Conserve MP Testing

[Byrthnoth]

Conserve MP Testing methodology:
I have my mule or myself spam Barsomething-ra (12 MP per) or Enspells (also 12 MP per) over and over and over and over in Abyssea. I use the program I posted in the other thread to count the number of times I start casting a spell. Thousands of MP, 0MP/tick Refresh. I know the cost of the spell, the number of casts, and the MP I've spent, so I know how much MP I've saved. First tests I did basically confirmed the way we think Conserve MP works (rounds down, 8-15/16 when it procs):
For (n=8:15), Barspell cost is (6, 6, 7, 8, 9, 9, 10, 11 MP = Average of 8.25)

If I have X MP and know I cast N Barspells in it, X/N = Average MP per cast
If Z is the Conserve MP% Proc rate, 8.25*Z + 12*(1-Z) = 12 - 3.75*Z = Average MP per cast

Test 0, Standardization: 30 Conserve MP (WHM/BLM, Magnetic Earring)
Prediction - 10.875 MP/cast
Test Results - 5163 MP Spent (three pools), 477 Casts of Barmagic :: 10.824 MP/cast <-- Good enough for me.
Stats: 31.36% proc rate +/- 4.16%
Conclusion: We do know how Conserve MP works.

Test 1, Additional Trait Tiers traits for Black Mage: 25 Conserve MP (BLM/WHM)
Prediction - 11.06 MP/cast
Test Results - 1720 MP Spent (one pool), 152 Casts of Barmagic :: 11.32 MP/cast <-- Higher than expected, not lower. Tested at 18% +/- 6.1% potency, so probably isn't over 25%.
Conclusion: No Trait Tiers for Black Mage

Test 2, Atma of the Undying: 30 Conserve MP (WHM/BLM, Magnetic Earring) + Atma
Test Results - 10328 MP spent (three pools), 1025 Casts of Barmagic :: 10.08 MP/cast
(12 - 10.08 )/3.75 = 51.3% +/- 3.0% Conserve MP
Conclusion: Atma of the Undying is 20 Conserve MP

Test 3, Atma of the Fetching Footpad: 25 Conserve MP (RDM/BLM) + Atma
Test Results - 7897 MP spent (three pools), 814 Casts of Barmagic :: 9.70 MP/cast
(12 - 9.7)/3.75 = 61.3% +/- 3.3% Conserve MP
Conclusion? Atma of the Fetching Footpad is about 35 Conserve MP, but this is the furthest off result I've gotten so far so....

Test 4, Atma of the Fetching Footpad: 0 Conserve MP (RDM/WHM) + Atma
Test Results - 7789 MP spent (three pools), 728 Casts of Barmagic :: 10.70 MP/cast
(12 - 10.7)/3.75 = 35.0% +/- 3.5% Conserve MP
Conclusion: Fetching Footpad is 35 Conserve MP.

Test 5, Atma of the Blinding Horn: 0 Conserve MP (RDM/WHM) + Atma
Test results - 10371 MP spent (three pools), 919 casts of Barmagic :: 11.285 MP/cast
(12-11.285)/3.75 = 20.42% +/- 2.6% Conserve MP
Conclusion: Blinding Horn is 20 Conserve MP

Test 6, 105 Conserve MP: Blinding Horn, Fetching Footpad, Undying, /BLM, Goliard Saio
Test results - 5867 MP spent (3 bars), 710 Spells cast :: 8.26 MP/cast ~= 99.6% Conserve MP proc rate
Conclusion: Yeah, you can hit a 100% Conserve MP proc rate but you (obviously) can't exceed it.
I didn't put confidence intervals here, because I think my assumptions run into the problem CDF points out when the values get closer to 100%. The fact that I can exceed a 100% occurrence with my measure (impossible in a binomial distribution) is problematic, and because I can, plugging the numbers I get near a 100% proc rate into the binomial equation and using the result is fairly meaningless. I think that my estimator underestimates the variance more any time I exceed a 50% Conserve MP rate.

Tests remaining:
7) Use my force conserve MP set to determine BLM set bonus proc rate and actual mechanism... as soon as I get one more Jewel of Ardor.


Stat Logic:
* I can't directly take the number I'm measuring and use a Binomial Distribution to model them, because I'm not measuring a 2-state system.
* If I solve for the Conserve MP proc rate, I *do* have a 2-state system (if we believe my assumptions about how Conserve MP works), and my N would be the number of casts.
* I can use this to make confidence intervals the same way that I always do.
*prz to has correct me if I'm wrong.

[CDF]

Your estimator is fine since its expected value is still the conserve MP rate.

The "problem" is that your estimator does not have minimal variance (the usual binomial proportion estimator does) so if you use the standard error for the usual binomial proportion (estimator) based on your actual estimator, your actual alpha error will be higher because the standard error you compute will be too low.

The actual estimated variance of your estimator is s^2(MP)/n/3.75^2 where s^2(MP) is the sample variance of your MP observations.

From simulation I got about a 91% confidence level for the first test (assuming 30% proc rate) so what you did seems to be a reasonable compromise in order to save time if you don't want to look at the actual MP observations

[Byrthnoth]

Alright, thanks! And yeah, I could probably automate recording the MP observations, but it would make my script much more spammy and much harder to analyze.