Margin of Error calculation

[Narse]

I'm interested in testing the effect of Treasure Hunter on Rare/Ex and regular drops, and am trying to find info on how to calculate margin of error. In my test I plan on killing one type of mob repeatly that has a rare/ex drop and at least one type of non-rare/ex drop. In order to calculate the base drop % on it's items (done without any treasure hunter, then I will go back and kill an equal number with /THF), I need a formula to calculate the % and give me the margin of error. Unfortunately I'm not able to find much when I google 'margin of error', and I recall people posting about it here before but I wasn't able to find it in search on the site. If anyone could help me out here it'd be greatly appreciated.

[thestalkmore]

If I remember anything from stats class...

You're going to want to calculate a confidence interval using a Z or a TSample test.

[Dense]

Hi Narse, Wiki is your friend!

http://en.wikipedia.org/wiki/Margin_of_error

[Narse]

Umm....


...yeah, I'm not good with understanding what you just said. I guess I'll try to explain my goal a little better, and maybe someone can give me my magic number.

Example:
I go and kill Sylvestres in Buburimu as PLD/WAR (no signet), which drop 3 Rare/Ex items, as well as saruta cotton. How many will I have to kill to be able to say "Saruta cotton drops 5% of the time, with a margin of error or < 1%"?

Or rather, for the calculation of this, I'll assume that a drop is placed on a table for X/256 places. How many kills would I need to say "Saruta Cotton is 64/256, with 99% accuracy, or 1% margin of error?" Just the answer to that question wold be appreciated, with the formula to being able to figure that out as well even more appreciated.

EDIT: Also, no, Wiki is not my friend. I saw that and I can't understand how to apply it to this situation.

EDIT 2: Is this better expressed in % instead of X/256? As in, "Saruta cotton drops 25% of the time, with 99% accurary, or 1% margin of error"? One way gives a population of 256, other way of 100.. I think I may be slowly figuring this out, hmm...

[Lynsy]

Umm....


...yeah, I'm not good with understanding what you just said. I guess I'll try to explain my goal a little better, and maybe someone can give me my magic number.

Example:
I go and kill Sylvestres in Buburimu as PLD/WAR (no signet), which drop 3 Rare/Ex items, as well as saruta cotton. How many will I have to kill to be able to say "Saruta cotton drops 5% of the time, with a margin of error or < 1%"?

Or rather, for the calculation of this, I'll assume that a drop is placed on a table for X/256 places. How many kills would I need to say "Saruta Cotton is 64/256, with 99% accuracy, or 1% margin of error?" Just the answer to that question wold be appreciated, with the formula to being able to figure that out as well even more appreciated.

EDIT: Also, no, Wiki is not my friend. I saw that and I can't understand how to apply it to this situation.

EDIT 2: Is this better expressed in % instead of X/256? As in, "Saruta cotton drops 25% of the time, with 99% accurary, or 1% margin of error"? One way gives a population of 256, other way of 100.. I think I may be slowly figuring this out, hmm...

Honestly, there isn't enough information there to give a margin for error. Margin of error and accuracy are also not inverse concepts.
However, if you want to assume 95% accuracy with a 25% drop rate and a sample size of 256, your margin of error would be ~5.3%. For 99% accuracy, your margin of error would be 6.9%

[Narse]

See, that's my problem. I don't know the sample size I need, and I can't figure it out because I don't know the drop rate. The drop rate is what I'm trying to figure out.

I guess it can be equated to a 'survey', a yes on the survey being a drop, and a no being no drop. Since we're dealing with drops and the tallying of hard data, there's 100% accuracy in the numbers that I'll have, and going for a margin of error of 1% or less, I need to figure out how many mobs I have to kill.

[Lynsy]

The best way to do it is to just do a fuckton and then calculate your margin of error at the end of it and take what you get. Trying to get a particular margin of error rarely works, as it measures uncertainty and its hard to be certain of how uncertain you might be.

[Narse]

Well, I've got one more option of comparision to explain what I'm looking to do, and I know there's a formula for it, as I've seen it, but for the life of me I can't find it now.

Ex:
Someone says they'll give me one 'point' every time I roll a 100 sided die to a number equal to or greater than the 1-100 integer he has pre-decided in his head. I can't see the number I roll, I can only see that I have gained or did not gain a point after each roll. My goal would be to figure out what the number this person is thinking of, within 1.

Now, for the sake of argument, we'll say my friend is thinking of 50. In an ideal situation, I'd roll 100 times and have 50 points. Of course, that's very unlikely to happen. What I need to figure out is how many times I must roll that die to figure out what number he is thinking of, within 1.

[Lynsy]

Now, for the sake of argument, we'll say my friend is thinking of 25. In an ideal situation, I'd roll 100 times and have 25 points. Of course, that's very unlikely to happen. What I need to figure out is how many times I must roll that die to figure out what number he is thinking of, within 1.

Actually, you would have 75 points.
But...
That's just another form of the margin of error equation.

But, for that example you'd need a sample size of ~12,280 to be within 1 number, reliably.

[Kaylia]

This?

Standard error = http://upload.wikimedia.org/math/7/3...52a1612cd0.png

p is the % and n is the size of the sample. Sample need to follow gauss stuff.


Radius of confidence:
Margin of error at 99% confidence = http://upload.wikimedia.org/math/a/f...d1a2749d40.png
Margin of error at 95% confidence = http://upload.wikimedia.org/math/e/3...eb147958b2.png
Margin of error at 90% confidence = http://upload.wikimedia.org/math/3/0...ff36809628.png


Everything is from the wiki link posted above. You should read it instead of asking again >_>


The rest is basic maths if you want the number

Find what "error" you need for 99% accuracy (1.69%) and find your sample size from there.

[Narse]

Now, for the sake of argument, we'll say my friend is thinking of 25. In an ideal situation, I'd roll 100 times and have 25 points. Of course, that's very unlikely to happen. What I need to figure out is how many times I must roll that die to figure out what number he is thinking of, within 1.

Actually, you would have 75 points.
But...
That's just another form of the margin of error equation.

But, for that example you'd need a sample size of ~12,280 to be within 1 number, reliably.

Yeah, I saw the error in that and fixed it, but not before you quoted apparently. 12,280 is the number I'm looking for, thank you (I was assuming it'd prolly be ~10,000 or so). Would you mind explaining how you got that?

[Narse]

This?

Standard error = http://upload.wikimedia.org/math/7/3...52a1612cd0.png

p is the % and n is the size of the sample. Sample need to follow gauss stuff.


Radius of confidence:
Margin of error at 99% confidence = http://upload.wikimedia.org/math/a/f...d1a2749d40.png
Margin of error at 95% confidence = http://upload.wikimedia.org/math/e/3...eb147958b2.png
Margin of error at 90% confidence = http://upload.wikimedia.org/math/3/0...ff36809628.png


Everything is from the wiki link posted above. You should read it instead of asking again >_>


The rest is basic maths if you want the number

Find what "error" you need for 99% accuracy (1.69%) and find your sample size from there.

As for this, I can't do that equation because I am trying to figure out what n should be. I did read it, and since it didn't apply to my situation, that is why I asked.

[Lynsy]

Yeah, I saw the error in that and fixed it, but not before you quoted apparently. 12,280 is the number I'm looking for (I was assuming it'd prolly be ~10,000 or so). Would you mind explaining how you got that?

Magix!

Nah, but seriously, its the same equation. You want a margin of error of 1% and a certainty of 1% meaning you get. Your multiplier is 2.58 for a 99%.
And wow, I totally fucked that up. (I, apparently, used 2.56...so its actually 12,480 not 12,280...blame it on being 5AM)
Anyway, I'll show my work MSPaint style.
http://i4.photobucket.com/albums/y116/lynsy/nequals.jpg

[Faris]

what the fuck maths

[sleekmotorwurkz]

only problem is when youre talking about killing a large number of mobs, you also have to take into account all the variables that are rumored to have effects on drop rate... for example you kill 200 things with no TH, but by the time you do your "With TH" test the moon phase has changed which increased the drop rate (who knows if it really does) but still it could affect the numbers.

If youre working with a large sample size (which should be the case if youre worried about margin of error) you should keep track at every day/moon phase change and do multiple tests during the same day/moon phase for more accurate results.

Even then there could be some random unknown variable that affects drop rate that nobody knows about which could screw up the numbers completely.

[Xaetica]

@sleekmotor

You do not account for those variables by testing under each one (I mean, you could, but there is a better ay). In statistics what you do is you make sure that your population includes all of those variables. Then at the end of your run, you normalize for each of the variables.

By doing this you can actually see if the variable in question is in fact a variable.

[Croaker]

I love that you are wanting to go out and test. Far too many people just like to say "it can't be done." Its actually very easy and fun. Here is a small sample of testing I have done to give you an idea on what you can do.

TH0 drop rate.

For this I used 75MNK/NIN.

Tabar Beaks killed: 200
Meat drops: 34 (17.0%)
Feather drops: 27 (13.5%)
Skin drops: 4 (2.0%)

TH4 drop rate.

For this I used 75THF/NIN.

Tabar Beaks killed: 200
Meat drops: 94 (47.0%)
Feather drops: 46 (23.0%)
Skin drops: 7 (3.5%)


*** Done in Sauromugue. Tabar in different zones have completely different drop percents.

Picking a mob with more than one drop makes your job a lot easier. Three things all pointing the same way will give you more confidence in what you are doing.

200 is a small sample size to determine the exact % drop. You would need 1000 or so kills to get a 99% confidence on the meat. That number goes way up as the drop % goes down, so you need even more for the skins.

What you can do though is do a t-test. These two groups were different with very high confidence. You can use that to discover a lot of interesting stuff with relativly little effort.

Don't worry about moons or days or weather. I can't say it has no effect at all, but from the data I have collected it is less than 3%. Insignificant compared to the difference from the TH. If you are worried about it, just spread your kills out over a week or so and it will average out anyway.

Happy Hunting :D

[Lynsy]

Oh, it can be done...It's just the number of tests that you need to come up with anything conclusive is so astronomical that no one wants to. Particularly when you take into consideration everything said to effect it.

[GraveRobberX]

http://www.cartoonstock.com/lowres/mba0049l.jpg

http://www-128.ibm.com/developerwork..._1401_fig7.jpg

Graph makes me happy, too much math = death

[Taj]

There's really no accurate way to get the drop rate of a monster, considering all the bizarre variables that are factored in. Moon phase, for one, almost definitely has an effect on drop rate. Then there's things like the max level of everyone in your party, and then even more superstitious stuff like the element of the day it was killed on, what direction it was facing, what direction you were facing, whether or not the ebon panels were in alignment, etc.