A new look at Treasure Hunter

[Toksyuryel]

I just did 100 more kills. 19/100 bringing my total to 48/200 or 24%. There is a party that came out and level sync'd on these so I'll let them have the camp for the rest of the night.

I am personally of the opinion that TH simply affects different items in different ways. Once a formula for one has been established we'd need to compare to something else, presumably something with a drastically different drop rate.

Much too early to tell, but based on this data point one could theorize that TH2 = 2.0 mod, and TH1 = 1.5 mod. Definitely needs more data though.

[Gaea]

I think the reason why it appears that Treasure Hunter affects things differently is because small variations in a drop rate can become much more noticeable as you apply TH. It seems you are getting a 64/256 drop rate with Treasure Hunter II on an item that seems to have a base value of 32/256.


Just a little thougth I had, though. I'm assuming that the client side software that determines this stuff is 32 bit. If by some strange chance it were actually 64 bit then that would double the amount of data processed in a single clock cycle and make n/512 a plausible option to these mechanics. However, I'm reaching here, and I'm pretty sure that the developers who designed the 32 bit application that we know also designed the server side software. Besides, back in 2002, and certainly in the couple years of development prior to that 64 bit computing was hardly anything close to mainstream.

[Dawin]

TH0 Results:

Carnivorous Crawler (Killed 342 times)
46 spool of silk thread [Drop Rate: 13.45 %]


TH1 Results:

Carnivorous Crawler (Killed 376 times)
60 spool of silk thread [Drop Rate: 15.96 %]

This alone disproves the OP with sufficient accuracy / sample size. If we were going by OP, we would have to assume the first case is a 12.5% drop rate, which would mean the second case is a 25% drop rate, however, it's almost 50% below expectation which is huge.

[Mojo]

I got a bit of data done tonight but not enough to post. Either way, just a few things before I head off to bed.

The target mob(Crawlers is West Saratabaruta) is working out nice and easy for this test. However, we're going to need a good sample size of at least 1000 of each TH0, TH1, TH2, TH3, and TH4 to really get a good study going. The chance of the drops being fractional instead of decimal is high given the way the rest of this game works. There are also other mobs around that can drop two items. Data will be needed on those mobs too, so slaughtering everything in sight is probably the best approach to this.

Also, as far as the theory presented in the original post. None of the data thus far is remotely close to what it predicted. STOP TRYING TO FORCE THE DATA TO FIT YOUR MODEL. If you really want to figure out the inner workings of treasure hunter, you will need to leave any preconceived notions behind and just observe the results.

[Delekii]

This alone disproves the OP with sufficient accuracy / sample size. If we were going by OP, we would have to assume the first case is a 12.5% drop rate, which would mean the second case is a 25% drop rate, however, it's almost 50% below expectation which is huge.

.5% drop rate is 50% below an expected drop rate of 1%, that doesn't mean anything. It could have been that the first test was sufficiently lucky to push what should have been 6% ish up to 12% ish... sample size is way too small in pretty much all cases in this thread.

[Dawin]

Gaea, the number of players in a Party seems to affect drop rates. This seems like it would mess with your theory a bit. If there's a difference for parties of 1,2,3,4,5,6, then the minimum difference possible on a 1/256 drop would be 6/256. Add in +4 for the minimum possible TH4 bonus, and that would imply that no drop is rarer than 10/256.

Granted this has a lot of assumptions, but it's something that needs to be accounted for one way or another.

Perhaps testing the affects of multiple players in a party on drops would give a clearer indication of the basics of the drop system, and should be done before TH testing.

[Toksyuryel]

After I finish solo-testing 1000 crawlers with no TH, I'll see if I can convince a friend to join me to duo 1000 more crawlers with no TH, and see if there's a difference.

[Dawin]

.5% drop rate is 50% below an expected drop rate of 1%, that doesn't mean anything. It could have been that the first test was sufficiently lucky to push what should have been 6% ish up to 12% ish... sample size is way too small in pretty much all cases in this thread.

After I finish solo-testing 1000 crawlers with no TH, I'll see if I can convince a friend to join me to duo 1000 more crawlers with no TH, and see if there's a difference.

He may also have to tag each mob or generate hate in some way. Don't neglect that.

[Dawin]

.5% drop rate is 50% below an expected drop rate of 1%, that doesn't mean anything. It could have been that the first test was sufficiently lucky to push what should have been 6% ish up to 12% ish... sample size is way too small in pretty much all cases in this thread.

60/376 on a 25%er is close to impossible, although you are right that running over double expectation on 12.5%ers on a similar sample size is more likely, but still incredibly rare.

Pathetically, I have a career in a field that deals with some statistical work, but I know nothing about confidence / probability formulas that would show how unlikely these results are.

[Wizerd]

This also isn't a crackpot theory. This is based on proven data from client side games which we can and have actually dissected, including Final Fantasy titles. All drop rates and encounter rates have been based on a base 2 system. On 8 bit Nintendo the highest possible rarity was 1/64 (8*8) and in 32 bit games like this the highest possible rarity is expressed as in 1 in 32*8 or 1/256.

All I'm doing is taken proven math in client side dissected games including Final Fantasy titles and applying it to something that we cannot see because this is all determined server side.

Simply put, if we can define a single object's drop rate with absolute certainty as x/256, we can ascertain the values of the bonuses applied by all forms of Treasure Hunter. Furthermore, using a different item that we've absolutely ascertained the value of with a value y/256 we can determine if the the relationship of Treasure Hunter is purely addition to the original value or an actual multiplier.

Needless to say, there's no doubt in my mind that every item has a base value n/256 that determines its drop rates. These mechanics seem to drive other SE titles before and after the creation of Final Fantasy XI. These mechanics are also apparent in other video games. From a programming point of view, it makes sense to make express a random variable in a 32 bit application as n/256. While my theory itself needs some broader testing for me to accept it as fact, I do believe I'm on the right path and that I am correct about the base value of an item's rarity being expressed as n/256.

Crackpot theory may have been a little harsh, I'll give you that. Your model, however, suggests that the drop rate of an item can be doubled simply by adding levels of Treasure Hunter. Treasure Hunter 2 (nevermind 4) doubling the drop rate of any given item is far too farfetched to be the case. When I did my Treasure Hunter testing, Giant Stingers jumped from about 5% to about 9% from TH0 to TH2, but Insect Wings only increased about 1.4x what they were before. I don't have the data on hand (I may not even have it anymore), but I do remember that much.

You seem bent on 1/256, but other things in the game have been figured out using lower fractional values such as 1/1024. There's no reason to believe the minimum drop rate is 1/256, especially considering the rarity of certain select items. If I had to place the minimum drop rate on a base 2 fraction, I would place it at 1/512, because when adding Treasure Hunter 4, the rate at which the item drops is often still well below 1/200ish.

I do not have a problem with your base theory. My primary problem with your theory is how you attempt to add Treasure Hunter on to it, and I don't believe any of your attempts are even remotely close.

[Toksyuryel]

Killed another 200 crawlers, total is now 44/400. 600 to go, still seems like anything could happen with the wild variances I keep getting.

[Gaea]

My original model overestimated Treasure Hunter. I didn't consider SE would make adjustments that much smaller. However, I still believe SE would logically try to conduct these acts of "randomness" in a single processing cycle as they and others have in other games.

Though I might have been off on Treasure Hunter (I was going by its application in a different Final Fantasy), I still believe have a base value of n/256.

Regarding "great drop rates" in parties, I think what you are referring to is multiple items (such as crystals) being added to the drop pool. Each of these items would have its own n/256 value.

The point is that to determine a drop you'd want to do it in as fewest processing cycles as possible. n/256 is the only logical choice. It's like having a 256-sided dice and if any number 1 through x comes up, your item drops.

The whole point of this was thread was to simply point out that past and present games are showing that our current model of Treasure Hunter is severely flawed by just assuming the drop rates are in raw, base 10, percentages.

Once we narrow a few items n/256 value, we can simply apply varying degrees of Treasure Hunter in BROAD testing and get an actual value for those degrees. Furthermore, we'll be able to apply actual base values of rarity to items in the game rather than shotty percentage data.

Also 1/512 would only be logical in a 64 bit system and this game began production almost 10 years ago so I sincerely doubt that's the case. Otherwise you are talking about twice the processing work to generate a random number and that makes no sense. 1/1024 would quadruple the processing work. Regardless, 1/256 is a .390625% chance. That's a little under .4%. A drop rate on anything in this game of less than half of a percentile is pretty low.

The only logical way to process a random result in FFXI would be the same way they've done it in past FF games and more recent ones and that's n/256.

My original model assumed that Treasure Hunter II might take a 32/256 item and turn it into a 128/256 item. Well, obviously, that was flawed and I didn't really think about that at the time I began trying to apply this to Final Fantasy XI. that doesn't mean that I'm wrong about the value for an item being n/256. I just believe Treasure Hunter additions are much more conservative.

Treasure Hunter I may only increase a 32/256 drop rate to 40/256. These increases may come in the form of multipliers such as 1.25 for TH1 which would mean 32*1.25/256 = 40/256.

Treasure Hunter plus equipment may just be another 1.1 multiplier of the base value.

[Wizerd]

The whole point of this was thread was to simply point out that past and present games are showing that our current model of Treasure Hunter is severely flawed by just assuming the drop rates are in raw, base 10, percentages.

Well to be fair, I don't think anyone ever really assumed drops were done in a percent basis. We don't *have* a current model of Treasure Hunter. Drop rates you see out there are just reported drop rates from player experiences.

I agree that a base 2 system is likely used. That would be difficult to dispute considering how many processes the server has to process per second and how much more complicated a decimal system would get for that. I just don't agree with how you outline it (even your revisions).

It would be simple enough for the dev team to give a drop a rate of 39/256, for example, in order to simulate a 15% drop rate on an item. Treasure Hunter could then modify that 39.

[Toksyuryel]

100 more crawlers down, total is now 54/500 (27/250 or 10.8%). I'm going to bed, I'll finish when I wake up.

EDIT: an interesting note about this figure, FF4DS actually uses a x/250 drop system, and TH doubles that value. It's possible this is the case here as well but it's probably something we'll never know for certain.

[Gaea]

Well, that I can somewhat agree with, and I'm sure drop rates that have been modified have been done so at this very fundamental level. I'm sure its possible I could be wrong that many items may have a base value that fits in a neat and tidy world of perfectly factorable fractions, but I don't think SE is trying to "simulate" any particular percentages. I think all this stuff about "percentages" is actually just something players conjured trying to make sense of a system that doesn't seem to be playing the same game as us.

With broad testing, I'm sure we can figure out the exact n/256 value of an item's base drop rate (no Treasure Hunter) and determine the true values of Treasure Hunter by applying it in steps.

[Amele]

I think the reason why it appears that Treasure Hunter affects things differently is because small variations in a drop rate can become much more noticeable as you apply TH. It seems you are getting a 64/256 drop rate with Treasure Hunter II on an item that seems to have a base value of 32/256.


Just a little thougth I had, though. I'm assuming that the client side software that determines this stuff is 32 bit. If by some strange chance it were actually 64 bit then that would double the amount of data processed in a single clock cycle and make n/512 a plausible option to these mechanics. However, I'm reaching here, and I'm pretty sure that the developers who designed the 32 bit application that we know also designed the server side software. Besides, back in 2002, and certainly in the couple years of development prior to that 64 bit computing was hardly anything close to mainstream.

well, obviously the math is all base 2 (it's a videogame)

I think the fundamental issue with your theory is that you assume that it has to be a shifted scale, 1/1024; 1/512; 1/256; etc. and that it *can't* be, say, 0000 0011 (3/256).

personally I think it works (mostly) like haste, dual wield delay, etc. if it works as integer math. - it is very possible that drop rate could be an unsigned 16-bit value (giving a minimum of 1/65,536) - even if they're trying to save space. they could even used a signed 16 bit value (giving you a maximum of 1/32k and change)

and have plenty of space to vary exact drop rates by as much or as little as .00003%

obviously they probably don't use the maximum resolution on drops; anyway - even if base drop rates are all buckets; it doesn't mean that TH has to be.

a 1/8 bucket could easily be 20h (8 bit wide) - and TH2 could just as easily make it 22h instead of 40h or 80h.

[Gaea]

an interesting note about this figure, FF4DS actually uses a x/250 drop system, and TH doubles that value. It's possible this is the case here as well but it's probably something we'll never know for certain.

FFIV uses n/256 as well. Many people are rounding values like ".390625%" to ".4%" which would suggest a 1/250 system but logically that fraction wouldn't many sense in a single processing cycle. FFIV has been pretty well dissected and the data it works with in memory isn't close to secret. If you poke around you'll be able to confirm the actual drop rate in the FFIV community.

personally I think it works (mostly) like haste, dual wield delay, etc. if it works as integer math. - it is very possible that drop rate could be an unsigned 16-bit value (giving a minimum of 1/65,536) - even if they're trying to save space. they could even used a signed 16 bit value (giving you a maximum of 1/32k and change)

It's not about space it's about processing cycles. The burden an n/65,536 would place on the FFXI servers is something I can't even begin to imagine. You are talking about 256 times the processing required to generate a single random result. Where's the logic behind that?

My theory simply states that on the server side, the stuff we can't see, FFXI is using a similar method for calculating drop rates as it has before and after this game came out and that to logically produce a random result with the greatest flexibility with the most minimum impact in the processing required to do so, that figure would have to be n/256.

[fishstix]

Within people's tests, are other variables being constant? Things like moon phase, elemental day. I know these can just be superstitions and arent being tested at the moment, but I don't think it should be ignored. I believe in it at least >.>

[SnickySnacks]

Ok, I have to call shenanigans here.
256 = 8 bits = 1 byte.
In a 32-bit variable, you technically can keep track of values from 0 to 4294967296-1, so I have no idea where you're getting the idea that a 32-bit processor is limited to n/256 values. Even if you're assuming the lowest common denominator (PS2) has to somehow deal with the floating point functions (it doesn't, it's all done server side), I still doubt their floating point operations would run on 8-bits, as you'd get no output resolution from that.

Please don't forget that there's a huge difference in calculating NES/SNES game values (8, 16-bit systems respectively) vs. calculating values on a modern computer (32, 64-bit).

Plus that's assuming we're even talking about floating point numbers.
The values could be stored as integers (0 to 4294967296-1) and just compared against a random value, as there's no reason the decimal percentage needs to be calculated for a drop. You can just say something drops 1000 out of 4294967296-1 times, roll a random number, and if it's less than 1000 it drops.

[Mojo]

It's already been proven that the math in FFXI goes as deep as 1/1024, so there really shouldn't be any question as to whether it's possible. It will never be proven either way seeing as how it would take around 20,000 kills on a given mob to accurately prove a difference of even 1/256.

Anyways, I've collected some data on TH1. I think other people should follow suit in the format I've done. I stopped every 100 crawlers and entered all of the information I've gathered from all of the mobs killed during that period of time into a spreadsheet on google docs. Doing it like that is a good way to keep track of variance, as there may be some lurking variables(day/weather/moonphase/time) that could be affecting drop rates, although I find it highly unlikely. Regardless, it's a good practice and should be done. Killing Bumblebees, Carrion Crows, and Crawlers was on the agenda(other people should be doing this too). I'll continue later when I have some time, but here's my data so far.

http://spreadsheets.google.com/pub?k...CuVYD1cgrgD-OA