OK, I've been trying to look into an idea someone said to me about a few days ago -- that maybe caster INT (cINT) and target INT (tINT) are multiplied by M seperately, and the rounding issue arises from this.
Hopefully this is not the case, as this would mean that with the same dINT but different cINT vs. tINT, the damage could vary slightly.
Also, the term "dINT" would become kind of meaningless.
But, my current M value for Thunder 5 being the actual value the game uses sounds pretty implausible, as it is something like 20 bit value so far.
So I need to look into different models like this, as Suiram suggested.
Before that, here's a list of some Thunder 5 casts that I've been working from.
D-V is the damage before adding the spell's V value, and before including terms like MAB, so it's just dINT*M but without knowing what the value of M is yet.
Code:
cINT tINT dINT D-V (aka. dINT*M)
356 110 246 565
356 107 249 572
999 106 893 2053
999 81 918 2111
999 71 928 2134
999 61 938 2156
999 55 944 2170
dINT+928 and dINT+938 are the most interesting ones as I've shown earlier, and dINT+701 would be nice to test...
Code:
dINT D-V (D-V)/dINT (D-V+1)/dINT
246 565 2.2967479675 2.3008130081
249 572 2.297188755 2.3012048193
893 2053 2.2989921613 2.3001119821
918 2111 2.2995642702 2.3006535948
928 2134 2.2995689655 2.3006465517
938 2156 2.2985074627 2.2995735608
944 2170 2.2987288136 2.2997881356
For dINT+701, D-V should be one of these two...
701 1611 2.2995720399
701 1612 2.2995720399
My current M value for Thunder 5 would result in 1611 for dINT+701...
602819/262144 2.2995719909668
The (D-V)/dINT column shows the minimum the M value needs to be to give the correct D-V answer.
The (D-V+1)/dINT column shows at what value M would be too high and cause D-V to be +1 higher than it should be.
Well so far I've testing the idea by using the following:
Code:
FLOOR(cINT * M) - FLOOR(tINT * M)
I may not have fully understood the idea correctly, but if that's correct then if I use just my dINT+928 and +938 D-V values, the M value can't be anything/2048.
My own idea at the moment is something along the lines of what I was trying to describe about recast time calculation in the "Cast and Recast" thread.
The greater values take up more bits leaving less precision for the fraction part of the value.
I think if it's something like that, then just finding the M value needed for the most precise situation (ie. between dINT+928, 701 & 938 in Thunder 5's case) should work for all the dINT which don't need such precision.