Testing out fast songs, but I could use some help

[Asmoranomar]

...20 values would get replaced by extra samples until the standard deviation of the 20 samples was lower than 3...

I don't know if you can actually do that. I don't think you can add in values you want to fit the std dev your looking for. I mean, which values would you know to be the correct ones?

You can do more tests and simply include them into the std dev, and eventually it should narrow down. If it never does, then your margin of error is as good as its going to get without changing something in your testing scheme.

[Yinnyth]

Yeah, I'm still kinda trying to justify it in my head as well. The idea is that I have a bunch of seemingly random values which have extremes of about 20-30 frames away from one another. These values are in constant fluctuation regardless of whether or not I'm currently testing them; they are infinite in number. Even 1,000 samples might end up giving me a poorly misleading average. So the idea is that I'm distilling the average by tossing out fringe numbers, or as you put it

threw out any values that were obviously incorrect (easy to notice)

Though my method of doing so...

I've also ran into situations where I lagged on the initial cast time, in those instances, I get a lower frame count - lower than the expected casting time of the spell. So I also throw out any values that are lower than the value I'm supposed to have (in the case of naked songs, I obviously cant cast faster than 8 seconds - it gets more difficult when you actually put fast cast gear on).

...seems a little bit more scientific than simply tossing out "impossible" numbers. If we know beyond any shadow of doubt that the base casting time of a song is 8 seconds, 480 frames, then why not toss out any result that gives us anything that isn't 480 frames? I'm kind of at a loss for what to do though. I need much better accuracy, but I can't afford to spend the time doing 250 samples per setup... There's a bullseye somewhere in the middle of all these values, and I desperately want an equation that will find it for me.



Another idea I had was to assume that songs take exactly 480 frames base casting time, and test the gear with their percentages based off of that. Obviously, lag will still push my values higher than they actually are, but I would only have lag as a variable on one side of the equation instead of both. The equation for finding my average lag would then be:

(1-[song reduction])*480=[frames]-[lag]

Instead of

(1-[song reduction])*([frames1]-[lag])=[frames2]-[lag]


I think tomorrow I start over yet again... Maybe I'll double my sample size, go based off the lowest value, and attempt to calculate my average lag.

[Asmoranomar]

...seems a little bit more scientific than simply tossing out "impossible" numbers.

What I meant by that is for situations where the number of frames is less than (or more than) the expected value. Getting 470- frames for a naked song that *should* be 480 frames means I obviously lagged at some point (likewise for 490+ frames). Not to mention, I never took them completely out of consideration - If I *had* put them in a std dev it would have been outside the scope I was looking for anyways. I guess the ironic thing was when I did a few tests above I never got slightly less than 480 frames. If I had, say 478 frames, I probably would have left it. I guess my issue is I never used a std dev because it was simple enough to catch. If I was having the issues you were, I would have kept all my vaules and used std dev.

I guess in your situation, it would be the same as getting less than 460 or more than 520 frames. I wouldn't even include that in my data set because it was so strikingly obvious. If that is what your doing then I don't see an issue with that, as increasing your std dev scope from 20 to 30..40...50...100(eep!) tests should eventually find that anyways.

However, this is only really applicable to naked songs, as you can't really tell what to expect when your testing gear (well, you still can because lag is actually more identifiable the faster you cast, but only to the degree I spoke about previously) - thats the point of the tests.

I really wish I could get access to my server to get the spreadsheets I have. Unfortunately I'm currently stationed outside the states, everything's in storage, and (like you) don't really feel like doing the tests all over again from scratch.

go based off the lowest value

You should base your tests off the average number in your std dev. High and Low values can both be attributed to lag.

[Yinnyth]

My problem is that I don't think I can compile enough values to get a reasonably accurate average before my life ends. In all of my tests, I have never gotten a frame count that was below the "expected" value. Hundreds of values, and none of them were below 480 frames * (1-reduction %). This would seem to indicate that all of the variation I'm getting is above the "actual" value, which means that my "truest" value would be my lowest tested value.

That being said, I will of course be using the average as well, but I think testing the lowest will give me a more accurate read. For today's testing I'm going to be trying 40 or 50 samples per tested setup, record an average value, lowest value, and standard deviation of the set. From both the average and lowest value, I will find the % reduction from 480 frames, and compare it to the expected value (240 frames for nightingale, 360 frames for minstrel's ring, etc) to find their deviation. I'll compile all the deviations, average them, and find their standard deviation. If the standard deviation of the "average value deviations" is lower than the standard deviation of the "lowest value deviations" then I'll have a pretty good idea that it's better to work my calculations off the average, rather than the lowest.

From that point, I will use the average of the deviation from expected values to help me in calculating unknown fast cast items such as ebur tam. This of course will be more time consuming than my previous tests, but I think aside from taking thousands of samples, this is my last hope...

[Asmoranomar]

Actually the more I think and read about it, I kind of want to get back into doing these tests again. I really hate not having a copy of my data with me while discussing this. If not simply because your results are so different than what I recall.

I have more FC/SST equipment now too (Full Marduk, Minstrel's Ring, Locq. Earring, Warlock's Mantle, Rostrum Pumps, Veela Cape, etc.) and I could easily synergize some SST evolith onto an ebon piece to test SST in 1% incriments.

...Or maybe it's just because I'm having one of those slow days at work.

...

On the other hand, all the trouble your having pretty much sums up what you originally wanted to do in the first place. Even if FC and -SST is rounded down and you got -48% Casting Time instead of -50% CT on your "Capped CT" setup, your not going to really notice or benefit from it due to lag and system delay - Pop off Nightingale and its going to be worse.


Edit: Maybe now would be a good time to tell you that when I was originally doing my tests, I really didn't question the stated vaules of the equipment. Sure I tested them, but that was to determine a baseline for my intended question - "How does FC and SST stack?" Most gear I tested was pretty accurate by itself, but when I went to try to do combined tests my numbers just completely fell out. I even tried working backwards, but that was even worse. It got inconclusive pretty fast, so much that I couldn't easily identify how SST by itself stacked (or FC for that matter). The more SST/FC gear I wore, the more horrible the results.

What does this mean for you? Well eventually I'd imagine once you get good vaules on individual pieces, your going to want to pair them up to see how they stack (this is the moment trying to measure 1/1024 differences mindfucks me). If you run into any of the issues I ran into, this is going to be much worse for you in this department. I would be interested in seeing results from a sample of your best FC/SST set. I wouldn't be surprised if you had enormous error margins there.

[Yinnyth]

I would be really interested to see some full marduk tests done, myself. I've got the pieces for the body myself, but I don't really have the funds or the desire to make it since this test is probably the only time I'd use it...

In my opinion the "perfect" fast song setup is ebur tam (-3% light song casting time evolith), sheikh manteel, rostrum pumps, veela cape, loq. earring, minstrel's ring, and dweomer knife (-5% augment). However, I'm very curious if full marduk really would prove to be faster than this setup.

Anyways, as soon as I said I never got any tests that resulted in a number lower than their expected value, I had my first troubadour song fire off in 718 frames (720 expected). Still, I'm thinking a -2 deviation will be closer (and more importantly consistant) than the average deviation turns out to be. I'm about 1/3 done with the baseline tests for my new system- hopefully I get done with these this morning.

[Yinnyth]

40 samples each, fraps 60FPS, assumed 8 seconds base casting time (480 frames).

Troubadour

+50% casting time (720 frames)
Average: 732.2 (+52.54%, 12.2 lag)
Lowest: 718 (+49.58%, -2 lag)
Standard deviation: 8.16

Nightingale

-50% casting time (240 frames)
Average: 250.55 (-47.8%, 10.55 lag)
Lowest: 243 (-49.38%, 3 lag)
Standard deviation: 4.04

Minstrel's Ring

-25% casting time (360 frames)
Average: 375 (-21.88%, 15 lag)
Lowest: 360 (-25%, 0 lag)
Standard deviation: 5.41

Sha'ir Manteel

-12% casting time (422.4 frames)
Average: 429.05 (-10.61%, 6.65 lag)
Lowest: 420 (-12.5%, -2.4 lag)
Standard deviation 7.97

Sheikh manteel

-13% casting time (417.6 frames)
Average: 426.55 (-11.14%, 8.95 lag)
Lowest: 421 (-12.29%, 3.4 lag)
Standard deviation: 5.63

Overall base tests

Average AvLag: 10.67 (3.17 standard deviation)
Average LowLag: 0.4 (2.72 standard deviation)

Conclusions:
I believe the difference in casting time between sha'ir and sheikh manteel is lower than the item description would lead one to think. Sha'ir is most likely 128/1024 (12.5%), while Sheikh is most likely 134/1024 (13.09%).

[rog]

I can't read.

[Yinnyth]

Finished testing the fast cast from /rdm, rostrum, loquacious, and veela. /rdm (Fast Cast II) did indeed seem to give song casting time -15% as is assumed. Loq, Veela, and Rostrum do not all give the same fast cast, however.

Loquacious Earring

-1% casting time (475.2 frames)
Average: 489.25 (+1.93%, 14.05 lag)
Lowest: 472 (-1.67%, -3.2 lag)
Standard deviation: 9.01

Veela Cape

-1% casting time (475.2 frames)
Average: 491.225 (+2.34%, 16.025 lag)
Lowest: 473 (-1.46%, -2.2 lag)
Standard deviation: 7.14

Rostrum Pumps

-2% casting time (470.4 frames)
Average: 472.4 (-1.58%, 2 lag)
Lowest: 466 (-2.92%, -4.4 lag)
Standard deviation: 6.88

Conclusions: just because all of these items lower recast time by roughly 1% does not mean they all decrease casting time by 2%. Loq. and Veela appear to be a 1% casting time reduction, though possibly as high as 1.5%. Rostrum pumps are roughly twice as much casting time reduction as Loq and Veela, and I would estimate them as being somewhere around 2-2.5% casting time reduction.

[Yinnyth]

Just finished testing Ebur Tam (all I had time for today). No big surprise, but wiki was pretty far off base how much fast cast it has.

Ebur Tam (-3% light song casting time evolith, minuets to test FC only)

-1% casting time (475.2 frames)
Average: 485.95 (+1.24%, 10.75 lag)
Lowest: 472 (-1.67%, -3.2 lag)
Standard deviation: 8.54

-1% is a rough guess based on the fact that the lowest value I got was so close to the lowest values I got for veela and loquacious even though the average was slightly lower. It is possible it's as high as -2%, but I'm confident it's the same as Loq and Veela.

Next I get to start running tests as to how song casting time is calculated. My guess is it's either (1-SST%)*(1-FC)*nightingale*troubadour or (1-SST%-FC)*nightingale*troubadour.

[Yinnyth]

I just finished my testing for the time being and posted the results on the OP. I don't have values for the marduk set because I really don't feel like spending a mint on an item I would only use to reduce casting time of non-song spells. However, if someone who has the marduk would be willing to test it, I would be interested to see how it stacks up against sheikh+ebur+rostrum.