Extensive Desynthesis Rate Research

[Amele]

Been a long time folks; ただいま!

I'm posting this on behalf of a retail friend who I know by the name of "Gloin", who undertook to research Desynthesis rate information on current Retail.

tl;dr: Basically he reconfirmed the information that the Japanese wiki has: here - 60% failure, 40% success, given success: 40% NQ, 30% HQ1, 20% HQ2, 10% HQ3 (given HQ: 3:2:1 ratio); given failure 50% chance of losing the material.
And no tiering, obviously, although as you can see he looked for it to be sure.


Tier                1(+23)   2(+43)   3(+63)   4(+83)    Total
Fails                 2530     2438     2328     6382    13678
Material Loss         1267     1206     1146     3691     6845
NQ                     742      662      598     1689     3691
HQ1                    499      461      454     1305     2719
HQ2                    327      328      338      844     1837
HQ3                    169      197      164      398      928
Fail Rate           59.29%   59.67%   59.97%   60.11%   59.85%
Material Loss Rate  50.08%   49.57%   49.23%   50.55%   50.04%
Success Rate        40.71%   40.33%   40.03%   39.89%   40.15%
NQ Rate             42.72%   40.17%   38.48%   39.87%   40.23%
HQ1 Rate            28.73%   27.97%   29.21%   30.81%   29.63%
HQ2 Rate            18.83%   19.90%   21.75%   19.92%   20.02%
HQ3 Rate             9.73%   11.95%   10.55%    9.40%   10.11%


All desynthesis attempts were performed on "Leather Highboots" at the listed skill deltas, with no moghancement active (to ensure no aura effects.), no food active, and no synthesis support.

He also has begun investigation/reconfirmation of the effect of Moghancement: Lightning. (Overwhelming so far, but based on conventional wisdom we don't expect that the aura strength will matter.)


With 5730 failures, 2331 resulted in a lost material (still leather boots.) which gives a prevalence of 40.68% and a wilson binomial confidence interval (95%) of 39.42 to 41.96%.


With a baseline of 50% for leather boots material loss (determined from the initial testing above) we can conclude that the rate change is either 10% absolute or 20% relative, which is a higher magnitude than I think most of us believed it was.

I will post more information as it comes, but wanted to get the initial findings somewhere public instead of on a discord since we agreed it was worth sharing with the wider community.

If anyone else is interested in helping us further investigate material break rates, please feel free to DM me but don't expect an immediate response, I'm not very active in the FFXI community these days.

[Spicyryan]

Thank you for taking the time to test. This is important information.

[Amele]

Some more findings; the big surprise here was that "Moghancement: Desynthesis" appears to impact success chance by quite a bit.

Moghancement lighting, Powerful aura:

With 6215 failures, 2460 resulted in a lost material, which gives a prevalence of 39.58% and a wilson binomial confidence interval (95%) or 38.37 to 40.80%

So probably no effect from aura strength.

Moghancement Desynthesis, Powerful aura :

with 7388 attempts, 3210 successes, for a prevalance of 43.45% success (~4% higher than anticipated) with a wilson binomial CI (95%) of 42.32 to 44.58%. My Guess is that it's a relative 10% increase in success applied to the base success rate of 40%

Moghancement Desynthesis, Overwhelming aura:

with 6302 attempts, 2822 successes, for a prevalance of 44.78% success (~4% higher than anticipated) with a wilson binomial CI (95%) of 43.56 to 46.01%. the 44% value (10% increase of 40%) is in the CI for both, so aura strength probably doesn't matter here either

Comparing the two, both increase yield. Mog: Lightning increases yield more by about 2.5%, Mog: Desynth reduces crafting time for a given quantity of materials by about 12.5%.

[Byrthnoth]

So powerful vs overwhelming matters for desynthesis moghancement but not lightning?

Eh, probably not based on the confidence intervals

[Amele]

I could ask him for p-values to double check... but I think the differences were not significant. Null Hypothesis was that aura strength did not matter and we definitely didn't get a result that would enable us to reject that null.

I don't know if these were enough tests to distinguish between whether it's multiplying by 10% exactly or +/- some fractional X/1024, but that's probably only of interest to the folks working on emulation?

[Byrthnoth]

Yeah, not of any practical value (0.0006 different) and the sample size you would need would be stupid.

I imagine that wind, water, etc. do the same things for their respective desynth recipes.

Do things that reduce item loss (food, ring, etc.) or increase success rate affect desynths?

[Amele]

Sorry, Took a Break for the American Holiday and then we took a few days to get back to it.

Do things that reduce item loss (food, ring, etc.) or increase success rate affect desynths?

Coconut Rusk (-6% material loss, 3% HQ increase) + Moghancement lightning

Material Loss:

Losses: 2634, Attempts: 7546, Rate: 34.91% Wilson CI (95%) : 33.84%-35.99%

HQ Rate:

HQs: 3128, Successes (NQ+HQ) : 5148, Rate: 60.76%, Wilson CI (95%) : 59.42%-62.09%

The observed Material loss rate seems to be consistent with a function like: "(Base - Food {- Gear?} ) * (1 - moghancement)". where moghancement lightning is 0.2 (so base only 50% * 0.8 = 40% observed in the original trial)

It is not consistent with either a 16% (10+6%) absolute or 26% (20+6%) relative so other possibilities include a misprint on the item (seems improbable) or a cap at 30% reduction in material loss which is not consistent with the Sept 11, 2017 Patch note ( JP Note - NA note ) linked both since they say different things (weird).

The observed HQ Rate fits within a marginal +3% (or +3/128 (?)) multiplier on HQ rates. (i.e. we're getting .6*1.03, not .6+.03 ) - which I believe is consistent with the testing for success synth on coconut rusk? Craftymath v2 But it is also possible that HQ adjustment for Rusk does not impact desynthesis (that's also within the 95% CI)


Still need to test Gear, I think that's next. Trying to find someone with a Shield since we suspect that's going to be a necessary part of the puzzle for investigating the absolute cap alluded to in the patch notes....

[Byrthnoth]

To summarize:

* Break 60% / Success 40% without Desynthesis Moghancement
* Break 56% / Success 44% with Desynthesis Moghancement
* Effect of Kitron Macarons unknown, but it would be unsurprising if they were -7% break rate.

* If Break:
* Material Loss 50% without an Elemental Moghancement
* Material Loss 40% with an Elemental Moghancement
* Material Loss 34% with Coconut Rusk (loss rate -6%)

* If Success (no observed skill tiers):
* NQ: 40%
* HQ1: 30%
* HQ2: 20%
* HQ3: 10%
* Not clear from the data whether Rusks affect this. (60.78% vs. 60% HQ Rate). Not too surprising given the very different HQ rates and apparent lack of tiers.


Unknown whether gear affects it, but it probably at least affects break rate.


If Rusks don't affect the desynthesis HQ rate and Macarons give the expected -7% break rate, then the best combination of food and moghancement is Elemental + Macaron regardless what gear does along basically every axis. Well, I say that, but Desynthesis + Macaron is almost as good wrt results and you do 10% fewer synths.

[Amele]

I have received some additional information regarding Desynthesis from another player I know as "Goose". I have a copy of all the raw data, but I've duplicated the summary table in this post for posterity/convenience.

This concerns *success chance as a function of skill* when below the skill cap (!). I took the liberty of truncating the fractions.

Methodology:

"using a character who is capped at a multiple of ten skill, identify a recipe that is +N levels over the skill of the character (e.g. "Capped at 10 Bonecraft, do Fang Necklace (15) desynthesis to check success rate at -5 skill"). where possible we did at least 1000 synths, although for some easily sourced mats we did more."

Results:

Difference	Item	Level	Total Synths	NQ	HQ	Break	Succ %	HQ %	Break Mat Loss
-1	Bronze Harness	11	1140	182	282	676	0.4070	0.6077	0.5118
-2	Silver Earring	22	1904	266	392	1246	0.3455	0.5957	0.4967
-3	Brass Rod	23	1181	149	216	816	0.3090	0.5917	0.5098
-4	Brass Hammer	24	1150	114	162	874	0.24	0.5869	0.4828
-5	Fang Necklace	15	1439	116	162	1161	0.1931	0.5827	0.4996
-6	Elm Shield	26	1356	93	120	1143	0.1570	0.5633	0.3928
-7	Scale Mail	17	1465	48	90	1327	0.0941	0.6521	0.3986
-8	Bone Mask	18	1296	43	54	1199	0.0748	0.5567	0.5004
-9	Elm Staff	19	1389	11	47	1331	0.0417	0.8103	0.4109
-10	Bone Axe	20	425	0	0	425	0	0.00	0.5152

Amele's Commentary:

The first thing I noticed is that the success rate for desynthesis at -1 skill delta appears to be the same 40% as you have at 0 skill (!!!).

I didn't put the effort in to calculate CI%s on this, I think it seems self-evidently to be a two segment linear function. at -1 it's still 40% success, same as 0 (!!!); from -2 to -7 it's ~5% less success per skill, down to 10%. then -8, -9, -10 are each probably close to -3.3% per level, which takes us to 0% success chance at -10.

So if I wanted to summarize this as a single statement, I'd say: "40% at -1 or higher skill, approximately -5% per level from -2 to -7, -3.3% per level to -10, 0% at -10".


Oh also, they did success synths, although that's not really in scope for the thread topic... Want to discuss that here, or should I make a new post on the subforum?

[Byrthnoth]

In no particular order:
1. The desynthesis HQ rate is basically what we'd expect normally. It's weird that the mat loss rate was either 50% or 40%. Maybe that depends on the synth? We already know that some synths have mats that are a 100% loss rate on break, so 40%, 50%, and 100% aren't that crazy. It doesn't seem to be skill dependent, anyway.

2. The impact of skill on break rate is the same as with success synths, to my knowledge, although I guess you'll tell me shortly. -5% per level under cap, although I didn't realize it drops to -3.33% below 10% success rate. It's weird that you can attempt a synth as long as you're within 27 levels of it, but the break rate becomes 100% at -10 skill.

3. If you have the raw data, could you please check whether they were getting HQ2/3 results with negative skill diffs? We recently realized that it's not possible to HQ2+3 success synths unless you're at least Tier 1, so I'm wondering if that applies to desynthesis as well.

4. You could make another thread if you want or discuss it here. I'll read it either way! lol

5. We don't really know the level of a desynthesis synth and the deltas are based off the success synth level, right? Instead of being -1 == 40% break rate, could it be that desynthesis recipes have a skill cap 1 below synthesis recipes?

[Amele]

Sweet, I have some responses.

1. ["goose" says:] "This is a result of sometimes having an appropriate crystal element: moghancement, which reduces material loss rate by 10%" [amele, says: see upthread]. "I didn't remark on it because it wasn't what we were testing for so we didn't bother to control for it."

2. It's similar but different, I'll make a full post here since that way it's all in the same place. Probably tomorrow since I'm pretty busy today.

3. I checked the raws, they were getting HQ2/3 - would you like me to calculate the rates and see if they differ from when it's overcap? I can do that but probably not for a day or two.

5. That's a good point, I'll see if I can get them to run a handful of different -1s and see if they're consistent or if we happened to use one that wasn't actually 11 skill. That said if it's not 40% at -1 then I'm surprised it was as high as 35% at -2. (put another way, if you knew -2, -3, -4 were ~35%, ~30%, and ~25% you'd predict -1 was 40% right?) But we should still double check since it's surprising.

edit: actually maybe we can uncap skill and see if there's any skillups on harness at 10. if we don't get any skillups then we'll know it's not 11 cap.

edit2:

It's weird that you can attempt a synth as long as you're within 27 levels of it

I think the maximum available +skill you can have these days is around +15? That probably has something to do with being able to succeed beyond -10 (and therefore to want to attempt that far away?) The attempt restriction is, I believe, based solely on your character-sheet skill level and not your effective skill level?

Otherwise; 425 is not technically enough attempts to rule out a non-zero but very low chance of success. (got me to calculate this one; the lower bound on the 99% CI (clopper-pearson in this case due to the expected value being close to 0%) is ~1.26%. So, statistically speaking this test was underpowered if you wanted to rule out success chances that were less than ~1.2% but greater than 0%. (e.g. 1/128 or something which is ~0.8%). Still, 0 in 425 seems good enough to conclude "don't do this", w.

oh also I rather suspect the crafting rings (+1% success chance) would work to raise your minimum success chance to 1% even at -27? (I wonder if anyone has tested this...)

[Byrthnoth]

1. Got it, okay!

2. Good point about the 27+15

3. I think it's not necessary. Desynthesis HQs apparently just work differently from success synths.

5. It might be more reliable to see whether you can attempt to desynthesize something whose success synth is 28 levels above you.

[Amele]

Got a hold of the player who did the Bronze Harness and they agreed to uncap their smithing; immediately started getting skillups:



so yeah- surprising result but this sure looks like 40% is the success rate for both at/above cap and -1 below cap for desynthesis.

I also got a player to try Battle Bow (woodworking 48) desynthesis at 20 WW skill today and they got the "beyond your current skill level" message, so it's probably not the case that a large number of synths cap differently for desynthesis.

Been busy with IRL work so I haven't had a chance to work up the post for synthesis rates, will get to that ASAP.

[Byrthnoth]

You get skillups up to 10 levels over the synthesis skill cap these days, so you'd expect to get skillups to 21 on a level 11 synth, right?


But yeah, the 28 level diff is pretty conclusive and I can't imagine they spent a bunch of time subtracting 1 from random recipe desynth levels.

[Amele]

Methodology:

Same as the desynth, we used capped characters for the tests.

Results:

Difference	Item	Level	Total Synths	NQ	HQ	Break	Succ %	HQ %
14	Slice of Bluetail	15	10079	8970	615	494	0.9509	0.0641
-1	Maple Sugar	21	3610	3216	0	394	0.8908	0
-2	Yew Lumber	22	1806	1545	0	261	0.8554	0
-3	Willow Lumber	13	2069	1682	0	387	0.8129	0
-4	Karakul Wool	14	2200	1535	0	665	0.6977	0
-5	Elm Lumber	25	1459	855	0	604	0.5860	0
-6	Brass Flowerpot	16	1422	735	0	687	0.5168	0
-7	Brass Hairpin	17	1979	824	0	1155	0.4163	0
-8	Chestnut Lumber	28	2965	922	0	2043	0.3109	0
-9	Walnut Lumber	19	1937	376	0	1561	0.1941	0
-10	Iron Arrowheads	20	2505	244	0	2261	0.0974	0
-11	Carapace Powder	21	1064	0	0	1064	0	0

As a reminder, at or above cap is 95% base success chance.

This one looks like it's 2-segment again. -5% per level to -3 delta skill, where you're now at 80%. then -10% per level after that.

notably this means you're still at 10% chance to succeed even at -10 skill, and don't get to 0% until -11.

I didn't include break rate data because they weren't controlling for it but I did notice that Slice of Bluetail (which used "Cooking Kit 15") had a 100% material loss rate in the raw data, so it's possible that the Crafting Kit materials are 100% loss... I'll see if I can get more testing done on that.

I know Goose has indicated interest in looking at how moghancement: Crystal Element interacts with 'guaranteed' loss materials in addition to normal materials so maybe we'll have more on that front in a few months.

[Byrthnoth]

I got the Reassembling Technique yesterday and started blowing through some stockpiled Crest Cards. I'll post full stats at the end, but with only ~1000 synths down:
* They are desynthesis recipes and follow the behavior we would expect in this thread
* I have a ~34% loss rate with Elemental Moghancement (-10%) and Ionis (-6%?)
* I have a ~49% break rate with Kitron Macarons (-7%), Artificer's Ring (-1%), and Ionis (-3%?)
* I have a 60% HQ rate (no change) with Craftman's Ring and Ionis

My guess is that the best build for desynthesizing cards is:
* Elemental Moghancement
* Artificier's Ring
* Patissier's Ring (I need to go fish up some Malicious Perch)
* Kitron Macaron
* Ionis

I don't think HQ+ stuff affects desynthesis.

I'm not sure I've seen it noted before, but desynthesis also does not seem to give Craftsmanship points for shields.

If Ionis is -3% break rate, then Ionis + Craftsman ring is -4% and floors break rate (1%) for single-craft, capped skill synths. News to me!

[Amele]

It's cool to have independent reconfirmation of some of these values, and to get a sense of the size of the Ionis bonus. Would need to know exact failure rates and loss rates to give you confidence intervals but those sound like reasonable guesses based on what you shared.

I am looking forward to the full stats when you finish!

[Byrthnoth]

Category : N : %
Break : 1849 : 47% of synths broke
Loss : 666 : 36% of breaks lost
Success : 2083: 53% of synths were successful
NQ : 838 : 40% of successes NQed
HQ1 : 574 : 28% of successes HQ1'd
HQ2 : 463 : 22% of successes HQ2'd
HQ3 : 208 : 10% of successes HQ3'd


This is with Elemental Moghancement, Craftsman's/Artisan's, Kitron Macaron, and Ionis.

Loss: -10% from Moghancement, -4% from Ionis?
Break: -1% from Artisan's, -7% from Kitron, -5% from Ionis?

For the sake of parsimony, I suspect Ionis is likely the same % for Loss and Break. (-5%?)

I fished up a Patissier's Ring and will farm another batch of cards, swapping it for Craftsman's.

[Byrthnoth]

Category : N : %
Break : 425 : 46% of synths broke
Loss : 102 : 24% of breaks lost
Success : 507: 54% of synths were successful
NQ : 198 : 39% of successes NQed
HQ1 : 137 : 27% of successes HQ1'd
HQ2 : 111 : 22% of successes HQ2'd
HQ3 : 61 : 12% of successes HQ3'd


This is with Elemental Moghancement, Patissier's/Artisan's, Kitron Macaron, Ionis, and HQ Crystals.

I blew 3mil or so on HQ crystals from the wedding NPC to test this out. The sample size is lower than I'd like so I may go toss more gil but thought it was worth posting.

My guess is that the loss rate reduction of HQ crystals is about 10%.

I don't think using HQ crystals is economical when desynthing for spheres, but I definitely understand why all of the cursed HQ2 Limbus stuff being made by shield crafters is signed.

[Byrthnoth]

I guess this means that shield crafters should be able to floor material loss rate:
* Base (50%)
* Elemental Moghancement (-10%)
* Shield stats (-5%)
* Enchantment (-15%)
* HQ Crystal (-10%)
* Rusk (-6%)
* Rusk Ring (-1%)
* Orvail Ring +1 (-2%)
* Ionis (-5%?)

Total: -4%

I'd be interested in what the actual floor is.

You won't be able to use the Elemental Moghancement when you need the skill from Mega Moglification, but you can HQ1 any synth you're Tier 0 on so in theory you could just grind it out with one set of mats until it worked if you didn't care about HQ2 and the loss rate was 0%. (moonbow items, etc.)