Item Search
     
BG-Wiki Search
+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 34

Thread: CraftyMath     submit to reddit submit to twitter

  1. #1
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    CraftyMath

    So, FFXIDB has been collecting synths for the last ~year. Arcon noticed that they were a bit messed up about 6 months ago and no one had any time to fix them, so we didn't. In the last few weeks we've been looking at them and we think that we know what is wrong with it and can fix it.

    Anyway, we're going to have to nuke the 7M synth database that we've built up in the last year. Before we do that, if I very tightly constrain my queries then I can get some very basic information out of it:

    Tier -1 HQ rate: 0.06% ( 16 / 24127 )
    Tier 0 HQ rate: 1.8% +/- 0.3% ( 144 / 8073 ) -- 95% confidence interval is likely underestimated because it's too close to a bound.
    Tier 1 HQ rate: 6.6% +/- 0.5% ( 660 / 9951 )
    Tier 2 HQ rate: 28.5% +/- 0.8% ( 3659 / 12865 )
    Tier 3 HQ rate: 50.6% +/- 0.7% ( 8835 / 17462 )

    Data:
    Spoiler: show
    Format is:
    Conservative Tier : NQ, HQ1, HQ2, HQ3
    The Tier estimation is "conservative" because I'm always assuming that any synthesis support buff present is Advanced and that any Furniture in the MH is giving its crafting moghancement. Thus I should be overestimating skill level rather than underestimating, which should bias the HQ rate estimates downwards. .... or I'd like to pretend that it matters, but in practice I ran the numbers with the opposite assumptions (furniture not active and NQ support) and it made almost no difference. Still, those are the qualifiers.

    Code:
    Silent Oil (Slime Oil recipe)
    Tier -1: 4827, 3, 0, 0, 0
    Tier  0: 436, 3, 0, 0, 0
    Tier  1: 563, 32, 8, 0, 0
    Tier  2: 2429, 603, 146, 53
    Tier  3: 5338, 4074, 1022, 363
    
    Holy Water (no Trit recipe)
    Tier -1: 19279, 12, 0, 1
    Tier  0: 7473, 138, 3, 0
    Tier  1: 7889, 396, 151, 0
    Tier  2: 5068, 1634, 419, 152
    Tier  3: 2728, 2144, 519, 189
    
    Holy Water (Trit recipe)
    Tier -1: 5, 0, 0, 0
    Tier  0: 20, 0, 0, 0
    Tier  1: 839, 57, 15, 1
    Tier  2: 1709, 493, 120, 49
    Tier  3: 561, 395, 90, 39


    I'll note that this should not be taken as the absolute truth and there's some chance that even crafting records that look good (real ingredients, real products) in the database could be faulty. That's why we're going to nuke the database. That said, the majority of the data is probably fine and paints the above story.

  2. #2
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    Does it account for things like food or rings?

  3. #3
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    I can check whether or not people are using food, but it has no way to know what food that is. For rings, I just went back into the database and did a breakdown of holy water and silent oil with and without rings. The numbers are slightly different because it has over a week worth of extra data:

    Code:
    Silent Oil, Ring:
    Tier -1 : 80, 0, 0, 0
    Tier  0 : 34, 0, 0, 0
    Tier  1 : 0, 0, 0, 0
    Tier  2 : 182, 41, 8, 1
    Tier  3 : 937, 741, 195, 77
    
    Silent Oil, No Ring:
    Tier -1 : 4747, 3, 0, 0
    Tier  0 : 402, 3, 0, 0
    Tier  1 : 563, 32, 8, 0
    Tier  2 : 2247, 562, 138, 52
    Tier  3 : 4401, 3333, 827, 286
    
    
    Holy Water, Ring:
    Tier -1 : 701, 0, 0, 0
    Tier  0 : 129, 2, 0, 0
    Tier  1 : 348, 16, 7, 0
    Tier  2 : 203, 109, 21, 8
    Tier  3 : 352, 266, 78, 13
    
    Holy Water, No Ring:
    Tier -1 : 18785, 12, 0, 1
    Tier  0 : 7396, 137, 3, 0
    Tier  1 : 7627, 386, 144, 0
    Tier  2 : 4930, 1562, 403, 145
    Tier  3 : 2381, 1880, 442, 176
    For Tier 2:
    Ring: 188 / 573 = 32.8% +/- 3.8%
    No Ring: 2862/9139 = 31.3% +/- 1.0%


    For Tier 3:
    Ring: 1370 / 2659 = 51.5% +/- 1.9%
    No Ring: 6944 / 13726 = 50.6% +/- 0.8%

    Not significantly different (but +1% observed.) We'd need some Tier 0 synth that is done in bulk to really evaluate their potency (and whether it's multiplicative or additive.)

  4. #4
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    So it has a way of knowing what rings were used while crafting? Neeto. When I get the chance I could add some personal food/no food data see how that compares. It's not as big a sample as this but might help

  5. #5
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    So dug up my numbers of no food or ring T1. 3003 nqs 195 hq. Will look for my food ones later

  6. #6
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    Did some T0 synthing grinding up goldsmithing 51-62 on Geodes.

    7/402 synths HQed.


    Macaron + No Ring

  7. #7
    Cerberus
    Join Date
    Nov 2008
    Posts
    419
    BG Level
    4
    FFXI Server
    Cerberus

    Great info, I thought ffxidb was defunct good to know its still collecting data that can be pivoted on

  8. #8
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    Damn I thought I had posted my T1 food testing. Maybe I did wrong thread lol

  9. #9
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    Quote Originally Posted by Neoex View Post
    Great info, I thought ffxidb was defunct good to know its still collecting data that can be pivoted on
    This thread is old, actually. It probably has more data now that I haven't looked at.

    Also, I think I solved the mystery of the non-0% T-1 HQ rate. If you start a synth with guild support it can wear off mid-synth and you can still HQ, but XIDB won't detect it is on when the synth completes.

  10. #10
    Melee Summoner
    Join Date
    Aug 2010
    Posts
    26
    BG Level
    1
    FFXI Server
    Bahamut

    Do you have a link to that food testing?
    I'm trying to figure out what the actual statistical average of cost for me would be but there's too many unknowns for HQ rings and food. If as Byrth posted above is right and it works that way for t0 synths too, then wouldn't it become 1.5 + 1 (Hq ring) + 1 (Orvail ring) + 1? (food) for a 4.5% chance or 1 in 22 instead of 1 in 64? Because that's a huge freakin difference.

  11. #11
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    The error bars on that data are so large that they do not distinguish between either possibility.

  12. #12
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    Okay, I built up >3000 Avarites so I decided to do a Rusk vs. No Rusk test.

    Goldsmithing skill: 106 goldsmithing skill with stall (5) and shades (1) for 112 skill total
    Additional gear: Craftmaster's Ring and Artificer's Ring
    Recipe: Avarite + Wind Crystal (Level 81, so Tier 2)
    Area: Bastok Markets, no Signet, Sigil, Sanction, or Ionis

    Rusk:
    NQ 2119 HQ1 541 HQ2 149 HQ3 41

    No Rusk:
    NQ 1868 HQ1 460 HQ2 103 HQ3 37

    The BGwiki description of how crafting works implies there are 3 independent probabilities:
    1) Break or Success
    I could potentially reanalyze my data for break rates if people particularly care, but at the moment I am ignoring the first comparison because that's not supposed to be related to what Rusks do.

    2) NQ or HQ (varies based on HQ Tier, would be 1/4 for this synth)
    Rusk:
    25.65% (731/2850)
    No Rusk:
    24.31% (600/2468)
    Chi squared p=0.261

    The raw HQ rate is not significantly different between these samples, but we have +/-1.6% 95% confidence intervals on these HQ rates (would need ~3x sample size to decrease it to 1%).


    3) HQ1 (12/16), HQ2 (3/16), or HQ3 (1/16)
    Chi squared p=0.318

    The HQ distributions are not significantly different between Rusk and No Rusk either. If I include NQs in the stats and assume steps 2 and 3 are not independent, the difference in the distributions becomes (p=0.316.


    I'll keep synthing avarites on this mule for fun and profit, and keep updating the stats accordingly.

  13. #13
    Salvage Bans
    Join Date
    Feb 2008
    Posts
    997
    BG Level
    5
    FFXI Server
    Bismarck

    I wouldn't mind seeing the break rates if it's not a hassle for you Byrth. Curious to see if it reflects my own personal rates when I use a rusk. Awesome job on the sample size as well, would have expected the rusk to be a lot higher.

  14. #14
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    Bad news. Apparently I stopped logging my mules because I was trying to cut down on dropbox space, and then I didn't turn logging back on when I started using them to craft. So I don't have any of the break data. I did set them up to log now, though.

    If you suspect that Rusks contribute to break rate, this is also what I have suspected.

  15. #15
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    Ok found some my old crafting data. Did a T1 synth with 2 accounts synthing exact same times side by side same gear lvl etcs. 1 with rusk 1 without. Was doing roasted corn so no different HQs and no plug in to tell me and before rings and such. Got bored and never finished so sample size isn't huge but 1890 synths for each

    Coconut Rusk: NQ - 1674 HQ -135 breaks - 81

    None: NQ - 1716 HQ -93 breaks - 81

    I think it might just be more noticeable because less NQs or often if not using rusk you'll use kitrons? Also not discounting the possibility of greater odds of actually breaking stuff when you fail but I didn't think of that at the time. That said rusk had a much larger effect for me it seems than you.... a lot of possibilities on that though from things working differently with multiple hqs to error to you having different break rates and hq rate being determined before success/fail instead of after success, rings fucking with things smaller changes in T2 etc

  16. #16
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    Hmm.. actually that difference is highly significant (p<=0.01) pretty much no matter how you handle it with a chi squared test.

    Did:
    * Rusk vs. non-rusk NQ/HQ contingency table (p<0.01)
    * Rusk vs. hypothetical 15:1 T1 NQ/HQ rate (p<0.01)
    * Rusk vs. non-rusk NQ/HQ/Break contingency table (p~=0.01)

    etc.

    I am confident from that data that Rusks do matter for that synth. Differences are:
    1) T1 vs. T2
    2) Level 2 synth vs. Level 81 synth
    3) I was wearing a Craftmaster/Artisan's ring pair, while you were not
    4) Simultaneous vs. not, but your non-Rusk is pretty much exactly what I'd predict from normal theory

  17. #17
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    Probably multiple + hq rate spaghetti doing crazy things. This was also data from like a couple years ago so stuff might have changed

    Either way probably needs more testing

  18. #18
    Old Merits
    Join Date
    Nov 2015
    Posts
    1,181
    BG Level
    6
    FFXI Server
    Asura
    WoW Realm
    Cho'gall

    I wonder what their reasoning is for purposely excluding rusks from the recent food item effect descriptions... Maybe the current devs don't even know how they work themselves? XD

    ..Still kicking myself for brainfarting and asking this question about Macarons instead of Rusks during the AMA. >_<;;

  19. #19
    Ridill
    Join Date
    Apr 2011
    Posts
    23,736
    BG Level
    10
    FFXI Server
    Bahamut

    It's also possible they work in some really wonky way... which assuming neither of us fucked up our test kind of seems to be the case.

  20. #20
    BG Content
    Join Date
    Jul 2007
    Posts
    21,104
    BG Level
    10
    FFXI Server
    Lakshmi
    Blog Entries
    1

    I want to know at what point I can stop the T1 Coconut Rusk test. The competition is Kitron Macarons, which were recently revealed to "increase the rate of synthesis success by 7%." My mule now has logger loaded and has been synthing darksteel ingots like they're going out of style while wearing Smith's / Artisan's rings and eating Kitron Macarons. We expect a 95% break rate for a single craft synth. I observe a ~1% break rate (4/667). I propose the success rate cap is 99% and I'm hitting it. If Kitron did nothing and the rings worked exactly as advertised, I'd expect a 97% success rate. I get a 99% success rate, so Kitron is doing something and it's not small.

    I bet Kitron is capable of capping my synth success rate all by itself. Thus, if we did 200 synths with a 50% mat loss rate...

    No Kitron:
    10 breaks - 5 lost, 5 retained and resynthed for an ultimate total of 4.875 successes
    194.875 successes

    Kitron:
    2 breaks - 1 lost, 1 retained and resynthed for an ultimate total of pretty damned close to 1 success
    199 successes

    That is effectively a multiplicative 2% boost in your HQ rate.
    T3: 50% -> 51%
    T2: 25% -> 25.5%
    T1: 6.25% -> 6.375%
    T0: 2%? -> essentially 2%

    So, unfortunately the effect of Kitron on HQs is going to be pretty negligible for the synths that people care the most about (and thus any Rusk test needs very considerable precision) but it might be a competitive alternative if you're making something like Darksteel Chains (T3, single craft).