Spellcast Help Thread (Link your XML using Pastebin.com for help!)

[Cymmina]

So functionally speaking, how would we reduce lag? Basically use a dummy spell instead of using aftercasts? This is essentially what I do in my SMN spellcast, because you need something like a 3 second lag swapping back to your idle set after a BP.

That doesn't need a dummy spell. Set your aftercast delay to 3.

Processing code is faster the fewer the commands there are to evaluate and execute. That's how programming works. It's actually pointed out in the SC docs (see "Performance Tips"). Even if a condition is false, it still costs processing time to evaluate the expression (unless it will be skipped when it breaks out of an if/elseif/else block).

[Byrthnoth]

That doesn't need a dummy spell. Set your aftercast delay to 3.

Oh yeaaaah <_<;

Processing code is faster the fewer the commands there are to evaluate and execute. That's how programming works. It's actually pointed out in the SC docs (see "Performance Tips"). Even if a condition is false, it still costs processing time to evaluate the expression (unless it will be skipped when it breaks out of an if/elseif/else block).

Unless I'm mistaken, Yugl's examples have the same number of comparisons in both cases (really, it has more in the no-lag case). It's one comparison that happens all the time, and one comparison that happens if the first comparison is false.

I guess I could rephrase my earlier post as, "What are we theoretically minimizing here? Number of equip commands?"

[Yugl]

My ultimate goal will be to have bigger brackets* and try to roll all <equip> commands through Dancing Chains. Effectively, trigger spells will solely be responsible for setting variables.

* By bracketing, I mean changing from

<if>
<elseif>
<elseif>
<elseif>
<elseif>
<elseif>

to

<if>
<elseif>

with a series of <if> + <elseif> between

I'll probably have . . .

<AllAutomaticProcesses>
<trigger spells>
<Magic|Songs|Ninjutsu>
<WeaponSkills>
<JAs>

I wouldn't recommend changing your XML until I make these changes and test it out though. Changes to sets might include the removal of $TP and change set names to Engaged. This would enable dancingchains to do <equip set="%Status-$VARTP|$Impetus|$Armor|$Movement (New addition thanks to Motenten). Since MNK exhibited the most lag, I'll see if it works.

[Motenten]

So functionally speaking, how would we reduce lag?

You can get a rough idea of how much processing the plugin uses by checking the debug output for various actions. If you use a weaponskill, or cast a spell, how many lines of debug output is generated? How many IF checks made, entering deeper levels of XML structure and then returning to higher levels? How much work is done constructing the final set of gear for each stage (precast/midcast/aftercast)?

First look at the simplest XML possible: idle set, JA set, and swap to the JA set gear when using the JA. Try again with a full weaponskill set. Compare those with how much debug output is generated with the full template versions.

Edit note: My own revision to Yugl's code generates about 630 lines of debug output (from initial command received to final aftercast set) for a weaponskill command.

I would probably start with the pattern matching. Eliminate glob patterns (*) as much as possible. For example, the very first line of the mnk xml rules is

<if notSpell="$TriggerSetOne|$TriggerSetTwo|$TriggerSetThree">

Where those three variables are defined as

        <var name="TriggerSetOne">Shackled*|Grim*|Dancing Chains|Vulcan*|Barbed*|*Schism|Carnal*</var>
        <var name="TriggerSetTwo">Poison V|Poisonga V|Scop's*|*Pastoral|*Fantasia|Raptor*</var>
        <var name="TriggerSetThree">Netherspikes|Foxfire|Diaga V|Banishga V|Goblin Gavotte</var>

That means *every single action* has to do 10 ambiguous pattern matching checks. With a proper regex library, that shouldn't be too big a deal, really, but it's extra work that doesn't need to be done.

Plus, we don't know how efficiently coded Spellcast really is. The 'easy' way to handle this type of coding is usually very slow, but acceptable as long as you're not doing hundreds of checks at a time. Going from 100 microseconds to 10 microseconds is inconsequential if you're only running the function a few times, but run it a few hundred times and the delay starts being serious.

Basically use a dummy spell instead of using aftercasts? This is essentially what I do in my SMN spellcast, because you need something like a 3 second lag swapping back to your idle set after a BP. We still need to keep precasts for stuff like weaponskills, JAs, etc.

The main problem I see with this is that there's likely to be a large amount of duplicated coding; you just gain due to moving some of it to a later point in time. I actually was just building a (very simple) smn xml yesterday while levelling it a bit; simply set the aftercast delay for blood pacts to 4.0.

I suppose one must ask, what is most of your xml code's time spent on? Constructing the precast, midcast, or aftercast sets?

It seems to me that aftercast is the largest element. Precast and midcast ranges from trivially simple (equip this piece of gear when activating the JA) to only mildly complicated (enfeebling magic (fixed value based on spell), mnd-based spell (easily defined spell set), X accuracy tier (set by user, generally fixed for large periods of time)). Aftercast is generally where all your state checking goes on (return to idle or melee? accuracy gear? PDT set? what buffs are active? etc), though some of the state checking applies to precast as well.

As such, yes you could defer the aftercast work to a delayed command. Is it worth it? I don't know, especially for jobs that are heavily JA-dependant, where the lag moves from before the current JA to before the next JA.


So, in theory you'd have:

Trigger actions which pass through immediately to set variables.

Normal actions which proceed:
<equip when="precast" set="$PrecastSet">
<equip when="midcast" set="$MidcastSet">
<command when="aftercast">/ma AftercastTrigger</command>

And then an explicit aftercast section that is only accessed by the AftercastTrigger action.

The problem is, suppose you do Boost+Weaonskill. Since the Boost's aftercast is a command trigger, it could be going off at the same time as the weaponskill's precast, with no way of saying which will happen first.

Still... It does seem like it has potential. I may put something together for Yugl to test.

[Yugl]

I'll definitely keep auto-aftercast equipment, but if I can change it so that it equips %Status-$VARTP|$Impetus|$Perfect Counter|$Armor|$Movement instead of equipping $TP, then I can reduce the number of equips going on at the same time in certain instances. For example, look at the following rule for footwork:

        <!--- Abilities: Footwork --->
        <if spell="Footwork">
            <var cmd="set STYLE KICK" />
            <var cmd="set TP TP-$STYLE-$VARTP|$PerfectCounterSet|$ImpetusSet|$Armor" />
            <if status="engaged">
				<equip when="midcast" set="$TP" />
			</if>
        </if>
        <elseif notBuffactive="Footwork">
            <var cmd="set STYLE H2H" />
            <var cmd="set TP TP-$STYLE-$VARTP|$PerfectCounterSet|$ImpetusSet|$Armor" />
            <if status="engaged">
				<equip when="midcast" set="$TP" />
			</if>
        </elseif>

This rule would continually try to equip $TP if Footwork is not active. If I can get the above plan to work, I can reduce the code to

        <!--- Abilities: Footwork --->
        <if spell="Footwork">
            <var cmd="set STYLE KICK" />
         </if>
        <elseif notBuffactive="Footwork" advanced='"$STYLE"="KICK"'>
            <var cmd="set STYLE H2H" />
        </elseif>

[Byrthnoth]

I think we're only really picking up ground if we condense things using more broad categories, like Types, followed by more specific categories, like "which WeaponSkill?" For instance, asking if the spell being used is one of the 6 specific Fomor WSes is 6 string comparisons, after which we have to ask which specific Fomor WS it is. By moving all the Fomor-WS related set adjustments under your <if Type="WeaponSkill> comparison, you stop them from processing most of the time and don't pick up any delay. It also lets you reduce the <elseif notSpell="$TriggerSetOne|$TriggerSetTwo|$TriggerSe tThree"> statement down to <else>.

I've somewhat majorly reorganized my Dancer xml based on this, and also I'm placing my if/elseif tree in the order of most frequent use. So Dancer-specific job abilities are followed by weaponskills which are followed by Ninjutsu and then things like Poison V/Poisonga V that are rarely used.

[Yugl]

Yes, that's what the post before my previous post described. Of course, bracketing alone won't be sufficient (Tested this already), so I'll need to couple this with the additional changes I proposed. Another major impact of using this is that many of the include rules will be dragged out or shifted to fit the correct bracket. If possible, I would like to maintain return rules as is, but if this produces too much lag, I'll have to break up the rule.

[Motenten]

Something else occurred to me when considering how Spellcast may be accessing sets.

The simplest and fastest method would be with hash tables, which give an O(1) lookup speed (constant). However they can't work for two reasons:

1) Wildcards in set names. I have, for example, the following sets:

        <set name="TP-Staff-Acc*">
        </set>

        <set name="TP-FW-Acc1">
        </set>

        <set name="TP-FW-Acc2" BaseSet="TP-FW-Acc1">
        </set>

        <set name="TP-FW-Acc3" BaseSet="TP-FW-Acc2">
        </set>

So I can specify the set with the construction: TP-$STYLE-Acc$AccLevel. In the above case, Footwork with accuracy 1, 2 or 3, or staff with any accuracy. TP-Staff-Acc* can't be put in a hash table because it isn't resolvable into a single defined value. I could have $AccLevel=Pumpernickel and it would still match.

2) Building on the above, ambiguous set names are resolved in declaration order. Example:

        <set name="MoonTP-1|MoonTP-2|MoonTP-3|etc..">
        </set>

        <set name="MoonTP-1*">
        </set>

        <set name="MoonTP-2*">
        </set>

If my moon TP set is defined as MoonTP-%MoonPct then moon phase 1% will catch set MoonTP-1, while moon phase 10% will be handled by MoonTP-1*. If MoonTP-1* had been declared above the MoonTP-1|MoonTP-2|etc set then it would have been the set used for moon phase 1%, and MoonTP-1 would never be used.


That means that set names Must have a linear order, and set name resolution must also be linear; that is, O(n) instead of O(1). For small numbers of sets this isn't an issue. However, as the number grow, this causes progressively increasing strain on the rules parser.

For my own mnk xml, I have.... 62 sets within the xml itself (some of which have wildcards, and not all sets written yet) and 94 imported sets (with two wildcards); a total of 156 sets that may potentially need to be searched through. Depending on set order, this could be very bad. For example, my utsusemi set is the last one in the list, so all 156 set names need to be compared every single time I cast that.

Yugl's is a far more conservative 50 sets total. Most example xmls tend to not have more than a couple dozen.


This leads to another issue: variable names. They -could- be stored in a hash table since variable names are required to be distinct, however it seems unlikely because of how variable name composition works.

A common one in early general includes was $%SpellElementStaff. In order to do the composition, it has to extract %SpellElement from that. However, it can't know beforehand how many letters are used to define that. While this example uses a predefined variable (%), it's the same thing if it used a user variable ($).

So, it has to look for %S, %Sp, %Spe, etc, all the way til it finds a match, then replace that text with the variable value and repeat the entire thing again.

I have, for example, a set of vars I use to change spells when checking for !! procs in Abyssea. If I'm checking for procs, it will convert the specified tier 3/tier 4/etc spell to the one of the appropriate element (specified separately). The composition is thus: $T3-$ProcElement. All nifty until a problem I had while constructing this comes up: it was impossible to reliably compose the names using $AM-$ProcElement, $GA3-$ProcElement, or $Brd-$ProcElement; I had to use $T-AM-$ProcElement, etc.

While I'm still not entirely sure why, it seemed to be related to how the variable names were evaluated, and especially the order in which they were evaluated. Lower alphabet values failed, higher alphabet values worked.

The sc var list command also lists variables in alphabetical order. While it is likely at least for convenience in finding variables, it seems highly likely that variable name searches are done in a linear, alphabetical manner. So, another O(n) search for every variable use, and I've (potentially, for mage jobs) got dozens of variables.



I can't really offer any suggestions yet on how this can help fixing things, but I think it's likely to be a heavy contributing factor to Spellcast's lag; it just doesn't scale well with complexity.

[Byrthnoth]

Well, did what I described and put the results here: http://pastebin.com/8M2Qj5wp

It didn't reduce the lag entirely, but I like to think it helped.

I guess next thing I could do is go through and put my sets in order of use frequency. I believe you're right about how it processes them, Motenten, and in addition I think we know that it replaces variable names alphabetically (potentially one pass per variable?) before interpreting anything. That's why you're advised to use "aaa" or "_" or something in front of the inner most nested variable so it will be processed before the outer variable. I forget where, but one of the spellcast makers/pioneers had a guide that sort of tangentially described the variable processing somewhere.

[Motenten]

A test of debug output.

Version 1: nothing but the precast/aftercast equip rules (prevented anything else from running), using normal vars:
23-266 = rules parsing
267-332 = processing (equip commands sent to windower, wait delay, next equip commands sent, etc)

Version 2: only equip rules, no variable set, only base sets (pretty much what version 1 would have evaluated to):
23-108 = rules parsing
109-172 = processing (equip commands sent to windower, wait delay, next equip commands sent, etc)

Version 3: full standard xml parsing:
23-583 = rules parsing
584-651 = processing (equip commands sent to windower, wait delay, next equip commands sent, etc)


So, 560 in full version, 243 with layered vars, and 85 with nothing but a simple set.

With the full version there were 29 instances of ParseRuleActions (equip, var setting, etc).


A lot of those actions are things that -shouldn't- need to be checked constantly. For example, Town idle vs Field idle. The zone is getting checked on every single action to see which idle set I want to return to. How to minimize the number of times these types of checks need to be made?

Well, I have one trigger which is a 'reset' action. I use it fairly frequently to put gear back into place after something was messed up (eg: having equip screen open during weaponskill, preventing returning to melee gear), but it's a far cry from every single action, and I could see isolating those types of checks to only occur on a reset request rather than every single JA and spell used.

Similarly, something like Yugl's Berserk check shouldn't need to be made every single time through the xml. He sets a variable when the JA is used, it should be expected to remain in place for quite some time. A command to re-run the reset after ~185 seconds to clear it if the buff is gone should be sufficient.


Going to go through revising with that in mind. Will see what I end up with.

[Yugl]

I understand some of the stuff you guys mention, but without a background in computer science/programming, much of it is difficult to translate into solutions. That said:

A lot of those actions are things that -shouldn't- need to be checked constantly. For example, Town idle vs Field idle. The zone is getting checked on every single action to see which idle set I want to return to. How to minimize the number of times these types of checks need to be made?

Well, I have one trigger which is a 'reset' action. I use it fairly frequently to put gear back into place after something was messed up (eg: having equip screen open during weaponskill, preventing returning to melee gear), but it's a far cry from every single action, and I could see isolating those types of checks to only occur on a reset request rather than every single JA and spell used.

Similarly, something like Yugl's Berserk check shouldn't need to be made every single time through the xml. He sets a variable when the JA is used, it should be expected to remain in place for quite some time. A command to re-run the reset after ~185 seconds to clear it if the buff is gone should be sufficient.


Going to go through revising with that in mind. Will see what I end up with.

That is the looming issue for me. I don't think it is effective to just place a 180 wait due to dispelled buffs, zoning, and clicking off JAs (Not to mention, this is an issue for <if Area> codes). AutoExec is the preferably way to handle this, but this 1) forces you to rely on AutoExec and 2) means you cannot reload the XML and expect sets to function properly. I haven't devised a method that will prevent perpetual checks outside the use of DancingChains or perhaps a huge <if BuffActive="Berserk|Impetus|PerfectCounter" or NotBuffActive="Berserk|Impetus|Counter" in its own bracket.

[Motenten]

That is the looming issue for me. I don't think it is effective to just place a 180 wait due to dispelled buffs, zoning, and clicking off JAs (Not to mention, this is an issue for <if Area> codes). AutoExec is the preferably way to handle this, but this 1) forces you to rely on AutoExec and 2) means you cannot reload the XML and expect sets to function properly. I haven't devised a method that will prevent perpetual checks outside the use of DancingChains or perhaps a huge <if BuffActive="Berserk|Impetus|PerfectCounter" or NotBuffActive="Berserk|Impetus|Counter" in its own bracket.

180 is just the default. If you do nothing else in that 180 seconds, it will call the reset and revert to the non-Berserk var. You can always manually trigger a reset check which will clear the var if Berserk got dispelled, etc. I prefer not to use autoexec either, so I don't write anything expecting it.

Anyway, after some tweaking:

Requirements for revision: Anything in the Reset section is only allowed to check for states and set variables. It should not interfere with any other actions, nor be dependant on what JA or spell is being used. Changing groups is allowed.

Long-term JAs should set their variable in their own rule (eg: If type="Ability", if spell="Berserk", set BerserkMode=Berserk), as well as provide a command that will activate the reset trigger. The reset section checks for those active states (but not the use of the ability itself), and resets them to defaults if the buffs are not up.

The reset section can also check for non-JA states such as weakness.

I set up a var for the reset command to make it simpler (this uses my default reset trigger):

    <var name="ResetTrigger">Tranquility</var>
    <var name="ResetCommand">/ja $ResetTrigger &lt;me&gt;</var>

For duration buffs, I then include this command:

    <command when="Aftercast">wait $Duration-%Spell;$ResetCommand</command>

Where the $Duration-%Spell value is 5 seconds longer than the JA duration, for a safety margin. Different vars used for multi-word JAs.


Full test weaponskill debug:

23-525: rules parsing
526-593: equipping

So shaved 60 lines of rules parsing off of it (10% of debug lines).

Revised versions:
mnk.xml - http://pastebin.com/3EJe9NjB
include3.xml - http://pastebin.com/3zkvMkpH

Unfortunately can't comment on any changes to lag; maybe Yugl can test.

[Yugl]

I'll have to check tomorrow, but it's odd that the proposed solution is at odds with the idea of bracketing. Namely, if I loop one trigger spell into another (Say Barbed Crescent into DancingChains), then I effectively double the search work for spellcast. However, would finding a true comparison terminate processes for the rest of the XML? For example:

<If Trigger spell>
<elseif Magic>
<elseif Ninjutsu>
<elseif WS>
<elseif JA>

If I loop it so that a JA activates DancingChains (A trigger spell), would it terminate at the first comparison?

Edit: First list for MNK (Stuff not filled out is not necessarily a part of MNK XML; for example, SCH JAs)

1. Generic Check
	1. ReturnRules
	2. Level Restriction
	3. Area Rule
	4. LockedWeaponsRule
	5. AutoChangeGear (For Aftercast)
	6. Spell Cancel Rules
	7. Powder Boots Rule
	8. Obi rules
	9. AutoChangeGroup (When Abyssea variable changes)
2. Trigger Spells
	1. ShackedFists
	1. Grim Halo
	2. Dancing Chains
	2. Vulcan Shot
	2. Aegis Schism
	2. Barbed Crescent
	2. Carnal Nightmare
	3. Poison V
	3. Poisonga V
	4. Utsusemi
	5. Class Specific 1
	5. Class Specific 2
	5. Class Specific 3
	5. Class Specific 4
	5. Class Specific 5

3. Magic
	1. MP Notification
	2. EnSpells
4. Ninjutsu
4. WS
	1. Critical hit modified WS
		1. Victory Smite
		1. Ascetic's Fury
	2. Basic WS
		1. Asuran Fists
		1. Retribution
		1. Full Swing
		1. Tornado Kick
		1. Dragon Kick
5. JA
	1. Normal use
		1. Boost
		1. Chi Blast
		1. Focus
		1. Dodge
		1. Counterstance
	2. Variable Change
		1. Hundred Fists
		2. Footwork
		3. Impetus
		4. Perfect Counter
		5. Berserk
6. COR
7. SCH
8. DNC

[Motenten]

No, not double, though it depends on the implementation. The main idea is to short-circuit the extra checks if you're using the trigger action (checking for the trigger early), and avoid most of those checks otherwise. And it's not at odds with bracketing; you still want to use that to isolate segments of condition checks.

[Yugl]

Added an example list for MNK (Generic rules will need revisiting), but I'm debating removing generic WS bracket with Crit-WS and Normal-WS brackets since that would make less checks on WS at the cost of other actions; might be better if I move WS to the bottom of the rules list?(tired though, so maybe not!). Anywho, leave notes and I'll check back.

Edit: Ugh, I realize I was inconsistent, but same number either means the same bracket or the same line divided by pipes.

[Rellikard]

Is there a way to lock in a specific gear set if I'm engaged to a certain monster?

[Gustave]

<if spelltargetname="Leaping Lizzy" />

From the Windower Wiki page

"SpellTargetName <WC> |Tiamat |2.20 Allows one to set rules based on the name of the spell's target"

What are trying to setup?

[Rellikard]

Been getting a few complaints about gear swaps in voidwatch on PLD which I can understand so basically when fighting certain NMs, just don't way my gear to change.

[Gustave]

I would recommend them to blinkmenot or stal/stpc but that's just me.


Are you going to eliminate some or all blinking? Wanna pastebin your xml? It may be easier to set variable "sets" for non visible gear swaps.

[Yugl]

Tell your mages to use <stal> or BlinkMeNot. Otherwise, put this rule at the top of the list of rules:

<if Target="VWNMName">
<return />
</if>