Working Parser?

[Issac]

Yes, GraniteParse ftw. :D

[Zhais]

Haven't have chance to open the source yet =.=
How come the add effect is not parsed?

as far as I know, added effects /should/ be parsed. if they are not, that's a bug


sourceforge project still under review, but there'll be a place for bugs and such once that opens up.

tho, probably the first thing anyone needs to do is go through everything and start commenting since we don't have divisor working on it anymore.

[Alleya]

DirectParse? Since it parses directly from FFXI program memory as opposed to indirectly through the logs.

Any objections or better suggestions?

that works.


and what is it alleya!

I was going to say, have a secret data column that takes the total damage done and converts it into $USD, then calculates how many mandaus you can buy with that dollar amount. >.> I have nothing against dvs, I'm just an asshole like that, lol.

[Amele]

Its really hard to get an accurate WS average with "OLD DVS" as it really is just your Hi/Low and the average is in between the two.

No it's not, the average is correct.

wasn't there an old bug where the displayed average would only update when a new high or low was recorded? (The average was still correct, it just didn't reupdate the field until you got a new hi/low.)

[dvsparse]

Its really hard to get an accurate WS average with "OLD DVS" as it really is just your Hi/Low and the average is in between the two.

No it's not, the average is correct.

wasn't there an old bug where the displayed average would only update when a new high or low was recorded? (The average was still correct, it just didn't reupdate the field until you got a new hi/low.)

Yes and I fixed it through a convoluted caching mechanism. problem is that if you score a normal hit, or a crit hit, or whatever, you want some columns (crit%, hi/avg/low, etc) to update, but other columns (spell damage, for example) aren't affected at all. if you have everything update every time, it will be too slow because of the rate at which lines come in and the fact that I used a virtual list control for the display. actually recalculating all that shit that many times per second will make it unusable (and it did when I first tried to fix that bug). so the caching method is a way that if a stat changes, you know all the other stats that were affected by it. if you look in one of those xml files you see a list of dependencies, that's how you specify what calculations are affected by what stats. so it will only recalculate the ones that had the potential to change, and leave the rest alone.

Anyway what the original guy was saying was that he thought Avg is just (Hi+Low)/2. On the contrary, if you take Total Skill Damage / Avg you get the number of WS used.


To answer VZX's question about the additional effect, it will never work correctly with any parser that exists now or in the future unless SE modifies their server. in the "header" that comes in with every single line there's 2 numbers. one is a message id, which is always different for every single message that ever comes across. another is what i called a "conversation id" which links together related messages, typically those that break across a line but sometimes messages which are related but don't actually break the line have the same conversation id. critical hit is an example, the part that tells you you got a crit and the part that tells you how much damage both have the same conversation ID. anyway long story short the message that tells you that you got an additional effect always has a unique conversation ID, so you can't link it back to which attack actually caused the additional effect. furthermore, there is no identifying information such as the player's name in the actual additional effect message. the way it works now is that it attributes the additional effect to the last person in the chat log who performed an actual attack. but this is complicated by the fact that additional effect messages are not guaranteed to come in immediately after the attack that caused it, and they often don't when there is a lot of battle spam. so they often get attributed to the wrong person in a party when you have them turned on. If you're solo and have your filters for Other people turned off, it should be pretty accurate.

Wall of text, but w/e.

[Zhais]

Its really hard to get an accurate WS average with "OLD DVS" as it really is just your Hi/Low and the average is in between the two.

No it's not, the average is correct.

wasn't there an old bug where the displayed average would only update when a new high or low was recorded? (The average was still correct, it just didn't reupdate the field until you got a new hi/low.)

old, new, and still current bug. edit: maybe fixed? thought it was still buggy


after talking with divisor and looking at code a bit more, not sure why MB stuff isnt working as its formatted the same as criticals... but additional effects aren't linked directly to the same entry as the weapon hit. That makes it difficult to assign which effect goes to which player when there's a lot of spam. Do other parsers accurately parse added effects? or do they just assign them to the previous player in the log.

At work/school, so can't test right now



edit: aaand, beaten by wall o' text :d

[Nameless]

When will drain spell damage ever be recognized?

[Amele]

yeah I knew you had fixed it (hence the old adjective) but I wasn't sure about the specific details. I figured it had to do with the xml but I don't have source at work so I couldn't look into it here.

zhais: most other parsers either don't list added effect at all or just assume that the order is right in the log (which is it often but not 100%). a few list them out as separate effects I believe.

we're doing the best job we can with them really - if SE supported third party tools at all (I wonder sometimes, since it does have the conversation IDs.. what these were intended for) we could request a change to the conversation IDs but I doubt SE will do anything with that request lol.

edit: Drain spell damage is listed as recovered hp, and the actual spell damage is hidden (drain at full HP still does damage to the mob but says '0 hp') so, never gonna be able to list drain spell damage

[Bestovian]

I will test again as soon as I get a chance.

Only reason I mentioned was because I was doing some Y/G/K and Kaiten testing about a week or so ago just to see how much it would actually be changing.

Well I will just post the Kaiten numbers, as that was the reason for me bringing it up.

This was only a 10 WS base average done.

WS #1 - 1415
WS #2 - 1678
WS #3 - 419
WS #4 - 1498
WS #5 - 1567
WS #6 - 1249
WS #7 - 732
WS #8 - 1871
WS #9 - 986
WS #10 – 1428

Total Damage from WS was 12,843.
WS Hi was 1871 with the Low of 419.
Average shown on the parse was 1163, which honestly was a little more then right in between but not much. However knowing I only did a total of 10 WS I knew the average was off because it should have shown 1284. The weird thing is the total WS Damage was exact to what I had written down just the average was off.

Just seemed weird is all is the reason I brought it up.

[Amele]

that's an interesting "average" (it's not an average of any subset 1..x of that set) in fact, I don't think it's an average of any contiguous subset of that set.

looks like something might still be broken after all.

[VZX]

To answer VZX's question about the additional effect, it will never work correctly with any parser that exists now or in the future unless SE modifies their server. in the "header" that comes in with every single line there's 2 numbers. one is a message id, which is always different for every single message that ever comes across. another is what i called a "conversation id" which links together related messages, typically those that break across a line but sometimes messages which are related but don't actually break the line have the same conversation id. critical hit is an example, the part that tells you you got a crit and the part that tells you how much damage both have the same conversation ID. anyway long story short the message that tells you that you got an additional effect always has a unique conversation ID, so you can't link it back to which attack actually caused the additional effect. furthermore, there is no identifying information such as the player's name in the actual additional effect message. the way it works now is that it attributes the additional effect to the last person in the chat log who performed an actual attack. but this is complicated by the fact that additional effect messages are not guaranteed to come in immediately after the attack that caused it, and they often don't when there is a lot of battle spam. so they often get attributed to the wrong person in a party when you have them turned on. If you're solo and have your filters for Other people turned off, it should be pretty accurate.

I know it won't work properly (for now), but is the function currently disabled?
Sometimes when I play RNG, it's only me, holy bolt RNG, that generates additional effect on the log. So, to me, making it parses the add effect is enough. No need to parse it correctly.

Will it be a simple fix?

[Zhais]

zhais: most other parsers either don't list added effect at all or just assume that the order is right in the log (which is it often but not 100%). a few list them out as separate effects I believe.

we're doing the best job we can with them really - if SE supported third party tools at all (I wonder sometimes, since it does have the conversation IDs.. what these were intended for) we could request a change to the conversation IDs but I doubt SE will do anything with that request lol.

I always figured the threadIDs and message type #s were for chat coloring client-side, and also for filtering purposes.

Will it be a simple fix?

mayyyyyyyyyybe

[Amele]

I thought coloring was done exclusively by chat ID and not conversation ID?

[dvsparse]

I know it won't work properly (for now), but is the function currently disabled?
Sometimes when I play RNG, it's only me, holy bolt RNG, that generates additional effect on the log. So, to me, making it parses the add effect is enough. No need to parse it correctly.

Will it be a simple fix?

Maybe I'm not understanding something, but there's already an option to display Additional Effects last I remember, all you have to do is open the dialog and check the correct box. Also I thought the color was specified in the header independently of the chat id and the conversation id, but i can't say I remember tbh.

[Zhais]

sourceforge project approved, I'll get the svn setup when I have time tonight or tomorrow, and get the correct files posted up.

sign up on sourceforge and pm me if you want to be added as a developer on it.

[Croaker]

Since it looks like a few people will be working on the parser, I would love to see a feature that allows for damage per second. It might be as simple as reading the timestamp when the parser starts and reading it again when the parser is paused or stopped. Or it might get more complicated and pause the clock in the interval between the mob defeated message and when damage resumed.

I mainly use the parser to measure self improvement and a DPS feature would allow me to test gear setups when solo.

[Sonomaa]

the primary reasoning to get the parser into other peoples hands is so it wont be taken down if somebody gets a leaky vagoo because they got caught rmting their mandood, in regards to the accusations of stolen code, Im going off a third partys notation of comparing the chat parsing timings to another popular application developed before dvsparse was started. The code in question (no, I havent looked at it) was involving the chat parser which pulls directly from FFXI code, which at the time very few app makers had figured out

Im giving you the benefit of the doubt that you didnt steal the code, but Im not going to be entirely convinced until I see some proof or lack of proof otherwise. You havent given me reason to believe you over other people, but Ill take your word for it for the moment and have some deeper digging done.

[Amele]

dps is both really simple and stupidly complex.

in an ideal world, as you said, dps would be running when you were actually engaged and would not be running when you aren't engaged (I haven't bothered searching to see if there is a flag for 'engaged' in memory; I would assume there is)

but then blackmages don't track unengaged magic damage....

it's very difficult to track monster death to start of next fight (especially since many monsters share names and it's not uncommon to start engaging monster #2 before #1 is dead)

in a simple world you just divide time since parse started to now into total damage - but that's a really coarse number and not *really* useful.


there's a bunch of options in between.


so, yeah; a dps feature could be added (relatively easily) but that doesn't mean a useful dps feature could be added relatively easily.

if there's a way to detect engaged status that might be best. if there's a way to detect *everyone's* engaged status that'd be even better. etc.

the primary reasoning to get the parser into other peoples hands is so it wont be taken down if somebody gets a leaky vagoo because they got caught rmting their mandood, in regards to the accusations of stolen code, Im going off a third partys notation of comparing the chat parsing timings to another popular application developed before dvsparse was started. The code in question (no, I havent looked at it) was involving the chat parser which pulls directly from FFXI code, which at the time very few app makers had figured out

Im giving you the benefit of the doubt that you didnt steal the code, but Im not going to be entirely convinced until I see some proof or lack of proof otherwise. You havent given me reason to believe you over other people, but Ill take your word for it for the moment and have some deeper digging done.

so, wait; they were comparing parse timings from two direct applications at executable speed without debuggers and without looking at the code? (and in either case: onus of proof is on the accuser in most democratic societies, regardless of prior wrongdoings by the accused. the person providing 'proof' here needs to be the accuser; especially since dvs has already provided his code as open source; whereas the accuser has not even provided the name of his app to the public)

I would be interested in seeing this 'timing comparison'. because I have trouble believing that whoever this is is seriously expecting anyone with any coding knowledge at all to believe that someone can distinguish duty cycles in the sub 500ms range with sufficient resolution to claim the underlying code is identical, without doing it via a debugger interface: and since they presumably have both sources it's highly suspect that they don't simply prove it by doing a source comparison and need to resort to 'timing comparisons' in an environment where most timings are defined by a single property on a built in timer interface and are readily and easily manipulated.


(also, the chat files are fairly easy to find and they're not *that* hard to track - the only difficult part is that it is divided into two sections but once you've monitored the whole space for at least 100 lines you'll be able to follow the logic and the header pointer to current line sticks out like a sore thumb (I had identified it before I had even identified that there was a structure of other useful data around it when I first went looking) the only thing left to do is trace the pointer references back until you find a stable pointer - which to be honest isn't easy - it's not at the head of its nearest pointer and it's got more than one dereference: but that's not 'hard' to figure out so much as it is time consuming; especially when there's 12 files sitting on the harddrive with all the information waiting for you right there.

not many people in the community necessarily had done it yet; but it doesn't take a genius to decode that memory map either - in fact, I would argue that it's easier to decode that map than it would be to deconstruct a closed source application (depending on specific platform) and try to determine the shape of the memory map from that application

[dvsparse]

the primary reasoning to get the parser into other peoples hands is so it wont be taken down if somebody gets a leaky vagoo because they got caught rmting their mandood, in regards to the accusations of stolen code, Im going off a third partys notation of comparing the chat parsing timings to another popular application developed before dvsparse was started. The code in question (no, I havent looked at it) was involving the chat parser which pulls directly from FFXI code, which at the time very few app makers had figured out

Im giving you the benefit of the doubt that you didnt steal the code, but Im not going to be entirely convinced until I see some proof or lack of proof otherwise. You havent given me reason to believe you over other people, but Ill take your word for it for the moment and have some deeper digging done.

If you're talking about Wade and LogMon, research it all you want. I talked to the guy once through PM and he didn't tell me anything I didn't already know. Not that it matters anyway since I figured everything out myself except what StarHawk told me, but the actual memory scanning code is about 1% of the entire project.

The more time you spend carrying out your childish vendetta against me even though you already technically "won" and have had the freedom to gloat about it for the past 3 months, the more I can sit back and laugh at how utterly pathetic you are, especially since you probably know jack shit about programming yet you're sitting here trying to accuse someone of stealing code. It's clear where your priorities lie, and they definitely dont' lie with trying to protect someone's intellectual property. Research away, shithead.

[Sonomaa]

did I do anything to you? I dont recall doing anything but changing your custom title to something more fitting, maybe you are getting me confused with taj. Take up your issues with him, and no, its not wade.