Has Anyone Noticed a Decrease in RMT tells?

[Zosi]

I'm not saying that we, as players, should be able to make a 100% call on whether a particular character is RMT or not. What I said was that with access to the game's logs/databases/etc, it would not be hard at all to do. There will still be some ambiguous cases, sure, but the large majority would be trivial.

I'm saying that without access to the game's logs/databases/etc, you don't have a fucking clue whether SE's banning innocent people or actual RMTs. You can't say that you'd do a better job, because you don't know whether they're right. They have MUCH better data than you. That was a major point of my post, which you seem to have not even read.

My major issue with this is the way SE seems to be implementing their detection methods. The problem of detecting RMTs needs to be looked at in the same way as detecting spam is. For those not too familiar with detection algorithms, there are two ways of classifying incorrect detections. A "false positive" is when your algorithm classifies an item as the type that it was meant to detect, when it should not have been detected. A "false negative" is when your algorithm should have detected it, but did not. So in terms of an RMT-detector, a false positive occurs when you classify a legitimate player as an RMT. A false negative occurs when you classify an RMT as a legitimate player.

It should be immediately obvious that a false positive is an extremely serious problem, whereas a false negative is relatively minor in the big picture. As I said, this is how spam detection algorithms are designed. If the odd piece of spam slips through into someone's inbox, it's not a big deal, they just delete it. As long as you blocked the other 150 blatant ones that they would have received that day, they'll hardly care. However, if that person was expecting an important piece of email and your algorithm trashes it as spam, that's a critical flaw.

What makes you assume that SE doesn't consider false negatives/positives, aside from the fact that their bans don't precisely match up with your preconceptions of who the RMTs are? Again, as you're so fond of pointing out, we don't have the data they do. From your position, it's extremely foolish to bluster about how trivial and easy it is to catch RMTs when you have no idea about the accuracy of their current process.

Overall though, if SE would just add some reasonable restrictions to the trial accounts, they (and by extension, we) wouldn't even be having this problem. Don't let trial accounts garden or fish. There's probably not much else that you can do in 14 days that'll generate a significant profit, so that'd plug most of money-making holes right there. Also, their communications need to be restricted somehow to prevent website spamming. There are many possible solutions, from extreme (don't let trial accounts send tells at all) to lenient (limit the rate at which trial accounts can send tells).

I agree with most of that, and they could learn a lot from WoW, where trial players are quite heavily restricted. However, they have to strike a balance, because more than most MMOs, FFXI is a social experience. In WoW or similar games, not being able to whisper other players or trade isn't a huge deal because you can be self-sufficient for a much longer period of time than your average FFXI newbie.

Edit: To summarize, you seem to be claiming the following:

1) Correctly identifying RMTs is trivial if you have access to player data
2) Square has access to player data
3) Despite #1 and #2, Square is ineffective at identifying RMTs

The logical problem is that without access to the player data yourself, you have no way of proving #3. The fact that they're banning people who don't fit your standard profile of being RMTs should alert you to the possibility that your profile may be incorrect, but instead of that, you seem to have jumped straight to "Well, SE must be doing a shitty job!"

[Kipling]

I know about it and I'm jealous.

I play on 360 so I obviously can't use it.

I would switch over to computer, but I would have to buy a new computer and at least a 30" flat screen TV to play it on. So that amounts to like 700-1000 just to play FFXI. Maybe when I have more money.

But I am really thrilled at this. Nothing more annoying then being in /tell chat with your friend and a RMT pops up and you end up mting the RMT.

And lol at the price jumping up on gil. Guess I should have bought gil, held on to it and RMTed it back to people for $30 more/mil.

Lol, I know how you feel.. or I should say I kind of do. I play on PC but my gf usually just uses her 360 and when I hear "Goddamnit, these fuckers won't leave me alone again!" to my left I know what it is without looking, lol ; ;

[Epik]

On a slight derail, has anyone seen a change in RMT crafting? I know on sylph the cookbots seem to be out to lunch at the moment, for the most part.

[Elites]

No way, got 2 tells in a row today.

[Blackbrock]

I *did* notice a decrease in RMT tells but then they saw this thread and today I got a tell asking for me to SELL MY GIL TO THEM.
Like wtf scoob?

[Rakiel]

I *did* notice a decrease in RMT tells but then they saw this thread and today I got a tell asking for me to SELL MY GIL TO THEM.
Like wtf scoob?

I got several tells like that today. Much lulz were had.

[Fallyn]

like $40/1Mil, too. SE hit them hard lol.

[Kyte]

My Brogame/Brosale /tells have greatly diminished, but I have gotten new /tells from some other company (forget the name, but the price was significantly more). In fact, the same bot /telled me 3 times over the course of just a few minutes.


EDIT: I think the name was getgold.com, or something like that.

[Deimos]

Giant pro-SE rant

Seriously? Have you ever even played FFXI? You're really going to take the position of "SE is infallible and would never screw something like this up, everyone else must be lying!"?

Have a look in the top few threads on this forum alone, there's one about someone getting (temp?) banned for naming their linkshell SpecialTaskForce, and one about the possibility of being auto-banned for NPCing items to Proud Beard.

Not even going to bother to write much of a response to that. I honestly can't figure out if you're trolling, or if you're just such a huge SE fanboy that you've blinded yourself to their general incompetence in handling things like this.

[McGoonagle]

What makes you assume that SE doesn't consider false negatives/positives, aside from the fact that their bans don't precisely match up with your preconceptions of who the RMTs are?

I know one person who got banned who is no more an RMT than Kate Moss is an aircraft carrier. She is also someone who puts a fair bit of effort into confounding RMTs and has filed more than her fair share of reports about them.

Unless you can prove, as opposed to imputing ignorance on my part, that she is the only false-positive, then you have to accept that there is a problem with the procedure, model, or algorithm used.

SE is keeping very quiet about this (even [GM]Dave), and I think they know they got it wrong. Let's hope they solve it quietly too.

[Zosi]

Seriously? Have you ever even played FFXI? You're really going to take the position of "SE is infallible and would never screw something like this up, everyone else must be lying!"?

No, I'm not going to take that position up, or anything close to it. Please, for the love of god, learn how to read. I didn't say that SE was infallible, and I certainly didn't say that everyone else must be lying. I said that you had no logical support for your assumption that SE has everything completely wrong, and that you're ignorant if you think that you would be able to do a better job given the same data.

Have a look in the top few threads on this forum alone, there's one about someone getting (temp?) banned for naming their linkshell SpecialTaskForce, and one about the possibility of being auto-banned for NPCing items to Proud Beard.

Again, I certainly didn't say that SE is infallible, simply that they have better data than you, and that it's rather foolish to claim that you could do a better job when you don't have *ANY* idea what the actual data is. If you'd like to contradict that, feel free to show me your data. You won't, because you have NONE, just your own arrogant belief that you could do a better job than the STF.

Not even going to bother to write much of a response to that. I honestly can't figure out if you're trolling, or if you're just such a huge SE fanboy that you've blinded yourself to their general incompetence in handling things like this.

If those are the only two possibilities you see, then you really need to open your eyes. How about a third? You're wrong, but are too stubborn to realize it, so you lash out at me and call me a fanboy or troll because you don't have an actual argument.

I'm no SE fanboy - I despise them for what they've done to a franchise that I've loved since I was renting FF games for my SNES with allowance money. I hate what they've done to FFXI end-game, and their continued inaction on very basic things that would make a world of difference. Despite that, even I have to admit that the STF has been quite effective since they started their crusade against RMTs. There certainly have been both false positives and false negatives, as happens with pretty much any kind of automated process - but again, we as players aren't in a particularly good position to even know when those are occurring, or what percentage of the whole they are.

For all YOU know, the people posting that they got autobanned for gardening may have been selling excess gil that they got using that method or buying gil and using gardening as an excuse to explain how they got it. Both are quite common possibilities, yet you assume that they're innocent based on nothing but their word. Few of them post screenshots of actual GM conversations, and as we all know, GMs aren't very good at relaying correct information even when they do tell you the conditions of a ban. Again, I'm not saying that they are RMTing, just that there's no way for us to know if they are or not. It's premature and reckless to assume either way, and you seem to be quite fond of blindly siding with anyone who says that SE is fucking everything up.

[Zosi]

I know one person who got banned who is no more an RMT than Kate Moss is an aircraft carrier. She is also someone who puts a fair bit of effort into confounding RMTs and has filed more than her fair share of reports about them.

Unless you can prove, as opposed to imputing ignorance on my part, that she is the only false-positive, then you have to accept that there is a problem with the procedure, model, or algorithm used.

SE is keeping very quiet about this (even [GM]Dave), and I think they know they got it wrong. Let's hope they solve it quietly too.

I have never claimed that there are no false positives, and I have certainly not claimed that their procedure, model or algorithm is perfect and flawless. My main assertion (which I think is entirely reasonable) is that the STF does a decent (no, not perfect) job, and that it's extremely unlikely that some random poster on BG (Deimos) would be able to do a better job. Try actually reading my posts rather than Deimos's ill-thought-out responses.

P.S. - Deimos, if you really want to prove me wrong, you're welcome to post your algorithms for finding RMTs here. Clearly, if it's as trivial as you claim in your posts, you should be able to write something up in a few minutes that will work better for finding RMTs than whatever the STF uses, right?

Double edit: "even [GM]Dave" makes it sound like you think that [GM]Dave is actually associated with SE. Are people really that gullible?

[McGoonagle]

Double edit: "even [GM]Dave" makes it sound like you think that [GM]Dave is actually associated with SE. Are people really that gullible?

Sarcasm detector offline?

[Deimos]

I'm honestly not sure what your problem with me seems to be, we seem to mostly agree. We both think that it's impossible for a normal player to make the call, but I lean towards the side that they're probably making some mistakes, whereas you seem to think that they're probably not. All I did was make a comment about how they seem to be banning innocent people, and you immediately attacked me with "oh yeah I'd like to see you do better" (which you're still doing). The next post, the first thing I said was that it's impossible for me to know as a player whether someone is RMT or not, but that if you have access to the data, it's a lot easier to make a call. Despite that, you seem to be focused on defending SE from my horrible unfounded accusations. I have no idea where all your hostility about it came from.

My overall point was that in the context of banning RMTs in FFXI, false positives are an extremely serious flaw. I am totally confident that if I had access to the game data, I could achieve zero false positives. I'm not some random kid here saying "lol se sux I could do dat job bettr", I have a degree in computer science, I work every day at my job with massive databases, I do a ton of data analysis. I'm a senior moderator on scorehero.com, an honor-system-based site for competitive Guitar Hero / Rock Band scores, and I've personally banned and subsequently investigated the appeals of thousands of users for posting fake scores on the site. I'm quite experienced with using intimate knowledge of a game's inner workings to determine when people are cheating/lying. I'm not just being an armchair quarterback here, I know how to do this kind of work.

Now, if I actually was doing the STF's job, and it was a key objective to have zero false positives (as it should be), would I have ended up banning less people overall than they did? Yes, absolutely, but that's the price of being careful. Get the really blatant ones automatically, flag the ambiguous ones and let a human review it. If there's any uncertainty at all, don't ban them. Many people have literally poured years of their life into their character, you can't just treat them as a statistic, an "acceptable loss".

Now, do I know for sure that they banned any people unjustly? No, and I'm not trying to say that I do. However, from reading the posts both by and about some of the "normal players" that got banned, it doesn't seem very likely that they were secretly RMTing/exploiting. Could they have been, and everyone vouching for them is either lying or just ignorant? Sure, that's possible. But looking at SE's history of making mistakes, it seems much more likely to me that their criteria just weren't quite perfect.

Hell, Sage Sundi even admitted in an interview that one of their early iterations of an exploit-detector was run on a live server and it immediately threw multiple innocent players into jail because it was buggy. It's a known fact that they've made mistakes with this kind of thing in the past.

Looking at some of the information related to these particular bannings, it seems clear that they're making at least some mistakes here as well. Not necessarily in the false-positive direction, because (as is exactly your point), we can't possibly know that. But they're definitely making some mistakes in the false-negative direction. Several people in the Allakhazam thread posted that they have multiple gardening accounts, and that not all of them got banned. They were all doing the same thing, all sending the money to the same main character, etc, but not all of them got banned. If that person was actually RMTing gil or using gardening exploits and just lying about it, why wouldn't all their accounts get banned? Note that this is not the same thing as people saying "no, I didn't use any exploits!" to save some face. There's absolutely no motivation for the player(s) to make up this kind of story, there's no benefit to them by pretending that they had multiple accounts, or that not all of them were banned.

If SE's detection algorithm is so good, why didn't it catch these obviously-linked accounts, registered to the same credit card, performing the same tasks, and sending gil to the same central player? It obviously has some holes, my assertion is just that there are probably holes in the other direction as well.

But don't get me wrong, despite my general pessimism towards SE, overall they did a very good job with this round of bannings. I switched from getting 10+ tells a day telling me where to buy gil with constantly-falling prices to tells begging me to sell them my gil, for over double the price I could have bought it last week. They obviously decimated a good-sized RMT operation. I just don't like the possiblity that some totally innocent players had to get caught up by it as well.

[McGoonagle]

I have never claimed that there are no false positives, and I have certainly not claimed that their procedure, model or algorithm is perfect and flawless. My main assertion (which I think is entirely reasonable) is that the STF does a decent (no, not perfect) job, and that it's extremely unlikely that some random poster on BG (Deimos) would be able to do a better job. Try actually reading my posts rather than Deimos's ill-thought-out responses.

The STF has done a pretty good job of getting rid of RMTs, just not this time. As Napoleon found out, you're only as good as your last battle, and this time they got it wrong.

Algorithmic banning may work, but it will have to work better than it has this time. SE will have to admit to the mistakes, make them good, and be more careful next time round. Until then, they might want to stick with more labor-intensive and less efficient methods with people in the loop.

For the algorithmic bannhammers, here's a little conundrum. If you saw a new character called "Vsegda" would you suspect an RMT? All the letters are clustered on the left side of a QWERTY keyboard. You might think it's a keyboard slapper.

You might, but it's also an anglicized spelling of the Russian word for "everywhere."

[Zosi]

I'm honestly not sure what your problem with me seems to be, we seem to mostly agree. We both think that it's impossible for a normal player to make the call, but I lean towards the side that they're probably making some mistakes, whereas you seem to think that they're probably not. All I did was make a comment about how they seem to be banning innocent people, and you immediately attacked me with "oh yeah I'd like to see you do better" (which you're still doing). The next post, the first thing I said was that it's impossible for me to know as a player whether someone is RMT or not, but that if you have access to the data, it's a lot easier to make a call. Despite that, you seem to be focused on defending SE from my horrible unfounded accusations. I have no idea where all your hostility about it came from.

Sigh, you still don't read. This post is what pissed me off:

Wow, this is pretty bad even for SE. Are they really that bad at distinguishing legitimate players from RMTs? In less than a minute of thought I can think of several ways that banning real players could have been avoided when you have full access to player data.

You specifically say that if you have access to SE's data, that YOU COULD DO BETTER. I've never asked you to do better *without* that data, what I have said is that without the data, you really have no idea about the quality of the bans as a whole. In other words, there's no way you could know that you could do better, because you're in the dark about how well they're actually doing. Your arrogance about your ability to judge with the facts is supported only by judgments made without the facts.

I think our major point of disagreement besides that is that you seem to view having zero false positives as being absolutely required, while I'm of the opinion that it's unreasonable to expect zero false positives out of any kind of useful system. I agree that minimizing false positives is a good goal and that SE hasn't done everything in their power to do that, but overall I view the system as satisfactory. I'll touch base on this a bit more in the other thread, but I think they have a fairly good balance now between banning people with hard, confirmed RMT histories (via their money trail-following tool or similar) and people who are simply extremely likely to be RMTs based on analysis of their behavior. The spot where I believe that they need the most improvement on is handling appeals. It *is* unacceptable that there's no predefined appeals process to contest an automated ban. False positives are inevitable, but they're ignoring opportunities to correct them afterwards and improve both the accuracy of their system (via tightening and correcting their models) and their reputation/image among players.

[Deimos]

I think the reason I view zero false positives as required is because of exactly what you're saying. There is no appeals process at all, the whole banning/unbanning system seems to be kept entirely secret, and they almost never give anyone any sort of closure. "You're banned, I can't tell you why or do anything about it, but I'll make a note that you called. You might hear back next week, maybe. If you're still banned after that, you can send a letter to this guy that may or may not even exist. It won't do anything, it never has for anyone else, but go ahead. Hope to see you online!"

Having seen them do absolutely nothing to improve their customer service over the 5+ years I've played this game, I think I've basically given up on that end of things. So if a proper appeals process doesn't exist and never will, yeah, you have to put more weight on doing your detection properly in the first place. If there was a reasonable way to appeal, I agree, it wouldn't be a big deal if a few innocent people got hit and were quickly reinstated with an apology and perhaps a free month for their inconvenience. The very act of trying to appeal is actually a pretty decent indicator that they're not RMTs, at least in the Chinese-sweatshop sense. Those types won't bother to appeal, they'll just start more trial accounts or whatever their current method of choice is.

And yes, I do think that I could do better. I don't know that I could, and I didn't really mean to imply that it was a certainty, or come off as arrogant about it. It's not like I'll ever have the chance to try, so it's kind of pointless to argue about whether I'd actually be able to do better or not. My point was just that, speaking as someone that's knowledgeable about the game (many of SE's staff seem not to be, though I can't speak to the STF specifically), and experienced with both data analysis and the process of investigating cheaters, it seems to me that they could do a much better job. Seeing as how Miori just posted in the LM-17 thread that she's been reinstated, there seems to have been at least one false positive, and the situation that she described for her account looked to me like a fairly easy one to exclude.

Anyway, hopefully this ends up in sparking some useful discussion in the other thread I started. It's a good topic, and people as deeply-involved in the game as general BG members should have some useful insight.

[Aksannyi]

I think the reason I view zero false positives as required is because of exactly what you're saying. There is no appeals process at all, the whole banning/unbanning system seems to be kept entirely secret, and they almost never give anyone any sort of closure. "You're banned, I can't tell you why or do anything about it, but I'll make a note that you called. You might hear back next week, maybe. If you're still banned after that, you can send a letter to this guy that may or may not even exist. It won't do anything, it never has for anyone else, but go ahead. Hope to see you online!"

Absolutely this.

[Liminality]

RMT sales are going down and the price of gil is going up. Simple economics.

[Siniroth]

To be honest, I'd never got a RMT tell before, just got one on my trial account that I'm poking around with for lack of funds to play Siniroth

Scratch that, TWO RMT tells in under 5 minutes, o_O