Sunday, 14 September 2014

Market History Lies

As I've said before, market history does NOT always reflect what really happens in the market.

Here's some more proof.

PLEX prices in The Forge (screenshot taken at Jita 4-4)

First off, according to market history there were 0 PLEX sold in The Forge on 2014-09-14. I know that for a fact that's completely out to lunch since I logged in several times throughout the day for the sole purpose of checking the price of PLEX on Jita.

Second, as recorded in my wallet transactions, I bought 5 PLEX myself on 2014-09-14. I'm also certain a bunch more sold to a large blocking order even lower than my own buy order since the stack size of that blocking order (which I've been watching quite closely) went down by at least 16 since earlier on the same day.

Third, I paid 33 million (and change) less for each of those 5 than market history says was the lowest price for the day. Yet history reports the low price for the very next day (the 15th) is even lower (by just 3 ISK). No, this isn't due to GMT time differences since both times are in game time (in fact it's still the 14th here in Canada as I'm writing this).

Fourth, how do you get non-zero low, high and avg numbers if there were no sales? That's something I see often with rare, low volume items that sell maybe 1 a week, month, or even less frequently. On those kind of items you'll see a whole bunch of 0 sales to 0 orders days with the occasional sale of 1 or 2 usually at a much lower price than the so called low, high and average prices.

I'd guess there was some kind of server error leading to the loss or corruption of market data for yesterday's PLEX sales and when that happens it defaults to numbers for the last day or average numbers for the last week/month or whatever. Not a big deal with high volume items like PLEX but a bad practice for more rare items that don't usually sell even 1 a week far less every day.

Obviously when there are no sales CCP must be substituting high/low/avg numbers derived from history in leiu of actual real data. In some cases it even looks like they might be using listed prices of items priced so high they are not selling and just sit there not selling for months. If they are using listed prices they should stop because not only is that very misleading, it's also highly subject to manipulation.

BTW, during the hour after I started writing this post, I sold 4 PLEX at 791 million each and bought another 3. After taxes and broker fees that's a little under 20 million profit per PLEX or about 2.5% profit at roughly 4.7% margin. Typical numbers for me in PLEX markets on the main hubs though I usually make at least 4-5%, sometimes as much as 10% or even a bit more in other places.

Considering I made nearly 80 million in an hour from PLEX alone on just one trader *without camping I have to wonder just how much ISK I could make per hour camping PLEX + a few other items on Jita during weekend prime time. Especially if I ground faction standings to chop another 1% off combined taxes and fees. It'd be interesting to find out but I hate camping and grinding.

Edit: After a short unscheduled server downtime now there is market history data for the 14th.

But the low price is still 27 million higher than I paid.


  1. Sometimes the market data gets lost. When there's zero entries on a given day, the prices get carried over from the previous day (the average I think goes across all 3, but it's early so gimme a break) so that the average price gets carried. Look at items that are nearly never sold and you'll see the same effect.

    As for pricing differences, there's some hidden criteria which drops certain sales off of the pricing to stop people being able to manipulate prices too easily. Strong outliers almost certainly get dropped, but there's other circumstances things will not qualify too. CCP keeps what the criteria is close to the chest for obvious reasons.

  2. As far as i understand this, the low/high are not simple min/max of all sales, but in itself represent a "moving min/max" value calculated over time.

    So to use the plex example, i would assume that getting plex at 755m was only in a short window of time and the rest of it is more close to the "low" value this day. The other problem is that it combines buy and sell orders and we have no separated history or i would expect the values would make more sense.

    I don't think the history "lies", since it never actually states that it will display the "lowest" singular sale this day. In fact the history chart has not a single value that displays singular sale/buy events. The idea for those charts is to display a "smoothed" or averaged record of what happened this day.
    So unless you account for 30-50% of the volume this day at 755m, it is totally fine to "record" the history of the price at min 782m and max 793m.

    I always interpret those values as "the most likely price" i can get. So i would expect that i could buy at 782m with little effort and at the average 791m at none.

    Don't get me wrong, i would also like more numbers and certainly see reasons to also include something like a daily 5% min/max of that day, but personally i'm not interested in the "lowest" or "highest" sales number, without also having the volume listed.

    So just interpret those numbers as "smoothed" values, rather than representing absolute or singular events.

    1. I meant "low" and "high" rather than "min", "max". This is the semantic distinction btw, since min/max would actually mean something different than more loose terms "low", "high".

    2. Yup, understood. I took it to mean high / low anyhow.

      I can't argue with anything you or lucas had to say. After all you seem to think it works pretty much the same way I think it does. I just don't like the way it works. I'd real prefer hard data with real numbers, not numbers massaged, manipulated and edited to show what some coder or economist thinks the numbers should be.

      Sure I can understand dropping extreme outliers (0.01 ISK sell orders and 10x typos kind of thing) but IMO they go way too far with that, it winds up distorting the data and misrepresenting what's really happening.

      Granted that's not a big deal with high volume items like PLEX but like I said in the OP it is a problem with slow moving items that don't sell every day.

      And yeah I'd love to see it further broken down with separate numbers for buy orders and sell orders but I don't see that happening.

    3. BTW, they could easily fix most of the problems with extreme outliers if they'd just change the order filling logic so when you enter a buy order way higher than the lowest sell order it fills your order from existing sell orders at the sell order price first. And just the opposite for sell orders lower than existing buy orders, fill them from existing buy orders at the buy order price.

      Just like how real life markets work.

    4. Just bug some CSM member or directly CCP about it. We already have corp transaction history and lately the corp blueprint list in the API, so maybe they can also add a "raw" history to the Crest/XML API, since it requires no changes to the client/game.

      We also need to keep in mind, that the ingame market values are designed for your average "Joe", so defining what are "good" low/high values is subjective. I personally would use something like this:

      1) remove the lowest and highest 1% of transactions this day.
      2) calculate the lowest 5% percentile and highest 5% percentile

      I guess for some items 10% or 15% percentile might be more informative to the average "Joe", so you see its not this easy to work this out.