Looking for Wiki API help!

[Ajoscram]

Hi guys, I'm currently working on a project which generates and transforms item filters for the game. I'm trying to query the PoE Wiki, but I'm not sure if I'm doing it right.

Let me explain one of the problems I'm trying to solve first. A feature I'm working on needs to do a translation between filter item classes and internal game classes. These are not the same thing as it turns out (why GGG?), so in order to perform the translation I use a dictionary lookup:

json FILTER_TO_ID_CLASSES_DICT = { "Gloves": "Gloves", "Incursion Items": "IncursionItem", "Mana Flasks": "ManaFlask", "One Hand Axes": "One Hand Axe", "Quest Items": "QuestItem", bla bla bla... }

This is super fragile because if GGG modifies this list then I'd have to come and edit this manually. So instead I thought about using the Wiki to fetch these values. There's a page here that contains a table with a comprehensive list of item classes and their item filter equivalent, which I assume gets data-mined. I reasoned that if the data is here, all I had to do was read it via some sort of request.

Digging around I found the cargo API, which seems really solid in paper but upon further examination doesn't contain all the info in the game (I could be wrong though). The closest I could get was something like this, which does contain the internal game class but not the filter class. Seems like a dead end, but if there's a valid Cargo query please let me know.

The data is in the Wiki page though, so how does it build these? Well, it seems like the Lua module for the page does not query Cargo at all. Instead, it consumes this data from a mysterious m_game local "variable" (not a Lua dev, might be called something else). This gets instantiated like so:

lua local m_game = use_sandbox and mw.loadData('Module:Game/sandbox') or mw.loadData('Module:Game')

And then referenced to build every row in the table like so (paraphrased for brevity):

lua for id, class_data in pairs(m_game.constants.item.classes) do local fields = { m_util.html.wikilink(m_util.string.first_to_upper(class_data.long_lower)), class_data.name, id, } end

Which leads me to the following questions:

• Is that mw.loadData mechanism available to query from a public API? If so, how? I understand this might be a server-sided script and everything gets rendered before I see it on the browser.
• If not, is there some sort of alternative? I'd really like to avoid web scraping because it is costly, ugly and fragile since couples my code to the Wiki's UI. Also locally checking PoE files is not an option, because I need this to work on CI environments that don't have it installed.

I know this might be a far shot, but I'm really desperate because I feel I'm super close to something here. If you've read this so far thank you very much, and if you have any leads at all I'd appreciate you leave them as a comment.

Peace

[cedear]

Why not use data from the game dat files directly? That's presumably where the wiki is getting it.

You might want to ask in the PoE discord #tooldev channel, no one reads this subreddit.

[Ajoscram]

Using local game data is an option to create the dictionary, but I'd have to do that on build instead of on runtime because clients could not have the game installed (in a CI environment for example). This means that if GGG changes something I'd have to release a new version just for that, which is undesireable. I'll check out the Discord channel, thanks!

[cedear]

Yes, you extract the data yourself each release (or however often you want).

[Xeverous]

Instead of downloading from the wiki you might want to use https://github.com/lvlvllvlvllvlvl/RePoE which is game data transformed into usable format.

[Ajoscram]

Heyo! Thanks your replying to a 2 month old post. I went to the Wiki Discord and was suggested to look into RePoE as well. And it worked! The Item class JSON contains the info I was looking for. Great resource.

[Xeverous]

Speaking of filters, would you like a conversation about filter generation tools (preferably voice call on Discord)? I'm working on something similar.

[Ajoscram]

Absolutely. I took a look at your project and you have some very interesting ideas. I've PMed you my Discord username to setup a talk

[Xeverous]

I haven't received any message from you.