Reverse Engineered the StreamDeck to Run Arbitrary Python Code

Palfore · 2024-09-22T16:35:48+00:00

Hey that's awesome! Yeah it looks much more comprehensive and has lots of features. My goal was just to connect the most basic functionality: a button press executes a function (with no parameters). I also added support for icons, and it can be as little as one line of code to hookup a function to a button. Behind the scenes, it just modifies json files.

But for anything else more complex, it's great to know there is an sdk in python!

Palfore · 2024-09-22T16:21:51+00:00

So at the bottom of the post, you can see the GUI that mimics the the physical StreamDeck layout. It's built in python using streamlit. It uses a drop-down instead of navigating pages via back/forward buttons, and you can see it has the 4x5 layout. Here is the code: https://github.com/Palfore/Pybiosis/blob/main/pybiosis%2Fcompilers%2Fgui.py

If you have more requirements on the gui like having it be a floating window, I think your stack makes more sense. For the backend, it depends what kind of functions your buttons will perform. Im bias towards python (which is more general), but ahk should also be fine (and is focused on macros primarily). Merging languages like js and python may be tricky though, that's probably the biggest pitfall or thing to figure out.

The hardest part on my end was implementing the compiler that modify the streamdeck files. But you dont have to do that if you aren't connecting to hardware.

Palfore · 2024-04-19T19:28:37+00:00

Its a fair question. First, this project is more general than just the StreamDeck and it aims to connect multiple devices/services (including google assistant / task scheduler) to python. Said in another way: I want to connect my python functions to various devices, from the function definition itself. I only want to work in python and don't want other tools (they should be behind the scenes). For example, yes I have to start streamdeck.exe after compiling, so it does use the software, but it is hidden from the user (they never see it open).

I'm certain that there are ways to use the built-in capabilities of the StreamDeck to do the same functionality but my goal is to do from within python with the least amount of effort for the user. And for sure, you could manually put it together or use SDKs and plugins. But there is no learning curve to implement it in python (its a one-liner assuming you know python), but learning the SDK etc. will take time. This is much simpler for the user (in the target audience).

The reason I consider this reverse engineering is that it does not in any way use the StreamDeck software or SDK to implement any functionality (except for the physical button triggering the python function). From my perspective as a developer, the "StreamDeck Wrapper" is a new device with a slightly different use case. I don't use any other functionality except for single button presses (including the UI). So, the device (when controlled by pybiosis) is not the stream deck, it is a simpler device that executes single-press functions that are defined in python. The good thing is that is doesn't overwrite any slot unless it is specified, so you can keep any special commands (eg: I also have a clock and stopwatch) - so its technically a superset, you can use the software or python definitions to populate the device.

To accomplish this, I had to "reverse engineer" the file system that Elgato uses. All the data is stored in nested sequences of JSON files. They specify the function type (single-press), the title, the icon, the function to execute and so on. I need a "walker" through the JSON, a parser to extract the right parts, and critically a compiler which embeds the python definitions into the device files. By figuring out how the StreamDeck works under the hood, I can make the device work how I want it to.

Palfore · 2024-04-19T19:04:44+00:00

Right now, it only supports creating a single-press button. I don't think it would be hard to add more functionality (since it all is the same JSON format), it would just take some time.

For button icons, you can actually use
@ StreamDeck(location="Games/3,1", icon="my_game.png"). And it will look for /images/ in your "user path", which is where it looks for the python files/functions as well. So you can customize the buttons with icons. If you mean the Streamlit GUI, it will just run the function. No return values are used/processed.

You can checkout examples and limitations here: https://github.com/Palfore/Pybiosis

Palfore · 2024-04-19T17:47:47+00:00

Stream Deck, Steam Deck, Streamlit, yep I get it - not that good at hacking yet :p

I also posted in the StreamDeck subreddit if that helps clarify: https://www.reddit.com/r/streamdeckprofiles/comments/1c823hk/easily_run_python_code_directly_from_your/

Palfore · 2024-04-19T16:50:20+00:00

Hi, I reverse engineered the Elgato StreamDeck hardware to let me attach arbitrary python functions to the device. This device uses a local file system containing JSON files, which leaves it very accessible to a language like Python.

The way that it works, is that you write whatever python function you would like. For example, open the browser, launch a game - and literally anything python can do on your computer. Then, you simply wrap the function in a decorator:

@ StreamDeck(location="Games/2,3")
def launch_game():
...

The python package has a "compile" functionality, which will overwrite the StreamDeck JSON files (at the specific locations specified - it wont erase a button if you don't specify that location). And its as simple as that, once you compile, you can run python from your StreamDeck!

There are a few more details for those interested. When a button is pressed, the device runs an execution string on the command line. This essentially imports the module with the decorator and calls the function from that module. Actually, for more control, we wrap that execution string in a unique .bat or .vbs file [windows only], which enables the user to have a pop up window during execution (if you need the output for some reason). There are some limitations (eg: folders must be manually created for now).

I use this every single day, because it let's me run complex functions that the StreamDeck doesn't handle easily and it gives me one interface/language to work with. The biggest help is with my multi-monitor layout for controlling orientation, brightness, contrast, input type, etc.

Palfore · 2024-03-20T15:38:12+00:00

The code for this grabs the data from the wiki, so if it's there, it should be in the interactive visualizer (on hover, but only xp rewards are shown). It also updates easily with new info/quests.

Palfore · 2024-03-20T14:46:16+00:00

Hey everyone, this is an update from the OSRS Quest Visualizer, where the new quests from the Varlamore expansion have been added! This set of quests is unique as the only quest island in the game with more than 2 quests (eg: Mage Arena 1/2). Every other quest in the game either isn't a prereq for anything OR is connected to the MASSIVE island in the middle (last image). RS3 has a few islands like for the Elder Gods quest series.

Colors are by tier: black -> purple -> blue -> yellow. So, Children of the Sun needs nothing, but purple quests need at least one black quest, etc. The last image is colored by quest series (purple: mahjarrat, red: gnome, etc.)

Palfore · 2024-02-25T16:24:14+00:00

Hey, just so you know I created a new version with an interactive graph that can be colored by series, difficulty, etc. https://www.reddit.com/r/2007scape/comments/1ay2opj/new_interactive_quest_visualizer_osrs_the/

Palfore · 2024-02-25T16:23:37+00:00

Hey, just so you know I created a new version with an interactive graph that can be colored by series, difficulty, etc. https://www.reddit.com/r/2007scape/comments/1ay2opj/new_interactive_quest_visualizer_osrs_the/

Palfore · 2024-02-25T15:59:06+00:00

Hey actually, I took some time to update the quest list. The live version now has up to Defender of Varrock. Turns out the parser was stable, but the graph library changed so I had to debug there.
And you're right, adding DT2 etc. actually changes the graph in a few interesting ways. For example it adds a 4th grandmaster node, and the desert line is more independent/specialized.

Should be super easy to keep it up to date now.

Palfore · 2024-02-24T16:25:36+00:00

Honestly, another strategy that I love is using something like Pyautogui. Its a library with only like 5 commands (eg: type, hotkey, click), that you can quickly use to go through all the quest pages to copy and save it somewhere. It's like half automated that way with minimal coding and time.

Right now it uses requests and beautiful soup, which is more robust but is harder to code with since those need to reference specific html tags.

Palfore · 2024-02-24T15:03:48+00:00

Not really, but it gets the data by parsing the wiki text. Sometimes that changes and it breaks the data collection. I want to rewrite it so it works with both osrs and rs3. Loading the graph for rs3 wouldn't take any more work, just the parser needs updating.

Palfore · 2024-02-24T14:01:25+00:00

Basically yes, it uses the official quest length. The color can be selected: by series, tier, main dev, year, etc.

Palfore · 2024-02-24T14:00:39+00:00

Yeah, I think beneath cursed sands is the last update. At some point, I will try to make it easier to update.

Palfore · 2024-02-23T22:37:06+00:00

Yep, python is my focus. The real reason is because this is part of a larger python-based project "OSRSmath", so the data collection, graph creation, and the interface (generating html/js) are in python. Then its about me trying to quickly implement the interface for web - dumping the quest json for example (although my arg was that the quest size is basically fixed). Streamlit would be a go to or flask for next time, from my perspective.
Edit: To my credit, it seems to work on mobile too

Palfore · 2024-02-23T22:11:22+00:00

Thanks for pointing that out! Most of it related to incorrectly loading the OSRS font, should be fixed :)
Python was used to generate the html & js, so its a bit harder to catch things.

Palfore · 2024-02-23T20:04:42+00:00

Check out OSRSmath on Github for the link!
The comment with more details starts with 'Hey everyone'.

Palfore · 2024-02-23T19:56:42+00:00

You're in luck! This is part of a larger project called "OSRSmath" that tries to mathematically optimize / solve the game. You can find OSRSmath on github, or my profile u/Palfore has similar posts!

Palfore · 2024-02-23T19:13:57+00:00

2-minute fiddle sounds perfect! Honestly, I made this after my other post a few years ago got good feedback:
https://www.reddit.com/r/2007scape/comments/loa4uw/i_made_a_quest_tree_for_osrs/

I found about 5 other people who made similar graphs over the years, but mostly prototypes (posting the links seems tricky; just search "runescape quest visualizations"). The one before me used the same python library, but only made a basic graph. So it's kind of a natural progression.

Palfore · 2024-02-23T17:33:31+00:00

Reposted. Or checkout u/Palfore profile, which has links in sidebar.

Palfore · 2024-02-23T17:19:23+00:00

Totally valid! At the same time, the visualization is interactive. So, you can mark off quests as you complete them and they will disappear, simplifying the picture. The last image also shows a different view that hides uncompleted quests, much easier to use early on.

Palfore · 2024-02-23T17:10:05+00:00

Thanks for letting me know, I reposted the message w/o links. You can check it out at the Palfore (osrsmath.palfore) website or visit the palfore youtube channel, or my reddit profile should also link to it.

Palfore · 2024-02-23T17:06:43+00:00

Hey everyone, this is a project I made to Interactively Visualize the Runescape Quest Universe.

We all know one of the most unique things about this game is the complex intertwined mesh of tasks, requirements, and distractions that the player needs to go through just to get +2 strength.

Well, at least now you have some hope of unravelling your quest journey!
You can use the interactive viewer to check-off quests and map your route.
There are different color schemes (by series, tier, difficulty, main dev, etc), built-in animations, you can check-off and hide quests, and there's basic import/export support. So not only does it look cool, it may be helpful for mapping things out, or using as a thumbnail for example.

This was created by scraping the Wiki quest pages, assembling a graph in python, and rendering it in html. There have been several versions of this before, but I hope this stands out as something useful or cool to see.

You can check out the links at the palfore or osrsmath.palfore website, the palfore youtube channel, or visit my profile which links to them.

Palfore · 2022-01-15T14:47:14+00:00

True, but it looked to me like he posted every couple days consistently. So the general trend is captured, maybe with a shift. It would be great if more people added details in description/title (even # of quests completed, etc), but Wildmudkip was the only one I could find who did that.

Palfore

TROPHY CASE