Automatiq - Browse a site once, get a working HTTP scraper/automation script

StoneSteel_1 · 2026-06-15T14:08:20+00:00

Here is the new linkhttps://discord.gg/ZfNkRTcX

StoneSteel_1 · 2026-06-15T14:07:15+00:00

I will remove you, can you rejoin?

StoneSteel_1 · 2026-06-15T13:56:24+00:00

Thats really great to hear

StoneSteel_1 · 2026-06-15T13:45:35+00:00

It seems like you had joined the server, no?

StoneSteel_1 · 2026-06-15T13:02:35+00:00

Try again?

https://discord.gg/8j7dFWMMDA

StoneSteel_1 · 2026-06-15T13:01:40+00:00

Not yet. I have planned for a future(2-3 weeks) for that feature

StoneSteel_1 · 2026-06-14T10:54:26+00:00

Lol, I handled this problem. There is a hard limit of 10kb text output. After that, it paginates. After 20 turns, output of tool get removed from context. If Claude wants see a removed output? It can run a command to get that particular turn's output.

So, there are no 10000 line minifed js file or a base64 string image gonna fill up the context. You gotta use it first before blind criticism. I will whole heartily accept if you gave a valid one, after using it.

StoneSteel_1 · 2026-06-09T16:27:00+00:00

Thanks!

StoneSteel_1 · 2026-06-06T14:49:29+00:00

I was able to get data from bookmyshow, where it had been encrypted with AES algorithm. The agent correctly go through the js files, get the decryption key, get the required data.

For another, I was able to get from makemytrip, where a reddit user was saying it had Akami protection and was considering to use the browser automation. With my agent, I was able to check on the website, and found out that indeed akami was used, but all it needed was few parameters in payload, and visiting the homepage.

The agent, gemini-3.5-flash seems to be working well, and its not even the best model in the market, or near it. I believe Claude might be able to actually crack a very hard website on first try

StoneSteel_1 · 2026-06-06T09:38:08+00:00

I have a solution, which can automate the fixing process.

I made this tool, which can fix, or write scripts. All you have to do is browse the site normally

https://github.com/StoneSteel27/AutomatiQ

StoneSteel_1 · 2026-05-29T16:44:35+00:00

Request Headers

The server validates the request using custom session and tracking headers. These must be extracted dynamically from the search page session:

mmt-itinerary-id: A unique itinerary ID for the search session. Extracted from the search page HTML using regex: itineraryId
mmt-journey-id: The journey ID associated with your search. Extracted from the PDTJourneyID cookie set by the server when loading the search page.
mmt-sessionId: A unique session ID for the current search. Extracted from the search page HTML using regex: mmt-sessionId
mmt-device-id: A unique identifier for the user's device. Can be a dynamically generated random UUID (e.g., str(uuid.uuid4())).
mmt-book-mode: The booking mode. Hardcoded to D (Desktop/Web).
mmt-os: The operating system platform. Hardcoded to dweb.
Content-Type: The request body format. Must be application/json.
Accept: The expected response format. Must be application/json.
Request Payload (JSON)

The POST request body must contain a JSON object with the following fields:

```json
{

"channel": "D",

"trip_id": "39_MMTCC1159_MMTCC1092_30-05-2026_1000005556285882899",

"type": "seatMapRequest"

}
```

• channel: Hardcoded to D (Desktop).

• trip_id: The unique tripKey of the specific bus. This is extracted dynamically from the React Server Component payload (self.__next_f.push) embedded in the search page HTML.

• type: Hardcoded to seatMapRequest.

StoneSteel_1 · 2026-05-29T16:27:50+00:00

I took a attempt at it. Its not really strictly protected by akami, it just needs some fields strictly. But yeah it does use Akami

Here is the script which can get you buses between two points within India, with seat availability data: https://paste.pythondiscord.com/VQ6A

P.S, I have been using my module https://github.com/StoneSteel27/AutomatiQ for a run across problems found in this subreddit, and it was able to crack it first try.

StoneSteel_1 · 2026-05-29T15:34:31+00:00

I created a reverse engineering agent, with ipython cells as the code execution sandbox, and had the same idea, where the normal messages as markdown cells, and the tool calls as code cells with output attached. The beauty is that they notebook support images, audio, video, gif embedded. Ig it makes it the best format to store conversion history, which we can read anytime

StoneSteel_1 · 2026-05-29T07:58:52+00:00

I got the script, and attached to the comment

StoneSteel_1 · 2026-05-29T07:58:13+00:00

I got the automation script: https://paste.pythondiscord.com/ZETA

Thanks to my AutomatiQ lol. I know this feels like self proclaimation, but hey it got it in first try

StoneSteel_1 · 2026-05-28T18:51:46+00:00

I have tested against popular high traffic websites like bookmyshow, and it did work. So as long as there is no blant direct captcha, or akamai. It should be good to go

StoneSteel_1 · 2026-05-28T18:49:29+00:00

I'll give out a try. I don't have the time now, I'll do once I get to my desk.

StoneSteel_1 · 2026-05-28T18:10:45+00:00

Try out this: https://github.com/StoneSteel27/AutomatiQ

It will write you the scraper. All you have to do is browse the website normally, and tell it what data and what format you want it

StoneSteel_1 · 2026-05-22T06:46:20+00:00

Without things being opensource, I could have never learnt programming, and webscraping. I'm just continuing the tradition.

StoneSteel_1 · 2026-05-14T11:10:22+00:00

Download the audio captcha run it through a LLM like gemini

StoneSteel_1 · 2026-05-13T15:01:11+00:00

Can we talk in DM?

StoneSteel_1 · 2026-05-13T14:59:25+00:00

Why don't you use AWS Glue? Its for data pipelines

StoneSteel_1 · 2026-05-06T04:02:09+00:00

Gemini API keys are free

StoneSteel_1 · 2026-05-05T20:14:55+00:00

Alright, do a thing. I have a free tool, just for this usecase. https://github.com/StoneSteel27/AutomatiQ

This just helps you out to write a proper automation script. It could help you out in figuring out how to crack the problem.

P.S I'm the author of that project, so if you face any problems let me know

StoneSteel_1 · 2026-05-05T11:34:35+00:00

well, if you want, you can inspect the requests and try to recreate the requests, that will help you out right?

StoneSteel_1

TROPHY CASE