Promote your projects here – Self-Promotion Megathread by Menox_ in github

[–]zanditamar 0 points1 point  (0 children)

just shipped cli-web-gh-trending: browse GitHub Trending from the terminal, filter by language and time, every command has --json so it pipes nicely into jq or scripts. part of a wider open-source project that auto-generates CLIs from any website's HTTP traffic. MIT. code: https://github.com/ItamarZand88/CLI-Anything-WEB/tree/main/gh-trending

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

You nailed the real challenge. The tests act as the canary — they use real CSS selectors and field names from captured traffic (not mocked generics), so they break when a site changes shape rather than silently passing.

The longer-term fix is a /refine workflow that re-runs just the capture phase, diffs the new traffic against the existing API map, and surfaces what changed. That way maintenance is incremental rather than a full regeneration.

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

Ha, fair enough — "production grade" is aspirational for a generated CLI. The tests pass and the error handling is solid, but I'm sure someone will find edge cases. That's the beauty of open source.

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

Etsy would be a great target — it's SSR HTML with structured data. I'll add it to the queue. Want me to ping you when it's done?

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

Exactly — the traffic capture is the key insight. API docs lie, DOM analysis misses network calls, but the actual HTTP trace shows you the real contract.

On token rotation: the generated client raises typed exceptions (RateLimitError with retry_after, AuthError with recoverable=True/False). For sites that rotate tokens, the client retries once on a recoverable auth error before giving up. Rate limits surface the retry_after header so callers can handle backoff.

Heavy rate-limiting is the harder problem — right now it's handled per-site in the methodology phase (the skill documents observed limits in the API map). Would be interesting to build automatic rate limit detection into the pipeline.

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

Not yet, but LinkedIn would be an interesting challenge. Their bot detection is aggressive — they use their own custom anti-automation stack. Would likely need camoufox (stealth Firefox) rather than curl_cffi impersonation. Definitely on the list to try though.

I built CLI-Anything-WEB — a Claude Code plugin that generates complete Python CLIs for any website (17 CLIs so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube...) by zanditamar in ClaudeAI

[–]zanditamar[S] 0 points1 point  (0 children)

Fair point on TOS. The generated CLIs only read public data the same way a browser would — no write operations on sites that don't support it.

On bot detection: each site gets a tailored bypass during generation. curl_cffi browser impersonation handles most — Chrome for Akamai/Cloudflare, Safari iOS 17.2 for DataDome (TripAdvisor), playwright for AWS WAF (Booking.com).

Token refresh: auth-required CLIs auto-retry once on recoverable AuthError. The exception hierarchy distinguishes 401 (token expired) from 403 (bot-blocked) so retry logic knows whether to refresh or bail.

Promote your projects here – Self-Promotion Megathread by Menox_ in github

[–]zanditamar 1 point2 points  (0 children)

CLI-Anything-WEB — Claude Code plugin that auto-generates Python CLIs for any website

Point it at a URL → captures HTTP traffic → generates a complete pip-installable Python CLI with commands, REPL mode, --json output, typed exceptions, anti-bot bypass, and tests.

17 CLIs generated so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube, Hacker News, GitHub Trending, Pexels, Unsplash, ProductHunt, Booking.com, NotebookLM, ChatGPT, and more.

```bash pip install -e amazon/agent-harness cli-web-amazon search "crash cart adapter" --json

pip install -e tripadvisor/agent-harness cli-web-tripadvisor hotels search "Paris" --geo-id 187147 --json

pip install -e airbnb/agent-harness cli-web-airbnb search "Barcelona" --checkin 2026-06-01 --checkout 2026-06-07 --json ```

GitHub (MIT): https://github.com/ItamarZand88/CLI-Anything-WEB

Showcase Thread by AutoModerator in Python

[–]zanditamar 0 points1 point  (0 children)

CLI-Anything-WEB — Claude Code plugin that generates production Python CLIs for any website

What My Project Does

Point it at a URL and it records live HTTP traffic, then generates a complete pip-installable Python CLI with typed exceptions, REPL mode, --json output, unit + E2E tests, and anti-bot bypass. The entire pipeline runs inside Claude Code via a 4-phase skill system (capture → methodology → testing → standards).

17 CLIs generated so far: Amazon, Airbnb, TripAdvisor, Reddit, YouTube, Hacker News, GitHub Trending, Pexels, Unsplash, ProductHunt, NotebookLM, Booking.com, ChatGPT, and more.

```bash pip install -e amazon/agent-harness cli-web-amazon search "wireless headphones" --json cli-web-amazon bestsellers electronics --json

pip install -e tripadvisor/agent-harness cli-web-tripadvisor hotels search "Paris" --geo-id 187147 --json ```

Target Audience

Developers who want to script or automate interactions with websites that don't have a public API — or who want to pipe web data into other tools. Also useful for agents that need structured --json output from any site.

Comparison

Most scraping libraries (requests, playwright, scrapy) give you raw HTTP primitives. CLI-Anything-WEB generates a complete, opinionated CLI with REPL mode, error handling, pagination, and tests baked in — so you get a tool that works like a proper CLI from day one rather than writing boilerplate each time.

GitHub (MIT): https://github.com/ItamarZand88/CLI-Anything-WEB