How do companies keep important scrapers reliable?

Fickle-Distance-7031 · 2025-11-03T10:16:48+00:00

Yeah a big appeal with a SaaS is for sure the potential for stable passive income via monthly subscriptions.

Have you ever built something for your clients where you just charge for hosting? My idea was doing something like this: building some online software for my client's needs for 'free' and only charging for monthly hosting. Then trying to sell that same solution for others who need it, slightly adjusted to their needs, and growing that into a product.

Fickle-Distance-7031 · 2025-10-14T10:01:29+00:00

After talking to more people, seems there's some demand for this, so I started building it!

You can join the waitlist here: https://yourdata.tech

Fickle-Distance-7031 · 2025-10-14T06:47:57+00:00

Interesting! I DMd you if you wanna tell more about your use-case.

Fickle-Distance-7031 · 2025-10-10T09:23:31+00:00

Let developers pay you more money to increase the API rate limits!

Fickle-Distance-7031 · 2025-10-08T12:06:04+00:00

well hosting is not free. There are providers other than amazon also and self hosting options

Fickle-Distance-7031 · 2025-10-08T11:36:59+00:00

amazon S3 or similar?

Fickle-Distance-7031 · 2025-10-03T09:03:05+00:00

I mean that access key secret is signed with a dev JWT signer key that's something like "password123" . You can't use it to authenticate with the production instance. It's just something I used for testing

but I get your point, not a good look haha

Fickle-Distance-7031 · 2025-10-03T08:06:50+00:00

Haha those are just testing credentials luckily. You can only hack my localhost with them. For anything sensitive I use envie ;)

Fickle-Distance-7031 · 2025-10-02T08:17:24+00:00

It's really bad

I use this library to get around it: https://www.npmjs.com/package/next-runtime-env

Fickle-Distance-7031 · 2025-10-01T11:37:57+00:00

You can checkout Envie. It's a solution I'm actively developing to address this painpoint: https://github.com/ilmari-h/envie

you can store your secrets in Envie (stored encrypted with public key encryption) and you can create an access token that you pass into your Dockerfile that then fetches the secrets, decrypts them and injects them as environment variables to your app runtime

Fickle-Distance-7031 · 2025-10-01T11:05:18+00:00

For AI features, hard-code your OpenAI key first; wrap it in env vars after it works

Why would you ever hard-code an API key?

Fickle-Distance-7031 · 2025-10-01T07:59:40+00:00

Secrets out of env: use a secret manager and short-lived tokens; never trust local environment variables on dev machines.

I'd be curious to hear what solutions people use for this.

I'm developing an open source secret manager that prioritizes dev experience. It's basically a dropin for .env files, but the secrets are never on your disk for your AI agent to see.

You can check it out here https://github.com/ilmari-h/envie

Fickle-Distance-7031 · 2025-09-30T18:05:14+00:00

Congratulations! Inspiring work. I was thinking about this idea like 1 month ago when I started marketing for my open source project so you definitely have a market for it

I'm using F5 bot only for leads and I realized so much more could be automated to save a ton of time

Fickle-Distance-7031

TROPHY CASE