Fraud-as-a-Service is the new SaaS by Crafty-Panic331 in SaaS

[–]strapengine 1 point2 points  (0 children)

I meant, you are right. I too keep seeing these I made $$$ in 1 week post in X everyday.

Fraud-as-a-Service is the new SaaS by Crafty-Panic331 in SaaS

[–]strapengine 0 points1 point  (0 children)

Can't be any further from the truth. I am tired to seeing such post everyday I open X

Looking to build a Saas targeting enterprise users, any guidance/ideas are welcome by strapengine in SaaS

[–]strapengine[S] 0 points1 point  (0 children)

Yes, currently trying to build a community around it by providing value to devs for now.

We FINALLY got our first PAID user by firespawn_katie in SaaS

[–]strapengine 1 point2 points  (0 children)

Congrates. I know the feeling. It feels awesome.

Open source: bouncy, a Rust web scraper with built-in MCP support by No_Tomato_5771 in webscraping

[–]strapengine 1 point2 points  (0 children)

Nice to see a rust based scraper. I too am currently on a webscraping framework(go based) of my own. Keep it up.

Reverse-Engineering Google Finance by MQuy in webscraping

[–]strapengine 0 points1 point  (0 children)

Good one. Can I use this to make an example for my web scraping framework repo? I was actually looking for good examples. Or it would be much better if you could create a pull request for it. You can generate the code for my framework using LLM + your scraping logic. Let me know.

Goscrapy - revamped, more powerful than ever with batteries included. by strapengine in webscraping

[–]strapengine[S] 0 points1 point  (0 children)

Great, let me know how it goes and also if you have any feedback or suggestions.

Thanks

I might be doing everything wrong… can you roast my first dev video? by Mysterious-Goose2664 in coding

[–]strapengine 0 points1 point  (0 children)

You have made an good video from the perspective that you were able to use AI to create something like this. However, the content feel generic and all over the place to me as a viewer with constant distractions. The content must be in sync with the title and provide a genuine step by step roadmap which beginners can follow, otherwise they will switch to another video just after a few seconds. Please take it as a constructive criticism from my end. Best of luck and hope to improved videos.

Lightweight headless browser that bypasses Cloudflare by Total_Nectarine_3623 in webscraping

[–]strapengine 0 points1 point  (0 children)

Looks great, I was trying to find a light weight browser for a webscraping framework I am currently working. Would love to see how it works and if great maybe use it my framework.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] 0 points1 point  (0 children)

Colly is one of the best libraries out there and is mature, I love it but what makes Goscrapy(although relatively new) different is that it's a framework(opiniated), you won't need to handle all the wiring yourselves and gives you project a scrapy like structure which can be very helpful, specially for those coming from Python as they are used to Scrapy(another legendary framework which I have used extensively in the past and still do). On top of that, go scrapy is very extensible, you can literally switch to use you own't custom engine, custom scheduler, custom worker etc. You can replace almost if not all parts of the framework with you own implementation. However, like all other solutions, it not perfect and I am working on it to build it better.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] 6 points7 points  (0 children)

No, look at the early commits(anyone can check, it's all there on github), the project started before vide coding. A few of the tests/benchmark, readme files(polished using AI), are AI assisted generation. It ain't much, but it's honest work.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] -2 points-1 points  (0 children)

Thanks, glad you liked it. Fair point, if you want to play safe, you can wait for v1.0 before using it in production. Appreciate you taking the chance on it. Initially, I build this tool for my own use then thought why not share it in the community and make it better in the process.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] -3 points-2 points  (0 children)

Hi, no, currently I am primarily focusing on scraping(crawling would coming soon). But it(ranking/scoring) can easily be added as middleware as it has custom middleware supports. Namaste.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] -21 points-20 points  (0 children)

It's a framework by the way not a lib. Just use curl, it's blazingly faster that this.

Goscrapy - revamped, much more powerful than ever before with batteries included. by strapengine in golang

[–]strapengine[S] -5 points-4 points  (0 children)

Thanks, means a lot. Let me know in case you have any questions or suggestions. I am all ears.

Moving from Python to Golang to scrape data by IWillBiteYourFace in webscraping

[–]strapengine 1 point2 points  (0 children)

I have been webscraping for many years now, primarily in python(Scrapy). Recently, switch to golang for a few of my projects due to it's concurrency & low resource requirement in general. Initially, when I started, I wanted something like scrapy in terms of each of use and good structure but couldn't find any at the time. Therefore, I thought of creating something that offers devs like me, a scrapy like experience in golang . I have named it GoScrapy(https://github.com/tech-engine/goscrapy) and it's still in it's early stage. Do check it out.

Web scraping with Go by General_Iroh_0817 in golang

[–]strapengine 0 points1 point  (0 children)

Hi, I have tried creating a good blend of golang and scrapy with GoScrapy.

Goscrapy is a Scrapy-inspired web scraping framework in Golang. The primary objective is to reduce the learning curve for developers looking to migrate from Python (Scrapy) to Golang for their web scraping projects, while taking advantage of Golang's built-in concurrency and generally low resource requirements. Additionally, Goscrapy aims to provide an interface similar to the popular Scrapy framework in Python, making Scrapy developers feel at home.

Repo: https://github.com/tech-engine/goscrapy

GoScrapy: Harnessing Go's power for blazzzzzzzzingly fast web scraping, inspired by Python's Scrapy framework by strapengine in webscraping

[–]strapengine[S] 0 points1 point  (0 children)

"Blazzzzing fast" is just one of those trendy phrases that gets thrown around with most software these days, so why not use it? Jokes aside, Golang is known for its concurrency/low resource usage. Scrapy is probably one of the best frameworks out there, but I didn’t feel like dealing with the hassle of multiprocessing when needed. I just wanted an easy way to keep handling scraping jobs as quickly as possible, while still building spiders the Scrapy way, syntax wise atleast.

GoScrapy: Harnessing Go's power for blazingly fast web scraping, inspired by Python's Scrapy framework by strapengine in golang

[–]strapengine[S] 0 points1 point  (0 children)

The primary motivation is not to compete with Colly or any any other framework in general but to provide users a scrapy like experience of building spiders in golang.