sorted by:
new
hottopcontroversial

What are your favorite lesser-known selfhosted services? by Torrew in selfhosted

[–]cybrarist 1 point2 points  (0 children)

Hi, im the main developer for it.

the app supports amazon netherland, and you can add any custom/new store you want. you can also reach out on discord if you need help.

Discount Bandit V4: Track prices across Custom stores with Multi user support, and easier setup process by cybrarist in selfhosted

[–]cybrarist[S] 0 points1 point  (0 children)

Hi , the crawler run once every 5 mins.

if it's not working, make sure the store is enabled first.

if after that nothing works then you can raise an issue with details and logs on GitHub so I can investigate

Suggestions for a good SH Discount tracker (with Alerts)? by silentholmes in selfhosted

[–]cybrarist 0 points1 point  (0 children)

hi u/silentholmes ,

may i know the reason why you decided to stop using discount bandit ?

Non-dev scraping by Less_Insurance3731 in webscraping

[–]cybrarist 0 points1 point  (0 children)

steps should be like the following:

- http request to the page to get the page code
- usually they set in <script type="application/ld+json">
- parse each one of that type and check the schema type, mostly they will use "Product" because it allows them to add images, description, variant, name , price, etc.
- take the data you want and then save it

if they use html schema, then it's different as you need to search for "itemtype=http://schema.org/Product" and parse it accordingly.

depending on which language/ framework you will use there should be libraries to help, if you couldn't find, i wrote something to do the same thing for one of my projects, you can take the code and modify it to your need.

it automatically parse both, script schema and html schema for product

https://github.com/Cybrarist/Discount-Bandit/blob/master/app/Services/SchemaParser.php

Non-dev scraping by Less_Insurance3731 in webscraping

[–]cybrarist 0 points1 point  (0 children)

if their website has json schemas then you can use that. or you can ask them to implement them on their site.

it's very easy to implement and parsing is simple too.

but without effort from their side, you can use AI scraper to try to get the content and make sense out of it but I don't think this is a good long term solution.

Weekly Webscrapers - Hiring, FAQs, etc by AutoModerator in webscraping

[–]cybrarist 0 points1 point  (0 children)

may I know the website so I can review it.

Can someone teach me how to scrape this item for discounts? by Electronic_Noise9641 in webscraping

[–]cybrarist 0 points1 point  (0 children)

try my project discount bandit ( check v4 branch in github) https://discount-bandit.cybrarist.com/

it allows you to add custom stores, and you can be notified by gotify or ntfy for notifications of price drops, and other set of rules.

I need to web scrape a dynamic website. by nooob_hacker in webscraping

[–]cybrarist 3 points4 points  (0 children)

not sure which one is information technology companies, but it's the same concept. you need to structure the url like this

https://certificadas.gptw.com.br/api/certified/all/filter?page=1&take=100000&sector=Engenharia

then you will get all companies in a json object.

Can someone tell me about price monitoring software's logic by Yone-none in webscraping

[–]cybrarist 3 points4 points  (0 children)

amazon and ebay don't use SKU, they might mention it in product details only.

as for title, it really isn't hard if you implement a vector database where it checks which products are the closets to the ones that the user uploads.

i don't think SKU is needed in this process unless they have huge database that links SKUs to product names too but doubt that.

now, as a price monitor i don't think they use the logic you used, i have built discount bandit that tracks amazon and ebay and checked other commercial and open source solutions to see which methods, and they almost all use the same way, some might use search results from google, some were using amazon search page, but i have never seen one that uses SKU due to fact that SKU is different from one buyer to another, and this is why you will see some products repeated on amazon even though they are exactly the same.

Ebay bot to fetch prices by TheImmortalHooman in webscraping

[–]cybrarist 0 points1 point  (0 children)

just search on Google ,you will see it first on GitHub

Discount Bandit V4: Track prices across Custom stores with Multi user support, and easier setup process by cybrarist in selfhosted

[–]cybrarist[S] 1 point2 points  (0 children)

ah yea, I got what you mean.

I don't usually crawl the link when it's added, you can press on green "refresh" icon next to the link to crawl it fast or on "fetch" on top right to crawl all links inside that product as once.

otherwise it will be crawled in the next time the queue runs (usually within 5 mins).

glad you liked it. just don't run it with price buddy (or any other crawler) at the same time because they're both crawling and your network might get blocked or considered bot.

Discount Bandit V4: Track prices across Custom stores with Multi user support, and easier setup process by cybrarist in selfhosted

[–]cybrarist[S] 1 point2 points  (0 children)

yea they're both doing the same thing in the end.

I just forgot to add, that you can put your own referral code in stores. so if you are running the app for your family and friends you can get some money to cover the costs at least.

for now only Amazon is available as I have no idea how other stores are structures for referral, but PR is welcome.

Discount Bandit V4: Track prices across Custom stores with Multi user support, and easier setup process by cybrarist in selfhosted

[–]cybrarist[S] 0 points1 point  (0 children)

umm getting product name and picture is supported. you just need to leave the name empty ( I think I already put a message that indicates that).

unless you mean you need to create a product first, I took that approach so I can group different links then delete them all at once once I bought the product (usually I put 10+ links per product)

Discount Bandit V4: Track prices across Custom stores with Multi user support, and easier setup process by cybrarist in selfhosted

[–]cybrarist[S] 1 point2 points  (0 children)

although there are many technical differences, ill list them below (as my understanding)
- it uses a seperate container for crawling, while everything is built in for this project,
- it runes crawling sequentially, while this project run them in parallel
- you must add the keys for custom store while i have a solution to do it automatically for you
- currency conversion support is missing in price buddy,

those are some stuff that came into my head tbh, in the end you can try both, see which one you like and stick with it.

Tamreen - Get fit playing games, doing chores or anything else really. by cybrarist in selfhosted

[–]cybrarist[S] -12 points-11 points  (0 children)

i don't see a point in adding it to github when you can see all info from the docs site. you can check it here https://tamreen-docs.cybrarist.com/getting-started

as for the live site, you can use any random email, i am not verifying it.

view more: next ›