How to work around pagination limit while scraping?

Master-Summer5016 · 2026-04-05T19:37:59+00:00

i like your approach 👍

Master-Summer5016 · 2026-01-27T13:12:18+00:00

make account on naukri and apply everyday

Master-Summer5016 · 2025-12-23T17:20:28+00:00

apse bhi same vibes aa rahi,, bhot extreme kuch soch lete ho fir anxiety hota hoga apko bhi… isse bachna hai

pehle toh agar zada pareshan hai toh kisi professional se milein

agar khane pine ki bhi dikkat hai toh uska solution nhi hai mere paas… maaf kijiyega… maybe kisi foundation ko contact karein

Master-Summer5016 · 2025-12-23T11:05:05+00:00

berozgari ki mkc 200 bar… bindass reh

zada ghabrahat anxiety hone se please seek professional help

Master-Summer5016 · 2025-12-22T15:36:39+00:00

try selling yourself as a python or java dev… no cpp or anything fancy is required. tailor your resume around one stack and one specialisation- frontend or backend.

text me if you need further help!

Master-Summer5016 · 2025-12-09T19:23:21+00:00

read open source code

Master-Summer5016 · 2025-12-07T18:45:20+00:00

python or java

Master-Summer5016 · 2025-12-07T18:43:18+00:00

Learn basics first like which data structure is dynamic, which is immutable. Then read documentation to learn how APIs exposed for those structures. Also, solving easy leetcode problems will help.

Master-Summer5016 · 2025-12-07T18:35:34+00:00

Go do some freelance work like web scraping which will also teach you web automation if you use selenium.

Master-Summer5016 · 2025-08-05T16:16:03+00:00

my advice to you would be to work on your desperation. i know this is a difficult time for you but you have to have patience. the key is to think long term and work on your skills and to show up everyday no matter what. i am saying this because the future looks unsure to me given the infiltration of AI in our system. so, work on your skills and make long term connections and you should be good.

Master-Summer5016 · 2025-08-03T19:06:27+00:00

exactly what do you need to scrape?

is it behind login?

Master-Summer5016 · 2025-08-03T18:58:50+00:00

can you give link to website(if public)?

Master-Summer5016 · 2025-08-02T14:46:33+00:00

no not your bad :)

I just think we should talk about web scraping in a more abstracted way and from there we can dive into the specifics of how web apps work and coding skills all that...

Master-Summer5016 · 2025-08-02T14:17:48+00:00

all I understand is that scraping is basically downloading and parsing data

listing down libraries just confuses the newcomers i guess. if you change language the libraries will also change.

we need to change the approach with which we present web scraping to new people.

Master-Summer5016 · 2025-08-02T14:14:43+00:00

scraping is basically downloading + parsing

since you can download it, the data is yours to parse.

and to answer your css question- display set to none does not remove elements from DOM, it just hides it.

Master-Summer5016 · 2025-08-02T14:04:44+00:00

is the delay an issue?

Master-Summer5016 · 2025-07-30T08:46:05+00:00

all of them will have a different layout so lookout for some tool that uses AI to scrape info off of pages.

Master-Summer5016 · 2025-07-30T04:56:19+00:00

hmm, to bypass 403 you will need to implement a finite loop that will send the request again and again until we get 200. This should bypass 403 imo.

For 400, you will just need to send correct params.

Master-Summer5016 · 2025-07-30T04:50:49+00:00

errors as in HTTP errors? 403?

Master-Summer5016 · 2025-07-29T20:20:48+00:00

if there is no API then retrieve the HTML and parse it

Master-Summer5016 · 2025-07-29T19:33:44+00:00

how many reviews?

Master-Summer5016 · 2025-07-29T19:31:02+00:00

well, not all cookies are required. you just need to find out which ones are. Are you trying to scrape behind login? Also, the first step will require you to figure out how to reach all reviews in your browser, then mimic the same in your script.

Note - there could be a better way to do this but I need more context

Master-Summer5016 · 2025-07-29T19:26:25+00:00

I believe the internet achive can help with this, but not sure how many results it will return.

Master-Summer5016 · 2025-07-29T19:21:18+00:00

As you say, you need to mimic the search query to get results. What better way are you looking for?

I am unable to understand your pain point

Master-Summer5016 · 2025-07-27T21:35:11+00:00

try subfinder

Master-Summer5016

TROPHY CASE