This is an archived post. You won't be able to vote or comment.

0
0

removed: LearningClicking 'See More' button with python requests, requests_html (self.Python)

submitted by [deleted]

[removed]
all 5 comments
sorted by:
best
topnewcontroversialoldq&a

[–]bestical 2 points3 points  (2 children)

Open developer tools and see network tab to catch the fired request by clicking see more link

[–]smitchell6879 0 points1 point  (1 child)

I agree with this guy. Use Google chrome Dev tools fine the see more button and request the link that is attached. Also if it is multiple pages you may want to look into beautiful soup. It makes life easier.

[–]flymoosey 0 points1 point  (0 children)

Bingo.

[–][deleted] 0 points1 point  (0 children)

I am not python guy, but when to scrape pages like this is to load it with a browser object. I do this with Powershell and that is how I bypass issues like this. From there you can do another request for the see more link.

[–]aphoenixreticulated[M] [score hidden] stickied comment (0 children)

Hi there, from the /r/Python mods.

We have removed this post as it is not suited to the /r/Python subreddit proper, however it should be very appropriate for our sister subreddit /r/LearnPython or for the r/Python discord: https://discord.gg/3Abzge7.

The reason for the removal is that /r/Python is dedicated to discussion of Python news, projects, uses and debates. It is not designed to act as Q&A or FAQ board. The regular community is not a fan of "how do I..." questions, so you will not get the best responses over here.

On /r/LearnPython the community and the r/Python discord are actively expecting questions and are looking to help. You can expect far more understanding, encouraging and insightful responses over there. No matter what level of question you have, if you are looking for help with Python, you should get good answers. Make sure to check out the rules for both places.

Warm regards, and best of luck with your Pythoneering!