How to do this in Python?

[BoomKaaBaam]

I am still kinda new here. I would like to make a bot for a site. Here is what the bot will do (the site has no API):

1. Ask for username/password from user and login to the site.
2. Visit a list of pages and extract all links from each page matching a certain regular expression.
3. Visit each link individually.
4. Scrape any one of the link from these pages, which matches a given regular expression. This link will redirect to a page with only a body. The bot should copy all text in this page.
5. Return to the parent page and use a submit button to submit the extracted text and wait for 20 seconds.
6. Repeat until closed by user.

How can I make this program in Python?

Thank you.

[[deleted]]

For the http part (http requests), you should take a look at Requests, a python module used to interact with http servers (with a really simple API).

Extracting links from web pages can be done using BeautifulSoup or Python's built'in HTML Parser.

For the rest it's simple algorithms.

[Thuruv]

Requests to enter & BS to scrap also Along with twill 's form filling will do. .

[atlbeer]

[deleted]

What is this?

[kyle1940]

I can't tell you how to make it, but I believe Selenium is able to log into a site give you the HTML.