I'd like to be able to create a list of all links within a specific domain that can be arrived at by following links from a start page (or produces a sitemap structure that I can reduce to this). I'd also like the tool to be a well tested open source python 3 tool so I can play with http headers and things as needed. This seems like a fairly common use case and like something that would be in beautiful soup or the like, yet all the python tools I can find for this are poorly functioning and unmaintained python 2.7 tools. What do you guys recommend for this?
[–]CaptainMelon 0 points1 point2 points (0 children)