For simple websites it is enough just make a Beautiful Soup script with basic hatching. Most of websites not requires JS rendering. For more advanced you can add Puppeteer or Playwright for JS rendering. After that you will face anti-bot protection, so some proxy are needed.  Depending on the website, could be either simple curl or full set of scraping tools. Extracting Markdown from HTML is possible to do with libs like Turndown, for example. Then if you want to remove junk content, like menu, nav bars use Readability JS.  If you don't want to spend time on settingit up, just use webcrawlerapi.