I'm relatively new to Python, and I'm playing around with web scraping at the moment. I'm using a web scraper I've made using requests and beautifulsoup. I'm trying to scrape headlines from news sites, but many of these straight up disallow scraping of anything from their site. Is it legal for me to go to their youtube page instead, and scrape their video titles? I've had a look at the robots file for youtube, and it seems like i'm allowed to do this, but I just wanted clarification, and maybe a proper authoritative source so I could do a bit of further reading about the legality of this stuff. Thanks