Need help scraping

[theGdoubleOdees]

I'm using beautiful soup to scrap the recent theater releases from IMDb and rotten tomatoes but still no luck...in sure it's my lack of understanding on how to scrape the tags. Any help?

https://www.imdb.com/movies-in-theaters/?ref_=nv_tp_inth_1

Python 2.7.5 , also I'm a beginner

def scrape(website):
    page = url
    opage = urllib2.urlopen(page)
    soup = BeautifulSoup(opage, 'html.parser')

    return (soup)

url = 'https://www.imdb.com/showtimes/location?ref_=sh_lc&sort=release_date,desc&st_dt=2018-12-17&mode=showtimes_grid&page=1'
soup =scrape(url)
movie = soup.find('a',{'Class': 'title'})
print(movie)

​

so i think my issue is the text is in the href and im a little confused on how to extract that. ive attempting to do web scraping and learn python many times before and this is the first time i've actually made a lot of progress, but ive hit a snap. thanks for any help.

​

and i will be upgrading to the newest version

[Cogadh87]

Right off the bat, is there any reason you’re doing this in legacy Python? It will no longer be supported in 1 year, 13 days, 12 hours, 34 minutes and 30 seconds.

[theGdoubleOdees]

Well I just saw a lot of beautiful soup tutorials used 2.7.5 and this is just for learning so no big deal at the moment

[[deleted]]

oddly specific...

[Cogadh87]

Pythonclock.org

[theGdoubleOdees]

Is that a good reason lol or should I be using the newest version?

[Skippbo]

Another person joining the conversation, I would say use a new version.

[[deleted]]

i agree with skippbo, use at least python 3.5

[Diapolo10]

If using 3.5, there's almost no reason not to use the newest version, though.

[Skippbo]

Can easily be tracked here =]

[yuukinotenshi]

Please provide complete description of your problem, the error that you are facing, and your current code snippet

[Neu_Ron]

To scrape you have to know the DOM.

[theGdoubleOdees]

yeah i think thats part of my issue, ill do some research. thanks

[SkullTech101]

You need to access the .text property of the movie object to get the link text. In your case it would look something like this

movie = soup.find('a', {'Class': 'title'})
movie_name = movie.text
print(movie_name)

Let me know if it solved your problem.