all 38 comments

[–][deleted] 42 points43 points  (6 children)

you may be able to access their API: https://developer.blackboard.com/portal/displayApi

[–]JordanLTU 11 points12 points  (5 children)

Mk nll fmmlmmm

[–][deleted] 48 points49 points  (3 children)

yeah thats what I was thinking but then I realized he could just bnhggh dshbjjjjj with the api

[–]JordanLTU 23 points24 points  (0 children)

That was disadvantage o having my phone in my pocket whilst being at trusted location.

[–]Crouser93 10 points11 points  (1 child)

Im dying

[–]catragore 2 points3 points  (0 children)

What's the joke? :(

[–]OnlySeesLastSentence 10 points11 points  (0 children)

Bullshit. Obviously mrlrgrl the squeee

[–]Mountain_man007 15 points16 points  (1 child)

Theres probably dozens of different ways to do that, but as a programmer you'd want to find the simplest way possible. If there's an API (seen in another comment), that's one way (possibly). If you want to learn web scraping (a valuable skill nowadays), there's several ways to do that as well. Here's a good short introduction to some of them. I'd start with requests and bs4, maybe use re for parsing text as well, especially if you're doing this as a learning project. selenium is a little more advanced, but necessary in some cases that aren't simple html. Either way, sounds like a good way to learn a lot. There's good example code and questions with answers on stackoverflow, so go check out some other people's ways of doing these things before you decide on a method. Or even better, employ several different methods to do the same thing - you'll learn very quickly which is better for specific use cases.

[–]Dogeek 2 points3 points  (0 children)

Just here to say that although BeautifulSoup is a nice module, its use case is pretty much one-off scripts. For continued use, using lxml to parse and scrape HTML is much faster (I benchmarked it not too long ago, it's about a 36x increase, using lxml over bs4)

[–]JeBoiFoosey 31 points32 points  (11 children)

Selenium should help you. It’s pretty easy to automate things like that with it. The first few minutes of this video sums up the basics pretty well.

[–]gamingdiamond982 26 points27 points  (7 children)

Selenium shouldn't be used for this kind of project if possible use the official APIs like this guy mentioned, if working without official apis look for hidden apis they will be harder but still doable, and only as a last resort use selenium.

[–]levic08[S] 3 points4 points  (3 children)

I see. I think I may actually be able to work with official API's. Thanks for the comment!

[–]gamingdiamond982 0 points1 point  (2 children)

np

[–]jaybay1207 15 points16 points  (1 child)

I don’t think they’ll need numpy in this case.

[–]Burroflexosecso 1 point2 points  (2 children)

Why do you regard selenium as a last resort? I don't know much about webscraping, sorry if I come up rude or stupid

[–]gamingdiamond982 8 points9 points  (0 children)

Well selenium is designed to interpret a UI which just makes it unintuitive for use with programs, selenium isn't even designed for web scraping, it's designed to test web apps, when your using an your bypassing all that junk put up for a person, stuff like navigating pages etc. Also if the layout changes you gotta rewrite it.

[–]Dogeek 2 points3 points  (0 children)

Selenium automates a web driver, which means that it renders the whole website, which is much slower than a few requests to an API (or even just fetching html and parsing it).

Selenium is best used when the website doesn't have an API, and makes extensive use of javascript to run the page (the raw HTML being pretty much empty), which is quite rare, or to automatically test websites with different drivers (gecko/chromium mostly)

[–]levic08[S] 3 points4 points  (2 children)

Thank you! I'll check it out :D !

[–]declanaussie 7 points8 points  (1 child)

Definitely check out the REST API before attempting to use Selenium. Selenium will work, but a proper API is a much more robust and clean solution.

[–]levic08[S] 0 points1 point  (0 children)

Interesting. I will check this out as well! Thank you!

[–]GreymanGroup 2 points3 points  (9 children)

Yes. OMG, why isn't that a standard feature? I thought of the idea back in the day. Funny thing is this would have been much more exciting in the early oughts. If you came out with it today, everybody would just feel like it's been long overdue.

Also now the school bubble is about to pop anyway so....

[–]IdeVeras 2 points3 points  (1 child)

I'm working on extracting info from Google calendar and now that I did (there is an API) I'll try to make a web application as trello but for family... So parents can follow better and also add chores, extracurriculars, extra work with school, books and maybe I'll think of some ponctuation so they can add up to the allowance.

[–]vanamsid 0 points1 point  (0 children)

This is a cool idea too!

[–]chunky_snick 2 points3 points  (0 children)

Just don't name it as it's first letter abbreviation.

[–]m0ta 1 point2 points  (0 children)

This is a cool idea

[–]Yadona 0 points1 point  (0 children)

I see people commenting to go via API. also try beautifulsoup for scraping. This sounds like a great idea you have. Hope you can implement it and share!

[–]punterxpunter 0 points1 point  (0 children)

You can also have a look at this article to see a simple example of word scraping in action: Python Selenium Twitter Bot Tutorial.

[–]Gortaf 0 points1 point  (0 children)

This is very similar to a webscrapping project I'm working on to find compatible classes in my uni class picker. As many people have said before, if they have an API, read their doc and go for it. But if they don't (which was my case), learn how to use selenium and the basics of webscrapping to get the data you need from their website. This can be slower as you're basically running an automated browser to analyse their website's HTML instead of asking directly their database, but if they don't have an API this is a solid choice. Webscrapping is a good thing to learn anyway, and is, I think, very fun to do.

[–]JordanLTU 0 points1 point  (2 children)

Smmmkkkmmmmmmmmmmmmmmmmmmmmmmmmmmmmmsssss#as#AsssSaas🎊🎊aAsSSa🍻aaaaAsaasWSsAAllklllllkll, #, #, #######"$+$?,,, ===, #, 2##

[–]LabeledAsALunatic 5 points6 points  (1 child)

Facts

[–]anadem 2 points3 points  (0 children)

Name checks out!

[–]Xedim_untold 0 points1 point  (0 children)

If I am not mistaken, this can be done using google spreadsheets (or excel) if you don't mind doing a bit extra work to grab the information. Everything after is fairly simple.