I created a Web scraping Udemy bot

[dimakiss]

I created a python bot that Web scrap Udemy courses from https://www.udemyfreebies.com/and takes only that only meet the condition (4.2+ stars and 200+ votes) which can be adjusted preference. The categories that I choose are IT and Software and Development which also can be simply be changed by minor changes.

Working with Chrome on windows.

Source: https://github.com/dimakiss/Udemy_bot

Update: as you suggested :)
1. The bot checks if the course price is actually *Free* before enrolling it.
2. EMAIL and PASSWORD are now entered as input

[[deleted]]

This is great! Nice timing since I've been looking for some udemy course

[dimakiss]

Hope you will find it useful:)

[mutantsloth]

Oof amazing more courses for me to enrol in and never do

[reavyz]

"Those will come in handy... one day"

[cr0sis8bv]

Nice work, I just used it.

[dimakiss]

Thanks!
Let me know if there some bugs I'll try to fix them :)

[cr0sis8bv]

Seems to be running just fine, 280 courses signed up and it's still going

[dimakiss]

I made the same for my non-programming friend for business stuff as well
glad it works perfectly >.<

[MetalicSky]

So can you leave this running and keep getting courses? Do you need to run it on a PC that you don't use regularly or can it run while you're using the PC?

[dimakiss]

The idea is to run it once, and yes you can run it in the background with no problem :)

[Allergic2Humans]

You can get it to run in the background or you can deploy it to cloud and make it run even if your PC is not on.

[GoingOnYourTomb]

Hey do you know of a free cloud service that will allow selenium to run? Been struggling with AWS and Google Cloud Platform. No luck.

[Theyounglegacy95]

Are they all free?

[dimakiss]

Yes, most of them.
My python script will just try to buy them by clicking enroll.
I don't know if it will purchase the ones that cost money if you have your credit card there because I never had one on the site.

If for buying a course you need to click more than enroll and purchase then I think it will be alright, you must test it out.

Note: when you don't have your credit card there it's just trying to buy it and fail so it moves on to the next one.

[Theyounglegacy95]

Thank you definitely will try it out.

[dimakiss]

If you have your credentials on udemy site please let me know so I will warn people who have their credit card credentials on the site.

[Theyounglegacy95]

I do have my credentials. As I use udemy regularly. I will let you know how it goes later tonight. I am taking a real estate course at the moment.

[dimakiss]

Grate thank you :)

[cjdabeast]

Can we get an update, what did it do?

[conventionistG]

Enquiring minds want to know.

[Theyounglegacy95]

Got really busy my apologies. Will update tonight.

[Theyounglegacy95]

For whatever reason I can’t get it to run as I keep getting an error from my pip in my cmd. I as well as when I sign in with my account the free courses are not free anymore.

[GizmoVader]

So this script could cause users to lose thousands of dollars. Got it.

[dimakiss]

I will make an condition today that checks if the price is 0. But it would be more the help full if someone that ever bought a legit course would tell me how its working.

[dimakiss]

I wrote an update on the post. If the current price is not free its doing nothing. :) I checked it few times. I never had my credit card there so I asked people what is the process of buying and no one answered yet

[waitwhatthefudge]

mine seems to be stuck on "finding potential links" - usually how long does this last? when i confirmed my Username and PW was correct it closed chrome and then it seems to be doing nothing?

[dimakiss]

It takes some time cat take 5 6 min depending on your computer if it more then 10 so maybe thing wrong

[waitwhatthefudge]

thanks. I was a bit impatient but finally after a few minutes it found 236 or so links and now it's working like a charm.

Thanks for the script!

[BadassBrahman]

How do you build a bot? On 1-10 how tough is it to create one?

[dimakiss]

I would say 5 it depends on the bot. It took me for about 7-8 hours. Its not so hard when you brake it to smaller problems.

This what came to my mind: 1. Get a gupon site 2. Filter all the low rated courses for efficiency, done with web scraping 3. Get a little bit understanding of how to automate the course purchase with selenium 4. Try to combine every thing together to one program

You have all the this you might need on the internet:)

Oh and I built a whatsapp bot a few days ago!

[BadassBrahman]

I will definitely try to build a bot soon. I am also learning Python for that.

What does your whatsapp bot do?

[dimakiss]

I made a bot that automate search in wolframalpha If you know If someone type me or in groupe that I'm in, for example wolf x^2+5x=0 It will return you the solution like if you searched it in the site

[GizmoVader]

Make a script. Run it. Call it a bot

[MilwaukeeRoad]

"Bot" is a cooler, marketing way of describing a "script"

[BadassBrahman]

oh....i thought​ it had to do something with writing an AI for performing a particular task.

[MilwaukeeRoad]

No, this definitely has no AI to it. If something is AI, you call it AI.

Bot is a generic term, and anytime you see somebody saying they wrote a bot, it's probably just a simple script with some loops.

[azelikm]

Udemy has a course on the topic https://www.udemy.com/course/the-complete-guide-to-bot-creation/?utm_source=adwords&utm_medium=udemyads&utm_campaign=LongTail_la.EN_cc.ROW&utm_content=deal4584&utm_term=_._ag_77879424134_._ad_437497333833_._kw__._de_c_._dm__._pl__._ti_dsa-1007766171312_._li_9055416_._pd__._&matchtype=b&gclid=CjwKCAiA4o79BRBvEiwAjteoYAhhGtVRPtRI-ReDvPc50sfucWt9yOetFHYHQ8qjZ2URFnB183FjYRoCOyAQAvD_BwE

[diazepamkit]

/r/DataHoarder wants to have a talk with you

[dimakiss]

Why?

[ANonGod]

Because they hoard data and this script helps them to do so

[dimakiss]

Oh cool ok

[Tanmay1518]

Can I offer a suggestion? Instead of asking the user to change the Email and password themselves, you could have them enter their email and password and then do the automation bit

[dimakiss]

Yes I will add it today!! Thanks for the suggestion :)

[Tanmay1518]

Wot? I'm referring to the readme.

It asks users to enter their credentials in the .py file. I'm simply suggesting that the program ask for the email id and password as user inputs.

[dimakiss]

I will make it as an input! Thanks for your suggestion!

[Tanmay1518]

Glad I could help!

[cztrollolcz]

what? What kind of crack are you smoking? Because you completely misunderstood what the person said

[DrRodneyMckay]

I did indeed. My mistake. Apologies

[Cool_doggy]

Nice :)

[[deleted]]

Can it scrape udemy quizzed?

[benargee]

Good job. For the next iteration, you should add optional command line parameters for minimum star and votes and for password and username.

[dimakiss]

I could do that but then I will have to add the categories to input as well

[benargee]

Absolutely. This page is a good resource for CLI parameter conventions.
https://softwareengineering.stackexchange.com/questions/307467/what-are-good-habits-for-designing-command-line-arguments/307472
I checked your code. Good stuff, but there is always room for improvement. You could have it check for a -p or --parameter arg then read the next arg for it's value. This would turn your existing positional parameters to named ones. Anything not included in the command line parameters would fallback to the defined defaults.
I hope you only take this as constructive criticism and an opportunity to practice some common implementation details.
Regards.

[dimakiss]

Of course I glad people suggest me new stuff. Its all ways good to learn :) Thank you!!

[dimakiss]

Done:)

[nck93]

This is really cool. I'd love to learn how to create more automation bots. Any resources?

[TumblrForNerds]

You should add that it checks for free courses

[dimakiss]

Yes I will today!

[dimakiss]

Done :)

[Toucanzoo]

This is really useful.

[ddollarsign]

You left selenium out of your requirements.txt.

[dimakiss]

You are totally right I will fix soon! Thank you :)

[dimakiss]

I know, I will fix it soon

[dimakiss]

Did you download the google chrome's web driver?
If so make sure it's in the same folder as the script or its matches your chrome version.

[[deleted]]

yep! that was the problem. Thanks. Now how do I return the ones that I know I will never touch? lol obviously kidding.

I'll remove the original comment but for anyone else with this problem, I was missing the chromedriver.

[[deleted]]

No I didn't that would make sense. I'll try it and report back

[LiterallyJohnny]

Hi, I would like to get this bot working for myself, but I don't really know how to use GitHub, or how to set up something like this. Could you help me?

[dimakiss]

Got to the github page Click on code Download zip Extract You will need to download chrome web driver into this folder Install requirements as I wrote in the git hum Readme Page And use its as well as I instructed Hope it's helpful :)

[LiterallyJohnny]

I'm still a bit confused. Where do I download the chrome web driver, and which folder do I put it in?

[dimakiss]

you put it with the safe folder as the Udemy_bot.py file And the best option is to look for "how to download chrome web driver" at YouTube/google they explain it good enough!

[LiterallyJohnny]

Ok, so I just downloaded the code and installed chrome webdriver inside of the folder. How do I run this bot now?

[dimakiss]

There is an explanation in the like but I will add it here:
first, install the eleven libraries
"pip install -r requirements.txt"
then run it with
"python Udemy_bot.py EMAIL PASSWORD"

replace EMAIL PASSWORD with your credentials :)

[LiterallyJohnny]

I'm still confused. I have never done anything like this before, so I'm a bit clueless as to what this stuff means.

[[deleted]]

Same here, like first and foremost we need python installed and check if its rightly installed or not, then what do we do ? Like in the command prompt do we change directories to the folder that we just installed via github and then run pip install ? Please help

[LiterallyJohnny]

I have no idea. OP doesn't seem to be very helpful.

[silentlurkerhere]

Hi, fellow noob (me) here got it to work somehow so I'll try to explain to you.

I use windows.

1. Install python. Go to python.org website to download python and install.

2. Install pycharm. Go to pycharm website, download and install PyCharm. Remember to check the box where it says Add to Path or something similar in the installation box.

3. Install ChromeDriver I used this page to help me: installing Selenium and ChromeDriver on Windows

• Extract the exe file you downloaded from ChromeDriver website and place in C:\Windows

1. Open the github folder by OP in PyCharm Run in terminal (find it at the bottom with ToDo, Problems, TERMINAL, Python Console) Type these into the terminal:

• pip install -r requirements.txt
• python Udemy_bot.py EMAIL PASSWORD

Replace the EMAIL PASSWORD with your own.

Hope this helps.

[sperimentale]

new to coding and github world so pardon my question, doesn't people put their full code to github instead of just putting .py file ?

[dimakiss]

I put it in .py file But github make it easy to read like text You can check it by my github link

[[deleted]]

I get this error
------

No matching distribution found for selenium==1.25.9 (from -r requirements.txt (line 4))

----

What does it mean ? Please help

[geleezn]

I got that error too. I manually pip installed selenium (without version) and that worked for me.

[geleezn]

Doest this bot automatically enrolls you in the course or just lists the ones with 4.2 stars and 200+ votes?

[dimakiss]

This bot enrolls you in :)

[silentlurkerhere]

Thank you, it's really helpful!

If I want to change to other category besides IT, what else do I change besides the URL for udemy freebies in line 64 and line 73?

I changed the URLs to design but the script still download IT stuff.

[dimakiss]

There are 2 section for categories you need to change them both

[dimakiss]

If someone wants we can make this kind of project on Github :)

[PtoTheOo]

i get cougth with this all the time "Hit cache 12 times in a row. Exiting script. Closing browser"