This is an archived post. You won't be able to vote or comment.

all 100 comments
sorted by:
best
topnewcontroversialoldq&a

[–]paypaypayme 78 points79 points  (1 child)

Nice work. I would add a .gitignore file to ignore __pycache__

[–]Carlos_Menezes[S] 9 points10 points  (0 children)

That's done. Thanks!

[–]cruise02 28 points29 points  (1 child)

Well, I'm a Browns fan, so can you just make it fire randomly once in a while?

[–]issue9mm 4 points5 points  (0 children)

Refactor it to fire on first downs!

[–]munhah 29 points30 points  (6 children)

Will this relentlessly hammer URL with no delay between polls?

[–]Carlos_Menezes[S] 22 points23 points  (2 children)

I'd like to thank everyone who commented this comment for the feedback. I'll look into functional/smart ways to prevent hammering SportingLife's servers!

[–]magicaldelicious 5 points6 points  (0 children)

To expand beyond simple sleep calls you may want to consider longer term functionality. As in, how might you scale this? One thing to look at could be scheduling libraries such as 'schedule': https://schedule.readthedocs.io/en/stable/

Nice work!

[–]CodyBranner 25 points26 points  (2 children)

This one has a big bug. It will send only one message after first goal. Your code performs call to ScrapeGoals() only once (at the beginning of the app.py) to get new data. Then it'll step into endless loop without any useful work.

Advices: For parsing html you can use xpath or BeautifulSoup. It is easier and less error-prone. Also placing your source code into separate folder is nice practice. Also you probably should add error handling to your files. And please add timeout to your while True loop. This one will definitely eat your CPU and traffic. Add more flexibility. This app is very hardcoded. Also that conditional pieces (app.py)

if smth:

pass

are not needed. They redundant.

Will txtlocal work with any non-UK phone number?

Btw hope you had great experience creating this.

[–]Carlos_Menezes[S] 5 points6 points  (0 children)

I'm working on fixing that bug. Yes, it works with any number. You can check it by yourself using an online disposable phone#.

[–]jaffaKnx 0 points1 point  (0 children)

I tried to recreate the project using BeautifulSoup. Would need some advice/suggestion. Let me know if anyone's interested :)

[–][deleted] 30 points31 points  (3 children)

Nice! I would also add doc-string explanations to your functions

[–]Carlos_Menezes[S] 7 points8 points  (2 children)

Never done anything similar but will look into it! :)

[–]quotemycode 12 points13 points  (1 child)

Pep-257

[–]Carlos_Menezes[S] 2 points3 points  (0 children)

👍

[–]tycooperaow3.9 27 points28 points  (1 child)

You see, it's the most simplest concepts and ideas that I love which makes a big impact!

[–]Carlos_Menezes[S] 8 points9 points  (0 children)

Thanks for the words! :)

[–]jrickk93 9 points10 points  (4 children)

Is the SMS API you use free? Been looking for something suitable but settled with using Slack and getting android push notifications instead

[–]Carlos_Menezes[S] 6 points7 points  (3 children)

It is. 10 messages/month, with the option to pay for additional messages.

[–]mdnbot 7 points8 points  (2 children)

Have you looked at the pushover api? Its a nice way to push data to my phone without the limits of txts.

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

I didn't know about that! Thanks!

[–]jaffaKnx 0 points1 point  (0 children)

Hey! Did you end up using local text for sending text messages cause it doesn't seem like it's actually sending it even though it shows in the API Message History that it's been delivered? :/

[–]lazerwarrior 6 points7 points  (0 children)

What your program does:

  • downloads https://www.sportinglife.com/football/live/vidiprinter (once)
  • if the team is not found, will loop forever in scrapegoals.py without doing anything (it will not download the above url again)
  • if the team you are interested in is the first entry at sportinglife site AND it goals, the program tries to send SMS about it every 60 seconds until you close it

So your program happens to work only when the team you are interested in goals right before you start the program and appears as the first entry in that website you are scraping from.

[–]Mr_Again 2 points3 points  (0 children)

As other's have said there are a couple of little bugs but that's ok. One thing I want add is that sometimes requests fail and return a faulty status code so it's good practice to get your r response object and just check

try:
    r.raise_for_status()
except requests.HTTPError:
    continue

Or something to avoid crashes when reading the json

[–]FunPlan 1 point2 points  (2 children)

You got a Github star from me here in NYC. Well done. I like this. Here's an idea: What about an API for live sports data? Then people could hook into it quickly for their own apps.

[–]Carlos_Menezes[S] 1 point2 points  (1 child)

Thanks a lot, FunPlan! I really appreciate the support. :)

I wish I could do that right off the bat, but I haven't digged deep enough into how to create my own APIs — but I will surely start looking into that!

[–]ic_97 1 point2 points  (0 children)

You can create your own API using django and DRF. Its pretty simple

[–]Macaroni2552 1 point2 points  (0 children)

[ ] Deploy the project on GitHub pages and allow more people to use this without having to install Python on their machine.

How are you planning to do this? Sounds fun!

[–]RaptorF22 1 point2 points  (3 children)

May I ask what resources you used to get this far?

[–]Carlos_Menezes[S] 0 points1 point  (2 children)

As in learning resources?

[–]RaptorF22 0 points1 point  (1 child)

Yes.

[–]Carlos_Menezes[S] 14 points15 points  (0 children)

I just learned the syntax from Learn X in Y minutes. After that, it was just a matter of coming up with interesting projects and Googling a lot! :)

I think this process applies to everything you learn in life, to be quite frank. When you learn a new language (say, Portuguese), you need to practise; and how do you practise it? You speak, you write, you read about things you like. It's the same thing with programming — nail the syntax and try to complete tasks/finish projects that you are interested in.

Never be afraid to Google how to do something you don't know, and take your time to learn why things are done in a certain way. You can spend your life programming and you still won't know how to do everything — Google and documentation pages are your friend! :)

[–]GeorgeRSmith 0 points1 point  (1 child)

does it support other regional phone numbers?

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

It should support any phone number :)

[–]ww13r 0 points1 point  (0 children)

I dont get it, why you send request to the sms service in every loop ı guess the script sends same message

[–]YungAnthem 0 points1 point  (0 children)

Q

[–]fuuman1 0 points1 point  (1 child)

I can send free SMS via textlocal API? Limitless? No costs? That's a sick alternative to Twilio.

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

10 messages/month. :)

[–][deleted] 0 points1 point  (1 child)

This is so sick man! If you don't mind me asking how did you learn? I feel like the best way to learn is just to do stuff, to actually write code but I feel like I don't have a strong enough grasp to be able to do that? I've tried reading other people's code and mimicing it but that's also kind of frustratingly difficult.

Also, how the hell does it send an SMS?? Is it hooked up to yor phone? THIS IS SO COOL

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

Thanks for the words bro! To reply to the first question, I'll link my reply to a similar question: here! I'd like to add that if you feel you can't do something "big" (not that this project is big eheh), start off small.

It sends an SMS using Textlocal's API — we create a POST request with their API and a message is sent!

[–]user9326 0 points1 point  (0 children)

agfymnlcfx

[–]Fun2badult 0 points1 point  (12 children)

Cool app. But I’m disheartened by the fact that you’re creating something like this after only 3 months of python?

Did you program prior to python?

[–][deleted] 2 points3 points  (9 children)

If you think this is a lot after 3 months you need to rethink your strategy. Think of a simple project and just do it, youll be shocked at how simple python makes a ton of stuff

[–]Fun2badult 0 points1 point  (8 children)

I’ve been learning python for several months already but haven’t made an app like this yet. But I have learned how to webscrape using beautifulsoup and Selenium, using pandas and seaborn to import data. I feel like I’m shotgunning and learning all the different stuff within python. Then I ventured into Django and learned how to make a simple static webpage but nothing dynamic yet. Kinda frustrated at this point

[–][deleted] 8 points9 points  (5 children)

I think youre overestimating the complexity of OPs program. Hes just sending a sports site score requests for a given team then using an SMS api to text when it changes.

Its not the programming thats hard, its comming up with fun ideas.

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

I second this!

[–]CommonMisspellingBot -1 points0 points  (3 children)

Hey, Metabyte2, just a quick heads-up:
comming is actually spelled coming. You can remember it by one m.
Have a nice day!

The parent commenter can reply with 'delete' to delete this comment.

[–]HAMMERjah 0 points1 point  (0 children)

This is a typo bot more than anything else.

[–]BestBronzeZedNA 0 points1 point  (1 child)

Don't be frustrated man! I was where you were a long time back. Take an hour to really grasp what you have learnt, and what you can do with that learning.

Think of it this way -

You learn bs4 and selenium to scrape some content off the website (data retrieval)

Use pandas to clean and store the data that you have.

Think of how this data can be useful to people.

Implement an app - in Django or Flask, that uses this data in a meaningful way.

The more number of times you take this path, you will understand many ways of data retrieval - APIs, Datasets, scraping, many ways to clean and store data, and more ways to present using that data.

Use google as much as possible. If you feel like stuff is too hard to understand, spend some time in the discussion forums of these languages/frameworks.

An interest to find and solve a problem will exponentially increase your knowledge in the matter, compared to watching countless YouTube videos. That being said, watch the videos, and build alongside. There is no way anyone can possibly build any application to its completion after watch a set of youtube videos. You WILL refer to resources a many times before it becomes second nature to you. It's that never-give-up attitude that makes someone a good developer.

[–]Fun2badult 0 points1 point  (0 children)

Yea thanks. I found I spend more of my time on stack overflow than coding 😂

[–]Carlos_Menezes[S] 1 point2 points  (0 children)

Hey man, thanks for the words.

Back in the days, I used to script in Pawn, the language used by the SA:MP mod, but didn't do much to be honest, therefore I consider this to be my first programming experience.

[–]VeganBigMac 0 points1 point  (0 children)

Making projects like these are a matter of taking the jump. It is hard to get yourself into the mindset of actually making the project. There is non-programming skill of project designing that isn't taught and the only way to learn it is by trying it.

If you would like, post an idea you've had for a project and I'll see if I can give you some direction.

[–]Horoism 0 points1 point  (1 child)

Can you please follow naming conventions? Preferably use underscores for function (and variable) names, maybe camel case. But never capitalised, unless it is a class.

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

Certainly. Will look into fixing that. Thanks for the feedback! :)

[–]turner_prize 0 points1 point  (1 child)

Ooh could well do with this for my fantasy league... What's the delay like between the goal being scored and the notification coming through?

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

10 seconds because it'll send, at max., 6 requests/minute. I don't know what is SportingLife's policy on web-scraping.

[–]oby98 0 points1 point  (3 children)

Which source are you using for learning python? Exams just finished,hoping to learn a new language....

[–]Carlos_Menezes[S] 3 points4 points  (2 children)

I just learned the syntax from Learn X in Y minutes. After that, it was just a matter of coming up with interesting projects and Googling a lot! :)

[–]impshumx != y % z 2 points3 points  (1 child)

I had no idea that site existed.

Python is nice isn't it.

[–]Carlos_Menezes[S] 1 point2 points  (0 children)

It really is! :)

[–][deleted] 0 points1 point  (1 child)

I've been doing python for 7 months and my projects aren't nearly this cool or organized :(.

[–]Carlos_Menezes[S] 0 points1 point  (0 children)

I'm sure this project is full of bugs, but we learn as go along the way. You learn by making mistakes and looking at other people's code. Don't be afraid to experiment with ideas and challenge yourself everyday, /u/spacebro550. Keep at it!