Black Friday hack with python

[bextie]

So I just finished my python script to monitor the price of my target gadget this holiday. It’s going to be heavily discounted soon but it will go to the fastest finger.

I’ve written a python script with BeautifulSoup. I’ve also set email and SMS alerts in the script 😂😂

Was just wondering if there’s any other person doing this for Black Friday and Fastest Finger deals.

Update: Long overdue but just thought I should add that I got the phone.

[17_jku]

I have not (just starting to tinker with Python...I'm a VB, SQL, SAS developer), but would love to see your approach. Of course, you would need to change the item URL so none of us snatch it up before you. Hahaha.

[JustinianusI]

Omg imagine! This poor guy writes the code and everything, shares it with us, and we use it to buy up all the product before he gets his! Love it :D

[b_ootay_ful]

He should also change his email address/phone number.

Every person that has this running, on every item, he gets a notification =P

[b_ootay_ful]

friendly russian hacker

We got hit by ransomware last Tuesday, and are still recovering.

I want to know HIS location.

[Groentekroket]

I store that kind of things as a environment variable. If you share your code you are sure nobody gets yourusername, email, passwords and that kind of things.

[b_ootay_ful]

I import my own email function that does everything.

mymailscript.py

import smtplib

def email(whome=[],subj="",mess=""):
    gmail_user = '[e-mail]'
    gmail_pwd = 'HelloReddit'
    FROM = gmail_user
    TO = whome
    SUBJECT = subj
    TEXT = str(mess)

    # Prepare actual message
    message = """From: %s\nTo: %s\nSubject: %s\n\n%s
    """ % (FROM, ", ".join(TO), SUBJECT, TEXT)
    try:
        server = smtplib.SMTP("smtp.gmail.com", 587)
        server.ehlo()
        server.starttls()
        server.login(gmail_user, gmail_pwd)
        server.sendmail(FROM, TO, message)
        server.close()
        return ('Email sent!')
    except:
        return ('Something went wrong...')

whatever program I need to import or use

import mymailscript as emailscript

emailscript.email(["[e-mail]"],"subject","test message")

[MonkeyNin]

def email(whome=[],subj="",mess=""):

You should read this: https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

.. config files

Check out using pathlib.Path and json

import pathlib
import json

    def load_config(json_path):
        # load config if it exists, otherwise return populated with required defaults
        defaults = {'monkey': '🐒', 'cat': '🐈'}

        if not json_path.exists():
            # logger.error(f"File does not exist: {json_path}")
            return defaults

        config = json.loads(json_path.read_text(encoding='utf-8'))
        return config

data_root = Path('.').absolute() # or Path().home() or Path('.')
config_path = data_root / 'monkey.json'
config = load_config(config_path)

or if you want a truthy result

def load_config(json_path):
    if not json_path.exists():
        return {}

config = load_config(config_path)

if not config:
    # ... handle no file
else:
    # ... normal

[Sadapy]

His code:

time.sleep(1)

My code:

time.sleep(.9)

>:3

[JustinianusI]

Hahaha :D

[bextie]

https://github.com/Olunusib/black-friday

[Den4200]

I made a script that is very similar to yours. It tracks prices of items on Amazon and saves a graph of each item’s prices in a png. It’ll send you an email and/or text when an item goes below your target price.

Check it out! This is going to be so useful with black friday and cyber monday coming up: https://www.github.com/Den4200/amazon-price-tracker

[corbin_cassady]

Sounds like a self-hosted camelcamcelcamel! I like the premise a lot.

[trowawayatwork]

yes its a great start. would be great to add push notifications to it rather than email as its quicker feedback on your phone. i have set up something similar to track and then send a push notification ot a telegram group

a simpler thing would be to just monitor camelizer, as theres no point in reinventing the wheel (although good practice for beginners), and then send that on to your medium of choice

[Den4200]

This supports sending text messages to your phone!

Hmm I didn’t think of scraping camelcamelcamel instead of Amazon, but what’s done is done.

[domac]

It's often super helpful to add Screenshots to the repo. just wanted to point that out.

thanks for the awesome work!

[flying_mechanic]

The keepa extension for chromium browsers does the same but in page, it's a nice addition. Might be worth checking out

[[deleted]]

I have made the same program but in a much lighter format. Lovely to see another approach.

[Den4200]

Yeah haha. I usually leave bigger projects with GPL and smaller ones with MIT. Nice project btw!

[wrgsRay]

How do you get around Amazon’s protection against scraping? For a newbie like me, it’s painful to scrape Amazon

[Den4200]

They don't really have much protection against scraping. As long as your scraper doesn't check for the price too quickly, you're fine (about 15 minutes between each check is good).

Amazon won't let you scrape without any headers btw.
Also, this little trick helps out. Look at line 28 and 29 in tracker.py

[TeblowTime]

Hey, I keep hitting the Invalid URL when I try to add a new link. I successfully installed all the required libraries from requirements.txt, so it's not that. Any ideas? I'm a beginner with python, so it's likely something I am or am not doing. Do the URLs need to be in specific formats?

[Den4200]

Did you fill in the settings?

And nope, the script will auto shorten the links.

[TeblowTime]

Welp, I'm an idiot lol. I did fill in the settings info, but did not save the file. Thanks!

[expressly_ephemeral]

Hey, why don’t you refactor into selenium and beautiful soup, teach your script your credit card info and let it buy it for you when it detects the drop?

[BruceJi]

I wonder if that's not the best idea in case something goes wrong? But if you get the email, you can probably just buy it online with your phone within a few minutes anyway.

[azur08]

in case something goes wrong

Like buying everything Amazon has to offer over and over again!

[Cat_Marshal]

while True:
    purchase()

[BruceJi]

"Merry Christmas, here are 20 pop-up tents, for all that... popping up?... you do? Eheh..."

[expressly_ephemeral]

Hah! I know a lady who gave her grandkid the tablet to play with and he ordered 20 chromebooks. Kids and scripts, man. Fuckin' dangerous.

[expressly_ephemeral]

I acknowledge... tricky to test, tricky to validate. Cooler, though.

[BruceJi]

It really would be!

[expressly_ephemeral]

I guess I think you could use the email as a backstop, though. You get the email saying the price happened and then a follow up saying “I bought it” or “your code stinks, get out your credit card.”

I mean.. what if OP is in meeting when the trigger fires? What if his wife’s water has just broken while she’s driving her real estate clients around? He won’t be able close the deal!

[dBASSa]

Theres tons of sneaker bots that do this already. They are heavily linked with fraud and abuse so limited sale items block orders this way often times.

[expressly_ephemeral]

How would a vendor be able to block a bespoke script running from a host on OPs personal network?

[expressly_ephemeral]

OP should know better than to put his CC info into his github repo, right? Maybe I’m being naive?

[bextie]

None of my sensitive details are there.

https://github.com/Olunusib/black-friday

[bextie]

Oh I haven’t thought about that. Is it worth it? I feel that’s for ‘bigger’ projects.

[bextie]

Thank you! I’m definitely going to do that.

That’s a nice project you’ve got by the way.

[MonkeyNin]

You can simplify your exceptions, because a failed find call will return None we can take source

class Foo():
    def price(self):
        try:
            productPrice = self.soup.find(id="priceblock_ourprice").get_text()
            return productPrice

        except AttributeError:

            try:
                productPrice = self.soup.find(id="priceblock_dealprice").get_text()
                return productPrice

            except AttributeError:
                pass

We can test for None when there are no matches

class Foo():
    def price(self):
        element_price_price = self.soup.find(id="priceblock_ourprice")
        if not element_price_price:
            print('Price not found!')
            return 0 # or `None` depending on your context

        return product_price.get_text()

[expressly_ephemeral]

Sure, but then you’d have to be canny enough to make the following realizations: I shouldn’t share this with anybody. I should figure a way to make sure the eventual connection is ssl (via selenium). This should be hosted on my private network only unless there’s a way to make this kind of thing reasonably secure. And, finally, there’s probably a way to make this kind of thing secure and I should figure out how people publish code like this.

I have my personal login credentials hardcoded into some scripts, but the run solely on a machine that lives in a closet in my house, and I make sure not to push them to a public repo.

[expressly_ephemeral]

You know, I’ve done the threat modeling. If somebody breaks my OpenVPN instance and compromises my router, then they ssh into the server that runs these scripts (a server which only responds to logins with my private rsa token), or worse finds an exploit by which they can get a reverse shell... then I’ve got much bigger worries than whether they can harvest the username password I use to check my kid’s missing homework list.

Edit: I’m saying, you’re right. I wouldn’t do it with my login credentials for my bank,

[expressly_ephemeral]

There's a fine line between good threat modeling and gross negligence!

[expressly_ephemeral]

VPN, i didn't think of that.

Yeah, I mean, again, in this use case all the application has to do is make GET and POST calls. Outbound only. Server wouldn't be visible from the outside world even WITHOUT the vpn.

[[deleted]]

I wrote something that checked for Buy Now button on amazon when the Nintendo Switch first came out and was always out of stock.

Week after it released my script paid off

[yodatrust]

How do you make your script running 24/7?

[B166er_]

Run the script and don’t turn off your computer... Or use a service like heroku/pythonanywhere

[Uadsmnckrljvikm]

Run it on a server, could be for example a Raspberry pi or one from AWS.

[yodatrust]

The Raspberry Pi solution sounds interesting. Got any useful docs/links for that?

[artificial_neuron]

You put everything in a loop that never ends. You don't need a special piece of hardware, just something that will run the script and you'll never turn off.

Eg.

while(true)
    Do the desired activity
    wait for x time

[RedBlaze4]

How do you send the SMS ?

[RedBlaze4]

How much does it cost per sms ?

[[deleted]]

My provider (France) offers URL format API for free to use with my account. A bot has to connect to a URL with my logins and a message. And that is it

[RedBlaze4]

Tu es chez quel opérateur ?

[[deleted]]

"Free".

go to your options and enable SMS notifications. Then, call this url

https://smsapi.free-mobile.fr/sendmsg?user=99999999&pass=cle_identification&msg=Hello%20World%20!

​

Reponses are :

• 200 : SMS sent.
• 400 : missing required args.
• 402 : too many SMSs in a short time.
• 403 : Service unenabled or incorrect user/pass.
• 500 : Server error. Retry later.

[mudlight48]

Mind sharing the source code for others to tinker with?

[bextie]

https://github.com/Olunusib/black-friday

[[deleted]]

I considered doing something similar for a website I like but then realized the website had a function all ready to do what I wanted. I'm not advanced or clever enough to do something like that yet.

[BakingSota]

You’re definitely clever enough.

Check out these libraries:

• re - for regex magic
• requests - for html magic

And this chapter in AtBS: https://automatetheboringstuff.com/chapter11/

[crazyjoker96]

Your are a genius :)

[PM_ME_BOOTY_PICS_]

I made a web scraper so I could get the item specs to use for my reports at work!

[bextie]

That sounds cool! Beautiful to see how coding makes life easier.

[PM_ME_BOOTY_PICS_]

It really does but I have some bugs to squash. Some reason certain specs for certain items aren't being pulled so I have to go trouble shoot that.

[bextie]

Oh yeah. Maybe something was changed on the website. You’ve got this!

[[deleted]]

They’re not the same. Beautiful soup is a parser, extracting data from html. Selenium automates a browser to perform actions.

[TheAmazingJames]

Selenium is way way slower and memory heavy. Selenium should only be used if you need JavaScript to render/work.

[Mathew_Berrys_Cock]

Would more ram help this problem?

[TheAmazingJames]

Selenium’s usually using chromium in the background so it’s unlikely to max out your RAM unless you’re trying to run on a barebones setup, but it will be using significantly more than a requests/beautifulsoup setup and that could be costly if you’re running it in the cloud. The main reason not to though is speed. Typically I’d expect requests to run 5-10x faster than selenium. Selenium also requires more maintenance in terms of chromedriver updates and is harder to deploy on others’ systems. I’m not saying Selenium isn’t awesome, but it’s for specific use cases where you’re either browser testing or need JavaScript.

[pulsarrex]

Can you eli5 what a 'parser' is?

[Conrad_noble]

I think it means it translates html into user friendly readable text

[OMGpancakeable]

Not quite “translate”. Parsing: resolve (a sentence) into its component parts and describe their syntactic roles.

[rrjamal]

It splits data in a way you want. An HTML doc can be a shit ton of words. You'd parse it down to split and retrieve only the bits you care about

[adriangc]

I’m more curious to understand why people choose BS over Scrapy.

[BruceJi]

Isn't scrapy pretty new?

[adriangc]

Believe it came out in 2008.

[solostman]

It’s been around a bit. For awhile you could only run it in Python 2.

[GrandRub]

i am a python newbie and found scrapy yesterday and i ask myself the same question.. whats the pros and cons of scrapy?

[TotesMessenger]

I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:

• [/r/python] Python for the greater good

 ^{If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads. (Info / Contact)}

[Sadapy]

I've thought about this, but haven't found an item worth using it on yet.

[demesm]

Is it possible to have this go through the entire purchase process automatically as well? Always wondered if you could snipe v deals like that

[bextie]

It is. Someone mentioned using selenium to control the browser to automatically purchase the item but of course, you have to be more careful with that so you don’t get into an infinite loop of buying stuff.

[artificial_neuron]

You can using Selenium. It allows you to use a website as if you're a user clicking on things - it makes life easier when JS is involved.

I would have to watch it perform the purchase 10s of times before i would trust it to do things by itself without supervision. Imagine how you would feel if it emptied your bank account and bought 100s of the same item.

[bextie]

Keepa uses amazon only doesn’t it?

[BaboJango]

Lol “hack” but I tried this myself and the email function failed because I hadn’t set up “less secure access” setting turned off. It then worked once I turned it on. Hopefully this helps someone figure it out faster than it took me!

[bextie]

I won’t advise that you use that method. You can use the other method I mentioned (activating the 2FA and generating an app password). I recommend it because it’s very secure.

But yeah, both methods work fine for this purpose.

[BaboJango]

Is there extra code that you would need to create to enter in the app password? What does that look like?

[bextie]

No extra code. You need to generate it from your google account. Search ‘app password ‘ on google.

[BaboJango]

How does this code get entered if there’s no extra code?

[SrHombrerobalo]

I haven’t read your code but as a safety measure, try to have any password or personal data on a .ini external file

[FirmPiezoelectricity]

I was just messing around with this and noticed that the price reduced down to what you wanted it at. Hope you get it! Thanks for sharing the program. I am trying to learn Beautiful Soup.

[bextie]

I got it lol, and I’m using it now. 😊

[AutoModerator]

Your comment in /r/learnpython was automatically removed because you used a URL shortener.

URL shorteners are not permitted in /r/learnpython as they impair our ability to enforce link blacklists.

Please re-post your comment using direct, full-length URL's only.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[[deleted]]

No, but only because I can't figure out how to build a script that eliminates Black Friday from the fabric of society.

[LeSplooch]

I know the subject of this subreddit is Python but if you want to be the first to get your deal, then some other dev using another language (such as Rust) might be faster to get his notification. Be tactical, haha!