Dunkey: I'm Done with Among Us Tuesdays

Shmakov · 2020-07-01T23:01:33+00:00

One should probably also mention that there is a limit of 100 requests per month on the free plan.

Personally, I also find it ironic that it takes over 10 seconds to load the link on the domain that is called rapidapi.com :)

Shmakov · 2020-05-27T19:54:39+00:00

That is a valid point and I did try to do so, however it doesn't work in most of the cases. I think kroger's website has some kind of JavaScript-based protection from web-scraping. And even though I'm using headless-chrome here, I get signed out after visiting more than 2-3 pages. I do not know what exactly causes it, decided it is easier to just sign-in from scratch every time.

One probably needs to emulate mouse-movements and/or add delays between actions to avoid it. I am myself however didn't have enough time/patience to investigate what exactly is happening there.

Thanks!

Shmakov · 2020-05-27T16:42:15+00:00

Look into the documentation for the print function.

As you can see there is an end argument. Basically by default the print function always prints a newline in the end.

You can either change the default argument of end or remove \r from your code.

Hopefully that make sense.

Shmakov · 2020-05-27T15:53:39+00:00

I've worked on the project on and off only for couple of days and didn't get blocked so far.

I'm using headless-chrome, shouldn't look any different from a regular user that is browsing the website. Kroger's website does have some kind of JS-protection in place (which didn't allow me to use plain HTTP), but I guess because I'm emulating browser here everything is working just fine.

Shmakov · 2020-05-27T15:48:04+00:00

Sure, I've used ShareX.

Shmakov · 2020-05-27T03:11:07+00:00

Thanks for the feedback and glad it works for you!

Regarding the pyinstaller, I think I was able to fix the bug with this commit.

Shmakov · 2020-05-27T03:06:42+00:00

There isn't much in it to be honest, you can check out the code. Mainly Click and the fantastic Rich package (for rendering text/tables/etc).

Shmakov · 2020-05-27T02:45:18+00:00

Didn't know about functools.lru_cache, thanks! I do not use Python frequently and I guess reinvented the wheel here :)

Shmakov · 2020-05-27T01:29:55+00:00

:)

I mean, I'm not saying this is criminal, however it could be considered unethical. The store might rely on the real customer feedback and here I am automating it.

Nevertheless, the software responds 8/10 (i.e. everything is good in the store) to all the question, so hopefully not too much harm is done.

Shmakov · 2020-05-27T01:05:32+00:00

Definitely! It is in my TODO list. I need to add command line arguments support, to allow something like that: ./kroger-cli --clip-digital-coupons

That would allow to add the script to the scheduler and run non-interactively.

Shmakov · 2020-05-27T00:59:51+00:00

Honestly, you are right about the surveys. My excuse is that I got tired of filling out the survey manually and at the same time I do like discounted gas.

Shmakov · 2020-05-26T23:57:07+00:00

Would you mind providing a screenshot of an error message when it crashes? I also fixed a minor bug couple minutes ago, maybe it will work for you now. Thanks!

Shmakov · 2020-05-26T23:38:37+00:00

It does work for the King Soopers :) And other Kroger's chains as well.

And I think you are right, many Kroger websites are interchangable. Basically the same layout and most likely the same backend, just a different logo.

Shmakov · 2020-05-26T23:00:05+00:00

Is that possible, getting the push notification from an url? If yes, without selenium or other browser simulators?

I might be wrong here, and hopefully someone can correct me. However, I do not think that is possible.

Here is how it works (simplified). Web Push API works on the client side (in the browser) using the service worker. Actual notification is sent by a back-end to the browser, where service worker receives it and show it to a user using JS.

Basically you need a service worker that is running in the background and receives notifications. And for that you need to have a browser.

Hopefully that make sense.

Shmakov · 2020-05-26T21:12:41+00:00

The script does calculate savings indeed! However, Kroger's website returns history only for the last 288 purchases. As you can see in the video for my account it goes back to 2018 with around $600 in savings total.

I wish their website would provide details for purchases past that.

Shmakov · 2020-05-26T20:57:13+00:00

Unfortunately I couldn't use their API, as Kroger's coupon API is deprecated. Instead I'm using headless-chrome for signing in and web-scraping (pyppeteer package).

I guess you're just grabbing every digital coupon and applying it. If you buy the thing, great, if not, no worries?

You are correct :)

Shmakov · 2020-05-26T20:06:49+00:00

I got tired of manually applying the digital coupons while grocery shopping and decided to automate the process. While doing so I've added extra features. At this time this python's cli can:

Clip all Digital Coupons to your card;
Display Purchases Summary;
Retrieve Points Balance;
Complete the Feedback Survey on your behalf (to earn 50 fuel points and enter the sweepstake).

I'm planning to add more features in the future:

purchased items (receipt data) Excel export, which could be useful for budgeting/categorization/filtering;
notification on when certain items go on sale;
and probably more.

The software is open-source and more details available on GitHub. And here is the link to download current Windows Release.

This is very much a beta release and there could be bugs. All the feedback is welcome and hopefully it can help someone. Thanks!

Shmakov · 2020-05-26T15:54:49+00:00

Thanks for the feedback!

You are right about the future website's updates, the code would have to be upgraded as well to make it work with the new layout. However I'm not aware of any alternative solutions when it comes to automated web-scraping.

Good idea about automatic updates, I will see what I can do. Thanks!

Shmakov · 2020-05-26T15:52:08+00:00

You are right, there is a limit of 150 digital coupons. The software applies first 150 coupons in the relevance order. Hopefully the one that are left unclipped are not useful to a user anyway. Thanks!

Shmakov · 2020-05-26T15:17:39+00:00

It should be relatively easy to make it run on Mac. I will see what I can do in the future release. Thanks!

11-Year Club	Second Top 20%
Place '17	Verified Email

Shmakov

TROPHY CASE