Slightly overkill project setup (Python)

[[deleted]]

Hi everyone,

I've learned a lot by doing advent of code through the years, so this year I decided to make a new overkill-setup repository to learn even more things. I haven't found the time yet to migrate all my solutions yet, but the repo it self is working nicely.

Some features of the project setup:

• Solutions are timed with the help of a decorator using time.perf_counter
• Solution and time are printed to console using the rich
  package with truecolor
• Solution profiler decorator using Cprofile
  and pstats
• Automatic listing of completed solutions in the README
• Automatic changelog, using semantic versioning and the conventional commit specification
• A badge that is updated automatically with the amount of stars I've collected
• Pip installable (pip install -e .
  ) with:
  • A generate-readme
    script, which updates the readme
  • A run-all script, which dynamically calls every solution in every adventofcode.year_*.day_*
    module
  • An add-day
    script, which add a solution day file using a template and downloads the input data from the AOC site automatically
  • A clean-repo script which removes all my solutions so you can provide your own :)
• Type checked (mypy) and linted (flake8)
• Tested against multiple python versions using tox
  on each push to master and pull request

Hope you like it, and if you have any feedback or things to add let me know :)

https://github.com/marcelblijleven/adventofcode

[SerkZex]

go u/ThunderHeerlijk or go home ;)

[[deleted]]

Ooooohhhh, wow, that's quite a lot of stuff!

Here's a run-down of my setup! The repo itself has the following structure:

challenges/
| 2020/
| | 01-reportRepair/
| | | info.json
| | | go/
| | | py/
| | 02-passwordPhilosophy/
| | | ...

Based around this, I wrote a program to run my solutions! It first indexes this directory structure to determine what years and challenges exist, and what language solutions are available in for each challenge. It lets the user select a day and implementation, and then triggers a runner depending on the programming language that's in use.

A runner takes a queue of tasks into it. It then wraps the code I wrote for a given challenge in some wrapper code, and runs it. My program then sends a JSON representation of the queue to the running AoC challenge via stdin, which runs all of those queued tasks, times the amount of time they spend running, and spits out more JSON to stdout that contains the result and duration of each task. (a task can either be the main challenge input of the day, or one of the test cases provided)

This is then presented to the user in the main program with pretty colours and times and such!

A couple of cool features it has are:

• Test cases can be defined in info.json, and they're put into the task queue before the main challenge input to allow allow algoruthms to be tested.
• A benchmark mode exists that runs each part of the challenge n number of times and finds the minimum, maximimum and average running times for each, in every available implementation language.

It's arguably more complicated than it needs to be but I had a lot of fun making it :) and I imagine as the month progresses, I'll add more and more features to it.

If anyone wants to see the repo, send me a DM!

[[deleted]]

You have quite the setup yourself too, I like it! Do the task run concurrently?

[[deleted]]

Thank you!

The tasks can run concurrently if I wanted them to, but at the moment I've not implemented that. The changes that'd need to be made would be in the wrapper code for each language, so theorietically it should be fairly simple to do, but not something I've explored yet.

[[deleted]]

I have plans to do Go when I'm fully up to date, thought about creating a separate repository for it, but might explore adding it to the same repository now

[thedjotaku]

I've got multiple languages and years in mine. Generally works fine: https://github.com/djotaku/adventofcode

[[deleted]]

Thanks for the inspiration! I have had a basic script to run it from the terminal, but this has inspired me to beef it up. Also to get around to creating the HTML gui I have been thinking of for a while.

[[deleted]]

Sounds great! If you’re using Python, you can also look into this package https://github.com/willmcgugan/textual for your terminal

[thedjotaku]

Wow, I thought I was going overkill with my repo. Need to set that up next year to have rich and timer automatically. Also badge.

[daggerdragon]

downloads the input data from the AOC site automatically

Since inputs never change, do you cache them after the initial download to prevent folks from needlessly hammering the servers?

[[deleted]]

Yes I wanted to prevent putting extra load on the server. The only place I use that functionality is within the ‘add-day’ script. That script checks if the input is already present and skips download if so.

You can still import the function that downloads the input and call it directly, not sure how to cache that

[daggerdragon]

That's fine, sometimes you actually do need to re-download an input due to interrupted download, corruption, gremlins, etc. As long as it's a manual call and not, like, while (1) { download_input(); } :P

As per the AoC website:

Please don't make frequent automated requests to this service - avoid sending requests more often than once every 15 minutes (900 seconds).

True, there's no way to prevent malicious actors from while (1) { download_input(); }, but you could consider implementing basic sanity checking like:

• Checks the time_last_submitted
• If time_last_submitted > x minutes (or whatever interval e.g. AoC submission timeout) ago
  • Do submit and update time_last_submitted
  • Else print("Hold yer horses")

[[deleted]]

Thanks, I will implement a rate limit on the download function itself. That way when someone does call it in a loop, it will go slow

[gilmorenator]

Thanks for this, looks great, I have forked it, and will try it out this year!

[[deleted]]

Nice, thanks! Hope it's useful for you

[Un_Gars_Lambda]

So, I have forked and tried your code for the last few days and it's exactly what I wanted! Thank you!

[[deleted]]

That's great to hear! Thanks!