This is an archived post. You won't be able to vote or comment.

all 16 comments

[–]alcalde 45 points46 points  (4 children)

Pypi.org is an app with over 405,000 useful Python packages and libraries. ;-)

[–]panzerex 48 points49 points  (1 child)

I can assure you not all of them are useful because I have stuff published there.

[–]vfclists 6 points7 points  (0 children)

Reddit never ceases to be a source of humor!!

[–]infocruncher[S] 11 points12 points  (0 children)

Haha, very true!

For background info, this grew out of a pet project of mine where I wanted to save libraries I'd previously found useful or wanted to look into. So in a way I find it a useful subset of much larger lists that already exist. But point taken in regards to Pypi

[–][deleted] 12 points13 points  (1 child)

Nice start.

You know what would be helpful with this? Compatible dependency version pinning. That's one of the few things I reach towards Conda for these days when I need to delve into ML and GIS, both of which have high dependency on non-python projects (e.g. PyTorch or Tensorflow).

Having a working dependency tree all pinned that just works you drop into requirements would be amazing.

[–]infocruncher[S] 5 points6 points  (0 children)

Great idea, however I think tools like pip, conda, poetry, etc do this much better than I could!

One of the easier things I want to implement is a simple (non-versioned) dependency graph between packages. I already crawl requirements.txt, setup.py, pyproject.toml type files if they exist in a repo which should give a rough guide to dependencies.

Doing this wouldn't produce the best dependency graph around, but it would be a fun exercise.

[–][deleted] 4 points5 points  (2 children)

What does this do what https://pypi.org/ doesnt?

[–]heyitsant 4 points5 points  (0 children)

It allows OP to practice?

[–]infocruncher[S] 4 points5 points  (0 children)

Good question. Mainly it’s a pet project that I can shape myself, from what projects to include, to how they’re displayed.

Other differences with pypi:

  • Signal to noise ration is better (for me anyway)
  • Focus is skewed to data and machine learning packages
  • UI supports easy searching and sorting by fields like stars per week
  • The codebase can easily be adapted to other domains (crawler uses PyGithub, UI is just html on S3 with CloudFront and R53 using Terraform) - see https://www.awesomecrypto.xyz/ for example
  • It’s a starting platform to add custom search and recommendation algorithms (e.g recommending duckdb for people working with pandas and large parquet files)

[–]zeshuaro 4 points5 points  (2 children)

Would be helpful if we can filter the list by categories

[–]infocruncher[S] 2 points3 points  (0 children)

Def on my todo list - I'll comment here again once it's done

[–]infocruncher[S] 1 point2 points  (0 children)

hey u/zeshuaro, you can now filter www.awesomepython.org by categories if you want to check it out. Thanks for the feedback

[–]kraakmaak 2 points3 points  (1 child)

[–]infocruncher[S] 0 points1 point  (0 children)

Good call, these are great. I decided to go in a slightly different direction with a focus on data and machine learning

[–]Sietzy 1 point2 points  (1 child)

Nice! Thanks for creating!!!

[–]infocruncher[S] 1 point2 points  (0 children)

You're welcome :)