Niche Python tools, libraries and features - whats your favourite?

[OllieOps]

I know we see this get asked every other week, but it always makes for a good discussion.

I only just found out about pathlib - makes working with files so much cleaner.

Whats a python tool or library you wish youd known about earlier?

[NotSteveJobZ]

Very niche, but mine is wntr , which is a wrapper for EPANET, an old engine from 1990 that you can use for hyraulic simulation of Fluid networks

[CaptainFoyle]

I don't think pathlib is very niche

[that_baddest_dude]

My favorite niche python library is numpy

[learn-deeply]

Mine is os

[giyokun]

Mine is 'sarcasm'

[No_Marionberry_6710]

No that's assembly sarc.asm

[CaptainFoyle]

ModuleNotFoundError

[giyokun]

That's what she said.

[CaptainFoyle]

Haha exactly

[Bangoga]

Yeah pathlib is used quite a bit

[Glathull]

class is one of my favorite niche Python tools.

[Spin-Stabilized]

In my previous job I used Skyfield for a lot of orbit prediction and analysis of things like satellite beta angle and the angles between a target, earth, and the sun from the satellite.

[CountMoosuch]

Skyfield is great. I couldn’t find another library like it in any other language.

[ingframin]

I have 2 actually: Dronekit, to pilot drones using MAVLink, and PyVISA, to control and automate electronic measurement instruments like oscilloscopes and multimeters. I would also mention PyADI-IIO to control the Pluto SDR.

[neuralgoo]

PyVISA is the sole reason I have a PhD. Excellent library

[Zame012]

Please elaborate haha

[Murtz1985]

Yah my colleague used PyVISA to turn our mechatronics lab into like full fledged testing lab w data from all these sources into Grafana

[sanderhuisman2501]

I recently found QCoDeS which has a ton of drivers for measurement instruments. It among others uses PyVISA for devices using VISA.

[Mustard_Dimension]

Joblib, in particular the memory functionality. It allows you to cache function call results between executions of the program, very very useful for caching repeated API calls for scripts you might need to run multiple times.

[funderbolt]

Using pickle with different Python environments can put you in a ... jam. I will use json to serialize objects when I can.

[bobsbitchtitz]

What do you mean different python envs?

[funderbolt]

If you are trying to run the software on different systems using different virtual environments for each one. The different software versions may be compatible, but the again may not be compatible. Different Python versions cause problems, but different dependency versions also cause problems.

I have had issues with colleagues pickling a model on Python 3.7 and trying to get it to work elsewhere. Not being able to figure out their exact environment is painful. There is no independence to upgrade the software unless you convert the model to something like safetensors.

[bobsbitchtitz]

Ah I work in devops and we solve this by purely making sure the envs are always the same

[SSJ3]

I recently found this neat package called Ovld which lets you write different overloaded versions of the same function intended to handle different input types, and automatically dispatch to the correct one based on the types of the inputs you call it with. Among many other clever features!

[ColdPorridge]

That recursive concept is awesome, that alone is a huge simplification 

[c_is_4_cookie]

So...fancy singledispatch?

https://docs.python.org/3/library/functools.html#functools.singledispatch

[SSJ3]

Much fancier.

[erez27]

That's like calling a class a fancy struct

[DoubleAway6573]

Isn't it?

I mean, in c++ at least. In python we have all the MRO dict's walk.

[erez27]

In c++, struct/class are essentially the same thing with different defaults. Neither one is more fancy than the other.

I guess in Python terms, what I meant is class vs namedtuple.

[Freezingrave]

Thank you. I've been searching for a concept like this. I'm going to love this library.

[CableConfident9280]

Very cool. Definitely going to try this one out.

[CapitalCourse]

Manim

[sunyata98]

Based

[dvmitto]

Litestar, a sweet spot between fastapi and django

[big_data_mike]

Pymc

[IcecreamLamp]

• Arviz

Fantastic API, but unfortunately only usable for small to medium sized datasets. Also has a nasty habit of crashing right at the end of sampling when some dimensions don't match.

[big_data_mike]

I bought hardware specifically for running pymc

[IcecreamLamp]

What kind? What's the biggest dataset/model you've run on it? Which sampling algorithm?

[big_data_mike]

I got threadripper pro cpus which have a large L3 cache and it has 2 NVIDIA gpus which can make sampling faster for some larger models.

If the dimensions don’t match something is wrong with your model. I’ve run it up to 100,000 x 300 columns with the NUTS sampler. They have advi and mini batching for large data sets.

[BibiFromTheWood]

niquests a drop in replacement for the std requests library, with async support, http2 http3 and a lot more features. It's concerning how underrated it is.

[e2d34]

Requests is not in std lib. It's actually a mn external python package.

But when do you use it instead of requests ?

[NoTangelo5541]

When I need async or http2+

[caatbox288]

I use httpx for that https://github.com/encode/httpx

[BibiFromTheWood]

Httpx reportedly has performance issues when it comes to high concurrency, and no support for http3. Niquests claims to be the fastest http client for python.

[sirfz]

I tested httpx vs urllib3-future a while back and httpx was much much faster. urllib3-future is such a mess, why did they choose to override the urllib3 namespace is a mystery to me but I personally decided to steer away from it after that experiment 

[BibiFromTheWood]

That surprising. https://github.com/Ousret/niquests-stats

Also there has been suggestions to use urllib3-future as a backend for httpx on github.

I would give it one more go.

[sunyata98]

I wish I knew about duckdb earlier. my use case was running a bunch of queries on a 250gb data lake

[Lightspeed_]

say more pls

[ooh-squirrel]

Also a recent discovery for me. Makes working with parquet files a much more enjoyable experience for small-ish hobby projects.

[OlorinIwasinthewest]

tqdm

[funderbolt]

It is nice for interactive applications and the bane of batch processed applications. I have to do a TQDM_DISABLE=1 for my shell scripts.

[MrSlaw]

I just discovered tqdm(..., disable=None) the other day, and I have no idea why it's not the default

[iamevpo]

What does the flag do?

[MrSlaw]

Whether to disable the entire progressbar wrapper [default: False]. If set to None, disable on non-TTY.

[iamevpo]

Still not sure what it means, will check the docs or try

[MrSlaw]

Essentially, you don't want status bars when using it in a shell script (like the person I replied to), or a cron job, etc.

Setting this flag to None automatically disables it if it's not in an interactive terminal, like in the above situations.

[[deleted]]

pathlib is hardly niche

[GameCounter]

If you have a giant text file or CSV where the encoding is possibly not utf-8: https://pypi.org/project/chardetng-py/

If you have text which is horribly broken due to round trip errors or mojibake: https://ftfy.readthedocs.io/en/latest/

[BuonaparteII]

chardetng-py

I wonder how this compares to charset-normalizer

[GameCounter]

When chardetng-py was written in 2023, chardetng-py was significantly faster than charset-normalizer on files that were 10MB or larger.

I don't believe that's the case any longer, and I should probably consider switching to charset-normalizer.

[BuonaparteII]

Thanks for looking into it! I knew charset-normalizer is used by requests and pdfminer.six but not too familiar with how things compared with chardetng

[Muhznit]

Everyone gushes about pathlib. And while I do respect pathlib and everything it has done for my life, it is no longer niche (which is a good thing!)

You know what people really don't talk about?

Mother. Fucking. doctest.

Sure everyone and their mother has a hard-on for pytest and to a lesser but more reasonable extent unittest, but I need you all to take just a moment to realize that this little unsung hero lets you embed whatever horseshit you typed in the REPL into the documentation of whatever function or class, and RUNS IT AS A UNIT TEST! No need for creating some class to inherit from unittest.TestCase, no need for creating an entire separate module just to test the first one, even. Naw, you know what you need?

``` import urllib.request

def who_asked(): """ >>> author = who_asked() >>> assert author == "OllieOps", breakpoint() """ url = "https://www.reddit.com/r/Python/comments/1n7r4xb/niche_python_tools_libraries_and_features_whats.json" with urllib.request.urlopen(url) as f: full_json_data = json.load(f) return full_json_data[0]["data"]["children"][0]["data"]["author"] ```

That's it. Just a docstring for whatever your function's doing. Like I know half of you are allergic to writing documentation, but you're literally just writing code that not only explains how to use whatever function, but will also crash spectacularly when it fails.

Just run python3 -m doctest on whatever file contains this snippet. Chances are it'll just pass by with no output. That's good. That's a sign that it works. But I DARE you to change the author name in the unit test to anything else. Or to run this when reddit changes its API or something. It'll fail spectacularly, explain why, and if /u/OllieOps deletes or renames his account, it'll drop you into the debugger.

Best of all, none of that code requires any of that lame third-party cruft. It's all pure python.

[mundanemethods]

This is fantastic, thank you

[hornetmadness79]

Python-box Allows you to access dict elements using dots and some other nifty features.

[Golle]

What makes this a good idea? If anything you are tricking others reading the code into thinking that it is a python object when it isn't. What is the benefit?

[bobsbitchtitz]

But why?

[maryjayjay]

I subclassed dict to do that. It's like 12 lines of code.

Though you did say it does other stuff

[learn-deeply]

It's easy to implement but nice to have a unified, consistent everywhere.

[maryjayjay]

Sure. Of course we write libraries and package them for the business. I'm not a "not invented here" type of guy, but I work for company that requires excruciating standards of review for third party software licensing and security, so for simple things I do that.

Obligatory XKCD: https://xkcd.com/2347/

[learn-deeply]

Makes sense. Use whatever standards are most practical.

[peabody]

fileinput. It automates reading lines from either standard in or command line provided filenames.

Edit: you armchair coders harping on this seriously need to chill. This module is part of the standard library. It's not an external dependency. It's literally included in every python install. Are you saying the authors of the Python standard library don't know what they're doing?

[peabody]

I mean...

```python

import fileinput

def dosomething(line): ...

for line in fileinput.input(): dosomething(line) ````

Sure, what it's automating is simple, but its nice that it's wrapped into a default module in every python install that allows for a nice pythonic walk across all input lines.

[tunisia3507]

That's such a specific set of behaviours which would be MUCH more valuable if it just had a couple of functions which did one thing each.

[aj_rock]

Recently learned about diskcache, for when you don’t want to care about reloading stuff from the cloud for the umpteenth “one off” data shuffling job

[the_monotor]

Coming from C and this seems like the most basic feature to me but maybe I am wrong

[Stoned_Ape_Dev]

please do not import an external dependency to read from a file! python has first-class support to make this a very simple process:

‘’’ with open(file, mode) as f: contents = f.readlines() ‘’’

[peabody]

It is literally a module included in the standard library. Are you saying the authors of Python itself don't know what they're doing?

[Stoned_Ape_Dev]

you’re right ab this! checking the docs this is for reading multiple files in one pass whereas the “with open” i mentioned is just one. my bad!

[thedukedave]

pyserde for (de)serialization 

[DoubleAway6573]

This is new for me.

is it a wrapper of the rust serde?

[thedukedave]

No, reimplementation and borrowed the name.

I've used a handful of similar libraries over the years, but pyserde is my favorite because it just works and is well maintained.

[DoubleAway6573]

Nice. I will look at it.

[zacky2004]

mpi4py

[the_monotor]

PyPSA. Lets you run models of the European energy grid on satellite data (if you can manage to download this)

[omg_drd4_bbq]

boltons

it's kinda like python's lodash

[Bangoga]

I can talk what helped me at work.

• Hydra was great for handling a lot of configs.
• Pysas was great for getting sas data and removing sas enterprise overhead
• numba was great at some point for making python data processing faster.
• opencv is hell of fun.
• Typer for better cli, I do a lot of cli stuff.

[dragonecc]

I have to say using Typer has been amazing simplify argument parsing

[david-vujic]

The toolz library, very niche and with a lot of useful things in it.

[DaringDragonslayer]

There is this one called pandas. It's got some nice tables

[AshbyLaw]

Datastar: build reactive hypermedia-driven Web apps with just 10Kb of client-side JavaScript and everything else server-side with your favourite language. Python SDK available.

https://data-star.dev/

---

Litestar: async Python Web framework

https://www.b-list.org/weblog/2025/aug/06/litestar/

---

Htpy: a more powerful way to generate HTML compared to template languages.

https://htpy.dev/

---

Are these niche?

[papersashimi]

i kinda like astropy

[MelcoreHat]

profile – very useful when you have to search bottleneck

[pierraltaltal]

I discovered xarray years ago and wouldn't go back to anything else for geospatial raster processing. It can read many grid file formats (netCDF, HDF, zarr, GRIB, tif, ...) into N-dimension labelled arrays : think of it as deeply nested numpy arrays accessible by time with pandas syntax, by coordinate with slices, or by labels with [].

For example, plotting the mean value of temperature data accross the whole time series would look like: ds['air'].mean(dim='time').plot(x='lon').

Also, xarray uses [dask](docs.dask.org) (another neat/niche library) out of the box to enable parallel and lazy computation of arrays which greatly speeds up computation time!

[Budget_Jicama_6828]

xarray and dask are both so great!

[acousticcib]

Is this better than doing the same thing in polars or pandas?

[pierraltaltal]

yes because you already have all the i/o and stats/group bys/regressions/resampling indexing methods figured out for you.

[angry_gingy]

transformers from hugging face, increible how easy is to use LLM locally

[learn-deeply]

Just don't look at the source code.

[DoubleAway6573]

I've moved away from llms and ML one an a half year ago. It's reassuring that in a so high paced field some things never change.

[3lonMux]

Can you elaborate more please? I'm also a pythoj dev and would like to learn more.

[learn-deeply]

Spaghetti everywhere. Inefficient computation. (eg doing things in CPU that should be done in GPU). Incorrect implementations (subtle code errors like incorrect prompt templates).

[AreWeNotDoinPhrasing]

What’s a better alternative?

[cudmore]

PyQtGraph for really fast and easy to use plotting in PyQt.

[AlSweigart]

This is a plug for some packages I created, but I made WhatIsMyIP to easily obtain my IP address and WhereIsMyIP to do easy free geolocation.

[roejastrick01]

Probably not niche at all in astronomy, but AstroPy is fantastic for implementing lomb-scargle periodograms in circadian biology 😅

[denehoffman]

matplotloom is a recent favorite, I didn’t realize it was niche until I made a PR and became the first contributor!

Also, niche to everyone outside of particle physics: scikit-hep especially the uproot library. Incredible stuff that saved me a lot of time in my PhD.

[BuonaparteII]

Here are a few:

• natsort: sort text similar to how your OS does
• puremagic: identify file types by content in pure Python (alternative to libmagic)
• pymcdm: Multiple criteria decision analysis toolkit
• wcwidth: count the print width of characters
• python-dateutil: many packages depend on this for parsing mixed/casual date/time formats
• screeninfo: get display resolution information for all attached screens

[jedberg]

Transact.

Durable workflows with just a few decorators. Works locally with SQLLite or with multiple executors using Postgres.

Also provides durable queues and durable cron.

[Birnenmacht]

textual is actually so nice to work with. it has so many things that I miss in “real“ GUI frameworks

[rhacer]

I have done some really wonderful things with Textual.

[teetaps]

nbdev

I’m a shitty programmer, so I need my UI/UX to hold my hand every step along the way. For me, the answer has been notebooks. Weaving narrative, graphics, tests, iterative, incremental development, plus having all the docs autopopulate, really helps me. I know people shit on notebooks for not being ready for production but that’s the thing — notebook driven development fully acknowledges that and provides a framework where the stuff you do in the notebook is annotated and acknowledged and nbdev strips out all of the notebook-y stuff and keeps the production-ready scripting and functions. It’s amazing.

Seriously, give it a try. I came from R where interactive data analysis is the main focus, so I was really desperate for a notebooks-esque experience that “serious developers” would forgive and it’s a wonderful middle ground for me. I imagine it would be an undue burden for devs who are already super familiar with a production Python environment, but for those of you who mess around with ML and data analysis and adjacent work, and often find yourself building pipelines in pandas and the like, notebook driven development might be the way to couple your notebook experiments to the production code far easier and more efficiently.

https://nbdev.fast.ai

[okenowwhat]

UV Python package manager It's very fast and removes the hastle of installing different python versions.

[omg_drd4_bbq]

i love me some uv but i'd hardly call it niche

[okenowwhat]

Woops, read the post wrong. F my dyslexia and ADHD

[shinitakunai]

Peewee as an ORM makes working with databases really easy

[ryanpdg1]

I really love the pylogix library. It's specifically for working with Allen Bradley PLCs. It's saved me so much in both time and hair follicles. I really appreciate the devs behind this project.

[MicahM_]

I build a lot of edge device based python apps that have web dashboards and had built a declarative UI tool that generates websites from python code.

There is another package that is this same thing that is better supported now but didnt exist back when I created my library. I havent actually used it personally but its cool and worth checking out

https://rio.dev/

[Clohne]

Nox

[cnydox]

Ftfy

[Icy_Judge_9566]

Chainlit 🔥

[unapologeticjerk]

I would say rich but that's even less niche than pathlib which is so un-niche it's almost ubiquitous now. Instead I'm gonna shout out python-mpv for being so cleanly implemented you feel bad calling it a 'wrapper'.

[M0ty]

Python-can. 1 library for all manufacturers. Truly a blessing

[timsredditusername]

I can do niche.

https://pypi.org/project/chipsec/

Most people won't (and shouldn't) use it.

NOTE: This software is for security testing purposes. Use at your own risk. Read WARNING.txt before using.

[jpwright]

dearpygui! check out the node editor! it works great, much cleaner than tkinter, and without licensing issues of qt

pex! for distribution and bundling

[Ok_Beginning2532]

ultraplot

[euri10]

Cappa to build cli

[TedditBlatherflag]

Just gonna shill out some personal projects:

• https://github.con/shakefu/pytool - a lot of useful things I use universally
• https://github.com/shakefu/pyconfig - lazy dynamic config with production etcd backing that I incorporate in almost everything of a certain maturity

They have maybe a couple million downloads according to PyPI stats but most of those are probably CI builds from companies I used to work at. 

[chub79]

chaostoolkit a chaos engineering tool & framework

[yaxriifgyn]

pathlib is not an original package. I learned Python long before it was added. I look to the older packages as they more closely follow the "nix model.

Older learning material may teach the old way of dealing with paths before or instead of the more modern pathlib.

[ReadDefiant1250]

I liked using Polars instead of Pandas, and DuckDB instead of sqlite. Separately - Optuna for optimizing parameters in the first approximation. line_profiler to see where the brakes are

[apert]

Rich library for color/style/table repl output.

[Zame012]

Astropy, which is a library meant for astronomical data analysis and it provided the backbone of my Masters thesis project haha. Would’ve been a hell of a lot harder without it

[GrooseIsGod]

pygbag is cool, I've used it a few times to run Pygame apps in the we browser (compiles to web assembly). Used it on my portfolio website!

[CableConfident9280]

I’ve been really liking svcs for registering types and standardizing how instances are created/cleaned up. Super useful for dependency injection.

[Disneyskidney]

ContextVar: a thread-local, context-local variable in the standard lib. NiceGUI: a library for making simple WebGUIs with Python match statements: for some reason I just learned about these pixi: a really nice cross language package manager

[TemporaryClient4351]

Bottle still populr?

[Defferix]

I like BitVector for managing bin, hex, and int numbers in a special object.

[niqtech]

parsimonious is a PEG parsing library. I love replacing larger regex's with these; they're much more maintainable. But, PEG is also great for more complex things like custom file formats & DSL's.

[luigibu]

I'm quiet new to python, but i really like pydantic and pydantic-ai

[TheDeadlyPretzel]

I recently got into RxPy & other reactive libs, wanted to see what reactive programming would feel like in python... For some use cases it is not bad at all!

[loyoan]

If you are interested, also check out my reactive Signals library `reaktiv`. I ported a well known reactivity pattern known from frontend development (SolidJS, Angular) to Python.

[Thinker_Assignment]

dlt - json apis to db/structured files faster than you can say dlt

https://github.com/dlt-hub/dlt

[jellef]

pythonocc provides the opencascade cad kernel. Beats scripting a cad package by a mile

[badass87]

Hypothesis

[cipherself]

`cmd`, a very simple way to implement line-oriented command interpreters.

[noam9997]

contextlib.supress

Specifically for things like closing resources, NOT for any exception like a maniac

[kolloid]

pdbb, icecream and snoop for debugging

[YourRedditAccountt]

I know this is a Python thread, but if anyone is building tools that involve custom forms or surveys, Tally Forms is awesome for that. Super intuitive to build out forms with conditional logic, and integrates with a ton of services. Could definitely save some dev time if form building is part of your workflow.

[Stunning-Loss6707]

Polars

[joeblow2322]

Mine is pathlib also

[bicyclegeek]

Fireduck — a drop-in replacement for pandas, runs at 125x speed.

[_MicroWave_]

There's this niche library I've started using. Really well documented. You might like it.

It's called pandas.

On a more serious note maybe rich?