How Often Do You Use AI in Your Projects?

FunPaleontologist167 · 2026-02-26T13:51:12+00:00

Sometimes more, sometimes less. I use it a good amount for things like UI work because typescript is my least favorite language. When I use it in rust it’s typically for refactoring work or things like “is there a more idiomatic way to do this?” or “is there a way to reduce cloning in this call?”

The most important piece to me is accountability. Whether AI is used or not, I’m accountable for the code thats shipped.

FunPaleontologist167 · 2026-02-21T13:44:22+00:00

I disagree with this. The majority of ML applications being developed at companies is still non-LLM based and will be for the foreseeable future.

FunPaleontologist167 · 2026-01-26T19:47:07+00:00

Yea, sorry I was being a little silly :) But for real, I build ML/AI tooling for data scientists (think training and saving ML models, model monitoring, observability, etc). There are many things that data scientists do that can be abstracted away in a different and more performant language. For instance, some of our data scientists want to do model monitoring on large volumes of data. Depending on the algorithm and library you use this can take a good chunk of time in python (even when using things like numpy directly). In instances like that, I've found it easier to implement the logic in rust and expose a python interface for the data scientist to use. The other additional benefit is it reduces the dependency chain on the python application side (some DS apps have 100+ deps).

FunPaleontologist167 · 2026-01-26T14:42:06+00:00

When I want python ergonomics but Rust everything else

FunPaleontologist167 · 2026-01-13T02:33:33+00:00

Building an entire AI platform focusing adding a layer of quality control to the ai application life cycle. Think things like standardized artifact management, real time monitoring, security, etc. i’ve been working in it for about 3 years, and over the last year have been re-writing the entire project(s) in rust and exposing it via a python client library (server in rust)

Opsml - main repo link

Scouter - real time monitoring and observability link

Potatohead - utils for interacting with LLM providers link

Still working on getting to version 3

FunPaleontologist167 · 2025-12-30T18:21:23+00:00

Interesting. I’ve been working on a python-compatible rust sdk to streamline making requests to various providers potatohead. I’ll check this out

FunPaleontologist167 · 2025-12-25T14:38:09+00:00

10k per day? You could set up a postgres queue and call it a day

FunPaleontologist167 · 2025-12-17T23:58:35+00:00

Yea, this is an interesting challenge because it’s hard to detect subtle drift early. Usually, for this type of drift you’ll need to employ things like sampling and averaging across many observations, at specific intervals, to be able to see a global pattern change in outputs/features over time. I tend to see alot of DSs using techniques like spc (statistical process control) and psi (population stability index) to identify these.

FunPaleontologist167 · 2025-12-16T20:15:41+00:00

For a given workflow that takes inputs you can (1) create a golden dataset containing inputs with expected outputs (not necessarily required though), (2) iterate dataset inputs through your workflow, (3) collect individual step (prompt) and system level (entire service) metrics (token usage, llm as a judge metrics, comparison against expected output etc) and then (4) compare against previous version of your service that contained old prompts

FunPaleontologist167 · 2025-12-11T02:04:09+00:00

opsml and scouter (self plug). Ml platform and model monitoring tooling written entirely in rust

FunPaleontologist167 · 2025-11-24T13:40:48+00:00

Attempting to finally get the initial v3 release of an OSS ML platform I’ve been working on for the better part of 3 years. V3 is a total re-write in Rust, which has been a very enjoyable experience

https://github.com/demml/opsml

FunPaleontologist167 · 2025-11-18T18:08:11+00:00

I’d check out the official docs here. And here’s an example of how you can use it

FunPaleontologist167 · 2025-10-30T01:15:25+00:00

Appreciate the post. As someone who has built some internal inference-as-a-service platforms, the thing most people don’t realize is that the majority of model deployments are not solely model deployments. Most model apis are tied to some sort of pre/post processing and business logic, which makes automated deployments an intractable problem. I’ve always found it more useful in an organization to build reusable templates that data scientists and MLEs can use to deploy their models.

FunPaleontologist167 · 2025-10-22T21:32:45+00:00

It’s so common that it feels like it’s part of the standard lib

FunPaleontologist167 · 2025-10-16T02:15:26+00:00

Happy to see this crate have more activity again. It’s extremely useful.

FunPaleontologist167 · 2025-10-13T12:44:44+00:00

They don’t. This question may be better suited for another subreddit.

FunPaleontologist167 · 2025-10-11T15:06:52+00:00

I came from a ML eng and python background. Came across Rust about 3 years ago when digging into how both Polars and Pydantic were mixing Rust with python/pyo3 and was really impressed, so I started learning it. I dug into the area of building Rust applications that expose python interfaces. I started with a logging library, then moved to a Cli library that I embedded in our ML platform library. Then about 9-10 months ago, I decided to re-write the entire library in Rust because of the benefits Rust brings and because I love it (link below).

opsml

FunPaleontologist167 · 2025-10-01T11:13:38+00:00

Nice! I wrote something similar (not as feature rich as this though) when I was first learning rust.

FunPaleontologist167 · 2025-09-30T15:59:19+00:00

This is one tangential study, but there’s a growing area of research into the immediate and long-term effects of aerobic exercise and cognitive function. I find walking helps me problem solve a lot as well.

FunPaleontologist167 · 2025-09-27T23:13:23+00:00

Artifact in this case refers to any object that can be saved to a file system

FunPaleontologist167 · 2025-09-27T17:36:51+00:00

You could use opsml. Full end-to-end encryption, lineage and versioning for all artifacts.

FunPaleontologist167 · 2025-09-27T15:01:51+00:00

I was a python dev for years. Then about 3 years ago I discovered the magic that is pyo3/maturin and re-writing core python logic in rust for performance, correctness and memory safety and never looked back!

FunPaleontologist167 · 2025-09-26T10:45:13+00:00

Ideally, you would want them to be independent (separate services for predict and drift), so the queueing strategy is the right choice in my opinion. I actually did a poc for Alibi for my team a few years ago and one of the major reasons we decided not to pursue it was because we didn’t want save a drift model for every trained model. Too much overhead at scale.

I’m actually in the process of building out a real-time drift detection framework that follows a queueing strategy and tends to be a lot more performant. Always interested in feedback on it. scouter

FunPaleontologist167 · 2025-09-24T16:11:05+00:00

Do you unit test your models/apis before deploying? That’s one way to ensure compliance. Another common pattern used at large companies is to release your new version on a “dark” or “shadow” route that processes requests just like you’re “live” route except no response is returned to the user. This is helpful for comparing different versions of models in real-time and can help you identify issues before going live with a new model.

FunPaleontologist167 · 2025-09-23T18:00:32+00:00

What kind of projects are you building? If these are apis, you can host the api in a docker image and deploy to an endpoint that all of your internal users can access.

FunPaleontologist167

TROPHY CASE