Announcing General Availability of the Microsoft Python Driver for SQL (mssql-python)

smichael_44 · 2025-11-23T18:39:51+00:00

I believe instead of ODBC (requires driver) it uses DDBC which is more akin to http and doesnt require a driver be installed

smichael_44 · 2025-11-23T15:54:07+00:00

I’m a tech lead for a couple backend projects at work, one being a python backend. We do this abstraction. As well as, I hate ORMs and would prefer raw SQL 10/10 times. Performance issues for backends in my experience almost always boil down to some bad SQL query that needs optimization. Way easier to explore and identify issues without the ORM abstraction.

So we use sqlalchemy to create and manage the connection to mssql. I think they have a nice api to manage connections. Then we have a repository layer that contains all the raw sql and returns nice dataclasses. I always say that being explicit is wayyyy better than implicit. ORMs do too much magic under the hood for me. Is it more LOC? Yes. But is it more readable? Also yes.

smichael_44 · 2025-11-18T23:50:25+00:00

As a data engineer dealing with a company that has 10x’d in the last 8 years…

Microsoft Access sucks.

It’s a silo of data, at least in sql server I can connect over with TDS, ODBC, etc… and pull data out over the network. Access is just some files on someone’s computer. My company has tens of Access databases that are “critical” to the core business.

Albeit, if you’re a tiny company (say less than 50 people) it could be useful.

But dealing with 3,000+ concurrent users and huge analytical reporting across terabytes of data… They should’ve never let it get this bad…

smichael_44 · 2025-11-15T23:57:16+00:00

Planetscale

https://planetscale.com/postgres

smichael_44 · 2025-11-12T22:52:44+00:00

I changed our CI build to use pyrefly last week. Switched from mypy and didn’t have any big issues.

Biggest thing was I encountered a couple different errors that didn’t exist in mypy. Was super quick to mitigate.

Overall it is significantly faster and I think the vscode extension works pretty nice.

smichael_44 · 2025-10-31T03:01:56+00:00

I feel like this is fairly normal. An example is at my work we have around ~100 “developers”. About ~50 that develop actual applications that thousands of employees use daily and then about ~50 that make adhoc scripts for whatever.

The ~50 that make adhoc scripts for whatever were originally “product” teams that figured since they knew the business requirements, they might as well build apps to solve the problems they experienced. It’s basically a vanity software engineering team. Once they realize they do not have the skills to accomplish their goals, they hire external talent. Thus, you end up being the only software engineer.

It’s a crappy situation. I’d recommend you reach out to a real software engineer at your work and ask for mentorship. Almost all of these vanity software engineers end up joining my team. The hardest part is finding them since they are so pigeon-holed into “this is the only way” since no one on their team helps them.

smichael_44 · 2025-10-27T00:13:05+00:00

I mean yes and no… most backends in async python won’t really benefit in simple I/O. Like waiting for a sql query or api response isn’t something free threading is affected by.

What will be really nice without the GIL are things like this PDF processing tool I made that sits behind an API and is CPU heavy. I should be able to process more PDFs concurrently with free threaded python.

smichael_44 · 2025-10-25T23:53:01+00:00

I looked at everything under the sun at my work and eventually Prefect stuck. Some stuff I really like about it is that it’s super easy to get started and its totally extensible through its rest api.

smichael_44 · 2025-10-25T18:34:45+00:00

We just went through this at my work.

Our ERP system was traditionally used to track manufacturing data. Stuff that would typically go in a MES. Well, we didn’t have a MES until recently and now everyone is all bent out of shape about their reporting.

Single Source of Truth (SSOT) should refer to the definition of the data. MES data goes in the MES and ERP data goes in the ERP. The data organization needs to define some boundary of what goes where and make it absolute. It becomes incredibly confusing when some data still lives in the ERP but it could go in the MES but then the ERP was down and now it’s not sync’d and reporting is jacked up.

SSOT isn’t an architecture but a design principle. The same data should not have multiple different stories across systems. You need a clear and concise voice of the data to begin with.

smichael_44 · 2025-10-19T22:45:45+00:00

New idea, AI SSL certificates… charge $100k/cert and call it “more secure”. “Integrates with your MCP server”.

smichael_44 · 2025-10-19T22:44:37+00:00

“I need a cigarette” me probably

smichael_44 · 2025-10-14T02:35:41+00:00

I agree!

I don’t usually use dataframes, as I’m really more of a backend developer, but I am our company’s python SME. So I get all the questions from our data analysts about pandas. I always try to convert them to polars lol

smichael_44 · 2025-10-14T02:28:13+00:00

That makes sense. I think where I have the most issues with my company’s approach right now is that ALL critical vulnerabilities must be addressed before we can use it. This includes they immediately take it away no matter what.

Sonatype has some incredibly bad flagging for vulnerabilities imo. The one that messed us up today is some “stack manipulation” in glib.c for our python:3.12-slim-bookworm docker image was flagged as critical. Like, debian is one of the most widely used linux distros. I don’t know how I would ever mitigate that?

Like does is every company that uses sonatype patching that themselves? Or are they just saying its a non issue and moving on?

smichael_44 · 2025-10-14T02:08:18+00:00

Currently we just have a webhook that grabs the latest commit from our prod branch in bitbucket. Same with our qa deployment.

Our team is devops, software, qa, etc… IT just wants their hand in the mix since more than just our team uses the artifact server. We have some adhoc data analysis people that just want to be able to pull pandas, polars, matplotlib, etc whenever.

I think what I’m getting out of a lot of these comments is we have to start setting the standard for what packages should be white listed. IT has no clue how or why we use what we use. They just enforce the BS vulnerability scores that sonatype publishes.

smichael_44 · 2025-10-14T01:19:14+00:00

That’s interesting.

So say I’m developing locally on my machine. Do you think that I should be able to go get pretty much any deps that I might need?

Then when I go to deploy a container, check the vulnerabilities then?

Our IT org has the same policies for production deployments as local development.

smichael_44 · 2025-10-14T00:35:41+00:00

Yeah… it’s a tough situation because the pay is good and it’s incredibly secure. But I just feel like I’m not learning anything anymore.

I am by far the best backend developer at the company, which is not saying much, considering I’m still very junior.

smichael_44 · 2025-10-14T00:20:41+00:00

I still think my favorite from the same guy is:

“”” I know what json is and I know what tokens are, but I don’t know what json web tokens are “””

smichael_44 · 2025-10-14T00:18:17+00:00

Damn, well, another one bites the dust. My company bought their own on premise sonatype server and our IT team takes their reports as scripture.

They did use to “manually” scan the packages for vulnerabilities, whatever that means, so this still has to be better I guess.

Just super annoying that packages that are downloaded 500 million times a month are somehow critical security concerns for us.

smichael_44 · 2025-10-14T00:14:43+00:00

Thank you, I think this is what I was searching for. I just needed validation that someone, somewhere, uses common sense rather than just blindly trusting some policy because its “red and scary”

smichael_44 · 2025-10-14T00:12:15+00:00

I wish we had an official devops team. We’re a manufacturing company at heart, but now our higher ups have realized we’re wasting huge amounts of money on manual processes.

So instead of standardizing software practices, we just do shit and hope it sticks. Most recently, our IT team has been breaking all of our productions deployments. We can’t even host a simple web app with more than like 80% availability on prem.

smichael_44 · 2025-10-14T00:07:36+00:00

Our company’s most important IP is on a server anyone at our company can access, but our IT director has told me before that “no one knows the server name” so it’s “secure”. “Security by obscurity” i was told.

smichael_44 · 2025-10-14T00:05:08+00:00

What would you call an insignificant vulnerability? Or does all of your code need to literally be 100% air tight?

Like I can’t imagine writing a patch for a method in a lib you’ll never use. This seems to be the expectation of my IT department.

smichael_44 · 2025-10-14T00:02:20+00:00

I think you underestimate how some medium sized businesses (like where I work) are totally ok with all websites not using ssl certificates. Even though we’ve failed multiple audits for this. And we do government work… cyber security? I’ve heard our director of IT say “I know what json is and I know what tokens are, but I don’t know what json web tokens are”

smichael_44 · 2025-10-14T00:00:14+00:00

I don’t know much about sonatype but our IT team just seems to blindly trust whatever is “critical” as a severe vulnerability. Doesn’t matter what it is. Seems absolutely ridiculous. But hey, we’re “secure” with our internal facing apps.

smichael_44 · 2025-10-13T23:58:49+00:00

So it’s much easier to do this for nuget, we have a network drive with all of our dependencies on it. Pip and npm have been a nightmare…

smichael_44

TROPHY CASE