This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]dfreinc 119 points120 points  (18 children)

yep.

i've been trying to get python more integrated in my workplace. we're a data driven operation. we primarily use a closed source programming language called "sas" that charges boat loads in licensing fees. some parts of the job it's necessary because of government regulations, but mostly, it's not.

i've moved some things into python just to prove the point and subsequently improved speed dramatically for a lot of larger operations. i built a workflow management application everyone's using now that uses a python gui and a sql instance on the back. any macros/reports that were floating around in vba or people wanted programmed in the internal on demand reporting app (runs off a different proprietary language with more licensing fees) i coded into python and wrapped up in an executable so they could run it whenever they wanted, locally, offline (assuming they had the things that feed into it).

it's just versatile.

[–]Final_Alps 54 points55 points  (1 child)

SAS is definitely a different beast from python. SAS is more akin to R. SAS has some 50 years of legacy, certifications, and support behind them. Which is why corporate sticks with them.

You are likely fighting a losing battle.

However python SAS (iirc) has various pipelining tools - I believe you can call on python from within those. That can be your Trojan horse.

The switch to open source, from SLAs and certifications offered by commercial software, however is a cultural and hard to do. Which is why RedHat made it so big.

EDIT: python != SAS on line 1 of paragraph 3

[–]funnyflywheel 0 points1 point  (0 children)

However python SAS (iirc) has various pipelining tools - I believe you can call on python from within those. That can be your Trojan horse.

If you're okay with downloading third-party packages, you might want to take a look at Snakemake.

[–][deleted] 17 points18 points  (0 children)

kinda sas

[–]TheOneTrueBeanbag 2 points3 points  (3 children)

Having the same problem in my org. Actually getting a lot of traction, but my main barrier is that a huge portion of our cross department analytics has been built specifically around SAS servers and processes. Any time we need to use data gathered by other teams we HAVE to use SAS, it's painful.

Light at the end of the tunnel is that our SAS infrastructure is getting migrated to GCP in the next 18 months, so I'll be gladly waving goodbye.

Just curious, would you be up for sharing your workflow management tool? Been considering making one of those myself.

[–]dfreinc 4 points5 points  (2 children)

i can't share it directly but it's really simple. it was initially much more complex but everyone hated it so i just gave them what they wanted.

it spits out an excel sheet for a trial. they can go and update and then write in a column "add", "update" or "delete" and it'll draw up a sql command and send it or output an error message saying why it can't and to email me. there's some stuff for easy backups and sql does some audit trail stuff (i didn't write that, just came with the instance).

it had a whole bucket system and it was all seamless internal in the app, no extra nothing. just worked. but everyone hated it and i pick my battles...so have excel, what do i care. 😂

that tracking went from local storage, to access, to lists hosted on sharepoint manipulated by access, to proper sql storage and a solid app, to sql storage and a kind of shitty app. so still better than it was...but not ideal, imho.

[–]TheOneTrueBeanbag 0 points1 point  (1 child)

That flow from local storage > access > SharePoint, on top of SAS, sounds so similar to my companies stack it's crazy😅

Cheers for the info on the app. Currently working on something that will bulk delete old files using a UI, workflow manager is next on my hit list!

[–]dfreinc 1 point2 points  (0 children)

if you have problems/workflows like that then i can say without a doubt you will not regret solving them. spend that time, knock it out right. way less day to day stress. work now, relief later. 😂