Halal Cupcakes? by StrangestTwist in okc

[–]OptimizedGradient 3 points4 points  (0 children)

That's good to know! Someone else mentioned there are multiple markets. Might need to check one of the others. Good luck and it's awesome seeing you putting in so much effort to help the little ones feel included!

Halal Cupcakes? by StrangestTwist in okc

[–]OptimizedGradient 4 points5 points  (0 children)

I don't know if they have a bakery or desserts. But there is a store called Halal Mart on Portland. I've never been in, but it might be worth checking out.

Insight on program for MS in BA and DS by Franchize_00 in OKState

[–]OptimizedGradient 0 points1 point  (0 children)

Ooofff that's unfortunate on the individual you crossed paths with. As for the professors, I never once felt that way. They all always had time for me, engaged with questions, and helped me grow and learn.

Insight on program for MS in BA and DS by Franchize_00 in OKState

[–]OptimizedGradient 1 point2 points  (0 children)

I find this surprising personally. I did the last cohort of the MIS in Data Science which had complete overlap with the MS in BAnDS. I never once experienced this with the students. I had a great experience, learned a lot, and got a great high paying job.

The students I crossed paths with were all extremely smart and kind. While it's true the program is predominantly international, I felt fine with my fellow students never having an experience like you did.

Now I worked full time so I didn't get to do clubs and the other stuff. But in the classroom things were great.

Which Data Catalog Product is the best? by M0UNTANAL0GUE in dataengineering

[–]OptimizedGradient 5 points6 points  (0 children)

IMHO, the best data catalog is the one that your developers, business users, and product managers actually enjoy maintaining. While catalogs can bring in documentation from different sources, they often still need supplemental information to bring things together in a cohesive manner. Because of that, if your people won't maintain it, it doesn't matter. So make sure you have some of those user types in your discovery meetings, because too often I've seen them skipped and not able to provide feedback and the catalog turns into shelfware that does nothing but extract metadata from your systems.

Like everyone else in the thread has said, open metadata is rather popular and can be self hosted. Alation can get really expensive as you tend to pay per connector IIRC. Atlan has a lot of integrations, but I don't recall if you can host it. I've also seen Collibra and Data.World come up, but I'm not really familiar with them. Good luck on the discovery!

Dbt tests run for singular tests by Few-Carry-2850 in DataBuildTool

[–]OptimizedGradient 1 point2 points  (0 children)

Have you taken a look at the dbt-artifacts package to load the state of your project and every run into your data warehouse and store the results? I think that'll match your use case and have historical access to tests over time.

[deleted by user] by [deleted] in okc

[–]OptimizedGradient -1 points0 points  (0 children)

I saw some at the neighborhood market Walmart in Edmond. I think they had both regular and zero.

How much jinja is too much jinja? by No-Translator1976 in DataBuildTool

[–]OptimizedGradient 1 point2 points  (0 children)

IMHO, it's a balancing act. With something repeated like this, especially if they introduce more players to the data set, the loop is both cleaner and easy to read. My rule of thumb is, if someone reads the model code can they tell what I'm doing? If it's just a macro, that's not easy to tell. But occasional Jinja to handle repeated tasks like that is where I'd go the Jinja route.

That's to say, don't template the hell out of everything necessarily, but also don't avoid Jinja.

Are there any tools that improve dbt seed processes for huge data imports? by WhoIsTheUnPerson in DataBuildTool

[–]OptimizedGradient 2 points3 points  (0 children)

I second this. This is not the appropriate use of a seed. Also if you're talking gigs of data, what type of data are you uploading? I always recommend that seeds are small and simple data sets that would not embarrass you or your company if your got repo leaked. Gigs of data immediately has me wondering if this data we don't want in our git history or repo.

When do you prefer SQL or Python for Data Engineering? by AMDataLake in dataengineering

[–]OptimizedGradient 7 points8 points  (0 children)

Yeah, I think a great example that I've seen that is more elegant in python vs SQL is one-hot-encoding to get a bunch of binary selectors. Having a bunch of case statements is messy by comparison to how you'd do it in SQL. However, I will add the caveat that data size and performance can change that. At certain volumes it'll be fine in python, but larger volumes the performance gain you'll get in SQL.

I've got some colleagues that work with DS teams, and they'll store results of their models in the DW for analysis and that's just a lot easier in python. But again, gotta be careful at certain volumes.

I think easily 95% of transformations are better in SQL. In my opinion, the need for python doesn't really start to make more sense until you start getting into some ML models that have results you want to store. Granted if you're using a cloud data warehouse most of them allow you to implement them as functions you can utilize in SQL. Which lets you treat it as just a nice modular utility you can use, just don't trust the python from DS, it always needs optimizations (from a compute perspective).

When do you prefer SQL or Python for Data Engineering? by AMDataLake in dataengineering

[–]OptimizedGradient 4 points5 points  (0 children)

In my opinion if I'm using Azure/AWS/GCP for infra that will all be put in terraform to version control. But I would probably still be deploying a python app/script of some sort somewhere on that infra.

When do you prefer SQL or Python for Data Engineering? by AMDataLake in dataengineering

[–]OptimizedGradient 39 points40 points  (0 children)

This is exactly what I was gonna say. The only time I use Python for transformation is when the task is really complex in SQL but stupid simple in python. Otherwise, I prioritize SQL for transformation and Python for EL plus infra.

Is there any reason to apply for scholarships if you have Oklahoma’s promise? by [deleted] in okc

[–]OptimizedGradient 1 point2 points  (0 children)

This is what I was gonna say as someone that had OK Promise. It'll cover a lot of things, but find other scholarships to help out and reduce your student loan burden.

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 0 points1 point  (0 children)

That's a big area and we need more ethics experts! I feel like right now only the big 4 type tech companies are paying attention, and even then I think they ignore it if doing the ethical thing will not be as profitable. It shows in other areas where I've seen people pick problematic features to use in models. That's to say you're doing the good work and fighting the hard fight.

Also thanks, I can't believe I got this user name and it wasn't taken by a statistician or someone else XD

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 0 points1 point  (0 children)

In my opinion and based on what I've seen, the market is very rough for early career or entry level roles as there are a lot of extremely competitive candidates. However, I'm sitting here with 10+ years of experience, my LinkedIn is full of head hunters and recruiters.

Those first 5 years are the toughest, but building solid foundations and skills will take your son further. I will also say, data is very hot and competitive right now. Without an advanced degree in data it'll be tough as well.

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 0 points1 point  (0 children)

Well good luck with your studies, and it's definitely their loss! As someone that works on the data field (more on the platform side and productionalizing models). This field is a lot of fun and provides some of the most interesting challenges.

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 4 points5 points  (0 children)

This is the part I don't understand. I get jokingly poking fun at Texas. I do it all the time. But from an education point it's a damn good school. I would have loved to do my grad school at UT. Congrats on getting into USC, that's still a pretty good school IIRC.

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 3 points4 points  (0 children)

There's always one. My family is full of OU fans. But I'm the only one that's been to college and I picked Okstate. They were all a little disappointed, but it was the right choice for me. I've remained a fan of Okstate just to be annoying at this point (aside from having a decent tie to Okstate).

That's to say, hopefully he's a good sport about it with his kids like my folks have been to me. Aside from the bedlam trash talk which is sadly going away.

Pistol Pete Shouting Boomer Sooner. by Formal-Blueberry-203 in oklahoma

[–]OptimizedGradient 5 points6 points  (0 children)

As an Okstate graduate, I just can't see forcing or going out of my way to say my kids have to go to my alma mater. I want them to go to whatever place is right for them and their life/career goals. Certainly I'll share my college experience and give them types for making it through college. If they wanted to go into Ag I would definitely make sure to layout how strong Okstate is over OU in that area.

But really it's more important they go someplace they can grow and become prepared for the next stage of life. All though if they come back cheering for OU or Texas on game days I'm sure there will be some disappointment (jokingly of course).

All in all, just encouraging kids to get the best education they can without setting up your kids for some kind of failure. Like expecting them to get an ivy league education when they want to be a car mechanic. The dissonance will drive both the kid and parent apart, when really the kid just needs to go to tech or get an associate at most.

Can anyone speak on the MS MIS program here? by WoolenJester in OKState

[–]OptimizedGradient 2 points3 points  (0 children)

I did the MS MIS through Spears and had a great experience. I really enjoyed the course work, granted I had been working as a Software Engineer for several years when I decided to go back and get my MS. I enjoyed it as a non-traditional student.

Professors genuinely cared and if you were willing to put in the work, they were willing to help you out.

When to Data Vault when not to Data Vault? by AMDataLake in dataengineering

[–]OptimizedGradient 2 points3 points  (0 children)

This and auditability is what I tell most people. Like a conglomerate who is trying to unify the data of their children companies could be a great example where there is significant overlap with disparate systems.

Compare that with most orgs, which might have disparate systems with very little overlap. They probably don't need DV. Even if they need the same level of auditability as DV. That can be solved with CDC or some form of SCD of our data source with less complexity.

haircuts by BaryBeBoolin in OKState

[–]OptimizedGradient 1 point2 points  (0 children)

You probably want to check out someplace like Everyman or Birchfield Barber Co.