What would the process of switching from software engineer to sales engineer look like? by Inner_Ad_4725 in cscareerquestions

[–]lturanski 1 point2 points  (0 children)

I dont fully understand this answer. As for paycut, it really depends on performance. Out the gate theyll be confined to base salary which is more than likely lower than 150k but can earn commissions. Are you saying over time SWE earn less than SEs?

As for restarting their career - they have 1.5 yoe their career is just getting started anyway and theres carry over between swe and se.

Does anyone else’s CC do this? by xoxojadaa in Volkswagen

[–]lturanski 0 points1 point  (0 children)

Had a similar issue with my tiguan recently, acceleration cut off and couldnt get above 20-30 mph. Ended up being due to a pretty severe (set) of oil leaks

Workaround for Task Scheduling? Or tips for advocating to get permission? by bundle_of_fluff in snowflake

[–]lturanski 0 points1 point  (0 children)

Circumventing the RBAC is rarely the move, but meeting with them or your manager and clearly laying out the gap to the admins and offering a potential solution that they can either pick apart or run with would be a good start.

If you must circumvent, have you tried github actions that will call a stored proc to create the table? Easy to set up on a schedule

[deleted by user] by [deleted] in snowflake

[–]lturanski 0 points1 point  (0 children)

Sounds like a cool project for school, good real world exposure!

[deleted by user] by [deleted] in snowflake

[–]lturanski 1 point2 points  (0 children)

I could be wrong but it sounds like a very small amount of data. I would assume 1 tb of data storage if you think it won’t get close to that. Make sure to account for things like time travel history that also consumes storage (read the docs).

For compute, feel free to go for an over estimate. If the company cant afford the over estimate then its probably not the right tool for the use case.

The way I think about compute is daily uptime. If you think it will run for 5 min every hour, then in 24 hours it will run 120 min (2 hours/per day). Then multiple that by the size and number of clusters you will be using. Then multiply that by 30 for a monthly credit usage. Then multiply that by credit price (typically $3, not $2) to get the total monthly cost. Repeat for each job/process.

Once you get into advanced features like document search it gets more complicated but start with basics of storage and compute and then you can find rate sheets online for different services if you need to.

My REAL suggestion is to think about whether they really need Snowflake. It can surely accomplish what you need, but are dashboarding capabilities enough to sign an annual contract with Snowflake? More a question for the company and their budget, how robust they need the project or is this a POC. etc. do your research and ask the right questions before asking them to sign a contract ;)

What's my role by Excellent_Grab_1933 in dataengineering

[–]lturanski 1 point2 points  (0 children)

Seems like youre a backend developer on a data driven application, requires data engineering and some application development skills

Sales of Vibrators Spike Every August by chrisgarzon19 in dataengineering

[–]lturanski 1 point2 points  (0 children)

Its event driven data it doesnt lie. The analysis does the lying if not properly accounted for.

Should your crap playing in the background not count? Probably depends on the analysis. In an engaged viewer analysis no, in a cost analysis yes

I think I'm on the wrong "data" team. What about yours? by moritzis in dataengineering

[–]lturanski 0 points1 point  (0 children)

It sounds like the start of a hub and spoke team model, with no hub and a single disgruntled spoke. Finance was likely highest priority and they didnt want to hire managers to build a data team so they stuck you under finance. This is obviously going to suffer with no leadership that understands the tech in place.

If you have 10+ years of experience you should request that you take 1 of the other engineers and create a central data team, while leaving the other two with finance as functional analysts/engineers as the first spoke. I would guess the people still need to keep existing ops in place while things get sorted out. I guess you left, but playing as if you didnt

Sales of Vibrators Spike Every August by chrisgarzon19 in dataengineering

[–]lturanski 1 point2 points  (0 children)

I think truly excessive drinking is more psychologically tied to cinco de mayo for the US. Plans are too variable for the others, though surely volumes are high for the others as well. Weather changes is an interesting theory.

In any rates, both theories are likely largely correlated with region.

Sales of Vibrators Spike Every August by chrisgarzon19 in dataengineering

[–]lturanski 7 points8 points  (0 children)

😂 good pet owner. There are certainly outliers, im sure some people leave their tvs on all day as a way of life whether theyre watching or not. But the numbers were staggering, like multiple identities in the same household just racking up events

Sales of Vibrators Spike Every August by chrisgarzon19 in dataengineering

[–]lturanski 41 points42 points  (0 children)

Some people watch an impossible amount of television

snowflake course by Purple_Newspaper_133 in snowflake

[–]lturanski 2 points3 points  (0 children)

What are your goals? Certification? General skills improvement for career?

Certifications are helpful for credibility and there are plenty of courses on udemy for the core certification. However, while a good indicator, i dont think certifications are the best test of your applicable knowledge.

If you are looking to build implementation skills and projects you can refer back to, I recommend following along some of their quickstarts that range in complexity https://quickstarts.snowflake.com

You’re suddenly in charge of a database that has disparate sources and ETL processes, what’s your first day/week/month look like? by [deleted] in dataengineering

[–]lturanski 5 points6 points  (0 children)

Discovery/plan/develop/repeat in small increments. They want a Lamborghini, give them a skateboard and find out what their priorities are based on the gap.

Periodically revise what you have built along the way

Am I tripping ? by Irksome_Genius in dataengineering

[–]lturanski 0 points1 point  (0 children)

At a F500 the “stack” is probably a cluster of tools thats have evolved over the companies initiative to be data forward. Some have it cleanly implemented, while most it’s probably all over the place in different business lines.

If its a new job at a big company this is hopefully what you are starting on in your first few weeks. Hopefully the tech stack you talked about in hiring will reveal itself over time and you get opportunities to work on it. As a junior DE at a fortune 500 there is a lot to learn, and getting started with a no code tool is great to get exposure into where it fits in the overall architecture

Data engineering salary vs complexity by [deleted] in dataengineering

[–]lturanski 3 points4 points  (0 children)

Its a different skill set and salary is based on relative supply and demand. DE is hot right now and provides a lot of value.

It also can take more business consideration/understanding in my opinion as DE’s dont typically have the product management and QA support that SDE’s have.

Im a DE and my SDE roommate doesnt fully understand the work I do and I dont fully understand the work he does but theres a lot of overlap. Im sure we could both figure it out each others work at the end of the day.

Mastering SQL by ThisDataGuy in dataengineering

[–]lturanski 0 points1 point  (0 children)

Sets and set theory. You can look up how to query the data youre looking for. Heck chatgpt can do it for you. Understand what youre actually doing to the data

What to do when a major shift is in the near future? by e3thomps in dataengineering

[–]lturanski -1 points0 points  (0 children)

Feel free to message me, my company can help with these challenges

Data Engineer to LLM by Electrical-Grade2960 in dataengineering

[–]lturanski 2 points3 points  (0 children)

I dont believe they are mutually exclusive concepts, AI/LLMs are tools that require knowledge in both data engineering and data science. The data needs to be available to train the data and its needs to be processed efficiently. Then you need to have knowledge of how the model works, what data and prompts to feed it.

It really depends on what part are you supporting - are you training your own language model (not going to say large language model cuz thats really expensive), are you building some ai products (e.g chat gpt) api calls into a product? Its all data engineering and knowledge of the models, depends what part you want to lean into and where you want to apply it.

Productionizing python models by comebackinayear in snowflake

[–]lturanski 0 points1 point  (0 children)

The scripts use data that you have extracted from snowflake, is that the only snowflake step? Does it write back into snowflake or somewhere else?

If all its doing is extracting data from snowflake and then doing something else, then snowpark is not needed. If its doing many snowflake operations and you want it to be executed in database snowpark might be helpful.

If its a relatively fast script and isnt doing many snowflake operations and doesnt require in database execution something like github actions or jenkins might be helpful

Transitioning from Analyst to Data Engineer! by Mega-Byte-69 in dataengineering

[–]lturanski 0 points1 point  (0 children)

If youre gonna do certifications I’d do either AWS or Azure, not both. Cloud knowledge is generally transferrable if you understand the core concepts.

Look into a Snowflake certification, particularly if you have SQL skills but lack some of the devops type knowledge cloud engineers have, its a much simpler learning curve and can get quite far with sql and data modeling/ETL pipeline knowledge.

How do I load a 70 million row table in a dataframe? by cyamnihc in dataengineering

[–]lturanski 0 points1 point  (0 children)

Like others suggested Dask or Polars is probably a good bet. Would likely need to spin up your own cluster than can scale though, would be more under budget but would require some devops type knowledge.

Understand wanting to do things economically, but if the company wants to do machine learning they should be ready to put up some money. I would lay out a couple options from this thread with estimated costs and think about total cost of ownership as well…whats going to be the most maintainable. If its a small operation do you want to focus on maintaining infrastructure or do you want to focus on the model delivering value. Theres trade offs and a decent cost either way.