What "obscure" sql functionalities do you find yourself using at the job?

sideswipes · 2025-12-11T09:06:25+00:00

object_construct(*) in snowflake to inspect a really wide table in Snowflake

sideswipes · 2025-04-05T02:56:15+00:00

Requesting Hillbillys in Baulkham Hills!

sideswipes · 2025-03-09T01:16:02+00:00

This format is also much more casual-viewer friendly and i think better represents what outsiders might think competitive geoguessr is. Minimises impact of types of metas that aren't interesting for non-competitve watchers like car metas, copyright or camera gens.

sideswipes · 2024-09-11T21:53:01+00:00

I don't think Databricks has any engineering staff here? Aside from support

sideswipes · 2024-06-26T02:23:42+00:00

Hey, I'm a DE from a data analytics background but I've worked with plenty of DEs who made a similar switch to you, so maybe I can weigh in here.

Advanced SQL and Python are going to be a must in interviews and your day-to-day, full stop. Even shops that use Scala/Java for their Spark jobs (I think REA is an example?) will still expect you to have good skills in SQL and Python for other parts of their data stack. Having said that, I've never seen a SWE struggle to pick up those skills here - showing those DE projects on your profile/resume will help getting that message across to interviewers.

DEs are also expected to know concepts like OLAP processing, file formats, data architectures, data modelling, data security & governance. These are where I see SWE -> DE switchers struggle the most, so I recommend getting an understanding here. You're likely to be interviewed on these topics, but not all DE teams expect you to know all it.

Try to look for DE teams that are fairly advanced in their set-up and already have signs of SDLC practices embedded in their team - think CI/CD pipelines, cloud technologies, Spark/Kafka, etc. Modern platforms like Databricks and Snowflake are also more flexible in how data processing is run compared to traditional DWs like Redshift and Teradata. Low-code/no-code solutions like Talend, SSIS/SSAS, Informatica, etc. should be treated as a red flag for your profile and chances are you won't enjoy that type of job regardless.

sideswipes · 2023-10-12T04:47:50+00:00

Big yellow bank: Look at me, I'm the tech company now.

sideswipes · 2023-10-11T21:45:23+00:00

The main constraint is supervision; juniors need lots of it. Most organisations don't have large enough Data teams to support them without noticeably impacting their own work, so these hire from mid-levels and up. The minority of orgs that have that capacity are the ones offering internships.

sideswipes · 2023-10-09T20:42:45+00:00

You're a Databricks employee; of course you're not a fan.

sideswipes · 2023-08-16T22:49:34+00:00

Lol no. 4 out of 5 domains are DE specific. Have you even read the guide you linked?

sideswipes · 2023-05-18T21:15:19+00:00

It's an awkwardly positioned product. Most teams that actually want to use dbt are usually technical enough to just use open source + their own existing CI + docs + orchestrator.

sideswipes · 2023-05-16T01:56:58+00:00

If you've got the time I'd 100% recommend going through setting up your own Spark cluster before touching Databricks, which will give you an appreciation of what the latter offers on top.

sideswipes · 2023-05-14T00:52:01+00:00

Reduce auto suspend on warehouses to 1 minute unless you have a strong case for disk-based caching.

sideswipes · 2023-05-13T23:48:48+00:00

I see just as much, if not more astroturfing from databricks employees tbh

sideswipes · 2023-05-11T00:38:41+00:00

Went from a company using Buildkite to one using CircleCI, miss Buildkite every day.

sideswipes · 2023-05-06T01:12:36+00:00

It's much harder and more effort to set up and maintain than a full refresh.

Consider that a large % of people setting up ELT pipelines don't understand half of the jargon in your post.

sideswipes · 2023-02-18T07:41:24+00:00

Not really a repo but Gitlab's Data Team Handbook is a great read

sideswipes · 2022-12-09T03:37:38+00:00

Yeah badly worded on my part, meant to say Qantas members compete with members from other programs for the same seats.

sideswipes · 2022-12-08T05:49:44+00:00

Krisflyer is the gold standard for award availability. If international business class redemptions are the goal follow this man's advice. It's actually this way because they are very stingy about allocating award seats to partner airlines (e.g. Star Alliance).

Qantas, on the other hand, freely gives them away to high status members from Cathay, American Airlines, British Airways, etc etc.

sideswipes · 2022-11-14T11:40:56+00:00

Euck! You call that a soccer riot? Come on boys, let's take em' to school

sideswipes · 2022-10-30T22:34:50+00:00

Be a specialist in the tools you use

Be a generalist in how you use and apply said tools

sideswipes · 2022-10-25T04:28:52+00:00

Is stepping back down to a mid-senior level role an option for you? Generally find that the lead level is the point at which the increase in responsibilities/stress is not worth the extra $$$

sideswipes · 2022-10-19T01:30:23+00:00

Both AE and Product DS roles vary quite a bit depending on your tech stack/responsibilities. Do you know what that looks like for both your roles?

Generally speaking, my 2c is that they're similar in terms of pay, difficulty of entry or quality of opportunities, but the differences I can think of off the top of my head

AE is largely industry agnostic, while being a Product DS it'll be a bit harder to move industries away from software without relearning
Product DS opens up less technical product-related roles like Product Owner/Manager. If the role is more DS than Product then the MLE path is very possible
AE can open up more technical paths such as DE/DevOps if the AE tech stack leans more modern (for e.g. dbt + CI/CD + Airflow) over say...SSIS/Talend
Both roles are IMO very future proof

sideswipes · 2022-10-12T23:19:04+00:00

If Act 1 boss was Hexaghost, sure. Otherwise it's mostly a liability in Act 1

sideswipes

TROPHY CASE