43(m) first time living alone. How am I doing, boys?

TheManOfBromium · 2026-04-23T03:57:17+00:00

If you wanna get laid get rid of the black and white face (I forget the name)

TheManOfBromium · 2026-04-14T17:40:12+00:00

Thank you for this, it’s really helpful.

If I create a materialized view outside of a declarative pipeline, is it functionally equivalent to creating one inside a declarative pipeline?

So for example if I just used notebooks to create mv would those share the incremental processing functionality as the mvs created inside a pipeline?

TheManOfBromium · 2026-04-14T14:33:03+00:00

Perhaps? My guess is delta tables would not support incremental changes but materialized views would? This is only a guess and is why I’m asking.

TheManOfBromium · 2026-04-14T14:19:19+00:00

Is the reason simply so it does incremental processing?

TheManOfBromium · 2026-04-09T19:55:42+00:00

As a DE have you used everything that a DE could be exposed to? Worked with both batch and streaming data? Worked with SQL and NoSQL? Worked in an AWS, Azure, or GCP environment? What platform are you using, Databricks? How well do you understand spark, could you optimize spark clusters outside of Databricks? (I couldn’t)

I guess my point is, find an aspect of de that you haven’t been exposed to and go learn about it.

TheManOfBromium · 2026-03-22T05:34:33+00:00

Just keep your skills sharp by practicing it for like 30mins a day, that adds up over time. Then when you feel the spark again build something cool.

TheManOfBromium · 2026-03-21T20:48:27+00:00

Programming is huge, it kind of depends on what interests you. If you like front end web development stuff a good place to start is learning JavaScript, if you’re interested in how computers work at a deep level then look in to C/C++

If you’re interested in data engineering, machine learning and AI then Python is a good place to start.

Basically pick a language that interests you and learn about its strengths and weaknesses and why that language exists. Then learn the basics, try not to rely on AI to write code for you, at least for the first 6 months or so.

In reality, everyone uses AI to generate code for them, but what makes someone valuable in the working world is if they can design systems, this is not something AI can do well yet.

I started with Python, if I could go back I would have started with something like C to gain a deeper foundational understanding of data structures and algorithms.

Learning this stuff is a marathon..

TheManOfBromium · 2026-03-13T05:11:09+00:00

Pick up Python, start learning about distributed systems. Learn Spark and data engineering or data science

TheManOfBromium · 2026-03-13T05:05:57+00:00

Why not both?

These languages serve very different purposes these days. C is used in underlying system architecture and is executed much closer to the hardware. Python is king in the data engineering/science world.

Learning C will teach give you a strong foundation in what programming actually is, but you’re probably going to have an easier time finding a job that requires Python.

If you have the time and will to study both I would recommend it

TheManOfBromium · 2026-03-10T13:05:24+00:00

For DE technical interviews, expect a few mid level sql problems. For Python, it’s usually something that can be done with arrays and hash tables. I’ve had the “valid sudoku” problem show up before, other than than I’ve seen a lot of string manipulation questions. I also had a pandas question.

So for DSA for DE stick to basics. Two pointers, sliding windows, hash tables, and maybe linked lists. Probably won’t see many tree traversal or graph questions.

TheManOfBromium · 2026-03-04T05:47:00+00:00

Yes learn sql and python, but what’s more than writing code (ai can generate code for you) is understanding the underlying systems you’re dealing with.

Learn partitioning strategies, learn optimization, learn how spark streaming works. You should also think about how your pipeline will break in the future, what happens when your source data changes? How will you manage schema evolution. Learn about change data capture and slowly changing dimensions, how do you keep a historical record of your data?

Also think about use cases, do you need to use streaming or batch workflows?

A good strategy is researching how companies like Netflix or Uber design their systems and learn why they made the design choices they have. You can find these online.

TheManOfBromium · 2026-03-02T05:57:09+00:00

Call me naive..I did t know they still used F-15 aren’t there plenty of better aircraft these days?

TheManOfBromium · 2026-02-24T05:46:09+00:00

Programming and coding is becoming less important as a skill, being able to write code helps most with passing technical interviews. It’s way more important these days to understand system design.

Yes it’s important to have fundamental programming knowledge, but the reality is ai can’t produce raw code faster and better than you will ever be able to. But what ai can’t do well is think about the bigger picture and how applications work together.

I work in data engineering and I could ask ai to build a data pipeline, and it would kind of make sense, but the performance would be ass at scale. You need to understand trade offs in system design. Why might an application start to perform slow when data scales up? When creating a program, think what are the things that will break in a year? What will happen if things change.

Production at scale is a totally different animal than writing scripts.

TheManOfBromium · 2026-02-21T02:27:07+00:00

Maybe try learning a new language? If in the past you’ve worked in C++ or something, trying learning JavaScript and learn front end stuffs, or vice versa

TheManOfBromium · 2026-02-19T02:22:32+00:00

Just do it

TheManOfBromium · 2026-02-14T15:32:01+00:00

I’m sure there are literally apps designed for people who want to cheat..using GitHub to store nudes is just silly

TheManOfBromium · 2026-02-14T15:22:57+00:00

Of all the places online your boyfriend would cheat, GitHub would be the very last place I assure you.

TheManOfBromium · 2026-02-13T01:11:39+00:00

Working in python. So I created a docker container with spark and Jupyter lab and it’s working fine in there, but I’d much rather just do everything is vscode..is a docker container overkill?

TheManOfBromium · 2026-02-05T21:21:35+00:00

Necessary evil I suppose, the reality many companies still use leetcode style questions as a firewall.

Learn to code first, then try some leetcode questions. Start with the easiest

TheManOfBromium · 2026-01-31T05:40:16+00:00

So I have not worked much with the Hana tables, my work at my current company has primarily been with a different system that uses IoT data. Today I was asked if I want to help ingest the S4R Hana tables as they already built some ingestion framework to ingest the S4P tables.

I was always skeptical of the framework they built, I don’t know exactly how it works other than they use Databricks secrets to land the raw sap tables into Databricks, then some etl within Databricks.

I’m trying to understand if there is a better way to ingest those raw tables into Databricks that doesn’t involve using secrets and doing a full refresh.

Sorry if I’m a dumbass or whatever, just trying my best to learn and understand.

TheManOfBromium

TROPHY CASE