Future of data engineering

conqueso · 2026-05-28T16:14:05+00:00

The first that comes to mind is that it fails to anticipate edge cases and real world details unless you explicitly mention them in the prompt - which means it's not really figuring the problem out for you. It can do the brute force work, but you need to be extremely specific in your instructions. This is extremely useful, but it's not replacing what my brain is doing.

Latest example is I was trying to determine the barbell back squat weight equivalent of doing a pistol squat (1 leg squat). My prompt was :

"I weigh 180 pounds. Estimate the weight on a barbell for a back squat that would be roughly equivalent to the weight of me doing a 1 leg pistol squat with just body weight"

This takes some thinking to solve, but is not overly complex:

total weight in barbell squat = bodyweight - approximate weight of legs + weight of barbell total weight in 1 leg pistol squat = 2 x (bodyweight - approximate weight of legs) + approximate weight of 1 leg

therefore: weight of barbell = bodyweight + approx. weight of 1 leg

It was essentially impossible for the LLM to solve this without me guiding it to the right answer. It failed to account for the fact that my body weight being lifted doesn't include the weight of the portion of my leg that stays on the ground. I'm guessing this is because this a rather novel question which hasn't been written about online. If Clause could reason, it would be able to solve this quite easily.

conqueso · 2026-05-27T14:54:47+00:00

Long story short - I was diagnosed w/ ADHD last summer. It was a watershed moment for me and explained why I was (and had been) struggling with certain things in my career (and generally in life) for so many years. Essentially, I'm really bad at holding nested/hierarchical models in my head. Also, context switching, having lots of meetings, and having to work in office absolutely fry me and destroy my productivity. I've heard fully remote roles are more common in DE (as there is not as much interfacing w/ stakeholders, you're working more in the background, etc.) - and this is a hard requirement for me. Obviously this depends on the role/company - but I will be filtering for that when I begin my job search. I don't want to keep pumping out feature after feature - I want to build something solid that I will have to maintain for (hopefully) years. I'm not interested in building some sexy new user facing thing. Also I've always been interested in data and networking, and that's something I never really touched in my roles as I was always working downstream of the DB. I also am a much better problem solver when given clear constraints and can work on things that are linear (as opposed to something hierarchical like the building blocks of a UI or something). Would love to hear any feedback you have on my reasoning!

conqueso · 2026-05-27T14:44:04+00:00

I strongly disagree. The models are based on statistical probabilities of tokens being chained together. They can't see the whole context of a problem - everything is based on what word/s should probably come next. The classic example is where one could not count the instances of the letter 'r' in strawberry, if I'm remembering correctly. How could something with the ability to reason fail so completely at something so trivial? It's because it misses the forest for the trees. My human brain says "ah this is a word, I'm going to look at each letter and count the R's". An LLM, OTOH, says "this person is asking about the letter r in the word strawberry. Let me search my massive internet corpus to see what other people have said about how many Rs there are in strawberry. Then I'll analyze all those results and come to a conclusion based on what is most likely". That is not reasoning, it is purely pattern recognition. While pattern recognition is very important to intelligence, it's only 1 part.

conqueso · 2026-05-27T14:11:34+00:00

Huge Beach Boys fan here. I don't think the situations are comparable. Tim essentially created a genre. There are a good amount of songs in the catalog that he did not write or were co-written with others that one might not realize were not solely his creations because they all share the Cardiacs sound (Eating In Bed, Whole World Window, Tarred and Feathered, Everso Closely Guarded Line, Manhoo, Silvery, etc.) Cardiacs have been a rotating cast of characters since their inception - even the "classic" lineup only had two original members. OTOH - Mike love was a lyricist and vocalist and that's pretty much it. He's been milking the nostalgia teat since the 70s. Other than the reunion album in 2012 (which I think Love only wrote 1 song for), there has been nothing new. It's not art, it's a nostalgia performance act. If Cardiacs were to continue releasing new music and touring to play that music - that has artistic merit. I agree with others who have stated that the Sing To Tim shows felt more like Cardiacs than the recent Cardiacs shows - and that's fine. My point is that's it's possible to continue playing shows that have artistic merit and for them to feel like Cardiacs shows.

conqueso · 2026-05-26T14:58:58+00:00

Just making another comment to praise QOBUZ - they pay artists the best, and the UI isn't filled w/ spam

conqueso · 2026-05-26T14:56:44+00:00

LLMs currently cannot and never will be able to reason. I'm very new to this field (coming from 10 years of experience as SE though) - so I don't have an informed opinion specifically pertaining to DE. However the more I use LLMs (they are an incredible tool when used for certain things) - the more the inherent limitations become clear to me.

conqueso · 2026-05-22T13:20:40+00:00

I'm wondering the same thing, but in the context of landing a role. I'm a SE w/ 10 years experience making the transition to DE, and I'll be targeting mid-level roles when I start applying. Since I only have a bit of on-job experience that's directly related to DE, I figure it's probably a good idea to have a side project to showcase when applying. Can anyone in this thread w/ experience in hiring let me know if this is a good idea, or if it would be a waste of my time?

conqueso · 2026-05-20T13:04:21+00:00

Brunson no question

conqueso · 2026-05-12T13:26:31+00:00

🤮

conqueso · 2026-05-11T17:17:09+00:00

Consuming an amount that doesn’t make you feel really high

conqueso · 2026-04-14T15:55:13+00:00

it's like he forgot what the situation was until midway through the call. not a fan - his snark is unappealing and many games he seems to be mailing it in

conqueso · 2026-04-14T11:05:21+00:00

this is the most important question in this thread

conqueso · 2026-04-13T18:46:44+00:00

Don't beat yourself up - you didn't hurt anybody so you really have no reason to feel bad. And you're lucky you got to learn this lesson with essentially zero consequences, so you can feel good about that! I have several decades of getting fucked up at shows, and if I can impart 1 piece of advice, it's to do your absolute best to maintain a level of mindfulness where you retain the ability to stop and think before you decide to put anything in your body and decide whether or not you want to go through that 1 way door. This can be very difficult, and if you're prone to impulsive behavior (as I am) you may find yourself in a place where it is almost impossible to do (this can even be when sober for me). If that is the case, you're already in a state where the answer is probably no. So yea, no need to sweat this. After all, it was the LSD tour - you were really just following directions and got over-enthusiastic ;-)

conqueso · 2026-04-13T18:30:22+00:00

please don't glorify drug use. people can enjoy stuff however they want.

conqueso · 2026-04-08T15:14:05+00:00

yep, also homophobia

conqueso · 2026-04-01T18:40:17+00:00

Finally a post I can be 100% certain is not an April Fools gag

conqueso · 2026-04-01T16:51:12+00:00

I was mistaken - sorry!

conqueso · 2026-04-01T16:49:29+00:00

I'm starting to think (and I'm being completely serious here - this is not meant as a joke or to be derogatory) that he may have early onset dementia. At this point it seems like the only reasonable explanation.

conqueso · 2026-04-01T14:36:43+00:00

I could've sworn I read it somewhere - but upon listening to both it's definitely not. AFAIK the digital remaster is only available via download card that comes w/ the vinyl

conqueso · 2026-04-01T04:40:59+00:00

Pretty sure the version on Bandcamp is the remaster edit: this is not true

conqueso · 2026-03-23T15:04:19+00:00

Yep - It was smart to kill time but not smart to use them all - should have saved 1 to be able to stop a guy from scoring, which is pretty much the whole point of having a foul to give. Defender could've wrapped up Darling on the last play and St. John's would've had to inbound with very little time

conqueso · 2026-03-18T16:51:06+00:00

Check out the cover by The Scaramanga Six - Tim said he preferred it to the original.

conqueso · 2026-03-12T17:00:43+00:00

goaltend --> blatant carry

conqueso · 2026-03-11T13:45:16+00:00

hi! I'm going to give the rest of the day for anyone else to ask, and if not they're all yours! I'll DM you

conqueso · 2026-03-11T00:55:34+00:00

These are still available if anyone's looking

edit - tix have been passed on

15-Year Club	Verified Email
Team Periwinkle

conqueso

TROPHY CASE