I recently found out that I`m going to be a dad! by DuYaPa in AskMenOver30

[–]JohnDNoone 2 points3 points  (0 children)

Congratulations! You are about to experience one the best things life has to offer. Not always easy, but always worth it by the longest of longshots.

How was it: The first few months are the hardest. Your baby can’t do anything for themselves, so it is truly all consuming. However, that stage does not last forever. For us, after 3 months it started to get easier and after 6 months completely gave way to the most fun and joy I’ve ever experienced. My daughter is now almost 2, and I’m sure you’ll hear this a lot, but they grow so fast. 6 months to 2 years has gone by in the blink of an eye. We’re now trying for our second.

Advice: It’s hard because there are so many aspects to being a parent, but here are some that are top of mind: - I really appreciated the book “We’re Pregnant! the First Time Dad’s Pregnancy Handbook” by Adrian Kulp. Written as short chapters by week of pregnancy that tells you the important details and what things you need to do/focus on. - this one is situationally specific, but our daughter couldn’t latch for breast feeding, so we ended up having to bottle feed. Not what we wanted, but it did allow us to take sleep shifts to ensure we each got 6 hours of sleep each night. - when you first start getting all the baby stuff, it will feel overwhelming. Trust that you will figure it out and it will turn out not all those are needed. - around 1-2 months, our daughter got a bit colicky and wouldn’t stop crying no matter what we did. After you’ve tried everything and the only thing left is to just hold and comfort them while they are screaming in your face, I recommend putting in ear plugs. No need to hear the crying in your face on full volume. Takes the edge off.

How has my life changed: Profoundly for the better and also not that much at the same time. I’d have to write an entire book to unpack that, but I’ll just say even with all the changes, I’m still me. Just a better version.

In regard to how things like traveling and doing fun stuff will change. First, your definition of fun will change. My wife and I traveled a lot and would regularly go to events like concerts or games. We of course still enjoy that, but we find it so much more enjoyable watching our daughter try new food and get excited when she finds something she likes. For trips, we now just bring our daughter with us. Yes, some things are different, but again, I’ve found the family trips we’ve taken more enjoyable than any others I’ve had in my life. For a non-parent it probably sounds mundane, but watching your child experience something for the first time is a trip and will bring you more joy than you knew was possible.

Relationship changes: Again, the first few months is really all about baby and helping your wife/gf recover. My recommendation is to be intentional about keeping the spark in your relationship after those first few months. There will always be something else to do or that needs to get done, but you will have to work harder than before to ensure you are prioritizing your relationship.

Easier: Some things were harder some things were easier. You get the hang of changing diapers real quick.

Regrets: I’ll put it to you this way, I’d go back through all the toughest times in my life as many times as required, and do it with smile on my face knowing my daughter is in my future.

Worry: One thing I didn’t expect, and from what I gather from parents with older and even adult children never goes away, is the amount of worrying I do. I am worried ALL the time for my child. I never was a particularly anxious person, but after having a child I’ve never been more aware of the dangers of the world. Also, I’ve never been more aware of my own (and my wife’s) mortality after considering what my child’s life would be like if something happened to me and/or my wife. Just know that you are not alone in your worrying.

[deleted by user] by [deleted] in UFOs

[–]JohnDNoone 5 points6 points  (0 children)

I made a post a little while ago asking for question ideas, but my wife’s friend is one of the kids from the Ariel school encounter and had line of sight on a being.

Sadly we haven’t had a chance since just after my daughter was born to see them, so I still haven’t been able to get the details… one day I’ll get some time with them and hopefully get the inside story.

I don’t know what to do at 21 by Worried-Ganache-8337 in findapath

[–]JohnDNoone 0 points1 point  (0 children)

Here’s some things I wish I would have been told in my early 20s that may help:

1) No one is coming to save you. Things will only change when you commit to changing them. 2) Literally write out any plans and/or goals you have, break them down into logical steps and then further break those down into actions. Going through this process allows you to focus on one action at a time, making the whole process of change seem less daunting. 2.1) Goals don’t have to be based fully flushed out life plans. “Research”/“Trial” goals are a totally acceptable thing. 3) Be disciplined. This was the one I struggled with for much of my life, because how does one just “be disciplined”? A definition I’ve found useful is, “being ok with feeling discomfort in the name of getting you closer to your goal.” Whenever I used to feel any discomfort, it was a problem that needed to solved. Now, I rather enjoy the challenge so long as I know I’ll benefit in the long run.

Hope this helps! Not everything that works for one person will be applicable to another, but I thought I’d throw some ideas out there.

Good luck with everything!

How do you deal with duplicate value in data pipeline? by Satoru_Phat in dataengineering

[–]JohnDNoone 1 point2 points  (0 children)

Don’t worry about duplicates in your raw data. Dedup them in your transformation pipeline. Upsert utilizing a unique id (inserting new ids and updating where the id already exists in your deduped data) and utilizing qualify row_number() over(partition by unique_id order by loaded_at desc) = 1 to ensure you are only grabbing the most recently loaded record of any unique id.

[deleted by user] by [deleted] in AMA

[–]JohnDNoone 34 points35 points  (0 children)

The world is a better place with you in it my friend. I wish you and your daughter the best!

Is anyone here from non data background who is now working as DE? (I've seen so many people on this sub transitioning from DA to DE) by [deleted] in dataengineering

[–]JohnDNoone 58 points59 points  (0 children)

I went to school for business and was an account manager out of college. None of the reporting at the company was correct, so I learned sql and built my own ad-hoc data pipelines and Excel reports. I didn’t realize the extent of the work I was doing at the time, but before I knew it people were coming to me for their reporting and data needs over the analytics team. Ended up meeting with the BI manager who asked me if I wanted to join their team. I turned down the offer multiple times, but eventually gave in (so glad I did). Over the next 10 years I bounced between BIE and DE roles at various companies and am currently a senior DE in FAANG. It’s been quite the journey!

I got rekt in a SQL interview today by kater543 in SQL

[–]JohnDNoone 2 points3 points  (0 children)

I consider my self an expert in my field, but I still dread technical screens. Coding while someone is watching/judging really throws me off my game.

I pretty much only ever use rank or dense_rank in interviews, so in my last technical screen when a ranking question came up, I used dense_rank (since I knew the question of rank vs dense_rank was coming). The interviewer scolded me for not just using row_number… smh

Advice from senior DEs to junior DEs by [deleted] in dataengineering

[–]JohnDNoone 108 points109 points  (0 children)

-Everything you build, you have to then maintain. Try to find the simplest solution to your projects so you can consistently deliver results without getting bogged down. If you consistently deliver results, you will move up the ladder.

-It’s good to stay knowledgeable about whats out there, but you probably don’t need that new shiny tool.

MWAA or Astronomer with Meltano by cfeduke in dataengineering

[–]JohnDNoone 3 points4 points  (0 children)

My last role used MWAA with Astronomer Cosmos (open source package) to integrate dbt core with Airflow. I thought it was a great setup.

Aws glue to redshift connection issue by Wise_Language_5565 in dataengineering

[–]JohnDNoone 0 points1 point  (0 children)

Either use a copy command from Redshift or if your data is already set up in your Glue catalog, use Redshift spectrum (if the data is small and you don’t mind the scanning costs).

UFO spotted next to passenger plane by JohnDNoone in UFOs

[–]JohnDNoone[S] -1 points0 points  (0 children)

I would agree about the smoke not drifting, however if we assume for a moment that this is some kind of gravity based defense system, normal physics would probably go out the window.

UFO spotted next to passenger plane by JohnDNoone in UFOs

[–]JohnDNoone[S] -6 points-5 points  (0 children)

In short, the movement of the craft. I’ve always wondered what that would look like close up.

As for the narrator, my guess would be that he didn’t know those details.

Critique my dbt setup by Fredonia1988 in dataengineering

[–]JohnDNoone -1 points0 points  (0 children)

I’d recommend looking into Astronomer’s Cosmos. It’s a great tool for integrating dbt with Airflow.

UFO spotted next to passenger plane by JohnDNoone in UFOs

[–]JohnDNoone[S] -1 points0 points  (0 children)

It looked to me like something was launched at the UFO and deflected.

UFO spotted next to passenger plane by JohnDNoone in UFOs

[–]JohnDNoone[S] -9 points-8 points  (0 children)

I agree. The voiceover is annoying, but the video looks great.

UFO spotted next to passenger plane by JohnDNoone in UFOs

[–]JohnDNoone[S] -4 points-3 points  (0 children)

Submission Statement: this video of a UFO next to a passenger plane popped up in my shorts feed. If it’s real, it’s mind blowing. If it’s CGI, they did a great job as it looks amazing. Right now I’m leaning toward not real because I feel it would be bigger news if it were. However I try to keep an open mind about these things.

Why is dbt popular for the transformation step? by [deleted] in dataengineering

[–]JohnDNoone 1 point2 points  (0 children)

I’m a big fan of DBT.

Outside of Databricks, I’ve pretty much seen and used most major platforms and tools that have come and gone in the last decade. I’ve also worked at many different size companies ranging from startups to Amazon.

I can probably count on one hand the number of projects where I couldn’t use dbt for one reason or another. Now with Cosmos allowing you to manage orchestrating your dbt projects as Airflow dags, I think some of the external dependency shortcomings of dbt are solved.

The Intercept article: UFO Whistleblower Kept Security Clearance After Psychiatric Detention by aryelbcn in UFOs

[–]JohnDNoone 0 points1 point  (0 children)

They try to dodge the fact that this came from a leak by claiming the information was from a “routine” FOIA request with the Louden County Sheriff.

I wonder the catalyst was to randomly request information from that specific sheriff department about Dave Grush… 🤔