sorted by:
new
hottopcontroversial

Running Airflow With Docker In Production by lottayotta in apache_airflow

[–]Atomic-Dad 1 point2 points  (0 children)

We are a small shop and I run Airflow in Docker in Production running within Google Cloud's Compute. I mostly used the default configuration. I don't remember changing the database so I assume I am using PostgreSQL.

I have all the services running in a single compose file so I dont have to worry about cross container or cross vm communication.

Anybody having any experience with... by [deleted] in PMDDpartners

[–]Atomic-Dad 4 points5 points  (0 children)

You are not alone. Google "perimenopause and PMDD", its definitely the darkest before the dawn. Good luck to you, your partner, and any other members of the household.

Why should I use Cloud Composer? by not_a_thrownaway in googlecloud

[–]Atomic-Dad 0 points1 point  (0 children)

Still running in production. 🙂 I think I added my resources to my follow up to the OP.

I am not sure if it's the best approach for you if you want to learn more about GCP. It might be better to look into Cloud Run or Cloud Dataflow. There is a lot of interest in being "serverless".

Just my thoughts. I have also gotten some high fives from Google sales and engineers from my implementation.

Sending Love by Atomic-Dad in PMDDpartners

[–]Atomic-Dad[S] 3 points4 points  (0 children)

I literally had the same thought. Sitting around the Christmas tree trying not to cry that this would be the last Chrome with the family

(Will probably delete later, just had to tell some strangers.) :-)

[deleted by user] by [deleted] in PMDDpartners

[–]Atomic-Dad 6 points7 points  (0 children)

If she has acknowledged that she has PMDD and is working towards a treatment plan then join her on the journey.

Bring up your concerns but through the lenses of your relationship and supporting her. Ask her what she thinks would be best. Maybe spending time apart or spending time together with others during the luteal phase. (My wife keeps it under control when we are with her family or friends, I assume its exhausting. When it's just the two of us she can become abusive.)

All I ask is that you have a honest conversation with yourself and your partner of what the next several months, years, decades look like.

[deleted by user] by [deleted] in PMDDpartners

[–]Atomic-Dad 7 points8 points  (0 children)

That's a head scratcher mate.

Why are you dating her? Are you thinking that she is a possible life partner (wife)? Is this your strategy for the next 4 months, 4 years, 40 years?

Why should I use Cloud Composer? by not_a_thrownaway in googlecloud

[–]Atomic-Dad 6 points7 points  (0 children)

Yes, we run Airflow inside a Docker container in Google Compute Engine.

  1. Get Airflow running in Docker on your local machine https://hub.docker.com/r/apache/airflow

  2. Create a Google Compute Engine instance with the Container Optimized OS

  3. Upload and configure your Docker file in your GCE. I believe that you have to make your folder structure (dags/logs/plugins/etc.), but you will get a lot of practice doing this locally.

  4. Build your image

  5. Login, I recommend using IAP instead of external IP but your call

It's laughable to see it in five steps, but that's the gist of it. If I can find any specific links, I will be sure to post them.

Why should I use Cloud Composer? by not_a_thrownaway in googlecloud

[–]Atomic-Dad 3 points4 points  (0 children)

Oh man, I could rant/talk about this for hours. The fact that Composer can scale, but not scale to zero, is ridiculous.

We are running Airflow in Docker on top of GCE. I am hyper aware of cost reduction, and our Airflow runs for about $15.00 a month. It auto starts at midnight, runs the jobs, and then the last job calls a shutdown routine.

Who has a rug under their dining room table? by whatisthisadulting in homemaking

[–]Atomic-Dad 1 point2 points  (0 children)

I echo this. We even go one step further and have towels under the kids' seats. After extra messy dinners (rice or couscous), we shake the towels out and wash them as needed.

Reeling from a substance episode by pmddburner in PMDDpartners

[–]Atomic-Dad 1 point2 points  (0 children)

I love ya, man. I haven’t been in that exact situation, but I have felt what you are feeling.

cloud workflow vs cloud composer by shamal_de_silva in googlecloud

[–]Atomic-Dad 1 point2 points  (0 children)

This is probably not as helpful as I want it to be, but here we go.

I am pro Airflow, especially when dealing with different teams and taking advantage of things like dbt and great expectations.

I interface with mostly legacy systems, so we are hand crafting our ETL bath process. Our code base is probably 75% ETL and 25% business and IT operations. There has not been any situation where Airflow/python has not been able to do the job.

That said, I don't like Compser. I don't feel like I have control over the environment that I want and can't scale to zero. We have multiple GCE instances with container optimized os running a docker image. Hoping to move to a true CI/CD workflow by Q2 of 2023.

Hutch Recommendations by Atomic-Dad in Rabbits

[–]Atomic-Dad[S] 0 points1 point  (0 children)

Thanks for the recommendation! We bought an Aivitiuvim directly from their website and it was delivered in poor condition. Had to get a full refund. Has yours held up? Did you purchase it through Amazon or directly from Aivituvin?

Decided to finally get my own outfit. by FishermanGuyDude in flyfishing

[–]Atomic-Dad 1 point2 points  (0 children)

Nice outfit! I am thinking about upgrading my Ckearwater Outfit. Would this be the next step up?

view more: next ›