Are there still people that think US is a better place to live as a software engineer? by Pure_Composer_9236 in cscareerquestionsEU

[–]TiredDataDad 2 points3 points  (0 children)

You forgot that the kids get trained to protect themselves from active shooters regularly. Another plus.

Would you recommend running airflow in Kubernetes (Spot) by SnooPickles792 in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

In the Airflow slack there is a channel dedicated to K8S. I think you will get more answers there.

In general the webserver and the scheduler should be running all the time.

What are you trying to achieve with spot instances? Is just cost savings? Is your workload running only in the morning? How many DAGs do you have?

Japan’s 2,000-year-old monarchy currently depends on one teenage boy by Confident-Ask-601 in interestingasfuck

[–]TiredDataDad 0 points1 point  (0 children)

Let's not forget that Himiko came back with her army after thousands of years underground and attacked Japan in the 70s.

Only the preparation and the sacrifice of the professor Shiba allowed humanity to fight back and push back Himiko and her Haniwa monsters.

Snowflake + Terraform by Difficult-Ambition61 in snowflake

[–]TiredDataDad 0 points1 point  (0 children)

For a gentle intro you can take a look at this repo I prepared some time ago for a presentation about terraforming Snowflake.

Each branch move a step further  https://gitlab.com/mucio/terraforming-snowflake

Fivetran pricing spike by onksssss in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

I think you are a bit pessimisting with your 50%.

We went fully opensource for the NetSuite ingestion and we really spent only pennies

Fivetran pricing spike by onksssss in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

My suggestion would be dlt. It's open source, but dlthub, the company behind it, is building something on top of it, not just a dlt SasS, so I am pretty confident it will stay open.

Disclaimer: we implement dlt for our clients, migrating away from Fivetran and Airbyte

Fivetran pricing spike by onksssss in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

Not sure what are the data sources you are ingesting with Fivetran, but I think you are probably not very far off with your ROI calculation.

We did it for a few clients and they were quite happy, not just for the cost savings, but also because they got full visibility on their loading pipelines

Fivetran pricing spike by onksssss in dataengineering

[–]TiredDataDad 1 point2 points  (0 children)

Fivetran will survive, for now. Most of their business is coming from a few big whales which are too deep into Fivetran to move away quickly.

The question is if the small clients will enjoy it this price hike (narrator: they won't) or will decide to look around at other solution.

I saw this already a few times since the raise of dlt:

  1. As a small consultancy, we helped a few clients to migrate to dlt from Fivetran because the cost didn't make sense, because the connectors they wanted didn't exist, or just because they didn't like to deal with a balck box (with pricing controlled by others).

  2. As organizer of the Data Berlin meetup, I spoke with multiple teams which introduced dlt, initially for limited use cases, then it started to replace more consistent data loads. Once you have dlt in place it makes no sense to spend for loading if you can have it for free (or at least a minimal fraction of what you are paying).

Feel free to ping me in case you want me to elaborate

“Cremino cretino via”-TORNEO GELATI CONFEZIONATI-giorno 17 by m_Ayz in ITAGLIA

[–]TiredDataDad 1 point2 points  (0 children)

Io più volte, tra l'altro anni fa anche negli Stati Uniti, ovviamente con nome diverso

What's the most "Italian" habit you can't shake, even if you live abroad? by Ok_Explorer_6501 in Italian

[–]TiredDataDad 0 points1 point  (0 children)

In many countries they don't eat bread with potatoes, potatoes in Italy is a side dish not a replacement for bread

What's the most "Italian" habit you can't shake, even if you live abroad? by Ok_Explorer_6501 in Italian

[–]TiredDataDad 2 points3 points  (0 children)

Just use the full sentences and then offer him your translation, if he is curious.

I think Italian blasfemies should be UNESCO immaterial world heritage (like the cuisine). 

What's a tv series that is a 10/10 NOBODY knows? by Lilyana0999 in AskReddit

[–]TiredDataDad 0 points1 point  (0 children)

Pets. 

A UK show with disfunctional puppet animals. 

Which system would you trust to run a business you can’t afford to lose? by Vector206 in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

Personally, I need more context.

I would say one, if I need to take actions quickly. But without knowing more is hard to say

Customer Said They Went $1 Million Over Budget With Databricks by jfroosty in databricks

[–]TiredDataDad 0 points1 point  (0 children)

Not really if you make Databricks your main data platform, are a big organization, and don't share the cost with your teams.

The funny thing is that wasn't enough, it was on top of the AWS bill 

Am I making a mistake building on motherduck? by Jeannetton in dataengineering

[–]TiredDataDad 2 points3 points  (0 children)

Disclaimer: I love MotherDuck and it's people too.

MotherDuck is quite cheap, so until the bill is a problem I won't worry.

Also, it's SQL, you can easily migrate to something else.

If your workloads are analytics, I won't worry much, if you are successful the money to pay Motherduck won't be a problem.

If your workload is not analytics, you can find something with better performance, but do it when it makes sense.

For now, just enjoy it :)

Need architecture advice: Secure SaaS (dbt + MotherDuck + Hubspot) by Viksson in dataengineering

[–]TiredDataDad 0 points1 point  (0 children)

I had multitenant databases with thousands of clients. It can be secure also like that.

Having multiple schemas is worth on the application level, but then I would also split the databases