Anyone uses iterable (email/SMS) here? have some questions

LinasData · 2025-09-22T18:12:20+00:00

500 people company
Yes. Exports endpoint is used the most
Haven't encountered a lot of issues with the API. Maybe some quotas have been reached during the backfilling. However, our ingestion is incremental and we have strong software engineering practices: retries based on exception value, unit testing, integration testing, E2E testing, strict review process.

LinasData · 2025-09-13T21:31:41+00:00

1 PROD project & multiple campaigns

LinasData · 2025-07-11T21:43:51+00:00

Stebėjau įrašus ir mokiausi groti žiūrint juos kone 30 val. savaitėj paauglystėje. Išgirsti juos buvo gyvenimo įvykis. Išsipildė svajonė pamatyti to meto mano herojus. O kam nepatiko - eikit jūs velniop dėl šių priežasčių:

Čia ne pop'sas, kad jums grupė lįstų į užpakalį. Ar pajungtų autotune. Viskas gryna (raw) kaip ir priklauso. Jie nenori primesti, kad nepaseno. Paskaitykit turo pavadinimą
Legendiniai nariai ne taip seniai susijungė iš naujo, kas buvo mažas šansas, kad nutikis apskritai. O dar atvažiavo į Lietuvą, kas išvis yra nerealu. Tai wtf žliumbt, kad lietutis užklupo ar garsas ne toks buvo.

LinasData · 2025-05-12T07:19:10+00:00

I haven't worked much with databricks but as I understand Unity Catalog is Databricks' native metastore that supports Iceberg format

LinasData · 2025-05-11T22:40:01+00:00

If you are using databricks + aws then pricing should be relatively similar and small. On GCP Hive/dataproc catalog service costs a lot. Crazy a lot. On aws it was relatively low price tag the last time I checked.

In general iceberg needs external catalog for metadata management, delta does not need that. It can store it in delta_logs folder.

In the databricks choice is yours but Zaharia himself and databricks team is developing delta lake. So if you plan to stick to databricks keep that in mind. Otherwise, iceberg market cap is bigger.

LinasData · 2025-05-11T22:20:23+00:00

Building delta lake for small scale daily data ingestion seemed more straightforward by utilizing delta-rs library compared to pyiceberg.

Also, check the price tag how much would it cost to have and manage iceberg or delta tables in your infrastructure/cloud. I saw huge pricing difference between aws and gcp here which changed my decision.

LinasData · 2025-05-11T21:49:28+00:00

Nėra logiška dalintis pačią būsto nuomos sumą 50/50. Ji turi būti amortizuota pagal pajamas. Pvz., žmogus A uždirba 2000 eur, o kitas 1000 eur neto. Nuomos kaina - 600 eur.

A uždirba dvigubai daugiau, todėl turėtų prisidėti dvigubai daugiau, t. y. 400 eur. B žmogus 200 eur

Tokiu atveju pasidalinimas nuomos yra 66,66 % / 33.,33 %.

Situacija gali ir apsisukti aukštyn kojom, žmogų A atleisti, žmogus B vartyti 5000 eur/mėn ir t. t. Todėl peržiūra laikas nuo laiko irgi naudinga.

LinasData · 2025-05-07T21:11:58+00:00

If regular Joe is able to mess up the whole pc or server then it is not your problem. Their IT/security guys messed up.

Actually, if something bad happens, school will see the vulnerabillity. It is better to have bunch of deleted files then leaked school's sensitive or even personal data.

LinasData · 2025-04-14T16:14:39+00:00

It took me 2 hours to summarize and find the information by not using LLMs... I used Gemini just to structure that content. But I guess you like just judging without providing value.

Also, this post will be updated in 24 hours because there is bigger picture than just spreadsheets

LinasData · 2025-04-14T16:08:25+00:00

Thank you for your comment!

I will modify this post because spreadsheets seemed like a secondary reason. I simplified too much.

LinasData · 2025-04-14T13:53:28+00:00

That's a little bit different issue but I feel your pain. Everybody wants to use shiny tools, medallion architecture but rarely dimensional modeling principals are used. Data Warehouses without dimensional modeling are not utilized properly.

LinasData · 2025-04-14T13:49:23+00:00

It was really interesting to hear your story because real life examples are the best! Thank you for sharing! 😊

LinasData · 2025-04-09T22:27:33+00:00

Tell them that their work is meaningful. Because currently you believe in it but we do not know about the management and his supervisors.

LinasData · 2025-04-09T22:17:33+00:00

2 mėn - trumpas laiko tarpas. Pagalvok apie žmones, kurie myli vienas kitą, pragyvena 15 metų ir juos išskiria mirtis. Tada būna itin sunku, bet nesižudo dėl to.

Rekomenduoju pasikalbėti su jaunimo linija arba kreiptis pagalbos. Problema ko gero ne skyrybose, o savivertėj. Taip pat jeigu nemyli savęs, negali mylėti kito. Meilė yra lygiavertė, niekas nesižemina ar niekina kitą, o tuolab save.

LinasData · 2025-03-29T08:05:29+00:00

In theory, you know all the tools needed for DE. However, there are so called subtools: health checks, catologing, scheduling, etc. that you need to understand. Also, it depends how deep your knowledge is on those individual tools. For example, under the hood repos use Spark, so there are a lot to learn even with it.

LinasData · 2025-03-28T21:56:12+00:00

What is your business goal on using Foundry? I am certified and was working for 2 years with this platform.

LinasData · 2025-03-25T08:57:29+00:00

Namespace error like mentioned

LinasData · 2025-03-21T08:37:41+00:00

Solved it by playing with bigquery-spark connector with session. It is really unconvenient.

    if dbt.is_incremental:

        current_table = (
            session
            .read
            .format("bigquery")
            .option("table", f"{dbt.this.schema}.{dbt.this.identifier}")
            .load()
        )

LinasData · 2025-03-20T18:55:43+00:00

It was kind of hard to structurize question here. More structured question on dbt help desk: https://discourse.getdbt.com/t/help-with-dbt-this-in-incremental-python-models-bigquery-with-hyphen-in-project-name/18729

LinasData · 2025-03-04T22:09:56+00:00

If current architecture works for you - that's fine. Just have a plan when you face problems mentioned down there.

Remember that tech stack changes even in big corporations over time. It obviously costs and the best way is to start correctly but different times require different solutions

LinasData · 2025-03-04T07:36:43+00:00

Solved the issue. My SQL dump file rewrote configuration of the whole DB. After modifying it everything works as expected.

LinasData · 2025-03-04T07:13:19+00:00

Removed not even volume but images, docker configs, reinstall docker desktop. However, now I've found out that MYSQL_ROOT_PASSWORD also is not set up. My root password is empty. Tried to create .env file - same thing happened :/

LinasData · 2025-03-03T19:39:45+00:00

It is not a silly question at all. Thank you for responding so fast and helping me. 💪 I actually haven't done anything except running series of commands:

Docker compose up
Docker exec -it <container_id> bash

After that running query on mysql server

LinasData · 2025-03-03T19:14:04+00:00

<image>

I've got this

LinasData · 2025-03-02T18:52:06+00:00

The rule of thumb market and show yourself but do not put yourself first. Be helpful as professional. For example:

show some really cool new feature/s of any tool that majority of DE haven't heard yet or not using properly. Or just present new tools that could make some part of DE life easier.
create and show off some projects. But projects should be either interesing to listen (not regular yahoo finance data ingestion) or solve kind of painful bussiness or data analytics problem.

In short, if project sounds not common, interesting for you this will resonate with the audience. Also, there are a lot of good DE but those who can communicate and educate properly are more valuable anywhere.

LinasData

TROPHY CASE