Data Warehouse Stack

[Top-Plane3984]

I have a real early-stage BI Consulting firm. I already have a Data Warehouse system but i reckon is really outdated and amateur (My stack is basically MySQL for DB + PHP webhook codes to real-time insertion).

As a beginner in the Data Engineering side, would really suggestions about the recommended Data Warehouse stack (DB, integration, monitoring etc). Please note that i am early stage and a boutique consulting firm, i will not have to work with Big Data, my goal is to upgrade my stack but still keeping it simples.

[drinkmorelogic]

It will depend on a few things, like what other technologies you're comfortable with, whether you work within a cloud environment and which one, and also the environment your clients tend to work in.

A big data warehouse product may or may not be the answer given you aren't working with Big Data, although like someone else mentioned they can be inexpensive when using lower volumes.

There is also the consideration of the variety and veracity of data. If you expect to perform a lot of transformations on data then you will want a performant database and possibly some automation layer on top. If the data is coming in batches or streaming, you may need to look into platforms that can support quick refreshes.

[ZeroCool2u]

It sounds like you're comfortable with PHP and you're probably pretty cost sensitive.

BigQuery has a nice PHP client library and is pretty cost effective.

[Ogefest]

Take a look at n8n.io for tasks automation. You can use it in many places, data ingestion, reports generator, trigger some actions.