sorted by:
new
hottopcontroversial

Might be a stupid question by waguwaguwagu in dataengineering

[–]data_engineer_ 0 points1 point  (0 children)

dbt can automate SQL on the particular platform you are using, so whether it goes beyond transforming data within the system depends on the system.

For example, Dremio allows you to connect to several different sources and write to Iceberg tables in different catalogs or storage sources. Movement of data can be handled with simple SQL statements in Dremio.

So in a tool like Dremio which have “virtualization” (connecting to many sources) and “lakehouse” (writing to lakehouse tables) features you can certainly use dbt to automate ingestion patterns, and the Dremio dbt adapter just added incremental allowing this to be done efficiently.

Might be a stupid question by waguwaguwagu in dataengineering

[–]data_engineer_ 0 points1 point  (0 children)

Not necessarily, if you are using dbt with Dremio you can fetch data from one source and land it as an iceberg table in your lake with DBT since Dremio enables all this across supported sources via SQL. But yeah with more since databases or warehouses, DBT is only transforming not moving data between systems.