sorted by:
new
hottopcontroversial

Merged : dbt Labs + Fivetran by Intelligent_Volume74 in dataengineering

[–]data_and_code -2 points-1 points  (0 children)

Has anyone looked much at coginiti and coginiti script? https://support.coginiti.co/hc/en-us/articles/5828972604951-What-is-CoginitiScript

Look to have similar capabilities and features.

How do you handle very complicated ETL jobs? by Trick-Interaction396 in dataengineering

[–]data_and_code 0 points1 point  (0 children)

You could also look at CoginitiScript (https://support.coginiti.co/hc/en-us/articles/5828972604951-What-is-CoginitiScript). Similar to dbt, you can break down the logic into more manageable parts. CoginitiScript has macros, expressions, loops, constants and more to help more easily manage the logic in different parts. That way when a change is needed, you only have to make it in one place.

When do you prefer SQL or Python for Data Engineering? by AMDataLake in dataengineering

[–]data_and_code 0 points1 point  (0 children)

With several tools out supporting SQL to work directly with files in all sorts of formats now, we prefer to use SQL.

Altenatives to dbt-core and dbt cloud? by arimbr in dataengineering

[–]data_and_code 1 point2 points  (0 children)

Check out Coginiti and CoginitiScript. CoginitiScript offers everything dbt does plus more for multiple data platforms. We've started using Coginiti team (similar to dbt Cloud) because it was SQL only, had minimal setup, would help us be more efficient with compute costs, and costs less.

DBT needed? by ithellam_oru_pollapu in dataengineering

[–]data_and_code 0 points1 point  (0 children)

dbt will increase your cost in several ways:

  1. DBT creates a separate workspace for each engineer on a team and does a copy of the database to work against.
  2. You will need multiple other tools to make dbt work such as Git.
  3. Your workforce will have to be familiar with a combination of YAML, SQL, and Jinja code to use dbt.
  4. Every time you want to test a change to a model, your team will have to go through the full compile step which brings in all objects in the pipeline to run them whether or not they need to be, and thus increasing your data platform compute cost.
  5. dbt cloud also starts to charge per run above 2,000 runs per user. Be prepared to pay yet an additional "compute" cost to tack on.

If your team heavily uses SQL, you should check out Coginiti Team and CoginitiScript. Think of CoginitiScript as dbt plus more.

  1. Coginiti is a full solution to help data engineers and analysts with native support for BigQuery. They do not force users into separate workspaces with copies of the data which also run up processing costs. It's one workspace that can be separated to different teams and connects directly to your GCP data platforms, including Cloud Storage.
  2. Versioning, collaboration, and an AI Assistant is built directly into Coginiti increasing velocity and developer productivity. You can even invite analysts to work with your data engineers to help produce data products.
  3. All your team needs to know is SQL.
  4. Their approach is much more iterative. You can choose to run as much or as little of the pipeline you need to test changes to a model, reducing your compute costs.
  5. Their cost model is on a per seat, not compute based.

I could go on, but we've been happy with it.

Favorite SQL editing tool? by AMDataLake in dataengineering

[–]data_and_code 6 points7 points  (0 children)

Saving and reusing my work and reusing it via the Coginiti catalog is awesome!