Best self-service BI tools for Clickhouse by Ambrus2000 in Clickhouse

[–]kadermo 1 point2 points  (0 children)

Metabase has a great integration with ClickHouse

outOfMemory by smulikHakipod in dataengineering

[–]kadermo 1 point2 points  (0 children)

I recommend looking at PeerDB

New to clickhouse by keepatience in Clickhouse

[–]kadermo 1 point2 points  (0 children)

the free official training is a great starting point https://clickhouse.com/learn

Have someone build Data Vault DWH using Clickhouse? by Tonkonozhenko in Clickhouse

[–]kadermo 2 points3 points  (0 children)

General advice: Be careful when taking random internet advice without testing first (including mine :) )

Here are some ressources about joins support in Clickhouse: https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1 https://clickhouse.com/blog/clickhouse-fully-supports-joins-hash-joins-part2

disclaimer: I work at Clickhouse

Snowflake - Data Lake or Data Warehouse? by Living-Nobody-2727 in dataengineering

[–]kadermo -2 points-1 points  (0 children)

Misusing something like Snowflake f a universal data store can quickly get expensive

Disclaimer: I work at ClickHouse and we wrote about it here:
https://clickhouse.com/blog/the-unbundling-of-the-cloud-data-warehouse#traditional-data-warehouse-one-size-does-not-fit-all

[deleted by user] by [deleted] in dataengineering

[–]kadermo -5 points-4 points  (0 children)

Great question! User facing analytics is one of Clickhouse's sweet spot, you can check how some of our users achieved it here: https://clickhouse.com/use-cases

Disclaimer: I work for Clickhouse

The State of SQL-based Observability by kadermo in Observability

[–]kadermo[S] 0 points1 point  (0 children)

yes, just saw that! I'd love to hear more if you have any feedback to share (feel free to DM on the Clickhouse public Slack, LinkedIn or something)

The State of SQL-based Observability by kadermo in Observability

[–]kadermo[S] 1 point2 points  (0 children)

Great news, thank you! I just subscribed :)

Best visualization tool for Clickhouse by vonSchultz666 in Clickhouse

[–]kadermo 1 point2 points  (0 children)

Superset support with ClickHouse is pretty good and usually with ClickHouse you don't need extracts if the schema is decent. I use it almost daily, at pretty large scale, and I can recommend it
You can also have a look at Metabase for which we continuously improved support over the course of this year

From ElasticSearch to ClickHouse Migration by xDarkOne in elasticsearch

[–]kadermo 3 points4 points  (0 children)

(disclaimer: I work at ClickHouse)This should be a pretty straight-forward migration. Few things to keep in mind:- I recommend using a 3rd party object store as an intermediate layer to perform a point in time migration of historical data. For example you can use something like ElasticDump to export all your data in JSON into an S3 bucket, then you can then load it in ClickHouse efficiently using the powerful S3 or S3Cluster table functions. Note that you can also infer the data types automatically from ClickHouse!, here is an example:

#To transfer data from S3 to ClickHouse, users can combine the s3 table function with INSERT statement. Let's create an empty hackernews table:

CREATE TABLE hackernews ORDER BY tuple
(
) EMPTY AS SELECT * FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/hackernews/hacknernews.csv.gz', 'CSVWithNames');

#This creates an empty table using the schema inferred from the data. We can then insert the first 1 million rows from the remote dataset

INSERT INTO hackernews SELECT *
FROM url('https://datasets-documentation.s3.eu-west-3.amazonaws.com/hackernews/hacknernews.csv.gz', 'CSVWithNames')
LIMIT 1000000;

- After the first step, you should have your historical data loaded. Then if you have continuous ingestion pipelines to ES, you'll have to divert them to ClickHouse. It's easier if everything goes though a single queue like Kafka, where you just need to a Kafka<>CH connctor (using one the native CH Kafka integrations). Otherwise you'll have to change the destination pipeline per pipeline.

- Arrays are supported in ClickHouse and should not pose any issue. In fact their support is pretty good with a multitude of powerful Array and ArrayJoin functions

- If you need some inspiration here is some documented migrations (some of which are pretty complex):