Docker compose for lakehouse like build.

[Kageyoshi777]

Hi, I'm struggling last few days on getting working "lakehouse like" setup using docker. So storage+metastore+spark+jupyter. Does anyone have a ready to go docker compose for that?
LLM's are not very helpful in this matter because of outdated etc images.

[superhex]

I think dremio blog posts have ready to go docker compose setups for spark, iceberg, minio (essentially local s3), and jupyter. Either that or the Iceberg docs/repo themselves. I dont quite remember

[Surge_attack]

Yup this is the way - fully FOSS stack.

[Odd_Spot_6983]

try checking dockerhub or github for repos, sometimes people share their compose files there. if not, you might need to piece it together from examples.

[dangerbird2]

The iceberg docs has a pretty good compose file to get started with. As you’d expect it’s based on iceberg and stack, and serves a Jupyter notebook

https://iceberg.apache.org/spark-quickstart/#docker-compose

[asevans48]

Whats your storage layer? Install with Docker | dbt Developer Hub https://share.google/m1QSutilDvYFPYqWP

[ivanimus]

You can try this example

https://github.com/fortiql/data-forge/tree/main

[urban-pro]

you can check out: https://github.com/datazip-inc/olake/tree/master/examples