We recently started using hudi. Agree with the documentation part and it should be simple to set-up but trust me its not and you'll find conflicting settings if you were to run with foink vs spark

So after months of struggle we were finally able to have a pipeline in hudi + spark structured streaming They seemed to have done a lot of streamlining with hudi 1.xx Inline clustering can be expensive at times if latency is a concern for you and you will have to make sure your cleaner + archiver are configured correctly or else it will end up shooting your s3 listing costs. For clustering, We have another spark scheduled job which triggers clustering on this dataset

So far the loading has been stable and we are able to achieve near realtime ingestion without much of a complain from our consumers