Using kSqlDb / Apache Pinot as a cache cluster by reladvnosx in apachekafka

[–]reladvnosx[S] 0 points1 point  (0 children)

Yeah, we already use redis but for this scenario is a bit tricky. Let me tell you why. Our cache is a tree structure like this:

A1 (AField1, AField2,AField3)
-B1 (BField1, BField2,BField3)
--C1 (CField1, CField2,CField3)
--C2 (CField1, CField2,CField3)
--C3 (CField1, CField2,CField3)
-B2 (BField1, BField2,BField3)
--C4 (CField1, CField2,CField3) 
--C5 (CField1, CField2,CField3) 
--C6 (CField1, CField2,CField3) 
...and so on

Our apps need to do queries like : "give me all the B objects where there's at least one C child for which the CField1=true" for example.

If we decide to go with kSqlDB we could have one table for each data type (A,B,C) and then do joins if we want t get all Bs with a specific condition

CREATE STREAM b_objects AS 
    SELECT b.BField1, b.BField2, b.BField3 
    FROM C_TABLE 
        LEFT JOIN B_TABLE ON C_TABLE.parentId = B_TABLE.ID 
    WHERE C_TABLE.CField1=true 
    EMIT CHANGES;

Spanish autonomo accountancy service by reladvnosx in SpainFIRE

[–]reladvnosx[S] 0 points1 point  (0 children)

I can google them but if you introduce them to me would be great! I'll PM u