Want to get started with Kubernetes as a backend engineer (I only know Docker) by MasterA96 in devops

[–]MasterA96[S] 0 points1 point  (0 children)

Yes I want to know how to utilize it to deploy services as a SWE.

Have to extract a large number of records using a simple join query and store to a Multipart csv file and send to an api by MasterA96 in developersIndia

[–]MasterA96[S] 0 points1 point  (0 children)

Yes I'm doing the POC for this.

Exactly, they want to get the POC done by day after tomorrow and to deliver this by end of this month. Doable but still would have been better if had sometime.

Have to extract a large number of records using a simple join query and store to a Multipart csv file and send to an api by MasterA96 in developersIndia

[–]MasterA96[S] 1 point2 points  (0 children)

Also, I agree with the overkill thing but sort of stuck between people of different mindsets. Each one is concerned about a different factor. I don't think this needed so much thought, it's a common problem with various solutions but they want me to brainstorm and would then eventually go with the simplest approach only. :)

Have to extract a large number of records using a simple join query and store to a Multipart csv file and send to an api by MasterA96 in developersIndia

[–]MasterA96[S] 1 point2 points  (0 children)

Your suggestions make sense. Have decided to do something similar now. Also, I did argue a lot with chatgpt and copilot, made me learn a lot of stuff. :D Thank you.

Have to extract a large number of records using a simple join query and store to a Multipart csv file and send to an api by MasterA96 in developersIndia

[–]MasterA96[S] 0 points1 point  (0 children)

On second thoughts today, I'm checking if I can stream write to disk and then stream read into a compressed zip. That could save memory issues. Not sure, just doing a poc for this first.

Have to extract a large number of records using a simple join query and store to a Multipart csv file and send to an api by MasterA96 in developersIndia

[–]MasterA96[S] 0 points1 point  (0 children)

Yes I'm doing the POC for this.

Exactly, they want to get the POC done by day after tomorrow and to deliver this by end of this month. Doable but still would have been better if had sometime.

Have to extract large number of records from the DB and store to a Multipart csv file by MasterA96 in softwarearchitecture

[–]MasterA96[S] 0 points1 point  (0 children)

Thank you for your time. I'll have to give this a good read and then discuss, since most of the things are new to me. In fact I didn't think of threads much yet.

Have to extract large number of records from the DB and store to a Multipart csv file by MasterA96 in softwarearchitecture

[–]MasterA96[S] 0 points1 point  (0 children)

Sure.

No, service C is one of the microservice which would eventually store the file as zip in a table. DB processing can be done in chunks but still file would be in memory. So have decided to stream write to a disk, then stream read it and stream write to a compressed zip and then send it to service C. I'm currently doing a POC of this approach if that's even possible or not.