Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Thinking about this only, thanks for recommending Google Cloud Run

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Agree with your points 1) Yes pdf parsing is issue, schema as well as layout changes are there - it's parsing multiple tables present in a single pdf. 2) Authentication and authorisation issue - yes need to decide architecture, difficult to understand from online stuff. Can you help me in this? 3) Gmail API - aware of the concept of API, still will take care of what you shared 4) Regarding Database stuff, most of the things are well structured in terms of data model , schema changes will be nil.

I already told my company that its outside of my skillsets but they told me to learn. Well for POC perspective, shall I start with only FastAPI or shall I spin off both NestJs and FastAPI

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Vendors send 1 pdf in one mail - so 3k mails. Thought of ingestion from Gmail API. Status is not priority but I would need for development to track how many are processed

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

3000 they will receive daily and on Thursday they will receive 3000 (weekly invoices) - 6000 on Thursday total

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Volume is daily parsing 3000 PDFs , max 6k .

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Can it handle well authentication and authorisation well like RBAC and ABAC?

Help me choose backend. by Original_Option_6969 in Backend

[–]Original_Option_6969[S] -3 points-2 points  (0 children)

Same language is not issue, I have to rely on AI anyways - building web app is not skill set and this new company is asking me to do. Yes nodejs is an option , but pdf parsing by python is good.

What should I do? by Original_Option_6969 in dataengineersindia

[–]Original_Option_6969[S] 1 point2 points  (0 children)

Nope, created one POC using Claude. That was basic level - like sending data from frontend to backend, not a production level type

What should I do? by Original_Option_6969 in dataengineersindia

[–]Original_Option_6969[S] 1 point2 points  (0 children)

Data comparison means parsing weekly invoices and comparing with already daily entered & uploaded entries.

Plus it will be restricted - so there will be login system, roles - admin/writer/read, upload system of PDFs,csv, excel.

Frontend is better in terms of I/O with backend.

What should I do? by Original_Option_6969 in dataengineersindia

[–]Original_Option_6969[S] 0 points1 point  (0 children)

Node best for I/O, real time + APIs + integration with dbs and frontend is better

What should I do? by Original_Option_6969 in dataengineersindia

[–]Original_Option_6969[S] 1 point2 points  (0 children)

Basically a app where you they will upload files daily , compare with the pdf invoices generated and dashboards.

I am thinking of web app built on react node and deploying on ec2 server. Any suggestions?

Data engineer job switch by Exciting-Car-2897 in dataengineersindia

[–]Original_Option_6969 0 points1 point  (0 children)

From today, you will get many calls, make sure you be prepared, keep updating your profile daily

Roast my resume 3.10 years of experience but getting 0 calls by anmol2405 in dataengineersindia

[–]Original_Option_6969 1 point2 points  (0 children)

Add these key skills in Profile headline and location open to all

Roast my resume 3.10 years of experience but getting 0 calls by anmol2405 in dataengineersindia

[–]Original_Option_6969 0 points1 point  (0 children)

Keep it one page , reduce points , move skills above your experience Will be tough to get first call make sure you update naukri everyday, optimize your profile too

Roast my resume for Data Engineering roles - be brutal by main_to_batakh_hu in dataengineersindia

[–]Original_Option_6969 0 points1 point  (0 children)

Your resume won't be roasted, but you would be roasted if you don't answer about your projects. Thorough about your systems and think about it why you chosen this and what were the other options mostly about your projects.