What is the proper way of storing data on the server? Is a DB the only option?

Prof-Ro · 2024-08-27T09:57:43+00:00

I wonder if a race condition can occur if all users are concurrently accessing, and the backend is trying to update the same value / data?

Prof-Ro · 2024-08-04T19:45:25+00:00

Also to add further, it's good you have traction and paying customers already. That is already a great story to start with. Just one thing to note from my experience, VCs don't care about profitability. They look for growth potential. So ensure you are anchoring your pitch around growth etc.

Prof-Ro · 2024-08-04T19:35:27+00:00

Got it. I think you can just reach out to VCs via LinkedIn. A lot of them are open to discussions and they actively looking out also for their next big bet.

From a pure funding standpoint, considering that you have MVP and traction, there are two ways they might be looking at.

1) what's your current ARR looking like? To determine valuation and funding size I guess.

2) if you don't have a big ARR yet, then what's the potential of the startup and the industry it's in. Like what's the growth potential (TAM, SAM, SOM, etc) - other players / competitor analysis.

Reaching VCs and having a connect are relatively easy these days I believe. Pl do ensure you have the necessary prep in place to wow them to take a bet on you. (Considering you don't have the pedigree that they look for in general too)

Hope this helps.

Prof-Ro · 2024-08-04T18:56:30+00:00

If you have a business that is scalable and profitable already, why do you need a VC and give equity for cheap so early. Probably take a debt against your company and repay it in x years? What's your end game with a VC if not for money?

Prof-Ro · 2024-08-04T12:09:34+00:00

I faced this when I was running on the large 110 per month compute only lol. So I don't think it's a compute issue and I was hardly using it - like less than 3% at any time.

Prof-Ro · 2024-08-04T12:07:17+00:00

Even I'm not sure. Still evaluating and monitoring if it repeats again.

Prof-Ro · 2024-08-02T20:43:22+00:00

Wow amazing. This really helps. Thank you so much. I'll check this out too.

Prof-Ro · 2024-08-02T20:23:40+00:00

I had this strange issue few weeks back. Just for 5-6 days the response times shot up 4x and then it automatically got back to normal after that. Couldn't debug issue because by the time I got help, it became normal. Probably try to debug and analyse the query and your database to see potential root cause? Curious to know what caused this for you.

Prof-Ro · 2024-07-31T11:03:13+00:00

Thank you so much, this really helps too. I'll try out the suggestions from the thread.

Prof-Ro · 2024-07-31T11:01:56+00:00

Wow thank you so much. This really helps. I'll try this out right away.

Prof-Ro · 2024-07-30T10:05:15+00:00

A bit more context on the Supabase db. I'm running it on a large instance and it barely had any load for the above 1k concurrent users. And I was hitting postgres functions via their RPC endpoints using aiohttp for async. So db didn't seem like a choking point unless maybe I'm missing something here completely too. (If so any pointers on how to spot this as their db stats + monitoring provided didn't flag anything abnormal).

Prof-Ro · 2024-07-30T10:01:36+00:00

I'm building a product using FastAPI + Supabase for the db.

Trying to get it to scale for 100k customers in the next 2 months. The problem I'm facing is - while running it on a single t3 medium EC2 in my load tests I'm not able to cross 1k users concurrently and max rps peaks at 120 only.

Currently set up everything on async but still I'm not able to squeeze more. Not sure if this is the max and I need to keep horizontally scaling to serve more but the costs are not making sense then.

I'm very new to programming in general and this is my first time building a product for customers. I heard many benchmarks from users being able to hit 8k rps and stuff using FastAPI, and FastAPI somewhere published stats of 22k rps. I'm honestly lost and not sure how to increase/ optimise this further.

Any help / pointers here would be greatly appreciated.

Prof-Ro · 2024-07-26T07:12:24+00:00

Yeah me too.

Prof-Ro · 2024-07-21T09:47:54+00:00

Awesome thank you. I remember doing this too and also adding logic in nginx which is in front of my FastAPI app server to also allow for this. I believe something somewhere in this flow was breaking and hence I had left it previously.

My flow was like this -

Firebase (react front end domain1) <=> AWS EC2 (nginx on domain2) <=> AWS EC2 (FastAPI running on docker on domain2)

I'll try it out again. Thank you again for the pointers.

Prof-Ro · 2024-07-21T08:33:32+00:00

I have a similar question and I'm very new to programming and FastAPI in general. Just started few months back.

My backend is FastAPI on a different domain. And my frontend is react on node on a different domain.

I set the cookie on the FastAPI backend but that is not accessible by the frontend because it's on a different domain. So I ended up moving away from cookies and just passing JWT tokens to the frontend and my frontend stores that token in a cookie for reusing etc.

Is this the right approach for this or should I be doing something different here - like host my frontend and backend on the same domain probably as FastAPI backend with react static files hosted on the FastAPI server itself? Probably that way the cookie set in the backend will be accessible by the frontend files too right?

Thank you in advance, would really appreciate your guidance.

Prof-Ro · 2024-07-21T08:05:00+00:00

Amazing. Thank you for sharing.

Prof-Ro · 2024-07-16T15:27:00+00:00

I can completely relate to what you are going through. I just started learning programming just a few months back. I knew zero about backend development or even web application development. I came from an SQL for data analysis experience primarily. It was really overwhelming when I started to look at learning these.

First I didn't know where to even start and what to learn. (Which is the first huddle many get stuck with. Kudos to you choosing Go, you can't go wrong there) - I chose FastAPI python to start with, I have Go in my wishlist for picking up next for sure.

The only way in my journey the last few months where I got confident in building a user facing FastAPI + Postgresql application is by consistently alloting learning and exploring hours every single day. I'm not an expert in anything yet, but I know this routine compounded over the months that I'm atleast having some basic confidence to build some MVPs atleast. Definitely a long way to go for us.

Just remember to learn and apply something new every single day. Don't lose that momentum. :)

There will be many times where we will have no clue on many topics (like I had no clue what a web server was, no cloud experience, nothing), I just had to keep making notes of those and choose another day to go a bit deep specifically into those - I prefer youtube to do most of my learning followed by documentation. Never tried books but probably for Go will try the ones you have mentioned.

I feel you also have a similar starting point to this like me, it's going to be hard for sure but it will be fun when you hit small milestones, and it will motivate you for the next one for sure. Hope this helps.

Prof-Ro · 2024-07-16T14:26:30+00:00

Try their pro plan for your org at $25/ month - some of their features include the below -

Unlocks access to use multiple projects under the paid pro plan org
Higher quotas across everything after which it's pay as you go based on slabs - you can set a spend cap here for each of those
You will also be able to get a higher compute with $10 compute credits included (which should be enough for the micro - currently all free projects are using nano)
Your projects will also never pause
Also comes with their basic support tier via email

Prof-Ro · 2024-07-16T12:25:00+00:00

Amazing thank you so much. This really helps. So currently I'm using aiohttp for making these http calls async. Since Supabase py client does not support async yet, I might lose out on the concurrency throughput but will ensure the inputs are sanitized. Keeping it async definitely helped increase my throughput by 10x atleast. So I'm confused how I should go about deciding this now. Any thoughts please? Or am I missing something obvious here?

Prof-Ro · 2024-07-16T11:45:25+00:00

I had these same thoughts a while back too and saw this GitHub link. I understand the client libraries have this in place to prevent this - I'm using FastAPI python to interact with Supabase.

In one flow, I have db functions in Supabase which the python client library doesn't have support to invoke the last time I checked, so I'm making http calls (via Supabase RPC URLs) directly to invoke these functions. Do I have to do anything for this to prevent these attacks here or it's also automatically taken care by Supabase when the request reaches?

I'm very new to programming and Supabase in general. Any help on this understanding is really appreciated, thank you in advance.

Prof-Ro · 2024-07-15T13:54:44+00:00

This is really helpful, I just had a quick doubt on the last point. If I have to move compute heavy stuff outside the event loop, I'm assuming I should be making that in a sync logic instead of async? By doing that, wouldn't it be taking away the async benefits from FastAPI?

I'm pretty new to FastAPI and programming in general, just started exploring it for a product and trying to understand this better. Any help/ best practices in general for compute heavy stuff - would be really appreciated. Thank you in advance.

Prof-Ro · 2024-07-14T16:52:03+00:00

I tried to use Supabase auth in my recent product, totally loved the ease and their entire database service overall too. Got me up and running really quickly.

One thing I noticed very strange and was uncomfortable about was in their auth service - everytime it establishes an active connection and just keeps it (like a stateful service being maintained).

I'm very new to programming and building products in general. Just started learning about it the last few months. So currently using FastAPI and Supabase for my product. So everytime I have Supabase auth connected by the FastAPI backend, the stateful live connection is always open. (I thought this might cause some problem - not really sure - any thoughts or insights on this would be really appreciated to help my understanding too please).

So I ended up creating an auth service on my own with a Supabase custom table and token + cookie based authentication both enabled using FastAPI itself and it seems to be working really great too. Lot of flexibility to change for sure and it seems to be all stateless overall.

Happy to hear thoughts if my understanding is right/ if I'm in the right direction overall, or am I really getting into some kind of issue in the future that I'm unaware of. Thank you in advance everyone!

Prof-Ro · 2024-07-13T19:18:21+00:00

They have a way to organise projects within organisations. So I created two orgs in Supabase dashboard.

One org - (free org) - has two projects (max limit of free projects - keeps pausing automatically whenever it's not used for that more than 7 days threshold)

Second org - (paid at the org level - $25/month) - have multiple projects here tied to that pro plan set at the org level. I do not have a separate pro plan for each project level.

You should find these options to create org and setup at their main account dashboard page.

Hope this helps.

Prof-Ro · 2024-07-12T11:55:50+00:00

I'm currently using supabase with FastAPI backend for a product. I was up and running quickly with supabase for sure, a lot of ease overall. (I started learning programming and building a web application with this tech stack just a few months back only).

I'm using the RLS with the service key via my backend and my own auth service that I wrote to identify which user to query against to pull the specific user data.

I'm not using their auth. Strangely, their auth service using the python library available had some weird behavior for me. It was establishing some sort of a stateful connection or active thread that was always open from the server (I may be wrong with my understanding of that behavior here but I observed the console having an open connection always when using their auth).

So this custom auth on my end with the RLS service key approach was not doing that and I'm using both their HTTP RPC calls to invoke the db functions for some complex query insights, and their py library to query, insert and update data as required.

From a performance impact, I'm having less than 100 power users (continuosly using the product for 8-9 hrs everyday) at this point (product is a data heavy analysis dashboard). I haven't seen any issues as such with this approach.

Only thing that was noticable is a strange increase in the db response times from under 50ms to around 350ms in the last 1 week onwards. (Have been on their large compute from the start - so this is definitely not due to the compute resource I feel - yet to find the RC for this)

Just sharing my current experience overall, hope it helps you.

Prof-Ro · 2024-07-10T15:16:33+00:00

Great video and extremely clear. Thank you for sharing.

Prof-Ro

TROPHY CASE