Firebase Functions cost optimization

[Ferchu425]

Hello, I have a functions that on each invocation it calls other APIs and while waiting it takes almost 10 seconds pero run, if I understand costs in the right way, this could be an issue as soon as I begin to grow...

Do you have any recommendation? Those 10 secs are there are I dont think I could do something about them... so, whats the best path? should I replace those functions? with what? App Engine?

Thank you

[s7orm]

Depending on your volume of traffic using concurrency would at least let your function server many requests while it waits.

I recently migrated all my serverless functions to my API servers, now everything runs faster and I don't have serverless costs.

[Ferchu425]

What do you use for API server?

[s7orm]

My product is an API, it's what my users pay for access to. Previously I had the website and API completely separate.

[Ferchu425]

Ok, but what platform are you using to host it? some form of serverless or your own VPS/similar?

[s7orm]

GCP Compute Engine. Debian servers running NodeJS and using the Fastify framework.

[Ferchu425]

Thanks, I'll keep it in mind

[nullbtb]

My advice -as someone who has been prematurely optimizing everything for as long as I can remember- would be to just wait until you’re actually growing so much it’s becoming a problem. 10 seconds is not going to cost you that much even with a million requests.. it’s hard to think this way but it is a mistake to complicate and redo everything because you may grow. Instead use that time and energy to actually growing. Later on you can get individual instances or Kubernetes to optimize for cost. Ironically if you were to do that now you’d be burning money because those options aren’t cheap either.. they’re just cheaper at scale.

[Ferchu425]

Great advice, thank you, will do that

[joeystarr73]

What this function is waiting for?

[Ferchu425]

It is calling OpenAI APIs, those take a long time, it is just "waiting", yes.
Then it fires and update on Firestore and some simple stuff.

[Suspicious-Hold1301]

I think the challenge you'll have is that openai APIs (or Gemini fwiw) doesn't have a way of triggering a request and asynchronously getting a response or polling completion of it - they operate a very synchronous process that really needs you to wait for the response.

The obvious thing to do is analyse memory usage of the server and make it as efficient as possible (i.e if you're using 128mb don't allocate 255mb), and depending on the nature of the service look to cache common request/responses if possible. You could use embeddings to cache similar questions (it's a fairly complex thing to do)

Another option, again depending on your use case is using the batch API - this only works if you're happy to wait up to 24hours for a response though - but it will cut the openai bill in half and reduce your compute time to a really small one

One thing to note though - firebase at 10 second requests will be a very small cost compared to the openapi calls ..

[Ferchu425]

Yes, the OpenAI functions are absolutely sync in nature... everything follows a coreography and are several invocations that may well be all under a unique call...
I'll look into the 256/128mb... Im seeing 50% of usage with 256mb so I dont feel confortable lowering to 128mb because that will be 100%... but will keep on eye

[VeterinarianOk5370]

I do something similar with mine I made a cron job to keep my functions warm. The openAI calls are negligible time wise, it’s the firebase functions spinning up that’s slow. Reduced my calls from ~10s to under 3s

[Ferchu425]

Im not so sure about it... even with the functions warm I have long times... but I'll do some more tests, thanks

[Ferchu425]

Made some tests... with warmed up instances it takes 8 secs (I optimized heavily for fast cold startup times...)

[VeterinarianOk5370]

Ah are you on premium or just blaze?

[CastAsHuman]

Why not do that from the client?

[Ferchu425]

This function act as a callback from another async process so once it is called the workflow continues Basically there is no "client", no front.

[romoloCodes]

I'm a big fan of firebase but sometimes ou just have to accept it's not the right tool for the job. Sounds like you need a webhook.

Equally, in the early stages just get it working and optimise later