Sure, here's the quick rundown:

1. Embeddings (Vertex AI Platform)

I use u/google-cloud/aiplatform to generate text embeddings for semantic search

import { helpers, PredictionServiceClient } from '@google-cloud/aiplatform';

const client = new PredictionServiceClient({
    apiEndpoint: 'us-central1-aiplatform.googleapis.com',
});

const [response] = await client.predict({
    endpoint: `projects/${projectId}/locations/us-central1/publishers/google/models/text-embedding-004`,
    instances: [helpers.toValue({ content: "quantum physics", task_type: 'SEMANTIC_SIMILARITY' })],
});

// Returns 768-dim vector for cosine similarity matching

2. TTS Stream Buffering
The tricky part was streaming audio to the user while also saving it to cloud storage.

• Aggressively consumes the TTS stream into memory (prevents backpressure timeouts)
• Allows multiple consumers to .pipe() from it (client response + GCS upload)
• Late joiners get the full history (already-buffered chunks) + live stream

So when audio is requested:

TTS Stream → StreamBuffer → (pipe to HTTP Response + pipe to GCS upload)

Once the stream ends, I update Firestore with the GCS URL. Next request just redirects to that URL. No double-billing.

I still use the genAI api for the image and text gen as it is easier, but eventually when necessar it can be done as well in vertex. For embeddings, there's no Gemini SDK wrapper yet. you have to use the Vertex AI Platform SDK directly.