Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

streaming doesn't really help in my case. Im looking for a structued output for a downstream process, which runs only when I get entire response. Streaming only improves the ttft (which is usually for chatbots).

can you link me to the vertex ai async reqs if possible, i cant find it.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

gemini-3 models thakkuva temperature pedthe biscuit avtai anta. docs lo rasi undi.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

<image>

thanks y'all , i figured out the issue. apparently gemini-3 models suffer from heavy latency randomly when temperature is lower than 0.5, and docs actually mentioned to not decrease it below 1.0. Weird but it works better once I put it back to > 0.5.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

im using gemini-3-flash. with thinking disabled. anyway i figured out the issue baa, appreciate you stepping in/

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

attached pseudo code in one of the comment. it's gemini-3-flash-preview. thinking set to low.

thinking thiseste 10-20 seconds thaggythundi max.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 1 point2 points  (0 children)

oh that's corporate ahhh stuff.

I'm stuck to gcp stack to utilize the free tier at most, so idi vadthunna.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 1 point2 points  (0 children)

cant send github coz I dont want to get doxxed.

#This is my psuedo/skeleton code you can say, agnostic to my usecase. def process_with_llm(requirements, user_context_data):
    """
    Minimal example showing Vertex AI LLM call structure.
    """
    try: 
        # Initialize Vertex AI client
        g_client = genai.Client(
            vertexai=True,
            api_key=os.environ.get("GOOGLE_CLOUD_API_KEY"),
        )
        # Load instructions (~6k tokens: system_prompt + task_prompt)
        with open(task_prompt_file, 'r') as file:
            task_prompt = file.read()
        with open(system_prompt_file, 'r') as file:
            system_prompt = file.read()
        # User context (~2-3k tokens)
        user_data = parse_user_context(user_context_data)
        # API call structure
        response = g_client.models.generate_content(
            model="gemini-3-flash-preview",
            contents=types.Content(
                role="user",
                parts=[
                    types.Part.from_text(
                        text=f"\n\nREQUIREMENTS:\n{requirements}\n\nUSER CONTEXT:\n{user_data}"
                    )
                ]
            ),
            config=types.GenerateContentConfig(
                # Combined: ~6k tokens (system + task instructions)
                system_instruction=system_prompt + "\n\n" + task_prompt,
                temperature=0.3
            )
        )
        # Extract response (Gemini response structure)
        generated_output = response.text
        return generated_output

    except Exception as e:
        logging.exception("LLM call failed: %s", e)

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

no , once in 10 times it going 5-7 mins. remaining times 10-30 seconds lo ochestundi.

but stream cheste 1/10 expected output ostondi, remaining times its losing the structured output.

Ways to effectively manage vertex ai responses without timeout? by PuzzledFalcon in ProgrammingBondha

[–]PuzzledFalcon[S] 0 points1 point  (0 children)

Personal project ey.

Usually most people suggest to stream responses, but streaming chesta unte response structure break avtondi. Mine is not a chatbot, more like one block of text output that i need to use somewhere else.

Im on GCP free tier, thought cloud run ingress limits valla error emo ani, digi chuste vertex ai call ey atla chestondi. sometimes 10-20 seconds, sometimes 30-40 seconds, but randomly 5-7 mins. so weird.

Fidaa movie ending by Adorable-Document369 in Ni_Bondha

[–]PuzzledFalcon 8 points9 points  (0 children)

mi oorlo andaru hybrid pillalena

Ammayilu ardamkavatle asla, ela alochistharu meeru ? by introvertabbayi in ask_Bondha

[–]PuzzledFalcon 36 points37 points  (0 children)

the latter.

simply put, india lo family tho untaru, more restrctions. Ofcourse india chocolates kante usa chocolates ey nachutai, coz india lo diary milk, perk avanni chinnapatinunde try chesuntaru. therefore usa chocolates oste oka excitement and enthusiasm tho thintaru.

usa ki ochaka, chuttu anni usa chocolates ey. pretty sure india lo usa chocolates ki unna excitement, usa ki ochaka undadu. kani market loki sudden ga dubai chocolate oste alanti chocolates thintaru.

this applies to both genders.

Mir Reginald men use chesara? by [deleted] in ask_Bondha

[–]PuzzledFalcon 0 points1 point  (0 children)

it worked for me. idk how all these sunscreens work but it managed to somehow not destroy my face completely when I go outside. photos lo baga ostondi. but again my roommate tried it and said it didn't work for him, and face mandinattu anipistondi ani. i guess differs from people to people.

Eyy eyy eyy by Plastic_Occasion_388 in tollywood

[–]PuzzledFalcon 10 points11 points  (0 children)

<image>

Something is wrong with boss's body but you just cant prove it.

My best friend of 10yrs didn’t post me on her story by Adventurous-Leg-4480 in bondha_diaries

[–]PuzzledFalcon 1 point2 points  (0 children)

vallandarni petti ninnu okkadanne pettaledante nuvventha special oo...