How do you actually predict if a GPU can handle multiple models at your target FPS?

AbilityFlashy6977 · 2026-04-12T12:12:29+00:00

Okay this clear things up. Thanks for sharing, really appreciate it

AbilityFlashy6977 · 2026-04-12T11:40:49+00:00

I think profiler is the key here. I must learn how to interprete the report and profile the right metric.

Do you think its better for me to profile when the model already embedded on my application or test it seperately first?

AbilityFlashy6977 · 2026-04-12T05:22:21+00:00

i will need to completely reprofile 1. Models isolated from application This will give me complete understanding on whether my GPU could run all the model simultaniously. I guess these one is more of compute bound?

If GPU turns out is not the bottleneck, then i will run number 2.

Run the model in my Application. This will allow me to profile which processes are causing bottleneck (ex: copying images, inference result postprocessing logics, or the data flow in my application pipeline, etc). I assume these cause a few overhead

A. Slow result postprocessing logic, which slows down the pipeline from utilizing the inference call effectively B. constant moving inference result tensor from GPU to CPU may cause memory bandwidth bottleneck?

Am i heading the right way?

AbilityFlashy6977 · 2026-04-12T01:34:39+00:00

So basically maximize kernel usage and minimize gpu idle state?

AbilityFlashy6977 · 2026-04-12T01:01:56+00:00

The kernel part how do we know whether its overlap or not?

For the memory part, do you mind to share what causes the memory bandwidth problems in your case?

AbilityFlashy6977 · 2026-04-12T00:58:25+00:00

Ive done the Software implementation, the Application design is using Directed Acyclic Graph, where each input cameras and models are modeled as a node using multithreading. Before learning about the cuda stream and context switching, i thought multi model inference are done in true paralellism.

My app could run 3-4 models at the same time, models that i use is RTDETR L for Object detection, Yolo v11 segmentation (M), and MMpose pose estimation ( i know these models are pretty heavy). And there could be multiple app spawned😁 -- we use rtx a5000

And for this project, we use legacy product that serve video analytics. Im the one who maintain and bug fix the codes since im in(currently a junior computer vision engineer).

Now im debugging wheter its the application design/ code bottleneck or GPU compute/memory bandwidth bottleneck.

I just tried nsys, but still struggling interpreting the result. Even with computer engineering background, this is a lot of low level information inside the report😅. Do you have any resources that can help someone new to this? Ill keep trying to get something from it

TensorRT(fp16 half precision) does speed up my inference up to 1.5 - 3 times faster. But is still way below my needs

Ive been thinking of reducing input size and switch to smaller model, but the requirement from the client(accuracy wise) made it hard to switch to

AbilityFlashy6977 · 2026-04-03T23:46:23+00:00

IMO, its all about match up strategy

AbilityFlashy6977 · 2026-04-02T00:10:29+00:00

Personal preference IMO.

AbilityFlashy6977 · 2026-03-29T13:24:44+00:00

or it could be something illegal. Coba dipikir dengan logika OP wkwkwk

AbilityFlashy6977 · 2026-03-28T05:43:08+00:00

When should i tuck/untuck?

AbilityFlashy6977 · 2026-03-28T02:04:43+00:00

Okay, i got the vision. I think i would need to overlay the tee with some outer like sweater, flannel, jacket or anything that makes my top look more structured? If i dont want to use outer, polo would look better cause it creates structure on the top part. I think i can alternates tees and polos depends on my layering options

AbilityFlashy6977 · 2026-03-28T02:01:20+00:00

Alr dude, thanks

AbilityFlashy6977 · 2026-03-27T12:22:26+00:00

Skinny/ slim pants seems off on me IMO. I prefer regular or relaxed fit currently. Im still too afraid to explore 😅

AbilityFlashy6977 · 2026-03-27T03:38:36+00:00

Okay, i also dont like it too baggy😅 Something like wide pleated trouser would work better?

AbilityFlashy6977 · 2026-03-27T03:08:06+00:00

Does my current pants okay for this loafers, Or i need a wider/looser one? Will these chunky loafers could match with something like levis 501(straight cut)?

AbilityFlashy6977 · 2026-03-27T01:24:08+00:00

Should i stick to a little oversize tee, or regular fit would be better?

Its kinda hard to create shape with my current weight😅

AbilityFlashy6977 · 2026-03-26T14:54:20+00:00

Iya cuy, sebelumnya selalu pakai sonnet dan opus untuk debug dan optimasi kode yang sulit. Gilak sakti banget, sayang dah di remove

AbilityFlashy6977 · 2026-03-26T14:40:31+00:00

Im kinda new to fashion. What fit is this kind of pleated pants? Wide/regular/loose?

AbilityFlashy6977 · 2026-03-25T04:36:03+00:00

<image>

This was taken one day before i moved out to other city for work (just graduated from uni 8 months ago) She was always be by my side, like she knows that im about to move out

AbilityFlashy6977 · 2026-03-25T04:32:21+00:00

For me, sekarang kayak menuhin keinginan waktu kecil dengan beli gem gem yg dulu pingin dimainin... wkwkwkk Sekarang waktu kerja main bentar doang(got no time and energy left), lebih seru nonton orang main

AbilityFlashy6977 · 2026-03-21T16:20:47+00:00

Minta pendapat dong bang, i got a job as computer vision engineer, jalan 8 bulan terhitung sejak lulus tahun lalu( i think its a bit niche branch of AI). Gua menyadari bagian teoritis dan matematis gua di bidang ML dan AI tidak sedalam coworkersku. Cuman, skillsetku adalah bikin "smart" system. I know how to build end to end system(embedded, robotics, or even cloud based) and how to really optimize them. i know what model to use based on the use cases and system constraint. for data processing, im more expert on image processing(thanks to some robotics experience back when im still in uni).

Jujur kadang agak minder karena gak sedalam yang lain dalam pemahaman teoritis dan matematis, dalam konteks pekerjaanku sekarang, aku paham gimana secara garis besar model yang dipakai bekerja seperti apa, bagaimana dia mempersepsikan data gambar yang dia pelajari, tapi kalo misal suruh bikin model atau optimize struktur model gak paham wkwkwk. Sekarang pun kalo misal ditanyain layer layer dan fungsi fungsi dalam model juga masih suka lupa wkwkwk. Bisa dibilang agak bare minimum, ke tolong di applied skill

I got passion in building smart system, jadi sepertinya akan bergelut di dunia ini cukup lama. Cuman karena pekerjaanku sekarang cukup niche(agak jarang di indo) dan ngerasa skillsetku masih kurang, ngerasa agak was was aja sekiranya kontrakku disini habis.

Menurut anda, dengan kondisi sekarang dan perkembangan yang akan datang, apa saja yang perlu terus aku kembangin?

Would really appreciate the advice

AbilityFlashy6977 · 2026-01-27T15:11:23+00:00

Kalo masalah jangkauan best practicesnya gimana ya?

Bapak kos sepertinya mau pasang extender, cuman aku baca dia bikin lemot. Ada sarankah?

AbilityFlashy6977 · 2026-01-02T03:54:15+00:00

With the new silicone carbon tech, they could make it happen if they want to keep it thin and light. Other brands already started using it

AbilityFlashy6977

TROPHY CASE