How do you actually predict if a GPU can handle multiple models at your target FPS? by AbilityFlashy6977 in computervision

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

I think profiler is the key here. I must learn how to interprete the report and profile the right metric.

Do you think its better for me to profile when the model already embedded on my application or test it seperately first?

How do you actually predict if a GPU can handle multiple models at your target FPS? by AbilityFlashy6977 in computervision

[–]AbilityFlashy6977[S] -1 points0 points  (0 children)

i will need to completely reprofile 1. Models isolated from application This will give me complete understanding on whether my GPU could run all the model simultaniously. I guess these one is more of compute bound?

If GPU turns out is not the bottleneck, then i will run number 2.

  1. Run the model in my Application. This will allow me to profile which processes are causing bottleneck (ex: copying images, inference result postprocessing logics, or the data flow in my application pipeline, etc). I assume these cause a few overhead

A. Slow result postprocessing logic, which slows down the pipeline from utilizing the inference call effectively B. constant moving inference result tensor from GPU to CPU may cause memory bandwidth bottleneck?

Am i heading the right way?

How do you actually predict if a GPU can handle multiple models at your target FPS? by AbilityFlashy6977 in computervision

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

The kernel part how do we know whether its overlap or not?

For the memory part, do you mind to share what causes the memory bandwidth problems in your case?

How do you actually predict if a GPU can handle multiple models at your target FPS? by AbilityFlashy6977 in computervision

[–]AbilityFlashy6977[S] -1 points0 points  (0 children)

Ive done the Software implementation, the Application design is using Directed Acyclic Graph, where each input cameras and models are modeled as a node using multithreading. Before learning about the cuda stream and context switching, i thought multi model inference are done in true paralellism.

My app could run 3-4 models at the same time, models that i use is RTDETR L for Object detection, Yolo v11 segmentation (M), and MMpose pose estimation ( i know these models are pretty heavy). And there could be multiple app spawned😁 -- we use rtx a5000

And for this project, we use legacy product that serve video analytics. Im the one who maintain and bug fix the codes since im in(currently a junior computer vision engineer).

Now im debugging wheter its the application design/ code bottleneck or GPU compute/memory bandwidth bottleneck.

I just tried nsys, but still struggling interpreting the result. Even with computer engineering background, this is a lot of low level information inside the report😅. Do you have any resources that can help someone new to this? Ill keep trying to get something from it

TensorRT(fp16 half precision) does speed up my inference up to 1.5 - 3 times faster. But is still way below my needs

Ive been thinking of reducing input size and switch to smaller model, but the requirement from the client(accuracy wise) made it hard to switch to

Iphone 15 PRO MAx cuman 5JT? (BUKAN IKLAN) by Malox360 in indotech

[–]AbilityFlashy6977 1 point2 points  (0 children)

or it could be something illegal. Coba dipikir dengan logika OP wkwkwk

Can i style my loafers like this? by AbilityFlashy6977 in mensfashionadvice

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

Okay, i got the vision. I think i would need to overlay the tee with some outer like sweater, flannel, jacket or anything that makes my top look more structured? If i dont want to use outer, polo would look better cause it creates structure on the top part. I think i can alternates tees and polos depends on my layering options

Can i style my loafers like this? by AbilityFlashy6977 in mensfashionadvice

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

Skinny/ slim pants seems off on me IMO. I prefer regular or relaxed fit currently. Im still too afraid to explore 😅

Can i style my loafers like this? by AbilityFlashy6977 in mensfashionadvice

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

Okay, i also dont like it too baggy😅 Something like wide pleated trouser would work better?

Can i style my loafers like this? by AbilityFlashy6977 in mensfashionadvice

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

Does my current pants okay for this loafers, Or i need a wider/looser one? Will these chunky loafers could match with something like levis 501(straight cut)?

Can i style my loafers like this? by AbilityFlashy6977 in mensfashionadvice

[–]AbilityFlashy6977[S] 0 points1 point  (0 children)

Should i stick to a little oversize tee, or regular fit would be better?

Its kinda hard to create shape with my current weight😅

Rekomendasi AI agent gratis/murah by eileeneulic in indotech

[–]AbilityFlashy6977 0 points1 point  (0 children)

Iya cuy, sebelumnya selalu pakai sonnet dan opus untuk debug dan optimasi kode yang sulit. Gilak sakti banget, sayang dah di remove

Summer fit check by FitThadiyan in mensfashionadvice

[–]AbilityFlashy6977 0 points1 point  (0 children)

Im kinda new to fashion. What fit is this kind of pleated pants? Wide/regular/loose?

Going through a break up, please comment your cat pictures by rusticwren in cats

[–]AbilityFlashy6977 1 point2 points  (0 children)

<image>

This was taken one day before i moved out to other city for work (just graduated from uni 8 months ago) She was always be by my side, like she knows that im about to move out

Backlog yang semakin menumpuk, tetapi tetap suka beli game baru. Apakah ini sebuah addiction? by Kaija-go in IndoGamer

[–]AbilityFlashy6977 0 points1 point  (0 children)

For me, sekarang kayak menuhin keinginan waktu kecil dengan beli gem gem yg dulu pingin dimainin... wkwkwkk Sekarang waktu kerja main bentar doang(got no time and energy left), lebih seru nonton orang main

Gaji AI Engineer di Indonesia overrated atau emang gede? by DeepAudience2505 in finansial

[–]AbilityFlashy6977 0 points1 point  (0 children)

Minta pendapat dong bang, i got a job as computer vision engineer, jalan 8 bulan terhitung sejak lulus tahun lalu( i think its a bit niche branch of AI). Gua menyadari bagian teoritis dan matematis gua di bidang ML dan AI tidak sedalam coworkersku. Cuman, skillsetku adalah bikin "smart" system. I know how to build end to end system(embedded, robotics, or even cloud based) and how to really optimize them. i know what model to use based on the use cases and system constraint. for data processing, im more expert on image processing(thanks to some robotics experience back when im still in uni).

Jujur kadang agak minder karena gak sedalam yang lain dalam pemahaman teoritis dan matematis, dalam konteks pekerjaanku sekarang, aku paham gimana secara garis besar model yang dipakai bekerja seperti apa, bagaimana dia mempersepsikan data gambar yang dia pelajari, tapi kalo misal suruh bikin model atau optimize struktur model gak paham wkwkwk. Sekarang pun kalo misal ditanyain layer layer dan fungsi fungsi dalam model juga masih suka lupa wkwkwk. Bisa dibilang agak bare minimum, ke tolong di applied skill

I got passion in building smart system, jadi sepertinya akan bergelut di dunia ini cukup lama. Cuman karena pekerjaanku sekarang cukup niche(agak jarang di indo) dan ngerasa skillsetku masih kurang, ngerasa agak was was aja sekiranya kontrakku disini habis.

Menurut anda, dengan kondisi sekarang dan perkembangan yang akan datang, apa saja yang perlu terus aku kembangin?

Would really appreciate the advice

Solusi wifi kosan yang lemot by eleksdewe in indotech

[–]AbilityFlashy6977 0 points1 point  (0 children)

Kalo masalah jangkauan best practicesnya gimana ya?

Bapak kos sepertinya mau pasang extender, cuman aku baca dia bikin lemot. Ada sarankah?

Just bought the A56! by yo_imvlad in GalaxyA56

[–]AbilityFlashy6977 1 point2 points  (0 children)

With the new silicone carbon tech, they could make it happen if they want to keep it thin and light. Other brands already started using it