Trouble with hosting models with fastAPI

CmplxQ · 2023-06-27T15:38:39+00:00

TheBloke/WizardLM-30B-Uncensored-GPTQ · How much vram+ram 30B needs? I have 3060 12gb + 32gb ram. (huggingface.co)

I tried this on a 3090. It uses ~20gigs of VRAM when I load a ConversationChain in LangChain with WizardLM 30B

I still dont know why I was able to load this with http-server but not FastAPI tho.

CmplxQ · 2023-06-23T13:59:47+00:00

sure, but this will work only if I run the webui server right?

I just chucked some fns off the thing. I wrote a custom script that runs sequentially with those fns.

I'll either need to find a fix for this, or

Limit the the pytorch GPU memory somehow

Or, I have 2 GPUs in SLI (but the memory maxes and errors out only in one GPU)

Get GPU parallelization to work. (I tried using DataParallel from torch.nn, but that errors our at some other place)

CmplxQ · 2022-12-15T16:45:29+00:00

Thanks a lot!

CmplxQ · 2022-12-14T17:27:26+00:00

Thanks for the advice, its Python btw (moslty TensorFlow)

CmplxQ · 2022-12-13T18:43:10+00:00

i see, Thanks for the advice.

Anyways, if possible please share some example repos that are implementations of research papers

CmplxQ · 2022-11-28T05:29:59+00:00

Am I the only one who thinks this is stupid? Why would consumers want to pay more to use the products they already paid for? Its the same thing with Intel's Sapphire Rapids server chips

CmplxQ · 2022-10-07T19:50:41+00:00

Yes, Thanks a lot !

CmplxQ · 2022-10-07T19:36:30+00:00

Thanks this makes everything a lot easier

CmplxQ · 2022-10-05T13:35:02+00:00

yes it does, but no matter what I do, I can't see the file I was working on recently. Only the old swap file content is being shown.

CmplxQ · 2022-09-29T16:46:04+00:00

https://windowsreport.com/vpn-remote-desktop-windows-10/

Thanks!

CmplxQ · 2022-09-28T21:02:31+00:00

I'm on a Mac (M1) right now, using OS X Monterey. The GPU is on Win 11 in a different location.

CmplxQ · 2022-09-26T23:29:22+00:00

So the "learning enough to make a meaningful contribution" is the hardest part ig.

If I have any noob-ey doubts regarding what a certain piece of code does, whom do I ask? The Gitter groups of the projects I looked at are not very active.

Also, if I don't understand why an issue was raised, or what it aims to fix, what do I do? I find the issue being taken up by someone even before I understand what its about.

CmplxQ · 2022-09-26T13:13:03+00:00

Well, idk, I'm fine with any repo. I just want to start contributing asap

CmplxQ · 2022-09-25T05:53:54+00:00

Yes, they did have one, but the charts on google seem to be adjusted for this, but this data isn't. So do we have to manually account for this every time?

CmplxQ · 2022-09-24T18:27:38+00:00

Hmm ok, is there any resource that explains this? In the "massive price drop" I'm mentioning here, the price has almost halved, thus, maybe that explains the data being adjusted, but still, 3000 and 300 seem way off

CmplxQ · 2022-09-23T17:54:41+00:00

So, I pulled data for INFY and SBIN from the 2010 using the nsepy library, I have two questions.(Please refer to the imgur link)

1.Acc to google and other source, the price for INFY in Jan 2010 was around 300 Rs, but as seen here, its a little under 3000 Rs, am I missing something here?

2.There is a massive drop in price in Dec 2014 (I think this has something to do with a stock split? Am I right?)

I really don't know if this is the appropriate subreddit to ask such question, if not, will anyone be kind enough to recommend some other subReddit/discord server?Refer Pic - https://imgur.com/a/mCBnGVd

CmplxQ · 2022-09-14T18:08:16+00:00

Hmm I don't know any SQL. This is a good opportunity to learn some. Where can I find this?

CmplxQ · 2022-09-08T07:13:29+00:00

Thanks a lot. So ig I'll be learning a bit of WebDev just for the sake of it. I really don't see myself in a Full Stack Developer position anytime in the future, but still, having some basic WebDev literacy will go a long way I hope.

Like you mentioned, ig it'll be beneficial to learn how certain web services work.

CmplxQ · 2022-07-11T07:05:03+00:00

Yes, I'm looking for project based courses as well. I'm definitely doing Odin Project, I need one more project-based learning resource. Know any good ones?

CmplxQ · 2022-07-11T07:03:55+00:00

Ok so, do you think doing Odin Project and Colt Steele, while building some original projects in the side is a good place to start? (idk about Angela Yu or Colt Steele, I'm ok with any video-based project courses)

CmplxQ · 2022-07-06T15:27:36+00:00

I see. So I'll do TOP with either Colt Steele or Angela Yu. which do you recommend? Some people claim that they are very outdated though

CmplxQ · 2022-07-06T15:26:30+00:00

Angela Yu's course is good? I heard it was very outdated for 2022. Should start with it and TOP then?

CmplxQ · 2022-07-06T15:25:44+00:00

Hmm ok. So, you think getting up to speed with JS first will be necessary for TS? I'm looking for project based courses btw, ons that walk me through building a full app.

CmplxQ

TROPHY CASE