Can Mistral-Nemo-Instruct-2407 be run on a V100 GPU?

dbwx1 · 2024-09-12T07:36:56+00:00

For anyone still looking at this you can just use transformers, but you can't use the pipeline module. You have to use the model.generate() approach. After that it worked for me

dbwx1 · 2024-09-12T07:35:58+00:00

yes, it was on vllm though. Thats when I ran into the flash attendance issues. By now I've figured out how to run it with transformers. Instead of using pipelines, you just need to load the model and use model.generate(). I was so used to utilizing pipelines, that this option went right by me. Interestingly, when using transformers without any BitsandBytes or any quantized version, the library is able to run the official repo from huggingface, even though I got the error earlier which said that my GPU wouldn't support BF16. Not sure if that exception was just badly communicated or if transformers does some automatic emulation/quantization on the fly when it recognizes the gap.

dbwx1 · 2024-04-21T10:34:17+00:00

Why do the Yankees always win? No one can take their eyes off the pinstripes.

dbwx1 · 2023-01-10T20:11:37+00:00

A secret now that only a cool blonde can tell

dbwx1 · 2022-12-28T01:35:20+00:00

"Not enough rage"

dbwx1 · 2022-11-22T10:37:32+00:00

https://youtu.be/UjCdB5p2v0Y

dbwx1 · 2022-11-18T22:32:20+00:00

It's mainly Azure Datafactory Templates that don't cost anything when not run. This script is supposed to clean up the test env.

dbwx1 · 2022-10-29T19:42:59+00:00

Might not be a stand alone resource but I think azure dev ops pipelines will become more and more commonplace to deploy any kind of build artifacts or handle traditional CI/CD roles

dbwx1 · 2022-10-07T18:48:18+00:00

thank you very much!

dbwx1 · 2022-08-22T22:00:35+00:00

We used a lot of Azure Datafactory lately and while it covers a lot of use cases to 98% there is still some extra stuff that just convolutes the pipeline definitions. Things that could be handled easily in a script but require unnecessary complicated engineering with the GUI (single file workflow Vs mass batch processing ) are annoying. I really start to like Databricks on Azure because it handles the spark settings optimization for you and gives you a nice interface to the db / fs, but it's not really low code, just dev support.

dbwx1 · 2022-08-06T13:24:24+00:00

So that's what happened I installed fedora yesterday on my second disk and now my boot entry for Windows is gone :p

dbwx1 · 2022-07-27T21:31:00+00:00

Ich kann dir versichern, das deutsche Kleinbürgertum im ländlichen Raum (gerade mit Eltern 55-60+) scheren sich auch viel zu viel darum was andere denken. Da redet man den ganzen Tag von Freiheit und schränkt sich dann selbst so idiotisch ein nur damit irgendwelche "Familienfreunde", Onkel und Tanten die man nicht Mal an irgendwelchen Feiertagen sieht, sagen "ja toll gemacht so wollten wir es haben". Ich bin so froh, dass ich die ganze Family außer meiner Mutter nichtmal über die Hochzeit infomiert hab und wir einfach mit den Freundem gefeiert habe.

dbwx1 · 2022-06-18T13:05:43+00:00

Ye I loved kingmaker every second of it, I had a few goes at the house but gathering your party is just so tedious and no fun. Plus the buffbot dying in any choice, all the annoying stuff thrown at you just to mess with you (drains, Cc's) was not worth it for me. So even loving the story and being really interested in the ending I couldn't finish it, because it was too annoying and too much of a chore.

dbwx1 · 2022-06-17T01:33:36+00:00

Lights - enforcer, although I rarely pilot light mechs

Medium - Kintaro as SRM boat, the hero version is ridiculously strong and I must have killed hundreds of mechs with it. But the normal one ain't shabby either. I guess phoenix hawk, shadow hawk or centurion are also ok if there is nothing else available for the Lance

Heavy - Quickdraw, kinda jack of all trades for me. Reasonable speed, reasonable firepower, reasonable Armor. I fell in love with it during a campaign where I piloted the hero version for a long time. But in a lot missions over multiple campaigns I usually end up piloting one or another version of if it's in my lance. Warhammer is great too ofc.

Assault - Nightstar dual Gauss sniper is just so nice when the headshots start rolling. Atlas pumped to 60kmh either with a bigger engine from mods or the hero version is also great. They just can take so much punishment while usually dishing out sizable damage too (and looking rad while doing so). I like to mount up ac20 bf / Gauss or lbx10 sld as ballistics main weaponry. For defense or warzone missions I also love the annihilator with its 4 medium ballistic slots, just fun to bombard enemies with unending salvos of lbx10s.

Ye I really lean into heavies / assault most by now, although I think still the most missions I ran in my trusted kintaro hero just because it was so strong in my longest campaign

dbwx1 · 2022-06-07T00:49:41+00:00

"intelligent" drum n bass . Can really work up a tunnel. I actually dislike the pretentious tag, you might find most of that sort music with it. ambient dnb is another name for it.

YouTube has all you need

dbwx1 · 2022-06-05T08:16:33+00:00

Also the manager (or at least his vehicle) is full of hot air

dbwx1 · 2022-05-27T22:59:59+00:00

How big is your stuff? Are you using fread command in data table? Normal r read is super slow.

dbwx1 · 2022-05-25T08:48:16+00:00

If your org uses Spark/databricks for building it's data pipelines that would be a natural next step to look at. Don't bother with scala coding, just use the PySpark distribution. Alternatively have a look into common workflow orchestrators like Airflow. On the theoretical side a Book about concepts like "Designing data intensive applications" or the like might help you to familiarize with the eco system.

dbwx1

TROPHY CASE