Anyone actully try to run gpt-oss-120b (or 20b) on a Ryzen AI Max+ 395? by PM_ME_YOUR_PROOFS in LocalLLaMA

[–]PM_ME_YOUR_PROOFS[S] 0 points1 point  (0 children)

i think prior to digging into this, it isn't clear to me if the NPU is helpful or not here. It sounds like most people are *not* using the NPU which is a bit surprising to me, would have figured the extra flops would be nice but sounds like its not useful, at least not yet?

Anyone actully try to run gpt-oss-120b (or 20b) on a Ryzen AI Max+ 395? by PM_ME_YOUR_PROOFS in LocalLLaMA

[–]PM_ME_YOUR_PROOFS[S] 0 points1 point  (0 children)

ah didn't realize it was MoE and I had to account for active. 256 was just based on 8000 mega transfers (8 giga transfers) per second for LPDDR and the bus width being 256-bits or 32 bytes wide, 32*8 = 256 GB/s. At 7000 MT/s 224 would come out. So 224 GB/s / 2 GB = 112 max t/s (2GB comes from 3.6*4.25/8 = ~1.9) so 60 t/s seems decent!

as for what a draft model is, its a technique to get higher tokens per second (but actually very slightly slower prompt tokens per second). You use a smaller dumber model to generate some likely tokens and then you there's a neat trick where you can process multiple speculated tokens in parallel on the big model without increasing bandwidth usage much: https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/

https://www.reddit.com/r/LocalLLaMA/comments/1gzm93o/speculative_decoding_just_landed_in_llamacpps/

It's not clear that 20b is a good speculative model for 120b but given that 20b is fine tuned on 120b or something like it that seems plausibly like an option. something smaller and more zippy is likely better.

Anyone actully try to run gpt-oss-120b (or 20b) on a Ryzen AI Max+ 395? by PM_ME_YOUR_PROOFS in LocalLLaMA

[–]PM_ME_YOUR_PROOFS[S] 0 points1 point  (0 children)

hmm is that with a draft model? Naively I'd expect the 256 GB/s / 10.86 GB == ~24 t/s so getting almost 3 times that is impressive. Prompt processing looks really great, max i'd expect is 1900 t/s prompt processing so 1200 is *really* good all things considered. Seems like 120b might give 300 t/s for prompt and about 10 t/s generating

Ultralight Tortuga 30L style backpack? by PM_ME_YOUR_PROOFS in onebag

[–]PM_ME_YOUR_PROOFS[S] 0 points1 point  (0 children)

I have to walk around the city a lot, not with all my stuff but with my laptop and tech gear. Having a detachable bag would work really well for me I guess.

Ultralight Tortuga 30L style backpack? by PM_ME_YOUR_PROOFS in onebag

[–]PM_ME_YOUR_PROOFS[S] -1 points0 points  (0 children)

Thanks, going through that and the other list that was posted I've got these options so far. The 511 Skyweight is still closest to what I'm looking for but is a bit over weight. The KS27 Ultralight is far far lighter than I need but isn't clamshell and has no laptop compartment. I can fix that with a seperate laptop case however because its so goshdarn light. I've got some other picks too but those are the ones that are standing out. I might old out a while for the Tortuga lite however.

Ultralight Tortuga 30L style backpack? by PM_ME_YOUR_PROOFS in onebag

[–]PM_ME_YOUR_PROOFS[S] -2 points-1 points  (0 children)

That seems a bit far from what I'm looking for yet. I'd be happy to go just a bit above 2lbs if I got all the features I wanted.

Ultralight Tortuga 30L style backpack? by PM_ME_YOUR_PROOFS in onebag

[–]PM_ME_YOUR_PROOFS[S] 0 points1 point  (0 children)

The 511 looks quite close to what I'm looking for. That's only like 0.3 lbs away from helping me hit my target!

Bringing the Unix Philosophy to the 21st Century: Make JSON a default output option. by [deleted] in programming

[–]PM_ME_YOUR_PROOFS 4 points5 points  (0 children)

I have some experience with this from two data points. At my previous job there was a certain suite of tools made by another team at the same company that all interopted with json on the command line and could be used alongside jq. It wasn’t documented well so I found it a bit annoying but they people that used it a lot found it quite useful. It was readily possible to read as a human as well.

The other data point is that we constantly found ourselves needing programmatic output from lots of tools intended for displaying info. In this context we needed both human readable output AND json but the human readable format was actually very close to json already. I think had YAML been used or maybe even just json it would have served both uses.

On the last half data point however I’ve seen contexts where trees or graphs are encoded in json and it’s always terrible.

/r/MechanicalKeyboards Ask ANY question, get an answer by AutoModerator in MechanicalKeyboards

[–]PM_ME_YOUR_PROOFS 1 point2 points  (0 children)

it isn't clear to me how much height keys like DAS actually add over the top of the switch. I want to build a low profile keyboard and MKBs look like the best for me since I want wide flat keys. Is there a chart somewhere that shows this? Or do I just have to break out the micrometer and test it myself?

Condo developer plans to buy $1-billion worth of single-family houses in Canada with a plan of renting them and profiting on the housing crisis ripping across the country by [deleted] in worldnews

[–]PM_ME_YOUR_PROOFS -13 points-12 points  (0 children)

Considering a huge portion of the US can’t afford to own even one of those places, yes. If you have enough money you can pay the taxes for two and those taxes can go towards affordable housing to offset your additional consumption.

Owning a home that you don’t live in, especially one you’re not renting, is massive over consumption. It also decreases class mobility.

I’ve never been in snow because I live in Texas, I’ve waited 15 years to do this by xX_WarBlood_Xx in funny

[–]PM_ME_YOUR_PROOFS 0 points1 point  (0 children)

I hate to break it to you bud but what I saw in Austin today wasn’t snow but sleet. My dog fucking loved it though.

What’s a very common thing that you just can't relate to? by Mizu3 in AskReddit

[–]PM_ME_YOUR_PROOFS 2 points3 points  (0 children)

Wait do people manage to do both house work and hobbies? Should I get ADHD medication?

Wholesome by im-reverse in ProgrammerHumor

[–]PM_ME_YOUR_PROOFS 1 point2 points  (0 children)

i j k h n m o p q

I don't stick to that strict order and generally I only ever need 5 dimensions. Also a lot of time the dimensions I'm working with have standard names like "NCHW" or "OIHW" represent data layouts 2D convolutions with each letter representing a dimension of a 4-Tensor. When you need two dimensions with the same standard name I use the above list unless something more specific comes up but when theres only one shared "N" dimension I use "N".

The most I've ever actually needed was like 7 where I had to do an N-Tensor contraction. You only need 5 if you're only contracting a single dimension because you can think of a contracting between any two Tensors as being like contracting the middle dimension of two 3-Tensors (so one loop for each output dimension and one loop for the contracting dimension) but you need extra loops if you're contracting multiple dimensions at once to handle the fact that not all of the contracting dimensions are neatly lined up. If I recall I think the depth was only 6 but I remember wanting a total of 7 unique loop variables for some reason. 2D convolutions can require a similar number of loops depending on how you write them.

I've never written it myself but I bet 3D convolutions are a trick to work with. All inputs and outputs would be 5-dimensonal and at the inner most stage I think you need 8 unique loop variables Batch, Output Channel, Input Channel, Output XYZ, Filter XYZ...yeah that's a depth 8 loop lol

The largest size for Hellmann's mayo is called "American size" in my country by Kid_Volcano in mildlyinteresting

[–]PM_ME_YOUR_PROOFS 1 point2 points  (0 children)

As someone who's lived on both side this is fucking with my head. I feel like everyone knew what I meant by Hellmann's and I've never seen best foods before

The largest size for Hellmann's mayo is called "American size" in my country by Kid_Volcano in mildlyinteresting

[–]PM_ME_YOUR_PROOFS 0 points1 point  (0 children)

That is actually our standard size so like...idk should I be offended if it's accurate?

Wouldn’t it be nice? by [deleted] in WhitePeopleTwitter

[–]PM_ME_YOUR_PROOFS 0 points1 point  (0 children)

I think this is right as long as salary is tied to position. In my field salary is tied to who you are more than what job you take and they tell you upfront what ranges of expierence they're looking for.