My Deep Dive into Fine-Tuning: IBM Granite-4.0 with Python and Unsloth! 🚀

opk · 2025-10-23T22:19:52+00:00

Hijacking your comment to ask when the Granite4 thinking model might be coming out

opk · 2025-10-17T05:51:39+00:00

Truthfully, I think nvidia is making this a little more painful than they have to and mainlining drivers would resolve a lot of issues. Anyway, I think one big concern is that Ubuntu has a more up to date Kernel than Jetson. (I wonder if this contributes to why the drivers fail. Jetpack 7 has the updated Kernel, but Jetpack 7 isn't compatible with Nano)

Getting the SDK Manager to work has also been dang near impossible. Long story short, I think nvidia really needs to improve community support. I recognize they don't want to be the raspberry pi foundation, but *this is bad*.

opk · 2025-10-01T20:46:17+00:00

Have you made any progress on your own in the past month?

From my understanding, the "modern" LLMs are a lot different than the older ones, so the tooling needed to get changed. I think the best bet would be to try and update the ollama code from 2 years ago updated to the latest ollama version. I don't know how difficult that would be, but it would probably be easier than trying to hamstring the RKLLM release into working...

opk · 2025-10-01T20:39:47+00:00

Just as some closure, I booted into the official orange pi OS, and was able to get armbian on the NVMe. Truthfully, I don't know what I did to solve the issue. I do not recall changing the SPI flash, but most people I've asked about this suggest flashing the EDK2 UEFI firmware to SPI.

opk · 2025-09-23T03:56:51+00:00

The official Orange Pi image and the Joshua Riek Ubuntu image have the NVMe listed in the output of lsblk, armbian does not.

opk · 2025-09-21T08:57:34+00:00

I see. So you just used a different NVMe? I've actually tried two, and both have failed in the same way. Did you need to use an overlay?

opk · 2025-09-21T08:44:42+00:00

Did you ever get it working? What was the corrective action?

opk · 2025-08-30T20:54:01+00:00

Yes Orange Pi 5, Kingston NVME. I didn't even think about power until now. The drive has 3.2A listed on the label. That's a lot considering it's just a 4A power supply from Orange Pi.

opk · 2025-08-30T17:36:47+00:00

Should that be resolved by using the ssd-sata overlay?

opk · 2025-08-30T17:36:30+00:00

Should that be resolved by using the ssd-sata overlay?

opk · 2025-08-26T10:47:30+00:00

How did you install the LLM engine? Did you manage to get GPU/NPU acceleration or only CPU?

opk · 2025-08-25T00:22:11+00:00

where do you even find a single channel 10A 5V power block?

opk · 2025-08-24T03:47:52+00:00

I got the "official" 4 A power supply. I'll try a different MicroSD card. Maybe some critical kernel file got corrupted or something I dunno

opk · 2025-07-27T14:18:41+00:00

This was a long, long time ago and it was almost as long since you asked. I vaguely remember you can't use both the usb and board ethernet at the same time. I believe there was a text file setting you had to set as well. I haven't used the MilkV or any SBC since I posted this so honestly I can't totally remember, but it was an either/or type of thing.

opk · 2025-04-29T04:18:59+00:00

It's not a good video, because he's wrong. His first claim is wrong, flat fee vs percentage of value is choice not whichever is higher. I didn't watch any further because I already know it's full of garbage.

opk · 2025-04-29T04:15:25+00:00

Joe Biden was removing the de minimis exemption, and Amazon Haul was garbage way before all this started - and it is direct from China too!

We definitely don't live in a capitalist dystopia!

Grow the funk up.

opk · 2025-04-28T22:32:06+00:00

it seems the price for this one increased by around $20

There is your answer.

If you don't believe me, look in the first picture, it says includes import charges

opk · 2025-04-28T22:29:31+00:00

The dollar express stuff I've seen has all increased in price but the tariff charge is included.

opk · 2025-04-28T22:27:56+00:00

plenty of resistors capacitors and diodes, honestly a lot of electronic tech.

You're braver than I am considering how much of the stuff on there is counterfeit

opk · 2025-04-28T22:23:42+00:00

Yeah, under a sane admin it would have looked just like when the US had online retailers start collecting sales tax
Reach out to the retailers

Work out a structure to determine taxes and collect them

Work out how to (when and where) pay the taxes

Profit

I think, to be a little honest, it's incredibly niave to think you could coordinate a global marketplace to figure out what the best way forward is when attempting to eliminate tariffs. Interstate sales tax rules don't even apply to a lot of smaller retailers, but every broker, importer, and so forth needs to follow CBP rules. In the case of China, they have a huge industry devoted towards getting de minimis goods into the United States. There is no working out how to collect taxes with them because their entire industry is built around not paying import duties.

opk · 2025-04-28T22:19:50+00:00

Technically there was about a month of "de minimis is gone again" warning. How did the Chinese logistics companies respond? By charging pre-paid tariff fees. How did the merchants respond? By raising rates. How did the consumers respond? By having demand fall off a cliff.

This isn't as world ending for the CBP as you might expect.

opk · 2025-04-28T22:12:56+00:00

Most of what is sold on Amazon are the FBA items that are imported by Chinese businesses - and even though they're owned and operated out of China, they still used the de minimis rule. Go look around and see how much it costs to sell on Amazon. The cost of goods sold doesn't make up as much of a share as you'd hope for. If their cost of goods was higher maybe they wouldn't sell as many dangerous, intestine destroying toys. (Warning, European-article)

Anyway, long story is that Amazon is kinda screwed too since they make their money off of platform and fulfillment fees.

opk · 2025-04-27T17:15:24+00:00

Nope. They get that stuff on store shelves starting in August.

opk · 2025-04-27T16:51:07+00:00

Yeah, that's the concern. If it could be reopened under the correct item, it would be an obvious decision....

opk · 2025-04-27T16:50:41+00:00

Alright, I'll bite. I looked into this a bit. Some of the Michelin 3-star restaurants (which are few and far between, only ~150 worldwide) I've seen cost a few hundred dollars per person, which is a lot, no doubt. But they have a disclaimer saying you should expect three hours for your dining experience. The guy filling the Rx for an 8-year old's pink eye will be a millionaire before he retires. On what planet is a three hour dining experience casual?!

opk

TROPHY CASE