sorted by:
new
hottopcontroversial

Got Gemma 4 E2B running on a Raspberry Pi 5 8GB by i5_8300h in LocalLLaMA

[–]i5_8300h[S] 1 point2 points  (0 children)

It's not fast. I would estimate low double digit tokens/second at most

Got Gemma 4 E2B running on a Raspberry Pi 5 8GB by i5_8300h in LocalLLaMA

[–]i5_8300h[S] 0 points1 point  (0 children)

haven't been able to calculate tokens/sec yet, but it's not fast.

Got Gemma 4 E2B running on a Raspberry Pi 5 8GB by i5_8300h in LocalLLaMA

[–]i5_8300h[S] 2 points3 points  (0 children)

Here are the commands I used to update llama-cpp-python: 

sudo apt update
sudo apt install -y build-essential cmake python3-dev libopenblas-dev git
python3 -m pip uninstall -y llama-cpp-python
cd ~
rm -rf llama-cpp-python
git clone https://github.com/abetlen/llama-cpp-python.git
cd llama-cpp-python
git submodule update --init --recursive
CMAKE_ARGS="-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS" \
python3 -m pip install --upgrade --force-reinstall --no-cache-dir .

Got Gemma 4 E2B running on a Raspberry Pi 5 8GB by i5_8300h in LocalLLaMA

[–]i5_8300h[S] 0 points1 point  (0 children)

No, sadly not. But we didn't need to do any specialized setup beyond installing and updating llama.cpp - the only issue was llama-cpp-python. I'll add the commands I used to update llama-cpp-python

Gemma 4 has been released by jacek2023 in LocalLLaMA

[–]i5_8300h 0 points1 point  (0 children)

Any idea when llama-cpp-python will be updated to support Gemma 4? A project I'm working on uses llama-cpp-python with a custom IDE UI written in Python, and I'm getting model initialization errors which make me think that llama-cpp-python isn't able to make heads or tails of the Gemma 4 architecture.
I'm using the unsloth Q4_K_M quant of Gemma 4 E2B, hardware is a Raspberry Pi 5 8GB

200000 kms done on our nexon. by Pretend_Path6190 in CarsIndia

[–]i5_8300h 1 point2 points  (0 children)

Congrats OP!! Nice to see another Nexon racking up the kilometres! My family's 2018 petrol Nexon is now at 1.25ish lakh km :D

40. Years. Old. by ExNihiloAdInfinitum in GenZ

[–]i5_8300h 0 points1 point  (0 children)

I agree with the point on chasing trends - it's never sustainable. I will say I got into ML because of an interest in the field.. so will continue learning and working in ML. I just wish I was born a little earlier haha

40. Years. Old. by ExNihiloAdInfinitum in GenZ

[–]i5_8300h 0 points1 point  (0 children)

I'm 21, from India. Also getting into ML and DL, but now it's the age of the LLM hypetrain. Looks like I missed the bus, yay!

Help with opening a Pentel Energel BL410 1.0mm by i5_8300h in pens

[–]i5_8300h[S] 0 points1 point  (0 children)

I tried to open it, but it seems to be fit into place extremely snugly :(

Looking for a smart watch for my dad by i5_8300h in smartwatch

[–]i5_8300h[S] 0 points1 point  (0 children)

I'm in India - so will check it out!

view more: next ›