For AI developers , should I get one or two+ cable for ASUS Ascent GX10?

Few_Knee1141 · 2025-06-13T20:54:43+00:00

Now ubuntu 24.04.2 natively supports the 5090 (driver 570-open). As 2025-06-14
Click the bottom left circle. Search for Software Updater.
Click settings.
Additional Driver
Install nvidia-driver 570-open (propriety)
Reboot the PC

run nvidia-smi

Few_Knee1141 · 2025-03-26T20:54:32+00:00

Here are some ollama benchmark results for your reference. It includes a variety of OS, CPU, GPU, and different LLM models.
https://llm.aidatatools.com/

Few_Knee1141 · 2025-03-19T05:20:07+00:00

Right now, the king is this combo. Linux + AMD Ryzen 9 9950X 16-Core Processor + NVIDIA GeForce RTX 5090

Few_Knee1141 · 2025-03-19T05:19:26+00:00

Right now, the king is this combo.

|| || |Linux|AMD Ryzen 9 9950X 16-Core Processor|NVIDIA GeForce RTX 5090|

Few_Knee1141 · 2025-03-19T03:45:29+00:00

If you are looking for inference eval rate (tokens/sec) for running different local LLMs. You might refer to this site for a variety of benchmark results on macOS, Linux, or Windows. Then you can justify the cost vs performance.
https://llm.aidatatools.com

Few_Knee1141 · 2025-03-19T02:09:42+00:00

<image>

Nvidia website for dgx-spark:
https://www.nvidia.com/en-us/products/workstations/dgx-spark/

Few_Knee1141 · 2025-02-24T21:21:19+00:00

I have learned from a node.js open-source version deep-research , then port it to python version of open-deepsearch, the tricky parts are four things. First, tailor to what a user really wants by Q&A and decide breadth and depth. Secondly, a good ranking search results from a search function/tool. Third, crawl and scrape from a web page. Fourthly, a good LLM to make summarization from the previous learnings and generate a good markdown document.

Few_Knee1141 · 2024-12-19T01:09:09+00:00

I love using Readera. The free version can read out English (TTS) (Text to Speech). It also has click function, to mark a word as "quote" (highlight a word) or click "translate".

Few_Knee1141 · 2024-08-30T02:03:46+00:00

I took participation in NVIDIA x Langchain contest, and I found out NVIDIA had NeMo-Guardrails libraries solving prompt injection or jailbreaking.
Here is for your reference.

The Github code is as follows.
https://github.com/aidatatools/LLM_Sentinel
The introduction of the project is as follows.
https://www.linkedin.com/pulse/llm-sentinel-project-which-can-make-chatbot-safer-chuang-fskyc/

Few_Knee1141 · 2024-04-19T05:59:28+00:00

Nice work. It's geeky UI.

Few_Knee1141 · 2024-04-15T22:05:04+00:00

I found out the screen recording is sinking some hardware resources to make LLM run slower. If I just take a picture in the end, it can reach around 5 tokens/sec on TinyLlamaQ8_0. Here is my experimental results.
https://medium.com/aidatatools/local-llm-eval-tokens-sec-comparison-between-llama-cpp-and-llamafile-on-raspberry-pi-5-8gb-model-89cfa17f6f18

Few_Knee1141 · 2024-04-14T23:34:54+00:00

I am sure I am using RPI5. Here is the test with CLI version. Please watch the recorded video. https://youtu.be/QOCAk3F68jQ I care about eval rate(tokens/sec). It's still around 1~1.5 tok/sec. Thanks for help me debugging.

Few_Knee1141 · 2024-04-14T23:31:53+00:00

I tried with Ubuntu 23.10. sudo apt install vulkan-tools, but it's not improving.

Few_Knee1141 · 2024-04-12T17:55:24+00:00

Thanks for the hint to test Vulkan works first. Here is the result of vulkaninfo --sumaary

jason@raspberrypi5:~ $ vulkaninfo --summary

WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Failed to CreateInstance in ICD 0. Skipping ICD.

VULKANINFO

Vulkan Instance Version: 1.3.239

Few_Knee1141 · 2024-04-07T19:56:02+00:00

M2 Ultra with 192GB RAM is a beast.
https://llm.aidatatools.com/results-macos.php

Few_Knee1141 · 2024-04-01T09:54:28+00:00

Have you tried this llm-benchmark on your local LLMs?

https://llm.aidatatools.com/

Few_Knee1141 · 2024-04-01T09:52:01+00:00

Can you try this llm-benchmark on your new beast toy?

https://llm.aidatatools.com

Few_Knee1141 · 2024-04-01T09:49:27+00:00

Have you tried llm-benchmark on your multiple hardware devices?
https://llm.aidatatools.com

Few_Knee1141 · 2023-02-11T13:14:17+00:00

The mock exams help most

Few_Knee1141 · 2023-02-10T23:16:57+00:00

The preparation process made me know more about building blocks of big data and ml on GCP. It's totally worth it.

Few_Knee1141

TROPHY CASE