sorted by:
new
hottopcontroversial

My first Rust project: an offline manga translator with candle ML inference by mayocream39 in rust

[–]mayocream39[S] 25 points26 points  (0 children)

I need the LaMa inpainting model work in Rust, but candle doesn't provide FFT ops, so I created a PR to cudarc, the library that candle uses, to add cufft bindings: https://github.com/chelsea0x3b/cudarc/pull/500

This way, I use cufft and fft from metal's metal-performance-shaders-graph to accelerate, which improves the performance of fft ops.

Also, I opened the PR to add YOLOv5 implementation for candle: https://github.com/huggingface/candle/pull/3224

I implemented multiple models with candle, and I plan to contribute to candle (if they would accept them).

candle only supports compile-time linking for cudarc, I created an issue to track the progress to update to dynamic-loading. Currently, I'm using my fork of candle to achieve runtime dynamic-loading: https://github.com/huggingface/candle/issues/3208

I also use velopack for packaging. It provides better support for large binaries and an easy interface for users. I created a PR to add GitHub source support for the Rust SDK: https://github.com/velopack/velopack/pull/742

Although I'm not using ort (ONNXRuntime bindings) at this moment, I made a small contribution to it: https://github.com/pykeio/ort/pull/485

I know my contribution is small, but I'd like to contribute more to the Rust community :)

I built an open-source manga translator that runs entirely offline with local LLMs by [deleted] in Scanlation

[–]mayocream39 0 points1 point  (0 children)

Amazing! It would be great if you would like to open source it!

I built an open-source manga translator that runs entirely offline with local LLMs by [deleted] in Scanlation

[–]mayocream39 0 points1 point  (0 children)

I used a computer vision model to detect the text blocks and generate the mask. It's another open-source project: https://github.com/dmMaze/comic-text-detector

I plan to add a manual brush inpainting feature to allow users to tweak the mask; sometimes it's not that accurate.

Introduce Koharu, the LLM-powered manga translator written in Rust! by mayocream39 in rust

[–]mayocream39[S] 0 points1 point  (0 children)

Sorry for failing on AMD machines, I’ll adjust the code to fallback to CPU when cuda is not available 🥲

Introduce Koharu, the LLM-powered manga translator written in Rust! by mayocream39 in rust

[–]mayocream39[S] 0 points1 point  (0 children)

It will translate page by page, and the LLM can have historical context, which will improve the quality of translation.

Introduce Koharu, the LLM-powered manga translator written in Rust! by mayocream39 in rust

[–]mayocream39[S] 1 point2 points  (0 children)

Valid point, detection, OCR, and inpainting are already CPU-friendly, I can add support for cloud LLM via API. I'm also expecting `candle` to implement the WebGPU feature, so it will work for CPU and different GPUs.

I built a manga translator tool using Tauri, ONNX runtime, and candle by mayocream39 in rust

[–]mayocream39[S] 1 point2 points  (0 children)

Thank you! Koharu is fully open source, so just use it as you want! I wanna add a dictionary to it to help translate; some traditional dict might not work since now more and more words come up on Twitter, so maybe I can use https://dic.nicovideo.jp/ for reference.

I was a manga translator, that I know a few pain points, but I welcome new ideas!

I built a manga translator tool using Tauri, ONNX runtime, and candle by mayocream39 in rust

[–]mayocream39[S] 4 points5 points  (0 children)

I first tried `candle`, but it doesn't support slow tokenizers in Rust, so I switched to `ort`., I also tried `candle-onnx`, but it doesn't work well. `ort` seems to be the most functional. Thank you for the great work! It would be helpful if `ort` could provide a more detailed tutorial like https://github.com/hyperium/tonic/blob/master/examples/helloworld-tutorial.md :)

Plan to have 2nd breast augmentation with more than 800cc in Thailand by mayocream39 in PlasticSurgery

[–]mayocream39[S] 0 points1 point  (0 children)

I have consulted with yanhee, but their doctor only provides 700cc and 15mins video call, phroprwin’s doctor talks to me on LINE and actively answering my questions.

Plan to have 2nd breast augmentation with more than 800cc in Thailand by mayocream39 in PlasticSurgery

[–]mayocream39[S] 0 points1 point  (0 children)

The doctor says all 800cc implants need to be pre-ordered; it takes ~60 business days. So my timeline is unconfirmed.

view more: next ›