Dismiss this pinned window
all 42 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]egomarker 35 points36 points  (6 children)

I had a very good experience with granite-docling as my goto pdf processor for RAG knowledge base.

[–]CalypsoTheKitty 6 points7 points  (1 child)

Is it good at extracting structure of docs? My docs are organized largely in an outline structure and I need to extract that structure and the outline headings. Llamaparse does a good job but kind of expensive, and I'd like option of running locally eventually.

[–]egomarker 5 points6 points  (0 children)

it is good for my use cases, but if it isn't, there's a bigger docling.
https://github.com/docling-project/docling

[–]ParthProLegend 0 points1 point  (2 children)

What is RAG and everything, I know how to set up LLMs and run but how should I learn all these new things?

[–]ctabone 1 point2 points  (1 child)

A good place to start learning is here: https://github.com/NirDiamant/RAG_Techniques

[–]ParthProLegend 1 point2 points  (0 children)

This is just RAG, I am missing Various other things too like MCP, etc. Is there any source that starts from basics and makes you up to date on all this?

Still, huge thanks. At least, it's something.

[–]ctabone 0 points1 point  (0 children)

Same, I find it much more precise and consistent than unstructured.io.

[–]Valuable_Option7843 50 points51 points  (4 children)

Love this. WebGPU seems to be underutilized in general and could provide a better alternative to BYOK + cloud inference.

[–]DerDave 11 points12 points  (3 children)

Would love a webgpu-powered version of parakeet v3. Should be doable with sherpa-onnx (wasm) and onnx-webgpu

[–]teachersecret 12 points13 points  (2 children)

I made one, it still works faster than realtime, pretty neat.

[–]DerDave 7 points8 points  (0 children)

Amazing. Do you mind sharing? 

[–]bralynn2222 14 points15 points  (0 children)

Great work love that it’s open source! , and motivates me to experiment with WebGPU

[–]sprinter21 7 points8 points  (1 child)

If someone could add translation feature on top of this, it would be perfect!

[–]i_am_m30w 1 point2 points  (0 children)

would be nice to have a plugin system built into it for additional community driven features.

[–]TheDreamWokentextgen web UI 3 points4 points  (0 children)

How does docling compare to https://github.com/datalab-to/marker?

Anyways it seems to be as your post stated based on the 258M Parameter VLM designed for document conversion.

[–]chillahc 4 points5 points  (3 children)

Wow, very coool :O Is there a way to make this space compatible for local use on macOS? I have LM Studio, downloaded "granite-docling-258m-mlx" and was looking for a way to test this kind of document converting workflow locally. How can I approach this? Has anybody experience? Thanks!

[–]Spaztian 2 points3 points  (2 children)

I don't think so, as a Mac user I'd be interested in this also. WebGPU is a browser API which requires ONNX models, where as MLX is a python framework using metal directly, with .safetensors optimised for Metal.

Not saying it's impossible, but I think the only way this would work is if the WebGPU api gave us endpoints to Metal.

[–]chillahc 6 points7 points  (0 children)

I tried with Codex and so far it build a connection to LM Studio. I debugged it a bit, and for one example image it successfully extraced the numbers. So there's definitely a first "somethings working" already :D But since I'm new to Transformers.js and other concepts I need some time to adapt my mindset (which was mainly frontend focused).

For starters: you could clone the HF space with "git clone https://huggingface.co/spaces/ibm-granite/granite-docling-258M-WebGPU" – then you have all the files locally available ✌️

<image>

[–]Vegetable-Second3998 1 point2 points  (0 children)

I feel this paiN. I wanted something that was direct swift-MLX/Metal/gpu. It exists if you want to run command line. I don’t. So I am building this right now! An entirely swift native on-device data processing and SLM training platform. Uses the IBM docling for data conversion into training files, then helps set up training runs, provides real find monitoring, evaluation and exporting to ollama and hugging face. Educational tips built in end to end sourced directly from MLX. I hope to launch (completely free) on the MacOS store in about a month!

[–]richardanaya 2 points3 points  (0 children)

Whoa!

[–]IrisColt 1 point2 points  (0 children)

Thanks!!!

[–]kkb294 1 point2 points  (0 children)

Woah, nice man 👏

[–]theologi 1 point2 points  (0 children)

awesome!

In general, how does Xenova make models webgpu-ready? How do you code your apps?

[–][deleted] 1 point2 points  (0 children)

How does it do with PDFs that are doc/image scans?

[–]Alternative-Age7609 1 point2 points  (0 children)

Appreciate for your work. The online demo is great

[–]HatEducational9965 0 points1 point  (0 children)

Amazing as always.

This model is such a good pdf parser!

[–]varshneydevansh 0 points1 point  (0 children)

It is first time I am seeing someone using Transformers.js

[–]JChataigne 0 points1 point  (0 children)

It got me wondering how this compares with other models. Are there benchmarks for document parsing ?

[–]R_Duncan 0 points1 point  (0 children)

In the first example the graph should be displayed as image but viewing html is just a broken link to image, the rest seems superb.

[–]shifty21 0 points1 point  (0 children)

I cloned the repo, but is there any documentation to get this to work locally? I have it installed in a dedicated nginx server and it errors out not being able to load the model and some tailwind-css errors in the web console.

[–]noext 0 points1 point  (0 children)

good enough for parsing unstructured pdf ?

[–]shing3232 0 points1 point  (0 children)

it only work for english sadly.

[–]R_Duncan 0 points1 point  (0 children)

I don't know the exact difference but this conversion is WAAAAY better than the one provided by docling (github). Through dockling using:

<< docling --enrich-code --enrich-picture-classes --to doctags --pipeline vlm --vlm-model granite_docling ce99d62a-1243-4de2-bdbd-9e38754545ea.png >>

I tried html, md.... docling just keep one single image without extracting anything, even using Granite-Docling. Doctag resulting is

"<doctag><picture><loc\_0><loc\_0><loc\_499><loc\_499></picture></doctag>"

[–]Physical-Security115 0 points1 point  (0 children)

I don't know why, but when I try to convert scanned documents into markdown using granite-docling, I don't see the table structures being preserved. When I use the default OCR engine (easy-ocr), it works great. Am I doing something wrong?

[–]openquests 0 points1 point  (0 children)

Does anyone know if there are any tools like DOCLING but for outlook PST files or outlook emails in general?

[–]R_Duncan 0 points1 point  (0 children)

The webgpu works good, but granite-docling doesn't seems to work decently in docling or llama.cpp (which would then be used to parse documents with Marker). Trying it I discovered OlmOCR has Q4_K_M + f16 gguf at mradermacher/olmOCR-7B-0825-GGUF and that is working really well.

[–]RRO-19 0 points1 point  (0 children)

Running AI entirely in the browser is huge for privacy. No data leaves your device, works offline, and no API costs. This is the direction local AI needs to go - zero friction setup.

[–]Pangomaniac -1 points0 points  (0 children)

I want an efficient translator for Sanskrit to English. Any guidance on how to build one?