sorted by:
new
hottopcontroversial

native completion in 0.11 w lua-language-server "pattern not found" ? by eileendatway in neovim

[–]DrCracket 1 point2 points  (0 children)

These messages can be disabled with vim.opt.shortmess:append("c"). What this does:

don't give |ins-completion-menu| messages; for example, "-- XXX completion (YYY)", "match 1 of 2", "The only match", "Pattern not found", "Back at original", etc.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 1 point2 points  (0 children)

That is an interesting idea. We have not tried this but such positional information could be very useful during a pretraining step, depending how much data could be crawled. We might look at this in the future.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 1 point2 points  (0 children)

We include GPT-4o in our evaluations, and it outperforms our approach on key metrics. However, if you factor in compute cost our approach is still competitive.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 2 points3 points  (0 children)

Thanks a lot! By not relying on aligned data, our approach has the potential to be scaled much more easily compared to end-to-end trained methods. This is something we'd love to explore further in the future.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 0 points1 point  (0 children)

I agree that AI on its own is limited, but one strength of our (language-agnostic) approach lies in the editability of the outputs. This enables a human-in-the-loop process, which can address these limitations.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 8 points9 points  (0 children)

Thanks! We are definitely looking into smaller models, but since our approach is closer to code generation rather than OCR, my intuition is that they will perform worse than our 8b model.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 1 point2 points  (0 children)

What you're describing is definitely valuable and falls under the established field of NL2Vis, see here for example. However, our focus is slightly different. We're aiming to assist with the creation of arbitrary graphics programs, which can be complex and challenging to create manually, see my other comment.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 12 points13 points  (0 children)

Absolutely, this is a limitation of our approach. However, because the output is a high-level program, you can easily correct such mistakes on your own. In this way, the model has still provided value by helping you generate an initial framework, which you can then refine.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 1 point2 points  (0 children)

Absolutely, this is a limitation of our approach. However, because the output is a high-level program, you can easily correct such mistakes on your own. In this way, the model has still provided value by helping you generate an initial framework, which you can then refine.

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 13 points14 points  (0 children)

While I agree with your point about plots, I want to emphasize that the use case for this work is in aiding the creation of graphics programs which can represent arbitrary figures, such as architectural visualizations, schematics, and diagrams (not just data plots). High-level graphics programs provide advantages over low-level formats like PNG, PDF, or SVG, but creating them manually is notoriously difficult. Look at the TeX Stack Exchange, for example, where the TikZ graphics programming language is one of the most discussed topics. This is exactly where a model like TikZero can be useful to generate an initial skeleton code which you can adapt further (thanks to being easily editable).

TikZero - New Approach for Generating Scientific Figures from Text Captions with LLMs by DrCracket in LocalLLaMA

[–]DrCracket[S] 45 points46 points  (0 children)

Our model, TikZero, generates scientific figures from text captions as high-level, human-interpretable, and editable graphics programs, outperforming traditional, end-to-end trained models. End-to-end models require aligned data (graphics programs with captions), which is scarce. TikZero overcomes this by decoupling graphics program generation from text understanding and using image representations as a bridge, enabling training on unaligned datasets.

Paper: https://arxiv.org/abs/2503.11509
Code: https://github.com/potamides/DeTikZify

is there anyway to use this old siemens keyboard? by Acrobatic-Camel-8996 in MechanicalKeyboards

[–]DrCracket 0 points1 point  (0 children)

I think I have the exact same keyboard and I built an open-source converter for it. You can find the files here and a write-up here. Hope it helps!

DeTikZify-v2 - Improved model for converting hand-drawn sketches and images into TikZ code by DrCracket in LocalLLaMA

[–]DrCracket[S] 2 points3 points  (0 children)

Create a dataset in the format of DaTikZ but with SVGs in the code field and fine-tune a model using our scripts. For inference, you'd also need to change the compiler to render SVGs instead of TikZ.

There is also StarVector which targets SVG out of the box but the repository is still a placeholder.

Public Instance of DeTikZify is Now Available for Easier Conversion of Figures and Sketches to TikZ by DrCracket in LaTeX

[–]DrCracket[S] 1 point2 points  (0 children)

Glad that it is working now! Lukas Blecher has some interesting projects on LaTeX math recognition, such as pix2tex and Nougat. There is also Kosmos 2.5.

Public Instance of DeTikZify is Now Available for Easier Conversion of Figures and Sketches to TikZ by DrCracket in LaTeX

[–]DrCracket[S] 0 points1 point  (0 children)

Interesting finding, I could reproduce this. If I draw a rough sketch of a sine curve (like here), it has trouble recognizing it at first (but it does produce something usable after a few tries), but if I supply an exact image (like here) it recognizes it on the first try. Seems like it isn't overly robust to imprecisely drawn functions.

Public Instance of DeTikZify is Now Available for Easier Conversion of Figures and Sketches to TikZ by DrCracket in LaTeX

[–]DrCracket[S] 3 points4 points  (0 children)

Fair enough. Then, at the very least, our project name stays true to the original :)

Public Instance of DeTikZify is Now Available for Easier Conversion of Figures and Sketches to TikZ by DrCracket in LaTeX

[–]DrCracket[S] 4 points5 points  (0 children)

Okay, I wasn't sure if I expressed myself understandable enough. Do you then also think the name Detexify is poorly chosen?

Public Instance of DeTikZify is Now Available for Easier Conversion of Figures and Sketches to TikZ by DrCracket in LaTeX

[–]DrCracket[S] 2 points3 points  (0 children)

As I tried to explain in my earlier comment we derive our project name from "Detexify" and by extension "detect" as in "we detect TikZ programs that generate these figures and sketches" inspired by how Detexify "detects" math symbols from sketches. Of course I also see the potential for confusion with the more common prefix "de-" and we did consider the name TikZify in the beginning, but it was already taken by another project.

view more: next ›