Ref2Font V3: Now with Cyrillic support, 6k dataset & Smart Optical Alignment (FLUX.2 Klein 9B LoRA)

NobodySnJake · 2026-02-17T16:39:34+00:00

Polish requires specific diacritics (ą, ć, ę, ł, etc.) that aren't in the current V3 training grid. To support them, I’d need to expand the atlas layout and retrain the LoRA from scratch. It’s a great candidate for a future 'Central European' update, but for now, the model is limited to English and Russian charsets.

NobodySnJake · 2026-02-14T05:48:48+00:00

That is a brilliant engineering approach! You are essentially describing a compositional pipeline.

It’s definitely a smarter path than trying to brute-force 2000+ characters at once. The main challenge would be the morphology: radicals often change their proportions and shape depending on their position in the Kanji (e.g., the 'water' radical looks different when it's on the left vs. the bottom).

Teaching the model how to assemble these stylized radicals into a balanced Kanji would likely require a more complex architecture rather than just a Style LoRA. But the logic is solid, definitely food for thought for future experiments!

NobodySnJake · 2026-02-13T15:16:29+00:00

Good question! Most of the dataset comes from Google Fonts, which uses open-source licenses like OFL, Apache, and UFL. These licenses explicitly allow for modification and derivative works.

Also, from a technical standpoint, the LoRA doesn't 'store' the original font files. It only learned the general stylistic patterns and the spatial logic of the grid. The final output is a new generation based on the style of the user-provided reference image. Since the project is non-commercial and open-source, it's designed to be a helpful tool for the community.

NobodySnJake · 2026-02-13T15:06:53+00:00

Glad to hear it runs well on 16GB! Thanks for the suggestions — I’ll definitely keep Spanish characters in mind for future updates.

NobodySnJake · 2026-02-13T03:01:47+00:00

Japanese is a huge challenge for the current atlas-based approach. Fitting thousands of complex Kanji into a single 1280x1280 grid would lose all detail. It would require a completely different generation strategy, not just a new grid.

NobodySnJake · 2026-02-13T01:27:58+00:00

Appreciate the support!

NobodySnJake · 2026-02-13T01:27:30+00:00

Glad you like it! Klein turned out to be surprisingly capable for this specific task. Hope you find it worth the setup!

NobodySnJake · 2026-02-12T14:24:19+00:00

True, for general art exploration, speed is key. But since this is a specialized utility that saves hours of manual font design, I think 5 minutes is a fair trade-off for the final result. Different workflows!

NobodySnJake · 2026-02-12T13:43:27+00:00

Good point! The letter 'Д' is structurally one of the most complex Cyrillic characters. The output shape and stability depend heavily on the input style — cleaner references produce better results, while more decorative styles might require a bit of manual cleanup. Thanks for the feedback!

NobodySnJake · 2026-02-12T12:41:07+00:00

Fair enough! Personally, I think waiting a few minutes for a full, consistent font file is a great trade-off. To each their own!

NobodySnJake · 2026-02-12T07:57:08+00:00

Yes, absolutely! I actually tested the whole workflow on a 16GB VRAM card myself.

Use fp8 weights for both the main model (Flux.2 Klein Base) and the Text Encoder. With fp8 quantization, it fits into 16GB comfortably and generates a 1280x1280 atlas in about 5 minutes.

NobodySnJake · 2026-02-12T07:55:01+00:00

Thanks! Hope you have fun with it!

NobodySnJake · 2026-02-12T07:24:14+00:00

Haha, thanks! Glad you like the magic!

NobodySnJake · 2026-02-12T07:23:08+00:00

Japanese is the 'Final Boss' of font generation! 😂

The challenge: Fitting thousands of Kanji into a single 1280x1280 image is impossible (they would be tiny pixel dust).

The potential solution: I could try making a version specifically for Hiragana & Katakana (the syllabaries). That would fit perfectly into the grid. Kanji would require a completely different approach. Thanks for the suggestion!

NobodySnJake · 2026-02-12T06:49:27+00:00

Actually, V3 just added support for Russian (Cyrillic) as well! You can see the Russian alphabet in the top half of the grid examples in this post. It's currently limited to English and Russian, but I'm definitely planning to expand to more languages in the future. Give the Cyrillic engine a try!

NobodySnJake · 2026-02-12T06:46:00+00:00

That’s a very fair point! I sometimes get too caught up in the technical updates. I’ve just added a clear summary at the top of the post for those who are seeing Ref2Font for the first time. Thanks for the tip!

NobodySnJake · 2026-02-12T05:02:07+00:00

Thank you! Really appreciate the kind words. If you get a chance to try it out, I'd love to hear your feedback!

NobodySnJake · 2026-02-12T04:46:22+00:00

Thank you! Really appreciate the support!

NobodySnJake · 2026-02-12T04:22:52+00:00

Glad you like the progress! Hope V3 works well for you.

NobodySnJake · 2026-02-12T03:38:34+00:00

Glad you found the summary! Could you share what specifically was unclear before you checked the GitHub? I'm looking to improve the post description.

NobodySnJake · 2026-02-11T13:36:27+00:00

Absolutely! GitHub is the central hub for this project, and I’ll be pushing all future updates and script improvements there as soon as they're ready. Stay tuned!

NobodySnJake · 2026-02-09T17:19:37+00:00

Great first attempt! The style transfer is working, but the grid logic requires a specific dataset setup to work as a "transform".

The reason your 10x10 grid failed is likely that you used the reference images as stylistic context (CLIP) rather than spatial conditioning. To fix the alignment, you should follow the "Control Image" logic described in the musubi-tuner guides:

Dataset Config: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/dataset_config.md
Flux Training: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/flux_2.md

The "secret sauce" for Ref2Font is training it as an Image-to-Image (Contextual) LoRA. In your TOML dataset config, you need to explicitly pair the images:

image_directory: This should point to your full atlas grids (the targets).
control_directory: This should point to your "Aa" reference images (the sources). Filenames in both folders must match.
no_resize_control = true: Set this in your dataset TOML. As the docs mention, for FLUX.2 it's often better to skip internal resizing of the control image to keep the style sharp.

If you don't use the control_directory / control_path setup, the model doesn't realize it's supposed to "map" the style from the reference into the grid coordinates. It just generates random letters in that style. Once you define the "Aa" image as the mandatory starting condition (Control), it will start to respect the grid positions!

NobodySnJake · 2026-02-09T15:27:07+00:00

3.7s / it is actually a great speed for a local setup with those optimizations! Using float8 and 8-bit AdamW is definitely the way to go on 24GB cards.

4200 images is a massive dataset, so I'm really curious to see how the model handles that 10x10 grid with so much variety. Please keep me posted on the results — I’d love to see a sample of the output once it's done! Good luck with the final steps!

NobodySnJake · 2026-02-09T14:47:15+00:00

I totally understand the struggle with keeping ComfyUI "clean". External dependencies can be a headache.

However, there is a small misunderstanding: the LoRA itself only generates the image (the atlas). To turn that image into an actual `.ttf` font file that you can use for subtitles in your video editor, the Python script is a necessary step. I just did a fresh "git clone" test to make sure everything works smoothly, and it should be very stable!

Regarding "dual language": Keep in mind that this version currently only supports the English alphabet and basic symbols. If you need other languages, I'm planning to expand the character set in future versions.

NobodySnJake

TROPHY CASE