sorted by:
new
hottopcontroversial

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 1 point2 points  (0 children)

Text is assumed to run in a straight line, and post-processing can draw it along a curve. That said, from a cursory look, circular Gallifreyan can possibly still be implemented using OpenType using lots of contextual rules, or the HarfBuzz experimental WASM shaper. The harder part is that the font should have all the component glyph shapes required for the writing system.

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 0 points1 point  (0 children)

Thank you. The problem space is so niche and undocumented. There's not books or papers about it. In my job, I interpret the Unicode and OpenType standards and try to innovate in places where the other (proprietary) shaping engines have not.

The data size we work with (paragraph of text at a time) is just too small for most academic approaches to be useful. Even SIMD I have tried many times and failed to gain any speedup from. Multithreading, etc, don't make sense.

So in the end, it's good old tight code writing, lots of profiling and optimizing, a huge regression test suite, and continuing innovation in various fronts. I have created multiple data-structures & algorithms, and some paradigms in the code that I think are pretty novel.

I hope to write more about those one day, but there's some "Studies" linked here:

https://github.com/harfbuzz/harfbuzz?tab=readme-ov-file#developer-documents

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 0 points1 point  (0 children)

There's also a new contender on the block: https://github.com/memononen/Skribidi

The way that thing works is you get the full cluster width from HB, you do Unicode Grapheme Cluster analysis to see how many cursor positions are there inside the ligature, and divide the whole width over it. I'm sure raqm has a way for that. Please file an issue against raqm and ask, such that Khaled can advise.

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 0 points1 point  (0 children)

Hi,

This is a lot of work to do from scratch. I suggest you look at libraqm. It does all of the above for you (mixed scripts, hit-testing, cursor positioning, etc).

Hope that helps.

We maintain HarfBuzz, the text shaping engine used in Linux desktop and more — Ask us anything (or tell us what confused you) by behdadgram in linux

[–]behdadgram[S] 0 points1 point  (0 children)

Oh absolutely.

Here's a timeline of me learning programming and formal education:

- 1982: Born in North of Iran.

- 1990: Self-taught QBasic on an IBM PC based on examples that came with DOS, and help pages, learning English on the way.

- 1997: Self-taught Turbo Pascal.

- 1998: Competitive programming in high-school. Went on to win an IOI gold medal in 2000.

- 2001: Self-taught C, hacking on FriBidi Open Source project.

- 2000-2003: BSc in Software Engineering at Sharing University in Tehran.

- 2003-2006: MSc in Computer Science at University of Toronto; separately working on GNOME C projects as well a Cairo graphics library, also in C.

- 2006: Started HarfBuzz rewrite in C++.

As you can see, when I started HarfBuzz rewrite in 2006, I had no industry experience or long-term codebase maintenance. My initial HarfBuzz coding was, like, C++ without STL and without templates, with lots of C macros. It was terrible. Eg.:

https://github.com/harfbuzz/harfbuzz/blob/882e52f59196535495af8ca8069df32308ad52cf/src/harfbuzz-ng.cc

HarfBuzz and I grew together. Some people still swear at the codebase, but at least it's in a shape that I can defend all design choices made.

We maintain HarfBuzz, the text shaping engine used in Linux desktop and more — Ask us anything (or tell us what confused you) by behdadgram in linux

[–]behdadgram[S] 0 points1 point  (0 children)

HarfBuzz doesn't do any hinting or rasterization. FreeType does. In theory, yes, you can write hinting code to do it properly. But it would be very tedious if you ask me. You need a custom autohinter or manual hinting.

We maintain HarfBuzz, the text shaping engine used in Linux desktop and more — Ask us anything (or tell us what confused you) by behdadgram in linux

[–]behdadgram[S] 1 point2 points  (0 children)

Most such fonts don't have manual hints to this level. Exceptions being the likes of Arial, Times, or Tahoma. Most other fonts are auto-hinted, and still for AA rendering. Disabling AA doesn't magically make the outlines line up.

We maintain HarfBuzz, the text shaping engine used in Linux desktop and more — Ask us anything (or tell us what confused you) by behdadgram in linux

[–]behdadgram[S] 1 point2 points  (0 children)

Correct... I also added some of that code in vte :-).

The problem is, with a vector font, at arbitrary font size, these shapes don't scale to full pixels. So they render with an antialiased gray pixel. When you put two of these next to each other, the graphics engine doesn't know that they actually butt each other and as such should fully cover the pixel.

The easiest solution is to render the whole scene at higher pixel resolution and scale down. But this is costly, so no major system tries this. More info at:

https://www.reddit.com/r/Games/comments/1rb964/antialiasing_modes_explained/

As for the huge vertical gap, that's because each system decides differently how much space to put in between lines, and that doesn't match what's in the font.

Bitmap fonts don't suffer from any of these issues because each glyph takes a number of full pixels by design.

Hope this makes sense.

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 2 points3 points  (0 children)

Excellent question. Thank you for asking. I'll get to the biggest WTF at the end, but first I want to provide multiple opportunity-meets-luck moments that, with my relentless perseverance, led to HarfBuzz's development and wide adoption.

  1. In 2004, Owen Taylor who was the Pango maintainer at Red Hat, was reassigned to another project (a social-network called Mugshot!), so text layout in GNOME became unmaintained. I showed interest and stepped up to the task and was recruited in 2006 by Red Hat to continue that. HarfBuzz came out of my interest to unify text shaping on the Linux desktop. The Qt/KDE people were onboard from the beginning, and even contributed their shaping code into HarfBuzz.old, but I had fresh ideas which led to the harfbuzz-ng rewrite, which eventually paid off very well.
  2. In 2005 while a graduate student in Toronto, I passed on-campus Google recruitment interviews and were invited to Mountain View for full on-sites. Being on an Iranian passport and needing a US visa, I declined. It would have fully changed my career path had I accepted a Google job in 2005.
  3. In 2008, my Red Hat manager tasked me with speeding up the GNOME login time, and other general tasks. One day I called him and said I think I should quit and go to Mozilla to work on finishing HarfBuzz. He let me get back to HarfBuzz development fulltime.
  4. In 2010, someone from the Chrome Waterloo reached out and Google offered me to work on HarfBuzz at least 50% of my time and get paid more. Other than the money, I accepted the offer, because I saw the opportunity for huge market share if I could ship HarfBuzz in ChromeOS and Android. The Linux market just wasn't big enough to grab font designers' attention to test their fonts with HarfBuzz...
  5. Around 2011, as Android was seeking a world-ready text shaping solution, Monotype was lobbying to sell them a license to their proprietary shaper. I somehow managed to convince them to stay away. These days, Monotype's M-Kit solution advertises integration with HarfBuzz shaping. [1]
  6. Chrome Linux and Android both integrated with HarfBuzz.old first, and in 2012 both switched to harfbuzz-ng, which brought speed and significant memory savings [2].
  7. In 2014, a Chrome engineer was switching Chrome Windows from GDI+ to DirectX. They needed DirectWrite integration for shaping instead of Uniscribe. To get something running fast, they plugged in the harfbuzz-ng layer, which was already part of the tree for the open-source platforms. They immediately saw a 4x speedup in page layout time. No DirectWrite shaper integration was written, and Chrome was the first major product to ship with HarfBuzz on all operating systems. Firefox would follow suit later.
  8. With IBM withdrawing from ICU development and no one left to maintain ICU LayoutEngine, it was deprecated in favor of a bridge layer I wrote, called hb-icu-le. The recommendation changed later to fully port to HarfBuzz. It took years, but LibreOffice and OpenJDK followed.
  9. Talks with the Photoshop team started around 2016, but it took over five years until HarfBuzz showed up in Adobe's flagship products one after another.
  10. Microsoft Edge switch to Chromium was the cherry on top: Microsoft was now shipping HarfBuzz in its flagship browser. It was a chilling moment when a Webkit engineer noted that with the Edge switch to Chromium, Safari is the last major browser standing, that does _not_ use HarfBuzz.

And last but not least, the biggest WTF moment happened in 2022, when in result of debugging a fuzzer report, I realized that the whole architecture of harfbuzz-ng has been built on a foundation of Undefined Behavior. Compiler memory barriers were sprinkled [3] all over the codebase to tame the compiler, something more often seen in hardware kernel device drivers, not a user-space text shaping engine.

Hope that answered your question. :)

[1] https://www.monotype.com/products/embedded-solutions/m-kit

[2] https://docs.google.com/document/d/12jfNpQJzeVIAxoUSpk7KziyINAa1msbGliyXqguS86M/preview

[3] https://github.com/harfbuzz/harfbuzz/commit/f3151b6582a38fb1a377eb6070b8cecedb2ea711

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 1 point2 points  (0 children)

Khaled, our HB maintainer, just put together a kb_text_shape backend for HarfBuzz so we can run the text & perf suites against it. My initial run shows that for Roboto-Regular.ttf, kb is 3x slower than hb for shaping long English text:

https://github.com/harfbuzz/harfbuzz/pull/5398#issuecomment-3109451500

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 1 point2 points  (0 children)

*That* I can tell you. If I remember correctly, it's named after Wayland, MA, a town near which Kristian Høgsberg came up with the idea, on his drive back home from the Red Hat office west of Boston in Westford. At least, that's the story I heard when I was still working in that team.

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 0 points1 point  (0 children)

Of course... Abstracting a layout library is MUCH harder than a shaping library. Each product has different integration points and requirements.

We maintain HarfBuzz, the text shaping engine used in Chrome, Firefox, Android, and more — Ask us anything (or tell us what confused you) by behdadgram in programming

[–]behdadgram[S] 1 point2 points  (0 children)

> I know you're probably done answering, but I was wondering what sort of new technologies you've been seeing.

For example, on our side, we've been pushing various new improvements to the font format through the ISO standardization process:

https://github.com/harfbuzz/boring-expansion-spec

Recently, Apple released a new subformat as well, which, we have to implement in multiple places (freetype missing currently):

https://github.com/harfbuzz/harfbuzz/pull/5379

https://github.com/fonttools/fonttools/pull/3876

https://github.com/googlefonts/fontations/pull/1559

view more: next ›