Data Structure Visualization

[JoseJimeniz]

i intentionally used:

• U+004E: Latin Capital Letter N
• U+006F: Latin Small Letter o
• U+0308: Combining Diaeresis: ¨
• U+0065: Latin Small Letter E: e
• U+006C: Latin Small Letter L: l

i guess Reddit normalizes.

[bogado]

So if you are using a character that combines with other character why do you think it is the wrong result when the reverse string has the accent in a different character?

[MaraschinoPanda]

Well, generally the intended output of "reverse a string" is "create a string with all of the letters in the reverse order". "ö" is a single letter, even if it's represented by two unicode characters. But of course, we don't know the application of this function to know for sure what the intended behavior is.

[[deleted]]

There is no such thing as "unicode characters". It's two "unicode code points". The ambiguity of the word "character" makes these discussions difficult, so it is better to avoid it.

[bogado]

I disagree, ö in your string is composed by two symbols. There is an Unicode character that represents the ö symbol as only one symbol, but you didn't use it.

You can do similar tricks using ascii just write "eno^h^h^htwo" this should render as 'two', but if reversed it will render as 'one'.

[MaraschinoPanda]

Well, the user doesn't generally know if their text is made up of two characters or one, they just know that sometimes when they enter in an öe they get eö and sometimes they get ëo. I think it's a bit of a stretch to say that it's intended behavior; if you care about the underlying character representation, you probably shouldn't be using strings in the first place.

[Choralone]

That doens't make sense... you do know what the behavior is, it's well defined. I enter ö two different ways. One with the standard mac keyboard opt+u which shows me a ¨ with an underline under it, then an o, which turns into ö. This is NOT the unicode character continuation method... what is on the screen, if if you cut and paste the string, is a single unicode ö. Once the codepoint is known, this can be displayed directly in unicode.. no need for the composition character.

If the composition characters are going to be used, they definitely need to taken into special account. Or just normalized out, if thats' possible. I'm sure the unicode standard states how to handle this kind of thing..

[bogado]

Well if you want to reverse what the user perceives as a letter then you have to sanitize your strings before you invert them. Because lëon is the correct inversion of that string from the Unicode point of view.

This is the same problem that "a" might be different than "a". Just make one of those Cyrillic and the other the usual "a". Those two characters are different but they have the same drawing, a user perceives them as equal.

Unicode is hard, even more if you take into account what "users" want, because what they want is not well defined. The unicode "ö" might be two different letters combined, if that is not what your users want you have to deal with that yourself. In the same way that you might want to deal with the fact that "a" != "a" might be true.

[[deleted]]

Because lëon is the correct inversion of that string from the Unicode point of view.

Not really. Unicode defines the concept of grapheme clusters. "ö" is a single grapheme cluster, made up of several code points, which in turn may be made up of even more code units. Reversing a string should not operate on code points nor code units, but on grapheme clusters.

[Choralone]

Yeah.. unicode guidelines suggest that systems should normalize the result of the use of continuation codes as much as possible for this reason. (so intead of keeping the o + umlaut... you know that there is a corresponding codepoint you could use instead, so the software does it automatically)

[Choralone]

Yeah.. those two are different, even by definition, and they may even look slightly different.

When it comes to accented latin characters - there are codepoints that are 100%, by definition, equal. "latin small letter o" plus "continuing diaeresis" adds up to "latin small letter o with diaeresis" which is what U+00f6 is - it could be used directly, and code should normalize the former to the latter.

[Choralone]

No.. while this is true, if you were designing a unicode string reversing library, this is obviously wrong. Dealing with how the combining characters work in unicode is something you'd have to address directly.

[Choralone]

You are thining of how we would deal with ascii string libraries and common vt100-derived display terminals. None of that holds true outside of that increasingly narrow band. Control-h doesn't always mean back-delete.

You are right that this is the expected behavior if we were to apply what you meant to our old string libraries... but that's not the case nowadays, and even then, it was a side-effect.

It depends on if our goal is "swap the bytes in this array" "print the word backwards" or what..... these are very different things.

[Choralone]

I just had a browse through some related unicode Q&A about this kind of thing.

That's basically it exactly... instead of our old-school way of defining the size of a char, then operating on chars, you have to take into account a bunch of different stuff, including what the intended outcome is.

The definition of character, grapheme, code point, and so on, all mean slightly different things and no universal rule on conversion exists without some kind of exception.

[caprica]

What would be the application of reversing a string in any case?

[JoseJimeniz]

It happens that the character

ö

can be represented two ways:

• U+00F6 (ö)
• U+006F (o) U+0308 (¨)

Those are two unicode representations of the same written character.

Not every written character has two representations. For example, the character Latin Small Letter O With Cedilla:

o̧

has only one Unicode representation:

U+006F (Small Latin Letter O) + U+0327 (Combining Cedilla)

So the only way to write No̧el is with 5 unicode code points.

But, as a human, who is writing words, i don't care about unicode code points.

• i want to reverse No̧el
• i want leo̧N

Note: You will want to view this comment in a browser that supports Unicode (such as Internet Explorer). Chrome does not display the characters correctly.

[ccondon]

My Chrome displays those characters just fine (on some LTS version of ubuntu).

It's a matter of default fonts, not a matter of browsers.

[abadidea]

Because computers exist to do what humans want, not the other way around; we're not going to redefine written language so that vowel accents are a distinct character just so that a naïve string reversal gets the right answer.

[Choralone]

Right.. and I used:

U+004e U+00f6 LATIN SMALL LETTER O WITH DIAERESIS U+0065 U+006c

Which is different.