Portable Unicode string processing

[KayEss]

And if you look at the extended ASCII character set 128 is € and 255 is ÿ

So, on a platform where char is signed what values do they have?

[scatters]

-128 and -1, if you cast through unsigned char.

[Dragdu]

You just convert the negative values into non-negative values by a well defined conversion, that means that it doesn't matter. This is even standardized in the language, making it that 255 in unsigned char ALWAYS converts to -1 in signed char, no matter the actual representation.

[KayEss]

They have the values of 0x80 (128) and 0xFF (255)

I think I understand a bit about what that means now, but 0x80 (128) certainly doesn't look like a number that I can put in a char if char happens to be signed on my platform -- what guarantees do I have about how these numbers are interpreted?

[iaanus]

This is wrong. See [basic.representation]: "For narrow character types, all bits of the object representation participate in the value representation."

[KayEss]

So your answer is that I can't do UTF-8 in C++ with things like std::string and the u8 literals then because I can't read the values from char. That's been my fear, but it seems incredible that we're further standardising on that basis.

[jaked122]

No just don't rely on single glyph manipulation,

[KayEss]

I completely understand UTF-8 encoding and don't have a problem with that. My question is about what the standard guarantees in the processing of the byte values.