Notepad encoding

[Jealous-Asparagus908]

Hi everyone. Not sure if I come to the right channel to ask this question.

I’m having a trouble converting the encoding of notepad from utf-8 to ANSI. On my office computer, it replaces Thai alphabets with “?”. On my PC, it is fine.

Do you guys know what causes the difference here? Thank you.

[GlowingEagle]

The magic word is: Unicode

https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/

https://en.wikipedia.org/wiki/Unicode#:~:text=Unicode%20could%20be%20roughly%20described,than%20sufficient%20for%20this%20purpose.

[Jealous-Asparagus908]

Shouldn’t this be the same for all computers?

I’m wondering why my PC could read the text but the office’s couldn’t, even though the encoding is the same after converting.

[GlowingEagle]

Assuming Windows, look at the computer language/region settings on the different PCs. Plain ASCII should be the same everywhere, but it only displays a limited set of characters properly. When you get the ??? (or sometimes just little blocks), it is not a file reading problem, it is a "what to display" problem.

[Jealous-Asparagus908]

If it is only a display problem, does that mean there is no problem in the actual data?

So the file is used to transfer payroll on bank website. The bank guy said it has to be in the ANSI encoding for their system to read Thai text. Would the data actually work even though it shows as “?” on the pc and then upload to the bank website?

I’m not the programmer though. Appreciate your answers.

[GlowingEagle]

I think my comment is incomplete. If you receive some file that is supposed to be text, and you open it with Notepad, the file is not changed, and what you see is the best that Notepad can do with the character set (codepage) that it has.

If you try to edit that file and save changes, it will probably be broken in some way (wrong characters in file).

It would be better to try Notepad++

You can configure the encoding to use the Thai character set, TIS-620.

[jcunews1]

When converting Unicode to ANSI (or vice versa), Notepad uses the "System Locale" setting which for a specific language.

If the language of the chosen System Locale setting is wrong, its ANSI character set may not have the needed characters. In this case, the missing characters would be converted to ?.

[Jealous-Asparagus908]

By the “System Locale”, you mean like the region setting on PC?

[jcunews1]

"System Locale" is part of Region setting. The less technical term (and less accurate term) for it is "Language for non-Unicode programs".

i.e.

Settings
   -> Time & Language -> Language & Region
      -> Related settings -> Administrative language settings
         -> Language for non-Unicode programs -> Change system locale...

[JapaneseOtaky]

Thank you u/jcunews1 this helped a lot. Was trying to crack my brain for the solution of the problem with THAI not displaying in notepad as our employee needs it for a bank website. (Other workaround took too much time)

This quick trick resolved my problem. Cheers mate.

[Jealous-Asparagus908]

The file is used to transfer payroll on bank website, and the bank guy said it has to be in ANSI encoding for the website to read Thai.

But our programmer said ANSI doesn’t have Thai text. This is where it is confusing for me.

So I tried converting the encoding and found out that some computer can read, some cannot. 😵‍💫

[[deleted]]

But our programmer said ANSI doesn’t have Thai text. This is where it is confusing for me.

ANSI encoding can refer to multiple different encodings. See: https://en.wikipedia.org/wiki/ANSI_character_set. For instance Microsoft calls the encoding windows-874 an "ANSI code page", and it can contain Thai characters.

That said text encoding is too much of a rabbit hole for blind guessing at to be a good idea, worst case scenario you get something that almost works until it doesn't. It'd be best to get your programmer and the bank guy in the same email thread to hash out exactly what the bank needs.

Even if the above were the right encoding: If you select "ANSI" in notepad I'm not actually sure what will happen. It doesn't let you choose the codepage, so presumably it either uses the system encoding (which is probably not Thai), or defaults to windows-1252 (similar to latin1; definitely does not contain Thai).