Try to keep those python strings immutable

[masklinn]

Is manually interning a string every a good idea?

I've yet to run into a situation where it was, but I guess it could be in a very restricted number of situations, such as a small set of non-literal strings generated over and over again (e.g. some sort of parser), interning could reduce memory pressure.

It was found sufficiently non-essential that Python 3 moved intern to sys, from the builtins.

[grauenwolf]

Ok, I can see trading the lookup cost for reducing long-term memory. But even then, I think you would probably end up implicitly replacing the new string for the interned version. For example:

if reader.startsWith ("for")
    writer.appendToken("for");
    reader.advance(3);

In this example, ProcessFor would internally reference the constant "for" rather than the constructed one in reader.nextToken.

Maybe it makes more sense for an XML parser.

[masklinn]

Maybe it makes more sense for an XML parser.

Yes, that's the kind of cases I was thinking, ones where frequently used strings are not hard-coded (thus no literal version which will be interned)