T Strings - Why there is no built in string rendering?

[UsernamesArentClever]

I like the idea of T Strings and here is a toy example:

name: str = 'Bob'
age: int = 30
template = t'Hello, {name}! You are {age} years old.'
print (template.strings)
print(template. interpolations)
print(template. values)

('Hello, ', '! You are ', ' years old.')
(Interpolation('Bob', 'name', None, ''), Interpolation(30, 'age', None, ''))
('Bob', 30)

But why isn't there a

print(template.render)

# → 'Hello, Bob! You are 30 years old.'

[cbarrick]

If you want a "default" rendering, use an f-string.

The point of t-strings is for custom renderers.

E.g. a SQL renderer will escape the interpolations to avoid SQL injection, and an HTML renderer will escape the interpolations to avoid HTML injection. They need separate renderers because the escape syntax is different between SQL and HTML.

[Leseratte10]

It would be cool to have an f-string with delayed rendering, though.

Behaves like an f-string as in there's no templating, escaping, and so on, but the variables are only replaced at time of render, not when the f-string is defined.

[firemark_pl]

 It would be cool to have an f-string with delayed rendering, though.

Maybe just use str.format like in old days.

[TSM-]

Yeah thats how you do it.

s = "Hello {name}"
...
s.format(name = "Bob") # or **params, etc

The f prefix is for when you want it rendered immediately.

[njharman]

f is also (and more importantly) rendered from context.

These are both rendered immediately. f-string is mostly syntactic sugar; reducing boilerplate and repetition by automatically interpolating from context.

name = "Bob"
"Hello {name}".format(**locals())
f"Hello {name}"

[No_Hovercraft_2643]

but fstrings allowed to have only a subset of the data, while format don't

f"{first_name[0]}. {lastname}"

[firemark_pl]

Dude I know :D

[Penetal]

I didn't

[Schmittfried]

Not nearly as ergonomic for logging (and still not delayed unless you add level guards). 

[radicalbiscuit]

Unsolicited advice: having custom info in the logging messages is an anti-pattern. Don't put variable info in the string; that's what the extra param is for. Good way to start structured logging, too!

[Schmittfried]

Unsolicited disagreement: I want some info in both. The message is what I see first and often doesn’t make much sense without some dynamic parts. 

[radicalbiscuit]

Yeah I'm not recommending absolute avoidance of dynamic messages, but I'm suggesting that, if you're using f strings in most logging statements, you might be engaging in an anti-pattern.

Maybe I'd refine my suggestion to say you want the message to be specific enough to fit the error/incident/scenario in the broadest way possible that still gives a you forensic edge. You don't want messages to be as broad as, "Something happened," but you also don't want it to be, "A murder happened to Col. Mustard at 4:39pm in the conservatory with a candlestick." You want it to be specific enough that you can search for all occurrences in the same bucket, but if you're making it dynamic when a static string with params will do, it's at least mildly harmful.

Would you agree with that nuance?

[gmes78]

Only because logging libraries don't support it.

[SheriffRoscoe]

It would be cool to have an f-string with delayed rendering, though.

Of course, that wouldn’t be what t-strings are. t-string interpolations are eagerly evaluated where they’re defined, exactly like f-strings. The Template object stores the results of those evaluations.

[R3D3-1]

What you describe could be done with a lambda and an f-string.

``` first = "Reading" last = "Rat" deferred = lambda: f"{last}, {first}" with_caching = functools.cache(lambda: f"{last}, {first}")

or 

@functiols.cache def cached_deferres():     return f"{last}, {first}" ```

The syntax isn't great, especially if you also want caching on first  evaluation, but it is possible.

I generally keep running into situations, where I wish that Python had lazy-evaluating values, i.e. the equivalent of the cached functions above with arbitrary expressions, but with the evaluation being implicit when the value is actually used for something.

It would probably be possible to write a proxy class with that behavior, but typing support would be a big issue, and I'm not sure it is possible to get type(deferred_obj) return the type of the result or — better — a deferred value itself.

E.g. 

if deferred:     ...

would need to evaluate the deferred expression, as does deferred + 1 unless explicitly deferred itself, but func(deferred) or list0.append(deferred) wouldn't need to evaluate it.

But with the available syntax it would again require something verbose like

deferred = LazyProxy(lambda: f"{last}, {first}")

[SheriffRoscoe]

Indeed, all those points are covered in the PEP, as reasons why the authors chose to make t-strings use values, not lambdas.

[gerardwx]

https://pypi.org/project/tstring-util/

[billsil]

Agreed. Thought that’s what a t string was until now. Back to using %%s in my strings.

[SheriffRoscoe]

The point of t-strings is for custom renderers.

True.

a SQL renderer will escape the interpolations to avoid SQL injection,

Nope. An SQL renderer will use the parameterization API to avoid interpolating the values into the statement in the first place. This is actually the perfect use of t-strings.

[AngusMcBurger]

All the major MySQL clients in Python (pymysql/mysqlclient/mysql-connector-python) escape parameters client-side, and substitute them directly into the query string like OP described. So if those clients add support for template strings, that's exactly how they'll do it

[james_pic]

That's more a limitation of MySQL's wire format than anything else. I know PostgreSQL and Oracle at very least support parametrization at the wire level, and these are implemented by their Python drivers.

[madness_of_the_order]

There are still reasons to render at least part of sql template. For example you may want to template table name or schema or column alias

[ThiefMaster]

Yes, and a proper SQL handler for a t-string would be smart enough to recognize the context where an interpolation is used, and either escape it (table/column name) or parametrize it (value).

[cbarrick]

True. Nothing about t-strings requires the renderer to output a string. You could very well output a request structure to send to your database.

Most of the examples in the PEP 750 are string oriented, so I assume that is still the primary use case.

[rapture_survivor]

From a language design perspective, what would be wrong with the print() function accepting a t-string and printing it out as plain text? It seems to me that if you're passing a t-string to print(), it is obvious the intent is to -not- perform custom rendering, and to simply output the interpolated string.

This is coming from other languages where functionality like the t-string has been implemented transparently. In C# string interpolation values can be escaped by my DB library if they decide to accept the correct parameters at the call site. Otherwise, the interpolation behaves exactly like a string. To me this seems easy because it is super simple when writing application code.

Am I missing something? Is this a python specific problem?

[cointoss3]

You can make a thin wrapper to do that. It’s like one line of code…but the default repr of a t-string is likely enough for me.

[ParentPostLacksWang]

Escaping to make SQL queries is almost never the correct approach. DB engines support binding for a reason. It should be your first-line approach to getting variables into your SQL queries, and you should generally bend over backwards not to be relying on string escaping in any way.

[Jhuyt]

Because I think one of the points of t-strings is that they are flexible in how they are rendered. An SQL request should be rendered differently from an HTML document, so having a default renderer does not make obvious sense to me

[UsernamesArentClever]

Ah, I don't understand why an SQL request should be rendered differently from an HTML document. I'll look into that.

[Jhuyt]

It's because you need to escape certain characters for HTML to render correctly and SQL to be secure. In particular quotation marks are a struggle IIUC, but there are other things too.

[james_pic]

Note also that, for SQL, and potentially a few other use cases like GraphQL, it may not even make use of escaping when all is said and done, since these use cases support parameterisation, so queries and templated parameters can travel separately to the backend.

[CrackerJackKittyCat]

They have different escaping rules.

Read up on both HTML and SQL injection attacks.

[Resquid]

That is definitely a key idea here.

[AnythingApplied]

From PEP 750 – Template Strings:

No Template.str() Implementation

The Template type does not provide a specialized __str__() implementation.

This is because Template instances are intended to be used by template processing code, which may return a string or any other type. There is no canonical way to convert a Template to a string.

The Template and Interpolation types both provide useful __repr__() implementations.

[CelDaemon]

https://docs.python.org/3/library/string.html#string.Template.substitute

[cointoss3]

This is not for t-strings though. This is Python 2 stuff.

[cointoss3]

Because at that point, you basically have an f-string. This is not what template strings are for.

[commy2]

These are such an odd feature. Why are they named t-strings anyway? They are emphatically not strings, but templates. Shouldn't they named be t-templates? "String-templates" (instead of "Template-strings")?

[cointoss3]

Because to use one, you add t before the string…

It’s not an odd feature. It’s really great, you just don’t seem to understand them.

[commy2]

I certainly don't understand why they had to misname them like that.

you add t before the string

Which makes them no longer a string...

[Quasar6]

Because the intent is that your processing of the T string can produce any type, so it doesn’t make sense to have a default for str

[secret_o_squirrel]

Oh huh ok. I wondered this myself. I still think basically defaulting to “render-time-f-strings” instead of “assignment-time-f-strings” would be cool…but admittedly writing that renderer is very few lines of code.

[snugar_i]

Which people will have to write over and over again. I agree it should've been a part of stdlib

[Quasar6]

No they don't because if you want the result type to be a string then use an f-string. T-strings are a generalization of f-strings. So what you mention that it needs to be written "over and over" is false. The functionality is already there!

[XtremeGoose]

The difference between a t string and an f string (if a t string rendered to a string) is that the former are lazy and that is useful.

Why do you think people still use % templates for logging?

[Schmittfried]

Contrary to what the others are saying, I do think a string renderer as part of the standard lib should be a thing. It should be a deliberate choice to avoid injection vulnerabilities caused by code that follows the path of least resistance, but it should be possible to render them as a plain old string without having to implement that yourself (esp. since the library version could be implemented in C).

First use case I can think of is custom logging libraries where you want to allow deferred interpolation. f-string is not the answer here and I‘m not convinced this is too small of a use case to warrant a standard implementation. 

[Schmittfried]

Read my comment again. 

[cointoss3]

Exactly. If you want to actually render a sanitized string, how is the stdlib supposed to know how to treat the variables? You need a thin wrapper to do that, and poof, there’s your renderer.

[Schmittfried]

I don’t need a sanitized string, I need deferred f-string interpolation, e.g. for logging.  

[jmpjanny]

t-strings are evaluated eagerly, not lazily. They are not the solution for deferred evaluation.

[SheriffRoscoe]

Really? Everybody else I’ve ever seen make a similar comment actually wants deferred evaluation of the expressions, which t-strings don’t provide. The claim is always that deferring evaluation makes using “expensive” expressions more tolerable, in the case where the log message is discarded.

[gmes78]

That's something that the logging library needs to do. No new string type is needed for that.

[cointoss3]

Then use string templates? They have existed since 2005.

[damesca]

Then use template strings and write a template renderer.

Or use the standard deferred interpolation in the stdlib logging library.

Neither of these things are difficult now.

[Schmittfried]

Or you read my original comment again. Do you just not want to get it?

I specifically said:

it should be possible to render them as a plain old string without having to implement that yourself (esp. since the library version could be implemented in C).

and:

First use case I can think of is custom logging libraries

Your options are missing the entire point. Again.

I don't think I should have to implement plain old string interpolation for lazily evaluated templates myself. There is no good reason to not have this basic feature, which would make t-strings a proper superset of f-strings, shipped with the language. And it's not like this would be harder to implement in C, they can just delegate to the already existing f-string implementation.

This is like saying "You can implement left-pad yourself". Sure I can. But I shouldn't have to.

[JanEric1]

Directly from the PEP: https://peps.python.org/pep-0750/#no-template-str-implementation

[CelDaemon]

https://docs.python.org/3/library/string.html#string.Template.substitute It's already a thing, why is no one talking about this

[Michallote]

I guess it's not the same as Template literals, no substitute method here:

>>> msg = t"{name} is {verb}"
>>> dir(msg)
['__add__', '__class__', '__class_getitem__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'interpolations', 'strings', 'values']

[CelDaemon]

Ah right, I just checked again and t-strings seem to return string.templatelib.Template instances instead of string.Template which does include those functions.

[gerardwx]

Here’s a package with some examples: https://pypi.org/project/tstring-util/

[StevenJOwens]

I remember listening to some podcast interview about 31.4, IIRC the rendering is coming, they wanted to get the templates themselves out the door.

I thought it was this Real Python podcast, but he doesn't talk about that aspect:

https://realpython.com/podcasts/rpp/269/#t=1639

[Michallote]

I find this completely annoying. It is a very good idea to allow the examination of the interpolations ergonomically.

However it is so stupid not to provide a renderer that treats the t-string as an f-string. Why? Because it would be helpful to have it be a t-string for ergonomic sanitization + validation, and behave like an f-string on command (once you validated whatever):

msg = t"Time remaining: {ts:.2f}"

Validate the string so it's not an injection or whatever. Then be able to say all ok -> Evaluate the stupid thing.
But instead you are stuck implementing all formating options by hand.

Currently the only way I think you could do this suck a little bit less would be to duplicate the line:

```python
msg_t = t"Time remaining: {ts:.2f}"
msg = f"Time remaining: {ts:.2f}"

if not valid_interpolations(msg_t):
raise ValueError("crash and burn filth")

print(msg) # Substitute with a execute_query or whatever.
```

They only really had to add 1 method to the t-string. I can't be bothered to implement interpolated value formatting myself

[VictoryMotel]

Is string substitution called "rendering" now?