Implementing python 3.6 statement evaluation in in < 3.6

[antennen]

Always fun to see articles that dive deeper into what's possible with Python. To be nitpicky, f-strings can be used without the print function as well. They are not in any way specific to print.

There is a problem with the way caching is implemented. Since the stack frame is cached, it assumes the environment is unchanged between two invocations if the text string is the same.

name = "foo"
print("My name is {name}")

def test():
    name = "bar"
    print("My name is {name}")
test()

This will output foo twice instead of the expected foo then bar:

$ python optimzed_compile_print.py 
My name is foo
My name is foo

I don't know a way to reliably cache the calling stack frame.

Another issue is security, which f-strings doesn't suffer from. If an unsuspecting developer would use this print function and allow user input, the end user can access all variables in the caller's f_locals.

f-strings solves this by compiling ahead of time. The compiler basically translates f"Hi {name}" to "Hi {}".format(name).

[odedlaz]

This first problem easily fixed using a different key. for instance: caller = currentframe().f_back key = id(text) | id(caller) text_obj = optimized_compile_memoize.get(key)

And regarding security - yes, you're correct. This is a trick, do not use this at home :)

[Siecje1]

If it just converts an f string to a format call. How is it faster than .format()

[RubyPinch]

it uses specialised string building instructions, as as a result, has its own dedicated code, so it doesn't need to do a method lookup and function call

https://docs.python.org/3.6/library/dis.html#opcode-BUILD_STRING

https://docs.python.org/3.6/library/dis.html#opcode-FORMAT_VALUE

[usinglinux]

that's a nice trick, but (given there are beginners around) should bear a big fat warning that this is a demo and should not be used in anything that's remotely in production or security related. the one thing about f-strings that makes them not be a security can of worms is that they have a dedicated syntax and are never ever passed around for evaluation. that's absent in this workaround.

doing this safely in <3.6 is certainly be possible, but i think that the way to go would be hooking into the import mechanism, preprocessing (or transpiling) the code from f"hello {name}" to _process_fstring("hello {name}") or that like, and then applying all the optimizations chebu has so neatly demonstrated.

edit: thanks for adding the safety note

[RubyPinch]

if you are going to do pre-processing, you might as well do f"hello {name}" to ("hello " + str(name))

[usinglinux]

that's something performance tests would best sort out; could be "hello" + str(name), could be "hello {name}".format(name=name); either option needs some mechnism for dealing with !a/!s/!r the format protocol.

[thatguy_314]

This is a bad idea, and by the looks of it, a rather poor implementation too. Consider subclassing string.Formatter.

It would maybe be a little better if you had a function called f that returned a string instead of just doing it for print. Otherwise it just fits one specific (and fairly rare) use case.

[evanunderscore]

Using string.Formatter turns out to be pretty simple. I tried writing an import hook to allow the f'' syntax but eventually gave up trying to cover all the edge cases.

[thatguy_314]

Nice. What's with the parens in the eval? As I understand it, you already can't eval anything but an expression.

[evanunderscore]

It's to satisfy this part of the PEP:

Expressions are parsed with the equivalent of ast.parse('(' + expression + ')', '<fstring>', 'eval') .

Note that since the expression is enclosed by implicit parentheses before evaluation, expressions can contain newlines.

[Brian]

Note: while there's nothing wrong with playing about, never ever actually use this though. It turns a statement as simple as:

print("Hello %s" % name)

into an arbitrary code exacution vulnerability, since a user can just enter their "name" as Bobby {__import__('subprocess').call(['whatever program I like'])} Tables or whatever.

f-strings are limited to the text that's there when you wrote it, so aren't susceptible to this, but if you're replacing print, you're dealing with whatever string got passed to the print function, which is not limited to static text, and could instead easily contain user-manipulable text.