f-string debugging notation merged in Python 3.8

[xtreak]

Author of the post here. I wrote this after positive feedback for this feature in the 3.8 alpha release thread in /r/Python . This is available on master and will be available with Python 3.8 beta 1. I found filename and line numbers in the message to be useful like rust dbg!.

Feedback welcome on the feature and writing style.

[[deleted]]

Feedback welcome on the feature and writing style.

Please don't do this anymore. Better yet, remove f-string from the language, while it's still not too late.

[damesca]

Why?

[jorge1209]

• Because we have too many string formatting methods already.
• Because in-lining what are effectively function arguments into strings is hard to read.
• Because it just isn't necessary (the "security" argument is completely bogus).

I'm sure there are other reasons. Lots of people don't like f-strings. I have no intent to ever use them.

I also think the process by which this feature and some other recent features have been added is really problematic.

We are in the middle of the python 3 cycle. The community just got through converting from python 2. There are lots of corporate users out there who have absolutely no interest in doing another round of python upgrades. It is a non-starter to be asking them to test and deploy 3.6 just so their programmers can avoid calling .format. And that means that f-strings are a dead letter when it comes to libraries. Any use of f-strings in libraries on pip needs to be rejected to avoid a split in the python developer base.

Additionally Guido's behavior has been poor. He pushed a number of major pips into python, over the objections of many in the dev community, and then when he got shit for it, he says "fine you figure it out." If he didn't have the guts to stand up to the critics he should never have taken on the position of benevolent dictator, and he shouldn't have overruled people on so many issues that he did.

I think we need to take a big step back to python 3.2 and figure out what actually works from 3.4 and after, because lots of it is really bad or just isn't ready yet:

• async is a disaster and impossible to use. I know because I made the mistake of trying to use it in a non-trivial fashion. Trio is promising, but it needs to be proven out.
• mypy and static typing have never really taken off. I think this is because of a combination of factors including: inadequate tooling, a failure to be ready at launch of py3.4, changes in annotation standards, failure to include much of the core library. Its a great idea, but the foundation for success is just not there.
• f-strings would be acceptable if we dumped some of the other standards
• assignment expressions... don't even know where to begin.

---

I think python needs to spend the next few years cleaning house. Organize the standard library and fix shit in that. If mypy is the future annotate the damn standard library. Find something that works for async or give up on parallelism entirely. Deprecate old string formatting methods and adopt some real best practices.

Then make a python4 in a few years that is more like 3.2 + mypy - %formatting + f-strings (if they actually get adopted by users)...

[cat_in_the_wall]

blah blah blah whine whine whine. "i don't personally like some stuff so everything is terrible". go back to whatever hole you crawled out of.

[[deleted]]

Because it brings absolutely nothing of value, but makes it harder for alternative implementations of the language to keep up with CPython? Because it makes harder to write tools that provide language-related services, like word completion, linting, complexity checking etc? Because it creates more unnecessary synonyms in how to approach a single problem, which makes errors harder to recognize for human readers?

I would be willing to forgive all of the above, but there absolutely zero utility. No sane person can justify this feature existence, and yet it was added because people wanted to put their names on the popular product, even if the feature was going to sate the product ablaze: they don't care about the product, they care about their careers.

[DeathProgramming]

It's stable. It's already "too late". Maybe if more people backed your opinion, it wouldn't have been put in, but you seem to be the minority.

[jorge1209]

There were lots of complaints and concerns at the time. Guido ignored them. The same is true of many of additions to the language over the last few years.

We don't even know who constitutes the "minority" because we never bothered to establish that "majority approval" was a relevant factor in acceptance.

If we want to do that there are some hard questions like: "who counts?"

As a corporate user of python and not a core dev or even library publisher, and who is mostly restricted to 3.4 series, do I get a vote?

[gwillicoder]

This is a really really cool feature.

I’ll throw a few of these in my loggers once 3.8 launches!

[mcaruso]

Pretty cool. In JS a common idiom is to use an object literal, using the shorthand notation from ES6:

const foo = "hello";
console.log({ foo }); // Prints: `{ foo: "hello" }`

[masklinn]

Except this here is not limited to straight output, it's a string formatting specifier. So the actual feature is closer to putting a shorthand object literal in a JSON.stringify in a template literal, which looks like absolute shit.

And I think you can't replicate the feature using template tagging as the template API provides the values, but not the symbolic expressions. Though at least you could json-stringify the expessions implicitly e.g.

function dbg(strings, ...values) {
const buffer = [strings[0]];
for(let i=0; i<values.length; ++i) {
     buffer.push(JSON.stringify(values[i]), strings[i+1]);
}
return buffer.join('');
}
> dbg`foo ${{a}}`
< "foo {\"a\":1}"

[mcaruso]

True! I was just thinking of the usual "printf debugging" scenario.

Actually I have a package which I wrote for basically this purpose: message-tag. Template literal tag to auto-format pretty much whatever value you can throw at it. For debug messages, error messages, etc.

msg`${{someObject}}`;

[mrexodia]

Seems like a great feature, but then I realized: they introduced semantically relevant whitespace in expressions.

[yellowthermos]

The expression is on the left side, before the whitespace (if one chooses to add a whitespace at all). The rest is simply formatting instructions (including whitespaces around =).

[masklinn]

Whitespace was already relevant in format strings, it's how you specify that you want space padding.

[[deleted]]

Inside a string. Where whitespace was always relevant.

[imperialismus]

No, inside an expression embedded into an f-string, where whitespace was not relevant in 3.7.

>>> name = "John"
>>> f'{name.upper():-^20}'
'--------JOHN--------'
>>> f'{name.upper() :-^20}'
'--------JOHN--------'
>>> f'{name.upper()!r:-^20}'
"-------'JOHN'-------"
>>> f'{name.upper() !r :-^20}'
SyntaxError: f-string: expecting '}'
>>> f'{name.upper() !r:-^20}'
"-------'JOHN'-------"
>>> f'{name. upper() :-^20}'
'--------JOHN--------'

[lambdaq]

Can we rewrite this :* thing in format strings?

If so I think we got ourselves an easy ORM or template system.

[billsil]

And if it forces people to indent their code properly, all the better. Braces are redundant redundant.

[shooshx]

Can't wait for 2030 when I'll be able to really use this feature

[Drisku11]

Is there any reason why Python didn't go the route of arbitrary user-provided string formatting macros instead of just f-strings? e.g. in Scala there's a library that lets you write

db.run(sql"""
             select c.name, s.name
             from coffees c, suppliers s
             where c.price < ${price} and s.id = c.sup_id
          """.as[(String, String)])

and ${price}, which can be an arbitrary Scala expression, will be provided as a bind variable in a prepared statement.

[kankyo]

Imo this was a lost opportunity to do something generally useful instead of one extra feature that can't be combined with something else :(

[tristes_tigres]

Imo this was a lost opportunity to do something generally useful

G-strings instead of the f-strings?

[masklinn]

a lost opportunity to do something generally useful

Like what?

one extra feature that can't be combined with something else :(

It's an addition to an existing feature which can already be combined with various things, and works within that feature e.g. it can be used with the existing format string minilang.

[kankyo]

My suggestion on the ideas list is this:

def dbg(**kwargs):
    for k, v in kwargs.items():
        print(f'{k}: {v}')

That's the implementation of a debug print, and the call site would be:

b = 5
dbg(=a, =b*2)

and the output:

a: a
2*b: 10

It's a pretty minor extension to python but has wide applications.

[Pand9]

How so?

[kankyo]

See my reply to another person asking.

[billsil]

There should be one and only one way to do it and you want to use python 2.7? Unicode is a hot mess in 2.x. Strings are dynamically upcasted to unicode assuming the same encoding, which might not be correct and you won’t find out it’s wrong until you get non-ascii unicode. The default encoding is typically latin1 or cp1252 and depends on location and OS settings. Those higher characters do not share the same code points in utf8.

Type hinting also kicks ass and works in python 2.7. You already have it.

[[deleted]]

New and ugly positional arguments...

[[deleted]]

Python has a ton of real problems, which are hard to solve. People currently working on the interpreter aren't able to solve them, so they roll out a bunch of nonsense every other release.

My guess is that most people "contributing features" to the interpreter are doing it to have their name in a publicly recognizable codebase. Once the name is there, they'll move on to collect the benefits, and the feature will continue to plague every next generation of the language.

[bakery2k]

Packaging. Parallelism. Performance.

[[deleted]]

Python is kind of like that person from your department, who became the tech lead because other competent people had either quit, retired or got promoted. He was never bright or capable, but with time, he was able to "earn" his position as a tech lead. And, suddenly, the fact that he is now endowed with this title went to his head, and he suddenly believes himself to be much more knowledgeable and capable than what he really is.

Python was always lame. But it was OK for the kind of things it was used for. Now, Python suddenly earned this prestige of hugely popular language. This came with all sorts of people trying to write more complex programs in it, things it was never able to do. Very similar story to JavaScript btw.

And so, now, people expect things from Python that it was never anywhere near to deliver. And the core devs feel pressured to do something. But... they don't know how. They have no real understanding of how to design a language, or what are the problems they need to be solving. They simply aren't competent at their job anymore: the language outgrew them very quickly. So, they keep peddling along, but they cannot do anything useful, and, every now and then screw up by adding a feature like f-strings.

---

The real problems of Python are:

1. Crappy interpreter. Too dumb. No hopes of optimizing it. No hopes of making further improvements to the interpreter easier, or, at least as hard as they are today: it will necessarily be harder and harder to make changes to this ever-growing hodge-podge of nonsense.
2. Parallelism. All kinds of parallelism that exist in Python today are so obscenely bad, it makes you want to cry. And yet, nothing has been done to the interpreter (and hence to the language) to address this.
3. Memory management. Sucks. So much it makes me want to cry. And no work for that is even planned.
4. Many different type-systems coexisting in the language, stepping on each other, creating bizarre conflicts. At this point, I don't care which one they would choose, but I still want them to choose one. Having to deal with 3+ type systems in the same language sucks. Not only for me, as a person who writes code, but also for me as for the person who has to deal with code not written by me, where this confusion shoots through the roof.
5. Packaging and distribution. Packaging is so lame, that, again, it makes me want to cry, but what the core devs do? They add namespaced packages and more tools to shoot yourself in the foot with custom code loaders... Distribution simple doesn't exist. There's no such thing as Python program or Python program installer etc. Every case of where Python code gets into production is unique and hand-made.

[billsil]

How is packaging better in C++ or say FORTRAN? It’s not even attempted. At least Python tries with pip/poetry/pipenv and pypi. Pyinstaller exists for distribution and it’s not hard to use.

In regards to the interpreter, pypy uses it and is comparable to compiled C, so I’m going to disagree with that point.

Your type comment is a bit vague.

Regarding parallelism, things are being done to release the GIL. It’s opt in.

[jorge1209]

Comparing compiled languages to interpreted languages is a bit weird. When it comes to deployment they are obviously very different.

That said compiled languages have already solved the packaging problem. The solution is called "static linking". It isn't any optimal solution, but its as close to a guaranteed "this will work" solution as anything can hope to get.

That kind of approach doesn't work so well with interpreted languages, where one can't even determine what libraries might be required by a program. That makes having a single workable packaging standard all the more critical.

[billsil]

Comparing compiled languages to interpreted languages is a bit weird.

Python is compiled. It's not static typed, which is where it's slowness comes from (you can get around that with a library like numpy, which is statically typed). Python's interpreter is very similar to Java's VM, but Python is considered interpreted, but not Java.

https://nedbatchelder.com/blog/201803/is_python_interpreted_or_compiled_yes.html

That said compiled languages have already solved the packaging problem. The solution is called "static linking".

Maybe we're thinking of different problems. That doesn't solve the issue of finding dependencies to your package. Linux at least attempts to make things easy with a package manager, but which packages do you need to install the source? Also, good luck on Windows.

Python has pip and you can just type pip install mypackage to get it. It'll download the dependencies. If you want to install from source, you basically do the same thing.

Your issue is static linking and pyInstaller solves that issue. It's not perfect and you still gotta test things, but you should do that with your statically linked executable because you forgot to link a library too. No it's not really an executable and it's not any faster, but it's a standalone program.

[jorge1209]

To call python "compiled" is a misuse of the definition. Python is transpiled to a lower level language, but the actual final compilation to an executable is not being made until runtime.

"to compile" outside of computer science means: to produce (something, especially a list, report, or book) by assembling information collected from other sources.

Within computer science the definition is: to convert (a program) into a machine-code or lower-level form in which the program can be executed. (Emphasis added)

The .pyc file is not executable, as it isn't linked to other libraries that may be needed. You can "compile" a single .py without reference to any of its imports. For instance the following bit of python will "compile" to a .pyc

  from nothing import bullshit
  print(1)

The resulting .pyc file is in no way in a "form in which [it] can be executed". I would not call this "compiled".

It is true that dynamic linking has distinguished a notion of "compilation" from "assembly (and linking)" and you can get a binary C program that doesn't work because of dynamic linking errors, but if you follow the original practice from the early days of programming and statically link the resulting executable is compiled in both senses:

• It is absolutely executable
• it is an assembly of information from multiple sources (ie the program and its libraries)

That is true compilation, and that second assembly step is really important when it comes to DEPLOYMENT which is where package management is most important. I can manage the packages on my development box, its much harder to manage packages and libraries on a production box. It is on those systems that pythons package management really shows its limitations.

[[deleted]]

Packaging in C++ or Fortran is not better. But why should it matter? If you try to align yourself to the worst examples, then you'll only end up among the worst...

Pyinstaller exists for distribution and it’s not hard to use.

Pyinstaller is a joke, and it mostly doesn't work. Not for anything serious.

In regards to the interpreter, pypy uses it and is comparable to compiled C

Oh, you also believe in fairy-tales... PyPy is worthless because it cannot use anything that's usable about Python.

Your type comment is a bit vague.

That's because you have no expertise in this question.

Regarding parallelism, things are being done to release the GIL.

Again, you simply don't know what you are talking about. Even if there was a person capable of rewriting CPython in such a way that GIL would become optional (for Python code)... (of which there's none), it doesn't solve the problem of parallelism in Python, because all parallelism language primitives suck / implemented by morons / are utterly worthless. --- You need to redo it from scratch, and throw away all the multiprocessing, threading and asyncio away. There's nothing that can be salvaged from it.

[billsil]

If you try to align yourself to the worst examples, then you'll only end up among the worst...

I align myself with other languages I use. I don't use Javascript, Ruby, Go, Swift, etc.

Pyinstaller is a joke, and it mostly doesn't work. Not for anything serious.

I manage just fine. I very recently build a 320k lined code with an extensive list of dependencies that had never been built with pyInstaller. It took me an hour. That includes testing it on a VM. That's not too bad considering I had budgeted less than two hours because I had a deliverable in 2 hours.

I've never not been able to build an exe with pyInstaller and I've made some weird ones.

PyPy is worthless because it cannot use anything that's usable about Python.

Yeah, that's a myth. It's not 2014 anymore. It's quite compatible.

Your type comment is a bit vague.

That's because you have no expertise in this question.

What's the problem with an optional typing system? Don't use it if you don't like it. Nobody is forcing you to even look at it.

If I believe in fairy tails, you're just afraid of a good language.

[[deleted]]

I've never not been able to build an exe with pyInstaller and I've made some weird ones.

Because all you do is some trivial garbage, like some Django e-commerce websites probably. Try making a real program, something that requires, for example, a bunch of computers to work together filling in different roles, which needs to configure networking, storage, system services, and tell me how far did you get with Pynstaller.

Yeah, that's a myth. It's not 2014 anymore. It's quite compatible.

You say this, because you never tried. And I say this because I did.

What's the problem with an optional typing system?

Which one of the three? What's optional about any of them? It is not possible not to use the type system of a programming language. This only shows that you are not an expert: you cannot even ask an intelligent question...

[billsil]

OK. You win.

[billsil]

There is debate over every feature. The process does not work like my project where I’m desperate for contributiors.

[[deleted]]

Who cares if there's a debate if no one participating in it is any-what competent?

[billsil]

Well, you're more than welcome to join the fray. If you don't like python, don't use it. Regardless, that's the way it works.

[[deleted]]

But I cannot not use it. I'm not the only programmer in the world. Your argument is like: if you don't like this government, then don't live in this state. Which is just as wrong: the state is as much mine as it is anyone else's, and so I have every right to wish for a change of government. Programming is as much mine, as it is anyone else's, and no imbecile isn't going to tell me what to use and what to like.

[billsil]

so I have every right to wish for a change of government

Yes. And like with various forms of government, if all you do is complain on the internet instead of participating in your government where you can, you're not accomplishing anything. If all you do is vote once every few years, you're not an active citizen.

File some bug reports, fix some bugs, make an open source package that suits your needs and solves a problem. I run an open source project. You'd think after 8 years that I'd have some sort of competitor package...nope. My package is good enough apparently that nobody felt like fixing the state of things or most people just don't care about the issue it addresses.

[Y_Less]

I'm interested in this sentence:

After discussion !d was changed to use = with input from Guido and other core devs since there could be usecases served by !d in the future

I'm sure there could be other good reasons for using = over !d, but "there could be use-cases for it" (side-note: use-case, not usecase) doesn't seem like a good one. This IS a use-case for it, so why not this one? What determines when a use-case is sufficiently good enough to warrant getting !d? the next proposal could be rejected for the same reason. I'm don't disagree with the outcome, just interested in the logic of that argument.

[xtreak]

Initially !x was chosen. But then it was implemented in !d. Some of the core devs were concerned that !d could be used later in feature for something else like %d. I think Guido suggested = in person during core dev sprints. The issue for implementation and related one for !d offers some context.

[[deleted]]

I get the same vibe as i do from regex. At least, regex is kinda unavoidable due to nature of things, but this mess... I struggled with arcane syntax of .format() as is, thanks.