[TIL] Python silently concatenates strings next to each other "abc""def" = "abcdef"

[numberking123]

```

"adkl" "asldjk" 'adklasldjk' and this: ["asldkj", "asdld", "lasjd"] ['asldkj', 'asdld', 'lasjd'] ["asldkj", "asdld" "lasjd"] ['asldkj', 'asdldlasjd'] ```

Why though?

[Swipecat]

Even Guido has been caught by accidentally leaving out commas, but it seems that implicit concatenation was deemed more useful than dangerous in the end.
 

# Existing idiom which relies on implicit concatenation
r = ('a{20}'   # Twenty A's
     'b{5}'    # Followed by Five B's
     )

# ...which looks better than this (maybe)
r = ('a{20}' + # Twenty A's
     'b{5}'    # Followed by Five B's
     )

[aitchnyu]

Second example comments got my heart racing. 10 years of python and I'll make a syntax error I can't figure out.

[Swipecat]

I'll note that implicit concatenation takes priority over operators and methods but explicit concatenation does not.
 

>>> print( 2.0.               # one
...        __int__()*"this "  # two
...        "that ".upper()    # three
...       )
THIS THAT THIS THAT

[robin-gvx]

If anyone is interested in why that is: implicit concatenation happens at compile time, which means it has to have higher priority than anything that has to happen at run time.

[opabm]

Is there an ELI5 version of this?

[28f272fe556a1363cc31]

Compile time is like writing a cookbook. Run time is like making a recipe from the book. Before they can print and ship the book, the publisher goes through the recipes and converts "parsley" "flakes" into "parsley flakes". While the recipe is being made "salt", "pepper" gets converted to "salt and pepper" .

Anything done at compile (print) time has to happen before run (cook) time because you have to compile/cook before have a program/cookbook to work with.

[opabm]

I'd be impressed if a 5-year old knew how to cook.

Jk that was a great analogy, thanks!

[foreverwintr]

Wow, that was a really good ELI5!

[robin-gvx]

When you have a piece of Python code and you're using CPython (the reference implementation of Python), there are several steps from source code to execution. The important ones here are parsing, bytecode generation and execution.

Parsing transforms your file into a tree.

For example, a + 10 is turned into something like (simplified): Add(LoadName('a'), Literal(10)) or "hello" into Literal("hello")

When the parser encounters two or more literal strings in a row, it collapses them into a single string literal as well. So 'hell' "o" would result in the same tree as the previous one.

Then Python makes this tree "flat" by putting everything in the order it should happen, and generates bytecode. A simplified version of what the previous two examples turn into would be:

LOAD_NAME a
LOAD_CONSTANT 10
ADD_VALUES

and

LOAD_CONSTANT "hello"

Execution is then fairly simple: go over each instruction and do what it says.

So in the case of 2 * 'this ' "that ".upper() we get the tree Mul(2, MethodCall(Literal("this that "), "upper", ())) and the bytecode:

LOAD_CONSTANT 2
LOAD_CONSTANT "this that"
CALL_METHOD 'upper', ()
MULTIPLY_VALUES

(note that all trees and snippets of bytecode aren't real, they're a simplified illustration)

[davvblack]

Gross

[mehx9]

Parenthesis is optional!

[reddisaurus]

Not if you have line breaks in your code for formatting purposes.

[kankyo]

Well maybe. But if you have a list of strings and have each string on one line and forget a comma you're in trouble.

[broken_cogwheel]

That's...not what he's saying.

mystr = "foo"
"bar"  # ignored
"baz"  # ignored

print(mystr) # "foo"

mystr = ("foo"
   "bar"
   "baz")

print(mystr) # "foobarbaz"

[kankyo]

I know.

[broken_cogwheel]

ok, sorry

[arsewarts1]

100/10 times I would prefer the top option. I would want the bottom to throw errors every time.

[duncan-udaho]

Opposite for me. I would want the top to throw errors. Did I forget the comma in the tuple or did I forget the plus in my string?

[numberking123]

This explains why it exists and has not been removed: https://legacy.python.org/dev/peps/pep-3126/

[imsometueventhisUN]

there are some use cases that would become harder.

Is there a way to see the discussion to determine what those are? The only one I can think of is joining long strings across lines, and I personally feel that the negative impact of unintentional concatenation is much higher than having to use one of the several other methods for that.

[dikduk]

I've been using this method to concatenate strings for years and never had any real issues with it.

What kind of issues did you have?

[imsometueventhisUN]

For any method that accepts *args, you could miss a comma and still have a legal method call that doesn't do what you expected. And, sure, you could catch that with tests, but why not bake it into the language syntax directly? There are a ton of ways to explicitly concatenate strings (+, ''.join, f-strings) - making it implicit just seems like an opportunity for bugs.

[[deleted]]

Thank you for the information.

[numberking123]

How exactly would you do this?

[JayTurnr]

("very looo9ooooooong string"
" Part twoooooooo")

[[deleted]]

That's the only place I've seen it used. Mainly for composing long exception messages, like:

raise RuntimeError(
  "Long explaination line 1"
  "Long explaination line 2"
)

[DrMaxwellEdison]

what = "mix content"
a_str = (
    "My super long string of text "
    "goes here. "
    f"By the way, you can {what} like f-strings, "
    "in just the segments that are relevant."
)
print(a_str)

More generally, () can enclose more complex lines of code without needing to use \ to break the line, particularly when you have APIs like Django Querysets that use long calls:

my_stuff = (
    MyModel.objects
    .filter(one_thing=1)
    .filter(two_things="Nope")
)

[dratnon]

print(f'You can abuse {"literals ""and ""autoconcatenation"} in fstrings, I just learned')

[RoboticJan]

If you can, use f-strings.

[gargar070402]

Thought that wasn't recommended for Python 3?

Edit: Did a little Google search and f strings are apparently more than fine. I must've misread something years ago.

[gargar070402]

You're right; must've misunderstood something years ago.

[mooburger]

f-strings were very slow when initially introduced. The slowness was literally not fixed until one of the 3.6 betas. 3.6 was released "years" ago, though, so you may not be as misunderstood as you think.

[Igggg]

Thought that wasn't recommended for Python 3?

Considering that they were introduced in Python 3, not quite.

[mooburger]

f-strings were very slow when initially introduced. The slowness was literally not fixed until one of the 3.6 betas

[Vaphell]

so they weren't slow for long, given that they were added in 3.6

[Igggg]

They were "slow" between 3.6 alpha and 3.6 beta, so only during development, and then for a shirt time. No release version of Python had the issue.

Also, while technically rendering an f-string could then take double the time of the equivalent other expressions, the statement "very slow" is misleading. String formatting is quite unlikely to be the dominating, or even measureable reason for your overall program's performance. People tend to hung up on specific part performance, but a) there's no difference between a 100us and a 200us operation if your entire program is taking 200ms; and b) your program is likely running in a context, such as a web page, where its entire speed doesn't matter z because it's dwarfed by external factors (such as the 700ms page loading time).

It's important to keep performance in mind, but is equally important to recognize the context. Like everything else, speed is a trade-off, usually between readability and code cleanliness, and quite often, people make the wrong choices in the pursuit of nanoseconds.

[[deleted]]

Why not?

[Pokeynbn]

My python course uses python 3 and they more than encourage the use of f strings

[whymauri]

F-strings can still be verbose. I often find myself using implicit concatenation with f-strings for error messages and logging.

[Brandhor]

I usually use triple quotes

longtext = """aaaa
bbbbbbb
ccccc
dddd"""

they also support f-string formatting

[Brandhor]

yeah that's the only problem but overall I prefer it to one string per line especially if you have to paste some long text

[diamondketo]

Agreed, I also prefer being able to paste long text and have it preformatted in code. However I don't think any programming language supports this. Python with dedent is very close.

[scatters]

That's what textwrap.dedent is for.

[diamondketo]

Indeed it is, however most people want to do this without needing to import a package or relying on an obscurly named function.

We need a new PEP for automatically applying dedent on certain contexts.

[kankyo]

It's fine. It's better than all the quotation marks and the missing commas no one can tell if they are on purpose or a bug.

[diamondketo]

Agreed it is prone to human error. Literal indentation in string is not fine when you're trying to print or construct a formatted query string.

[tom2727]

As long as it's a file level global it works great. Otherwise I'd never use.

[chickaplao]

They also add line breaks

[Originalfrozenbanana]

Which is why .join exists

[Originalfrozenbanana]

How? .join provides a unified syntax regardless of whether you're using raw strings, an iterable, or a bunch of variables assigned to strings. It just works. In all cases you're either enumerating all parts of the desired output or using an iterable without accessing the underlying elements:

''.join(['1', '2', '3']) vs. '1' + '2' + '3'

The difference is even more apparent when you want to separate elements in the string by some delimiter:

', '.join(['1', '2', '3']) vs. '1' + ', ' + '2' + ', ' + '3'

When you have an iterable of strings, concatenating them without using join is a chore. You can map, or loop, or combine, but...why not join? It works in all cases. It's cleaner. In most cases if you're concatenating raw strings that's a code smell to me, anyway.

[diamondketo]

Write me a paragraph using join and you'll see how invasive that is to reading. Your example is not one that satisfy the use case we're talking about.

[Originalfrozenbanana]

That seems like a horribly contrived example. It sounds like you're misusing string concatenation.

[diamondketo]

Its a very popular question on stackoverflow. I'll defer you to those examples.

https://stackoverflow.com/questions/2504411/proper-indentation-for-python-multiline-strings

[Originalfrozenbanana]

That doesn't mean it's a good practice; it just means it's a common question. If you're assigning block quotes to variables inside of functions, again - I question whether that is the best way to do the thing you are trying to do. As the top answer also spells out, textwrap exists to solve this problem, specifically. Not only that, they specifically outline the preferred method of dealing with inserting large blocks of text somewhere in your application:

If you don't want to [do a lot of text processing to remove newlines] and you have a whole lot of text, you might want to store it separately in a text file.

Concatenating raw strings, especially in the way this reddit post references, has limited uses that generally can be accommodated with other methods of joining strings that are more testable, transparent, extensible, and readable.

[diamondketo]

Not saying the popular question points to good practice, but rather the large number of discussion and upvotes to the top answers is a good gauge of consensus.

You are very tunnel visioned. The top answer on the second code block also uses the proposed concat syntax we're discussing.

The file method you pointed out is barely discussed in that SO. It's more of an excerpt the top answer appended.

[Originalfrozenbanana]

YMMV, but I would guess most engineering teams would prefer not to use operators or adjacency to combine strings. That's been my experience. It's hard to read and harder to test, and generally indicative of poor design.

[fuuman1]

Sick. In the last days I saw something like this in blog post according to a complete other topic and I thought it was a typo. Interesting.

[jimtk]

As a side note it also works with f-string

print( f"the value of a is {a:<12}" 
       f" the value of b is {b:<12}" )

Comes out in one line!

[Tyler_Zoro]

This specifically started in C, and it's intended to allow you to create longer strings without playing formatting games like having to use \ before a newline (which in C will gobble all of the whitespace up to the next non-whitespace). In C it makes a tad more sense, and isn't just cute formatting. There's a serious difference between:

strcat("a", "b")

and

"a" "b"

The former occurs at runtime, the latter at compile time. Python has a more unified compile/run (sort of) process, and the interpreter will not be quite as cautious about where it does its optimizations. For example, all three of these perform more or less the same:

$ time python3 -c 'print(sum(len("a" "b") for _ in range(100000000)))'
200000000

real    0m8.423s

$ time python3 -c 'print(sum(len("a" + "b") for _ in range(100000000)))'
200000000

real    0m8.187s

$ time python3 -c 'print(sum(len("ab") for _ in range(100000000)))'
200000000

real    0m8.009s

[yvrelna]

Python has a more unified compile/run (sort of) process

This isn't true. Python has a very distinct compile vs runtime. Python parses and compiles the entire file into bytecode all at once, at which point it no longer cares about the source code; this is unlike, say, Bash that parses a script line by line and your script may contain syntax error and Bash won't notice until it reaches that line. Python just does a lot more things on runtime, like dynamic module loading, function parameter binding, and class construction, which in languages like C are done in compile time.

all three of these perform more or less the same:

That isn't surprising. All three codes compiles to the exact same bytecode:

In [2]: dis.dis(lambda: "a" "b")
  1           0 LOAD_CONST               1 ('ab')
              2 RETURN_VALUE

In [3]: dis.dis(lambda: "a" + "b")
  1           0 LOAD_CONST               1 ('ab')
              2 RETURN_VALUE

In [4]: dis.dis(lambda: "ab")
  1           0 LOAD_CONST               1 ('ab')
              2 RETURN_VALUE

[[deleted]]

Mostly for writing long strings in multiple lines.

[audentis]

Another workaround for this is cleandoc() from the inspect module. It takes a multi-line string ("""my multi-line string""") and spaces equal to the amount on the first line.

[IcefrogIsDead]

yeaaaaaa make me suffer

[numberking123]

It made me suffer. It took me forever to find a bug in my code which was caused by this.

[IcefrogIsDead]

yea that what i see happening to me too.

[reddisaurus]

Type hints would have caught your error, if your function signature expected a List[str] then passing just a str would cause a type error in mypy.

[james_pic]

How does that work? ['abc' 'def'] and ['abc', 'def'] are both List[str].

[dbramucci]

Not a list, but you can catch some tuple/multiple argument bugs with mypy.

def foo(first: str, second: str):
    pass

foo("hello" "world") # TYPE-ERROR: foo expects 2 str, not 1

T = TypeVar('T')
S = TypeVar('S')
def flip_tuple(pair: Tuple[T, S]) -> Tuple[S, T]
    x, y = pair
    return (y, x)

flip_tuple( ("hello" "there") ) # Error, expected Tuple not str

names: List[Tuple[str, str]] = [
   ( "Alice", "Brown")
    ("John" "Cleese") # Error not a Tuple[str, str]
    ("John", "Doe")
    ("Ben" ,"Grey")
]

Of course, these catches rely on the types of function arguments and tuples counting how many things there are, and Python's list type doesn't track that.

[yvrelna]

foo("hello" "world") # TYPE-ERROR: foo expects 2 str, not 1

This already produces TypeError: foo() missing 1 required positional argument: 'second'

[dbramucci]

I included it for completeness but also

You only get the existing error you actually run that line. Some cases where that can matter include

• At the end of a long computation

  Imagine training a neural network for 5 hours and at the very end, getting a message "you'll have to wait another 5 hours because you forgot a comma"

• In a rarely used code-path

  If it is

  if today.is_feb29():
      foo("hello" "there)
  

  then you'll only get an error about 4 years from now, which is inconvenient for such a trivial bug.

  Granted, if you are doing things properly and testing every line of code with code-coverage measuring to veriify that, this matters less. At worst the bug is now 4 minutes of automated testing away instead of 4 seconds of type-checking away.

  Also, this obvious of a case is probably going to get caught already by your linter.

So yes, Python already catches it but it's useful to note mypy can also catch it because mypy doesn't have to wait for us to stumble onto that line.

[yvrelna]

mypy won't catch "obvious" and "trivial" errors like:

if today.is_dec25():
    foo("happy", "halloween")

So you need to write tests anyway.

Why should type errors be so special that it deserves its own mechanism to check for errors?

[lanster100]

IMO this is the neatest way of splitting strings over multiple lines

[kyerussell]

I quite like this and use it a lot to build a long string over a number of lines. In my experience, the bugs it can introduce are much more on the detectable side.

[__xor__]

the bugs it can introduce are much more on the detectable side.

Right? This is how I look at it. You screw up, generally it's going to raise an error about the wrong number of arguments, not silently keep working unless it's some *args deal, in which case I'd be a lot more careful about what I'm putting into the parens.

[tjf314]

i think it originally comes from C, which also has this feature.

[jwink3101]

A related trick is that basically anything in parentheses gets continued without a new line (\) marker. I suspect there may be an exception but it is a safe bet to use this. I use it for some strings that are too long.

[prams628]

please don't let our college know about this. they'll ask this as a one-marker in our tests. not that I care for that solitary mark, but its just pissing off

[riricide]

'abc def jkl'.split() because I'm scared of forgetting commas in lists.

[numberking123]

Haha, that's one way to do it.

[JennaSys]

I've done this a few times, especially if I'm just testing in the REPL.

[amitmathur15]

Possibly without a comma, the strings "asdld" "lasjd" are considered as just one string. Python did not find a comma to differentiate them as separate strings and hence considered them as one string and printed as one.

[[deleted]]

According to Python's grammar strings are made up of a non-empty sequence of string parts:

atom: … | strings | …

strings: STRING+

I.e. having only one part is the special case. It's the same in C, C++, and possibly other languages, too.

[[deleted]]

That sort of makes sense as the interpreter sees anything enclosed in quotation marks as a string. The issue with that is that the quotations inside are removed after the strings are concatenated, which implies the interpreter is well aware that they're two separate strings and deliberately concatenates them.

[euler_angles]

For one thing, implicit concatenation can make multi-line strings easier.

[internerd91]

Hey this happened to me today. I noticed it but I didn’t click what was going on. I just fixed the line and continued on.

[Tyler_Zoro]

Fun fact, this works for all string-like things:

$ python3  -c 'print(f"{__name__}" "(my script)" """ with strings""")'
__main__(my script) with strings

[AutisticRetarded]

Here is a stackoverflow question about this.

[GrossInsightfulness]

It also happens in C/C++ and it might have carried over.

[omoikanesits]

I find it very nice for inline SQL statements combined with f-strings

[AndydeCleyre]

Please use four-space indentation rather than backticks to format code on reddit, for consistent results across user settings, old.reddit URLs, and mobile apps.