Which is the preferred way to concatenate a string in Python

Question

Since Python s string can t be changed  I was wondering how to concatenate a string more efficiently   I can write like it   s    stringfromelsewhere   or like this   s      s append somestring   later  s      join s    While writing this question  I found a good article talking about the topic   http   www skymind com  ocrow python string   But it s in Python 2 x   so the question would be did something change in Python 3

User · Answer

In Python    3 6  the new f-string is an efficient way to concatenate a string    gt  gt  gt  name    some name   gt  gt  gt  number   123  gt  gt  gt   gt  gt  gt  f Name is  name  and the number is  number     Name is some name and the number is 123

User · Answer

While somewhat dated  Code Like a Pythonista  Idiomatic Python recommends join   over   in this section  As does PythonSpeedPerformanceTips in its section on string concatenation  with the following disclaimer      The accuracy of this section is disputed with respect to later   versions of Python  In CPython 2 5  string concatenation is fairly   fast  although this may not apply likewise to other Python   implementations  See ConcatenationTestCode for a discussion

User · Answer

You write this function  def str join  args       return    join map str  args     Then you can call simply wherever you want  str join  Pine      Returns   Pine str join  Pine    apple      Returns   Pineapple str join  Pine    apple   3     Returns   Pineapple3

User · Answer

Using in place string concatenation by     is THE WORST method of concatenation in terms of stability and cross implementation as it does not support all values  PEP8 standard discourages this and encourages the use of format    join   and append   for long term use    As quoted from the linked  Programming Recommendations  section      For example  do not rely on CPython s efficient implementation of in-place string concatenation for statements in the form a    b or a   a   b  This optimization is fragile even in CPython  it only works for some types  and isn t present at all in implementations that don t use refcounting  In performance sensitive parts of the library  the    join   form should be used instead  This will ensure that concatenation occurs in linear time across various implementations

User · Answer

The best way of appending a string to a string variable is to use   or     This is because it s readable and fast  They are also just as fast  which one you choose is a matter of taste  the latter one is the most common  Here are timings with the timeit module   a   a   b  0 11338996887207031 a    b  0 11040496826171875   However  those who recommend having lists and appending to them and then joining those lists  do so because appending a string to a list is presumably very fast compared to extending a string  And this can be true  in some cases  Here  for example  is one million appends of a one-character string  first to a string  then to a list   a    b  0 10780501365661621 a append b   0 1123361587524414   OK  turns out that even when the resulting string is a million characters long  appending was still faster   Now let s try with appending a thousand character long string a hundred thousand times   a    b  0 41823482513427734 a append b   0 010656118392944336   The end string  therefore  ends up being about 100MB long  That was pretty slow  appending to a list was much faster  That that timing doesn t include the final a join    So how long would that take    a join a   0 43739795684814453   Oups  Turns out even in this case  append join is slower   So where does this recommendation come from  Python 2   a    b  0 165287017822 a append b   0 0132720470428 a join a   0 114929914474   Well  append join is marginally faster there if you are using extremely long strings  which you usually aren t  what would you have a string that s 100MB in memory    But the real clincher is Python 2 3  Where I won t even show you the timings  because it s so slow that it hasn t finished yet  These tests suddenly take minutes  Except for the append join  which is just as fast as under later Pythons   Yup  String concatenation was very slow in Python back in the stone age  But on 2 4 it isn t anymore  or at least Python 2 4 7   so the recommendation to use append join became outdated in 2008  when Python 2 3 stopped being updated  and you should have stopped using it   -    Update  Turns out when I did the testing more carefully that using   and    is faster for two strings on Python 2 3 as well  The recommendation to use    join   must be a misunderstanding   However  this is CPython  Other implementations may have other concerns  And this is just yet another reason why premature optimization is the root of all evil  Don t use a technique that s supposed  faster  unless you first measure it   Therefore the  best  version to do string concatenation is to use   or     And if that turns out to be slow for you  which is pretty unlikely  then do something else   So why do I use a lot of append join in my code  Because sometimes it s actually clearer  Especially when whatever you should concatenate together should be separated by spaces or commas or newlines

User · Answer

You can do in different ways   str1    Hello  str2    World  str list     Hello    World   str dict     str1    Hello    str2    World      Concatenating With the   Operator print str1         str2     Hello World    String Formatting with the   Operator print   s  s     str1  str2      Hello World    String Formatting with the     Operators with str format   print        format str1  str2      Hello World print   0  1   format str1  str2      Hello World print   str1   str2   format str1 str dict  str1    str2 str dict  str2        Hello World print   str1   str2   format   str dict      Hello World    Going From a List to a String in Python With  join   print     join str list      Hello World    Python f strings -- gt  3 6 onwards print f  str1   str2       Hello World   I created this little summary through following articles    Python 3 s f-Strings  An Improved String Formatting Syntax  Guide   Speed test also included  Formatted string literals String Concatenation and Formatting Splitting  Concatenating  and Joining Strings in Python

User · Answer

If the strings you are concatenating are literals  use String literal concatenation  re compile            A-Za-z            letter or underscore           A-Za-z0-9         letter  digit or underscore         This is useful if you want to comment on part of a string  as above  or if you want to use raw strings or triple quotes for part of a literal but not all   Since this happens at the syntax layer it uses zero concatenation operators

User · Answer

You can use this more efficient  too   https   softwareengineering stackexchange com questions 304445 why-is-s-better-than-for-concatenation   s      s    stringfromelsewhere

User · Answer

As  jdi mentions Python documentation suggests to use str join or io StringIO for string concatenation  And says that a developer should expect quadratic time from    in a loop  even though there s an optimisation since Python 2 4  As this answer says      If Python detects that the left argument has no other references  it calls realloc to attempt to avoid a copy by resizing the string in place  This is not something you should ever rely on  because it s an implementation detail and because if realloc ends up needing to move the string frequently  performance degrades to O n 2  anyway    I will show an example of real-world code that naively relied on    this optimisation  but it didn t apply  The code below converts an iterable of short strings into bigger chunks to be used in a bulk API   def test concat chunk seq  split by       result            for item in seq          if len result -1     len item   gt  split by               result append             result -1     item     return result   This code can literary run for hours because of quadratic time complexity  Below are alternatives with suggested data structures   import io  def test stringio chunk seq  split by       def chunk            buf   io StringIO           size   0         for item in seq              if size   len item   lt   split by                  size    buf write item              else                  yield buf getvalue                   buf   io StringIO                   size   buf write item          if size              yield buf getvalue        return list chunk     def test join chunk seq  split by       def chunk            buf              size   0         for item in seq              if size   len item   lt   split by                  buf append item                  size    len item              else                  yield    join buf                                  buf clear                   buf append item                  size   len item          if size              yield    join buf       return list chunk      And a micro-benchmark   import timeit import random import string import matplotlib pyplot as plt  line      join random choices      string ascii uppercase   string digits  k 512       n  x      y concat      y stringio      y join      n   5 for i in range 1  11       x append i      seq    line     20   2    20    len line       chunk size   i   2    20     y concat append          timeit timeit lambda  test concat chunk seq  chunk size   number n    n      y stringio append          timeit timeit lambda  test stringio chunk seq  chunk size   number n    n      y join append          timeit timeit lambda  test join chunk seq  chunk size   number n    n  plt plot x  y concat  plt plot x  y stringio  plt plot x  y join  plt legend   concat    stringio    join    loc  upper left   plt show

User · Answer

If you are concatenating a lot of values  then neither  Appending a list is expensive  You can use StringIO for that  Especially if you are building it up over a lot of operations   from cStringIO import StringIO   python3   from io import StringIO  buf   StringIO    buf write  foo   buf write  foo   buf write  foo    buf getvalue      foofoofoo    If you already have a complete list returned to you from some other operation  then just use the    join aList   From the python FAQ  What is the most efficient way to concatenate many strings together      str and bytes objects are immutable  therefore concatenating many   strings together is inefficient as each concatenation creates a new   object  In the general case  the total runtime cost is quadratic in   the total string length       To accumulate many str objects  the recommended idiom is to place them   into a list and call str join   at the end   chunks      for s in my strings      chunks append s  result      join chunks         another reasonably efficient idiom is to use io StringIO       To accumulate many bytes objects  the recommended idiom is to extend a   bytearray object using in-place concatenation  the    operator    result   bytearray   for b in my bytes objects      result    b      Edit  I was silly and had the results pasted backwards  making it look like appending to a list was faster than cStringIO  I have also added tests for bytearray str concat  as well as a second round of tests using a larger list with larger strings   python 2 7 3   ipython test example for large lists of strings  try      from cStringIO import StringIO except      from io import StringIO  source     foo   1000    timeit buf   StringIO   for i in source      buf write i  final   buf getvalue     1000 loops  best of 3  1 27 ms per loop    timeit out      for i in source      out append i  final      join out    1000 loops  best of 3  9 89 ms per loop    timeit out   bytearray   for i in source      out    i   10000 loops  best of 3  98 5   s per loop    timeit out      for i in source      out    i   10000 loops  best of 3  161   s per loop     Repeat the tests with a larger list  containing    strings that are bigger than the small string caching     done by the Python source     foo   1000    cStringIO   10 loops  best of 3  19 2 ms per loop    list append and join   100 loops  best of 3  144 ms per loop    bytearray        100 loops  best of 3  3 8 ms per loop    str        100 loops  best of 3  5 11 ms per loop

User · Answer

The recommended method is still to use append and join

User · Answer

my use case was slight different  I had to construct a query where more then 20 fields were dynamic  I followed this approach of using format method  query    insert into  0   1   2   3   values  4    5    6    query format  users   name   age   dna   suzan  1010  nda     this was comparatively simpler for me instead of using   or other ways

[python] Which is the preferred way to concatenate a string in Python?

Examples related to python

Examples related to string

Examples related to python-3.x

Examples related to concat