How to replace multiple substrings of a string

Question

I would like to use the  replace function to replace multiple strings   I currently have    string replace  condition1         but would like to have something like  string replace  condition1       replace  condition2    text     although that does not feel like good syntax  what is the proper way to do this  kind of like how in grep regex you can do  1 and  2 to replace fields to certain search strings

User · Answer

Here my  0 02  It is based on Andrew Clark s answer  just a little bit clearer  and it also covers the case when a string to replace is a substring of another string to replace  longer string wins   def multireplace string  replacements               Given a string and a replacement map  it returns the replaced string        param str string  string to execute replacements on      param dict replacements  replacement dictionary  value to find  value to replace       rtype  str                Place longer ones first to keep shorter substrings from matching       where the longer ones should take place       For instance given the replacements   ab    AB    abc    ABC   against        the string  hey abc   it should produce  hey ABC  and not  hey ABc      substrs   sorted replacements  key len  reverse True         Create a big OR regex that matches any of the substrings to replace     regexp   re compile     join map re escape  substrs           For each match  look up the new string in the replacements     return regexp sub lambda match  replacements match group 0    string    It is in this this gist  feel free to modify it if you have any proposal

User · Answer

You should really not do it this way  but I just find it way too cool    gt  gt  gt  replacements     cond1   text1    cond2   text2    gt  gt  gt  cmd    answer   s   gt  gt  gt  for k v in replacements iteritems     gt  gt  gt      cmd      replace  s   s     k v   gt  gt  gt  exec cmd    Now  answer is the result of all the replacements in turn  again  this is very hacky and is not something that you should be using regularly  But it s just nice to know that you can do something like this if you ever need to

User · Answer

Note  Test your case  see comments  Here s a sample which is more efficient on long strings with many small replacements  source    quot Here is foo  it does moo  quot   replacements          is    was     replace  is  with  was       does    did                   def replace source  replacements       finder   re compile  quot   quot  join re escape k  for k in replacements keys       matches every string we want replaced     result          pos   0     while True          match   finder search source  pos          if match                cut off the part up until match             result append source pos   match start                   cut off the matched part and replace it in place             result append replacements source match start     match end                  pos   match end           else                the rest after the last match             result append source pos                break     return  quot  quot  join result   print replace source  replacements   The point is in avoiding many concatenations of long strings  We chop the source string to fragments  replacing some of the fragments as we form the list  and then join the whole thing back into a string

User · Answer

this is my solution to the problem  I used it in a chatbot to replace the different words at once     def mass replace text  dct       new string          old string   text     while len old string   gt  0          s              sk              for k in dct keys                if old string startswith k                   s   dct k                  sk   k         if s              new string  s             old string   old string len sk            else              new string  old string 0              old string   old string 1       return new string  print mass replace  The dog hunts the cat     dog   cat    cat   dog        this will become The cat hunts the dog

User · Answer

I feel this question needs a single-line recursive lambda function answer for completeness  just because  So there    gt  gt  gt  mrep   lambda s  d  s if not d else mrep s replace  d popitem     d    Usage    gt  gt  gt  mrep  abcabc     a    1    c    2     1b21b2    Notes    This consumes the input dictionary   Python dicts preserve key order as of 3 6  corresponding caveats in other answers are not relevant anymore  For backward compatibility one could resort to a tuple-based version     gt  gt  gt  mrep   lambda s  d  s if not d else mrep s replace  d pop     d   gt  gt  gt  mrep  abcabc      a    1      c    2         Note  As with all recursive functions in python  too large recursion depth  i e  too large replacement dictionaries  will result in an error  See e g  here

User · Answer

This is just a more concise recap of F J and MiniQuark great answers  All you need to achieve multiple simultaneous string replacements is the following function   def multiple replace string  rep dict       pattern   re compile     join  re escape k  for k in sorted rep dict key len reverse True     flags re DOTALL      return pattern sub lambda x  rep dict x group 0    string    Usage    gt  gt  gt multiple replace  Do you like cafe  No  I prefer tea      cafe   tea    tea   cafe    like   prefer     Do you prefer tea  No  I prefer cafe     If you wish  you can make your own dedicated replacement functions starting from this simpler one

User · Answer

Here is a short example that should do the trick with regular expressions   import re  rep     condition1        condition2    text     define desired replacements here    use these three lines to do the replacement rep   dict  re escape k   v  for k  v in rep iteritems      Python 3 renamed dict iteritems to dict items so use rep items   for latest versions pattern   re compile     join rep keys     text   pattern sub lambda m  rep re escape m group 0     text    For example    gt  gt  gt  pattern sub lambda m  rep re escape m group 0       condition1  and --condition2--       and --text--

User · Answer

Another example   Input list  error list      br      ex     Something   words     how    much ex     is br     the    fish br     noSomething    really     The desired output would be  words     how    much    is    the    fish    no    really     Code     n 0  0  if len n 0   else n 1  for n in    w replace e     for e in error list if e in w  w  for w in words

User · Answer

Or just for a fast hack   for line in to read      read buffer   line                   stripped buffer1   read buffer replace  term1            stripped buffer2   stripped buffer1 replace  term2            write to file   to write write stripped buffer2

User · Answer

Here is another way of doing it with a dictionary   listA  The cat jumped over the house  split   modify    word word for number word in enumerate listA   modify  cat   modify  jumped    dog   walked  print     join modify x  for x in listA

User · Answer

You could just make a nice little looping function  def replace all text  dic       for i  j in dic iteritems            text   text replace i  j      return text  where text is the complete string and dic is a dictionary     each definition is a string that will replace a match to the term  Note  in Python 3  iteritems   has been replaced with items    Careful  Python dictionaries don t have a reliable order for iteration  This solution only solves your problem if   order of replacements is irrelevant it s ok for a replacement to change the results of previous replacements  Update  The above statement related to ordering of insertion does not apply to Python versions greater than or equal to 3 6  as standard dicts were changed to use insertion ordering for iteration  For instance  d      quot cat quot    quot dog quot    quot dog quot    quot pig quot   my sentence    quot This is my cat and this is my dog  quot  replace all my sentence  d  print my sentence   Possible output  1   This is my pig and this is my pig   Possible output  2  This is my dog and this is my pig   One possible fix is to use an OrderedDict  from collections import OrderedDict def replace all text  dic       for i  j in dic items            text   text replace i  j      return text od   OrderedDict    quot cat quot    quot dog quot      quot dog quot    quot pig quot     my sentence    quot This is my cat and this is my dog  quot  replace all my sentence  od  print my sentence   Output   quot This is my pig and this is my pig  quot    Careful  2  Inefficient if your text string is too big or there are many pairs in the dictionary

User · Answer

Here is a variant of the first solution using reduce  in case you like being functional      repls     hello     goodbye    world     earth   s    hello  world  reduce lambda a  kv  a replace  kv   repls iteritems    s    martineau s even better version   repls     hello    goodbye      world    earth   s    hello  world  reduce lambda a  kv  a replace  kv   repls  s

User · Answer

You can use the pandas library and the replace function which supports both exact matches as well as regex replacements  For example   df   pd DataFrame   text     Billy is going to visit Rome in November    I was born in 10 10 2010    I will be there at 20 00      to replace   Billy   Rome   January February March April May June July August September October November December     d 2   d 2      d 2   d 2   d 4    replace with   name   city   month   time    date    print df text replace to replace  replace with  regex True     And the modified text is   0    name is going to visit city in month 1                      I was born in date 2                 I will be there at time   You can find an example here  Notice that the replacements on the text are done with the order they appear in the lists

User · Answer

I needed a solution where the strings to be replaced can be a regular expressions  for example to help in normalizing a long text by replacing multiple whitespace characters with a single one  Building on a chain of answers from others  including MiniQuark and mmj  this is what I came up with   def multiple replace string  reps  re flags   0           Transforms string  replacing keys from re str dict with values      reps  dictionary  or list of key-value pairs  to enforce ordering            earlier items have higher priority             Keys are used as regular expressions      re flags  interpretation of regular expressions  such as re DOTALL             if isinstance reps  dict           reps   reps items       pattern   re compile     join    P lt   d gt  s      i  re str 0                                     for i  re str in enumerate reps                             re flags      return pattern sub lambda x  reps int x lastgroup 1     1   string    It works for the examples given in other answers  for example    gt  gt  gt  multiple replace   condition1  and --condition2--                          condition1        condition2    text        and --text--    gt  gt  gt  multiple replace  hello  world     hello     goodbye    world     earth     goodbye  earth    gt  gt  gt  multiple replace  Do you like cafe  No  I prefer tea                           cafe    tea    tea    cafe    like    prefer     Do you prefer tea  No  I prefer cafe     The main thing for me is that you can use regular expressions as well  for example to replace whole words only  or to normalize white space    gt  gt  gt  s    I don t want to change this name  n  Philip II of Spain   gt  gt  gt  re str dict    r  bI b    You   r   n t            gt  gt  gt  multiple replace s  re str dict   You don t want to change this name  Philip II of Spain    If you want to use the dictionary keys as normal strings  you can escape those before calling multiple replace using e g  this function   def escape keys d           transform dictionary d by applying re escape to the keys         return dict  re escape k   v  for k  v in d items      gt  gt  gt  multiple replace s  escape keys re str dict    I don t want to change this name  n  Philip II of Spain    The following function can help in finding erroneous regular expressions among your dictionary keys  since the error message from multiple replace isn t very telling    def check re list re list           Checks if each regular expression in list is well-formed          for i  e in enumerate re list           try              re compile e          except  TypeError  re error               print  Invalid regular expression string                      at position           format i  e     gt  gt  gt  check re list re str dict keys      Note that it does not chain the replacements  instead performs them simultaneously  This makes it more efficient without constraining what it can do  To mimic the effect of chaining  you may just need to add more string-replacement pairs and ensure the expected ordering of the pairs    gt  gt  gt  multiple replace  button     but    mut    mutton    lamb     mutton   gt  gt  gt  multiple replace  button      button    lamb                                      but    mut      mutton    lamb      lamb

User · Answer

I built this upon F J s excellent answer   import re  def multiple replacer  key values       replace dict   dict key values      replacement function   lambda match  replace dict match group 0       pattern   re compile     join  re escape k  for k  v in key values    re M      return lambda string  pattern sub replacement function  string   def multiple replace string   key values       return multiple replacer  key values  string    One shot usage    gt  gt  gt  replacements    u caf     u tea     u tea   u caf       u like   u love    gt  gt  gt  print multiple replace u Do you like caf    No  I prefer tea     replacements  Do you love tea  No  I prefer caf      Note that since replacement is done in just one pass   caf    changes to  tea   but it does not change back to  caf      If you need to do the same replacement many times  you can create a replacement function easily    gt  gt  gt  my escaper   multiple replacer                 t      t     gt  gt  gt  many many strings    u This text will be escaped by  my escaper                           u Does this work  tYes it does                          u And can we span nmultiple lines  t Yes twe tcan      gt  gt  gt  for line in many many strings          print my escaper line       This text will be escaped by   my escaper   Does this work  tYes it does And can we span multiple lines  t  Yes twe tcan      Improvements    turned code into a function added multiline support fixed a bug in escaping easy to create a function for a specific multiple replacement   Enjoy    -

User · Answer

I would like to propose the usage of string templates  Just place the string to be replaced in a dictionary and all is set   Example from docs python org   gt  gt  gt  from string import Template  gt  gt  gt  s   Template   who likes  what    gt  gt  gt  s substitute who  tim   what  kung pao    tim likes kung pao   gt  gt  gt  d   dict who  tim    gt  gt  gt  Template  Give  who  100   substitute d  Traceback  most recent call last         ValueError  Invalid placeholder in string  line 1  col 10  gt  gt  gt  Template   who likes  what   substitute d  Traceback  most recent call last         KeyError   what   gt  gt  gt  Template   who likes  what   safe substitute d   tim likes  what

User · Answer

I don t know about speed but this is my workaday quick fix   reduce lambda a  b  a replace  b           o   W      t   X     iterable of pairs   oldval  newval         tomato   The string from which to replace values             but I like the  1 regex answer above   Note - if one new value is a substring of another one then the operation is not commutative

User · Answer

In my case  I needed a simple replacing of unique keys with names  so I thought this up   a    This is a test string   b     i    I    s    S   for x y in b items        a   a replace x  y   gt  gt  gt  a  ThIS IS a teSt StrIng

User · Answer

I was struggling with this problem as well  With many substitutions regular expressions struggle  and are about four times slower than looping string replace  in my experiment conditions    You should absolutely try using the Flashtext library  blog post here  Github here   In my case it was a bit over two orders of magnitude faster  from 1 8 s to 0 015 s  regular expressions took 7 7 s  for each document   It is easy to find use examples in the links above  but this is a working example       from flashtext import KeywordProcessor     self processor   KeywordProcessor case sensitive False      for k  v in self my dict items            self processor add keyword k  v      new string   self processor replace keywords string    Note that Flashtext makes substitutions in a single pass  to avoid a --  b and b --  c translating  a  into  c    Flashtext also looks for whole words  so  is  will not match  this    It works fine if your target is several words  replacing  This is  by  Hello

User · Answer

Why not one solution like this   s    The quick brown fox jumps over the lazy dog  for r in    brown    red      lazy    quick         s   s replace  r    output will be   The quick red fox jumps over the quick dog

User · Answer

For replace only one character  use the translate and str maketrans is my favorite method    tl dr   result string   your string translate str maketrans dict mapping      demo  my string    This is a test string   dict mapping     i    s    s    S   result good   my string translate str maketrans dict mapping   result bad   my string for x  y in dict mapping items        result bad   result bad replace x  y  print result good     ThsS sS a teSt Strsng  print result bad      ThSS SS a teSt StrSng

User · Answer

Starting Python 3 8  and the introduction of assignment expressions  PEP 572      operator   we can apply the replacements within a list comprehension     text    The quick brown fox jumps over the lazy dog    replacements      brown    red      lazy    quick     text    text replace a  b  for a  b in replacements    text    The quick red fox jumps over the quick dog

User · Answer

Starting from the precious answer of Andrew i developed a script that loads the dictionary from a file and elaborates all the files on the opened folder to do the replacements  The script loads the mappings from an external file in which you can set the separator  I m a beginner but i found this script very useful when doing multiple substitutions in multiple files  It loaded a dictionary with more than 1000 entries in seconds  It is not elegant but it worked for me  import glob import re  mapfile   input  Enter map file name with extension eg  codifica txt     sep   input  Enter map file column separator eg        mask   input  Enter search mask with extension eg  2010 txt for all files to be processed     suff   input  Enter suffix with extension eg   NEW txt for newly generated files      rep        creation of empy dictionary  with open mapfile  as temprep    loading of definitions in the dictionary using input file  separator is prompted     for line in temprep           key  val    line strip   n   split sep          rep key    val  for filename in glob iglob mask     recursion on all the files with the mask prompted      with open  filename   r   as textfile    load each file in the variable text         text   textfile read              start replacement          rep   dict  re escape k   v  for k  v in rep items    commented to enable the use in the mapping of re reserved characters         pattern   re compile     join rep keys             text   pattern sub lambda m  rep m group 0    text            write of te output files with the prompted suffice         target   open filename  -4    NEW txt    w           target write text          target close

[python] How to replace multiple substrings of a string?

Examples related to python

Examples related to text

Examples related to replace