Insert line at middle of file with Python

Question

Is there a way to do this  Say I have a file that s a list of names that goes like this    Alfred Bill Donald   How could I insert the third name   Charlie   at line x  in this case 3   and automatically send all others down one line  I ve seen other questions like this  but they didn t get helpful answers  Can it be done  preferably with either a method or a loop

User · Answer

The accepted answer has to load the whole file into memory, which doesn't work nicely for large files. The following solution writes the file contents with the new data inserted into the right line to a temporary file in the same directory (so on the same file system), only reading small chunks from the source file at a time. It then overwrites the source file with the contents of the temporary file in an efficient way (Python 3.8+).

from pathlib import Path
from shutil import copyfile
from tempfile import NamedTemporaryFile

sourcefile = Path("/path/to/source").resolve()
insert_lineno = 152  # The line to insert the new data into.
insert_data = "..."  # Some string to insert.

with sourcefile.open(mode="r") as source:
    destination = NamedTemporaryFile(mode="w", dir=str(sourcefile.parent))
    lineno = 1

    while lineno < insert_lineno:
        destination.file.write(source.readline())
        lineno += 1

    # Insert the new data.
    destination.file.write(insert_data)

    # Write the rest in chunks.
    while True:
        data = source.read(1024)
        if not data:
            break
        destination.file.write(data)

# Finish writing data.
destination.flush()
# Overwrite the original file's contents with that of the temporary file.
# This uses a memory-optimised copy operation starting from Python 3.8.
copyfile(destination.name, str(sourcefile))
# Delete the temporary file.
destination.close()

EDIT 2020-09-08: I just found an answer on Code Review that does something similar to above with more explanation - it might be useful to some.

User · Answer

There is a combination of techniques which I found useful in solving this issue   with open file   r    as fd      contents   fd readlines       contents insert index  new string     new string should end in a newline     fd seek 0     readlines consumes the iterator  so we need to start over     fd writelines contents     No need to truncate as we are increasing filesize   In our particular application  we wanted to add it after a certain string   with open file   r    as fd      contents   fd readlines       if match string in contents -1      Handle last line to prevent IndexError         contents append insert string      else          for index  line in enumerate contents               if match string in line and insert string not in contents index   1                   contents insert index   1  insert string                  break     fd seek 0      fd writelines contents    If you want it to insert the string after every instance of the match  instead of just the first  remove the else   and properly unindent  and the break    Note also that the and insert string not in contents index   1   prevents it from adding more than one copy after the match string  so it s safe to run repeatedly

User · Answer

You can just read the data into a list and insert the new record where you want   names      with open  names txt    r    as fd      for line in fd          names append line split      -1  strip         names insert 2   Charlie     element 2 will be 3  in your list     fd seek 0      fd truncate        for i in xrange len names            fd write   d   s n    i   1  names i

User · Answer

If you want to search a file for a substring and add a new text to the next line  one of the elegant ways to do it is the following   import fileinput for line in fileinput FileInput file path inplace 1       if  TEXT TO SEARCH  in line          line line replace line line  NEW TEXT       print line

User · Answer

location of line   0 with open filename   r   as file you want to read        readlines in file and put in a list      contents   file you want to read readlines          find location of what line you want to insert after      for index  line in enumerate contents               if line startswith  whatever you are looking for                      location of line   index   now you have a list of every line in that file context insert location of line   whatever you want to append to middle of file   with open filename   w   as file to write to          file to write to writelines contents    That is how I ended up getting whatever data I want to insert to the middle of the file   this is just pseudo code   as I was having a hard time finding clear understanding of what is going on      essentially you read in the file to its entirety and add it into a list     then you insert your lines that you want to that list  and then re-write to the same file     i am sure there are better ways to do this  may not be efficient  but it makes more sense to me at least  I hope it makes sense to someone else

User · Answer

Below is a slightly awkward solution for the special case in which you are creating the original file yourself and happen to know the insertion location  e g  you know ahead of time that you will need to insert a line with an additional name before the third line  but won t know the name until after you ve fetched and written the rest of the names   Reading  storing and then re-writing the entire contents of the file as described in other answers is  I think  more elegant than this option  but may be undesirable for large files   You can leave a buffer of invisible null characters    0   at the insertion location to be overwritten later   num names   1 000 000      Enough data to make storing in a list unideal max len   20               The maximum allowed length of the inserted line line to insert   2         The third line is at index 2  0-based indexing   with open filename   w    as file      for i in range line to insert           name   get name i                       Returns  Alfred  for i   0  etc          file write F  i   1    name  n        insert position   file tell                 Position to jump back to for insertion     file write   0    max len     n             Buffer will show up as a blank line      for i in range line to insert  num names           name   get name i          file write F  i   2    name  n          Line numbering now bumped up by 1     Later  once you have the name to insert    with open filename   r    as file               Must use  r   to write to middle of file      file seek insert position                   Move stream to the insertion line     name   get bonus name                       This lucky winner jumps up to 3rd place     new line   F  line to insert   1    name       file write new line  max len                Slice so you don t overwrite next line   Unfortunately there is no way to delete-without-replacement any excess null characters that did not get overwritten  or in general any characters anywhere in the middle of a file   unless you then re-write everything that follows  But the null characters will not affect how your file looks to a human  they have zero width

User · Answer

This is a way of doing the trick   f   open  path to file    r   contents   f readlines   f close    contents insert index  value   f   open  path to file    w   contents      join contents  f write contents  f close      index  and  value  are the line and value of your choice  lines starting from 0

User · Answer

You don t show us what the output should look like  so one possible interpretation is that you want this as the output    Alfred Bill Charlie Donald    Insert Charlie  then add 1 to all subsequent lines   Here s one possible solution   def insert line input stream  pos  new name  output stream     inserted   False   for line in input stream      number  name   parse line line      if number    pos        print  gt  gt  output stream  format line number  new name        inserted   True     print  gt  gt  output stream  format line number if not inserted else  number   1   name   def parse line line     number str  name   line strip   split     return  get number number str   name   def get number number str     return int number str split      0    def format line number  name     return add dot number          name  def add dot number     return str number         input stream   open  input txt    r   output stream   open  output txt    w    insert line input stream  3   Charlie   output stream   input stream close   output stream close

User · Answer

A simple but not efficient way is to read the whole content  change it and then rewrite it   line index   3 lines   None with open  file txt    r   as file handler      lines   file handler readlines    lines insert line index   Charlie    with open  file txt    w   as file handler      file handler writelines lines

User · Answer

Parse the file into a python list using file readlines   or file read   split   n   Identify the position where you have to insert a new line  according to your criteria  Insert a new list element there using list insert    Write the result to the file

[python] Insert line at middle of file with Python?

Examples related to python