Objective-C Reading a file line by line

Question

What is the appropriate way of dealing with large text files in Objective-C  Let s say I need to read each line separately and want to treat each line as an NSString  What is the most efficient way of doing this   One solution is using the NSString method      id stringWithContentsOfFile  NSString   path        encoding  NSStringEncoding enc        error  NSError    error    and then split the lines with a newline separator  and then iterate over the elements in the array  However  this seems fairly inefficient  Is there no easy way to treat the file as a stream  enumerating over each line  instead of just reading it all in at once  Kinda like Java s java io BufferedReader

User · Answer

Here s a nice simple solution i use for smaller files   NSString  path     NSBundle mainBundle  pathForResource   Terrain1  ofType   txt    NSString  contents    NSString stringWithContentsOfFile path encoding NSASCIIStringEncoding error nil   NSArray  lines    contents componentsSeparatedByCharactersInSet  NSCharacterSet characterSetWithCharactersInString    r n     for  NSString  line in lines        if  line length            NSLog   line       line

User · Answer

The appropriate way to read text files in Cocoa Objective-C is documented in Apple s String programming guide  The section for reading and writing files should be just what you re after  PS  What s a  line   Two sections of a string separated by   n   Or   r   Or   r n   Or maybe you re actually after paragraphs  The previously mentioned guide also includes a section on splitting a string into lines or paragraphs   This section is called  Paragraphs and Line Breaks   and is linked to in the left-hand-side menu of the page I pointed to above  Unfortunately this site doesn t allow me to post more than one URL as I m not a trustworthy user yet    To paraphrase Knuth  premature optimisation is the root of all evil  Don t simply assume that  reading the whole file into memory  is slow  Have you benchmarked it  Do you know that it actually reads the whole file into memory  Maybe it simply returns a proxy object and keeps reading behind the scenes as you consume the string   Disclaimer  I have no idea if NSString actually does this  It conceivably could   The point is  first go with the documented way of doing things  Then  if benchmarks show that this doesn t have the performance you desire  optimise

User · Answer

Just like  porneL said  the C api is very handy   NSString  fileRoot     NSBundle mainBundle  pathForResource   record  ofType   txt    FILE  file   fopen  fileRoot UTF8String    r    char buffer 256   while  fgets buffer  256  file     NULL       NSString  result    NSString stringWithUTF8String buffer       NSLog       result

User · Answer

I am adding this because all other answers I tried fell short one way or another  The following method can handle large files  arbitrary long lines  as well as empty lines  It has been tested with actual content and will strip out newline character from the output   -  NSString  readLineFromFile  FILE   file       char buffer 4096       NSMutableString  result    NSMutableString stringWithCapacity 1000        int charsRead      do           if fscanf file    4095   r n  n    n r    buffer   amp charsRead     1                 result appendFormat    s   buffer                     else               break                  while charsRead    4095        return result length   result   nil      Credit goes to  Adam Rosenfield and  sooop

User · Answer

This answer is NOT ObjC but C   Since ObjC is  C  based  why not use fgets   And yes  I m sure ObjC has it s own method - I m just not proficient enough yet to know what it is

User · Answer

That s a great question  I think  Diederik has a good answer  although it s unfortunate that Cocoa doesn t have a mechanism for exactly what you want to do   NSInputStream allows you to read chunks of N bytes  very similar to java io BufferedReader   but you have to convert it to an NSString on your own  then scan for newlines  or whatever other delimiter  and save any remaining characters for the next read  or read more characters if a newline hasn t been read yet   NSFileHandle lets you read an NSData which you can then convert to an NSString  but it s essentially the same process    Apple has a Stream Programming Guide that can help fill in the details  and this SO question may help as well if you re going to be dealing with uint8 t  buffers   If you re going to be reading strings like this frequently  especially in different parts of your program  it would be a good idea to encapsulate this behavior in a class that can handle the details for you  or even subclassing NSInputStream  it s designed to be subclassed  and adding methods that allow you to read exactly what you want   For the record  I think this would be a nice feature to add  and I ll be filing an enhancement request for something that makes this possible   -     Edit  Turns out this request already exists  There s a Radar dating from 2006 for this  rdar   4742914 for Apple-internal people

User · Answer

Use this script  it works great    NSString  path      Users xxx Desktop names txt   NSError  error  NSString  stringFromFileAtPath    NSString stringWithContentsOfFile  path                                                            encoding  NSUTF8StringEncoding                                                               error   amp error   if  stringFromFileAtPath    nil        NSLog   Error reading file at    n     path   error localizedFailureReason      NSLog   Contents      stringFromFileAtPath

User · Answer

from  Adam Rosenfield s answer   the formatting string of fscanf would be changed like below     4095   r n  n    n r     it will work in osx  linux  windows line endings

User · Answer

This should do the trick    include  lt stdio h gt   NSString  readLineAsNSString FILE  file        char buffer 4096           tune this capacity to your liking -- larger buffer sizes will be faster  but        use more memory     NSMutableString  result    NSMutableString stringWithCapacity 256           Read up to 4095 non-newline characters  then read and discard the newline     int charsRead      do               if fscanf file    4095   n  n  c   buffer   amp charsRead     1               result appendFormat    s   buffer           else             break        while charsRead    4095        return result      Use as follows   FILE  file   fopen  myfile    r       check for NULL while  feof file         NSString  line   readLineAsNSString file          do stuff with line  line is autoreleased  so you should NOT release it  unless you also retain it beforehand    fclose file     This code reads non-newline characters from the file  up to 4095 at a time   If you have a line that is longer than 4095 characters  it keeps reading until it hits a newline or end-of-file   Note  I have not tested this code   Please test it before using it

User · Answer

I see a lot of these answers rely on reading the whole text file into memory instead of taking it one chunk at a time  Here s my solution in nice modern Swift  using FileHandle to keep memory impact low   enum MyError       case invalidTextFormat    extension FileHandle        func readLine maxLength  Int  throws - gt  String               Read in a string of up to the maximum length         let offset   offsetInFile         let data   readData ofLength  maxLength          guard let string   String data  data  encoding   utf8  else               throw MyError invalidTextFormat                       Check for carriage returns  if none  this is the whole string         let substring  String         if let subindex   string firstIndex of    n                 substring   String string string startIndex     subindex             else               substring   string                       Wind back to the correct offset so that we don t miss any lines         guard let dataCount   substring data using   utf8  allowLossyConversion  false   count else               throw MyError invalidTextFormat                   try seek toOffset  offset   UInt64 dataCount           return substring            Note that this preserves the carriage return at the end of the line  so depending on your needs you may want to adjust the code to remove it   Usage  simply open a file handle to your target text file and call readLine with a suitable maximum length - 1024 is standard for plain text  but I left it open in case you know it will be shorter  Note that the command will not overflow the end of the file  so you may have to check manually that you ve not reached it if you intend to parse the entire thing  Here s some sample code that shows how to open a file at myFileURL and read it line-by-line until the end   do       let handle   try FileHandle forReadingFrom  myFileURL      try handle seekToEndOfFile       let eof   handle offsetInFile     try handle seek toFileOffset  0       while handle offsetInFile  lt  eof           let line   try handle readLine maxLength  1024             Do something with the string here           try handle close   catch let error       print  Error reading file    error localizedDescription

User · Answer

As others have answered both NSInputStream and NSFileHandle are fine options  but it can also be done in a fairly compact way with NSData and memory mapping   BRLineReader h   import  lt Foundation Foundation h gt    interface BRLineReader   NSObject   property  readonly  nonatomic  NSData  data   property  readonly  nonatomic  NSUInteger linesRead   property  strong  nonatomic  NSCharacterSet  lineTrimCharacters   property  readonly  nonatomic  NSStringEncoding stringEncoding   -  instancetype initWithFile  NSString   filePath encoding  NSStringEncoding encoding  -  instancetype initWithData  NSData   data encoding  NSStringEncoding encoding  -  NSString   readLine  -  NSString   readTrimmedLine  -  void setLineSearchPosition  NSUInteger position    end   BRLineReader m   import  BRLineReader h   static unsigned char const BRLineReaderDelimiter     n     implementation BRLineReader       NSRange  lastRange     -  instancetype initWithFile  NSString   filePath encoding  NSStringEncoding encoding       self    super init       if  self            NSError  error   nil           data    NSData dataWithContentsOfFile filePath options NSDataReadingMappedAlways error  amp error           if    data                NSLog         error localizedDescription                       stringEncoding   encoding           lineTrimCharacters    NSCharacterSet whitespaceAndNewlineCharacterSet              return self     -  instancetype initWithData  NSData   data encoding  NSStringEncoding encoding       self    super init       if  self             data   data           stringEncoding   encoding           lineTrimCharacters    NSCharacterSet whitespaceAndNewlineCharacterSet              return self     -  NSString   readLine       NSUInteger dataLength     data length       NSUInteger beginPos    lastRange location    lastRange length      NSUInteger endPos   0      if  beginPos    dataLength               End of file         return nil             unsigned char  buffer    unsigned char     data bytes       for  NSUInteger i   beginPos  i  lt  dataLength  i              endPos   i          if  buffer i     BRLineReaderDelimiter  break                End of line found      lastRange   NSMakeRange beginPos  endPos - beginPos   1       NSData  lineData     data subdataWithRange  lastRange       NSString  line     NSString alloc  initWithData lineData encoding  stringEncoding        linesRead         return line     -  NSString   readTrimmedLine       return   self readLine  stringByTrimmingCharactersInSet  lineTrimCharacters      -  void setLineSearchPosition  NSUInteger position        lastRange   NSMakeRange position  0        linesRead   0      end

User · Answer

I found response by  lukaswelte and code from Dave DeLong very helpful  I was looking for a solution to this problem but needed to parse large files by  r n not just  n    The code as written contains a bug if parsing by more than one character  I ve changed the code as below    h file    import  lt Foundation Foundation h gt    interface FileChunkReader   NSObject       NSString   filePath       NSFileHandle   fileHandle      unsigned long long currentOffset      unsigned long long totalFileLength       NSString   lineDelimiter      NSUInteger chunkSize      property  nonatomic  copy  NSString   lineDelimiter   property  nonatomic  NSUInteger chunkSize   -  id  initWithFilePath  NSString   aPath   -  NSString    readLine  -  NSString    readTrimmedLine    if NS BLOCKS AVAILABLE -  void  enumerateLinesUsingBlock  void    NSString   BOOL    block   endif   end    m file    import  FileChunkReader h    interface NSData  DDAdditions   -  NSRange  rangeOfData dd  NSData   dataToFind    end   implementation NSData  DDAdditions   -  NSRange  rangeOfData dd  NSData   dataToFind        const void   bytes    self bytes       NSUInteger length    self length        const void   searchBytes    dataToFind bytes       NSUInteger searchLength    dataToFind length       NSUInteger searchIndex   0       NSRange foundRange    NSNotFound  searchLength       for  NSUInteger index   0  index  lt  length  index              if    char   bytes  index       char   searchBytes  searchIndex                   the current character matches             if  foundRange location    NSNotFound                    foundRange location   index                            searchIndex                if  searchIndex  gt   searchLength                                return foundRange                          else               searchIndex   0              foundRange location   NSNotFound                       if  foundRange location    NSNotFound          amp  amp  length  lt  foundRange location   foundRange length                    if the dataToFind is partially found at the end of  self bytes              then the loop above would end  and indicate the dataToFind is found            when it only partially was          foundRange location   NSNotFound             return foundRange      end   implementation FileChunkReader   synthesize lineDelimiter  chunkSize   -  id  initWithFilePath  NSString   aPath       if  self    super init             fileHandle    NSFileHandle fileHandleForReadingAtPath aPath           if  fileHandle    nil                return nil                     lineDelimiter      n           currentOffset   0ULL                 chunkSize   128           fileHandle seekToEndOfFile           totalFileLength    fileHandle offsetInFile             we don t need to seek back  since readLine will do that            return self     -  void  dealloc        fileHandle closeFile       currentOffset   0ULL      -  NSString    readLine       if  currentOffset  gt   totalFileLength                return nil              autoreleasepool            NSData   newLineData    lineDelimiter dataUsingEncoding NSUTF8StringEncoding            fileHandle seekToFileOffset currentOffset           unsigned long long originalOffset   currentOffset          NSMutableData  currentData     NSMutableData alloc  init           NSData  currentLine     NSData alloc  init           BOOL shouldReadMore   YES            while  shouldReadMore                if  currentOffset  gt   totalFileLength                                break                             NSData   chunk    fileHandle readDataOfLength chunkSize                currentData appendData chunk                NSRange newLineRange    currentData rangeOfData dd newLineData                if  newLineRange location    NSNotFound                     currentOffset   originalOffset   newLineRange location   newLineData length                  currentLine    currentData subdataWithRange NSMakeRange 0  newLineRange location                     shouldReadMore   NO               else                  currentOffset     chunk length                                    if  currentLine length    0  amp  amp  currentData length  gt  0                        currentLine   currentData                     return   NSString alloc  initWithData currentLine encoding NSUTF8StringEncoding            -  NSString    readTrimmedLine       return   self readLine  stringByTrimmingCharactersInSet  NSCharacterSet whitespaceAndNewlineCharacterSet        if NS BLOCKS AVAILABLE -  void  enumerateLinesUsingBlock  void    NSString   BOOL   block       NSString   line   nil      BOOL stop   NO      while  stop    NO  amp  amp   line    self readLine              block line   amp stop            endif   end

User · Answer

A lot of these answers are long chunks of code or they read in the entire file  I like to use the c methods for this very task   FILE  file   fopen  path to my file    r     size t length  char  cLine   fgetln file  amp length    while  length gt 0        char str length 1       strncpy str  cLine  length       str length      0        NSString  line    NSString stringWithFormat    s  str                 Do what you want here       cLine   fgetln file  amp length       Note that fgetln will not keep your newline character  Also  We  1 the length of the str because we want to make space for the NULL termination

User · Answer

To read a file line by line  also for extreme big files  can be done by the following functions   DDFileReader   reader     DDFileReader alloc  initWithFilePath pathToMyFile   NSString   line   nil  while   line    reader readLine        NSLog   read line       line      reader release     Or   DDFileReader   reader     DDFileReader alloc  initWithFilePath pathToMyFile    reader enumerateLinesUsingBlock   NSString   line  BOOL   stop      NSLog   read line       line        reader release     The class DDFileReader that enables this is the following   Interface File   h     interface DDFileReader   NSObject       NSString   filePath       NSFileHandle   fileHandle      unsigned long long currentOffset      unsigned long long totalFileLength       NSString   lineDelimiter      NSUInteger chunkSize      property  nonatomic  copy  NSString   lineDelimiter   property  nonatomic  NSUInteger chunkSize   -  id  initWithFilePath  NSString   aPath   -  NSString    readLine  -  NSString    readTrimmedLine    if NS BLOCKS AVAILABLE -  void  enumerateLinesUsingBlock  void    NSString   BOOL    block   endif   end   Implementation   m    import  DDFileReader h    interface NSData  DDAdditions   -  NSRange  rangeOfData dd  NSData   dataToFind    end   implementation NSData  DDAdditions   -  NSRange  rangeOfData dd  NSData   dataToFind        const void   bytes    self bytes       NSUInteger length    self length        const void   searchBytes    dataToFind bytes       NSUInteger searchLength    dataToFind length       NSUInteger searchIndex   0       NSRange foundRange    NSNotFound  searchLength       for  NSUInteger index   0  index  lt  length  index              if    char   bytes  index       char   searchBytes  searchIndex                   the current character matches             if  foundRange location    NSNotFound                    foundRange location   index                            searchIndex                if  searchIndex  gt   searchLength    return foundRange              else               searchIndex   0              foundRange location   NSNotFound                      return foundRange      end   implementation DDFileReader  synthesize lineDelimiter  chunkSize   -  id  initWithFilePath  NSString   aPath       if  self    super init             fileHandle    NSFileHandle fileHandleForReadingAtPath aPath           if  fileHandle    nil                 self release   return nil                     lineDelimiter     NSString alloc  initWithString    n             fileHandle retain           filePath    aPath retain           currentOffset   0ULL          chunkSize   10           fileHandle seekToEndOfFile           totalFileLength    fileHandle offsetInFile             we don t need to seek back  since readLine will do that            return self     -  void  dealloc        fileHandle closeFile        fileHandle release   fileHandle   nil       filePath release   filePath   nil       lineDelimiter release   lineDelimiter   nil      currentOffset   0ULL       super dealloc      -  NSString    readLine       if  currentOffset  gt   totalFileLength    return nil         NSData   newLineData    lineDelimiter dataUsingEncoding NSUTF8StringEncoding        fileHandle seekToFileOffset currentOffset       NSMutableData   currentData     NSMutableData alloc  init       BOOL shouldReadMore   YES       NSAutoreleasePool   readPool     NSAutoreleasePool alloc  init       while  shouldReadMore            if  currentOffset  gt   totalFileLength    break            NSData   chunk    fileHandle readDataOfLength chunkSize           NSRange newLineRange    chunk rangeOfData dd newLineData           if  newLineRange location    NSNotFound                   include the length so we can include the delimiter in the string             chunk    chunk subdataWithRange NSMakeRange 0  newLineRange location  newLineData length                 shouldReadMore   NO                     currentData appendData chunk           currentOffset     chunk length              readPool release        NSString   line     NSString alloc  initWithData currentData encoding NSUTF8StringEncoding        currentData release       return  line autorelease      -  NSString    readTrimmedLine       return   self readLine  stringByTrimmingCharactersInSet  NSCharacterSet whitespaceAndNewlineCharacterSet        if NS BLOCKS AVAILABLE -  void  enumerateLinesUsingBlock  void    NSString   BOOL   block     NSString   line   nil    BOOL stop   NO    while  stop    NO  amp  amp   line    self readLine          block line   amp stop          endif   end   The class was done by Dave DeLong

User · Answer

Using category or extension to make our life a bit easier   extension String        func lines   - gt   String            var lines    String            self enumerateLines    line  stop  - gt     in             lines append line                    return lines              then for line in string lines            do the right thing

User · Answer

Mac OS X is Unix  Objective-C is C superset  so you can just use old-school fopen and fgets from  lt stdio h gt   It s guaranteed to work    NSString stringWithUTF8String buf  will convert C string to NSString  There are also methods for creating strings in other encodings and creating without copying

User · Answer

This will work for general reading a String from Text  If you would like to read longer text  large size of text   then use the method that other people here were mentioned such as buffered  reserve the size of the text in memory space    Say you read a Text File   NSString  filePath        file path    NSString  fileRoot     NSBundle mainBundle                  pathForResource filePath ofType   txt      You want to get rid of new line      read everything from text NSString  fileContents           NSString stringWithContentsOfFile fileRoot         encoding NSUTF8StringEncoding error nil       first  separate by new line NSArray  allLinedStrings           fileContents componentsSeparatedByCharactersInSet         NSCharacterSet newlineCharacterSet        then break down even further  NSString  strsInOneLine           allLinedStrings objectAtIndex 0       choose whatever input identity you have decided  in this case   NSArray  singleStrs           currentPointString componentsSeparatedByCharactersInSet         NSCharacterSet characterSetWithCharactersInString           There you have it

User · Answer

You can use NSInputStream which has a basic implementation for file streams  You can read bytes into a buffer  read maxLength  method   You have to scan the buffer for newlines yourself

[objective-c] Objective-C: Reading a file line by line

Examples related to objective-c

Examples related to file-io

Examples related to nsstring

Examples related to enumerate

Examples related to nsstream