How to obtain the total numbers of rows from a CSV file in Python

Question

I m using python  Django Framework  to read a CSV file  I pull just 2 lines out of this CSV as you can see  What I have been trying to do is store in a variable the total number of rows the CSV also   How can I get the total number of rows   file   object myfilePath fileObject   csv reader file  for i in range 2       data append fileObject next       I have tried   len fileObject  fileObject length

User · Answer

If you are working on a Unix system  the fastest method is the following shell command cat FILE NAME CSV   wc -l  From Jupyter Notebook or iPython  you can use it with a      cat FILE NAME CSV   wc -l

User · Answer

try  data   pd read csv  data csv   data shape   and in the output you can see something like  aa bb  where aa is the   of rows

User · Answer

numline   len file read readlines

User · Answer

After iterating the whole file with csv reader   method  you have the total number of lines read  via instance variable  line num  import csv with open  csv path file   as f      csv reader   csv reader f      for row in csv reader          pass     print csv reader line num   Quoting the official documentation   csvreader line num The number of lines read from the source iterator   Small caveat   total number of lines  includes the header  if the CSV has

User · Answer

Use  list  to fit a more workably object   You can then count  skip  mutate till your heart s desire   list fileObject   list values  len list fileObject     get length of file lines  list fileObject  10     skip first 10 lines

User · Answer

might want to try something as simple as below in the command line  sed -n      filename  or wc -l filename

User · Answer

Several of the above suggestions count the number of LINES in the csv file   But some CSV files will contain quoted strings which themselves contain newline characters   MS CSV files usually delimit records with  r n  but use  n alone within quoted strings   For a file like this  counting lines of text  as delimited by newline  in the file will give too large a result   So for an accurate count you need to use csv reader to read the records

User · Answer

row count   sum 1 for line in open filename   worked for me    Note   sum 1 for line in csv reader filename   seems to calculate the length of first line

User · Answer

To do it you need to have a bit of code like my example here   file   open  Task1 csv   numline   len file readlines    print  numline    I hope this helps everyone

User · Answer

import pandas as pd data   pd read csv  data csv    totalInstances len data

User · Answer

You need to count the number of rows   row count   sum 1 for row in fileObject     fileObject is your csv reader   Using sum   with a generator expression makes for an efficient counter  avoiding storing the whole file in memory   If you already read 2 rows to start with  then you need to add those 2 rows to your total  rows that have already been read are not being counted

User · Answer

First you have to open the file with open  input file   open  nameOfFile csv   r      Then use the csv reader for open the csv  reader file   csv reader input file    At the last  you can take the number of row with the instruction  len   value   len list reader file     The total code is this   input file   open  nameOfFile csv   r    reader file   csv reader input file  value   len list reader file     Remember that if you want to reuse the csv file  you have to make a input file fseek 0   because when you use a list for the reader file  it reads all file  and the pointer in the file change its position

User · Answer

This works for csv and all files containing strings in Unix-based OSes   import os  numOfLines   int os popen  wc -l  lt  file csv   read    -1     In case the csv file contains a fields row you can deduct one from numOfLines above   numOfLines   numOfLines - 1

User · Answer

2018-10-29 EDIT Thank you for the comments  I tested several kinds of code to get the number of lines in a csv file in terms of speed  The best method is below  with open filename  as f      sum 1 for line in f   Here is the code tested  import timeit import csv import pandas as pd  filename      sample submission csv   def talktime filename  funcname  func       print f quot    funcname  quot       t   timeit timeit f  funcname   quot  filename  quot     setup f from   main   import  funcname    number   100    100     print  Elapsed time      t      print  n      func filename       print   n    def sum1forline filename       with open filename  as f          return sum 1 for line in f  talktime filename   sum1forline   sum1forline   def lenopenreadlines filename       with open filename  as f          return len f readlines    talktime filename   lenopenreadlines   lenopenreadlines   def lenpd filename       return len pd read csv filename     1 talktime filename   lenpd   lenpd   def csvreaderfor filename       cnt   0     with open filename  as f          cr   csv reader f          for row in cr              cnt    1     return cnt talktime filename   csvreaderfor   csvreaderfor   def openenum filename       cnt   0     with open filename  as f          for i  line in enumerate f 1               cnt    1     return cnt talktime filename   openenum   openenum   The result was below    sum1forline Elapsed time    0 6327946722068599 n    2528244     lenopenreadlines Elapsed time    0 655304473598555 n    2528244     lenpd Elapsed time    0 7561274056295324 n    2528244     csvreaderfor Elapsed time    1 5571560935772661 n    2528244     openenum Elapsed time    0 773000013928679 n    2528244  In conclusion  sum 1 for line in f  is fastest  But there might not be significant difference from len f readlines     sample submission csv is 30 2MB and has 31 million characters

User · Answer

You can also use a classic for loop   import pandas as pd df   pd read csv  your file csv    count   0 for i in df  a column        count   count   1  print count

User · Answer

import csv count   0 with open  filename csv    rb   as count file      csv reader   csv reader count file      for row in csv reader          count    1  print count

User · Answer

I think we can improve the best answer a little bit  I m using   len   sum 1 for   in reader    Moreover  we shouldnt forget pythonic code not always have the best performance in the project  In example  If we can do more operations at the same time in the same data set Its better to do all in the same bucle instead make two or more pythonic bucles

[python] How to obtain the total numbers of rows from a CSV file in Python?

Examples related to python

Examples related to csv

Examples related to count