How to split CSV files as per number of rows specified

Question

I ve CSV file  around 10 000 rows   each row having 300 columns  stored on LINUX server   I want to break this CSV file into 500 CSV files of 20 records each   Each having same CSV header as present in original CSV    Is there any linux command to help this conversion

User · Answer

This should work      file name   Name of the file you want to split   10000   Number of rows each split file would contain  file part    Prefix of split file name  file part 0 file part 1 file part 2  etc goes on      split -d -l 10000 file name csv file part

User · Answer

Use the Linux split command   split -l 20 file txt new       Split the file  file txt  into files beginning with the name  new  each containing 20 lines of text each   Type man split at the Unix prompt for more information  However you will have to first remove the header from file txt  using the tail command  for example  and then add it back on to each of the split files

User · Answer

Made it into a function  You can now call splitCsv  lt Filename gt   chunkSize  splitCsv         HEADER   head -1  1      if   -n  quot  2 quot     then         CHUNK  2     else          CHUNK 1000     fi     tail -n  2  1   split -l  CHUNK -  1 split      for i in  1 split    do         sed -i -e  quot 1i HEADER quot   quot  i quot      done    Found on  http   edmondscommerce github io linux linux-split-file-eg-csv-and-keep-header-row html

User · Answer

I have a one-liner answer  this example gives you 999 lines of data  and  one header row per file  cat bigFile csv   parallel --header   --pipe -N999  cat  gt file     csv   https   stackoverflow com a 53062251 401226

User · Answer

This should do it for you - all your files will end up called Part1-Part500      bin bash FILENAME 10000 csv HDR   head -1  FILENAME      Pick up CSV header line to apply to each file split -l 20  FILENAME xyz    Split the file into chunks of 20 lines each n 1 for f in xyz                 Go through all newly created chunks do    echo  HDR  gt  Part  n       Write out header to new file called  Part n      cat  f  gt  gt  Part  n         Add in the 20 lines from the  split  command    rm  f                     Remove temporary file      n                       Increment name of output part done

[linux] How to split CSV files as per number of rows specified?

Examples related to linux

Examples related to unix

Examples related to csv

Examples related to split