If splitting very large files, the solution I found is an adaptation from this, with PowerShell "embedded" in a batch file. This works fast, as opposed to many other things I tried (I wouldn't know about other options posted here).
The way to use mysplit.bat
below is
mysplit.bat <mysize> 'myfile'
Note: The script was intended to use the first argument as the split size. It is currently hardcoded at 100Mb. It should not be difficult to fix this.
Note 2: The filname should be enclosed in single quotes. Other alternatives for quoting apparently do not work.
Note 3: It splits the file at given number of bytes, not at given number of lines. For me this was good enough. Some lines of code could be probably added to complete each chunk read, up to the next CR/LF. This will split in full lines (not with a constant number of them), with no sacrifice in processing time.
Script mysplit.bat
:
@REM Using https://stackoverflow.com/questions/19335004/how-to-run-a-powershell-script-from-a-batch-file
@REM and https://stackoverflow.com/questions/1001776/how-can-i-split-a-text-file-using-powershell
@PowerShell ^
$upperBound = 100MB; ^
$rootName = %2; ^
$from = $rootName; ^
$fromFile = [io.file]::OpenRead($from); ^
$buff = new-object byte[] $upperBound; ^
$count = $idx = 0; ^
try { ^
do { ^
'Reading ' + $upperBound; ^
$count = $fromFile.Read($buff, 0, $buff.Length); ^
if ($count -gt 0) { ^
$to = '{0}.{1}' -f ($rootName, $idx); ^
$toFile = [io.file]::OpenWrite($to); ^
try { ^
'Writing ' + $count + ' to ' + $to; ^
$tofile.Write($buff, 0, $count); ^
} finally { ^
$tofile.Close(); ^
} ^
} ^
$idx ++; ^
} while ($count -gt 0); ^
} ^
finally { ^
$fromFile.Close(); ^
} ^
%End PowerShell%