[linux] How to join multiple lines of file names into one with custom delimiter?

I would like to join the result of ls -1 into one line and delimit it with whatever i want.

Are there any standard Linux commands I can use to achieve this?

This question is related to linux bash shell parsing merge

The answer is


The combination of setting IFS and use of "$*" can do what you want. I'm using a subshell so I don't interfere with this shell's $IFS

(set -- *; IFS=,; echo "$*")

To capture the output,

output=$(set -- *; IFS=,; echo "$*")

Don't reinvent the wheel.

ls -m

It does exactly that.


You can use chomp to merge multiple line in single line:

perl -e 'while (<>) { if (/\$/ ) { chomp; } print ;}' bad0 >test

put line break condition in if statement.It can be special character or any delimiter.


Adding on top of majkinetor's answer, here is the way of removing trailing delimiter(since I cannot just comment under his answer yet):

ls -1 | awk 'ORS=","' | head -c -1

Just remove as many trailing bytes as your delimiter counts for.

I like this approach because I can use multi character delimiters + other benefits of awk:

ls -1 | awk 'ORS=", "' | head -c -2

EDIT

As Peter has noticed, negative byte count is not supported in native MacOS version of head. This however can be easily fixed.

First, install coreutils. "The GNU Core Utilities are the basic file, shell and text manipulation utilities of the GNU operating system."

brew install coreutils

Commands also provided by MacOS are installed with the prefix "g". For example gls.

Once you have done this you can use ghead which has negative byte count, or better, make alias:

alias head="ghead"

The sed way,

sed -e ':a; N; $!ba; s/\n/,/g'
  # :a         # label called 'a'
  # N          # append next line into Pattern Space (see info sed)
  # $!ba       # if it's the last line ($) do not (!) jump to (b) label :a (a) - break loop
  # s/\n/,/g   # any substitution you want

Note:

This is linear in complexity, substituting only once after all lines are appended into sed's Pattern Space.

@AnandRajaseka's answer, and some other similar answers, such as here, are O(n²), because sed has to do substitute every time a new line is appended into the Pattern Space.

To compare,

seq 1 100000 | sed ':a; N; $!ba; s/\n/,/g' | head -c 80
  # linear, in less than 0.1s
seq 1 100000 | sed ':a; /$/N; s/\n/,/; ta' | head -c 80
  # quadratic, hung

If you version of xargs supports the -d flag then this should work

ls  | xargs -d, -L 1 echo

-d is the delimiter flag

If you do not have -d, then you can try the following

ls | xargs -I {} echo {}, | xargs echo

The first xargs allows you to specify your delimiter which is a comma in this example.


sed -e :a -e '/$/N; s/\n/\\n/; ta' [filename]

Explanation:

-e - denotes a command to be executed
:a - is a label
/$/N - defines the scope of the match for the current and the (N)ext line
s/\n/\\n/; - replaces all EOL with \n
ta; - goto label a if the match is successful

Taken from my blog.


This command is for the PERL fans :

ls -1 | perl -l40pe0

Here 40 is the octal ascii code for space.

-p will process line by line and print

-l will take care of replacing the trailing \n with the ascii character we provide.

-e is to inform PERL we are doing command line execution.

0 means that there is actually no command to execute.

perl -e0 is same as perl -e ' '


You can use:

ls -1 | perl -pe 's/\n$/some_delimiter/'

ls produces one column output when connected to a pipe, so the -1 is redundant.

Here's another perl answer using the builtin join function which doesn't leave a trailing delimiter:

ls | perl -F'\n' -0777 -anE 'say join ",", @F'

The obscure -0777 makes perl read all the input before running the program.

sed alternative that doesn't leave a trailing delimiter

ls | sed '$!s/$/,/' | tr -d '\n'

This replaces the last comma with a newline:

ls -1 | tr '\n' ',' | sed 's/,$/\n/'

ls -m includes newlines at the screen-width character (80th for example).

Mostly Bash (only ls is external):

saveIFS=$IFS; IFS=$'\n'
files=($(ls -1))
IFS=,
list=${files[*]}
IFS=$saveIFS

Using readarray (aka mapfile) in Bash 4:

readarray -t files < <(ls -1)
saveIFS=$IFS
IFS=,
list=${files[*]}
IFS=$saveIFS

Thanks to gniourf_gniourf for the suggestions.


ls has the option -m to delimit the output with ", " a comma and a space.

ls -m | tr -d ' ' | tr ',' ';'

piping this result to tr to remove either the space or the comma will allow you to pipe the result again to tr to replace the delimiter.

in my example i replace the delimiter , with the delimiter ;

replace ; with whatever one character delimiter you prefer since tr only accounts for the first character in the strings you pass in as arguments.


Quick Perl version with trailing slash handling:

ls -1 | perl -E 'say join ", ", map {chomp; $_} <>'

To explain:

  • perl -E: execute Perl with features supports (say, ...)
  • say: print with a carrier return
  • join ", ", ARRAY_HERE: join an array with ", "
  • map {chomp; $_} ROWS: remove from each line the carrier return and return the result
  • <>: stdin, each line is a ROW, coupling with a map it will create an array of each ROW

If Python3 is your cup of tea, you can do this (but please explain why you would?):

ls -1 | python -c "import sys; print(','.join(sys.stdin.read().splitlines()))"

just bash

mystring=$(printf "%s|" *)
echo ${mystring%|}

Python answer above is interesting, but the own language can even make the output nice:

ls -1 | python -c "import sys; print(sys.stdin.read().splitlines())"


It looks like the answers already exist.

If you want a, b, c format, use ls -m ( Tulains Córdova’s answer)

Or if you want a b c format, use ls | xargs (simpified version of Chris J’s answer)

Or if you want any other delimiter like |, use ls | paste -sd'|' (application of Artem’s answer)


To avoid potential newline confusion for tr we could add the -b flag to ls:

ls -1b | tr '\n' ';'

I think this one is awesome

ls -1 | awk 'ORS=","'

ORS is the "output record separator" so now your lines will be joined with a comma.


EDIT: Simply "ls -m" If you want your delimiter to be a comma

Ah, the power and simplicity !

ls -1 | tr '\n' ','

Change the comma "," to whatever you want. Note that this includes a "trailing comma"


Parsing ls in general is not advised, so alternative better way is to use find, for example:

find . -type f -print0 | tr '\0' ','

Or by using find and paste:

find . -type f | paste -d, -s

For general joining multiple lines (not related to file system), check: Concise and portable “join” on the Unix command-line.


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to parsing

Got a NumberFormatException while trying to parse a text file for objects Uncaught SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) Python/Json:Expecting property name enclosed in double quotes Correctly Parsing JSON in Swift 3 How to get response as String using retrofit without using GSON or any other library in android UIButton action in table view cell "Expected BEGIN_OBJECT but was STRING at line 1 column 1" How to convert an XML file to nice pandas dataframe? How to extract multiple JSON objects from one file? How to sum digits of an integer in java?

Examples related to merge

Pandas Merging 101 Python: pandas merge multiple dataframes Git merge with force overwrite Merge two dataframes by index Visual Studio Code how to resolve merge conflicts with git? merge one local branch into another local branch Merging dataframes on index with pandas Git merge is not possible because I have unmerged files Git merge develop into feature branch outputs "Already up-to-date" while it's not How merge two objects array in angularjs?