[bash] How to perform a for loop on each character in a string in Bash?

I have a variable like this:

words="??????"

I want to make a for loop on each of the characters, one at a time, e.g. first character="?", then character="?", character="?", etc.

The only way I know is to output each character to separate line in a file, then use while read line, but this seems very inefficient.

• How can I process each character in a string through a for loop?

I'm surprised no one has mentioned the obvious bash solution utilizing only while and read.

while read -n1 character; do
    echo "$character"
done < <(echo -n "$words")

Note the use of echo -n to avoid the extraneous newline at the end. printf is another good option and may be more suitable for your particular needs. If you want to ignore whitespace then replace "$words" with "${words// /}".

Another option is fold. Please note however that it should never be fed into a for loop. Rather, use a while loop as follows:

while read char; do
    echo "$char"
done < <(fold -w1 <<<"$words")

The primary benefit to using the external fold command (of the coreutils package) would be brevity. You can feed it's output to another command such as xargs (part of the findutils package) as follows:

fold -w1 <<<"$words" | xargs -I% -- echo %

You'll want to replace the echo command used in the example above with the command you'd like to run against each character. Note that xargs will discard whitespace by default. You can use -d '\n' to disable that behavior.

I just tested fold with some of the Asian characters and realized it doesn't have Unicode support. So while it is fine for ASCII needs, it won't work for everyone. In that case there are some alternatives.

I'd probably replace fold -w1 with an awk array:

awk 'BEGIN{FS=""} {for (i=1;i<=NF;i++) print $i}'

Or the grep command mentioned in another answer:

grep -o .

FYI, I benchmarked the 3 aforementioned options. The first two were fast, nearly tying, with the fold loop slightly faster than the while loop. Unsurprisingly xargs was the slowest... 75x slower.

Here is the (abbreviated) test code:

words=$(python -c 'from string import ascii_letters as l; print(l * 100)')

testrunner(){
    for test in test_while_loop test_fold_loop test_fold_xargs test_awk_loop test_grep_loop; do
        echo "$test"
        (time for (( i=1; i<$((${1:-100} + 1)); i++ )); do "$test"; done >/dev/null) 2>&1 | sed '/^$/d'
        echo
    done
}

testrunner 100

Here are the results:

test_while_loop
real    0m5.821s
user    0m5.322s
sys     0m0.526s

test_fold_loop
real    0m6.051s
user    0m5.260s
sys     0m0.822s

test_fold_xargs
real    7m13.444s
user    0m24.531s
sys     6m44.704s

test_awk_loop
real    0m6.507s
user    0m5.858s
sys     0m0.788s

test_grep_loop
real    0m6.179s
user    0m5.409s
sys     0m0.921s

Another way is:

Characters="TESTING"
index=1
while [ $index -le ${#Characters} ]
do
    echo ${Characters} | cut -c${index}-${index}
    index=$(expr $index + 1)
done

It is also possible to split the string into a character array using fold and then iterate over this array:

for char in `echo "??????" | fold -w1`; do
    echo $char
done

Another approach, if you don't care about whitespace being ignored:

for char in $(sed -E s/'(.)'/'\1 '/g <<<"$your_string"); do
    # Handle $char here
done

You can use a C-style for loop:

foo=string
for (( i=0; i<${#foo}; i++ )); do
  echo "${foo:$i:1}"
done

${#foo} expands to the length of foo. ${foo:$i:1} expands to the substring starting at position $i of length 1.

The C style loop in @chepner's answer is in the shell function update_terminal_cwd, and the grep -o . solution is clever, but I was surprised not to see a solution using seq. Here's mine:

read word
for i in $(seq 1 ${#word}); do
  echo "${word:i-1:1}"
done

I believe there is still no ideal solution that would correctly preserve all whitespace characters and is fast enough, so I'll post my answer. Using ${foo:$i:1} works, but is very slow, which is especially noticeable with large strings, as I will show below.

My idea is an expansion of a method proposed by Six, which involves read -n1, with some changes to keep all characters and work correctly for any string:

while IFS='' read -r -d '' -n 1 char; do
        # do something with $char
done < <(printf %s "$string")

How it works:

• IFS='' - Redefining internal field separator to empty string prevents stripping of spaces and tabs. Doing it on a same line as read means that it will not affect other shell commands.
• -r - Means "raw", which prevents read from treating \ at the end of the line as a special line concatenation character.
• -d '' - Passing empty string as a delimiter prevents read from stripping newline characters. Actually means that null byte is used as a delimiter. -d '' is equal to -d $'\0'.
• -n 1 - Means that one character at a time will be read.
• printf %s "$string" - Using printf instead of echo -n is safer, because echo treats -n and -e as options. If you pass "-e" as a string, echo will not print anything.
• < <(...) - Passing string to the loop using process substitution. If you use here-strings instead (done <<< "$string"), an extra newline character is appended at the end. Also, passing string through a pipe (printf %s "$string" | while ...) would make the loop run in a subshell, which means all variable operations are local within the loop.

Now, let's test the performance with a huge string. I used the following file as a source:
https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt
The following script was called through time command:

#!/bin/bash

# Saving contents of the file into a variable named `string'.
# This is for test purposes only. In real code, you should use
# `done < "filename"' construct if you wish to read from a file.
# Using `string="$(cat makefiles.txt)"' would strip trailing newlines.
IFS='' read -r -d '' string < makefiles.txt

while IFS='' read -r -d '' -n 1 char; do
        # remake the string by adding one character at a time
        new_string+="$char"
done < <(printf %s "$string")

# confirm that new string is identical to the original
diff -u makefiles.txt <(printf %s "$new_string")

And the result is:

$ time ./test.sh

real    0m1.161s
user    0m1.036s
sys     0m0.116s

As we can see, it is quite fast.
Next, I replaced the loop with one that uses parameter expansion:

for (( i=0 ; i<${#string}; i++ )); do
    new_string+="${string:$i:1}"
done

The output shows exactly how bad the performance loss is:

$ time ./test.sh

real    2m38.540s
user    2m34.916s
sys     0m3.576s

The exact numbers may very on different systems, but the overall picture should be similar.

I've only tested this with ascii strings, but you could do something like:

while test -n "$words"; do
   c=${words:0:1}     # Get the first character
   echo character is "'$c'"
   words=${words:1}   # trim the first character
done

${#var} returns the length of var

${var:pos:N} returns N characters from pos onwards

Examples:

$ words="abc"
$ echo ${words:0:1}
a
$ echo ${words:1:1}
b
$ echo ${words:2:1}
c

so it is easy to iterate.

another way:

$ grep -o . <<< "abc"
a
b
c

or

$ grep -o . <<< "abc" | while read letter;  do echo "my letter is $letter" ; done 

my letter is a
my letter is b
my letter is c

#!/bin/bash

word=$(echo 'Your Message' |fold -w 1)

for letter in ${word} ; do echo "${letter} is a letter"; done

Here is the output:

Y is a letter o is a letter u is a letter r is a letter M is a letter e is a letter s is a letter s is a letter a is a letter g is a letter e is a letter

TEXT="hello world"
for i in {1..${#TEXT}}; do
   echo ${TEXT[i]}
done

where {1..N} is an inclusive range

${#TEXT} is a number of letters in a string

${TEXT[i]} - you can get char from string like an item from an array

To iterate ASCII characters on a POSIX-compliant shell, you can avoid external tools by using the Parameter Expansions:

#!/bin/sh

str="Hello World!"

while [ ${#str} -gt 0 ]; do
    next=${str#?}
    echo "${str%$next}"
    str=$next
done

or

str="Hello World!"

while [ -n "$str" ]; do
    next=${str#?}
    echo "${str%$next}"
    str=$next
done

sed works with unicode

IFS=$'\n'
for z in $(sed 's/./&\n/g' <(printf '???')); do
 echo hello: "$z"
done

outputs

hello: ?
hello: ?
hello: ?

I share my solution:

read word

for char in $(grep -o . <<<"$word") ; do
    echo $char
done