[bash] How to loop over files in directory and change path and add suffix to filename

I need to write a script that starts my program with different arguments, but I'm new to Bash. I start my program with:

./MyProgram.exe Data/data1.txt [Logs/data1_Log.txt].

Here is the pseudocode for what I want to do:

for each filename in /Data do
  for int i = 0, i = 3, i++
    ./MyProgram.exe Data/filename.txt Logs/filename_Log{i}.txt
  end for
end for

So I'm really puzzled how to create second argument from the first one, so it looks like dataABCD_Log1.txt and start my program.

This question is related to bash for-loop filenames glob

The answer is


for file in Data/*.txt
do
    for ((i = 0; i < 3; i++))
    do
        name=${file##*/}
        base=${name%.txt}
        ./MyProgram.exe "$file" Logs/"${base}_Log$i.txt"
    done
done

The name=${file##*/} substitution (shell parameter expansion) removes the leading pathname up to the last /.

The base=${name%.txt} substitution removes the trailing .txt. It's a bit trickier if the extensions can vary.


You can use finds null separated output option with read to iterate over directory structures safely.

#!/bin/bash
find . -type f -print0 | while IFS= read -r -d $'\0' file; 
  do echo "$file" ;
done

So for your case

#!/bin/bash
find . -maxdepth 1 -type f  -print0 | while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done

additionally

#!/bin/bash
while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done < <(find . -maxdepth 1 -type f  -print0)

will run the while loop in the current scope of the script ( process ) and allow the output of find to be used in setting variables if needed


Looks like you're trying to execute a windows file (.exe) Surely you ought to be using powershell. Anyway on a Linux bash shell a simple one-liner will suffice.

[/home/$] for filename in /Data/*.txt; do for i in {0..3}; do ./MyProgam.exe  Data/filenameLogs/$filename_log$i.txt; done done

Or in a bash

#!/bin/bash

for filename in /Data/*.txt; 
   do
     for i in {0..3}; 
       do ./MyProgam.exe Data/filename.txt Logs/$filename_log$i.txt; 
     done 
 done

Sorry for necromancing the thread, but whenever you iterate over files by globbing, it's good practice to avoid the corner case where the glob does not match (which makes the loop variable expand to the (un-matching) glob pattern string itself).

For example:

for filename in Data/*.txt; do
    [ -e "$filename" ] || continue
    # ... rest of the loop body
done

Reference: Bash Pitfalls


Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to for-loop

List append() in for loop Prime numbers between 1 to 100 in C Programming Language Get current index from foreach loop how to loop through each row of dataFrame in pyspark TypeScript for ... of with index / key? Is there a way in Pandas to use previous row value in dataframe.apply when previous value is also calculated in the apply? Python for and if on one line R for loop skip to next iteration ifelse How to append rows in a pandas dataframe in a for loop? What is the difference between ( for... in ) and ( for... of ) statements?

Examples related to filenames

Rename multiple files in a folder, add a prefix (Windows) How to loop over files in directory and change path and add suffix to filename Why do I get a SyntaxError for a Unicode escape in my file path? Git copy file preserving history A html space is showing as %2520 instead of %20 How do I get the file name from a String containing the Absolute file path? DateTime.ToString() format that can be used in a filename or extension? Get only filename from url in php without any variable values which exist in the url Obtaining only the filename when using OpenFileDialog property "FileName" Build the full path filename in Python

Examples related to glob

How to loop over files in directory and change path and add suffix to filename glob exclude pattern Regular Expression usage with ls How can I search sub-folders using glob.glob module? Loop through all the files with a specific extension Deleting all files from a folder using PHP? Python glob multiple filetypes How to count the number of files in a directory using Python Get a filtered list of files in a directory How to use glob() to find files recursively?