I am very, very new to UNIX programming (running on MacOSX Mountain Lion via Terminal). I've been learning the basics from a bioinformatics and molecular methods course (we've had two classes) where we will eventually be using perl and python for data management purposes. Anyway, we have been tasked with writing a shell script to take data from a group of files and write it to a new file in a format that can be read by a specific program (Migrate-N).
I have gotten a number of functions to do exactly what I need independently when I type them into the command line, but when I put them all together in a script and try to run it I get an error. Here are the details (I apologize for the length):
#! /bin/bash grep -f Samples.NFCup.txt locus1.fasta > locus1.NFCup.txt grep -f Samples.NFCup.txt locus2.fasta > locus2.NFCup.txt grep -f Samples.NFCup.txt locus3.fasta > locus3.NFCup.txt grep -f Samples.NFCup.txt locus4.fasta > locus4.NFCup.txt grep -f Samples.NFCup.txt locus5.fasta > locus5.NFCup.txt grep -f Samples.Salmon.txt locus1.fasta > locus1.Salmon.txt grep -f Samples.Salmon.txt locus2.fasta > locus2.Salmon.txt grep -f Samples.Salmon.txt locus3.fasta > locus3.Salmon.txt grep -f Samples.Salmon.txt locus4.fasta > locus4.Salmon.txt grep -f Samples.Salmon.txt locus5.fasta > locus5.Salmon.txt grep -f Samples.Cascades.txt locus1.fasta > locus1.Cascades.txt grep -f Samples.Cascades.txt locus2.fasta > locus2.Cascades.txt grep -f Samples.Cascades.txt locus3.fasta > locus3.Cascades.txt grep -f Samples.Cascades.txt locus4.fasta > locus4.Cascades.txt grep -f Samples.Cascades.txt locus5.fasta > locus5.Cascades.txt echo 3 5 Salex_melanopsis > Smelanopsis.mig echo 656 708 847 1159 779 >> Smelanopsis.mig echo 154 124 120 74 126 NFCup >> Smelanopsis.mig cat locus1.NFCup.txt locus2.NFCup.txt locus3.NFCup.txt locus4.NFCup.txt locus5.NFCup.txt >> Smelanopsis.mig echo 32 30 30 18 38 Salmon River >> Smelanopsis.mig cat locus1.Salmon.txt locus2.Salmon.txt locus3.Salmon.txt locus4.Salmon.txt locus5.Salmon.txt >> Smelanopsis.mig echo 56 52 24 29 48 Cascades >> Smelanopsis.mig cat locus1.Cascades.txt locus2.Cascades.txt locus3.Cascades.txt locus4.Cascades.txt locus5.Cascades.txt >> Smelanopsis.mig
The series of greps are just pulling out DNA sequence data for each site for each locus into new text files. The Samples...txt files have the sample ID numbers for a site, the .fasta files have the sequence information organized by sample ID; the grepping works just fine in command line if I run it individually.
The second group of code creates the actual new file I need to end up with, that ends in .mig. The echo lines are data about counts (basepairs per locus, populations in the analysis, samples per site, etc.) that the program needs information on. The cat lines are to mash together the locus by site data created by all the grepping below the site-specific information dictated in the echo line. You no doubt get the picture.
For creating the shell script I've been starting in Excel so I can easily copy-paste/autofill cells, saving as tab-delimited text, then opening that text file in TextWrangler to remove the tabs before saving as a .sh file (Line breaks: Unix (LF) and Encoding: Unicode (UTF-8)) in the same directory as all the files used in the script. I've tried using
chmod +x FILENAME.sh and
chmod u+x FILENAME.sh to try to make sure it is executable, but to no avail. Even if I cut the script down to just a single grep line (with the #! /bin/bash first line) I can't get it to work. The process only takes a moment when I type it directly into the command line as none of these files are larger than 160KB and some are significantly smaller. This is what I type in and what I get when I try to run the file (HW is the correct directory)
localhost:HW Mirel$ MigrateNshell.sh -bash: MigrateNshell.sh: command not found
I've been at this impass for two days now, so any input would be greatly appreciated! Thanks!!
chmod 777 ./MigrateNshell.sh
Or, add your program to a directory recognized in your $PATH variable. Example: Path Variable Example
Which will then allow you to call your program without
Unix has a variable called
PATH that is a list of directories where to find commands.
$ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/david/bin
If I type a command
foo at the command line, my shell will first see if there's an executable command
/usr/local/bin/foo. If there is, it will execute
/usr/local/bin/foo. If not, it will see if there's an executable command
/usr/bin/foo and if not there, it will look to see if
/bin/foo exists, etc. until it gets to
If it can't find a command
foo in any of those directories, it tell me command not found.
There are several ways I can handle this issue:
foois a shell script.
$PWD/fooor just plain
$PATHvariable to add the directory that contains your commands to the PATH.
You can modify
.bash_profile doesn't exist. I did that to add in
/usr/local/bin which I placed first in my path. This way, I can override the standard commands that are in the OS. For example, I have Ant 1.9.1, but the Mac came with Ant 1.8.4. I put my
ant command in
/usr/local/bin, so my version of
antwill execute first. I also added
$HOME/bin to the end of the PATH for my own commands. If I had a file like the one you want to execute, I'll place it in $HOME/bin to execute it.
Make sure you are not using "PATH" as a variable, which will override the existing PATH for environment variables.
For security reasons, the shell will not search the current directory (by default) for an executable. You have to be specific, and tell
bash that your script is in the current directory (
Add below lines in your .profile path
PATH=$PATH:$HOME/bin:$Dir_where_script_exists export PATH
Now your script should work without
I'm new to shell scripting too, but I had this same issue. Make sure at the end of your script you have a blank line. Otherwise it won't work.
#!/bin/bash may be the correct location on your system, that's not universal. Additionally, that may not be the user's preferred bash.
#!/usr/bin/env bash will select the first bash found in the path.
chmod u+x MigrateNshell.sh
Also try to
dos2unix the shell script, because sometimes it has Windows line endings and the shell does not recognize it.
$ dos2unix MigrateNshell.sh
This helps sometimes.
#! /bin/bash ^---
remove the indicated space. The shebang should be
Also make sure /bin/bash is the proper location for bash .... if you took that line from an example somewhere it may not match your particular server. If you are specifying an invalid location for bash you're going to have a problem.
Change the first line to the following as pointed out by Marc B
Then mark the script as executable and execute it from the command line
chmod +x MigrateNshell.sh ./MigrateNshell.sh
or simply execute bash from the command line passing in your script as a parameter