Difference: ProtocolsUnixCommandsQuickReference (1 vs. 8)

Revision 82024-07-09 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...
Changed:
<
<

Disk Space

>
>

Alphabetical Reference

Deleted:
<
<
The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).
  %CODE{bash}%
Deleted:
<
<
du -hc %ENDCODE%

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

%CODE{bash}% tr "\r" "\n" < input.tab >.converted.tab %ENDCODE%

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

%CODE{bash}% paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8 %ENDCODE%

Adding your path to the command prompt

Add to your .bashrc, .zshrc, or similar shell startup script: %CODE{bash}% export PS1='\w\$ ' %ENDCODE%

Useful Unix Commands

%CODE{bash}%

 cat yourfile.txt %ENDCODE% Prints the contents of the given file to the console.

%CODE{bash}% cd put/target/directory/here %ENDCODE% Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.

%CODE{bash}%

Added:
>
>
du -hc %ENDCODE% Get a human-readable output of disk space usage. If you add * then it will give you this for all files in the current directory.

%CODE{bash}%

 fgrep "words" yourfile.txt %ENDCODE% Print only lines in a file containing the given "words" in a particular file.

%CODE{bash}% find word* %ENDCODE% Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

%CODE{bash}% kill process %ENDCODE% Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

%CODE{bash}% ls %ENDCODE% Lists the contents of the working directory. Add -a to include invisible files. Add -l to show more information about each file such as its owner and permission flags.

%CODE{bash}% mkdir directoryname %ENDCODE% Makes a directory with a name specified by the user.

%CODE{bash}% nl yourfile.txt %ENDCODE% Numbers all the lines of a file.

%CODE{bash}% ps %ENDCODE% Lists all process currently running.

%CODE{bash}% pwd %ENDCODE% Prints the working directory path to the console.

%CODE{bash}% rm yourfile.txt %ENDCODE% Deletes a given file.

%CODE{bash}% rmdir yourdirectory %ENDCODE% Deletes a given directory.

%CODE{bash}% source yourfile %ENDCODE% Reads and executes commands from the given file in the current environment.

%CODE{bash}% which yourprogram %ENDCODE% Prints the the full path of the program to the console (note that its directory must be in your $PATH). Adding an -a after which prints all instances of the program.

Changed:
<
<

Adding Directories to the PATH Variable

>
>

Fixing Line Endings

 
Changed:
<
<
When you invoke commands such as python3 via at the command line, your shell searches all file directories listed in your $PATH in order to execute that command. Errors such as "command not found" when you try to run a program mean you need to add the directory containing that program to your PATH.

>
>
If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.
 
Deleted:
<
<
To show the current directories that are in your $PATH use this:
 %CODE{bash}%
Changed:
<
<
echo $PATH
>
>
tr "\r" "\n" < input.tab >.converted.tab
 %ENDCODE%
Changed:
<
<
To add a directory to your $PATH you can run this command
>
>

Merging commands to be serial on TACC

Added:
>
>
Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.
 %CODE{bash}%
Changed:
<
<
PATH=/your/directory/here:$PATH
>
>
paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8
 %ENDCODE%
Changed:
<
<
%COLOR{red}%Be sure that you include the colon and the $PATH part of this. If you leave them off then your shell will not know where to look for built-in commands like ls, cd, etc.!
>
>

Adding your path to the command prompt

 
Changed:
<
<
Generally, you want to add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be the one that is run. Because this can lead to confusion, there is even a command you can use that gives you the path to the executable that will be run if you type a command:
>
>
Add to your .bashrc, .zshrc, or similar shell startup script:
Deleted:
<
<
 %CODE{bash}%
Changed:
<
<
which
>
>
export PS1='\w\$ '
 %ENDCODE%
Added:
>
>
Now your current working directory will show up as part of your prompt. This can save you a lot of time typing ls.

Revision 72024-07-09 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

Changed:
<
<
The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).
>
>
The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).
 
Changed:
<
<
du -hc
>
>
%CODE{bash}%
Added:
>
>
du -hc %ENDCODE%
 

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.
Changed:
<
<
tr "\r" "\n" < input.tab >.converted.tab
>
>
%CODE{bash}%
Added:
>
>
tr "\r" "\n" < input.tab >.converted.tab %ENDCODE%
 

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

Changed:
<
<
paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8
>
>
%CODE{bash}%
Added:
>
>
paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8 %ENDCODE%
 

Adding your path to the command prompt

Changed:
<
<
Add to your .bashrc or similar bash startup script:
>
>
Add to your .bashrc, .zshrc, or similar shell startup script:
Added:
>
>
%CODE{bash}% export PS1='\w\$ ' %ENDCODE%
 
Changed:
<
<
export PS1='\w\$ '
>
>

Useful Unix Commands

 
Changed:
<
<

Useful Bash Commands

These commands are also compatible with the Terminal (Mac OS X) as well as the Windows Bash emulator called Cmder.

>
>
%CODE{bash}% cat yourfile.txt
Added:
>
>
%ENDCODE% Prints the contents of the given file to the console.
 
Changed:
<
<
cat yourfile.txt
Prints the contents of the given file to the console.

cd put/target/directory/here
Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.

fgrep "words" yourfile.txt
Print only lines in a file containing the given "words" in a particular file.

find word*
Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

>
>
%CODE{bash}% cd put/target/directory/here %ENDCODE% Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.
Deleted:
<
<
kill process
Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

ls
Lists the contents of the working directory.

mkdir directoryname
Makes a directory with a name specified by the user.

nl yourfile.txt
Numbers all the lines of a file.

ps
Lists all process currently running.

pwd
Prints the working directory path to the console.

rm yourfile.txt
Deletes a given file.

rmdir yourdirectory
Deletes a given directory.

source yourfile
Reads and executes commands from the given file in the current environment.

which yourprogram
Prints the the full path of the program to the console (note that its directory must be in your PATH). Adding an -a after which prints all instances of the program.

 
Added:
>
>
%CODE{bash}% fgrep "words" yourfile.txt %ENDCODE% Print only lines in a file containing the given "words" in a particular file.

%CODE{bash}% find word* %ENDCODE% Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

%CODE{bash}% kill process %ENDCODE% Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

%CODE{bash}% ls %ENDCODE% Lists the contents of the working directory. Add -a to include invisible files. Add -l to show more information about each file such as its owner and permission flags.

%CODE{bash}% mkdir directoryname %ENDCODE% Makes a directory with a name specified by the user.

%CODE{bash}% nl yourfile.txt %ENDCODE% Numbers all the lines of a file.

%CODE{bash}% ps %ENDCODE% Lists all process currently running.

%CODE{bash}% pwd %ENDCODE% Prints the working directory path to the console.

%CODE{bash}% rm yourfile.txt %ENDCODE% Deletes a given file.

%CODE{bash}% rmdir yourdirectory %ENDCODE% Deletes a given directory.

%CODE{bash}% source yourfile %ENDCODE% Reads and executes commands from the given file in the current environment.

%CODE{bash}% which yourprogram %ENDCODE% Prints the the full path of the program to the console (note that its directory must be in your $PATH). Adding an -a after which prints all instances of the program.

 

Adding Directories to the PATH Variable

Changed:
<
<
When you invoke commands such as python3 via Bash, it searches all file directories listed in your PATH in order to execute that command. Errors such as "command not found" (Linux) or "not recognized as an internal or external command, operable program or batch file" (Windows) when you try to invoke something mean you need to add the directory containing that program to your PATH.

echo %path% shows everything in your PATH on Windows while echo $PATH does the same for Linux/Unix systems.

PATH=$PATH:~/your/directory/here and PATH=~/your/directory/here:$PATH add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be what is invoked. This is for Linux/Unix.

The Windows equivalent on Cmder is PATH=%PATH%;C:\your\directory\here.

>
>
When you invoke commands such as python3 via at the command line, your shell searches all file directories listed in your $PATH in order to execute that command. Errors such as "command not found" when you try to run a program mean you need to add the directory containing that program to your PATH.

To show the current directories that are in your $PATH use this:

Added:
>
>
%CODE{bash}% echo $PATH %ENDCODE%

To add a directory to your $PATH you can run this command %CODE{bash}% PATH=/your/directory/here:$PATH %ENDCODE%

%COLOR{red}%Be sure that you include the colon and the $PATH part of this. If you leave them off then your shell will not know where to look for built-in commands like ls, cd, etc.!

Generally, you want to add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be the one that is run. Because this can lead to confusion, there is even a command you can use that gives you the path to the executable that will be run if you type a command:

%CODE{bash}% which %ENDCODE%

Revision 62020-02-14 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).

du -hc

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

tr "\r" "\n" < input.tab >.converted.tab

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8

Adding your path to the command prompt

Add to your .bashrc or similar bash startup script:

export PS1='\w\$ '

Useful Bash Commands

These commands are also compatible with the Terminal (Mac OS X) as well as the Windows Bash emulator called Cmder.

cat yourfile.txt
Prints the contents of the given file to the console.

cd put/target/directory/here
Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.

fgrep "words" yourfile.txt
Print only lines in a file containing the given "words" in a particular file.

find word*
Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

kill process
Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

ls
Lists the contents of the working directory.

mkdir directoryname
Makes a directory with a name specified by the user.

nl yourfile.txt
Numbers all the lines of a file.

ps
Lists all process currently running.

pwd
Prints the working directory path to the console.

rm yourfile.txt
Deletes a given file.

rmdir yourdirectory
Deletes a given directory.

source yourfile
Reads and executes commands from the given file in the current environment.

which yourprogram
Prints the the full path of the program to the console (note that its directory must be in your PATH). Adding an -a after which prints all instances of the program.

Adding Directories to the PATH Variable

When you invoke commands such as python3 via Bash, it searches all file directories listed in your PATH in order to execute that command. Errors such as "command not found" (Linux) or "not recognized as an internal or external command, operable program or batch file" (Windows) when you try to invoke something mean you need to add the directory containing that program to your PATH.

echo %path% shows everything in your PATH on Windows while echo $PATH does the same for Linux/Unix systems.

PATH=$PATH:~/your/directory/here and PATH=~/your/directory/here:$PATH add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be what is invoked. This is for Linux/Unix.

The Windows equivalent on Cmder is PATH=%PATH%;C:\your\directory\here.

Deleted:
<
<

Contributors

Dr. Jeffrey Barrick, Lucy LeBlanc

Revision 52016-04-07 - LucyLeblanc

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).

du -hc

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

tr "\r" "\n" < input.tab >.converted.tab

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8

Adding your path to the command prompt

Add to your .bashrc or similar bash startup script:

export PS1='\w\$ '

Added:
>
>

Useful Bash Commands

These commands are also compatible with the Terminal (Mac OS X) as well as the Windows Bash emulator called Cmder.

cat yourfile.txt
Prints the contents of the given file to the console.

cd put/target/directory/here
Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.

fgrep "words" yourfile.txt
Print only lines in a file containing the given "words" in a particular file.

find word*
Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

kill process
Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

ls
Lists the contents of the working directory.

mkdir directoryname
Makes a directory with a name specified by the user.

nl yourfile.txt
Numbers all the lines of a file.

ps
Lists all process currently running.

pwd
Prints the working directory path to the console.

rm yourfile.txt
Deletes a given file.

rmdir yourdirectory
Deletes a given directory.

source yourfile
Reads and executes commands from the given file in the current environment.

which yourprogram
Prints the the full path of the program to the console (note that its directory must be in your PATH). Adding an -a after which prints all instances of the program.

Adding Directories to the PATH Variable

When you invoke commands such as python3 via Bash, it searches all file directories listed in your PATH in order to execute that command. Errors such as "command not found" (Linux) or "not recognized as an internal or external command, operable program or batch file" (Windows) when you try to invoke something mean you need to add the directory containing that program to your PATH.

echo %path% shows everything in your PATH on Windows while echo $PATH does the same for Linux/Unix systems.

PATH=$PATH:~/your/directory/here and PATH=~/your/directory/here:$PATH add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be what is invoked. This is for Linux/Unix.

The Windows equivalent on Cmder is PATH=%PATH%;C:\your\directory\here.

Contributors

Dr. Jeffrey Barrick, Lucy LeBlanc

Revision 42016-02-29 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...
Changed:
<
<

Disk Space

>
>

Disk Space

 The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).

du -hc

Changed:
<
<

Fixing Line Endings

>
>

Fixing Line Endings

 If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

tr "\r" "\n" < input.tab >.converted.tab

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8

Added:
>
>

Adding your path to the command prompt

Add to your .bashrc or similar bash startup script:

export PS1='\w\$ '

Revision 32016-01-08 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).

du -hc

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

tr "\r" "\n" < input.tab >.converted.tab

Added:
>
>

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8

Revision 22014-05-30 - JeffreyBarrick

Changed:
<
<
META TOPICPARENT name="ProtocolList"
>
>
META TOPICPARENT name="ComputationList"
 

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).
Changed:
<
<
$du -hc
>
>
du -hc
Added:
>
>

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

tr "\r" "\n" < input.tab >.converted.tab

Revision 12012-12-26 - JeffreyBarrick

 
META TOPICPARENT name="ProtocolList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...

Disk Space

The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).

$du -hc

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright ©2025 Barrick Lab contributing authors. Ideas, requests, problems? Send feedback