Difference: ProtocolsUnixCommandsQuickReference (7 vs. 8)

Revision 82024-07-09 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Unix Commands Quick Reference

Useful commands and flags that we get tired of looking up...
Changed:
<
<

Disk Space

>
>

Alphabetical Reference

Deleted:
<
<
The du command is verbose and confusing if you run it without options. Here is how to get a human-readable output and a grand total for the path argument (omit for current directory).
  %CODE{bash}%
Deleted:
<
<
du -hc %ENDCODE%

Fixing Line Endings

If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.

%CODE{bash}% tr "\r" "\n" < input.tab >.converted.tab %ENDCODE%

Merging commands to be serial on TACC

Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.

%CODE{bash}% paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8 %ENDCODE%

Adding your path to the command prompt

Add to your .bashrc, .zshrc, or similar shell startup script: %CODE{bash}% export PS1='\w\$ ' %ENDCODE%

Useful Unix Commands

%CODE{bash}%

 cat yourfile.txt %ENDCODE% Prints the contents of the given file to the console.

%CODE{bash}% cd put/target/directory/here %ENDCODE% Changes the working directory to whatever path is typed after cd. Without an additional argument, cd brings one back to the home directory.

%CODE{bash}%

Added:
>
>
du -hc %ENDCODE% Get a human-readable output of disk space usage. If you add * then it will give you this for all files in the current directory.

%CODE{bash}%

 fgrep "words" yourfile.txt %ENDCODE% Print only lines in a file containing the given "words" in a particular file.

%CODE{bash}% find word* %ENDCODE% Locates and gives file names of all files in the working directory containing word in the beginning of the file name. The * represents a wildcard so it can be placed before, after, or both in order to find files that contain the given query in a particular part of the file name.

%CODE{bash}% kill process %ENDCODE% Ends a particular process. Note that you can type killall to end multiple processes that match the name that is input.

%CODE{bash}% ls %ENDCODE% Lists the contents of the working directory. Add -a to include invisible files. Add -l to show more information about each file such as its owner and permission flags.

%CODE{bash}% mkdir directoryname %ENDCODE% Makes a directory with a name specified by the user.

%CODE{bash}% nl yourfile.txt %ENDCODE% Numbers all the lines of a file.

%CODE{bash}% ps %ENDCODE% Lists all process currently running.

%CODE{bash}% pwd %ENDCODE% Prints the working directory path to the console.

%CODE{bash}% rm yourfile.txt %ENDCODE% Deletes a given file.

%CODE{bash}% rmdir yourdirectory %ENDCODE% Deletes a given directory.

%CODE{bash}% source yourfile %ENDCODE% Reads and executes commands from the given file in the current environment.

%CODE{bash}% which yourprogram %ENDCODE% Prints the the full path of the program to the console (note that its directory must be in your $PATH). Adding an -a after which prints all instances of the program.

Changed:
<
<

Adding Directories to the PATH Variable

>
>

Fixing Line Endings

 
Changed:
<
<
When you invoke commands such as python3 via at the command line, your shell searches all file directories listed in your $PATH in order to execute that command. Errors such as "command not found" when you try to run a program mean you need to add the directory containing that program to your PATH.

>
>
If you get odd errors after transferring a text file from a PC or Mac to a Unix machine, it's likely that you have a problem with newline characters. This is especially common when editing files in Excel and saving them to tab-delimited or comma-delimited files for input into. You can generally avoid this problem and fix the line endings by using an industrial strength text editor. This command converts Mac line endings in a saved Excel file to Unix line endings.
 
Deleted:
<
<
To show the current directories that are in your $PATH use this:
 %CODE{bash}%
Changed:
<
<
echo $PATH
>
>
tr "\r" "\n" < input.tab >.converted.tab
 %ENDCODE%
Changed:
<
<
To add a directory to your $PATH you can run this command
>
>

Merging commands to be serial on TACC

Added:
>
>
Sometimes you have 96 short jobs that you want to run serially 8 at a time on 12 cores rather than requesting 96 cores. This command will combine every 8 lines in a file into one line separated by && so that these commands are now run on one core.
 %CODE{bash}%
Changed:
<
<
PATH=/your/directory/here:$PATH
>
>
paste -s -d'#######\n' commands | sed "s/#/ \&\& /g" > commands8
 %ENDCODE%
Changed:
<
<
%COLOR{red}%Be sure that you include the colon and the $PATH part of this. If you leave them off then your shell will not know where to look for built-in commands like ls, cd, etc.!
>
>

Adding your path to the command prompt

 
Changed:
<
<
Generally, you want to add the given directory to the end or the beginning of your PATH variable list, since when you invoke a command, the directories will be searched from beginning to end and the first match will be the one that is run. Because this can lead to confusion, there is even a command you can use that gives you the path to the executable that will be run if you type a command:
>
>
Add to your .bashrc, .zshrc, or similar shell startup script:
Deleted:
<
<
 %CODE{bash}%
Changed:
<
<
which
>
>
export PS1='\w\$ '
 %ENDCODE%
Added:
>
>
Now your current working directory will show up as part of your prompt. This can save you a lot of time typing ls.
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright ©2025 Barrick Lab contributing authors. Ideas, requests, problems? Send feedback