Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

 Barrick Lab  >  ComputationList  >  CommandLineBox

Contributors to this topic edittopic JeffreyBarrick, DanielDeatherage, MattMcGuffie
Topic revision: r6 - 24 Jan 2020 - 00:07:11 - Main.MattMcGuffie
 
This site is powered by the TWiki collaboration platformCopyright ©2020 Barrick Lab contributing authors. Ideas, requests, problems? Send feedback