Although there is no hard quota in place on UCSF Box, the storage is limited and not really meant to store large data sets. There are reports that the UCSF Box maintainers have started to reach out to users who misuse it to store huge amounts of data. Please use it responsibly.
Below instructions work on data transfer nodes only. Attempts to use them on development nodes will result in “Access failed: 401 Unauthorized” errors.
In order to do access UCSF Box as describe below, you need to:
With a UCSF Box-specific password (see above), you can use, for instance, the
curl tools to access to your UCSF Box account. Start by logging in to one of the data-transfer nodes, either directly from outside or via a login node, e.g.
[alice@log1 ~]$ ssh dt2 alice1@dt2:s password: XXXXXXXXXXXXXXXXXXX [alice@dt2 ~]$
Then, verify that your UCSF Box setup is correct by logging into the root of your UCSF Box folder using your UCSF Box-specific password (not your Wynton HPC password):
[alice@dt2 ~]$ lftp --user firstname.lastname@example.org ftps://ftp.box.com Password: XXXXXXXX <== UCSF Box password here! lftp email@example.com@ftp.box.com:~> ls drwx------ 1 owner group 0 Jun 12 2014 Grant_R01.pdf drwx------ 1 owner group 0 Sep 30 2016 Securefirstname.lastname@example.org lftp email@example.com@ftp.box.com:~> exit [alice@dt2 ~]$
🛑 Never specify your password via a command-line argument! If you do, it will be visible to all other users via commands such as
lftp as above, you need to manually enter your password, which can be tedious or even prevent automatic file transfers in batch scripts. A solution to this is to set up the FTPS credentials in
~/.netrc. Here is what it could look like:
[alice@dt2 ~]$ cat ~/.netrc machine ftp.box.com login firstname.lastname@example.org password AliceSecretPwd2017
~/.netrc file must be kept private, otherwise its content could be readable to other users.
Since the password is fully visible in plain text, make sure to keep this file private at all times, otherwise users on the system can see all your credentials, i.e.
[alice@dt2 ~]$ chmod 600 ~/.netrc [alice@dt2 ~]$ ls -l ~/.netrc -rw------- 1 alice alice 72 Jul 3 15:10 /home/alice/.netrc
To verify that the automatic authentication works, try to log in again. You should no longer be prompted for your password - instead
lftp gets it automatically from
~/.netrc. For example:
[alice@dt2 ~]$ lftp --user email@example.com ftps://ftp.box.com lftp firstname.lastname@example.org@ftp.box.com:~> ls drwx------ 1 owner group 0 Jun 12 2014 Grant_R01.pdf drwx------ 1 owner group 0 Sep 30 2016 Secureemail@example.com lftp firstname.lastname@example.org@ftp.box.com:~> exit $
curl also recognizes
~/.netrc credentials, e.g.
[alice@dt2 ~]$ curl --netrc -O ftps://ftp.box.com/Grant_R01.pdf % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 15.6M 100 15.6M 0 0 1561k 0 0:00:10 0:00:10 --:--:-- 3918k [alice@dt2 ~]$ ls -la Grant_R01.pdf -rw-r--r-- 1 alice cluster 16453180 Jul 10 21:13 Grant_R01.pdf
To upload a file, we can do:
[alice@dt2 ~]$ curl --netrc --upload-file notes.txt ftps://ftp.box.com/