Rclone is a powerful tool for transferring files to, from, and within Wynton. Here will give some examples on how to use it from your local computer to access files on the Wynton file system over a remote SFTP connection. At the end, you will find instructions on how to “mount” your Wynton home folder such that you can access your Wynton files as if they were located on your local computer. This means that you can use your favorite file manager to browse files and directories, and you can view the files on your local computer, e.g. use your favorite image viewer to display PNG files generated by your Wynton jobs.
SSH key access to Wynton: The below instructions assume that you
have configured SSH access via SSH keys, i.e. you can SSH to Wynton
without having to manually enter your Wynton password each
time. Make sure this works for you before attempting to use
rclone. If have not done this setup, please see the ‘Log in
without Password’ instruction.
Rclone on your local computer: The instructions also assume that you have Rclone installed on your local computer. It is available for macOS, Linux, and Windows and straightforward to install. Please see https://rclone.org/install/ for installation instructions for your operating system.
Please do not continue below this point if the above is not working.
Before we can start, we have to make sure Rclone can connect to Wynton from your local machine. Specifically, we want it to connect to one of Wynton’s data transfer nodes, because those are the most efficient nodes for file transfers.
To configure an Rclone “remote” to dt2.wynton.ucsf.edu over SFTP, start with the following command on your local computer:
{local}$ rclone config create dt2.wynton.ucsf.edu sftp user=alice host=dt2.wynton.ucsf.edu pass="$(rclone obscure "")" key_use_agent=true
[dt2.wynton.ucsf.edu]
type = sftp
host = dt2.wynton.ucsf.edu
pass = *** ENCRYPTED ***
key_use_agent = true
user = alice
Importantly, do not forget pass="$(rclone obscure "")" - it is a
required workaround for an obscure Rclone bug - without it you will
get errors such as “Failed to create file system for “dt2.wynton.ucsf.edu:”: NewFs: couldn’t connect SSH: ssh:
handshake failed: ssh: unable to authenticate, attempted methods
[none], no supported methods remain”.
Next, we need to make sure Rclone has access to your private SSH keys. We recommend that you use the SSH agent that runs on your local machine for this. If you don’t have one running, you can start it with:
{local}$ eval "$(ssh-agent -s)"
Agent pid 1015799
After this, add your private SSH key;
{local}$ ssh-add ~/.ssh/laptop_to_wynton
Enter passphrase for /home/alice/.ssh/laptop_to_wynton:
Identity added: /home/alice/.ssh/laptop_to_wynton (alice@local)
That’s it! To verify that Rclone now has access to this remote, use:
{local}$ rclone about dt2.wynton.ucsf.edu:
Total: 11.477 PiB
Used: 8.016 PiB
Free: 3.461 PiB
Please note the trailing colon (:) - it must not be forgotten.
The output summarizes the disk quota on /wynton/, similarly to what
df -h /wynton outputs when logged in to Wynton.
With the above dt2.wynton.ucsf.edu Rclone remote set up,
you now have easy access to the Wynton file system from your local
computer.
To list all files in your Wynton home directory, use:
{local}$ rclone lsd dt2.wynton.ucsf.edu:
-1 2024-02-13 23:56:29 -1 .apptainer
-1 2024-05-20 17:27:01 -1 authorized_keys
-1 2025-11-18 14:34:32 -1 .bash_history
...
To list all files in your Wynton ~/my-analysis/, use:
{local}$ rclone lsd dt2.wynton.ucsf.edu:my-analysis/
Note, you must not use ~ to refer to your Wynton home directory - it
will not work, because it refers to your home directory on your local
machine. Instead, drop ~/, which works because the start location is
from your Wynton home directory.
To list all files in your group’s project directory, use:
{local}$ rclone lsd dt2.wynton.ucsf.edu:/wynton/group/boblab/
To copy a single file on your local file system to your Wynton home directory, use:
{local}$ rclone copy one_file.tsv dt2.wynton.ucsf.edu:
To copy a single file from Wynton to your local computer, use:
{local}$ rclone copy dt2.wynton.ucsf.edu:one_file.tsv .
Rclone has many more features around file transfers, e.g. synchronize the files in a local directory with those in a Wynton directory. To see what Rclone offers, see https://rclone.org/docs/.
You can mount your Wynton home directory remotely so that your
Wynton files appear to be regular files on your local computer. To
mount your Wynton home directory such that it appears under
~/wynton-home/ on your local computer, use:
{local}$ mkdir -p ~/mnt/wynton-home
{local}$ rclone mount dt2.wynton.ucsf.edu: ~/mnt/wynton-home
There will be no output displayed, and you will not get back to the
prompt until you terminate the above rclone mount using
Ctrl-C.
To check that it works, open another terminal on your local computer, and do:
{local}$ ls -al ~/wynton-home
total 757
drwxr-xr-x. 57 alice boblab 230 Nov 18 16:04 .
drwxr-xr-x. 3 root root 1 Apr 9 2019 ..
drwx------. 4 alice boblab 2 Feb 13 2024 .apptainer
-rw-r--r--. 1 alice boblab 0 May 20 2024 authorized_keys
-rw-------. 1 alice boblab 38967 Nov 18 15:49 .bash_history
...
Note that these files do not live on your local computer - instead,
rclone mount makes it appear as if they do. This means that you can
list and view files, browse directories, copy and remove files, and
even edit them as if they were local files. For example, to view the
first six lines of your Wynton ~/.bashrc file, you can do:
{local}$ cd ~/mnt/wynton-home
{local}$ head -n 6 .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
To copy a local file to your Wynton home directory, use:
{local}$ cp ~/one_file.tsv ~/mnt/wynton-home/
There are options to run this command in the background, but we recommend that you do not do this, because it is harder to see errors and harder to terminate the mount when done.
To mount all of Wynton’s file system, including /wynton/home/,
/wynton/groups/, /wynton/scratch/, among others, use:
{local}$ mkdir ~/mnt/wynton
{local}$ rclone mount dt2.wynton.ucsf.edu:/ ~/mnt/wynton
After this, you can list, e.g., /wynton/scratch/alice/ as:
{local}$ cd ~/mnt/wynton/scratch/alice/
{local}$ ls -la
...
Note that you can also access the Rclone-mounted files and directories using your local file navigator, e.g. Finder on macOS, File Explorer on MS Windows, and GNOME Files on Ubuntu Linux.
If you don’t use an SSH agent, Rclone can connect only if you have an SSH key without a passphrase. In such cases, you can modify the above remote settings as:
{local}$ rclone config create dt2.wynton.ucsf.edu sftp user=alice host=dt2.wynton.ucsf.edu pass="$(rclone obscure "")" key_file=~/.ssh/laptop_to_wynton
[dt2.wynton.ucsf.edu]
type = sftp
key_file = /home/alice/.ssh/laptop_to_wynton
user = alice
host = dt2.wynton.ucsf.edu
pass = *** ENCRYPTED ***
You can use all of Rclone from Wynton. It is installed on the data-transfer nodes (only). It is also available on development nodes via the CBI software stack, but we strongly recommend to transfer files on a data-transfer node, because it is faster in most cases.