Access to SBGrid software on Wynton HPC is limited to paying members of the SBGrid Consortium.
For bug reports related to SBGrid software, please use the SBGrid’s Report Software Bug form The Wynton sysadmins do not fix problems in SBGrid software.
SBGrid is a collection of hundred of programs for structural biology. It is kept up to date by the SBGrid team without needing intervention by the Wynton admins. Each lab using SBGrid must pay a yearly fee. This covers usage by anyone in the lab, both on Wynton HPC and any other Linux or macOS computer they own. The annual price depends on how many other labs from UCSF join. Many UCSF labs are already members. See the SBGrid website for information on how to join.
The SBGrid software stack is installed on Wynton HPC, but
is available only to paying members, which are listed on the
SBGrid Member Labs webpage. As a Wynton HPC user part
of a paying lab, you will have access to SBGrid on the
Wynton HPC cluster from being part of the sbgrid Unix
group. You can verify that this is get case by running:
[alice@dev2 ~]$ groups
boblab sbgrid
If sbgrid is not part of your output, then you currently do not
have access to the SBGrid software on Wynton HPC. If you
think this is a mistake, please contact us so we can add you.
SBGrid is available for interactive use on the development nodes and for use in job scripts on compute nodes. It is not available on the login or data-transfer nodes.
In order to use SBGrid software, the SBGrid environment must be
enabled. To enable the SBGrid, in the shell or in a job script, call
source /programs/sbgrid.shrc, e.g.
[alice@dev2 ~]$ source /programs/sbgrid.shrc
********************************************************************************
Software Support by SBGrid (www.sbgrid.org)
********************************************************************************
Your use of the applications contained in the /programs directory constitutes
acceptance of the terms of the SBGrid License Agreement included in the file
/programs/share/LICENSE. The applications distributed by SBGrid are licensed
exclusively to member laboratories of the SBGrid Consortium.
Run sbgrid-accept-license to remove the above message.
********************************************************************************
SBGrid was developed with support from its members, Harvard Medical School,
HHMI, and NSF. If use of SBGrid compiled software was an important element
in your publication, please include the following reference in your work:
Software used in the project was installed and configured by SBGrid.
cite: eLife 2013;2:e01456, Collaboration gets the most out of software.
********************************************************************************
SBGrid installation last updated: 2025-01-21
Please submit bug reports and help requests to: <bugs@sbgrid.org> or
<https://sbgrid.org/bugs>
For additional information visit https://sbgrid.org/wiki
********************************************************************************
********************************************************************************
After having enabled SBGrid, as explained above, SBGrid software tools
work like regular tools. For example, the SBGrid version of relion,
[alice@dev2 ~]$ sbgrid-info -l relion
Version information for: /programs/x86_64-linux/relion
Default version: 4.0.1_cu11.6
In-use version: 4.0.1_cu11.6
Installed versions: 4.0.1_cu11.6 5.0-beta_cu12.2 5.0-beta_cu11.6 5.0-beta_cu10.1 4.0.1_cu12.2 4.0.1_cu12.1 4.0.1_cu10.1_legacy 4.0.0_cu11.6 4.0.0_cu10.1_legacy 4.0.0_cu10.1 4.0-beta2_cu11.5 4.0-beta2_cu11.4.1 4.0-beta2_cu10.2 4.0-beta_cu9.2 3.1.4_cu11.8 3.1.4_cu11.6 3.1.4_cu10.1_legacy 3.1.4_cu10.1 3.1.3_cu10.2 3.1.1_cu9.2 3.0.8_cu10.1 2.1_cu8.0 1.4-randomphase3d 1.4b 1.4 1.3
Other available versions: 5.0-beta_cu12.2 5.0-beta_cu11.6 5.0-beta_cu10.1 4.0.1_cu12.2 4.0.1_cu12.1 4.0.1_cu10.1_legacy 4.0.0_cu11.6 4.0.0_cu10.1_legacy 4.0.0_cu10.1 4.0-beta2_cu11.5 4.0-beta2_cu11.4.1 4.0-beta2_cu10.2 4.0-beta_cu9.2 3.1.4_cu11.8 3.1.4_cu11.6 3.1.4_cu10.1_legacy 3.1.4_cu10.1 3.1.3_cu10.2 3.1.1_cu9.2 3.0.8_cu10.1 2.1_cu8.0 1.4-randomphase3d 1.4b 1.4 1.3
Overrides use this shell variable: RELION_X
can be launched as:
[alice@dev2 ~]$ relion --version
RELION version: 4.0.1
Precision: BASE=double
Many SBGrid programs run only on “modern” CPUs. Broadly speaking, there are four generations of CPUs, on the cluster - x86-64-v1, x86-64-v2, x86-64-v3, and x86-64-v4 - and SBGrid often requires x86-64-v3 compute nodes or newer. If your job fails with an obscure error such as:
Illegal instruction (core dumped)
or
*** caught illegal operation ***
address 0x2b3a8b234ccd, cause 'illegal operand'
it most likely ended up on a x86-64-v1 or x86-64-v2 compute node,
while the program required something newer. To avoid this from
happening, specify the x86-64-v=<level> resource to request a
compute node with x86-64-v3 or newer. Either specify command-line
option -l x86-64-v=3 when you submit the job, or add it is as an SGE
declaration in your script:
#$ -l x86-64-v=3 ## Script requires a CPU compliant with x86-64-v3 or newer
In the unlikely case that the SBGrid documentation says a program is compatible with pre-Haswell CPUs, it can run on also x86-64-v2 CPUs, and in some cases even x86-64-v1. If so, you can relax the x86-64 level requirement.
All our development nodes, including the GPU ones, have x86-64-v3 or newer CPUs.
If you are using SBGrid programs with GPU support, please note that
SBGrid programs are compiled for specific versions of
CUDA. Sometimes SBGrid provide different builds for multiple CUDA
versions. For example, the same RELION version is available for
different CUDA versions. Because of this, you have to make sure you
load a corresponding CUDA environment
module,
e.g. module load cuda/10.1.
As of 2022-09-16, most default versions of GPU-compatible SBGrid
programs are not compiled against CUDA 11.0.2, or greater. However,
note the newest NVIDIA A40 GPUs on the qb3-atgpu* compute nodes
require the use of programs compiled against CUDA 11.0.2, or
greater, which means those compute nodes may not be compatible with
the SBGrid program you want to run. You may need to specify a beta
version of the SBGrid programs, or avoid the qb3-atgpu* nodes. See
the SBGrid documentation for your specific program.