Software on Greene
Software Overview
There are different types of software packages available
- Use
module availcommand to see preinstalled software.- This includes the licensed software listed below
- Singularity Containers
- You can find those already built and ready to use, at location
/scratch/work/public/singularity/ - For more information on running software with Singularity, click here.
- You can find those already built and ready to use, at location
- Python/R/Julia packages can be installed by a user
If you need another linux program installed, please contact us at hpc@nyu.edu
Software and Environment Modules
Lmod, an Environment Module system, is a tool for managing multiple versions and configurations of software packages and is used by many HPC centers around the world. With Environment Modules, software packages are installed away from the base system directories, and for each package, an associated modulefile describes what must be altered in a user's shell environment - such as the $PATH environment variable - in order to use the software package. The modulefile also describes dependencies and conflicts between this software package and other packages and versions.
To use a given software package, you load the corresponding module. Unloading the module afterwards cleanly undoes the changes that loading the module made to your environment, thus freeing you to use other software packages that might have conflicted with the first one.
Below is a list of modules and their associated functions:
| Command | Function |
|---|---|
module unload <module-name> | unload a module |
module show <module-name> | see exactly what effect loading the module will have with |
| module purge | remove all loaded modules from your environment |
module load <module-name> | load a module |
module whatis <module-name> | find out more about a software package |
| module list | check which modules are currently loaded in your environment |
| module avail | check what software packages are available |
module help <module-name> | A module file may include more detailed help for the software package |
Package Management for R, Python, & Julia, and Conda in general
- Conda environments (Python, R)
- Using virtual environments for Python
- Managing R packages with renv
- Singularity with Miniconda
Examples of software usage on Greene
Examples can be found under /scratch/work/public/examples/ and include the following
| alphafold | knitro | Singularity |
| amd GPUs | lammps | slurm |
| comsol | matlab | spark |
| c-sharp | mathematica | stata |
| crystal17 | namd | squashfs |
| fluent | orca | trinity |
| gaussian | quantum-espresso | vnc |
| hadoop-streaming | R | vscode |
| julia | sas | xvfb |
| jupyter notebooks | schrodinger |
Accessing Datasets with Singularity
Licensed Software
SCHRODINGER
Schrödinger provides a complete suite of software solutions with the latest advances in pharmaceutical research and computational chemistry. The NYU New York campus has a limited number of licenses for the Biologics Suite (ConfGen, Epik, Jaguar, Jaguar pKa, MacroModel, Prime, QSite, SiteMap), BioLuminate and the Basic Docking Suite.
Schrödinger can be used for non-commercial, academic purposes ONLY.
Using SCHRODINGER on HPC Cluster
To load Schrodinger module execute
$ module load schrodinger/2021-1
Using SCHRODINGER on NYU Lab Computers
- Request your account at: https://www.schrodinger.com/request-account
- Download the software at: https://www.schrodinger.com/downloads/releases
- Contact NYU-HPC team to request your license file.
These license servers are accessible from NYU subnet.
Please see the following links for installation of the license file:
To check licenses status
# module load schrodinger/2021-1 # load schrodinger if not already loaded
# licadmin STAT
# licutil -jobs
## For example:
[wang@cs001 ~]$ licutil -jobs
######## Server /share/apps/schrodinger/schrodinger.lic
Product & job type Jobs
BIOLUMINATE 10
BIOLUMINATE, Docking 1
BIOLUMINATE, Shared 10
CANVAS 50
COMBIGLIDE, Grid Generation 11
COMBIGLIDE, Library Generation 50
COMBIGLIDE, Protein Prep 11
COMBIGLIDE, Reagent Prep 1
EPIK 11
GLIDE, Grid Generation 11
GLIDE, Protein Prep 11
GLIDE, SP Docking 1
GLIDE, XP Descriptors 1
GLIDE, XP Docking 1
IMPACT 11
JAGUAR 5
JAGUAR, PKA 5
KNIME 50
LIGPREP, Desalter 1
LIGPREP, Ionizer 3511
LIGPREP, Ligparse 1
LIGPREP, Neutralizer 1
LIGPREP, Premin Bmin 1
LIGPREP, Ring Conf 1
LIGPREP, Stereoizer 1
LIGPREP, Tautomerizer 1
MACROMODEL 5
MACROMODEL, Autoref 5
MACROMODEL, Confgen 5
MACROMODEL, Csearch Mbae 5
MAESTRO, Unix 1000
MMLIBS 3511
PHASE, CL Phasedb Confsites 1
PHASE, CL Phasedb Convert 1
PHASE, CL Phasedb Manage 1
PHASE, DPM Ligprep Clean Structures 1
PHASE, DPM Ligprep Generate Conformers 5
PHASE, MD Create sites 1
PRIME, CM Build Membrane 2
PRIME, CM Build Structure 2
PRIME, CM Edit Alignment 2
PRIME, CM Struct Align 18
PRIME, Threading Search 2
QSITE 5
SITEMAP 10
Schrodinger Example Files
Example SBATCH jobs and outputs are available to review here:
/scratch/work/public/examples/schrodinger/
COMSOL
COMSOL is a problem-solving simulation environment, enforcing compatibility guarantees consistent multiphysics models. COMSOL Multiphysics is a general-purpose software platform, based on advanced numerical methods, for modeling and simulating physics-based problems. The package is cross-platform (Windows, Mac, Linux). The COMSOL Desktop helps you organize your simulation by presenting a clear overview of your model at any point. It uses functional form, structure, and aesthetics as the means to achieve simplicity for modeling complex realities.
This license is for academic use only with Floating Network Licensing in nature i.e., authorized users are allowed to use the software on desktops. Please contact hpc@nyu.edu for the license. However, COMSOL is also available on NYU HPC cluster Greene.
In order to check what Comsol licenses are available on Greene use comsol_licenses command in your terminal session.
Several versions of COMSOL are available on the HPC cluster. To use COMSOL on the Greene HPC cluster, please load the relevant module in your batch job submission script:
module load comsol/5.6.0.280
To submit a COMSOL job in a parallel fashion, running on multiple processing cores, follow the steps below:
- Create a directory on "scratch" as given below.
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example
- Copy example files to your newly created directory
cp /scratch/work/public/examples/comsol/run-comsol.sbatch /scratch/<net_id>/example/
cp /scratch/work/public/examples/comsol/test-input.mph /scratch/<net_id>/example/
- Edit the slurm batch script file (run-comsol.sbatch) to match your case (for example chance location of the run directory).
- Once the slurm batch script file is ready, it can be submitted to the job scheduler using sbatch. After successful completion of job, verify output log file for detail output information.
sbatch run-comsol.sbatch
MATHEMATICA
Mathematica is a general computing environment with organizing algorithmic, visualization, and user interface capabilities. The many mathematical algorithms included in Mathematica make computation easy and fast.
To run Mathematica on the Greene HPC cluster, please load the relevant module in your batch job submission script:
module load mathematica/12.1.1
In the example below the module is loaded already in the sbatch script.
To submit a batch Mathematica job for running in a parallel mode on multiple processing cores, follow below steps:
- Create a directory on "scratch" as given below.
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example
- Copy example files to your newly created directory.
cp /scratch/work/public/examples/mathematica/basic/example.m /scratch/<net_id>/example/
cp /scratch/work/public/examples/mathematica/basic/run-mathematica.sbatch /scratch/<net_id>/example
- Edit the slurm batch script file (run-mathematica.sbatch) to match your case (for example chance location of the run directory).
- Once the sbatch script file is ready, it can be submitted to the job scheduler using sbatch. After successful completion of job, verify output log file generated.
sbatch run-mathematica.sbatch
SAS
SAS is a software package which enables programmers to perform many tasks, including:
- Information retrieval
- Data management
- Report writing & graphics
- Statistical analysis and data mining
- Business planning
- Forecasting and decision support
- Operations research and project management
- Quality improvement
- Applications development
- Data warehousing (extract, transform, load)
- Platform independent and remote computing.
There are licenses for 2 CPUs on the HPC Cluster.
Running a parallel SAS job on HPC cluster (Greene):
To submit a SAS job for running on multiple processing elements, follow below steps:
- Create a directory on "scratch":
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example
- Copy example files to your newly created directory.
cp /scratch/work/public/examples/sas/test.sas /scratch/<net_id>/example/
cp /scratch/work/public/examples/sas/test2.sas /scratch/<net_id>/example/
cp /scratch/work/public/examples/sas/run-sas.sbatch /scratch/<net_id>/example/
- Submit as shown below. After successful completion of job, verify output log file generated.
sbatch run-sas.sbatch
MATLAB
MATLAB is a technical computing environment for high performance numeric computation and visualization. MATLAB integrates numerical analysis, matrix computation, signal processing, and graphics in an easy to use environment without using traditional programming.
MATLAB on personal computers and laptops
NYU has a Total Academic Headcount (TAH) license which provides campus-wide access to MATLAB, Simulink, and a variety of add-on products. All faculty, researchers, and students (on any NYU campus) can use MATLAB on their personal computers and laptops and may go to the following site to download the NYU site license software free of charge.
https://www.mathworks.com/academia/tah-portal/new-york-university-618777.html
MATLAB can be used for non-commercial, academic purposes.
There are several versions of Matlab available on the cluster and the relevant version can be loaded.
module load matlab/2020b
module load matlab/2021a
In order to run MATLAB interactively on the cluster, start an interactive slurm job, load the matlab module and launch an interactive matlab session in the terminal.
Mathworks has provided a Greene Matlab User Guide that presents useful tips and practices for using Matlab on the cluster.
STATA
Stata is a command and menu-driven software package for statistical analysis. It is available for Windows, Mac, and Linux operating systems. Most of its users work in research. Stata's capabilities include data management, statistical analysis, graphics, simulations, regression and custom programming.
Running a parallel STATA job on HPC cluster (Greene):
To submit a STATA job for running on multiple processing elements, follow below steps.
- Create a directory on "scratch":
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example
- Copy example files to your newly created directory.
cp /scratch/work/public/examples/stata/run-stata.sbatch /scratch/<net_id>/example/
cp /scratch/work/public/examples/stata/stata-test.do /scratch/<net_id>/example/
- Submit using sbatch. After successful completion of job, verify output log file generated.
sbatch run-stata.sbatch
GAUSSIAN
Gaussian uses basic quantum mechanic electronic structure programs. This software is capable of handling proteins and large molecules using semi-empirical, ab initio molecular orbital (MO), density functional, and molecular mechanics calculations.
The NYU Gaussian license only covers PIs at the Washington Square Park campus. We will grant access to you after verifying your WSP affiliation. For access, please email hpc@nyu.edu.
Running a parallel Gaussian job on HPC cluster (Greene):
To submit a Gaussian job for running on multiple processing elements, follow below steps.
- Create a directory on "scratch":
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example #Copy example files to your newly created directory.
cp /scratch/work/public/examples/gaussian/basic/test435.com /scratch/<net_id>/example/
cp /scratch/work/public/examples/gaussian/basic/run-gaussian.sbatch /scratch/<net_id>/example/
- Once the sbatch script file is ready, it can be submitted to the job scheduler using sbatch. After successful completion of job, verify output log file generated.
sbatch run-gaussian.sbatch
Knitro
Knitro is a commercial software package for solving large scale mathematical optimization problems. Knitro is specialized for nonlinear optimization, but also solves linear programming problems, quadratic programming problems, systems of nonlinear equations, and problems with equilibrium constraints. The unknowns in these problems must be continuous variables in continuous functions; however, functions can be convex or nonconvex. Knitro computes a numerical solution to the problem—it does not find a symbolic mathematical solution. Knitro versions 9.0.1 and 10.1.1 are available.
Running a parallel Knitro job on HPC cluster (Greene):
To submit a Knitro job for running on multiple processing elements, follow below steps.
- Create a directory on "scratch":
mkdir /scratch/<net_id>/example
cd /scratch/<net_id>/example
- Copy example files to your newly created directory.
cp /scratch/work/public/examples/knitro/knitro.py /scratch/<net_id>/example/
- There is no sample sbatch script available for knitro.
- After creating your own sbatch script you can execute it as follows:
sbatch <script>.sbatch