Voyager is an innovative AI system designed specifically for science and engineering research at scale. Voyager is focused on supporting research in science and engineering that is increasingly dependent upon artificial intelligence and deep learning as a critical element in the experimental and/or computational work.
Jobs
Voyager runs Kubernetes. Kubernetes is an open-source platform for managing containerized workloads and services. A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. The application workloads are executed by placing containers into Pods to run on nodes. The resources required by the Pods are specified in YAML files.
Basic YAML Example:
apiVersion: v1kind: Podmetadata: name: hpu-test-podspec: restartPolicy: Never containers: - name: gaudi-container image: vault.habana.ai/gaudi-docker/1.8.0/ubuntu22.04/habanalabs/tensorflow-installer-tf-cpu-2.8.4:1.8.0-690-20230214 command: ["hl-smi"] resources: limits: habana.ai/gaudi: 1 hugepages-2Mi: 3800Mi memory: 32G cpu: 1 requests: memory: 32G cpu: 1
For computer, inference, or gaudi examples: Basic Jobs
Queue specifications
| Name | Purpose | Nodes | CPU cores / node | GPUs / node | Node RAM | Jobs
30 days
|
Wait Time
30-day trend
|
Wall Time
30-day trend
|
|---|---|---|---|---|---|---|---|---|
| gaudi | Designed for high-performance AI training using Intel Habana Gaudi training processors. | 42 | Intel Xeon Gold 6336 | — | 512 GB | — | — | — |
| goya | Dedicated for Habana Gaudi model inference, utilizing 2 first-generation nodes. | 2 | Xeon Gold 6240 (40 cores) | — | 384 GB | — | — | — |
| compute | General-purpose pre/post-data processing | 36 | Intelx86 (2 cores) | — | 384 GB | — | — | — |
Storage
File System
| Directory | Path | Quota | Purge | Backup | Notes |
|---|---|---|---|---|---|
| home | /home/username | 200 GB | Not backed up | The home directory is limited in space and should be used only for source code storage. User will have access to 200GB in /home. Users should keep usage on $HOME under 200GB. | |
| projects | /voyager/projects/project/username | 153 GB | Not backed up | NSF mounted project space | |
| scratch | /voyager/ceph/users/username | Purged when the pod is removed from the node. | Not backed up | Needs to be mounted as an emptyDir. Only exists while the Pod is running on the node. | |
| ceph | /voyager/ceph/users/username | Not backed up | The SDSC Ceph file system ( /voyager/ceph/users/$USER) IS NOT an archival file |
External Storage
For more information, see: Storage
File Transfer
All of Expanse's NFS and Lustre filesystems are accessible via the Globus endpoint xsede#expanse. See Data Movement.
| Supported Methods | Data Transfer Node | URL |
|---|---|---|
| GLOBUS (COMING SOON) | https://www.sdsc.edu/systems/voyager/user_guide.html#narrow-wysiwyg-5 |
Login to SDSC Voyager AI System
Voyager uses ssh key pairs for access.
Approved users will need to send their ssh public key consult@sdsc.edu to gain access to the system. To generate an SSH public key, open your terminal (macOS/Linux) or PowerShell/Command Prompt (Windows) and run the command ssh-keygen -t ed25519 -C "your_email@example.com". See Creating SSH Keys.
To log in to Voyager from the command line, run:
ssh <your_username>@login.voyager.sdsc.edu
Notes and hints
- Voyager will not maintain local passwords, your public key will need to be appended to your ~/.ssh/authorized_keys file to enable access from authorized hosts. RSA, ECDSA and ed25519 keys are accepted. Make sure you have a strong passphrase on the private key on your local machine.
- Do not use the login node for computationally intensive processes. The login nodes are meant for file editing, simple data analysis, and other tasks that use minimal compute resources. All computationally demanding jobs should be run using kubernetes.
See this page for more information about System Access.
SSH Login
$ ssh <your_username>@login.voyager.sdsc.edu