FABRIC is an NSF-funded, nationwide research infrastructure for at-scale experimentation in networking, cybersecurity, distributed computing and storage, machine learning, and science applications. It provides a programmable network fabric connecting site-distributed compute and storage resources with directly attached experimental hardware, including SmartNICs, GPUs, FPGAs, and NVMe devices. Selected sites also provide P4-programmable switches for experiments involving programmable data planes.
Unlike a conventional batch-scheduled HPC cluster, FABRIC is a configurable scientific instrument and distributed laboratory. Experimenters create project-scoped slices containing virtual machine slivers, network services, storage, and optional hardware components across one or more FABRIC sites. Experiments can be designed, deployed, and managed through the FABRIC Portal, JupyterHub, or programmatically through FABlib and the FABRIC APIs.
Jobs
FABRIC does not use Slurm or another batch scheduler and has no queues or partitions. Experiments are provisioned as project-scoped slices containing virtual machine slivers, network services, storage, and permitted experimental hardware. FABRIC work is therefore created, submitted, monitored, renewed, and deleted through the FABRIC Portal or FABlib rather than with commands such as sbatch, squeue, or scancel.
Before creating a slice, review current site and component availability through the FABRIC Resource Overview or query the current inventory through FABlib. Available capacity varies by site and changes as experiments are created and released.
A minimal FABlib example that creates and submits one virtual machine sliver is:
from fabrictestbed_extensions.fablib.fablib import FablibManager
fablib = FablibManager()
slice = fablib.new_slice(name="access-example")
node = slice.add_node(name="node1")
slice.submit()
slice.show()
print(node.get_ssh_command())For a real experiment, specify the required site, CPU cores, RAM, disk size, image, network services, and optional hardware components when building the slice. Specialized components such as GPUs, FPGAs, SmartNICs, NVMe devices, or programmable switches may require additional project permissions and are subject to site availability.
Useful slice-management operations include:
slice.show() # Display current slice information
print(slice.get_state()) # Display the current slice state
slice.renew(days=1) # Extend the lease by one day
slice.delete() # Release all resources in the sliceslice.delete() permanently releases the resources assigned to that slice. Run it only after the experiment and any required data transfers are complete.
FABRIC slices have finite leases rather than scheduler wall-time limits. Renew a slice before its lease expires when continued use is permitted, and delete the slice as soon as the experiment is complete so the resources return to the shared testbed.
See the FABlib API and programming guide, the complete FABlib class and method reference, the FABRIC example notebooks, and the live FABRIC resource inventory.
Storage
FABRIC VM slivers may use a built-in VM disk, an attachable NVMe SSD, or persistent rotating block storage at supported sites. The built-in disk size is requested when the VM is created. NVMe attachment requires the appropriate project permission and is subject to site availability.
Persistent block storage outlives the slice. A Project Owner requests a volume for specified sites, with a volume name, size, and expected duration. FABRIC presents the allocation as a bare block device that the experimenter must format and manage. The VM and volume must be at the same site and belong to the same project, and a volume can be attached to only one VM at a time. Persistent-storage allocations typically last 12 months and are reviewed for continued retention at the end of the allocation period.
The FABRIC Artifact Manager provides versioned, shareable storage for experiment artifacts with author, project, or public visibility. Artifact contents are uploaded as tar archives and are separate from block storage attached to a running slice.
File Transfer
Transfer files directly to or from a provisioned virtual machine sliver. Do not use the FABRIC bastion as the transfer destination. Use the sliver's assigned management address, the private sliver key, and the FABRIC SSH configuration.
scp -i /path/to/sliver_private_key \
-F /path/to/fabric_ssh_config \
local-file \
VM_USERNAME@VM_MANAGEMENT_ADDRESS:/remote/path/The virtual machine username depends on the selected image. Use the SSH details returned by FABlib for the provisioned node when determining the correct username, key, and management address.
| Supported Methods | Data Transfer Node | URL |
|---|---|---|
| SCP | RECOMMENDED | Provisioned VM sliver management address | Transfer files to and from a FABRIC VM sliver |
Login to Fabric
FABRIC requires a separate FABRIC experimenter account. Enroll through the FABRIC Portal and authenticate through CILogon, normally by selecting your institutional identity provider. Use the same identity provider for every FABRIC login so that CILogon continues to associate the login with the same FABRIC account.
Complete the enrollment workflow, verify the requested email address, and wait for FABRIC approval. Access to testbed resources also requires membership in an active FABRIC project. A Project Lead or Project Owner can add members and request any additional project permissions required for specialized hardware or services.
FABRIC does not document a separate FABRIC-managed two-factor authentication enrollment. If the selected institutional identity provider requires multi-factor authentication, complete that provider's MFA process when prompted.
The recommended starting environment is FABRIC JupyterHub. It provides a private environment with FABlib, example notebooks, and the configuration needed to interact with FABRIC. Users may also create and manage slices through the FABRIC Portal or run FABlib from another properly configured system.
Access to provisioned virtual machine slivers requires two separate SSH keypairs:
- A bastion keypair used to pass through the FABRIC bastion.
- A sliver keypair whose public key is installed on the virtual machines created in a slice.
In a configured FABlib environment, generate the keys and SSH configuration with:
from fabrictestbed_extensions.fablib.fablib import FablibManager
fablib = FablibManager()
fablib.create_bastion_keys()
fablib.create_sliver_keys()
fablib.create_ssh_config()Keep both private keys secure and never upload, email, or share them. FABRIC uses bastion.fabric-testbed.net only as an SSH ProxyJump. The bastion is not an interactive login host and should not be used as the destination for SSH, SCP, or SFTP sessions.
After a slice is active, FABlib can display the complete SSH command for a provisioned node:
print(node.get_ssh_command())The command includes the appropriate bastion, key, username, and management-address information. The virtual machine username depends on the image selected when the node was created.