Data Transfer Nodes (DTNs)
The Data Transfer Nodes (DTNs) are hosts specifically designed to provide optimized data transfer between OLCF systems and systems outside of the OLCF network. These nodes perform well on local-area transfers as well as the wide-area data transfers for which they are tuned. The OLCF recommends that users use these nodes to improve transfer speed and reduce load on computational systems’ login and service nodes. OLCF provides two sets of DTNs: one for systems in our moderate enclave and a second for systems in the open enclave.
DTN access is automatically granted to all enabled OLCF users. For interactive access to DTNs (
sftp), connect to
dtn.ccs.ornl.gov (for the moderate enclave) or
opendtn.ccs.ornl.gov (for the open enclave). For example:
For more information on connecting to OLCF resources, see Connecting for the first time.
Access From Globus Online
DTNs are also accessible via the “OLCF DTN” (for moderate) and “NCCS Open DTN” (for open) Globus endpoints. For more information on using Globus at OLCF see Globus.
Batch Queue (Slurm)
The moderate DTNs also support batch jobs. The system contains 8 nodes accessible through the DTN batch system.
Most OLCF resources now use the Slurm batch scheduler, including the DTNs. Below is a table of useful commands for Slurm.
View batch queue
Submit batch script
Submit interactive batch job
0 - 24 hrs
max 1 job running per user
Submitting jobs to Frontier
Submitting to the Frontier batch queue is supported from the moderate DTNs using the
-M flag; however, we will need to ensure that the programming environment is not inheritied from the DTNs. The
--export=NONE sbatch submission flag in combination with the
SLURM_EXPORT_ENV environment variable should prevent the DTN environment from copying to the job, but does not wipe the environment on the compute nodes.
sbatch -M frontier –export=NONE …
Inside of the slurm script:
We will also need to use the SBATCH directive
threads-per-core to alter the DTNs default setting to match Frontier’s default setting.