Vampir

Overview

Vampir is a software performance visualizer focused on highly-parallel applications. It presents a unified view on an application run including the use of programming paradigms like MPI, SHMEM, OpenMP, PThreads, HIP, CUDA, OpenCL and OpenACC. Vampir can also incorporate file I/O, hardware performance counters and other performance data sources. Various interactive displays offer detailed insight into the performance behavior of the analyzed application. Vampir’s scalable analysis server and visualization engine enable interactive navigation of large amounts of performance data.

Score-P and TAU are two instrumentation and runtime-measurement tools that can generate OTF2 trace files for Vampir to visualize. These support various instrumentation options including source level instrumentation as well as compile/link time instrumentation.

See https://vampir.eu/ for details about Vampir.

Usage

Vampir can be run a few different ways depending on a couple of factors. If you have a large trace file it could benefit from utilizing VampirServer to process the trace file with the system’s compute power while connected to the Vampir GUI at the user-site. If the trace file is small enough (< ~1 GB), it would do just fine viewing on a login node.

VampirServer is the backend software component that can run across multiple compute nodes taking advantage of the machine’s memory, this in turn provides an increase in performance for viewing large trace files i.e. >1GB. Note: VampirServer does not take advantage of GPU acceleration.

The following sections will cover the 3 different methods for using Vampir. For each method, you will need to enable X11 forwarding on an OLCF machine to allow a GUI to be launched to a remote screen. To do so, you can use the ssh -X (using option ‘-X ‘) as shown below

$ ssh -X <userid>@<machine>.olcf.ornl.gov
# On Frontier: ssh -X <userid>@frontier.olcf.ornl.gov

Please visit this link if you need more information for different ORNL Systems

Vampir on a Login Node

Note

Do not run Vampir on a login node for trace files > 1 GB! Please see the next 2 sections for running larger trace files.

After logging onto your machine with X11 forwarding, execute the series of commands below:

$ ssh -X <userid>@<machine>.olcf.ornl.gov
...
<machine>$ module load vampir
<machine>$ vampir &

Once the GUI pops up (might take a few seconds), you can load a file resident on the file system by selecting Local File for file selection.

Vampir Using VampirServer

Note

Please use this connection method for trace files larger than (> 1 GB), and see the next section Vampir Tunneling to VampirServer for an even more optimal solution. Attempting to visualize large trace files (> 1 GB) will be very slow over X11 forwarding and can cause decreased performance on the shared login nodes for other users

After connecting to your ORNL System using X11 forwarding you will need to load the Vampir module and start the VampirServer.

$ module load vampir
$ vampirserver start -- -A <Account/Project_ID> -t <walltime in HH:MM:SS> -N <num of nodes>
# Example: vampirserver start --  -A ABC123 -t 60 -N 1

Successful VampirServer startup messages should appear in terminal window. You will need this information!

[login_node]$ vampirserver start -- -A ABC123 -t 00:05:00 -N 1
Launching VampirServer...
Submitting to slurm a 1 node job with a walltime of 00:05:00...
...
VampirServer 10.5.0 Professional (00a4a90a)
Licensed to ORNL
Running 4 analysis processes... (abort with vampirserver stop 19361)
User: <USERID>
Password: XXXXXXXXXXXX
VampirServer <19361> listens on: frontier00018:30092

Launch the Vampir GUI

$ vampir &

Once the GUI has opened, you will need to connect to the VampirServer using the Remote File option as shown below. If there is a ‘recent files’ window open, select ‘open other’. Enter the node ID and the port number from above. Select Encrypted password from the Authentication dropdown option. Then press Connect.

When the server authentication window pops up, you will need to enter the USERID and the VampirServer password that was printed on the terminal screen. Once authenticated, you will be able to navigate through the filesystem to your .otf2 files

Vampir Tunneling to VampirServer

This connection method is more complex than the other 2 methods, however it also can provide a more optimal experience for very large trace files. The Vampir GUI client will be on your local machine and the server will be on the compute nodes.

Note

This method will require you to have a local copy of the Vampir GUI already installed on your machine.

If you do not have a local copy, please reach out to the help desk at help@olcf.ornl.gov for instructions on getting a local copy.

Similar to the previous methods outlined above, you will start by connecting to your ORNL System using X11 forwarding Once connected you will then need to start the VampirServer.

$ module load vampir
# Start the VampirServer
$ vampirserver start -- -A <Account/Project_ID> -t <walltime in HH:MM:SS> -N <num of nodes>
# Example: vampirserver start --  -A ABC123 -t 60 -N 1

Once you have successfully authenticated, you will need the information printed on the terminal window. That includes:

• Node ID

• Port Number

• password

Once the VampirServer is started, in a fresh terminal window on your Local machine you can then initiate the port forward command. This will open a secure tunnel from your local machine to the backend server running VampirServer.

Port Forwarding

[local_machine]$ ssh -L <localport>:<Node ID>:<Remote port>  <USERID>@<SYSTEM>.olcf.ornl.gov

# Example on Frontier: ssh -L 30000:frontier00018:30092 <USERID>@frontier.olcf.ornl.gov
# Localhost port is 30000.  Vampirserver listens on frontier00018 at port 30092.

Note

The local port number can be any unused port number on your local machine…try a number between 30000-30030. To check if the port you picked is open run:

$ netstat -ab | grep "<selected port number>"
# This can take a minute to return anything. If nothing is returned, your selected port is open

After submitting the port forward command as seen above, it will ask for your login information to access the OLCF system. Leave this terminal window open!

Launch the Vampir GUI on your local machine

Similar to how we have connected Vampir to the VampirServer in the previous section, you will follow the same steps except you will use localhost for the server name and your local machine port number you selected. Press ‘Connect’ and this should open the authentication window where you will enter your UserID and the VampirServer password printed after a successful connection.

Once the authentication step is complete, it should open up the remote filesystem for you to navigate to and load your .otf2 trace file.

Vampir GUI Demo

Please see the 30-minute video below on 2023 Trace-Based Performance Analysis with Score-P + Vampir to get a brief introduction to Vampir and Score-P. This recording is taken from the Frontier Training Workshop (August 2023), Friday, August 25th, 2023, presented by Bill Williams, TU-Dresden.

2023 Trace-Based Performance Analysis with Score-P + Vampir from OLCF on Vimeo.