6.0 (revision 14673)
Score-P Measurement Configuration

Introduction

Score-P allows to configure several measurement parameters via environment variables. After the measurement run you can find a 'scorep.cfg' file in your experiment directory which contains the configuration of the measurement run. If you did not set any configuration values explicitly, this file will contain the default values. This file is safe to be used as input for a POSIX shell. For example, if you want to reuse the same configuration from a previous measurement run, do something like this:

$ set -a
$ . scorep.cfg
$ set +a

Measurement configuration variables have a specific type which accepts certain values.

Supported Types

String

An arbitrary character sequence, no white space trimming is done.

Path

Like String but a path is expected. Though no validation is performed.

Boolean

A Boolean value, no white space trimming is done. Accepted Boolean values for true are case insensitive and the following:

Everything else is interpreted as the Boolean value false.

Number

A decimal number, white space trimming is done.

Number with size suffixes

Like Number, but also accepts unit case insensitive suffixes after optional white space:

The 'b' suffix can be omitted.

Set

A symbolic set. Accepted members are listed in the documentation of the variable. Multiple values are allowed, are case insensitive, and are subject to white space trimming. They can be separated with one of the following characters:

Acceptable values can also have aliases, which are listed in the documentation and separated by '/'. Values can be negated by preceeding it with '~'. Order of evaluation is from left to right.

Option

Like Set, but only one value allowed to be selected.

List of Configuration Variables

This is the list of all known configure variables to control a Score-P measurement.

Note
Not all variables are supported by one Score-P installation. Use the scorep-info config-vars command to list only those supported by the used installation.
SCOREP_ENABLE_PROFILING

Enable profiling


Type:
Boolean
Default:
true


SCOREP_ENABLE_TRACING

Enable tracing


Type:
Boolean
Default:
false


SCOREP_ENABLE_UNWINDING

Enables recording calling context information for every event


Type:
Boolean
Default:
false

The calling context is the call chain of functions to the current position in the running program. This call chain will also be annotated with source code information if possible.
This is a prerequisite for sampling but also works with instrumented applications.
Note that when tracing is also enabled, Score-P does not write the usual Enter/Leave records into the OTF2 trace, but new records.
See also SCOREP_TRACING_CONVERT_CALLING_CONTEXT_EVENTS.
Note also that this supresses events from the compiler instrumentation.


SCOREP_VERBOSE

Be verbose


Type:
Boolean
Default:
false


SCOREP_TOTAL_MEMORY

Total memory in bytes per process to be consumed by the measurement system


Type:
Number with size suffixes
Default:
16000k

SCOREP_TOTAL_MEMORY will be split into pages of size SCOREP_PAGE_SIZE (potentially reduced to a multiple of SCOREP_PAGE_SIZE). Maximum size is 4 GBminus one SCOREP_PAGE_SIZE.


SCOREP_PAGE_SIZE

Memory page size in bytes


Type:
Number with size suffixes
Default:
8k

If not a power of two, SCOREP_PAGE_SIZE will be increased to the next larger power of two. SCOREP_TOTAL_MEMORY will be split up into pages of (the adjusted) SCOREP_PAGE_SIZE. Minimum size is 512 bytes.


SCOREP_EXPERIMENT_DIRECTORY

Name of the experiment directory as child of the current working directory


Type:
Path
Default:
""

The experiment directory is created directly under the current working directory. No parent directories will be created. The experiment directory is only created if it is requested by at least one substrate. When no experiment name is given (the default) Score-P names the experiment directory `scorep-measurement-tmp' and renames this after a successful measurement to a generated name based on the current time.


SCOREP_OVERWRITE_EXPERIMENT_DIRECTORY

Overwrite an existing experiment directory


Type:
Boolean
Default:
true

If you specified a specific experiment directory name, but this name is already given, you can force overwriting it with this flag. The previous experiment directory will be renamed.


SCOREP_MACHINE_NAME

The machine name used in profile and trace output


Type:
String
Default:
"Linux"

We suggest using a unique name, e.g., the fully qualified domain name. The default machine name was set at configure time (see the INSTALL file for customization options).


SCOREP_EXECUTABLE

Executable of the application


Type:
Path
Default:
""

File name, preferably with full path, of the application's executable. This is a fallback if Score-P cannot determine the exetuable's name automatically. The name is required by some compiler adapters. They will complain if this environment variable is needed.


SCOREP_ENABLE_SYSTEM_TREE_SEQUENCE_DEFINITIONS

Use system tree sequence definitions


Type:
Boolean
Default:
false

Enables an internal system tree representation that specifies a sequence of system tree nodes with one record instead of creating one record per system tree node, location group or location. It is more scalable and has less memory requirements than single-node records. However, it costs inidividual names of nodes, but simply enumerates them based on types. Currently, system tree sequence definitions support only MPI (and trivially single-process) applications.


SCOREP_FORCE_CFG_FILES

Force the creation of experiment directory and configuration files


Type:
Boolean
Default:
true

If this is set to 'true' (which is the default), the experiment directory will be created along with some configuration files, even if no substrate writes data (i.e., profiling and tracing are disabled and no substrate plugin registered for writing).
If this is set to 'false', the directory will only be created if any substrate actually writes data.


SCOREP_TIMER

Timer used during measurement


Type:
Option
Default:
tsc

The following timers are available for this installation:

tsc

Low overhead time stamp counter (X86_64) timer.

gettimeofday

gettimeofday timer.

clock_gettime

clock_gettime timer with CLOCK_MONOTONIC_RAW as clock.


SCOREP_NM_SYMBOLS

Application's symbol table obtained via 'nm -l' for compiler instrumentation


Type:
Path
Default:
""

File name, preferably with full path, of <file> that contains the <application>'s symbol table that was obtained by the command:


$ nm -l <application> 2> /dev/null > <file>


Only needed if generating the file at measurement initialization time fails, e.g., if using the 'system()' command from the compute nodes isn't possible.


SCOREP_DEBUG_UNIFY

Writes the pre-unified definitions also in the local definition trace files


Type:
Boolean
Default:
true


SCOREP_PROFILING_TASK_EXCHANGE_NUM

Number of foreign task objects that are collected before they are put into the common task object exchange buffer


Type:
Number with size suffixes
Default:
1K

The profiling creates a record for every task instance that is running. To avoid locking, the required memory is taken from a preallocated memory block. Each thread has its own memory block. On task completion, the created object can be reused by other tasks. However, if tasks migrate, the data structure migrates with them. Thus, if there is an imbalance in the migration from a source thread that starts the execution of tasks towards a sink thread that completes the tasks, the source thread may continually creating new task objects while in the sink, released task objects are collected. Thus, if the sink collected a certain number of tasks it should trigger a backflow of its collected task objects. However, this requires locking which should be avoided as much as possible. Thus, we do not want the locking to happen on every migrated task, but only if a certain imbalance occurs. This environment variable determines the number of migrated task instances that must be collected before the backflow is triggered.


SCOREP_PROFILING_MAX_CALLPATH_DEPTH

Maximum depth of the calltree


Type:
Number
Default:
30


SCOREP_PROFILING_BASE_NAME

Base for construction of the profile filename


Type:
String
Default:
"profile"

String which is used as based to create the filenames for the profile files.


SCOREP_PROFILING_FORMAT

Profile output format


Type:
Option
Default:
default

Sets the output format for the profile.


The following formats are supported:

none

No profile output. This does not disable profile recording.

tau_snapshot

Tau snapshot format.

cube4

Stores the sum for every metric per callpath per location in Cube4 format.

cube_tuple

Stores an extended set of statistics in Cube4 format.

thread_sum

Sums all locations within a location group and stores the data in Cube4 format.

thread_tuple

Sums all locations within a location group and store in addition some statistical data about the distribution among the location of a location group.

key_threads

Stores the initial location, the slowest location and the fastest location per process. Sums all other locations within a location group. The result is stored in Cube4 format.

cluster_threads

Clusters locations within a location group if they have the same calltree structure. Sums locations within a cluster. Stores the result in Cube4 format.

default

Default format. If Cube4 is supported, Cube4 is the default else the Tau snapshot format is default.


SCOREP_PROFILING_ENABLE_CLUSTERING

Enable clustering


Type:
Boolean
Default:
true


SCOREP_PROFILING_CLUSTER_COUNT

Maximum cluster count for iteration clustering


Type:
Number with size suffixes
Default:
64


SCOREP_PROFILING_CLUSTERING_MODE

Specifies the level of strictness when comparing call trees for equivalence


Type:
Option
Default:
subtree

Possible levels:

none/0

No structural similarity required.

subtree/1

The sub-trees structure must match.

subtree_visits/2

The sub-trees structure and the number of visits must match.

mpi/3

The structure of the call-path to MPI calls must match.
Nodes that are not on an MPI call-path may differ.

mpi_visits/4

Like above, but the number of visits of the MPI calls must match, too.

mpi_visits_all/5

Like above, but the number of visits must match also match on all nodes on the call-path to an MPI function.


SCOREP_PROFILING_CLUSTERED_REGION

Name of the clustered region


Type:
String
Default:
""

The clustering can only cluster one dynamic region. If more than one dynamic region are defined by the user, the region is clustered which is exited first. If another region should be clustered instead you can specify the region name in this variable. If the variable is unset or empty, the first exited dynamic region is clustered.


SCOREP_PROFILING_ENABLE_CORE_FILES

Write .core files if an error occurred


Type:
Boolean
Default:
false

If an error occurs inside the profiling system, the profiling is disabled. For debugging reasons, it might be feasible to get the state of the local stack at these points. It is not recommended to enable this feature for large scale measurements.


SCOREP_TRACING_USE_SION

Whether or not to use libsion as OTF2 substrate


Type:
Boolean
Default:
false


SCOREP_TRACING_MAX_PROCS_PER_SION_FILE

Maximum number of processes that share one sion file (must be > 0)


Type:
Number with size suffixes
Default:
1K

All processes are than evenly distributed over the number of needed files to fulfill this constraint. E.g., having 4 processes and setting the maximum to 3 would result in 2 files each holding 2 processes.


SCOREP_TRACING_CONVERT_CALLING_CONTEXT_EVENTS

Write calling context information as a sequence of Enter/Leave events to trace


Type:
Boolean
Default:
false

When recording the calling context of events (instrumented or sampled) than these could be stored in the trace either as the new CallingContext records from OTF2 or they could be converted to the legacy Enter/Leave records. This can be controlled with this variable, where the former is the false value.
This is only in effect if SCOREP_ENABLING_UNWINDING is on.
Note that enabling this will result in an increase of records per event and also of the loss of the source code locations.
This option exists only for backwards compatibility for tools, which cannot handle the new OTF2 records. This option my thus be removed in future releases.


SCOREP_ONLINEACCESS_ENABLE

Enable online access interface


Type:
Boolean
Default:
false


SCOREP_ONLINEACCESS_REG_PORT

Online access registry service port


Type:
Number
Default:
50100


SCOREP_ONLINEACCESS_REG_HOST

Online access registry service hostname


Type:
String
Default:
"localhost"


SCOREP_ONLINEACCESS_BASE_PORT

Base port for online access server


Type:
Number
Default:
50010


SCOREP_ONLINEACCESS_APPL_NAME

Application name to be registered


Type:
String
Default:
"appl"


SCOREP_FILTERING_FILE

A file name which contain the filter rules


Type:
Path
Default:
""


SCOREP_SUBSTRATE_PLUGINS

Specify list of used plugins


Type:
String
Default:
""

List of requested substrate plugin names that will be used during program run.


SCOREP_SUBSTRATE_PLUGINS_SEP

Separator of substrate plugin names


Type:
String
Default:
","

Character that separates plugin names in SCOREP_SUBSTRATE_PLUGINS.


SCOREP_METRIC_PAPI

PAPI metric names to measure


Type:
String
Default:
""

List of requested PAPI metric names that will be collected during program run.


SCOREP_METRIC_PAPI_PER_PROCESS

PAPI metric names to measure per-process


Type:
String
Default:
""

List of requested PAPI metric names that will be recorded only by first thread of a process.


SCOREP_METRIC_PAPI_SEP

Separator of PAPI metric names


Type:
String
Default:
","

Character that separates metric names in SCOREP_METRIC_PAPI and SCOREP_METRIC_PAPI_PER_PROCESS.


SCOREP_METRIC_RUSAGE

Resource usage metric names to measure


Type:
String
Default:
""

List of requested resource usage metric names that will be collected during program run.


SCOREP_METRIC_RUSAGE_PER_PROCESS

Resource usage metric names to measure per-process


Type:
String
Default:
""

List of requested resource usage metric names that will be recorded only by first thread of a process.


SCOREP_METRIC_RUSAGE_SEP

Separator of resource usage metric names


Type:
String
Default:
","

Character that separates metric names in SCOREP_METRIC_RUSAGE and SCOREP_METRIC_RUSAGE_PER_PROCESS.


SCOREP_METRIC_PLUGINS

Specify list of used plugins


Type:
String
Default:
""

List of requested metric plugin names that will be used during program run.


SCOREP_METRIC_PLUGINS_SEP

Separator of plugin names


Type:
String
Default:
","

Character that separates plugin names in SCOREP_METRIC_PLUGINS.


SCOREP_METRIC_PERF

PERF metric names to measure


Type:
String
Default:
""

List of requested PERF metric names that will be collected during program run.


SCOREP_METRIC_PERF_PER_PROCESS

PERF metric names to measure per-process


Type:
String
Default:
""

List of requested PERF metric names that will be recorded only by first thread of a process.


SCOREP_METRIC_PERF_SEP

Separator of PERF metric names


Type:
String
Default:
","

Character that separates metric names in SCOREP_METRIC_PERF and SCOREP_METRIC_PERF_PER_PROCESS.


SCOREP_SAMPLING_EVENTS

Set the sampling event and period: <event>[<period>]


Type:
String
Default:
"perf_cycles@10000000"

This selects the interrupt source for sampling.
This is only in effect if SCOREP_ENABLE_UNWINDING is on.


Possible values:
 - perf event (perf_<event>, see "perf&nbsp;list")
     period in number of events, default: 10000000
     e.g., perf_cycles@2000000
 - PAPI event (PAPI_<event>, see "papi_avail")
     period in number of events, default: 10000000
     e.g., PAPI_TOT_CYC@2000000
 - timer (POSIX timer, invalid for multi-threaded)
     period in us, default: 10000
     e.g., timer@2000


SCOREP_SAMPLING_SEP

Separator of sampling event names


Type:
String
Default:
","

Character that separates sampling event names in SCOREP_SAMPLING_EVENTS


SCOREP_TOPOLOGY_PLATFORM

Record hardware topology information for this platform, if available.


Type:
Boolean
Default:
true


SCOREP_TOPOLOGY_PROCESS

Record the Process x Thread topology.


Type:
Boolean
Default:
true


SCOREP_TOPOLOGY_USER

Record topologies provided by user instrumentation


Type:
Boolean
Default:
true


SCOREP_TOPOLOGY_MPI

Record MPI cartesian topologies.


Type:
Boolean
Default:
true


SCOREP_SELECTIVE_CONFIG_FILE

A file name which configures selective recording


Type:
Path
Default:
""


SCOREP_MPI_MAX_COMMUNICATORS

Determines the number of concurrently used communicators per process


Type:
Number
Default:
50


SCOREP_MPI_MAX_WINDOWS

Determines the number of concurrently used windows for MPI one-sided communication per process


Type:
Number
Default:
50


SCOREP_MPI_MAX_EPOCHS

Maximum amount of concurrently active access or exposure epochs per process


Type:
Number
Default:
50


SCOREP_MPI_MAX_GROUPS

Maximum number of concurrently used MPI groups per process


Type:
Number
Default:
50


SCOREP_MPI_ENABLE_GROUPS

The names of the function groups which are measured


Type:
Set
Default:
default

Other functions are not measured.


Possible groups are:

all

All MPI functions

cg

Communicator and group management

coll

Collective functions

default

Default configuration.
Includes:
 - cg
 - coll
 - env
 - io
 - p2p
 - rma
 - topo
 - xnonblock

env

Environmental management

err

MPI Error handling

ext

External interface functions

io

MPI file I/O

p2p

Peer-to-peer communication

misc

Miscellaneous

perf

PControl

rma

One sided communication

spawn

Process management

topo

Topology

type

MPI datatype functions

xnonblock

Extended non-blocking events

xreqtest

Test events for uncompleted requests

none/no
Disable feature


SCOREP_MPI_MEMORY_RECORDING

Enable tracking of memory allocations done by calls to MPI_ALLOC_MEM and MPI_FREE_MEM


Type:
Boolean
Default:
false

Requires that the MISC group is also recorded.


SCOREP_MPI_ONLINE_ANALYSIS

Enable online mpi wait states analysis


Type:
Boolean
Default:
false


SCOREP_SHMEM_MEMORY_RECORDING

Enable tracking of memory allocations done by calls to the SHMEM allocation API


Type:
Boolean
Default:
false


SCOREP_CUDA_ENABLE

CUDA measurement features


Type:
Set
Default:
no

Sets the CUDA measurement mode to capture:

runtime

CUDA runtime API

driver

CUDA driver API

kernel

CUDA kernels

kernel_serial

Serialized kernel recording

kernel_counter

Fixed CUDA kernel metrics

memcpy

CUDA memory copies

sync

Record implicit and explicit CUDA synchronization

idle

GPU compute idle time

pure_idle

GPU idle time (memory copies are not idle)

gpumemusage

Record CUDA memory (de)allocations as a counter

references

Record references between CUDA activities

flushatexit

Flush CUDA activity buffer at program exit

default/yes/1

CUDA runtime API and GPU activities.
Includes:
 - runtime
 - kernel
 - memcpy

none/no
Disable feature


SCOREP_CUDA_BUFFER

Total memory in bytes for the CUDA record buffer


Type:
Number with size suffixes
Default:
1M


SCOREP_CUDA_BUFFER_CHUNK

Chunk size in bytes for the CUDA record buffer (ignored for CUDA 5.5 and earlier)


Type:
Number with size suffixes
Default:
8k


SCOREP_OPENCL_ENABLE

OpenCL measurement features


Type:
Set
Default:
no

Sets the OpenCL measurement mode to capture:

api

OpenCL runtime API

kernel

OpenCL kernels

memcpy

OpenCL buffer reads/writes

default/yes/true/1

OpenCL API and GPU activities.
Includes:
 - api
 - kernel
 -  memcpy

none/no
Disable feature


SCOREP_OPENCL_BUFFER_QUEUE

Memory in bytes for the OpenCL command queue buffer


Type:
Number with size suffixes
Default:
8k


SCOREP_OPENACC_ENABLE

OpenACC measurement features


Type:
Set
Default:
no

Sets the OpenACC measurement mode to capture:

regions

OpenACC regions

wait

OpenACC wait operations

enqueue

OpenACC enqueue operations (kernel, upload, download)

device_alloc

OpenACC device memory allocations

kernel_properties

Record kernel properties such as the kernel name as well as the gang, worker and vector size for kernel launch operations

variable_names

Record variable names for OpenACC data allocation and enqueue upload/download

default/yes/1

OpenACC regions, enqueue and wait operations.
Includes:
 - regions
 - wait
 - enqueue

none/no
Disable feature


SCOREP_MEMORY_RECORDING

Memory recording


Type:
Boolean
Default:
false

Memory (de)allocations are recorded via the libc/C++ API.