Alumni Project
Tools for Performance Evaluation Research
PI: David H. Bailey (LBNL); Co-PIs: Bronis de Supinski (LLNL), Jack Dongarra (U. Tenn.), Thomas Dunigan (ORNL), Paul Hovland (ANL), Jeffrey Hollingsworth (U. Mar.), Boyanna Norris (ANL), Daniel Quinlan (LLNL), Celso Mendes (U. Ill.), Shirley Moore (U. Tenn.), Daniel Reed (U. Ill.), Allan Snavely (SDSC), Erich Strohmaier (LBNL), Jeffrey Vetter (LLNL), Patrick Worley (ORNL); SciDAC ISICs: David Brown (TSTT); Phil Colella (APDEC), David Keyes (TOPS); SciDAC Applications: Donald Batchelor (WPI), Mark Gordon (EST), Kwok Ko (AST), Anthony Mezzacappa (TSI), Robert Malone (CCSM), Robert Sugar (QCD)
Summary
The PERC tools effort is developing a software infrastructure for monitoring and collecting performance data from execution of scientific applications on high performance computers, as well as novel mechanisms for optimizing application performance automatically. In many cases, prototypes of these tools already existed, but they had limited functionality or platform coverage. Our work focuses on adding necessary tool features, porting them to all platforms of interest (to provide comparable data for our PERC modeling and analysis), and enhancing tool reliability and usability for application scientists. The resulting improved tools are being applied to improve performance of SciDAC project codes.
PERC performance tools cover the spectrum from end-user tools to low-level
infrastructure. One focus is end-user tools that gather data used by
PERC performance models, with concurrent work on tools to improve application
performance. We are modifying existing tools and developing new ones
to collect data not obtainable with previous performance tools. Our
goal is to produce an interoperable suite of measurement, analysis,
and tuning tools through four tightly coupled efforts:
- Creation of end-user tools that integrate measurement and analysis, providing a common interface for comparing measurements across platforms and executions and correlating these data with benchmarks and application codes.
- Development of flexible instrumentation systems for capturing hardware and software interactions, instruction frequencies, memory reference behavior, and execution overheads.
- Creation of a domain-specific analysis and source-to-source optimization infrastructure to simplify application optimization and tool implementation.
- Development of data management software to track performance experiments and data across time and space.
End User Tools
Illinois’ SvPablo is a graphical environment for instrumenting source code and browsing dynamic performance data. It supports performance capture, analysis, and presentation for C/F77/F90 codes, and exploits hardware performance counters via the PAPI toolkit (see below). We are extending, documenting, deploying and hardening the SvPablo toolkit as the PERC project’s basis for integrated performance characterization and prediction. With PERC support, SvPablo has been ported to new platforms, including Linux IA-64, Compaq Alpha, IBM Power3, and the NEC SX-6. SvPablo now supports analysis of OpenMP codes and interoperates with Dyninst; ROSE integration is underway. The enhanced toolkit has been deployed at HPC centers and applied to several SciDAC codes analyzed in PERC.
The SIGMA project is a joint research effort of Maryland and IBM Research. With PERC support, we have extended SIGMA to use Dyninst for memory operation instrumentation. This has provided support for additional platforms (the original SIGMA only ran on IBM Power systems) and allowed dynamic control of when instrumentation is enabled. This has greatly improved SIGMA’s scalability. We are extending SIGMA for OpenMP and will support “what-if” predictions for changing OpenMP directives.
We are also developing a new methodology, called performance assertions (PA), that allows users to assert performance properties for code constructs explicitly. The PA runtime implicitly gathers performance data based on user assertions and verifies the expectations at runtime. The runtime system jettisons raw data for successful assertions while reacting to failures. A prototype implementation of PA, discussed in an SC2002 paper, has been used to validate manual FLOPS in PETSc, a key component of the TOPS ISIC solver infrastructure.
Flexible Instrumentation Systems
The goal of the PAPI activity within PERC is to create an easy-to-use, common set of interfaces to the hardware performance counters available on all major processor platforms, providing application developers with the data needed to tune application software on different platforms. The latest release of PAPI, version 2.3.1, added support for Compaq Alpha Tru64 UNIX, Linux IA-64, IBM POWER4/AIX5, and preliminary support for Pentium-4/Linux.
A second goal of the PAPI project is to enable straightforward instrumentation of multi-threaded and multi-process codes, including dynamic instrumentation. We recently released the Dynaprof toolkit, which inserts performance instrumentation into a running application’s address space and includes wall-clock and PAPI probes.
For dynamic object code instrumentation, the dyninst API provides a machine independent interface for creation of tools and applications that use runtime code patching. Under PERC, this API has been implemented on additional platforms and a memory instrumentation capability has been introduced. This capability enables other tools, such as the SIGMA trace generator, to dynamically enable and disable memory instrumentation during program execution.
Domain-Specific Analysis and Source-to-Source Optimization
The ROSE tool is defining new ways to use compilation techniques to build source-to-source translators and customized analysis-oriented preprocessors and tools. Thus, it functions as both an end-user tool and as tool infrastructure. A ROSE-based preprocessor optimizes operators defined within TSTT, a SciDAC ISIC, using domain-specific optimizations impossible for standard compilers. ROSE permits tools to read, use, or even rewrite the application source code in a way that enhances the utility of existing tools and makes many new tools possible. This infrastructure is also used in a next-generation automatic differentiation tool (ADIC 2.0), which has shown great promise on TOPS applications.
Data and Experiment Management
The Repository-in-a-Box (RIB) effort provides a toolkit for building and main-taining interoperable metadata repositories. With PERC support, we have established a repository of tools and data on performance evaluation. This repository facilitates collection and retrieval of diverse performance data without imposing special formats or data structures, and it is being populated with PERC performance data.
For further information please contact:
Prof. Dan Reed, PERC Tools Project Lead
University of Illinois, Dept. of Comp. Sci.
Tel: 217-244-0078 Em: reed@cs.uiuc.edu
back to project page
