|
ORNL Future Tech Wiki
Scalable HeterOgeneous Computing (SHOC) Benchmark Suite
The Scalable Heterogeneous Computing Benchmark Suite (SHOC) is a collection of benchmark programs testing the performance and stability of systems using computing devices with non-traditional architectures for general purpose computing, and the software used to program them. Its initial focus is on systems containing Graphics Processing Units (GPUs) and multi-core processors, and on the OpenCL programming standard. It can be used on clusters as well as individual hosts.
OpenCL is an open standard for programming a variety of types of computing devices. The OpenCL specification describes a language for programming kernels to run on an OpenCL-capable device, and an Application Programming Interface (API) for transferring data to such devices and running kernels on them. The OpenCL specification was ratified by The Khronos Group in late 2008. At the time of this writing, OpenCL implementations are just becoming publicly available. These early OpenCL implementations support running OpenCL kernels on GPUs and commodity multi-core processors, though not all implementations support both device types.
In addition to OpenCL-based benchmark programs, SHOC also includes a Compute Unified Device Architecture (CUDA) version of many of its benchmarks for comparison with the OpenCL version. CUDA, developed by NVIDIA, is an approach for programming NVIDIA GPUs for general purpose computing that predates OpenCL. Like OpenCL, CUDA-based programs use a host program running on the system's CPU to run kernels on an accelerator device (in this case, a GPU).
Features
*
Multiple benchmark applications written in both OpenCL and CUDA
*
Cluster-level parallelism with MPI
*
Node-level parallelism for multiple GPUs per node
*
Harness for running and easy reporting (in spreadsheet format) of the suite
*
Stability tests for large scale cluster resiliency testing
The Benchmarks
The SHOC benchmark suite is divided into two primary categories: stress tests and performance tests. The stress tests use computationally demanding kernels to identify OpenCL devices with bad memory, insufficient cooling, or other component problems. The performance tests are further subdivided according to their complexity and the nature of the device capability they exercise. This categorization is similar in spirit to that used in the BLAS API. Currently, the levels are:
*
Stability Tests
*
Performance Tests
o
Level 0: Very low level device characteristics (so-called “feeds and speeds”) such as bandwidth across the bus connecting the GPU to the host or peak floating point operations per second
o
Level 1: Device performance for low-level operations such as vector dot products and sorting operations
Download
Please see the download page.
Documentation
User Manual
1.
Getting started with the benchmarks.
2.
The SHOC Manual
Papers
1.
Danalis, A., Marin, G., McCurdy, C., Meredith, J., Roth, P., Spafford, K., Tipparaju, V., Vetter, J. (2010). The Scalable HeterOgeneous Computing (SHOC) Benchmark Suite. Proceedings of the Third Workshop on General-Purpose Computation on Graphics Processors (GPGPU 2010). PDF. Mar 2010.
Contributors
Future Technologies group @ ORNL
Contact Information
Please send any questions to shoc-help@email.ornl.gov
Known Issues
Known problems and workarounds can be found here. |
|