Skip to content

Home

Jump to bottom
Jim Galarowicz edited this page Oct 27, 2016 · 5 revisions

Welcome to the cbtf wiki!

Component Based Tool Framework (CBTF) Project Technical Overview

The goal of the CBTF project is to provide users and tool developers a tool development framework which expedites the ability to combine compatible reusable components into performance tools of differing levels of complexity. In other words, the CBTF project provides support for creating a compatible interface for components, finding compatible components and defining an interface API which facilitates communication between components. In the CBTF framework, a component is defined as a “black box” entity with zero or more inputs and zero or more outputs.

This diagram shows the abstract “black box” view of a component and the concrete, implementation through an API, on the right side. The diagram also illustrates the definition of a component. Components are based on a data-flow model, accepting zero or more inputs, performing arbitrary processing, and emitting zero or more outputs. They are written using C++, inheriting from a base class that provides basic infrastructure. They provide metadata in form of the component’s type, version, and input/output names and types. All components are versioned and multiple versions the same component can co-exist. Finally, components can be packaged in a variety of different ways.

XML Diagram

The definition of components and the connection of components can be done using the light weight CBTF library by making the appropriate API calls to generate the single process component network. The diagram below gives a high level overview of how the CBTF library would facilitate the creation of a single process example tool from components built using the CBTF framework. Each network contains one or more component instances and connections between those component instances. Any topology is possible, from pipelines to graphs with cycles. Networks are recursive in the sense that the network is itself a component, with input and outputs, which can be instantiated and further connection to other networks.

Component Diagram

Additionally, the single process component network can be generated automatically by specifying the component network definition and component connection using a network specification written in XML. The XML specification file contains all the necessary information about each of the components and how they are connected to each other. Using the CBTF XML and CBTF base libraries, a tool builder can create a tool from a set of components by specifying an XML specification file that defines the components and how they are connected. Specification of the component network in XML is now done manually, but it could be automated by the development of a GUI tool. A GUI tool is planned for creating the description file, as these files are verbose in nature.

The distribution of the CBTF component network across nodes of a distributed system is also supported. The CBTF library named libcbtf-mrnet supports instantiation of component networks at various levels of a MRNet based tree network. The following diagram illustrates how single component or multiple component networks can be deployed at the backend, communication, and frontend node levels within the MRNet tree based network.

Distributed Network Diagram

The purpose of the diagram above is to show the definition distributed component networks. These are (currently) MRNet based. A complete component network is located on each node in the MRNet network. These component networks are connected via symbolically-named MRNet streams (shown in red) which can move data either up or down the MRNet network. As with non-distributed component networks, distributed component networks are recursively defined and specified via XML files. Finally, light-weight MRNet is supported by allowing messages with specific MRNet tags to be associated with named streams.

SOURCES:

To obtain the CBTF project sources these are the four CBTF based project pages which point to the, open source, source repositories:

CBTF project contains the base code for CBTF that supports creating components, component networks and the support to connect these components and component networks into sequential and distributed network tools.

CBTF Krell project contains the Krell Institute contributions to the CBTF project. These contributions include many performance data collectors and support libraries as well as some example tools that drive the data collection at HPC levels of scale.

CBTF Argo Navis project contains the CUDA collector and supporting libraries that was done as a result of a DOE SBIR grant.

CBTF LANL project contains a memory tool and data center type system command monitoring tool.

Clone this wiki locally