Skip to content

OMPD 5 and limited OMPD support for cuda devices #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 69 commits into
base: ompd
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
d8edcc7
Merge branch 'master' into ompd
Jun 21, 2018
c836b8d
[OMPD] Fixing runtime ompd type references
manorom Jun 22, 2018
73fe9de
[OMPD] add cmake modules necessary to build gdb-wrapper
manorom Jun 22, 2018
255356b
[OMPD] align libompd and odb w/ new OpenMP spec
manorom Jun 25, 2018
0c3427e
Add ompd_get_thread_handle for cuda.
manorom Jun 29, 2018
63309d5
[OMPD] Adds some support for omp states on cuda
manorom Jul 2, 2018
e1949d3
[OMPD] Further align code to spec
manorom Jul 2, 2018
f9cc9b4
[OMPD] Aligned callback names with spec
manorom Jul 3, 2018
cffa430
Fix formatting mistake
manorom Jul 3, 2018
e72f4a7
[OMPD] Add parallel and reduction state to npvtx
manorom Jul 3, 2018
27e8cc8
[OMPD] Add reduction state + save thread coords
manorom Jul 5, 2018
322f3f6
[OMPD] cleanup and + ompd parallel handle for cuda
manorom Jul 10, 2018
8a25e59
[OMPD] Remove unsupported api functions + fixes
manorom Jul 13, 2018
485b132
[OMPD] add support for some ICVs to libompd
manorom Jul 13, 2018
f37acc3
[OMPD] Add serial state in cuda device rtl
manorom Jul 13, 2018
4c40f17
[OMPD] Add some icvs + move values in ompd_types.h
manorom Jul 13, 2018
528b3ed
[OMPD] Align ompd.h formatting to spec
manorom Jul 16, 2018
c612600
[OMPD] changed some comments for current spec
manorom Jul 16, 2018
5dd42cc
[OMPD] Some cleanup in gdb-wrapper
manorom Jul 17, 2018
3473deb
[OMPD] fix omp version
manorom Jul 17, 2018
64b73b0
[OMPD] Add type compatibillity for new spec to odb
manorom Jul 17, 2018
43d1e58
[OMPD] Code clean-up in odb
manorom Jul 17, 2018
66775c1
[OMPD] Fix bug introd. by type changes f. new spec
manorom Jul 17, 2018
bf0ea8a
[OMPD] Add "tasks" command to odb
manorom Jul 18, 2018
ee03881
Merge remote-tracking branch 'origin/ompd' into ompd-devices
manorom Jul 18, 2018
504e7be
[OMPD] Add get_task_function and test code in odb
manorom Jul 19, 2018
d208dc1
[OMPD] Fix some formatting
manorom Jul 20, 2018
94c3081
[OMPD] Fix formatting mistakes
manorom Jul 20, 2018
267a230
[OMPD] Fix linking for gdb-wrapper
manorom Jul 23, 2018
14b0cea
[OMPD] Save cuda kernel info in ompd handles
manorom Jul 23, 2018
33ccbfa
Merge remote-tracking branch 'origin/master' into ompd-devices
manorom Jul 23, 2018
7a79281
[OMPD] Set correct omp states for all modes (cuda)
manorom Jul 30, 2018
5bec325
[OMPD] Fix thread handles in all exec modes (cuda)
manorom Jul 30, 2018
b3de3ae
[OMPD] Add ompd break points for thread begin/end
manorom Aug 1, 2018
b96b31c
[OMPD] Add some support: parallel handles (cuda)
manorom Aug 7, 2018
3609e86
[OMPD] remove usage of kernel id in odb
manorom Aug 8, 2018
a619134
[OMPD] Add ompd_get_thread_in_parallel for cuda
manorom Aug 8, 2018
011a376
[OMPD] Move memory segment values to ompd_types.h
manorom Aug 8, 2018
8d49887
[OMPD] Re-enable checking for ompd state tracking
manorom Aug 8, 2018
300c218
Remove accidentally committed .bak file
manorom Aug 9, 2018
e0538aa
Merge remote-tracking branch 'origin/ompd' into ompd-devices
manorom Aug 10, 2018
c806b3e
Merge remote-tracking branch 'origin/ompd' into ompd-devices
manorom Aug 10, 2018
783262c
[OMPD] Make odb initialization at first command
manorom Aug 13, 2018
17f53de
[OMPD] Add code to support some ICVs on cuda devs
manorom Aug 13, 2018
598ec68
[OMPD] removed ompdAllocatable class
manorom Aug 14, 2018
21d2ad7
[OMPD] Remove _ompd_device_handle_s struct
manorom Aug 14, 2018
6351c6f
[OMPD] Work around for cuda-gdb name mangling bug
manorom Aug 23, 2018
4b6f24b
[OMPD] Add support for cuda icvs to odb
manorom Aug 23, 2018
0322eab
[OMPD] Fix enclosing_parallel + enable target ICVs
manorom Aug 27, 2018
098dd55
[OMPD] Fix ompd_get_thread_in_parallel + add test
manorom Aug 30, 2018
66b0339
[OMPD] Update ompd_types.h + remove kernelId field
manorom Sep 4, 2018
09ed60b
[OMPD] Fix ompd_types.h
manorom Sep 5, 2018
b2be2d6
[OMPD] Add some support for tasks for cuda
manorom Sep 7, 2018
24e9595
[OMPD] Fix parallel level for explicit tasks
manorom Sep 17, 2018
d15db76
[OMPD] omp_device_t -> ompd_device_t
manorom Sep 18, 2018
58e5df5
[OMPD] Add examples to test ompd on cuda
manorom Sep 21, 2018
42982e4
[OMPD] Add ompd breakpoints
manorom Sep 21, 2018
23f5ea0
[OMPD] Fix task/parallel handle interaction
manorom Sep 21, 2018
8c4c7ae
[OMPD] Make OMPD configurable for deviceRTLs
manorom Sep 25, 2018
901ff1b
[OMPD] Clean up by removing unnecesarry header
manorom Sep 25, 2018
6a825eb
[OMPD} Remove deprecated ompd_process_handle_t
manorom Sep 25, 2018
bc9c1e3
[OMPD] Remove TODOs
manorom Sep 27, 2018
d861bcd
[OMPD] Add some kernel info + removed TODO notes
manorom Sep 28, 2018
c1b4c5a
Add information on scheduling parent for the master task in a paralle…
jprotze Oct 12, 2018
66b0f8d
Initialize all address variables
Oct 17, 2018
726b21b
Fix ompd_get_thread_id
jprotze Dec 14, 2018
36fcaf3
Initialize lwt field for ompd_get_task_in_parallel
Jan 16, 2019
e1c6cad
Fix ompd_get_task_function for implicit tasks
jprotze Jan 17, 2019
f9bb739
Use the right team to get the function pointer
Jan 17, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[OMPD] align libompd and odb w/ new OpenMP spec
This commit aligns libompd and gdb-wrapper with a newer OpenMP spec.
* Some functionality was desiabled as it relies on information no longer
exposed via OMPD.
* The OMPD callback functions still have non-standard names (but
  signatures should be correct now)
* Cuda device initialization should work now, but not much other OMPD
  device funtionality.
  • Loading branch information
manorom committed Jun 25, 2018
commit 255356b0b315c9c92ba15aeccda4fed43d587466
20 changes: 12 additions & 8 deletions libompd/gdb-wrapper/Callbacks.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,7 @@ void initializeCallbacks(const GdbProcessPtr &proc)
cb.dmemory_alloc = CB_dmemory_alloc;
cb.dmemory_free = CB_dmemory_free;
cb.print_string = CB_print_string;
cb.get_thread_context_for_osthread = CB_thread_context;
cb.get_containing_process_context = CB_process_context;
cb.get_thread_context_for_thread_id = CB_thread_context;
cb.tsizeof_prim = CB_tsizeof_prim;
cb.tsymbol_addr = CB_tsymbol_addr;
cb.read_tmemory = CB_read_tmemory;
Expand Down Expand Up @@ -78,14 +77,14 @@ ompd_rc_t CB_dmemory_free (

ompd_rc_t CB_thread_context (
ompd_address_space_context_t *context,
ompd_osthread_kind_t kind,
ompd_thread_id_kind_t kind,
ompd_size_t sizeof_osthread,
const void* osthread,
ompd_thread_context_t **tcontext
)
{
ompd_rc_t ret = context ? ompd_rc_ok : ompd_rc_stale_handle;
if (kind == ompd_osthread_cudalogical) {
if (kind == ompd_thread_id_cudalogical) {
*tcontext = ((OMPDContext*)context)->getContextForThread((CudaThread*)osthread);
}
else {
Expand Down Expand Up @@ -135,7 +134,12 @@ ompd_rc_t CB_tsizeof_prim(
inited=1;
init_sizes();
}
memcpy(sizes, prim_sizes, sizeof(prim_sizes[0])*ompd_type_max);
sizes->sizeof_char = prim_sizes[ompd_type_char];
sizes->sizeof_short = prim_sizes[ompd_type_short];
sizes->sizeof_int = prim_sizes[ompd_type_int];
sizes->sizeof_long = prim_sizes[ompd_type_long];
sizes->sizeof_long_long = prim_sizes[ompd_type_long_long];
sizes->sizeof_pointer = prim_sizes[ompd_type_pointer];

return ret;
}
Expand Down Expand Up @@ -175,7 +179,7 @@ ompd_rc_t CB_tsymbol_addr(
parser.matchAddressValue(gdb->readOutput().c_str(), addr);

if (strlen(addr) > 0)
symbol_addr->address = (ompd_taddr_t) strtoull (addr, NULL, 0);
symbol_addr->address = (ompd_addr_t) strtoull (addr, NULL, 0);
else if (strlen(addr) == 0)
ret = ompd_rc_error;

Expand Down Expand Up @@ -267,7 +271,7 @@ ompd_rc_t CB_write_tmemory (
ompd_address_space_context_t *context,
ompd_thread_context_t *tcontext,
ompd_address_t addr,
ompd_tword_t nbytes,
ompd_word_t nbytes,
const void *buffer)
{
return ompd_rc_unsupported;
Expand All @@ -277,7 +281,7 @@ ompd_rc_t CB_read_tmemory (
ompd_address_space_context_t *context,
ompd_thread_context_t *tcontext,
ompd_address_t addr,
ompd_tword_t nbytes,
ompd_word_t nbytes,
void *buffer)
{
if (!context)
Expand Down
6 changes: 3 additions & 3 deletions libompd/gdb-wrapper/Callbacks.h
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ ompd_rc_t CB_dmemory_free (

ompd_rc_t CB_thread_context (
ompd_address_space_context_t *context,
ompd_osthread_kind_t kind,
ompd_thread_id_kind_t kind,
ompd_size_t sizeof_osthread,
const void* osthread,
ompd_thread_context_t **tcontext);
Expand All @@ -71,15 +71,15 @@ ompd_rc_t CB_read_tmemory (
ompd_address_space_context_t *context,
ompd_thread_context_t *tcontext,
const ompd_address_t addr,
ompd_tword_t nbytes,
ompd_word_t nbytes,
void *buffer
);

ompd_rc_t CB_write_tmemory (
ompd_address_space_context_t *context,
ompd_thread_context_t *tcontext,
const ompd_address_t addr,
ompd_tword_t nbytes,
ompd_word_t nbytes,
const void *buffer
);

Expand Down
90 changes: 64 additions & 26 deletions libompd/gdb-wrapper/OMPDCommand.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@
* Author: Ignacio Laguna
* Contact: ilaguna@llnl.gov
*/
#include <config.h>
//#include <config.h>
#include "OMPDCommand.h"
#include "OMPDContext.h"
#include "Callbacks.h"
#include "OutputString.h"
#include "Debug.h"
#include "omp.h"
#include "ompd.h"
#include "ompd_test.h"
//#include "ompd_test.h"
#define ODB_LINUX
#include "CudaGdb.h"

#include <cstdlib>
Expand All @@ -38,9 +40,9 @@ OMPDCommandFactory::OMPDCommandFactory()

// Load OMPD DLL and get a handle
#ifdef ODB_LINUX
functions->ompdLibHandle = dlopen("libompd_intel.so", RTLD_LAZY);
functions->ompdLibHandle = dlopen("libompd.so", RTLD_LAZY);
#elif defined(ODB_MACOS)
functions->ompdLibHandle = dlopen("libompd_intel.dylib", RTLD_LAZY);
functions->ompdLibHandle = dlopen("libompd.dylib", RTLD_LAZY);
#else
#error Unsupported platform!
#endif
Expand Down Expand Up @@ -76,7 +78,7 @@ FOREACH_OMPD_API_FN(OMPD_FIND_API_FUNCTION)
// Initialize OMPD library
ompd_callbacks_t *table = getCallbacksTable();
assert(table && "Invalid callbacks table");
ompd_rc_t ret = functions->ompd_initialize(table);
ompd_rc_t ret = functions->ompd_initialize(0, table);
if (ret != ompd_rc_ok)
{
out << "ERROR: could not initialize OMPD\n";
Expand Down Expand Up @@ -191,11 +193,11 @@ void OMPDThreads::execute() const
for(auto i: thread_ids) {
ompd_thread_handle_t* thread_handle;
ompd_rc_t ret = functions->ompd_get_thread_handle(
addrhandle, ompd_osthread_pthread, sizeof(i.second),
addrhandle, ompd_thread_id_pthread, sizeof(i.second),
&(i.second), &thread_handle);
if (ret == ompd_rc_ok)
{
ompd_state_t state;
ompd_word_t state;
ompd_wait_id_t wait_id;
ret = functions->ompd_get_state(thread_handle, &state, &wait_id);
printf(" %-12u %p 0x%lx\t%i\t%lx\n",
Expand All @@ -211,31 +213,38 @@ void OMPDThreads::execute() const
int omp_cuda_threads = 0;
vector<OMPDCudaContextPool*> cuda_ContextPools;
map<uint64_t, bool> device_initialized;
map<ompd_taddr_t, ompd_address_space_handle_t*> address_spaces;
map<ompd_addr_t, ompd_address_space_handle_t*> address_spaces;

for(auto i: cuda.threads) {
if (!device_initialized[i.coord.cudaContext]) {
cout << "Cuda device with context " << i.coord.cudaContext << "not initialized as OpenMP device. Trying to initialize\n";
OMPDCudaContextPool* cpool;
cpool = new OMPDCudaContextPool(&i);
ompd_rc_t result;

device_initialized[i.coord.cudaContext] = true;
result = functions->ompd_device_initialize(
cpool->getGlobalOmpdContext(),
i.coord.cudaContext,
ompd_device_kind_cuda,
addrhandle,
cpool->getGlobalOmpdContext(),
ompd_device_kind_cuda,
sizeof(i.coord.cudaContext),
&i.coord.cudaContext,
&cpool->ompd_device_handle);

if (result != ompd_rc_ok)
{
cout << "Could not initalize device with context " << i.coord.cudaContext << ". Probably not a OpenMP device\n";
continue;
}
cout << "Device initialized\n";

address_spaces[i.coord.cudaContext] = cpool->ompd_device_handle;
}

ompd_thread_handle_t* thread_handle;
ompd_rc_t ret = functions->ompd_get_thread_handle(
address_spaces[i.coord.cudaContext],
ompd_osthread_cudalogical,
ompd_thread_id_cudalogical,
sizeof(i.coord), &i.coord,
&thread_handle);

Expand Down Expand Up @@ -321,7 +330,7 @@ void OMPDCallback::execute() const

/*ompd_rc_t CB_read_tmemory (
ompd_context_t *context,
ompd_taddr_t addr,
ompd_addr_t addr,
ompd_tword_t bufsize,
void *buffer
);*/
Expand All @@ -333,7 +342,7 @@ void OMPDCallback::execute() const
return;
}
long long temp=0;
ompd_taddr_t addr = (ompd_taddr_t)strtoll(extraArgs[1].c_str(), NULL, 0);
ompd_addr_t addr = (ompd_addr_t)strtoll(extraArgs[1].c_str(), NULL, 0);
int cnt = atoi(extraArgs[2].c_str());
ret = CB_read_tmemory(
host_contextPool->getGlobalOmpdContext(), NULL, {0,addr}, cnt, &temp);
Expand All @@ -345,7 +354,7 @@ void OMPDCallback::execute() const
/*ompd_rc_t CB_tsymbol_addr (
ompd_context_t *context,
const char *symbol_name,
ompd_taddr_t *symbol_addr);*/
ompd_addr_t *symbol_addr);*/

if (extraArgs[0] == "tsymbol_addr")
{
Expand Down Expand Up @@ -388,6 +397,7 @@ void OMPDApi::execute() const

if (extraArgs[0] == "get_threads")
{
#if 0 // MARKER_MR: TODO: reimplement this functionality with breakpoints
if(extraArgs.size()>1)
{
hout << "Usage: odb api get_threads" << endl;
Expand All @@ -406,6 +416,8 @@ void OMPDApi::execute() const
sout << "0x" << hex << thread_handle_array[i] << ", ";
}
sout << endl << "";
#endif
hout << "The 'odb api threads' command has been temporarily removed for the migration to a new ompd standard\n";
}

}
Expand All @@ -424,7 +436,7 @@ vector<ompd_thread_handle_t*> odbGetThreadHandles(ompd_address_space_handle_t* a
{
ompd_thread_handle_t* thread_handle;
ret = functions->ompd_get_thread_handle(
addrhandle, ompd_osthread_pthread, sizeof(i.second) ,&(i.second), &thread_handle);
addrhandle, ompd_thread_id_pthread, sizeof(i.second) ,&(i.second), &thread_handle);
if (ret!=ompd_rc_ok)
continue;
thread_handles.push_back(thread_handle);
Expand All @@ -437,7 +449,7 @@ vector<ompd_parallel_handle_t*> odbGetParallelRegions(OMPDFunctionsPtr functions
ompd_rc_t ret;
ompd_parallel_handle_t * parallel_handle;
vector<ompd_parallel_handle_t*> parallel_handles;
ret = functions->ompd_get_top_parallel_region(
ret = functions->ompd_get_current_parallel_handle(
th, &parallel_handle);
while(ret == ompd_rc_ok)
{
Expand All @@ -450,6 +462,10 @@ vector<ompd_parallel_handle_t*> odbGetParallelRegions(OMPDFunctionsPtr functions

bool odbCheckParallelIDs(OMPDFunctionsPtr functions, vector<ompd_parallel_handle_t*> phs)
{
sout << "Checking of parallel IDs has been disabled for upgrade of ompd in branch ompd-devices\n";
// MARKER_MR: TODO: fix checking of parallel ids
return true;
#if 0
bool res=true;
// ompd_rc_t ret;
int i=0;
Expand All @@ -466,10 +482,15 @@ bool odbCheckParallelIDs(OMPDFunctionsPtr functions, vector<ompd_parallel_handle
if (ompt_res != ompd_res) res=false;
}
return res;
#endif
}

bool odbCheckParallelNumThreads(OMPDFunctionsPtr functions, vector<ompd_parallel_handle_t*> phs)
{
sout << "Checking of parallel IDs has been disable for upgrade of ompd in branch ompd-devices\n";
// MARKER_MR: TODO: fix checking of parallel ids for num threads
return true;
#if 0
bool res=true;
// ompd_rc_t ret;
int i=0;
Expand All @@ -486,10 +507,15 @@ bool odbCheckParallelNumThreads(OMPDFunctionsPtr functions, vector<ompd_parallel
if (ompt_res != ompd_res) res=false;
}
return res;
#endif
}

bool odbCheckTaskIDs(OMPDFunctionsPtr functions, vector<ompd_task_handle_t*> ths)
{
sout << "Checking of task IDs has been disable for upgrade of ompd in branch ompd-devices\n";
// MARKER_MR: TODO: fix checking of task ids
return true;
#if 0
bool res=true;
// ompd_rc_t ret;
int i=0;
Expand All @@ -506,37 +532,47 @@ bool odbCheckTaskIDs(OMPDFunctionsPtr functions, vector<ompd_task_handle_t*> ths
if (ompt_res != ompd_res) res=false;
}
return res;
#endif
}

vector<ompd_task_handle_t*> odbGetTaskRegions(OMPDFunctionsPtr functions, ompd_thread_handle_t* th)
{
ompd_rc_t ret;
ompd_task_handle_t * task_handle;
ompd_task_handle_t *task_handle;
vector<ompd_task_handle_t*> task_handles;
ret = functions->ompd_get_top_task_region(
ret = functions->ompd_get_current_task_handle(
th, &task_handle);
while(ret == ompd_rc_ok)
{
task_handles.push_back(task_handle);
ret = functions->ompd_get_ancestor_task_region(
task_handle, &task_handle);
ret = functions->ompd_get_generating_task_handle(
task_handle, &task_handle); // MARKER_MR: TODO: is it generating or scheduling task or something different?
}
return task_handles;
}

vector<ompd_task_handle_t*> odbGetImplicitTasks(OMPDFunctionsPtr functions, ompd_parallel_handle_t* ph)
{
// ompd_rc_t ret;
ompd_task_handle_t** task_handles;
int num_tasks;
int num_tasks = evalGdbExpression("call omp_get_num_threads()");
vector<ompd_task_handle_t*> return_handles;
/*ret = */functions->ompd_get_implicit_task_in_parallel(

for (int i=0; i < num_tasks; ++i) {
ompd_task_handle_t* task_handle;
functions->ompd_get_task_in_parallel(
ph, i, &task_handle);
return_handles.push_back(task_handle);
}
#if 0
ompd_task_handle_t* task_handles;
/*ret = */functions->ompd_get_task_in_parallel(
ph, &task_handles, &num_tasks);
for(int i=0; i<num_tasks; i++)
{
return_handles.push_back(task_handles[i]);
}
free(task_handles);
#endif
return return_handles;
}

Expand Down Expand Up @@ -573,10 +609,12 @@ void OMPDTest::execute() const
auto implicit_task_h = odbGetImplicitTasks(functions, ph);
for(auto ith: implicit_task_h)
{
#if 0 //MARKER_MR: TODO: fix this
uint64_t tid;
functions->ompd_get_task_id(
ith, &tid);
sout << "0x" << hex << ith << " (" << tid << "), ";
#endif
sout << "0x" << hex << ith << " (" << "DISABLED IN ompd-devices" << "), ";
functions->ompd_release_task_handle(ith);
}
sout << endl;
Expand All @@ -587,7 +625,7 @@ void OMPDTest::execute() const
}
sout << endl;
pthread_t osthread;
functions->ompd_get_osthread(thr_h, ompd_osthread_pthread, sizeof(pthread_t), &osthread);
functions->ompd_get_thread_id(thr_h, ompd_thread_id_pthread, sizeof(pthread_t), &osthread);
host_contextPool->getThreadContext(&osthread)->setThisGdbContext();
odbCheckParallelIDs(functions, parallel_h);
odbCheckTaskIDs(functions, task_h);
Expand Down
Loading