Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 33 additions & 11 deletions doc/tutorial.txt
Original file line number Diff line number Diff line change
Expand Up @@ -285,16 +285,16 @@ scheduler point of view.
*** NUMA memory binding

* nodes_membind: Array of Integer. Define the NUMA binding of the thread.
Default value is all NUMA nodes of the system.
Default value is all NUMA nodes of the system.
An example : "nodes_membind" : [0, 2, 3]

"nodes_membind" Can be specified at task level or phase level.
"nodes_membind" Can be specified at task level or phase level.
As an example, below creates two threads.
One thread will run with memory binding to nodes 2 and 3.
The second task will run first phase with memory binding to nodes 0 and 1,
One thread will run with memory binding to nodes 2 and 3.
The second task will run first phase with memory binding to nodes 0 and 1,
second phase with memory binding to node 2.
Please note, that we follow an "event based policy", which means that
rt-app changes memory binding when there is a "nodes_membind" event
Please note, that we follow an "event based policy", which means that
rt-app changes memory binding when there is a "nodes_membind" event
and don't do anything otherwise.

"tasks" : {
Expand Down Expand Up @@ -339,8 +339,32 @@ unsigned int max_nbr_tgs = 32' at compile time.

*** events ***

events are simple action that will be performed by the thread or on the
thread. They have to be listed by execution order.
events are simple action that will be performed by the phase or on the
thread.

Two formats are supported:
* Either specified directly in the thread or phase body, in execution order:

"run": 1000,
"timer": {"ref": "helloworld", "period": 16000}

* As objects inside an array stored under the "events" key:

"events": [
{"run": 1000},
{"timer": {"ref": "helloworld", "period": 16000}},
{"run": 9000},
{"sleep": 1000}
]

Each object in the array must contain a single event key. This format allows
specifying the same event more than once (like "run" in the example), and is
friendlier to languages with map types that are not preserving insertion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also the case with current mode if you read the beg of the file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean via workgen or the way rt-app handles things currently ?

order.

Both formats cannot be mixed together.

Available events:

* run : Integer. Emulate the execution of a load. The duration is defined in
usec but the run event will effectively run a number of time a loop that waste
Expand Down Expand Up @@ -650,7 +674,7 @@ The generated events are of these main categories:

- rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=0
- rtapp_loop: event=end thread_loop=0 phase=0 phase_loop=0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't mix typo and new feature in the same patch or PR

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah looks like my text editor stripped the lines, sorry about that

Reporting the start and end of workload's phases and loops.

* Workload's events, for example:
Expand Down Expand Up @@ -756,5 +780,3 @@ below)
20000 ++--+----+---+----+---+---+----+---+----+--++ 494000
17560556057560556057560656065606756065606756075607
Loop start time [msec]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


81 changes: 69 additions & 12 deletions src/rt-app_parse_config.c
Original file line number Diff line number Diff line change
Expand Up @@ -983,37 +983,94 @@ parse_task_phase_data(struct json_object *obj,
{
/* used in the foreach macro */
struct lh_entry *entry; char *key; struct json_object *val; int idx;
int i;
struct json_object *array_val;
size_t i;
struct json_object *events_array = NULL;
bool is_first_entry;

log_info(PFX "Parsing phase");

data->nbevents = 0;
data->events = NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need this ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data->events=NULL is probably an artifact from before I introduced events_array variable, it's indeed not necessary anymore

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please try to not move code unnecessarily


/* loop */
data->loop = get_int_value_from(obj, "loop", TRUE, 1);

/* Count number of events */
data->nbevents = 0;
foreach(obj, entry, key, val, idx) {
if (obj_is_event(key))
if(!strncmp("events", key, 6)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this code seems overly complicated and transforms a simple straight forward function in something really complex.; From 2 foreach, it now needs 5 foreach/for loops

events_obj = get_in_object(obj, "events", TRUE); will say you if there is an events array or not

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I was looking for a function like that in json-c but it turns out it's a helper in that file.

if(!json_object_is_type(val, json_type_array)) {
log_critical(PIN "\"events\" key must be an array");
exit(EXIT_INV_CONFIG);
}
events_array = val;
break;
}
}

foreach(obj, entry, key, val, idx) {
if (obj_is_event(key)) {
/* Check that we are not trying to mix old and new styles */
if (events_array) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if an events array is detected we should simply ignore old style

Copy link
Author

@douglas-raillard-arm douglas-raillard-arm Mar 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really an improvement though ? The foreach is still needed to count the number of events anyway and silently ignoring events is not really something users would thank us for.

EDIT: count the number of old-style event*

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but when we have found an array we can skip this counting entirely because we don't care about old style events

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that this code cannot be removed, if someone is using old-style, this foreach needs to be somewhere. Silently ignoring part of the configuration file in order to save one iteration over a map that has a few tens of keys at most does not sound like a good deal. Even more considered the fact that in well-formed JSON files, the number of keys is bounded by the number of settings, so the only ones "paying a performance cost" are people writing a conf file that is ambiguous and will result in an error. (bearing in mind, we are parsing a configuration file, ahead of time of execution, in C, with an order of magnitude of a few tens or hundreds of keys in extreme cases).

On top of that, this can allow in the future to reclaim some event names for some settings. This sounds a bit crazy given that they have "verb" names for the most part, but the current syntax is lenient: we only match the prefix. This means that something like "synchronous-io" key is currently impossible to add, but would be possible with the "events" array as long as we forbid events outside of it when in use.

log_critical(PIN "Event key %s cannot be used in conjunction with \"events\" key", key);
exit(EXIT_INV_CONFIG);
/* Count number of events */
} else {
data->nbevents++;
}
}
}

if (events_array)
/* Assumes that all item in the array will be valid. If an item
* does not yield an entry in data->events, we must ensure it won't
* be used by exiting
*/
data->nbevents = json_object_array_length(events_array);

if (data->nbevents == 0) {
log_critical(PIN "No events found. Task must have events or it's useless");
exit(EXIT_INV_CONFIG);

}

log_info(PIN "Found %d events", data->nbevents);

data->events = malloc(data->nbevents * sizeof(event_data_t));

/* Parse events */
i = 0;
foreach(obj, entry, key, val, idx) {
if (obj_is_event(key)) {
log_info(PIN "Parsing event %s", key);
parse_task_event_data(key, val, &data->events[i], tdata, opts);
i++;
if (events_array) {
for (i=0; i < data->nbevents; i++) {
array_val = json_object_array_get_idx(events_array, i);
is_first_entry = true;
foreach(array_val, entry, key, val, idx) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the purpose of this foreach ?
isn't all object inside the array an event

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The array items are themselves objects:

[{event1: param1}, {event2: param2}, ...]

The alternative would be to use sub-arrays like a tuple instead of objects, but the object approach is the standard way of doing things, as it is pretty clean in YAML:

events:
    - run: 42
    - sleep: 53

(and the sub-array solution does not save any typing anyway)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but that was already the case with the old style and obj_is_event() is there to check that those objects are events

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After checking parse_task_event_data() it looks like it's doing a validation on the event name and will exit if it's given a non-event so we can skip this check here.

We still need to check the number of keys though, as:

  • 0 keys would mean that the length of the array (and therefore the size of data->events) does not match the actual number of events

  • more than one key does not make sense (at least for now, maybe one day this could be used to attach some sort of metadata to the event)

I need to fix the code to check we are not in the 0 keys situation, currently it would trigger a UB with data->events[i] not being initialized.

if (!is_first_entry) {
log_critical(PIN "Encountered a second key %s in an \"events\" item", key);
exit(EXIT_INV_CONFIG);
} else {
is_first_entry = false;
}

if (obj_is_event(key)) {
log_info(PIN "Parsing event %s", key);
parse_task_event_data(key, val, &data->events[i], tdata, opts);
} else {
/* We must exit in this branch,
* otherwise we will have a non-initialized
* entry in data->events that would be an
* undefined behavior
*/
log_critical(PIN "Encountered non-event key %s in \"events\" array", key);
exit(EXIT_INV_CONFIG);
}
}

}
} else {
i = 0;
foreach(obj, entry, key, val, idx) {
if (obj_is_event(key)) {
log_info(PIN "Parsing event %s", key);
parse_task_event_data(key, val, &data->events[i], tdata, opts);
i++;
}
}
}
parse_cpuset_data(obj, &data->cpu_data);
Expand Down