Add agent tracing support

[jodh-intel]

Now that kata-containers/agent#415 has landed, we need to add the ability to enable agent tracing from the runtime.

Static agent tracing can already be enabled by setting agent.trace* in kernel_params= in the configuration.toml file.

However, dynamic tracing (which is the preferred trace model) will require:

• A new config file option to enable agent tracing.
• Calls to the two new gRPC APIs: StartTracing() and StopTracing().

[jodh-intel]

Actually, the static tracing is only partially supported at the moment; to make this work, we need to disable the QMP quit call to allow the agent to shutdown the VM. This is why dynamic tracing is preferred since the runtime retains control of the VM whereas with static tracing, the agent is in control of the shutdown.

@sboeuf, @mcastelino - I'm thinking that rather than adding a simple enable_tracing=true here, we may want instead:

[agent.kata]
trace_type = [static|dynamic]
trace_subtype = [isolated|collated]

By default:

• trace_type would be set to dynamic but commented out
  (meaning "all tracing is disabled by default").
• trace_subtype would be set to isolated but commented out to disable it.
  The reason for this being that dynamic tracing only supports the isolated mode.

Related to that, iff the user sets trace_type = static, the following will happen:

• The runtime will add the appropriate guest kernel option itself.

  That would probably be cleaner in a sense [*].

• The runtime will not call QMP QUIT.

---

[*] - recall that any kernel options the user sets in kernel_params= take priority over those set by the runtime.

[sboeuf]

@jodh-intel this sounds good to me. I like the fact that the default will be the one we want (dynamic + isolated), but disabled because commented out.

One comment here, maybe we could call trace_type the trace_mode or tracing_mode, which would let us call trace_subtype the tracing_type or trace_type.
I think having a mode and a type is less confusing than a type and a subtype.

[jodh-intel]

I think having a mode and a type is less confusing than a type and a subtype.

Agreed. I'll update the agent trace docs for that first...

[jodh-intel]

This work is going to be blocked on #1152.

[jodh-intel]

It will also be blocked on kata-containers/agent#480.