DWARF for WebAssembly Target

[yurydelendik]

I started documenting the findings that were done during my work on saving LLVM debug information as custom sections (see D44184 and D45118). The LLVM prefers DWARF format, and it is doable to package entire DWARF data into wasm custom section and convert it into wasm binaries source maps later. These findings can be found at https://github.com/yurydelendik/webassembly-dwarf/. Also, I attempted to match the findings with @fitzgen's WebAssembly Debugging Capabilities just for information purpose.

This issue is an attempt to open discussion about if it will be valuable to continue packaging debug data as custom sections in DWARF format.

[yurydelendik]

There is D52634 opened. Let's outline the short-term goal here. The long-term goal is being documented (I recently updated) in the finding document.

The debug information, produced by LLVM, is not capable to expose WebAssembly specific items in the DWARF expressions. I considered the idea of using registers operands to express WebAssembly locals, globals, an operand stack, etc. It complicates logic around the encoding/decoding of such items and it still do not provide any value -- the WebAssembly specific DWARF expressions need to be transformed or special-cased during an evaluation process.

So the short-term goal is to extend DWARF expression with WebAssembly specific items such as locations for locals, globals and stack operands. The DW_OP_WASM_location with code 0xED can be added to the DWARF expression language. This operation will have two operands.

The DW_OP_breg operations are used to express WebAssembly location. The first operand (register number) defines the type of the WebAssembly location. It is encoded as ULEB128 constant. The second operand (the offset) is a location item index and it is encoded as SLEB128 constant.

Type	Description	Second Parameter
1	Local or Argument	The index of a WebAssembly function local/parameter
2	Global	The index of a WebAssembly module global
3	Operand Stack Item	The depth of the item (0 - points the bottom of the stack)

The dump tools, such as llvm-dwarfdump, will be extended to display such expression in e.g. wasm-location(type, index) format. The existing debuggers (e.g. lldb or gdb) or other tools will not be able to decode such expressions, and that is okay since they cannot evaluate in the WebAssembly context anyway. The idea for the latter is to transform WebAssembly specific locations into native locations (registers, memory, etc) during AOT or JIT compilation.

Current TODOs:

• Introduce the DWARF expression format
• Add DBG_VALUE with local operands location D52634
• Use WebAssembly location in DW_AT_frame_base location (for shadow stack)

[yurydelendik]

Pre reviewers require, replacing DW_OP_WASM_location with DW_OP_breg operators. See changes above

[justinclift]

With that doc, should the date on the sub heading be updated from "21 May 2018"? It sounds like it hasn't been changed in ~6 months. 😄

[yurydelendik]

With that doc, should the date on the sub heading be updated from "21 May 2018"? It sounds like it hasn't been changed in ~6 months. 😄

The "WebAssembly Debugging Capabilities" doc is 6 months old, yes. The "DWARF for WebAssembly Target" at https://yurydelendik.github.io/webassembly-dwarf/ is last modified 13 December 2018.

[justinclift]

Thanks @yurydelendik. 😄

[AndrewScheidecker]

In this proposal, I'm using a "function label index space" to refer to labels from the binary name section.

I wrote a rationale for using function label indices over binary offsets that I'll paste here:

This proposal defines a "function label index" space in addition to the scoped label index space already defined in the spec. This function label index space assigns indices to the labels in a function in the order their corresponding structured control instruction occurs in the function body.

Other proposals that deal with references to instructions seem to use binary format offsets:

1. DWARF for WebAssembly
2. The display conventions for WebAssembly locations
3. Source maps applied to WebAssembly binaries

The reasons for using the function label index space over a binary offset in this proposal are:

1. Any in-memory representation of the WebAssembly abstract syntax must already be able to encode the order of the structured control instructions, and that order implicitly defines the function label index space. Using binary offsets would require an implementation to keep track of the binary offset a structured control instruction was read from.
2. The function label index space also densely maps integers to names, while binary offsets would sparsely map integers to names. That makes it more practical to use a simple data structure to store the names in memory.
3. Finally, a binary offset would mean that a far greater proportion of WebAssembly module transforms would need to also transform the binary offsets in the name section. The function label index space is not changed by any transform that doesn't change the structured control instructions.

I believe parts of this rationale also apply to the code addresses in DWARF for WebAssembly; you reference instructions through a binary offset relative to the code section. Have you considered using an index into a function's instruction sequence instead?

[yurydelendik]

Have you considered using an index into a function's instruction sequence instead?

Yes, I did. Function Index + instruction index creates a compound key and it is harder to maintain, as well as module wide instruction index. Since most of WebAssembly tooling adapted bytecode offset, there is no reason to invent something that will not be useful, e.g. the optimization tools change the instructions (order), so instructions indices will be changed thus giving no advantage over bytecode offset for processing debug information.

[AndrewScheidecker]

It's true that it will be harder for a transform to avoid the need to transform the DWARF sections, but there are some useful transforms that don't change the instruction sequence of a function:

1. Translating a binary module to a text module renders the binary offsets meaningless.
2. A "layer 1" WASM binary format would have a hard time preserving DWARF code offsets.
3. Linking WASM object files needs some special relocation types for binary offsets that AFAICT are only needed to relocate DWARF binary offsets. (see R_WASM_FUNCTION_OFFSET_I32 and R_WASM_SECTION_OFFSET_I32 in https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md).

[yurydelendik]

Translating a binary module to a text module renders the binary offsets meaningless.

Annotations needs to be introduced into text format, similar to LLVM's .ll approach. I don't think it will reasonable to preserve DWARF sections data as-is during round trip of text format without annotated instructions and data segments.

Linking WASM object files needs some special relocation types for binary offsets that AFAICT are only needed to relocate DWARF binary offsets. (see R_WASM_FUNCTION_OFFSET_I32 and R_WASM_SECTION_OFFSET_I32 in https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md).

The R_WASM_SECTION_OFFSET_I32 will be needed for DWARF sections regardless of bytecode/index key. R_WASM_FUNCTION_OFFSET_I32 will be replaced with something different to relocate compound "(function index, instruction index)" key reference.

[AndrewScheidecker]

Annotations needs to be introduced into text format, similar to LLVM's .ll approach. I don't think it will reasonable to preserve DWARF sections data as-is during round trip of text format without annotated instructions and data segments.

The reason why it's easier to round trip DWARF through the text format applies the same whether you're translating the binary DWARF data to some annotated text syntax or not.

The abstract syntax that ties your binary and text format together will need some way to represent code addresses that's independent of their binary serialization. If the serialized format uses binary offsets, then serializing those code addresses will be tightly coupled to serializing the code section. For example, when you serialize the code section, you'll need to produce a map between instruction indices and binary offsets that you can use when you serialize the DWARF sections.

compound "(function index, instruction index)" key reference

I was thinking that the function index could be implied by the context, but I can see the DWARF format occasionally uses "code addresses" outside of the context of a function. Maybe it's possible to interpret code addresses to be function indices in some contexts (e.g. to define the code in a compilation unit) and instruction indices in other contexts.

[yurydelendik]

The reason why it's easier to round trip DWARF through the text format applies the same whether you're translating the binary DWARF data to some annotated text syntax or not.

A round trip wasm->text->wasm without changing the text is very narrow/insignificant use case and shall not be used to make decision how the instructions must be identified in the DWARF format IMHO. In most of the (superset) cases the text will be changed, which requires changes in the DWARF data.

function index could be implied by the context,

The .debug_line (and .debug_frame) section has no "contexts" and requires function instructions to be uniquely identified.

[dschuff]

There seems to be general consensus that some flavor of DWARF is what we want for at least the LLVM-based family of language and there are now several interoperable implementations of this spec. Can we check something into this repo that describes the current prototype? Then it would be easier to open specific issues or PRs than to add more comments here. e.g. I want to talk some more about the topic above (section vs binary offsets vs some abstract index space), but it would be better to have separate topics.
Maybe an md doc would be best for now, for easy editing and since we don't know whether we eventually want bikeshed or what for the doc in the future.