Question: Can Unstructured track hierarchical header info, such as H1, H2, H3, etc.?

[aaronsteers]

In HTML and Markdown, we have a concept of nested and hierarchical header text, and I think it could be helpful to track for TitleText.

Does Unstructured currently have any tracking for?:

1. Identifying the hierarchical level of a title during parsing.
2. Storing this hierarchy level (H1, H2, etc.) in the metadata.
3. Exposing the hierarchy of relevant headers on sub-elements, such as on metadata of a NarrativeText element?

Totally understandable if this is out of scope or perhaps not relevant to mainstream use cases. For my part, I just wanted to better understand if this is something that Unstructured can do, or if not if there are existing plans to add something like this in the future.

Thanks!

[qued]

Hi @aaronsteers ! There is currently an effort to discover and expose hierarchy metadata for element types.

Currently though, we do store tag info in the tag and ancestortags properties of instances of the HTMLText class, which are the element types that come back from partitioning an HTML file. This also applies to certain other file types that get converted to HTML for processing, including markdown.

However, I don't believe this information is retained in the metadata when the element is converted to JSON, so if you're using the API this may not be helpful. This is something we could add, though.

[cragwolfe]

There are plans to support distinguishing between H1, H2, etc. This will be accomplished by the category_depth metadata field being introduced in #1308 .

E.g., H3 would be a Title element that has metadata.category_depth=3.

[newelh]

This feature was added in #1268! Let us know if you run into any issues

[aaronsteers]

@newelh - Fantastic! I'll take a look. 🙏