[Question]: node_content is serialized within metadata and hence stored in the database

[kaniosm]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Hi,

I noticed an additional attribute in the metadata column (node_content), being generated and store together with the metadata.
From what I understand this attribute is used during search for simplicity purposes.
Could this be moved outside the metadata column to avoid storing duplicate data.
We are updating the metadata from our ORM framework, which requires updating the embedded object as well, can the update happen from the store directly without having to delete and regenerate the embedding?

[dosubot]

🤖

Hey @kaniosm! Great to see you diving into the details of LlamaIndex again. How's everything going?

Based on the current design of LlamaIndex, the node_content attribute is stored within the metadata column for each node. This design choice is likely due to the need for efficient search operations, as you've correctly pointed out.

Moving the node_content attribute outside of the metadata column would require significant changes to the underlying data model and could potentially impact the performance of search operations. This is because the node_content attribute is used in the _postprocess_nodes method of the MetadataReplacementPostProcessor class, where it is either replaced with the value from the metadata or remains the same.

As for updating the metadata from your ORM framework, the current design requires updating the embedded object as well. This is because the node_content attribute is tightly coupled with the metadata of each node. Direct updates to the store without regenerating the embedding might lead to inconsistencies between the node_content attribute and the metadata.

However, it's worth noting that the LlamaIndex is designed to be flexible and extensible. If you find that the current design does not meet your specific needs, you could consider extending the MetadataReplacementPostProcessor class or creating a new postprocessor class that better suits your requirements. This would allow you to control how the node_content attribute is generated and stored, and could potentially allow for direct updates to the store without regenerating the embedding.

Please refer to the LlamaIndex documentation and the source code of the MetadataReplacementPostProcessor class for more information on how to extend or customize the postprocessor classes.

I hope this helps! If you have any further questions, feel free to ask.

Dosu

Sources

• llama-index-legacy/llama_index/legacy/postprocessor/metadata_replacement.py
• llama-index-core/llama_index/core/node_parser/file/html.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[logan-markewich]

What vector store are you using? The _node_content field shouldn't be duplicating data (hopefully), most vector stores remove the text from this field if the text is already stored in another field

[kaniosm]

Hi Logan,

Thanks for the prompt reply!
We are using PgVector, and in our case the node_content appears in the metadata column in our PG table:

[logan-markewich]

Correct! So looking at the soruce code, there shouldn't be any redundant text

node_content is a serialization of the entire node, but the text content is removed before serializing

[kaniosm]

From what I understand in the source code, the remove_text flag is used to remove the text field from metadata and not the _node_content.

Just to add some info around this... initially we tried to change the top level attributes in the metadata_ column in PG, which had no effect in our app. We then realized that the same attribute under the node_content attribute in metadata is actually the one considered during retrieval. The top level attributes exists just to hold the data the first time the document is embedded, and those attributes the _node_content gets generated (jsondumps(node_dict)). i.e. node_to_metadata_dict is only called when addind notes.

[dosubot]

Hi, @kaniosm,

I'm helping the LlamaIndex team manage their backlog and am marking this issue as stale. The issue involves the storage of the "node_content" attribute within the metadata column, with a suggestion to move it outside to avoid storing duplicate data. There have been discussions with myself and @logan-markewich about the current design and potential ways to customize the postprocessor classes. As of now, the issue remains unresolved.

Could you please confirm if this issue is still relevant to the latest version of the LlamaIndex repository? If it is, kindly let the LlamaIndex team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation. If you have any further questions or updates, feel free to reach out.

[ramarnat]

The issue we are seeing with node content being added to the metadata is with relationships. We have a reader that reads a Document, and then a parser that creates the nodes from the document, adding the source node from the reader into the the node.relationships as SOURCE. The issue with that is that when the metadata gets sent to the vector store, it is including the full content of the original source node including the text of that source node.

[ramarnat]

Hmm, maybe we need to create RelatedNodeInfo instead of the actual source Node instead?

[ramarnat]

Even with RelatedNodeInfo, the node_content can get large if there are many related nodes (parent-child) for example.

Do related nodes need to be fully encapsulated in node content? Could rated nodes just be a reference with the node id and lazily retrieved?

The other option is to have some sort of EXCLUDE keys in the as_related_node method or reference only setting that only returns the id. I don't know what sort of downstream need is required of related nodes' metadata.

[mlevtov]

We are seeing the same issue.
We are using the SentenceWindowNodeParser with the MetadataReplacementPostProcessor. The _node_content field of each node contains the window and original_text of the nodes (which is a duplicate of those nodes fields), but also the window and original_text of the related nodes. Is the data from the related nodes used?

The problem is that the _node_content field ends up being about 10,000-20,000 characters for each node in our case, massively increasing our vector store resource requirement and cost.

Are there any workarounds to reduce this?