ipfs-core: dag.get does not return nested CID data

[stephhuynh18]

Version: ipfs-core 0.12.2

Severity:

High - The main functionality of the application does not work, API breakage, repo format breakage, etc.

Description:

We are currently upgrading to the latest versions of ipfs-related packages. We found that ipfs-core may have an issue with dag.get.

If we put JSON-like data on an IPFS node using dag.put, and this data contains a key that corresponds to a valid CID (nestedCID), when we get that data using dag.get and a path to that key, the nestedCID is returned instead of the data contained at the nestedCID.

  const regularContent = { test: '123' };
  const cid1 = await ipfs.dag.put(regularContent);
  const linkedContent = { link: cid1 };
  const cid2 = await ipfs.dag.put(linkedContent);

  const atPath = await ipfs.dag.get(cid2, { path: '/link' });
  // expected: { test: '123' }
  // received: CID(xxx)

This is not consistent with ipfs-core v0.7, where the data stored at the nestedCID is returned.

Is returning the nestedCID the desired behavior?

I also noticed that the nestedCID data is retrieved here:

js-ipfs/packages/ipfs-core/src/utils.js

Line 248 in 6178708

value = await load(value)

. Nothing is done with the data as yield is not called with this data. If yield is called with this data, the data is returned.

Steps to reproduce the error:

This issue is demonstrated here: https://github.com/stephhuynh18/ipfs-dag-get-issue

CC @ukstv

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[lidel]

This needs to be revisited after #3917

[achingbrain]

Looks like this behaviour changed with #3556 - the migration from ipld to multiformats.

The difference is the old ipld codecs used to implement traversal, the new versions do not.

At the IPFS level traversal is implemented by stepping through the properties of deserialized objects (with dag-pb treated as a special case since it's what UnixFS is based on) - this can be unreliable as the code may be dealing with a codec that doesn't deserialize to an object or implements paths differently than plain object properties (like dag-pb, for example).

I think in this case if there's a value we've loaded that hasn't been yielded, it's reasonable to yield it after leaving the while loop, so this is probably a bug.