Captain.log - IPLD v1 spec

[nicola]

31-6 Aug

Fellow explorers, I am jumping on this spaceship, taking control on the pilot cabin and claim myself captain. Thank you to all the people that are contributing and have contributed on IPLD, as well to everyone using and willing to use IPLD. In this repo we will "make IPLD great again" so that it can truly be bring interoperability, secure permanent links, authenticated data structure to the whole web (worldwide and interplanetary).

I will keep this issue updated weekly (or so), I will write about (1) what is happening, (2) where help is needed the most. Please subscribe to this issue, or watch the repo for updates.

What is happening

• We are refining the spec and we have a list of issues to be solved and we need to reach on agreements:
  • Spec refining: Relative paths in IPLD #1 : Relative paths: yes or not?
    • yes: (pro) circular relations!, (cons) new errors (parent does not exist!)
    • no: (pro) simple design!, (cons) no circular relations across objects
  • Spec refining: Properties in links #2 : Properties in links: can link have user properties beyond /?
    • yes: (pro) can have hidden properties, (cons) complex!
    • no: (pro) simple! can use this space for IPLD specific keywords, (cons) NOTHING!
  • Spec refining: Support of IPLD pointers as links #3 : Support links to have pointers that are hash + /path/to/whatever
  • Spec refining: Multiple links in the link object #5 : Can I specify multiple links with different hash functions in the link object?
    • yes: (pro) hash can break, links still exist (cons) need to detect if links' references are equal, complex typing
    • no: (pro) clean link object (cons) no survival through hash break
  • Spec refining: Do not bind the canonical representation to CBOR #6 : CBOR should be upgradeable
  • Spec refining: requiring namespace in IPLD pointers #7 : Namespace for IPLD
  • Spec refining: IPLD pointers must point to IPLD objects only #8 : Should IPLD object point to IPLD objects only?
  • Spec refining: Mutable links in IPLD #9 : How to handle mutable links
  • Spec refining: listing down the possible errors to be handled #11 : Errors to be handled
• New writing of the spec is happening: here is a draft New readme and rearranging specs #10
• IPLD moved into its own repo (however it is still in progress Moving IPLD into its own org ipfs/specs#128 - help needed by @dignifiedquire, @jbenet, @diasdavid in moving all the things)
• New proposal: CID and multicodec-packed Experimental Proposal: CIDv1 -- IPLD, Multicodec-packed, and more ipfs/specs#130

Where is the help needed

• We are having a call next monday to discuss the above IPLD spec refining call - round 1 ipfs/team-mgmt#124
• There is a need to agree on most of the issues above
• Provide examples of your use cases and needs

This is all for now!

Thank-yous

• Thank you to @dignifiedquire and @RichardLitt for going over some of the previous PRs and issues
• Thanks to @diasdavid and @jbenet for the updates on CID & co

cc @mildred, @Stebalien, @dignifiedquire

[nicola]

7-14 Aug

I will write this log in advance so that I don't miss pieces!

tl;dr

• Issues that need more discussion
• Issues that we already agreed upon

What happened (done)

• We did a call on Monday and here are the minutes
• Finalized agreement on:
  • 🎉 Spec refining: Do not bind the canonical representation to CBOR #6 : Spec refining: Do not bind the canonical representation to CBOR needs spec spec
  • 🎉 Spec refining: Multiple links in the link object #5 : Spec refining: Multiple links in the link object needs spec spec
  • 🎉 Spec refining: Support of IPLD pointers as links #3 : Spec refining: Support of IPLD pointers as links needs spec spec
  • 🎉 Spec refining: Properties in links #2 : Spec refining: Properties in links needs spec spec

What is happening (todo)

• Skeleton of IPLD v1 spec #12 : Spec: Getting DRAFT of v1 IPLD done (at least the skeleton)
• Spec refining: listing down the possible errors to be handled #11 : Spec: Listing down errors to be handled
• Spec refining: Terminology IPLD vs Merkle #15 : Spec: Terminology prefixing everything with IPLD or with Merkle (bike-shedding post)
• Create me website#1 : Website: Starting up IPLD.io

Need to find agreement from past week

• Spec refining: Relative paths in IPLD #1 : Relative paths: yes or not? (in new comments @nicola argues on why it is important)
• Spec refining: IPLD pointers must point to IPLD objects only #8 : Forcing IPLD to only point to IPLD paths (apparently it is a no, but need more agreement)
• Spec refining: requiring namespace in IPLD pointers #7 : Requiring prefixing (full CID and/or namespace) can this be implicit?
• Spec refining: Mutable links in IPLD #9 : Mutable links on IPLD (although we kind of agreed that we want them, but we need a distinguisher and this could be the namespace, needs more agreement)
• Experimental Proposal: CIDv1 -- IPLD, Multicodec-packed, and more ipfs/specs#130 : CID (we seems to agree that it solves a lot of problems)

Where is the help needed

• We still have pending issues over here: issues labelled spec
• Authenticated RDF (or RDF/JSON-ld on IPLD) ipfs/notes#152 : Bringing RDF on IPLD (or authenticating RDF)
• PubSub notes from the IPFS Workshop - recursion and corecursion, PubSub API and Self Certified Streams ipfs/notes#154 : PubSub & corecursive IPFS: how to listen to events, maybe related to IPLD

Thank-yous

• Thank you to @dignifiedquire for helping with the tooling so that we now have needs spec label if we agree and we just need to write the spec
• Thank you to @em-ly @jbenet @dignifiedquire @diasdavid @EdwardSilverton @flyingzumwalt @Stebalien for joining the call and participate to this week's activities

[nicola]

Aug 15-22

It has been a calm week. A lot of conversations. We are planning a new call soon!

What happened (done)

• merged Corrected typos for traverse example #21 wrong hash #22 thanks to @tjgillies and @EliasGabrielsson

What is happening (todo)

• Idea for permanent mutable links #19 : IPNS+snapshot links discussion
• Is this thin? #18 : Conversation whether IPLD is too big to be the thin layer
• see the unchecked previous todos for the work TODO!

What is happening from past weeks

• Skeleton of IPLD v1 spec #12 : Spec: Getting DRAFT of v1 IPLD done (at least the skeleton)
• Create me website#1 : Website: Starting up IPLD.io

Need to find agreement from past weeks

See previous post, but as a summary:

• Spec refining: Terminology IPLD vs Merkle #15 : Spec: Terminology prefixing everything with IPLD or with Merkle (bike-shedding post)
• Spec refining: listing down the possible errors to be handled #11 : Spec: Listing down errors to be handled
• Spec refining: Relative paths in IPLD #1 : Relative paths: yes or not? (in new comments @nicola argues on why it is important)
• Spec refining: IPLD pointers must point to IPLD objects only #8 : Forcing IPLD to only point to IPLD paths (apparently it is a no, but need more agreement)
• Spec refining: requiring namespace in IPLD pointers #7 : Requiring prefixing (full CID and/or namespace) can this be implicit?
• Spec refining: Mutable links in IPLD #9 : Mutable links on IPLD (although we kind of agreed that we want them, but we need a distinguisher and this could be the namespace, needs more agreement)
• Experimental Proposal: CIDv1 -- IPLD, Multicodec-packed, and more ipfs/specs#130 : CID (we seems to agree that it solves a lot of problems)

What is going on in the community

• 👪 IPLD preparations mediachain/mediachain#158 : IPLD and Mediachain!
• 👪 Nix and IPFS NixOS/nix#859 : IPLD/FS and NixOS

Where is the help needed

• We still have pending issues over here: issues labelled spec
• Authenticated RDF (or RDF/JSON-ld on IPLD) ipfs/notes#152 : Bringing RDF on IPLD (or authenticating RDF)
• PubSub notes from the IPFS Workshop - recursion and corecursion, PubSub API and Self Certified Streams ipfs/notes#154 : PubSub & corecursive IPFS: how to listen to events, maybe related to IPLD

Thank-yous

• @Stebalien, @jbenet, @tjgillies, @harlantwood, @Ericson2314

[jbenet]

(Awesome captain log! Cc @whyrusleeping @diasdavid)
On Mon, Aug 22, 2016 at 13:40 Nicola Greco notifications@github.com wrote:

15-22

Nothing much have happened (I have been traveling)
What happened (done)

• merged Corrected typos for traverse example #21 Corrected typos for traverse example #21 wrong hash #22
  wrong hash #22 thanks to @tjgillies
  https://github.com/tjgillies and @EliasGabrielsson
  https://github.com/EliasGabrielsson

What is happening (todo)

• Idea for permanent mutable links #19 Idea for permanent mutable links #19 : IPNS+snapshot links
  discussion
• Is this thin? #18 Is this thin? #18 : Conversation whether
  IPLD is too big to be the thin layer
• see the unchecked previous todos for the work TODO!

What is happening from past weeks

• Skeleton of IPLD v1 spec #12 Skeleton of IPLD v1 spec #12 : Spec: Getting DRAFT of
  v1 IPLD done (at least the skeleton)
• Create me website#1 Create me website#1 : Website:
  Starting up IPLD.io

Need to find agreement from past weeks

• Spec refining: Terminology IPLD vs Merkle #15 Spec refining: Terminology IPLD vs Merkle #15 : Spec: Terminology
  prefixing everything with IPLD or with Merkle (bike-shedding post)
• Spec refining: listing down the possible errors to be handled #11 Spec refining: listing down the possible errors to be handled #11 : Spec: Listing down
  errors to be handled
• Spec refining: Relative paths in IPLD #1 Spec refining: Relative paths in IPLD #1 : Relative paths: yes or
  not? (in new comments @nicola https://github.com/nicola argues on
  why it is important)
• Spec refining: IPLD pointers must point to IPLD objects only #8 Spec refining: IPLD pointers must point to IPLD objects only #8 : Forcing IPLD to only
  point to IPLD paths (apparently it is a no, but need more agreement)
• Spec refining: requiring namespace in IPLD pointers #7 Spec refining: requiring namespace in IPLD pointers #7 : Requiring prefixing
  (full CID and/or namespace) can this be implicit?
• Spec refining: Mutable links in IPLD #9 Spec refining: Mutable links in IPLD #9 : Mutable links on IPLD
  (although we kind of agreed that we want them, but we need a distinguisher
  and this could be the namespace, needs more agreement)
• Experimental Proposal: CIDv1 -- IPLD, Multicodec-packed, and more ipfs/specs#130 Experimental Proposal: CIDv1 -- IPLD, Multicodec-packed, and more ipfs/specs#130 : CID (we
  seems to agree that it solves a lot of problems)

Where is the help needed

• We still have pending issues over here: issues labelled spec
  https://github.com/ipld/specs/issues?q=is%3Aissue+is%3Aopen+label%3Aspec
• Authenticated RDF (or RDF/JSON-ld on IPLD) ipfs/notes#152 Authenticated RDF (or RDF/JSON-ld on IPLD) ipfs/notes#152 : Bringing
  RDF on IPLD (or authenticating RDF)
• PubSub notes from the IPFS Workshop - recursion and corecursion, PubSub API and Self Certified Streams ipfs/notes#154 PubSub notes from the IPFS Workshop - recursion and corecursion, PubSub API and Self Certified Streams ipfs/notes#154 : PubSub &
  corecursive IPFS: how to listen to events, maybe related to IPLD

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#13 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIcoRcZxjq_weRvZq4VOUC1cw5Zahfoks5qid8cgaJpZM4JdM3i
.

[nicola]

The captain is off the ship until the 5th September

[nicola]

5-11 Sept

Academic term has started, devs are going back to code, this is a great time to start with all the energy that we collected during the summer.

---

What happened (done)

Not much has happened I and other contributors have been out and silent

What is happening (todo)

• see the unchecked previous todos for the work TODO!
• 📞 IPLD refining call 2 ipfs/team-mgmt#181 : New call coming up

What is happening from past weeks

• Skeleton of IPLD v1 spec #12 : Spec: Getting DRAFT of v1 IPLD done (at least the skeleton)
• Create me website#1 : Website: Starting up IPLD.io

Need to find agreement from past weeks

See previous unchecked issues

What is going on in the community

• 👪 Implement databases over IPFS using Persistent Balanced Trees ipfs/notes#161 : Persistent balanced trees on IPFS

Where is the help needed

• We still have pending issues over here: issues labelled spec
• Authenticated RDF (or RDF/JSON-ld on IPLD) ipfs/notes#152 : Bringing RDF on IPLD (or authenticating RDF)
• PubSub notes from the IPFS Workshop - recursion and corecursion, PubSub API and Self Certified Streams ipfs/notes#154 : PubSub & corecursive IPFS: how to listen to events, maybe related to IPLD

---

Thank-yous

Thank you to no one in particular :)

[nicola]

11-18 Sept

A lot of conversations on relative links, how to have circular relations (on a data structure level and on an application level). This happened during the call ipfs/team-mgmt#181, (notes are here: ipfs/team-mgmt#196)

---

What is happening this week

• Skeleton of IPLD v1 spec #12 : (Finally) Writing the skeleton of the new IPLD v1 spec
• 📞 IPLD refining call 3 ipfs/team-mgmt#197 : There will be a call squeezed in the IPFS (going to happen after libp2p)

What happened (done)

• Is this thin? #18 : Is this thin?
• No more @link. #17 : No more @link.

To Agree

• Spec refining: Terminology IPLD vs Merkle #15 : Spec refining: Terminology IPLD vs Merkle
• Spec refining: listing down the possible errors to be handled #11 : Spec refining: listing down the possible errors to be handled
• Spec refining: Mutable links in IPLD #9 : Spec refining: Mutable links in IPLD
• Spec refining: IPLD pointers must point to IPLD objects only #8 : Spec refining: IPLD pointers must point to IPLD objects only
• Spec refining: requiring namespace in IPLD pointers #7 : Spec refining: requiring namespace in IPLD pointers

Agreed (needs spec writing)

• 🎉 Spec refining: Relative paths in IPLD #1 : Spec refining: Relative paths in IPLD !!!!
• 🎉 Spec refining: Merkle-paths to an array's index #20 : Spec refining: Merkle-paths to an array's index
• 🎉 Spec refining: Do not bind the canonical representation to CBOR #6 : Spec refining: Do not bind the canonical representation to CBOR
• 🎉 Spec refining: Multiple links in the link object #5 : Spec refining: Multiple links in the link object
• 🎉 Spec refining: Support of IPLD pointers as links #3 : Spec refining: Support of IPLD pointers as links
• 🎉 Spec refining: Properties in links #2 : Spec refining: Properties in links

Others (proposals, question, ..)

• Idea for permanent mutable links #19 : Idea for permanent mutable links

---

Thank-yous

• @Stebalien for writing the minutes of the previous call (and the work on cycles/relative paths/co)

[nicola]

19-02 Oct

Now there are weekly meetings on IPLD (see ipfs/pm) for the current sprint. A lot of progress on how IPLD code structure should look like and several conversations regarding relative paths (again!)

What is happening this week

• Call in which we need an update from @diasdavid and @jbenet on CID and IPLD interface & new 🚲 🏚 !@
• Going over a skeleton for the new spec

What happened (ongoing)

• Map the entire Ethereum State into IPFS with ipld #27 : Map the entire Ethereum State into IPFS with ipld
• Implementation: Make resolver into its own module #26 : Implementation: Make resolver into its own module
• Implementation: separate IPLD from CBOR #25 : Implementation: separate IPLD from CBOR
• Implementation: Tracking implementation effort for IPLD v1 #24 : Implementation: Tracking implementation effort for IPLD v1
• IPLD v2: Laying out Future plans #23 : IPLD v2: Laying out Future plans

What happened (done)

• Fix bracket mismatch in example. #28 : Fix bracket mismatch in example.

To Agree

• Spec refining: Terminology IPLD vs Merkle #15 : Spec refining: Terminology IPLD vs Merkle
• Spec refining: listing down the possible errors to be handled #11 : Spec refining: listing down the possible errors to be handled
• Spec refining: Mutable links in IPLD #9 : Spec refining: Mutable links in IPLD
• Spec refining: IPLD pointers must point to IPLD objects only #8 : Spec refining: IPLD pointers must point to IPLD objects only
• Spec refining: requiring namespace in IPLD pointers #7 : Spec refining: requiring namespace in IPLD pointers

Agreed (needs spec writing)

• 🎉 Spec refining: Merkle-paths to an array's index #20 : Spec refining: Merkle-paths to an array's index
• 🎉 Spec refining: Do not bind the canonical representation to CBOR #6 : Spec refining: Do not bind the canonical representation to CBOR
• 🎉 Spec refining: Multiple links in the link object #5 : Spec refining: Multiple links in the link object
• 🎉 Spec refining: Support of IPLD pointers as links #3 : Spec refining: Support of IPLD pointers as links
• 🎉 Spec refining: Properties in links #2 : Spec refining: Properties in links
• 🎉 Spec refining: Relative paths in IPLD #1 : Spec refining: Relative paths in IPLD

Others (proposals, question, ..)

• Idea for permanent mutable links #19 : Idea for permanent mutable links

Thanks you-s

@flyingzumwalt, @dignifiedquire, @RichardLitt

[daviddias]

@nicola update:

See ipld/js-ipld#60, we got the interface down :D

#27 : Map the entire Ethereum State into IPFS with ipld

It is not going to be mapped, it is going to be resolved natively, thanks to "IPLD formats" :)

Implementation: separate IPLD from CBOR

We got this with the creation of the interface-ipld-format and the IPLD Formats identified by multicodecs multiformats/multicodec#16