🚣 captain.log - IPFS JavaScript implementation 🌟

[daviddias]

Aye mates!

This is your captain speaking, the IPFS Node.js ship as been to set sail and good winds are blowing! Thank you for all the messages and enthusiasm shared so far, we truly appreciate your support and since some of you are eager to get their hands on code and help with the development, I'll keep this issue updated every week with what has been going on and where you can help the most. To make sure you don't miss any updates, click the 'watch' button on the repo.

LOG - Week of 21st of September

Plan:

• Finish Swarm revisit
  • https://github.com/diasdavid/node-ipfs-swarm/issues/8
  • https://github.com/diasdavid/node-ipfs-swarm/tree/revisit
• Implement node-libp2p bindings for every other module that is part of libp2p and have a alpha release people can experiment with
  • https://github.com/diasdavid/node-libp2p

Where you can help during this week:

• If crypto is your thing, take a look at
  • "Elliptic curve crypto in IPNS" daviddias/webcrypto#1
  • Node.js crypto and the WebCrypto API nodejs/node#2833
• If dream with tests and code coverage, there is a bunch of repos that could use some love
  • https://github.com/ipfs/node-ipfs#roadmap

[daviddias]

Aye! Check my new entry, arr :)

LOG - Week of 28th of September

Plan:

• Have a interoperable version of libp2p in Go, to that the primitives and coverage become stronger so that then the remaining parts of IPFS can implemented with confidence (cli, bitswap, chunker, etc)

Where you can help during this week:

• Making SPDY become a duplex stream - be a duplex stream spdy-http2/spdy-transport#14
• MOAR transports. Create other transports that swarm can use and tests to see them working at the same time
  • libp2p-udp
  • libp2p-udt
  • libp2p-utp
• If dream with tests and code coverage, there is a bunch of repos that could use some love
  • https://github.com/ipfs/node-ipfs#roadmap

[daviddias]

LOG - Week of December 21

Update:

• We have now IPFS Repo implemented in JS https://github.com/ipfs/js-ipfs-repo
• Bitswap work has started
  • https://github.com/diasdavid/js-ipfs-bitswap
  • WIP: exchange spec specs#53

Next:

• Make a IPLD Data Importing spec IPLD Data Importing - Set of Importers #41 (comment)
• jsipfs init - init #42
• jsipfs daemon

Notes:

• For a specific libp2p log, check ⛵️ Captain.log - libp2p JavaScript Implementation 🌟 libp2p/js-libp2p#6

[daviddias]

LOG - 26 of January

Update:

• You can now do npm i -g jsipfs and use the already implemented commands, full list on project status https://github.com/ipfs/js-ipfs#project-status
• We have an MerkleDAG implementation that interops with go-ipfs MerkleDAG implementation (this means you can write and read blocks from the same repo and will work for both implementations)
• We have currently a WIP on browser testing, with only one obstacle from storing properly binary data in the browser
• Lot's of good stuff

Next:

• files add do the whole file/dir -> MerkleDAG thing, following the Data Importing spec DEX specs#57

Where you can help?

In lots of places, check https://github.com/ipfs/js-ipfs/issues for descriptions, there is a huge amount of things that were unblocked once IPFS Repo and MerkleDAG were implemented and that can be done in parallel :) Once we get this all done, we will focus again on libp2p to get the networking stack ready for prime time!

Thank you!

[daviddias]

LOG - February 23 🚣

We've been pretty busy working on the JavaScript implementation of IPFS. There are more things that you can use now and all of them interop with the Go implementation. Before starting with the formal update and next steps, I just wanted give a shout out to everyone that has been submitting patches or contributing with features to js-ipfs lately. Special thanks goes to:

• @vijayee
• @dignifiedquire
• @xicombd
• @nginnever

Update

• IPFS MerkleDAG (DAGNode + DAGLink) and DAGService https://github.com/vijayee/ipfs-merkle-dag
• IPFS Blocks (block data structure and Block Service) https://github.com/ipfs/js-ipfs-blocks
• IPFS unixfs https://github.com/ipfs/js-ipfs-unixfs
• IPFS Architecture Overview https://github.com/ipfs/js-ipfs#ipfs-core-implementation-architecture
• Import files and directories into IPFS by converting them to their MerkleDAG representations with ipfs-data-importing https://github.com/ipfs/js-ipfs-data-importing
• object API is feature complete
• all js-ipfs core code is tested and it behaves correctly in the browser
• js-peer-id and js-peer-info now use node-forge library to be able to run on the browser and generate the public key from a private key. (before it was built to know how to generate key pairs, but since go-ipfs stores only the private key, we had to adapt the js code)
• jsipfs daemon
• complete update on all the feature threads on the js-ipfs repo, adding more information and clear action items https://github.com/ipfs/js-ipfs/issues
• level up js-ipfs tests

Extra - @RichardLitt has been putting an incredible amount of work in documenting the HTTP-API of IPFS, giving us a clear path on how to implement those routes correctly. You can find this endeavour in the http-api-spec repo

Goals

Our short-term goals for js-ipfs is to have complete feature parity with go-ipfs, with the exception of the DHT (now known as Peer Routing and Record Store). What this means is that if this goal is obtained, we will be able to:

• swarm listen and connect between js-ipfs and go-ipfs nodes
• use bitswap
• run IPFS in the browser

This will mark a very significant milestone on the track of the js-ipfs development, enabling other developers to start building apps and services using the JavaScript libraries.

Peer Routing and Record Store are the stretch goals, since it would require go-ipfs to learn about IPLD and IPRS.

Good foundations

We've invested a significant portion of time in making tests from the beginning of the development of this project. Tests makes the codebase less brittle and help contributors feel confident that their patches and features don't break any other feature.

js-ipfs is divided at the macro level into 'ipfs-core', 'http-api' and 'cli'. It all starts with core: this is where all the functionality that IPFS is composed of is exposed through a standardized interface (WIP). Each feature that gets implemented must be accompanied by tests and these must run in Node.js and the Browser. In addition, each time one of these features manipulates blobs of data that exit the memory of the program (e.g. when they are written to disk), they must be validated and compared to the ones that are produced by go-ipfs, ensuring interoperability and parity (example)

Then, the http-api is built, which uses core and exposes its features through a RPC like HTTP API, specified by the http-api-spec. Tests are built by testing the http-api directly and by using the js-ipfs-api, a module that is up do date with go-ipfs implemented http-api.

Finally, the CLI should come. This should be tested by running commands while a daemon is both online and offline.

Let's look at an example, the jsipfs id feature

• core implementation https://github.com/ipfs/js-ipfs/blob/master/src/ipfs-core/index.js#L44-L66
• core tests https://github.com/ipfs/js-ipfs/blob/master/tests/test-core/test-id.js
• http api implementation https://github.com/ipfs/js-ipfs/blob/master/src/http-api/routes/id.js and https://github.com/ipfs/js-ipfs/blob/master/src/http-api/resources/id.js
• http-api tests https://github.com/ipfs/js-ipfs/blob/master/tests/test-http-api/test-id.js#L7-L24
• http-api tests with js-ipfs-api https://github.com/ipfs/js-ipfs/blob/master/tests/test-http-api/test-id.js#L28-L44
• cli implementation https://github.com/ipfs/js-ipfs/blob/master/src/cli/commands/id.js
• cli tests with js-ipfs daemon not running https://github.com/ipfs/js-ipfs/blob/master/tests/test-cli/test-id.js#L8-L34
• cli tests with js-ipfs daemon running https://github.com/ipfs/js-ipfs/blob/master/tests/test-cli/test-id.js#L36-L76

Where can you help

• By completing one of the action items available on the 'help wanted' section https://github.com/ipfs/js-ipfs/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22
• Data Importing "export" feature https://github.com/ipfs/js-ipfs-data-importing/blob/master/src/index.js#L174-L176
• Improve code coverage - Help us reach 100% across every JavaScript repo!
• Review the last http-ipfs-api spec PR https://github.com/ipfs/http-api-spec/pulls

[daviddias]

LOG - April 5 📖

We’re close!™. That is right, we are indeed pretty close to get a workable JavaScript implementation of IPFS that can be ran on the browser. I want to thank first, everyone that has been actively contributing to js-ipfs, with features, tests, documentation and experimentation! This participation is key to get the js-ipfs implementation out the door as soon as possible.

Update

We are now able to use Web Sockets with SPDY stream muxing, giving us the change to use js-ipfs on the browser and exchange blocks with any IPFS Node that is available through a public IP. Big ^5 to @dignifiedquire who fixed pako, the JavaScript implementation of zlib, the last mile to make this work completely.

@ngginever has been leading the charge on the files api front, jsipfs files get is almost in a ready state to be merged, powered by pifs-data-importing pipeline.

@noffle finished jsipfs init now works and made it fully compatible with go-ipfs. While at it, some documentation for it was also created as a bonus.

Overall js-ipfs keeps getting more and more complete. @xicombd lead most of the charges on getting the http-api endpoints and cli commands implemented, using the same IPFS core codebase.

We’ve updated our ROADMAP to make it even more clear where we are and where he are headed.

Goal

We’ve set up a goal of getting js-ipfs milestone 1 ready by the 25th of April. It is a ambitious and there might be some hiccups that delay this goal, but we feel confident as we are pretty close to be fulfil all the goals. Nevertheless, hitting milestone 1 is not just about getting something that works, it is about getting something that works and that people feel confident to use, that being said, one of the things we want to invest time in is on having good documentation of all the pieces that compose js-ipfs and test coverage.

Where you can help

There are a lot of things where everyone can help, I’ve opened a issue for each of them with a task list, some focused on ‘features’, others focused in ‘modules’.

Here is the list of issues with what we are looking for to ship before milestone 1:

• track: Files API + MFS #60
• js-libp2p-ipfs-browser #126
• example: Use js-ipfs to load the entire assets of a WebSite/WebApplication #127
• Video player with js-ipfs #128
• track: jsipfs swarm #92
• util module for repo setup/teardown #95
• Shared Core API (standardise across js-ipfs and js-ipfs-api) #110
• js-ipfs-api #111
• js-ipfs-unixfs-engine #112
• js-ipfs-unixfs #113
• js-ipfs-repo #114
• js-ipfs-blocks #115
• js-peer-info #116
• js-peer-id #117
• js-multihash #118
• js-multihashing #119
• js-multiaddr #120
• js-multistream #121
• js-libp2p-swarm #122
• js-libp2p-tcp #123
• js-libp2p-spdy #124
• js-libp2p-ipfs #125

You will always be able to find in this repo, a collection of issues tagged with help wanted. This tag means that there is an action item that is currently not blocked by any other endeavour.