Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ds-query + ipfs refs local #531

Merged
merged 13 commits into from
Jan 12, 2015
Merged

ds-query + ipfs refs local #531

merged 13 commits into from
Jan 12, 2015

Conversation

jbenet
Copy link
Member

@jbenet jbenet commented Jan 10, 2015

This command introduces datastore.Query and
ipfs refs local, which is a commant that simply
lists all refs stored locally.

HUGE WARNING: this commit changes how blocks are stored,
making all past blocks unlikely to be read. I am considering
putting in place an "upgrade" script to make sure everyone's
operation continues soundly. We'll need things like this in
place later on.

@jbenet jbenet added the status/in-progress In progress label Jan 10, 2015
@jbenet
Copy link
Member Author

jbenet commented Jan 10, 2015

This makes #414/#383 possible.

@jbenet
Copy link
Member Author

jbenet commented Jan 10, 2015

RFCR, but NOT ready for merge.

@btc
Copy link
Contributor

btc commented Jan 10, 2015

Are resources reclaimed when clients don't read all results?

@jbenet
Copy link
Member Author

jbenet commented Jan 11, 2015

Major changes. Includes the new and improved datastore.Query, which responds to context cancellation (...that took a while).

Also, I made the commands library pass a context -- right now it's pretty ugly, but will get removed soon.

@whyrusleeping
Copy link
Member

<3 Ill CR

// * spawn another goroutine that waits on <-child.Closed()
//
// go func() {
// <-child.Closed()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably update this comment to reflect the new strategy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, yep will do

@jbenet
Copy link
Member Author

jbenet commented Jan 12, 2015

jenkins aborted the last test: https://build.protocol-dev.com/job/all-the-tests/489/ -- https://build.protocol-dev.com/job/network-test/673/console -- any idea why @briantigerchow ?

ipfs blocks local returns _all_ local refs. For now this is one
long op. future commits will make it async.
@jbenet
Copy link
Member Author

jbenet commented Jan 12, 2015

jenkins acting up again: https://build.protocol-dev.com/job/all-the-tests/496/ way too many commits there, it should only be f9ca67e...e244fff (as of this message)

@whyrusleeping
Copy link
Member

@jbenet ive seen that before, where jenkins has a TON of commits on its plate.. no idea why

Queries now can be cancelled and the resources collected
using Header().Set after WriteHeader() has no effect.
cc @mappum
cc @mappum can we do this for the copyChunks case?
the prefix should be as short as possible, as this is a
per-block overhead.
@jbenet
Copy link
Member Author

jbenet commented Jan 12, 2015

I'm going to merge this.

warning to users: datastore migration needed

After this commit, you wont be able to get the data, as the format changed a bit. If you'd like to make sure you dont get problems, and you can use the data, bug me here or on irc for a migration script. (happy to write one if someone needs it).

jbenet added a commit that referenced this pull request Jan 12, 2015
ds-query + ipfs refs local
@jbenet jbenet merged commit cfbe440 into master Jan 12, 2015
@jbenet jbenet removed the status/in-progress In progress label Jan 12, 2015
@jbenet jbenet deleted the ds-query branch January 12, 2015 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants