implement ipfs files command #1769

whyrusleeping · 2015-09-30T04:33:12Z

@jbenet

@diasdavid @mappum Heres the mfs api, Its gonna make things pretttty cooooool.

License: MIT
Signed-off-by: Jeromy jeromyj@gmail.com

License: MIT Signed-off-by: Jeromy <jeromyj@gmail.com>

whyrusleeping · 2015-09-30T07:05:08Z

terrible luck with tests... lets try that again.

jbenet · 2015-09-30T07:24:52Z

pretttty cooooool.
-- @diasdavid

jbenet · 2015-09-30T07:27:19Z

what looks better?

ipfs unix read
ipfs unix write
ipfs unix mv
ipfs unix cp
ipfs unix ls
ipfs unix mkdir
ipfs unix stat
ipfs unix rm

or

ipfs files read
ipfs files write
ipfs files mv
ipfs files cp
ipfs files ls
ipfs files mkdir
ipfs files stat
ipfs files rm

jbenet · 2015-09-30T07:28:34Z

is vendoring broken? the circleci tests look odd

https://circleci.com/gh/ipfs/go-ipfs/1391

jbenet · 2015-09-30T07:31:38Z

core/commands/files/files.go

+		}
+
+		res.SetOutput(&Object{
+			Hash: k.B58String(),


probably want more here, maybe:

size

cumulative size

of files (if dir)

jbenet · 2015-09-30T07:42:10Z

is vendoring broken? the circleci tests look odd

https://circleci.com/gh/ipfs/go-ipfs/1391

jbenet · 2015-09-30T07:42:15Z

core/commands/files/files.go

+			return
+		default:
+			res.SetError(errors.New("unrecognized type"), cmds.ErrNormal)
+		}


i'd put this (the meat) ls into the mfs module (or a subpkg like mfs/cmds or something. i like how @rht has been moving add and cat into core. these Commands should really just be glue between the commands library, and the functionality. (i.e. so that users who import IPFS as a library could use them)

this is just glue though, its converting the output from the mfs libs into the output we use in the cmds lib.

it's doing something meaningful, like selecting what to do depending on the Directory or FIle, etc. this is the meat of ls, even though ls is small. and we should be able to call ls from importing Go code.

Yup, agreed with @jbenet. This is the kind of walk you usually don't want to take again and again.

you mean walkFn and walk (for ls -R)

rht · 2015-09-30T23:25:36Z

core/core.go

+
+	switch {
+	case err == ds.ErrNotFound || val == nil:
+		nd = &merkledag.Node{Data: unixfs.FolderPBData()}


Should this be uio.NewEmptyDirectory()? as in https://github.com/ipfs/go-ipfs/blob/master/core/corehttp/gateway_handler.go#L277

oh, yeah. that works too.

whyrusleeping · 2015-10-01T00:15:33Z

pretty sure circleCI cant handle PRs that arent based on master.

cryptix · 2015-10-01T01:35:00Z

core/commands/files/files.go

+
+			for _, o := range out.Entries {
+				if long {
+					fmt.Fprintf(buf, "%s\t%s\t%d\n", o.Name, o.Hash, o.Size)


You might want to try tabwriter next time :)

rht · 2015-10-01T02:21:02Z

plan9

mv file1 file2
mv file ... directory

http://man.cat-v.org/plan_9/1/cp

rht · 2015-10-01T02:35:05Z

core/commands/files/files.go

+			return
+		}
+
+		res.SetOutput(&Object{


ipfs object stat also has LinksSize, and if Blocks should be NumLinks

Except for the hash, why not reuse Stat() as in ipfs object stat? And the marshalers as well.

jbenet · 2015-10-02T21:19:38Z

good tests! write fails need a bit of improvement:

on failures: check things haven't actually changed

also:

some silly whitespace things
path len(0) check

this is looking pretty good to me! 👍 excited to land this

License: MIT Signed-off-by: Jeromy <jeromyj@gmail.com>

whyrusleeping · 2015-10-03T01:13:52Z

@jbenet Alright, those are some tests!

whyrusleeping · 2015-10-04T02:44:42Z

@jbenet thoughts here?

jbenet · 2015-10-04T06:52:33Z

@whyrusleeping investigating why the circle-ci tests are failing.

for one, trying to run make vendor, need to get ipfs/go-log:

> go get github.com/ipfs/go-log
# github.com/ipfs/go-log
./oldlog.go:20: unknown logrus.TextFormatter field 'TimestampFormat' in struct literal
./oldlog.go:21: unknown logrus.TextFormatter field 'TimestampFormat' in struct literal

~~forget to push or something?~~

nvm. outdated logrus. fixed.

implement ipfs files command

jbenet · 2015-10-04T08:26:54Z

mfs (`ipfs files` for now) has shipped!

solid work from @whyrusleeping to land this.

remaining questions:

this is still an experimental command that might change a bit
play with the UI/UX and let us know how it is.
let us know what you prefer: ipfs files or ipfs unix.
- ipfs unix may be more accurate/less misleading even if a bit more confusing at first
- ipfs mfs -- could use this for merkledag-fs or something. (but mfs may be reserved for the "with mounts and hooks" one)

jbenet · 2015-10-04T08:27:17Z

(this is in 0.8.3-dev, 0.8.3 will be out soon)

daviddias · 2015-10-04T21:01:25Z

so rad! Thank you for putting all of this together so quickly @whyrusleeping !! :D

whyrusleeping · 2015-10-05T17:06:09Z

(this is in 0.8.3-dev, 0.8.3 will be out soon)

I actually moved it forward from the distant future of 0.8.3 to the nearer future of 0.4.0

jbenet · 2015-10-05T17:59:45Z

Oh sorry, whoops!

daviddias · 2015-10-11T00:26:47Z

core/commands/files/files.go

+
+var FilesCmd = &cmds.Command{
+	Helptext: cmds.HelpText{
+		Tagline: "Manipulate unixfs files",


@whyrusleeping will we use unixfs, files and mfs interchangeably? (looking also at the remaining help descriptions for the commands)

good question. i'm really not sure what the cleanest thing to do is. Posix files + dirs we use today across all OSes are unix derived, hence unixfs.

i think this should not be named mfs, but {unix, unixfs, files}. (mfs will be confusing to people unless we really push it and use that name everywhere. i think the other thing -- with mounts -- is likely better target to keep for the mfs name anyway. let's just say ipfs unix or ipfs files)

jbenet added the status/in-progress In progress label Sep 30, 2015

whyrusleeping force-pushed the mfs-api-redux branch 2 times, most recently from 48bb130 to ea69975 Compare September 30, 2015 05:42

implement ipfs files command

de41a9f

License: MIT Signed-off-by: Jeromy <jeromyj@gmail.com>

whyrusleeping force-pushed the mfs-api-redux branch from ea69975 to de41a9f Compare September 30, 2015 07:04

jbenet reviewed Sep 30, 2015
View reviewed changes

ghost mentioned this pull request Sep 30, 2015

Sprint Sep 28 ipfs/team-mgmt#35

Closed

88 tasks

rht reviewed Sep 30, 2015
View reviewed changes

whyrusleeping force-pushed the mfs-api-redux branch from f46286e to 0dbc807 Compare October 1, 2015 00:14

cryptix reviewed Oct 1, 2015
View reviewed changes

rht reviewed Oct 1, 2015
View reviewed changes

whyrusleeping force-pushed the mfs-api-redux branch from 0dbc807 to 67b15a7 Compare October 1, 2015 05:33

address comments from CR

d69891f

License: MIT Signed-off-by: Jeromy <jeromyj@gmail.com>

whyrusleeping force-pushed the mfs-api-redux branch from 67b15a7 to d69891f Compare October 2, 2015 22:28

jbenet added a commit that referenced this pull request Oct 4, 2015

Merge pull request #1769 from ipfs/mfs-api-redux

b2695b0

implement ipfs files command

jbenet merged commit b2695b0 into dev0.4.0 Oct 4, 2015

jbenet removed the status/in-progress In progress label Oct 4, 2015

jbenet deleted the mfs-api-redux branch October 4, 2015 08:16

daviddias reviewed Oct 11, 2015
View reviewed changes

jbenet mentioned this pull request Jul 29, 2016

Unite the Files API 🗂 ipfs/specs#98

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement ipfs files command #1769

implement ipfs files command #1769

whyrusleeping commented Sep 30, 2015

whyrusleeping commented Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet Sep 30, 2015

whyrusleeping Sep 30, 2015

jbenet Sep 30, 2015

cryptix Oct 1, 2015

rht Oct 1, 2015

rht Sep 30, 2015

whyrusleeping Oct 1, 2015

whyrusleeping commented Oct 1, 2015

cryptix Oct 1, 2015

rht commented Oct 1, 2015

rht Oct 1, 2015

rht Oct 1, 2015

jbenet commented Oct 2, 2015

whyrusleeping commented Oct 3, 2015

whyrusleeping commented Oct 4, 2015

jbenet commented Oct 4, 2015

jbenet commented Oct 4, 2015

jbenet commented Oct 4, 2015

daviddias commented Oct 4, 2015

whyrusleeping commented Oct 5, 2015

jbenet commented Oct 5, 2015

daviddias Oct 11, 2015

jbenet Oct 11, 2015

implement ipfs files command #1769

implement ipfs files command #1769

Conversation

whyrusleeping commented Sep 30, 2015

whyrusleeping commented Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet commented Sep 30, 2015

jbenet commented Sep 30, 2015

Choose a reason for hiding this comment

of files (if dir)

jbenet commented Sep 30, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whyrusleeping commented Oct 1, 2015

Choose a reason for hiding this comment

rht commented Oct 1, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbenet commented Oct 2, 2015

whyrusleeping commented Oct 3, 2015

whyrusleeping commented Oct 4, 2015

jbenet commented Oct 4, 2015

jbenet commented Oct 4, 2015

mfs (ipfs files for now) has shipped!

remaining questions:

jbenet commented Oct 4, 2015

daviddias commented Oct 4, 2015

whyrusleeping commented Oct 5, 2015

jbenet commented Oct 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfs (`ipfs files` for now) has shipped!