Skip to content

Commit

Permalink
Merge pull request trapexit#883 from trapexit/link-symlink
Browse files Browse the repository at this point in the history 
new features: follow-symlinks, rename-exdev, link-exdev
  • Loading branch information
@trapexit
trapexit authored Feb 25, 2021
2 parents 50daf84 + 8adebc9 commit 3e69114
Show file tree
Hide file tree
Showing 26 changed files with 6,991 additions and 461 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ USE_XATTR = 1
UGID_USE_RWLOCK = 0

ifeq ($(DEBUG),1)
OPT_FLAGS := -O0 -g
OPT_FLAGS := -O0 -g -fsanitize=undefined
else
OPT_FLAGS := -O2
endif
Expand Down
51 changes: 51 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ These options are the same regardless you use them with the `mergerfs` commandli
* **statfs=base|full**: Controls how statfs works. 'base' means it will always use all branches in statfs calculations. 'full' is in effect path preserving and only includes drives where the path exists. (default: base)
* **statfs_ignore=none|ro|nc**: 'ro' will cause statfs calculations to ignore available space for branches mounted or tagged as 'read-only' or 'no create'. 'nc' will ignore available space for branches tagged as 'no create'. (default: none)
* **nfsopenhack=off|git|all**: A workaround for exporting mergerfs over NFS where there are issues with creating files for write while setting the mode to read-only. (default: off)
* **follow-symlinks=never|directory|regular|all**: Turns symlinks into what they point to. (default: never)
* **link-exdev=passthrough|rel-symlink|abs-base-symlink|abs-pool-symlink**: When a link fails with EXDEV optionally create a symlink to the file instead.
* **rename-exdev=passthrough|rel-symlink|abs-symlink**: When a rename fails with EXDEV optionally move the file to a special directory and symlink to it.
* **posix_acl=BOOL**: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false)
* **async_read=BOOL**: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true)
* **fuse_msg_size=UINT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256)
Expand Down Expand Up @@ -240,6 +243,54 @@ In Linux 4.20 a new feature was added allowing the negotiation of the max messag
Since there should be no downsides to increasing `fuse_msg_size` / `max_pages`, outside a minor bump in RAM usage due to larger message buffers, mergerfs defaults the value to 256. On kernels before 4.20 the value has no effect. The reason the value is configurable is to enable experimentation and benchmarking. See the BENCHMARKING section for examples.


### follow-symlinks

This feature, when enabled, will cause symlinks to be interpreted by mergerfs as their target (depending on the mode).

When there is a getattr/stat request for a file mergerfs will check if the file is a symlink and depending on the `follow-symlinks` setting will replace the information about the symlink with that of that which it points to.

When unlink'ing or rmdir'ing the followed symlink it will remove the symlink itself and not that which it points to.

* never: Behave as normal. Symlinks are treated as such.
* directory: Resolve symlinks only which point to directories.
* regular: Resolve symlinks only which point to regular files.
* all: Resolve all symlinks to that which they point to.

Symlinks which do not point to anything are left as is.

WARNING: This feature works but there might be edge cases yet found. If you find any odd behaviors please file a ticket on [github](https://github.com/trapexit/mergerfs/issues).


### link-exdev

If using path preservation and a `link` fails with EXDEV make a call to `symlink` where the `target` is the `oldlink` and the `linkpath` is the `newpath`. The `target` value is determined by the value of `link-exdev`.

* passthrough: Return EXDEV as normal.
* rel-symlink: A relative path from the `newpath`.
* abs-base-symlink: A absolute value using the underlying branch.
* abs-pool-symlink: A absolute value using the mergerfs mount point.

NOTE: It is possible that some applications check the file they link. In those cases it is possible it will error or complain.


### rename-exdev

If using path preservation and a `rename` fails with EXDEV:

1. Move file from **/branch/a/b/c** to **/branch/.mergerfs_rename_exdev/a/b/c**.
2. symlink the rename's `newpath` to the moved file.

The `target` value is determined by the value of `rename-exdev`.

* passthrough: Return EXDEV as normal.
* rel-symlink: A relative path from the `newpath`.
* abs-symlink: A absolute value using the mergerfs mount point.

NOTE: It is possible that some applications check the file they rename. In those cases it is possible it will error or complain.

NOTE: The reason `abs-symlink` is not split into two like `link-exdev` is due to the complexities in managing absolute base symlinks when multiple `oldpaths` exist.


### symlinkify

Due to the levels of indirection introduced by mergerfs and the underlying technology FUSE there can be varying levels of performance degradation. This feature will turn non-directories which are not writable into symlinks to the original file found by the `readlink` policy after the mtime and ctime are older than the timeout.
Expand Down
2 changes: 1 addition & 1 deletion libfuse/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ INSTALLUTILS :=
endif

ifeq ($(DEBUG),1)
OPT_FLAGS := -O0 -g
OPT_FLAGS := -O0 -g -fsanitize=undefined
else
OPT_FLAGS := -O2
endif
Expand Down
17 changes: 12 additions & 5 deletions libfuse/include/fuse.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,13 +121,13 @@ struct fuse_operations
int (*rmdir) (const char *);

/** Create a symbolic link */
int (*symlink) (const char *, const char *);
int (*symlink) (const char *, const char *, struct stat *, fuse_timeouts_t *);

/** Rename a file */
int (*rename) (const char *, const char *);

/** Create a hard link to a file */
int (*link) (const char *, const char *);
int (*link) (const char *, const char *, struct stat *, fuse_timeouts_t *);

/** Change the permission bits of a file */
int (*chmod) (const char *, mode_t);
Expand Down Expand Up @@ -794,9 +794,16 @@ int fuse_fs_rename(struct fuse_fs *fs, const char *oldpath,
const char *newpath);
int fuse_fs_unlink(struct fuse_fs *fs, const char *path);
int fuse_fs_rmdir(struct fuse_fs *fs, const char *path);
int fuse_fs_symlink(struct fuse_fs *fs, const char *linkname,
const char *path);
int fuse_fs_link(struct fuse_fs *fs, const char *oldpath, const char *newpath);
int fuse_fs_symlink(struct fuse_fs *fs,
const char *linkname,
const char *path,
struct stat *st,
fuse_timeouts_t *timeouts);
int fuse_fs_link(struct fuse_fs *fs,
const char *oldpath,
const char *newpath,
struct stat *st,
fuse_timeouts_t *timeouts);
int fuse_fs_release(struct fuse_fs *fs,
fuse_file_info_t *fi);
int fuse_fs_open(struct fuse_fs *fs, const char *path,
Expand Down
158 changes: 88 additions & 70 deletions libfuse/lib/fuse.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1650,25 +1650,30 @@ fuse_fs_rmdir(struct fuse_fs *fs,
}

int
fuse_fs_symlink(struct fuse_fs *fs,
const char *linkname,
const char *path)
fuse_fs_symlink(struct fuse_fs *fs_,
const char *linkname_,
const char *path_,
struct stat *st_,
fuse_timeouts_t *timeouts_)
{
if(fs->op.symlink == NULL)

if(fs_->op.symlink == NULL)
return -ENOSYS;

fuse_get_context()->private_data = fs->user_data;
if(fs_->debug)
fprintf(stderr,"symlink %s %s\n",linkname_,path_);

if(fs->debug)
fprintf(stderr,"symlink %s %s\n",linkname,path);
fuse_get_context()->private_data = fs_->user_data;

return fs->op.symlink(linkname,path);
return fs_->op.symlink(linkname_,path_,st_,timeouts_);
}

int
fuse_fs_link(struct fuse_fs *fs,
const char *oldpath,
const char *newpath)
fuse_fs_link(struct fuse_fs *fs,
const char *oldpath,
const char *newpath,
struct stat *st_,
fuse_timeouts_t *timeouts_)
{
if(fs->op.link == NULL)
return -ENOSYS;
Expand All @@ -1678,7 +1683,7 @@ fuse_fs_link(struct fuse_fs *fs,
if(fs->debug)
fprintf(stderr,"link %s %s\n",oldpath,newpath);

return fs->op.link(oldpath,newpath);
return fs->op.link(oldpath,newpath,st_,timeouts_);
}

int
Expand Down Expand Up @@ -2541,6 +2546,37 @@ update_stat(struct node *node_,
node_->mtim = stnew_->st_mtim;
}

static
int
set_path_info(struct fuse *f,
fuse_ino_t nodeid,
const char *name,
struct fuse_entry_param *e)
{
struct node *node;

node = find_node(f,nodeid,name);
if(node == NULL)
return -ENOMEM;

e->ino = node->nodeid;
e->generation = node->generation;

pthread_mutex_lock(&f->lock);
update_stat(node,&e->attr);
pthread_mutex_unlock(&f->lock);

set_stat(f,e->ino,&e->attr);
if(f->conf.debug)
fprintf(stderr,
" NODEID: %llu\n"
" GEN: %llu\n",
(unsigned long long)e->ino,
(unsigned long long)e->generation);

return 0;
}

static
int
lookup_path(struct fuse *f,
Expand All @@ -2550,43 +2586,18 @@ lookup_path(struct fuse *f,
struct fuse_entry_param *e,
fuse_file_info_t *fi)
{
int res;
int rv;

memset(e,0,sizeof(struct fuse_entry_param));

res = ((fi == NULL) ?
fuse_fs_getattr(f->fs,path,&e->attr,&e->timeout) :
fuse_fs_fgetattr(f->fs,&e->attr,fi,&e->timeout));

if(res == 0)
{
struct node *node;

node = find_node(f,nodeid,name);
if(node == NULL)
{
res = -ENOMEM;
}
else
{
e->ino = node->nodeid;
e->generation = node->generation;
rv = ((fi == NULL) ?
fuse_fs_getattr(f->fs,path,&e->attr,&e->timeout) :
fuse_fs_fgetattr(f->fs,&e->attr,fi,&e->timeout));

pthread_mutex_lock(&f->lock);
update_stat(node,&e->attr);
pthread_mutex_unlock(&f->lock);

set_stat(f,e->ino,&e->attr);
if(f->conf.debug)
fprintf(stderr,
" NODEID: %llu\n"
" GEN: %llu\n",
(unsigned long long)e->ino,
(unsigned long long)e->generation);
}
}
if(rv)
return rv;

return res;
return set_path_info(f,nodeid,name,e);
}

static
Expand Down Expand Up @@ -3236,26 +3247,28 @@ fuse_lib_rmdir(fuse_req_t req,

static
void
fuse_lib_symlink(fuse_req_t req,
const char *linkname,
fuse_ino_t parent,
const char *name)
fuse_lib_symlink(fuse_req_t req_,
const char *linkname_,
fuse_ino_t parent_,
const char *name_)
{
struct fuse *f = req_fuse_prepare(req);
struct fuse_entry_param e;
int rv;
char *path;
int err;
struct fuse *f;
struct fuse_entry_param e = {0};

err = get_path_name(f,parent,name,&path);
if(!err)
f = req_fuse_prepare(req_);

rv = get_path_name(f,parent_,name_,&path);
if(rv == 0)
{
err = fuse_fs_symlink(f->fs,linkname,path);
if(!err)
err = lookup_path(f,parent,name,path,&e,NULL);
free_path(f,parent,path);
rv = fuse_fs_symlink(f->fs,linkname_,path,&e.attr,&e.timeout);
if(rv == 0)
rv = set_path_info(f,parent_,name_,&e);
free_path(f,parent_,path);
}

reply_entry(req,&e,err);
reply_entry(req_,&e,rv);
}

static
Expand Down Expand Up @@ -3298,27 +3311,32 @@ fuse_lib_rename(fuse_req_t req,
reply_err(req,err);
}

static void fuse_lib_link(fuse_req_t req,fuse_ino_t ino,fuse_ino_t newparent,
const char *newname)
static
void
fuse_lib_link(fuse_req_t req,
fuse_ino_t ino,
fuse_ino_t newparent,
const char *newname)
{
struct fuse *f = req_fuse_prepare(req);
struct fuse_entry_param e;
int rv;
char *oldpath;
char *newpath;
int err;
struct fuse *f;
struct fuse_entry_param e = {0};

f = req_fuse_prepare(req);

err = get_path2(f,ino,NULL,newparent,newname,
rv = get_path2(f,ino,NULL,newparent,newname,
&oldpath,&newpath,NULL,NULL);
if(!err)
if(!rv)
{
err = fuse_fs_link(f->fs,oldpath,newpath);
if(!err)
err = lookup_path(f,newparent,newname,newpath,
&e,NULL);
rv = fuse_fs_link(f->fs,oldpath,newpath,&e.attr,&e.timeout);
if(rv == 0)
rv = set_path_info(f,newparent,newname,&e);
free_path2(f,ino,newparent,NULL,NULL,oldpath,newpath);
}

reply_entry(req,&e,err);
reply_entry(req,&e,rv);
}

static
Expand Down
Loading

0 comments on commit 3e69114

Please sign in to comment.