Skip to content

Commit

Permalink
Orangefs: kernel client part 6
Browse files Browse the repository at this point in the history
Signed-off-by: Mike Marshall <hubcap@omnibond.com>
  • Loading branch information
hubcapsc committed Oct 3, 2015
1 parent 1182fca commit 74a552a
Show file tree
Hide file tree
Showing 2 changed files with 224 additions and 0 deletions.
87 changes: 87 additions & 0 deletions Documentation/ABI/stable/sysfs-fs-orangefs
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
What: /sys/fs/orangefs/perf_counters/*
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Counters and settings for various caches.
Read only.


What: /sys/fs/orangefs/perf_counter_reset
Date: June 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
echo a 0 or a 1 into perf_counter_reset to
reset all the counters in
/sys/fs/orangefs/perf_counters
except ones with PINT_PERF_PRESERVE set.


What: /sys/fs/orangefs/perf_time_interval_secs
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Length of perf counter intervals in
seconds.


What: /sys/fs/orangefs/perf_history_size
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
The perf_counters cache statistics have N, or
perf_history_size, samples. The default is
one.

Every perf_time_interval_secs the (first)
samples are reset.

If N is greater than one, the "current" set
of samples is reset, and the samples from the
other N-1 intervals remain available.


What: /sys/fs/orangefs/op_timeout_secs
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Service operation timeout in seconds.


What: /sys/fs/orangefs/slot_timeout_secs
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
"Slot" timeout in seconds. A "slot"
is an indexed buffer in the shared
memory segment used for communication
between the kernel module and userspace.
Slots are requested and waited for,
the wait times out after slot_timeout_secs.


What: /sys/fs/orangefs/acache/*
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Attribute cache configurable settings.


What: /sys/fs/orangefs/ncache/*
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Name cache configurable settings.


What: /sys/fs/orangefs/capcache/*
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Capability cache configurable settings.


What: /sys/fs/orangefs/ccache/*
Date: Jun 2015
Contact: Mike Marshall <hubcap@omnibond.com>
Description:
Credential cache configurable settings.
137 changes: 137 additions & 0 deletions Documentation/filesystems/orangefs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
ORANGEFS
========

OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal
for large storage problems faced by HPC, BigData, Streaming Video,
Genomics, Bioinformatics.

Orangefs, originally called PVFS, was first developed in 1993 by
Walt Ligon and Eric Blumer as a parallel file system for Parallel
Virtual Machine (PVM) as part of a NASA grant to study the I/O patterns
of parallel programs.

Orangefs features include:

* Distributes file data among multiple file servers
* Supports simultaneous access by multiple clients
* Stores file data and metadata on servers using local file system
and access methods
* Userspace implementation is easy to install and maintain
* Direct MPI support
* Stateless


MAILING LIST
============

http://beowulf-underground.org/mailman/listinfo/pvfs2-users


DOCUMENTATION
=============

http://www.orangefs.org/documentation/


USERSPACE FILESYSTEM SOURCE
===========================

http://www.orangefs.org/download

Orangefs versions prior to 2.9.3 would not be compatible with the
upstream version of the kernel client.


BUILDING THE USERSPACE FILESYSTEM ON A SINGLE SERVER
====================================================

When Orangefs is upstream, "--with-kernel" shouldn't be needed, but
until then the path to where the kernel with the Orangefs kernel client
patch was built is needed to ensure that pvfs2-client-core (the bridge
between kernel space and user space) will build properly. You can omit
--prefix if you don't care that things are sprinkled around in
/usr/local.

./configure --prefix=/opt/ofs --with-kernel=/path/to/orangefs/kernel

make

make install

Create an orangefs config file:
/opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf

for "Enter hostnames", use the hostname, don't let it default to
localhost.

create a pvfs2tab file in /etc:
cat /etc/pvfs2tab
tcp://myhostname:3334/orangefs /mymountpoint pvfs2 defaults,noauto 0 0

create the mount point you specified in the tab file if needed:
mkdir /mymountpoint

bootstrap the server:
/opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf -f

start the server:
/opt/osf/sbin/pvfs2-server /etc/pvfs2.conf

Now the server is running. At this point you might like to
prove things are working with:

/opt/osf/bin/pvfs2-ls /mymountpoint

You might not want to enforce selinux, it doesn't seem to matter by
linux 3.11...

If stuff seems to be working, turn on the client core:
/opt/osf/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core

Mount your filesystem.
mount -t pvfs2 tcp://myhostname:3334/orangefs /mymountpoint


OPTIONS
=======

The following mount options are accepted:

acl
Allow the use of Access Control Lists on files and directories.

intr
Some operations between the kernel client and the user space
filesystem can be interruptible, such as changes in debug levels
and the setting of tunable parameters.

local_lock
Enable posix locking from the perspective of "this" kernel. The
default file_operations lock action is to return ENOSYS. Posix
locking kicks in if the filesystem is mounted with -o local_lock.
Distributed locking is being worked on for the future.


DEBUGGING
=========

If you want the debug (GOSSIP) statments in a particular
source file (inode.c for example) go to syslog:

echo inode > /sys/kernel/debug/orangefs/kernel-debug

No debugging (the default):

echo none > /sys/kernel/debug/orangefs/kernel-debug

Debugging from several source files:

echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug

All debugging:

echo all > /sys/kernel/debug/orangefs/kernel-debug

Get a list of all debugging keywords:

cat /sys/kernel/debug/orangefs/debug-help

0 comments on commit 74a552a

Please sign in to comment.