From patchwork Fri Jul 17 17:16:46 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Marshall X-Patchwork-Id: 6818361 Return-Path: X-Original-To: patchwork-linux-fsdevel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D185C9F380 for ; Fri, 17 Jul 2015 17:18:01 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A035C205B3 for ; Fri, 17 Jul 2015 17:18:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4816420582 for ; Fri, 17 Jul 2015 17:17:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755536AbbGQRRg (ORCPT ); Fri, 17 Jul 2015 13:17:36 -0400 Received: from mail-qg0-f50.google.com ([209.85.192.50]:34200 "EHLO mail-qg0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755520AbbGQRRX (ORCPT ); Fri, 17 Jul 2015 13:17:23 -0400 Received: by qgep37 with SMTP id p37so50080957qge.1 for ; Fri, 17 Jul 2015 10:17:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dYrVgmDEmJtFlqKEtWScZVvTTawVpuY+SaNSKX0t7iU=; b=MgxZPAeD4TIO+UYt619+l7gIJQBRQ47QEs/yAx6D3cJeZ6vAq7URN0HdyyFAZ0t0wq 0/v4JnhrSG006yiCBzPHG2xDJ1V5u6LIgn0Eye+z3usV+FX7Z9CTJyZTpzSWkcVYpEKc S45ZDIo1CLAXpU3O8zVR1K7pf3Xq3YyzXd5Cay1R+ahZgSQZIdru5qfUH9jYF0KHdRyK TAB/BSVVUZCNKvd4bUr57jMdLB5kvl1cgHSqE8NbSOSRQXgY7xWQq6nKn9w5qUKFBjX6 RvlIStS8xDMxK0+NWZ5qrEtjV2G2G9KITAkTKqQ2rFN0lWL7ycImr/n0wMTQJEqHXzbD K4Sw== X-Gm-Message-State: ALoCoQnzFSfwCuy+M6zZMhWrDB3+oAfrDeFzIwoxsD+ux5Z79IUAFiVj5hMxoUsecVWSzImT/CPb X-Received: by 10.140.91.81 with SMTP id y75mr26902698qgd.3.1437153442849; Fri, 17 Jul 2015 10:17:22 -0700 (PDT) Received: from logtruck.clemson.edu (logtruck.clemson.edu. [130.127.148.78]) by smtp.gmail.com with ESMTPSA id f207sm6125352qhc.41.2015.07.17.10.17.21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Jul 2015 10:17:22 -0700 (PDT) From: Mike Marshall X-Google-Original-From: Mike Marshall To: viro@zeniv.linux.org.uk Cc: Mike Marshall , linux-fsdevel@vger.kernel.org Subject: [PATCH V3 6/7] Orangefs: kernel client part 6 Date: Fri, 17 Jul 2015 13:16:46 -0400 Message-Id: <1437153407-19222-7-git-send-email-root@logtruck.clemson.edu> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1437153407-19222-1-git-send-email-root@logtruck.clemson.edu> References: <1437153407-19222-1-git-send-email-root@logtruck.clemson.edu> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Spam-Status: No, score=-8.1 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mike Marshall Signed-off-by: Mike Marshall --- Documentation/ABI/stable/sysfs-fs-orangefs | 87 ++++++++++++++++++ Documentation/filesystems/orangefs.txt | 137 +++++++++++++++++++++++++++++ 2 files changed, 224 insertions(+) create mode 100644 Documentation/ABI/stable/sysfs-fs-orangefs create mode 100644 Documentation/filesystems/orangefs.txt diff --git a/Documentation/ABI/stable/sysfs-fs-orangefs b/Documentation/ABI/stable/sysfs-fs-orangefs new file mode 100644 index 0000000..affdb11 --- /dev/null +++ b/Documentation/ABI/stable/sysfs-fs-orangefs @@ -0,0 +1,87 @@ +What: /sys/fs/orangefs/perf_counters/* +Date: Jun 2015 +Contact: Mike Marshall +Description: + Counters and settings for various caches. + Read only. + + +What: /sys/fs/orangefs/perf_counter_reset +Date: June 2015 +Contact: Mike Marshall +Description: + echo a 0 or a 1 into perf_counter_reset to + reset all the counters in + /sys/fs/orangefs/perf_counters + except ones with PINT_PERF_PRESERVE set. + + +What: /sys/fs/orangefs/perf_time_interval_secs +Date: Jun 2015 +Contact: Mike Marshall +Description: + Length of perf counter intervals in + seconds. + + +What: /sys/fs/orangefs/perf_history_size +Date: Jun 2015 +Contact: Mike Marshall +Description: + The perf_counters cache statistics have N, or + perf_history_size, samples. The default is + one. + + Every perf_time_interval_secs the (first) + samples are reset. + + If N is greater than one, the "current" set + of samples is reset, and the samples from the + other N-1 intervals remain available. + + +What: /sys/fs/orangefs/op_timeout_secs +Date: Jun 2015 +Contact: Mike Marshall +Description: + Service operation timeout in seconds. + + +What: /sys/fs/orangefs/slot_timeout_secs +Date: Jun 2015 +Contact: Mike Marshall +Description: + "Slot" timeout in seconds. A "slot" + is an indexed buffer in the shared + memory segment used for communication + between the kernel module and userspace. + Slots are requested and waited for, + the wait times out after slot_timeout_secs. + + +What: /sys/fs/orangefs/acache/* +Date: Jun 2015 +Contact: Mike Marshall +Description: + Attribute cache configurable settings. + + +What: /sys/fs/orangefs/ncache/* +Date: Jun 2015 +Contact: Mike Marshall +Description: + Name cache configurable settings. + + +What: /sys/fs/orangefs/capcache/* +Date: Jun 2015 +Contact: Mike Marshall +Description: + Capability cache configurable settings. + + +What: /sys/fs/orangefs/ccache/* +Date: Jun 2015 +Contact: Mike Marshall +Description: + Credential cache configurable settings. diff --git a/Documentation/filesystems/orangefs.txt b/Documentation/filesystems/orangefs.txt new file mode 100644 index 0000000..ec9c841 --- /dev/null +++ b/Documentation/filesystems/orangefs.txt @@ -0,0 +1,137 @@ +ORANGEFS +======== + +OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal +for large storage problems faced by HPC, BigData, Streaming Video, +Genomics, Bioinformatics. + +Orangefs, originally called PVFS, was first developed in 1993 by +Walt Ligon and Eric Blumer as a parallel file system for Parallel +Virtual Machine (PVM) as part of a NASA grant to study the I/O patterns +of parallel programs. + +Orangefs features include: + + * Distributes file data among multiple file servers + * Supports simultaneous access by multiple clients + * Stores file data and metadata on servers using local file system + and access methods + * Userspace implementation is easy to install and maintain + * Direct MPI support + * Stateless + + +MAILING LIST +============ + +http://beowulf-underground.org/mailman/listinfo/pvfs2-users + + +DOCUMENTATION +============= + +http://www.orangefs.org/documentation/ + + +USERSPACE FILESYSTEM SOURCE +=========================== + +http://www.orangefs.org/download + +Orangefs versions prior to 2.9.3 would not be compatible with the +upstream version of the kernel client. + + +BUILDING THE USERSPACE FILESYSTEM ON A SINGLE SERVER +==================================================== + +When Orangefs is upstream, "--with-kernel" shouldn't be needed, but +until then the path to where the kernel with the Orangefs kernel client +patch was built is needed to ensure that pvfs2-client-core (the bridge +between kernel space and user space) will build properly. You can omit +--prefix if you don't care that things are sprinkled around in +/usr/local. + +./configure --prefix=/opt/ofs --with-kernel=/path/to/orangefs/kernel + +make + +make install + +Create an orangefs config file: +/opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf + + for "Enter hostnames", use the hostname, don't let it default to + localhost. + +create a pvfs2tab file in /etc: +cat /etc/pvfs2tab +tcp://myhostname:3334/orangefs /mymountpoint pvfs2 defaults,noauto 0 0 + +create the mount point you specified in the tab file if needed: +mkdir /mymountpoint + +bootstrap the server: +/opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf -f + +start the server: +/opt/osf/sbin/pvfs2-server /etc/pvfs2.conf + +Now the server is running. At this point you might like to +prove things are working with: + +/opt/osf/bin/pvfs2-ls /mymountpoint + +You might not want to enforce selinux, it doesn't seem to matter by +linux 3.11... + +If stuff seems to be working, turn on the client core: +/opt/osf/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core + +Mount your filesystem. +mount -t pvfs2 tcp://myhostname:3334/orangefs /mymountpoint + + +OPTIONS +======= + +The following mount options are accepted: + + acl + Allow the use of Access Control Lists on files and directories. + + intr + Some operations between the kernel client and the user space + filesystem can be interruptible, such as changes in debug levels + and the setting of tunable parameters. + + local_lock + Enable posix locking from the perspective of "this" kernel. The + default file_operations lock action is to return ENOSYS. Posix + locking kicks in if the filesystem is mounted with -o local_lock. + Distributed locking is being worked on for the future. + + +DEBUGGING +========= + +If you want the debug (GOSSIP) statments in a particular +source file (inode.c for example) go to syslog: + + echo inode > /sys/kernel/debug/orangefs/kernel-debug + +No debugging (the default): + + echo none > /sys/kernel/debug/orangefs/kernel-debug + +Debugging from several source files: + + echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug + +All debugging: + + echo all > /sys/kernel/debug/orangefs/kernel-debug + +Get a list of all debugging keywords: + + cat /sys/kernel/debug/orangefs/debug-help