From patchwork Thu Feb 26 14:28:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Bluemle X-Patchwork-Id: 5893771 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 8633C9F37F for ; Thu, 26 Feb 2015 14:28:22 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CF4D420395 for ; Thu, 26 Feb 2015 14:28:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1C47620381 for ; Thu, 26 Feb 2015 14:28:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932266AbbBZO2N (ORCPT ); Thu, 26 Feb 2015 09:28:13 -0500 Received: from mail.itxperts.de ([212.202.108.166]:64131 "EHLO mail.itxperts.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932186AbbBZO2M (ORCPT ); Thu, 26 Feb 2015 09:28:12 -0500 Received: from itxmail.itxperts.de (qgate_dmz.itxperts.de [192.168.160.1]) by mail.itxperts.de (8.14.0/8.14.0) with ESMTP id t1QESAUd028173 for ; Thu, 26 Feb 2015 15:28:10 +0100 Received: from doppio (doppio.itxperts.de [192.168.140.7]) by itxmail.itxperts.de (Postfix) with ESMTP id 9F80313F02 for ; Thu, 26 Feb 2015 15:19:52 +0100 (CET) Date: Thu, 26 Feb 2015 15:28:07 +0100 From: Andreas Bluemle To: Ceph Development Subject: FileStore performance: coalescing operations Message-ID: <20150226152807.5b71a93c@doppio> Organization: ITXperts GmbH X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.10; i686-pc-linux-gnu) Mime-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (mail.itxperts.de [192.168.160.166]); Thu, 26 Feb 2015 15:28:10 +0100 (CET) Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi, during the performance weely meeting, I had mentioned my experiences concerning the transaction structure for write requests at the level of the FileStore. Such a transaction not only contains the OP_WRITE operation to the object in the file system, but also a series of OP_OMAP_SETKEYS and OP_SETATTR operations. Find attached a README and source code patch, which describe a prototype for coalescing the OP_OMAP_SETKEYS operations and the performance impact f this change. Regards Andreas Bluemle diff --git a/src/os/FileStore.cc b/src/os/FileStore.cc index f6c3bb8..29382b2 100644 --- a/src/os/FileStore.cc +++ b/src/os/FileStore.cc @@ -2260,10 +2260,24 @@ int FileStore::_check_replay_guard(int fd, const SequencerPosition& spos) } } +void FileStore::_coalesce(map &target, map &source) +{ + for (map::iterator p = source.begin(); + p != source.end(); + p++) { + target[p->first] = p->second; + } + return; +} + unsigned FileStore::_do_transaction( Transaction& t, uint64_t op_seq, int trans_num, ThreadPool::TPHandle *handle) { + map collected_aset; + coll_t collected_cid; + ghobject_t collected_oid; + dout(10) << "_do_transaction on " << &t << dendl; #ifdef WITH_LTTNG @@ -2282,6 +2296,22 @@ unsigned FileStore::_do_transaction( _inject_failure(); + if (op->op == Transaction::OP_OMAP_SETKEYS) { + collected_cid = i.get_cid(op->cid); + collected_oid = i.get_oid(op->oid); + map aset; + i.decode_attrset(aset); + _coalesce(collected_aset, aset); + continue; + } else { + if (collected_aset.empty() == false) { + tracepoint(objectstore, omap_setkeys_enter, osr_name); + r = _omap_setkeys(collected_cid, collected_oid, collected_aset, spos); + tracepoint(objectstore, omap_setkeys_exit, r); + collected_aset.clear(); + } + } + switch (op->op) { case Transaction::OP_NOP: break; diff --git a/src/os/FileStore.h b/src/os/FileStore.h index af1fb8d..a039731 100644 --- a/src/os/FileStore.h +++ b/src/os/FileStore.h @@ -449,6 +449,8 @@ public: int statfs(struct statfs *buf); + void _coalesce( map &target, map &source); + int _do_transactions( list &tls, uint64_t op_seq, ThreadPool::TPHandle *handle);