From patchwork Fri Jul 8 18:26:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 9221677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AE0FD60467 for ; Fri, 8 Jul 2016 18:29:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A39E32836F for ; Fri, 8 Jul 2016 18:29:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9554628387; Fri, 8 Jul 2016 18:29:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 482AF2836F for ; Fri, 8 Jul 2016 18:29:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bLaUE-0003wZ-SN; Fri, 08 Jul 2016 18:26:50 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bLaUB-0003qf-6F for xen-devel@lists.xenproject.org; Fri, 08 Jul 2016 18:26:47 +0000 Received: from [85.158.139.211] by server-2.bemta-5.messagelabs.com id 1B/EE-13025-660FF775; Fri, 08 Jul 2016 18:26:46 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFuphkeJIrShJLcpLzFFi42JxWrrBXjftQ32 4wf4j7Bbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8byaVeYC+4IVJx9MYW1gXERbxcjB4eEgL/E rt1xXYycHGwCuhJNW/6ygYRFBFQkbu81ADGZBUIldrywBKkQBjLf/z7IBmKzAFW83HODCcTmF fCUWNS/gxnElhCQkzh//CeYzQkU37XkJjuILSTgIdH8q4kRZKSQgJrE3PXxEK2CEidnPmEBsZ kFJCQOvnjBDHEXt8TfbvsJjHyzkFTNQlK1gJFpFaNGcWpRWWqRrrGBXlJRZnpGSW5iZo6uoYG pXm5qcXFiempOYlKxXnJ+7iZGYCjVMzAw7mCcsMrvEKMkB5OSKO++Z/XhQnxJ+SmVGYnFGfFF pTmpxYcYZTg4lCR4c94D5QSLUtNTK9Iyc4BBDZOW4OBREuEtB0nzFhck5hZnpkOkTjEqSonzp oEkBEASGaV5cG2wSLrEKCslzMvIwMAgxFOQWpSbWYIq/4pRnINRSZi3HmQKT2ZeCdz0V0CLmY AWGwSALS5JREhJNTD27Jwl/3DPY+11K5ZrN23Y8Hrllju8j5383/VfK79lpn38w/LK5RIeis/ MuSd3139slL7XdEJ6qmXNkrLpE03SO3t3dUY4v63SeWOfx1VoW3BYuHj/pZrrOyV4v/it9VI+ 5Lqu67qvnojBcTOjUM3Kgw+C3h7e7P/IrPeOheJrnpLopXlLhZWUWIozEg21mIuKEwGL90/Qn wIAAA== X-Env-Sender: prvs=9904f56b2=Ian.Jackson@citrix.com X-Msg-Ref: server-5.tower-206.messagelabs.com!1468002404!49194160!2 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20532 invoked from network); 8 Jul 2016 18:26:45 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-5.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 8 Jul 2016 18:26:45 -0000 X-IronPort-AV: E=Sophos;i="5.28,331,1464652800"; d="scan'208";a="372092125" From: Ian Jackson To: Date: Fri, 8 Jul 2016 19:26:08 +0100 Message-ID: <1468002385-4407-17-git-send-email-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1468002385-4407-1-git-send-email-ian.jackson@eu.citrix.com> References: <1468002385-4407-1-git-send-email-ian.jackson@eu.citrix.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: Ian Jackson Subject: [Xen-devel] [OSSTEST PATCH 16/33] Database locking: Tcl: Retry only on DEADLOCK DETECTED X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Use the new errorCode coming out of db-execute* to tell when the error is that we got a database deadlock, which is the situation in which we should retry. This involves combining the two catch blocks, so that there is only one error handling strategy. Previously errors on COMMIT would be retried and others would not. Now errors anywhere might be retried but only if the DB indicated deadlock. We now unconditionally execute ROLLBACK. This is more correct, since we always previously executed BEGIN. And, we pass the errorInfo and errorCode from the $body to the caller. I have tested this with a test db instance, using contrived means to generate a database deadlock, and it does actually retry. Signed-off-by: Ian Jackson Acked-by: Ian Campbell --- tcl/JobDB-Executive.tcl | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl index ed9abbb..63db4f0 100644 --- a/tcl/JobDB-Executive.tcl +++ b/tcl/JobDB-Executive.tcl @@ -283,25 +283,27 @@ proc transaction {tables script} { db-execute "SET TRANSACTION ISOLATION LEVEL SERIALIZABLE" lock-tables $tables uplevel 1 $script + db-execute COMMIT } result] - if {!$rc} { - if {[catch { - db-execute COMMIT - } emsg]} { - puts "commit failed: $emsg; retrying ..." - db-execute ROLLBACK - if {[incr retries -1] <= 0} { - error \ - "commit failed, too many retries: $emsg\n$errorInfo\n$errorCode\n" + set ei $errorInfo + set ec $errorCode + if {$rc} { + db-execute ROLLBACK + switch -glob $errorCode { + {OSSTEST-PSQL * 40P01} { + # DEADLOCK DETECTED + puts "transaction deadlock ($result) retrying ..." + if {[incr retries -1] <= 0} { + error \ + "transaction failed, too many retries: $result\n$errorInfo\n$errorCode\n" + } + after 500 + continue } - after 500 - continue } - } else { - db-execute ROLLBACK } db-close - return -code $rc $result + return -code $rc -errorinfo $ei -errorcode $ec $result } }