From patchwork Wed Aug 29 12:17:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Motai.Hirotaka@aj.MitsubishiElectric.co.jp" X-Patchwork-Id: 10580233 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F2D7175A for ; Wed, 29 Aug 2018 12:42:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DECC2B014 for ; Wed, 29 Aug 2018 12:42:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 023CC2B019; Wed, 29 Aug 2018 12:42:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0CB942B014 for ; Wed, 29 Aug 2018 12:42:50 +0000 (UTC) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 72EE6DBC; Wed, 29 Aug 2018 12:36:52 +0000 (UTC) X-Original-To: ltsi-dev@lists.linuxfoundation.org Delivered-To: ltsi-dev@mail.linuxfoundation.org Received: from smtp2.linuxfoundation.org (smtp2.linux-foundation.org [172.17.192.36]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 90E60D8E for ; Wed, 29 Aug 2018 12:36:46 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mx06.melco.co.jp (mx06.melco.co.jp [192.218.140.146]) by smtp2.linuxfoundation.org (Postfix) with ESMTPS id 4D5EF1DCED for ; Wed, 29 Aug 2018 12:36:40 +0000 (UTC) Received: from mr06.melco.co.jp (mr06 [133.141.98.164]) by mx06.melco.co.jp (Postfix) with ESMTP id 112893A2E45 for ; Wed, 29 Aug 2018 21:17:27 +0900 (JST) Received: from mr06.melco.co.jp (unknown [127.0.0.1]) by mr06.imss (Postfix) with ESMTP id 420l4R01zvzRkBT for ; Wed, 29 Aug 2018 21:17:27 +0900 (JST) Received: from mf03_second.melco.co.jp (unknown [192.168.20.183]) by mr06.melco.co.jp (Postfix) with ESMTP id 420l4Q6r3BzRk5W for ; Wed, 29 Aug 2018 21:17:26 +0900 (JST) Received: from mf03.melco.co.jp (unknown [133.141.98.183]) by mf03_second.melco.co.jp (Postfix) with ESMTP id 420l4Q6jlczRk8v for ; Wed, 29 Aug 2018 21:17:26 +0900 (JST) Received: from JPN01-OS2-obe.outbound.protection.outlook.com (unknown [23.103.139.143]) by mf03.melco.co.jp (Postfix) with ESMTP id 420l4Q679kzRk38 for ; Wed, 29 Aug 2018 21:17:26 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mitsubishielectricgroup.onmicrosoft.com; s=selector1-mitsubishielectricgroup-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z5O6ENshjF3ZaKm5twOe4SuxTYsd3hI2bnHAqQQ55MQ=; b=YlEKjLxznwbzw/Jomo8y9fFrn6XWpECMSEgPmzgOhxh/L9yE5BlJFUtZ7Y+dIe0suuNbd6kdwG1KdJ2XQdSVzvMB2W2pKEnB7llfunf3KNxshZX/aVtqoh2F/IjkLC5tCwJBnnZr11ct6c7ReBWKFxgrH5gwMazIqOoBTfgoOr0= Received: from TY1PR01MB1692.jpnprd01.prod.outlook.com (52.133.160.145) by TY1PR01MB1708.jpnprd01.prod.outlook.com (52.133.160.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1080.15; Wed, 29 Aug 2018 12:17:26 +0000 Received: from TY1PR01MB1692.jpnprd01.prod.outlook.com ([fe80::d04e:e6f6:c782:22fa]) by TY1PR01MB1692.jpnprd01.prod.outlook.com ([fe80::d04e:e6f6:c782:22fa%2]) with mapi id 15.20.1080.015; Wed, 29 Aug 2018 12:17:26 +0000 From: "Motai.Hirotaka@aj.MitsubishiElectric.co.jp" To: "ltsi-dev@lists.linuxfoundation.org" Thread-Topic: [LTSI-dev] [PATCH 13/54] ring-buffer: Redefine the unimplemented RINGBUF_TYPE_TIME_STAMP Thread-Index: AdQ/iiotqVbKydeLSKOsOGwdK0SWlA== Date: Wed, 29 Aug 2018 12:17:05 +0000 Deferred-Delivery: Wed, 29 Aug 2018 12:15:00 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-melpop: 1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=Motai.Hirotaka@aj.MitsubishiElectric.co.jp; x-originating-ip: [153.231.200.201] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; TY1PR01MB1708; 6:uIST8kDVfApNOJ1OhS5PdG0NuHbjW6Q87xu8tij2pwgqfo5NZAjP0mp0QZC8vURPvduMSxr2eSjEhoTTsWut9HksnN77MSczNTdn+jFUBby9517QANmgXdQGy2jGTKRvj5H955cJcALbbbGW+vviXYb4FPE4HdN04BxpRsPPrBc4unWEtfvFtfgqw7a+i6kwFEsHWWo8Ae8VXYmQ4jqUJao4diLyFCBMLxFzxoUK2RiHQEpF31QOYvJu+u7jz18dbR82ovy9WMUxZvZW0xsUkIrjHKqbDkY6Uf8D1Pjx0UBIqnZDxd9MN8baShk+gwZZead4knVtCdkHUemmHRpLfu72qsC6FAmWMi+OhiyXEJLXjkOnrJ2mY5UmSF1iBLYzHVIntuCWOQR7wCDhQaPm+h98VuAUnL9SKQ2luPdAfUIpYTOIwj/mG+JCU+0cw3flr73lQ9vPBSjkh/oQqlk0ag==; 5:iKrBtORS1OdV0HJfYr3cvgpIWXXyFxWuof5O993W7UZPcGlF13ZqyRdqfleYLh0CKN12vaEmqoWycK5BMplC0jjPEVtW0edNTZaAXHh5Yx8tqLlQZ/WyK6z2Qk8MC+Owq39M6id4bdTZk3hs+bPqhD56XPicU6/a2fbZgWpb5wM=; 7:FSs4APhRbfPpQBcBJbXVzS/L/KgieFpKFz9CS2raJ6VVNqzSEloh7CVs04m6ZRDG1TUYtDTZedjYQRhig31uqQ4Tjx9pYBW0eq+I0io45AXobT5tV6rnq8fp5S/SR8K/j99A66LKPt+ZuiOtQr2dHiKtUwfmpu8iGDCK5nAwppZftYAywMb0//rfE2jOJGw8Npmw/grRIHT1ii371DD0dPIfCyMb358oCy6rBH/ulZC1qOQtlkhjT0hvvCSfptSs x-ms-office365-filtering-correlation-id: 5cf136da-a601-4cb6-beca-08d60da961c8 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:TY1PR01MB1708; x-ms-traffictypediagnostic: TY1PR01MB1708: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(42068640409301)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231311)(944501410)(52105095)(149027)(150027)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699016); SRVR:TY1PR01MB1708; BCL:0; PCL:0; RULEID:; SRVR:TY1PR01MB1708; x-forefront-prvs: 077929D941 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(366004)(396003)(39860400002)(346002)(376002)(189003)(199004)(966005)(25786009)(74482002)(81156014)(8676002)(106356001)(5660300001)(105586002)(6506007)(14454004)(81166006)(33656002)(26005)(7736002)(74316002)(186003)(72206003)(305945005)(66066001)(68736007)(97736004)(478600001)(476003)(102836004)(2906002)(8936002)(6666003)(6916009)(486006)(6116002)(1857600001)(316002)(7696005)(2900100001)(53946003)(5640700003)(3846002)(55016002)(99286004)(53936002)(14444005)(86362001)(5024004)(256004)(6436002)(6306002)(2501003)(5250100002)(9686003)(575784001)(2351001); DIR:OUT; SFP:1102; SCL:1; SRVR:TY1PR01MB1708; H:TY1PR01MB1692.jpnprd01.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:0; received-spf: None (protection.outlook.com: aj.MitsubishiElectric.co.jp does not designate permitted sender hosts) x-microsoft-antispam-message-info: 2Qn17SjTPpZtxUoMvwQ5pS3AX23Xuq2bzTKgINJkH92siQxnrA8Rjn9q4kqyFQRU/wgn9mTBeCcJDre9UhahdvsATsFBXfXAUzLFN68N6ZRUbCdMAm9sB3uFW4gxFNfJTPnVZDnQslWYfcCelMQH65ns9ku7eWYVc6pBeWdAyqmp26vhbcqEjZrJLyiySVGplBGqP4UNveAmOVNP0RmCWEh8s0hCUBagji/4cqagSvBPJCE9xH0V2yJ2Gcghfj4psBO7jgJe0NxRMMYQ1zIdGXTyr81DKan8tL5/aNoMUsLqyj13hFHhZ7+43QDMhGRtFKZlmow7ZOcsUo6dQw1Zv7Edxtp5zKzOiIAWeLCw1Hg= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: aj.MitsubishiElectric.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 5cf136da-a601-4cb6-beca-08d60da961c8 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Aug 2018 12:17:22.6307 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: c5a75b62-4bff-4c96-a720-6621ce9978e5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY1PR01MB1708 Subject: [LTSI-dev] [PATCH 13/54] ring-buffer: Redefine the unimplemented RINGBUF_TYPE_TIME_STAMP X-BeenThere: ltsi-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "A list to discuss patches, development, and other things related to the LTSI project" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ltsi-dev-bounces@lists.linuxfoundation.org Errors-To: ltsi-dev-bounces@lists.linuxfoundation.org X-Virus-Scanned: ClamAV using ClamSMTP RINGBUF_TYPE_TIME_STAMP is defined but not used, and from what I can gather was reserved for something like an absolute timestamp feature for the ring buffer, if not a complete replacement of the current time_delta scheme. This code redefines RINGBUF_TYPE_TIME_STAMP to implement absolute time stamps. Another way to look at it is that it essentially forces extended time_deltas for all events. The motivation for doing this is to enable time_deltas that aren't dependent on previous events in the ring buffer, making it feasible to use the ring_buffer_event timetamps in a more random-access way, for purposes other than serial event printing. To set/reset this mode, use tracing_set_timestamp_abs() from the previous interface patch. Link: http://lkml.kernel.org/r/477b362dba1ce7fab9889a1a8e885a62c472f041.1516069914.git.tom.zanussi@linux.intel.com Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (VMware) (cherry picked from commit dc4e2801d400b0346fb281ce9cf010d611e2243c) Signed-off-by: Hirotaka MOTAI --- include/linux/ring_buffer.h | 12 +++-- kernel/trace/ring_buffer.c | 104 ++++++++++++++++++++++++++---------- 2 files changed, 83 insertions(+), 33 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index adffb56b..6c2a6b3f 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -29,42 +29,44 @@ struct ring_buffer_event { * If time_delta is non zero: * array[0] holds the actual length * size = 4 + length (bytes) * * @RINGBUF_TYPE_TIME_EXTEND: Extend the time delta * array[0] = time delta (28 .. 59) * size = 8 bytes * - * @RINGBUF_TYPE_TIME_STAMP: Sync time stamp with external clock - * array[0] = tv_nsec - * array[1..2] = tv_sec - * size = 16 bytes + * @RINGBUF_TYPE_TIME_STAMP: Absolute timestamp + * Same format as TIME_EXTEND except that the + * value is an absolute timestamp, not a delta + * event.time_delta contains bottom 27 bits + * array[0] = top (28 .. 59) bits + * size = 8 bytes * * <= @RINGBUF_TYPE_DATA_TYPE_LEN_MAX: * Data record * If type_len is zero: * array[0] holds the actual length * array[1..(length+3)/4] holds data * size = 4 + length (bytes) * else * length = type_len << 2 * array[0..(length+3)/4-1] holds data * size = 4 + length (bytes) */ enum ring_buffer_type { RINGBUF_TYPE_DATA_TYPE_LEN_MAX = 28, RINGBUF_TYPE_PADDING, RINGBUF_TYPE_TIME_EXTEND, - /* FIXME: RINGBUF_TYPE_TIME_STAMP not implemented */ RINGBUF_TYPE_TIME_STAMP, }; unsigned ring_buffer_event_length(struct ring_buffer_event *event); void *ring_buffer_event_data(struct ring_buffer_event *event); +u64 ring_buffer_event_time_stamp(struct ring_buffer_event *event); /* * ring_buffer_discard_commit will remove an event that has not * ben committed yet. If this is used, then ring_buffer_unlock_commit * must not be called on the discarded event. This function * will try to remove the event from the ring buffer completely * if another event has not been written after it. * diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index eed0e343..45fc9155 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -36,16 +36,18 @@ int ring_buffer_print_entry_header(struct trace_seq *s) trace_seq_puts(s, "\ttype_len : 5 bits\n"); trace_seq_puts(s, "\ttime_delta : 27 bits\n"); trace_seq_puts(s, "\tarray : 32 bits\n"); trace_seq_putc(s, '\n'); trace_seq_printf(s, "\tpadding : type == %d\n", RINGBUF_TYPE_PADDING); trace_seq_printf(s, "\ttime_extend : type == %d\n", RINGBUF_TYPE_TIME_EXTEND); + trace_seq_printf(s, "\ttime_stamp : type == %d\n", + RINGBUF_TYPE_TIME_STAMP); trace_seq_printf(s, "\tdata max type_len == %d\n", RINGBUF_TYPE_DATA_TYPE_LEN_MAX); return !trace_seq_has_overflowed(s); } /* * The ring buffer is made up of a list of pages. A separate list of pages is @@ -135,22 +137,25 @@ int ring_buffer_print_entry_header(struct trace_seq *s) #define RB_ALIGN_DATA __aligned(RB_ARCH_ALIGNMENT) /* define RINGBUF_TYPE_DATA for 'case RINGBUF_TYPE_DATA:' */ #define RINGBUF_TYPE_DATA 0 ... RINGBUF_TYPE_DATA_TYPE_LEN_MAX enum { RB_LEN_TIME_EXTEND = 8, - RB_LEN_TIME_STAMP = 16, + RB_LEN_TIME_STAMP = 8, }; #define skip_time_extend(event) \ ((struct ring_buffer_event *)((char *)event + RB_LEN_TIME_EXTEND)) +#define extended_time(event) \ + (event->type_len >= RINGBUF_TYPE_TIME_EXTEND) + static inline int rb_null_event(struct ring_buffer_event *event) { return event->type_len == RINGBUF_TYPE_PADDING && !event->time_delta; } static void rb_event_set_padding(struct ring_buffer_event *event) { /* padding has a NULL time_delta */ @@ -204,17 +209,17 @@ rb_event_length(struct ring_buffer_event *event) * Return total length of time extend and data, * or just the event length for all other events. */ static inline unsigned rb_event_ts_length(struct ring_buffer_event *event) { unsigned len = 0; - if (event->type_len == RINGBUF_TYPE_TIME_EXTEND) { + if (extended_time(event)) { /* time extends include the data event after it */ len = RB_LEN_TIME_EXTEND; event = skip_time_extend(event); } return len + rb_event_length(event); } /** @@ -226,34 +231,34 @@ rb_event_ts_length(struct ring_buffer_event *event) * returns the size of the event itself. With the exception * of a TIME EXTEND, where it still returns the size of the * data load of the data event after it. */ unsigned ring_buffer_event_length(struct ring_buffer_event *event) { unsigned length; - if (event->type_len == RINGBUF_TYPE_TIME_EXTEND) + if (extended_time(event)) event = skip_time_extend(event); length = rb_event_length(event); if (event->type_len > RINGBUF_TYPE_DATA_TYPE_LEN_MAX) return length; length -= RB_EVNT_HDR_SIZE; if (length > RB_MAX_SMALL_DATA + sizeof(event->array[0])) length -= sizeof(event->array[0]); return length; } EXPORT_SYMBOL_GPL(ring_buffer_event_length); /* inline for ring buffer fast paths */ static __always_inline void * rb_event_data(struct ring_buffer_event *event) { - if (event->type_len == RINGBUF_TYPE_TIME_EXTEND) + if (extended_time(event)) event = skip_time_extend(event); BUG_ON(event->type_len > RINGBUF_TYPE_DATA_TYPE_LEN_MAX); /* If length is in len field, then array[0] has the data */ if (event->type_len) return (void *)&event->array[0]; /* Otherwise length is in array[0] and array[1] has the data */ return (void *)&event->array[1]; } @@ -270,16 +275,37 @@ EXPORT_SYMBOL_GPL(ring_buffer_event_data); #define for_each_buffer_cpu(buffer, cpu) \ for_each_cpu(cpu, buffer->cpumask) #define TS_SHIFT 27 #define TS_MASK ((1ULL << TS_SHIFT) - 1) #define TS_DELTA_TEST (~TS_MASK) +/** + * ring_buffer_event_time_stamp - return the event's extended timestamp + * @event: the event to get the timestamp of + * + * Returns the extended timestamp associated with a data event. + * An extended time_stamp is a 64-bit timestamp represented + * internally in a special way that makes the best use of space + * contained within a ring buffer event. This function decodes + * it and maps it to a straight u64 value. + */ +u64 ring_buffer_event_time_stamp(struct ring_buffer_event *event) +{ + u64 ts; + + ts = event->array[0]; + ts <<= TS_SHIFT; + ts += event->time_delta; + + return ts; +} + /* Flag when events were overwritten */ #define RB_MISSED_EVENTS (1 << 31) /* Missed count stored at end */ #define RB_MISSED_STORED (1 << 30) #define RB_MISSED_FLAGS (RB_MISSED_EVENTS|RB_MISSED_STORED) struct buffer_data_page { @@ -2223,22 +2249,25 @@ rb_move_tail(struct ring_buffer_per_cpu *cpu_buffer, /* reset write */ rb_reset_tail(cpu_buffer, tail, info); return NULL; } /* Slow path, do not inline */ static noinline struct ring_buffer_event * -rb_add_time_stamp(struct ring_buffer_event *event, u64 delta) +rb_add_time_stamp(struct ring_buffer_event *event, u64 delta, bool abs) { - event->type_len = RINGBUF_TYPE_TIME_EXTEND; + if (abs) + event->type_len = RINGBUF_TYPE_TIME_STAMP; + else + event->type_len = RINGBUF_TYPE_TIME_EXTEND; - /* Not the first event on the page? */ - if (rb_event_index(event)) { + /* Not the first event on the page, or not delta? */ + if (abs || rb_event_index(event)) { event->time_delta = delta & TS_MASK; event->array[0] = delta >> TS_SHIFT; } else { /* nope, just zero it */ event->time_delta = 0; event->array[0] = 0; } @@ -2271,17 +2300,19 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer, if (unlikely(!rb_event_is_commit(cpu_buffer, event))) delta = 0; /* * If we need to add a timestamp, then we * add it to the start of the resevered space. */ if (unlikely(info->add_timestamp)) { - event = rb_add_time_stamp(event, delta); + bool abs = ring_buffer_time_stamp_abs(cpu_buffer->buffer); + + event = rb_add_time_stamp(event, info->delta, abs); length -= RB_LEN_TIME_EXTEND; delta = 0; } event->time_delta = delta; length -= RB_EVNT_HDR_SIZE; if (length > RB_MAX_SMALL_DATA || RB_FORCE_8BYTE_ALIGNMENT) { event->type_len = 0; @@ -2459,17 +2490,17 @@ static __always_inline void rb_end_commit(struct ring_buffer_per_cpu *cpu_buffer !local_read(&cpu_buffer->committing)) { local_inc(&cpu_buffer->committing); goto again; } } static inline void rb_event_discard(struct ring_buffer_event *event) { - if (event->type_len == RINGBUF_TYPE_TIME_EXTEND) + if (extended_time(event)) event = skip_time_extend(event); /* array[0] holds the actual length for the discarded event */ event->array[0] = rb_event_data_length(event) - RB_EVNT_HDR_SIZE; event->type_len = RINGBUF_TYPE_PADDING; /* time delta must be non zero */ if (!event->time_delta) event->time_delta = 1; @@ -2503,20 +2534,21 @@ rb_update_write_stamp(struct ring_buffer_per_cpu *cpu_buffer, /* * A commit event that is first on a page * updates the write timestamp with the page stamp */ if (!rb_event_index(event)) cpu_buffer->write_stamp = cpu_buffer->commit_page->page->time_stamp; else if (event->type_len == RINGBUF_TYPE_TIME_EXTEND) { - delta = event->array[0]; - delta <<= TS_SHIFT; - delta += event->time_delta; + delta = ring_buffer_event_time_stamp(event); cpu_buffer->write_stamp += delta; + } else if (event->type_len == RINGBUF_TYPE_TIME_STAMP) { + delta = ring_buffer_event_time_stamp(event); + cpu_buffer->write_stamp = delta; } else cpu_buffer->write_stamp += event->time_delta; } } static void rb_commit(struct ring_buffer_per_cpu *cpu_buffer, struct ring_buffer_event *event) { @@ -2686,17 +2718,17 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, /* set write to only the index of the write */ write &= RB_WRITE_MASK; tail = write - info->length; /* * If this is the first commit on the page, then it has the same * timestamp as the page itself. */ - if (!tail) + if (!tail && !ring_buffer_time_stamp_abs(cpu_buffer->buffer)) info->delta = 0; /* See if we shot pass the end of this buffer page */ if (unlikely(write > BUF_PAGE_SIZE)) return rb_move_tail(cpu_buffer, tail, info); /* We reserved something on the buffer */ @@ -2763,18 +2795,21 @@ rb_reserve_next_event(struct ring_buffer *buffer, goto out_fail; info.ts = rb_time_stamp(cpu_buffer->buffer); diff = info.ts - cpu_buffer->write_stamp; /* make sure this diff is calculated here */ barrier(); - /* Did the write stamp get updated already? */ - if (likely(info.ts >= cpu_buffer->write_stamp)) { + if (ring_buffer_time_stamp_abs(buffer)) { + info.delta = info.ts; + rb_handle_timestamp(cpu_buffer, &info); + } else /* Did the write stamp get updated already? */ + if (likely(info.ts >= cpu_buffer->write_stamp)) { info.delta = diff; if (unlikely(test_time_stamp(info.delta))) rb_handle_timestamp(cpu_buffer, &info); } event = __rb_reserve_next(cpu_buffer, &info); if (unlikely(PTR_ERR(event) == -EAGAIN)) { @@ -3462,24 +3497,23 @@ rb_update_read_stamp(struct ring_buffer_per_cpu *cpu_buffer, { u64 delta; switch (event->type_len) { case RINGBUF_TYPE_PADDING: return; case RINGBUF_TYPE_TIME_EXTEND: - delta = event->array[0]; - delta <<= TS_SHIFT; - delta += event->time_delta; + delta = ring_buffer_event_time_stamp(event); cpu_buffer->read_stamp += delta; return; case RINGBUF_TYPE_TIME_STAMP: - /* FIXME: not implemented */ + delta = ring_buffer_event_time_stamp(event); + cpu_buffer->read_stamp = delta; return; case RINGBUF_TYPE_DATA: cpu_buffer->read_stamp += event->time_delta; return; default: BUG(); @@ -3493,24 +3527,23 @@ rb_update_iter_read_stamp(struct ring_buffer_iter *iter, { u64 delta; switch (event->type_len) { case RINGBUF_TYPE_PADDING: return; case RINGBUF_TYPE_TIME_EXTEND: - delta = event->array[0]; - delta <<= TS_SHIFT; - delta += event->time_delta; + delta = ring_buffer_event_time_stamp(event); iter->read_stamp += delta; return; case RINGBUF_TYPE_TIME_STAMP: - /* FIXME: not implemented */ + delta = ring_buffer_event_time_stamp(event); + iter->read_stamp = delta; return; case RINGBUF_TYPE_DATA: iter->read_stamp += event->time_delta; return; default: BUG(); @@ -3724,16 +3757,18 @@ static int rb_lost_events(struct ring_buffer_per_cpu *cpu_buffer) static struct ring_buffer_event * rb_buffer_peek(struct ring_buffer_per_cpu *cpu_buffer, u64 *ts, unsigned long *lost_events) { struct ring_buffer_event *event; struct buffer_page *reader; int nr_loops = 0; + if (ts) + *ts = 0; again: /* * We repeat when a time extend is encountered. * Since the time extend is always attached to a data event, * we should never loop more than once. * (We never hit the following condition more than twice). */ if (RB_WARN_ON(cpu_buffer, ++nr_loops > 2)) @@ -3760,22 +3795,27 @@ rb_buffer_peek(struct ring_buffer_per_cpu *cpu_buffer, u64 *ts, return event; case RINGBUF_TYPE_TIME_EXTEND: /* Internal data, OK to advance */ rb_advance_reader(cpu_buffer); goto again; case RINGBUF_TYPE_TIME_STAMP: - /* FIXME: not implemented */ + if (ts) { + *ts = ring_buffer_event_time_stamp(event); + ring_buffer_normalize_time_stamp(cpu_buffer->buffer, + cpu_buffer->cpu, ts); + } + /* Internal data, OK to advance */ rb_advance_reader(cpu_buffer); goto again; case RINGBUF_TYPE_DATA: - if (ts) { + if (ts && !(*ts)) { *ts = cpu_buffer->read_stamp + event->time_delta; ring_buffer_normalize_time_stamp(cpu_buffer->buffer, cpu_buffer->cpu, ts); } if (lost_events) *lost_events = rb_lost_events(cpu_buffer); return event; @@ -3790,16 +3830,19 @@ EXPORT_SYMBOL_GPL(ring_buffer_peek); static struct ring_buffer_event * rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts) { struct ring_buffer *buffer; struct ring_buffer_per_cpu *cpu_buffer; struct ring_buffer_event *event; int nr_loops = 0; + if (ts) + *ts = 0; + cpu_buffer = iter->cpu_buffer; buffer = cpu_buffer->buffer; /* * Check if someone performed a consuming read to * the buffer. A consuming read invalidates the iterator * and we need to reset the iterator in this case. */ @@ -3842,22 +3885,27 @@ rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts) return event; case RINGBUF_TYPE_TIME_EXTEND: /* Internal data, OK to advance */ rb_advance_iter(iter); goto again; case RINGBUF_TYPE_TIME_STAMP: - /* FIXME: not implemented */ + if (ts) { + *ts = ring_buffer_event_time_stamp(event); + ring_buffer_normalize_time_stamp(cpu_buffer->buffer, + cpu_buffer->cpu, ts); + } + /* Internal data, OK to advance */ rb_advance_iter(iter); goto again; case RINGBUF_TYPE_DATA: - if (ts) { + if (ts && !(*ts)) { *ts = iter->read_stamp + event->time_delta; ring_buffer_normalize_time_stamp(buffer, cpu_buffer->cpu, ts); } return event; default: BUG();