mbox series

[v4,00/20] libtraceeval histogram: Updates

Message ID 20230817204528.114577-1-rostedt@goodmis.org (mailing list archive)
Headers show
Series libtraceeval histogram: Updates | expand

Message

Steven Rostedt Aug. 17, 2023, 8:45 p.m. UTC 
From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

This patch set is based on top of:

 https://lore.kernel.org/all/20230809175340.3066-1-stevie.6strings@gmail.com/

I added a sample program task-eval which is one of the tools that will be
using this library. The first patch adds task-eval but that is still using
the old API (defined in trace-analysis.c).

The next patches modify the new API to fit with the use case of task-eval.
One is to use "pointer" as I'm not sure exactly the usecase of the dynamic
structure.

The cmp and release callbacks are changed to be more generic, and they get
called if they simply exist for a given type. I can imagine wanting a
release function for event the most mundane types (like number_32).

The cmp was also updated to pass in the traceeval descriptor, as I found
that I needed access to it while doing a compare (although, I rewrote the
code a bit where that use case isn't in the tool anymore).

Some fixes were made to the query.

I also did a bit of code restructuring and add the hash and iterator logic.

The last patch updates the sample code task-eval.c and has it give pretty
much the same logic as the original.

That sample could be updated to implement the code consolidation that Ross
suggested. I may do that later.

Happy programming!


Changes since v3: https://lore.kernel.org/all/20230817013310.88582-1-rostedt@goodmis.org/

- Added patch to reordered the parameters of copy_traceeval_data() and renamed
  copy to dst and orig to src to match that of memcpy() and strcpy().

- Added patch to rename copy_traceeval_data_set() to dup_traceeval_data_set()
  to not have the parameter order to be confusing with the typical copy order.

- Fixed comment of traceeval_remove() (Ross Zwisler)

Changes since v2: https://lore.kernel.org/all/20230811053940.1408424-1-rostedt@goodmis.org/

  - Fixed comments on traceeval_dyn_release_fn and traceeval_dyn_cmp_fn (Ross Zwisler)

  - Change parameter ordering of compare_traceeval_data (Ross Zwisler)

  - Added (size_t) to offset_of() macro (Ross Zwisler)

  - Fixed iter->current_bucket off by one bug (Ross Zwisler)

  - Removed HASH_MASK() from make_hash()

  - Fixed goto fail_release in traceeval_init() (Ross Zwisler)

  - Free sort and direction in traceeval_iterator_put() (Ross Zwisler)

  - Use num_levels variable instead of (level + 1) (Ross Zwisler)

  - Return 0 on success of update_entry (Ross Zwisler)

  - Fixed the cleanup of old values in update_entry() (Ross Zwisler)

  - Fixed resetting iter->next = 0 in traceeval_iterator_sort_custom() (Ross Zwisler)

  - Added failure checks to task-eval.c sample for all traceeval_query() calls.

  - Added traceeval_remove() to remove an item from traceveal (opposite of traceeval_insert())

Changes since v1: https://lore.kernel.org/all/20230809031313.1298605-1-rostedt@goodmis.org/

 - Lots!

   - Converted to using a hash table

   - Removed the unused compare code. With the other updates, it was
     taking too much time to keep updating them.

   - Added checks and labels to the types to have them know what type
     they are, and index they are at.

   - Added stat logic

   - Added iterator logic

   - Have a working sample with the new code!

Steven Rostedt (Google) (20):
  libtraceeval histograms: Fix traceeval_results_release() error message
  libtraceeval: Reverse params of copy_traceeval_data()
  libtraceeval: Rename copy_traceeval_data_set() to dup_traceeval_data()
  libtraceeval: Add sample task-eval program
  libtraceeval hist: Add pointer and const string types
  libtraceeval histogram: Have cmp and release functions be generic
  libtraceeval histograms: Add traceeval struct to compare function
  libtraceeval histogram: Remove comparing of traceeval and types
  libtraceeval: Convert hist array into a hash table
  libtraceeval histograms: Move hash functions into their own file
  libtraceeval histogram: Label and check keys and values
  libtraceeval histogram: Add updating of stats
  libtraceeval histogram: Add iterator APIs
  libtraceeval histogram: Add data copy callback
  libtraceeval histogram: Do the release on updates
  libtraceeval histogram: Use stack for old copy in update
  libtraceeval histogram: Add traceeval_iterator_sort_custom()
  libtraceeval histogram: Have traceeval_query() just give the pointer
    to results
  libtraceeval samples: Update task-eval to use the histogram logic
  libtraceeval: Add traceeval_remove()

 Makefile                 |   4 +-
 include/traceeval-hist.h | 117 +++--
 include/traceeval-test.h |  16 -
 samples/Makefile         |  29 ++
 samples/task-eval.c      | 966 +++++++++++++++++++++++++++++++++++++++
 src/Makefile             |   1 +
 src/eval-local.h         | 123 +++++
 src/hash.c               | 123 +++++
 src/histograms.c         | 945 ++++++++++++++++++++++++++++----------
 9 files changed, 2023 insertions(+), 301 deletions(-)
 delete mode 100644 include/traceeval-test.h
 create mode 100644 samples/Makefile
 create mode 100644 samples/task-eval.c
 create mode 100644 src/eval-local.h
 create mode 100644 src/hash.c

Comments

Steven Rostedt Aug. 17, 2023, 8:48 p.m. UTC | #1 
On Thu, 17 Aug 2023 16:45:08 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> Changes since v3: https://lore.kernel.org/all/20230817013310.88582-1-rostedt@goodmis.org/

The diff between this version and the last version:

diff --git a/include/traceeval-hist.h b/include/traceeval-hist.h
index 8e5a6451f399..0cee9bcbeb83 100644
--- a/include/traceeval-hist.h
+++ b/include/traceeval-hist.h
@@ -83,8 +83,8 @@ typedef int (*traceeval_data_hash_fn)(struct traceeval *teval,
 				      const union traceeval_data *data);
 
 typedef int (*traceeval_data_copy_fn)(const struct traceeval_type *type,
-				      union traceeval_data *copy,
-				      const union traceeval_data *origin);
+				      union traceeval_data *dst,
+				      const union traceeval_data *src);
 
 typedef int (*traceeval_cmp_fn)(struct traceeval *teval,
 				const union traceeval_data *Akeys,
diff --git a/src/histograms.c b/src/histograms.c
index f35d1b2e583d..96f0926f062c 100644
--- a/src/histograms.c
+++ b/src/histograms.c
@@ -500,65 +500,65 @@ static int get_entry(struct traceeval *teval, const union traceeval_data *keys,
 }
 
 /*
- * Copy @orig to @copy with respect to @type.
+ * Copy @src to @dst with respect to @type.
  *
  * Return 0 on success, -1 on error.
  */
 static int copy_traceeval_data(struct traceeval_type *type,
 			       struct traceeval_stat *stat,
-			       const union traceeval_data *orig,
-			       union traceeval_data *copy)
+			       union traceeval_data *dst,
+			       const union traceeval_data *src)
 {
 	unsigned long long val;
 
 	if (type->copy)
-		return type->copy(type, copy, orig);
+		return type->copy(type, dst, src);
 
-	*copy = *orig;
+	*dst = *src;
 
 	switch(type->type) {
 	case TRACEEVAL_TYPE_NUMBER:
 		if (type->flags & TRACEEVAL_FL_SIGNED)
-			val = (long long)copy->number;
+			val = (long long)dst->number;
 		else
-			val = (unsigned long long)copy->number;
+			val = (unsigned long long)dst->number;
 		break;
 
 	case TRACEEVAL_TYPE_NUMBER_64:
 		if (type->flags & TRACEEVAL_FL_SIGNED)
-			val = (long long)copy->number_64;
+			val = (long long)dst->number_64;
 		else
-			val = (unsigned long long)copy->number_64;
+			val = (unsigned long long)dst->number_64;
 		break;
 
 	case TRACEEVAL_TYPE_NUMBER_32:
 		if (type->flags & TRACEEVAL_FL_SIGNED)
-			val = (long long)copy->number_32;
+			val = (long long)dst->number_32;
 		else
-			val = (unsigned long long)copy->number_32;
+			val = (unsigned long long)dst->number_32;
 		break;
 
 	case TRACEEVAL_TYPE_NUMBER_16:
 		if (type->flags & TRACEEVAL_FL_SIGNED)
-			val = (long long)copy->number_16;
+			val = (long long)dst->number_16;
 		else
-			val = (unsigned long long)copy->number_16;
+			val = (unsigned long long)dst->number_16;
 		break;
 
 	case TRACEEVAL_TYPE_NUMBER_8:
 		if (type->flags & TRACEEVAL_FL_SIGNED)
-			val = (long long)copy->number_8;
+			val = (long long)dst->number_8;
 		else
-			val = (unsigned long long)copy->number_8;
+			val = (unsigned long long)dst->number_8;
 		break;
 
 	case TRACEEVAL_TYPE_STRING:
-		copy->string = NULL;
+		dst->string = NULL;
 
-		if (orig->string)
-			copy->string = strdup(orig->string);
+		if (src->string)
+			dst->string = strdup(src->string);
 
-		if (!copy->string)
+		if (!dst->string)
 			return -1;
 		return 0;
 	default:
@@ -619,14 +619,14 @@ static void data_release_and_free(size_t size, union traceeval_data **data,
 }
 
 /*
- * Copy @orig to @copy with respect to @size and @type.
+ * Duplicate a traceeval_data @orig into an newly allocated @copy.
  *
  * Returns 1 on success, -1 on error.
  */
-static int copy_traceeval_data_set(size_t size, struct traceeval_type *type,
-				    const union traceeval_data *orig,
-				    struct traceeval_stat *stats,
-				    union traceeval_data **copy)
+static int dup_traceeval_data_set(size_t size, struct traceeval_type *type,
+				  struct traceeval_stat *stats,
+				  const union traceeval_data *orig,
+				  union traceeval_data **copy)
 {
 	size_t i;
 
@@ -640,7 +640,7 @@ static int copy_traceeval_data_set(size_t size, struct traceeval_type *type,
 
 	for (i = 0; i < size; i++) {
 		if (copy_traceeval_data(type + i, stats ? stats + i : NULL,
-					orig + i, (*copy) + i))
+					 (*copy) + i, orig + i))
 			goto fail;
 	}
 
@@ -745,13 +745,13 @@ static int create_entry(struct traceeval *teval,
 		goto fail_entry;
 
 	/* copy keys */
-	if (copy_traceeval_data_set(teval->nr_key_types, teval->key_types,
-				keys, NULL, &new_keys) == -1)
+	if (dup_traceeval_data_set(teval->nr_key_types, teval->key_types,
+				   NULL, keys, &new_keys) == -1)
 		goto fail_stats;
 
 	/* copy vals */
-	if (copy_traceeval_data_set(teval->nr_val_types, teval->val_types,
-				vals, entry->val_stats, &new_vals) == -1)
+	if (dup_traceeval_data_set(teval->nr_val_types, teval->val_types,
+				   entry->val_stats, vals, &new_vals) == -1)
 		goto fail;
 
 	entry->keys = new_keys;
@@ -794,7 +794,7 @@ static int update_entry(struct traceeval *teval, struct entry *entry,
 		old[i] = copy[i];
 
 		if (copy_traceeval_data(types + i, stats + i,
-					vals + i, copy + i))
+					copy + i, vals + i))
 			goto fail;
 	}
 	data_release(size, old, types);
@@ -936,7 +936,7 @@ int traceeval_insert(struct traceeval *teval,
 
 /**
  * traceeval_remove - remove an item from the traceeval descriptor
- * @teval: The descriptor to insert into
+ * @teval: The descriptor to removed from
  * @keys: The list of keys that defines what is being removed
  *
  * This is the opposite of traceeval_insert(). Instead of inserting


-- Steve
Ross Zwisler Aug. 17, 2023, 8:59 p.m. UTC | #2 
On Thu, Aug 17, 2023 at 04:45:08PM -0400, Steven Rostedt wrote:
> From: "Steven Rostedt (Google)" <rostedt@goodmis.org>
> 
> This patch set is based on top of:
> 
>  https://lore.kernel.org/all/20230809175340.3066-1-stevie.6strings@gmail.com/
> 
> I added a sample program task-eval which is one of the tools that will be
> using this library. The first patch adds task-eval but that is still using
> the old API (defined in trace-analysis.c).
> 
> The next patches modify the new API to fit with the use case of task-eval.
> One is to use "pointer" as I'm not sure exactly the usecase of the dynamic
> structure.
> 
> The cmp and release callbacks are changed to be more generic, and they get
> called if they simply exist for a given type. I can imagine wanting a
> release function for event the most mundane types (like number_32).
> 
> The cmp was also updated to pass in the traceeval descriptor, as I found
> that I needed access to it while doing a compare (although, I rewrote the
> code a bit where that use case isn't in the tool anymore).
> 
> Some fixes were made to the query.
> 
> I also did a bit of code restructuring and add the hash and iterator logic.
> 
> The last patch updates the sample code task-eval.c and has it give pretty
> much the same logic as the original.
> 
> That sample could be updated to implement the code consolidation that Ross
> suggested. I may do that later.
> 
> Happy programming!

Looks good, you can add:

Reviewed-by: Ross Zwisler <zwisler@google.com>
> 
> Changes since v3: https://lore.kernel.org/all/20230817013310.88582-1-rostedt@goodmis.org/
> 
> - Added patch to reordered the parameters of copy_traceeval_data() and renamed
>   copy to dst and orig to src to match that of memcpy() and strcpy().
> 
> - Added patch to rename copy_traceeval_data_set() to dup_traceeval_data_set()
>   to not have the parameter order to be confusing with the typical copy order.
> 
> - Fixed comment of traceeval_remove() (Ross Zwisler)
> 
> Changes since v2: https://lore.kernel.org/all/20230811053940.1408424-1-rostedt@goodmis.org/
> 
>   - Fixed comments on traceeval_dyn_release_fn and traceeval_dyn_cmp_fn (Ross Zwisler)
> 
>   - Change parameter ordering of compare_traceeval_data (Ross Zwisler)
> 
>   - Added (size_t) to offset_of() macro (Ross Zwisler)
> 
>   - Fixed iter->current_bucket off by one bug (Ross Zwisler)
> 
>   - Removed HASH_MASK() from make_hash()
> 
>   - Fixed goto fail_release in traceeval_init() (Ross Zwisler)
> 
>   - Free sort and direction in traceeval_iterator_put() (Ross Zwisler)
> 
>   - Use num_levels variable instead of (level + 1) (Ross Zwisler)
> 
>   - Return 0 on success of update_entry (Ross Zwisler)
> 
>   - Fixed the cleanup of old values in update_entry() (Ross Zwisler)
> 
>   - Fixed resetting iter->next = 0 in traceeval_iterator_sort_custom() (Ross Zwisler)
> 
>   - Added failure checks to task-eval.c sample for all traceeval_query() calls.
> 
>   - Added traceeval_remove() to remove an item from traceveal (opposite of traceeval_insert())
> 
> Changes since v1: https://lore.kernel.org/all/20230809031313.1298605-1-rostedt@goodmis.org/
> 
>  - Lots!
> 
>    - Converted to using a hash table
> 
>    - Removed the unused compare code. With the other updates, it was
>      taking too much time to keep updating them.
> 
>    - Added checks and labels to the types to have them know what type
>      they are, and index they are at.
> 
>    - Added stat logic
> 
>    - Added iterator logic
> 
>    - Have a working sample with the new code!
> 
> Steven Rostedt (Google) (20):
>   libtraceeval histograms: Fix traceeval_results_release() error message
>   libtraceeval: Reverse params of copy_traceeval_data()
>   libtraceeval: Rename copy_traceeval_data_set() to dup_traceeval_data()
>   libtraceeval: Add sample task-eval program
>   libtraceeval hist: Add pointer and const string types
>   libtraceeval histogram: Have cmp and release functions be generic
>   libtraceeval histograms: Add traceeval struct to compare function
>   libtraceeval histogram: Remove comparing of traceeval and types
>   libtraceeval: Convert hist array into a hash table
>   libtraceeval histograms: Move hash functions into their own file
>   libtraceeval histogram: Label and check keys and values
>   libtraceeval histogram: Add updating of stats
>   libtraceeval histogram: Add iterator APIs
>   libtraceeval histogram: Add data copy callback
>   libtraceeval histogram: Do the release on updates
>   libtraceeval histogram: Use stack for old copy in update
>   libtraceeval histogram: Add traceeval_iterator_sort_custom()
>   libtraceeval histogram: Have traceeval_query() just give the pointer
>     to results
>   libtraceeval samples: Update task-eval to use the histogram logic
>   libtraceeval: Add traceeval_remove()
> 
>  Makefile                 |   4 +-
>  include/traceeval-hist.h | 117 +++--
>  include/traceeval-test.h |  16 -
>  samples/Makefile         |  29 ++
>  samples/task-eval.c      | 966 +++++++++++++++++++++++++++++++++++++++
>  src/Makefile             |   1 +
>  src/eval-local.h         | 123 +++++
>  src/hash.c               | 123 +++++
>  src/histograms.c         | 945 ++++++++++++++++++++++++++++----------
>  9 files changed, 2023 insertions(+), 301 deletions(-)
>  delete mode 100644 include/traceeval-test.h
>  create mode 100644 samples/Makefile
>  create mode 100644 samples/task-eval.c
>  create mode 100644 src/eval-local.h
>  create mode 100644 src/hash.c
> 
> -- 
> 2.40.1
>