[v2,1/3] livepach: Add .livepatch.hooks functions and test-case

Commit Message

Konrad Rzeszutek Wilk Aug. 4, 2016, 3:49 p.m. UTC

From: Ross Lagerwall <[e-mail]>

Add hook functions which run during patch apply and patch revert.
Hook functions are used by livepatch payloads to manipulate data
structures during patching, etc.

One use case is the XSA91. As Martin mentions it:
"If we have shadow variables, we also need an unload hook to garbage
collect all the variables introduced by a hotpatch to prevent memory
leaks.  Potentially, we also want to pre-reserve memory for static or
existing dynamic objects in the load-hook instead of on the fly.

For testing and debugging, various applications are possible.

In general, the hooks provide flexibility when having to deal with
unforeseen cases, but their application should be rarely required (<
10%)."

Furthermore include a test-case for it.

Signed-off-by: Ross Lagerwall <[e-mail]>
Signed-off-by: Konrad Rzeszutek Wilk <[e-mail]>
---
v12: s/xsplice/livepatch/

Cc: Andrew Cooper <[e-mail]>
Cc: George Dunlap <[e-mail]>
Cc: Ian Jackson <[e-mail]>
Cc: Jan Beulich <[e-mail]>
Cc: Stefano Stabellini <[e-mail]>
Cc: Tim Deegan <[e-mail]>
Cc: Wei Liu <[e-mail]>
---
 docs/misc/livepatch.markdown        | 23 +++++++++++++++++
 xen/arch/x86/test/xen_hello_world.c | 36 ++++++++++++++++++++++++++
 xen/common/livepatch.c              | 50 ++++++++++++++++++++++++++++++++++++-
 xen/include/xen/livepatch_payload.h | 49 ++++++++++++++++++++++++++++++++++++
 4 files changed, 157 insertions(+), 1 deletion(-)
 create mode 100644 xen/include/xen/livepatch_payload.h

Comments

Jan Beulich Aug. 5, 2016, 3:35 p.m. UTC | #1

>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
> In general, the hooks provide flexibility when having to deal with
> unforeseen cases, but their application should be rarely required (<
> 10%)."

But the greater flexibility of course comes with increased chances
of screwing things up. I'm therefore still not entirely convinced that
such XSAs wouldn't then better not be live patched.

> --- a/xen/arch/x86/test/xen_hello_world.c
> +++ b/xen/arch/x86/test/xen_hello_world.c
> @@ -4,14 +4,50 @@
>   */
>  
>  #include "config.h"
> +#include <xen/lib.h>
>  #include <xen/types.h>
>  #include <xen/version.h>
>  #include <xen/livepatch.h>
> +#include <xen/livepatch_payload.h>
>  
>  #include <public/sysctl.h>
>  
>  static char hello_world_patch_this_fnc[] = "xen_extra_version";
>  extern const char *xen_hello_world(void);
> +static unsigned int cnt;
> +
> +static void apply_hook(void)
> +{
> +    printk(KERN_DEBUG "Hook executing.\n");
> +}
> +
> +static void revert_hook(void)
> +{
> +    printk(KERN_DEBUG "Hook unloaded.\n");
> +}
> +
> +static void hi_func(void)
> +{
> +    printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
> +};
> +
> +/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
> +static void Z_check_fnc(void)

And that Z_ prefix is supposed to guarantee that being last? Isn't
it that upper case letters sort before lower case ones?

> +{
> +    printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
> +    BUG_ON(cnt == 0 || cnt > 2);
> +    cnt = 0; /* Otherwise if you revert, apply, revert the value will be 4! */

Isn't this an indication of a general weakness: Shouldn't a patch
module be allowed to assume it's being run in a clean state, i.e.
without any of its static data altered from their load time values?

> @@ -70,7 +71,11 @@ struct payload {
>      unsigned int nsyms;                  /* Nr of entries in .strtab and 
> symbols. */
>      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
>      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
> -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
> +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */

These both seem like they want a const in the middle.

> @@ -515,6 +520,27 @@ static int prepare_payload(struct payload *payload,
>          }
>      }
>  
> +    sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.load");

Is the .hooks infix really needed?

> +    if ( sec )
> +    {
> +        if ( !sec->sec->sh_size ||

What's wrong with a zero size section (holding no hooks)? We've
had that same case elsewhere in the original series ...

> @@ -999,6 +1025,17 @@ static int apply_payload(struct payload *data)
>  
>      arch_livepatch_quiesce();
>  
> +    /*
> +     * The hooks may call common code which expects spinlocks to be certain
> +     * type, as such disable this temporarily.
> +     */
> +    spin_debug_disable();
> +    for ( i = 0; i < data->n_load_funcs; i++ )
> +        data->load_funcs[i]();
> +    spin_debug_enable();

I have to admit that I can't really make sense of the comment, and
hence the code.

> --- /dev/null
> +++ b/xen/include/xen/livepatch_payload.h
> @@ -0,0 +1,49 @@
> +/*
> + * Copyright (C) 2016 Citrix Systems R&D Ltd.
> + */
> +
> +#ifndef __XEN_LIVEPATCH_PAYLOAD_H__
> +#define __XEN_LIVEPATCH_PAYLOAD_H__
> +
> +/*
> + * The following definitions are to be used in patches. They are taken
> + * from kpatch.
> + */
> +typedef void livepatch_loadcall_t(void);
> +typedef void livepatch_unloadcall_t(void);
> +
> +/*
> + * LIVEPATCH_LOAD_HOOK macro
> + *
> + * Declares a function pointer to be allocated in a new
> + * .livepatch.hook.load section.  This livepatch_load_data symbol is later
> + * stripped by create-diff-object so that it can be declared in multiple
> + * objects that are later linked together, avoiding global symbol
> + * collision.  Since multiple hooks can be registered, the
> + * .livepatch.hook.load section is a table of functions that will be
> + * executed in series by the livepatch infrastructure at patch load time.
> + */
> +#define LIVEPATCH_LOAD_HOOK(_fn) \
> +    livepatch_loadcall_t *__attribute__((weak)) \
> +        livepatch_load_data_##_fn __section(".livepatch.hooks.load") = _fn;

There's again a const desirable here, to render the section r/o.

Jan

Konrad Rzeszutek Wilk Aug. 5, 2016, 9:08 p.m. UTC | #2

On Fri, Aug 05, 2016 at 09:35:49AM -0600, Jan Beulich wrote:
> >>> On 04.08.16 at 17:49, <[e-mail]> wrote:
> > In general, the hooks provide flexibility when having to deal with
> > unforeseen cases, but their application should be rarely required (<
> > 10%)."
> 
> But the greater flexibility of course comes with increased chances
> of screwing things up. I'm therefore still not entirely convinced that
> such XSAs wouldn't then better not be live patched.
> 
> > --- a/xen/arch/x86/test/xen_hello_world.c
> > +++ b/xen/arch/x86/test/xen_hello_world.c
> > @@ -4,14 +4,50 @@
> >   */
> >  
> >  #include "config.h"
> > +#include <xen/lib.h>
> >  #include <xen/types.h>
> >  #include <xen/version.h>
> >  #include <xen/livepatch.h>
> > +#include <xen/livepatch_payload.h>
> >  
> >  #include <public/sysctl.h>
> >  
> >  static char hello_world_patch_this_fnc[] = "xen_extra_version";
> >  extern const char *xen_hello_world(void);
> > +static unsigned int cnt;
> > +
> > +static void apply_hook(void)
> > +{
> > +    printk(KERN_DEBUG "Hook executing.\n");
> > +}
> > +
> > +static void revert_hook(void)
> > +{
> > +    printk(KERN_DEBUG "Hook unloaded.\n");
> > +}
> > +
> > +static void hi_func(void)
> > +{
> > +    printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
> > +};
> > +
> > +/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
> > +static void Z_check_fnc(void)
> 
> And that Z_ prefix is supposed to guarantee that being last? Isn't
> it that upper case letters sort before lower case ones?

Yes. And the code is actually flexible enough not to be the last one.

B/c:
> 
> > +{
> > +    printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
> > +    BUG_ON(cnt == 0 || cnt > 2);
> > +    cnt = 0; /* Otherwise if you revert, apply, revert the value will be 4! */
> 
> Isn't this an indication of a general weakness: Shouldn't a patch
> module be allowed to assume it's being run in a clean state, i.e.
> without any of its static data altered from their load time values?

Of this bug. (which I obviously need to fix).
> 
> > @@ -70,7 +71,11 @@ struct payload {
> >      unsigned int nsyms;                  /* Nr of entries in .strtab and 
> > symbols. */
> >      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
> >      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
> > -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> > +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
> > +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
> 
> These both seem like they want a const in the middle.

I did that initially and found out that the calling ->load_funcs[i] resulted
in _no_ code being called.

I did not dig in the assembler output to figure out what happend, let me
do that now.


> 
> > @@ -515,6 +520,27 @@ static int prepare_payload(struct payload *payload,
> >          }
> >      }
> >  
> > +    sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.load");
> 
> Is the .hooks infix really needed?
> 
> > +    if ( sec )
> > +    {
> > +        if ( !sec->sec->sh_size ||
> 
> What's wrong with a zero size section (holding no hooks)? We've
> had that same case elsewhere in the original series ...

That would be a bit odd (having zero size).

But yes there is nothing wrong with it.  I will fix it up.

> 
> > @@ -999,6 +1025,17 @@ static int apply_payload(struct payload *data)
> >  
> >      arch_livepatch_quiesce();
> >  
> > +    /*
> > +     * The hooks may call common code which expects spinlocks to be certain
> > +     * type, as such disable this temporarily.
> > +     */
> > +    spin_debug_disable();
> > +    for ( i = 0; i < data->n_load_funcs; i++ )
> > +        data->load_funcs[i]();
> > +    spin_debug_enable();
> 
> I have to admit that I can't really make sense of the comment, and
> hence the code.

Perhaps:

Since we are running with IRQs disabled and the hooks may call common
code - which expects the spinlocks to run with IRQs enabled - we temporarly
disable the spin locks "IRQ affinity state"

?
> 
> > --- /dev/null
> > +++ b/xen/include/xen/livepatch_payload.h
> > @@ -0,0 +1,49 @@
> > +/*
> > + * Copyright (C) 2016 Citrix Systems R&D Ltd.
> > + */
> > +
> > +#ifndef __XEN_LIVEPATCH_PAYLOAD_H__
> > +#define __XEN_LIVEPATCH_PAYLOAD_H__
> > +
> > +/*
> > + * The following definitions are to be used in patches. They are taken
> > + * from kpatch.
> > + */
> > +typedef void livepatch_loadcall_t(void);
> > +typedef void livepatch_unloadcall_t(void);
> > +
> > +/*
> > + * LIVEPATCH_LOAD_HOOK macro
> > + *
> > + * Declares a function pointer to be allocated in a new
> > + * .livepatch.hook.load section.  This livepatch_load_data symbol is later
> > + * stripped by create-diff-object so that it can be declared in multiple
> > + * objects that are later linked together, avoiding global symbol
> > + * collision.  Since multiple hooks can be registered, the
> > + * .livepatch.hook.load section is a table of functions that will be
> > + * executed in series by the livepatch infrastructure at patch load time.
> > + */
> > +#define LIVEPATCH_LOAD_HOOK(_fn) \
> > +    livepatch_loadcall_t *__attribute__((weak)) \
> > +        livepatch_load_data_##_fn __section(".livepatch.hooks.load") = _fn;
> 
> There's again a const desirable here, to render the section r/o.

<nods> Let me dig in this. I can't recall when I initially tried
making the livepatch_loadcall_t be const whether I had this as const or not..

Thank you for your quick review!
> 
> Jan

Jan Beulich Aug. 8, 2016, 6:23 a.m. UTC | #3

>>> On 05.08.16 at 23:08, <[e-mail]> wrote:
> On Fri, Aug 05, 2016 at 09:35:49AM -0600, Jan Beulich wrote:
>> >>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>> > In general, the hooks provide flexibility when having to deal with
>> > unforeseen cases, but their application should be rarely required (<
>> > 10%)."
>> 
>> But the greater flexibility of course comes with increased chances
>> of screwing things up. I'm therefore still not entirely convinced that
>> such XSAs wouldn't then better not be live patched.
>> 
>> > --- a/xen/arch/x86/test/xen_hello_world.c
>> > +++ b/xen/arch/x86/test/xen_hello_world.c
>> > @@ -4,14 +4,50 @@
>> >   */
>> >  
>> >  #include "config.h"
>> > +#include <xen/lib.h>
>> >  #include <xen/types.h>
>> >  #include <xen/version.h>
>> >  #include <xen/livepatch.h>
>> > +#include <xen/livepatch_payload.h>
>> >  
>> >  #include <public/sysctl.h>
>> >  
>> >  static char hello_world_patch_this_fnc[] = "xen_extra_version";
>> >  extern const char *xen_hello_world(void);
>> > +static unsigned int cnt;
>> > +
>> > +static void apply_hook(void)
>> > +{
>> > +    printk(KERN_DEBUG "Hook executing.\n");
>> > +}
>> > +
>> > +static void revert_hook(void)
>> > +{
>> > +    printk(KERN_DEBUG "Hook unloaded.\n");
>> > +}
>> > +
>> > +static void hi_func(void)
>> > +{
>> > +    printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
>> > +};
>> > +
>> > +/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
>> > +static void Z_check_fnc(void)
>> 
>> And that Z_ prefix is supposed to guarantee that being last? Isn't
>> it that upper case letters sort before lower case ones?
> 
> Yes. And the code is actually flexible enough not to be the last one.
> 
> B/c:
>> 
>> > +{
>> > +    printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
>> > +    BUG_ON(cnt == 0 || cnt > 2);
>> > +    cnt = 0; /* Otherwise if you revert, apply, revert the value will be 4! */
>> 
>> Isn't this an indication of a general weakness: Shouldn't a patch
>> module be allowed to assume it's being run in a clean state, i.e.
>> without any of its static data altered from their load time values?
> 
> Of this bug. (which I obviously need to fix).

And the fix of which then should perhaps become a prereq to this
change.

>> > @@ -70,7 +71,11 @@ struct payload {
>> >      unsigned int nsyms;                  /* Nr of entries in .strtab and 
>> > symbols. */
>> >      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
>> >      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
>> > -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
>> > +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
>> > +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
>> 
>> These both seem like they want a const in the middle.
> 
> I did that initially and found out that the calling ->load_funcs[i] resulted
> in _no_ code being called.
> 
> I did not dig in the assembler output to figure out what happend, let me
> do that now.

Indeed - I can't see how a const modifier here could alter whether
or not the pointed to function gets called.

>> > +    if ( sec )
>> > +    {
>> > +        if ( !sec->sec->sh_size ||
>> 
>> What's wrong with a zero size section (holding no hooks)? We've
>> had that same case elsewhere in the original series ...
> 
> That would be a bit odd (having zero size).

What's odd there? A simplistic generation tool could simply produce
all sections, no matter whether some of them have no contents.

>> > @@ -999,6 +1025,17 @@ static int apply_payload(struct payload *data)
>> >  
>> >      arch_livepatch_quiesce();
>> >  
>> > +    /*
>> > +     * The hooks may call common code which expects spinlocks to be certain
>> > +     * type, as such disable this temporarily.
>> > +     */
>> > +    spin_debug_disable();
>> > +    for ( i = 0; i < data->n_load_funcs; i++ )
>> > +        data->load_funcs[i]();
>> > +    spin_debug_enable();
>> 
>> I have to admit that I can't really make sense of the comment, and
>> hence the code.
> 
> Perhaps:
> 
> Since we are running with IRQs disabled and the hooks may call common
> code - which expects the spinlocks to run with IRQs enabled - we temporarly
> disable the spin locks "IRQ affinity state"

Apart from that last thing in quotes (I'd at least drop "affinity" and
perhaps make it "IRQ state checks", and then without quotes) this
sounds quite a bit better.

Jan

George Dunlap Aug. 8, 2016, 1:42 p.m. UTC | #4

On 05/08/16 16:35, Jan Beulich wrote:
>>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>> In general, the hooks provide flexibility when having to deal with
>> unforeseen cases, but their application should be rarely required (<
>> 10%)."
> 
> But the greater flexibility of course comes with increased chances
> of screwing things up. I'm therefore still not entirely convinced that
> such XSAs wouldn't then better not be live patched.

But this functionality is optional, right?  So although I tend to agree
with you, we can let the person / organization preparing the patch take
that risk if they want to?

 -George

Jan Beulich Aug. 8, 2016, 2:01 p.m. UTC | #5

>>> On 08.08.16 at 15:42, <[e-mail]> wrote:
> On 05/08/16 16:35, Jan Beulich wrote:
>>>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>>> In general, the hooks provide flexibility when having to deal with
>>> unforeseen cases, but their application should be rarely required (<
>>> 10%)."
>> 
>> But the greater flexibility of course comes with increased chances
>> of screwing things up. I'm therefore still not entirely convinced that
>> such XSAs wouldn't then better not be live patched.
> 
> But this functionality is optional, right?  So although I tend to agree
> with you, we can let the person / organization preparing the patch take
> that risk if they want to?

That's a valid point, albeit I think patch producer and patch consumer
might have different views, and the patch consumer may not know
(or even know to care) whether (s)he's got handed a patch with or
without use of hooks.

Jan

George Dunlap Aug. 8, 2016, 2:10 p.m. UTC | #6

On 08/08/16 15:01, Jan Beulich wrote:
>>>> On 08.08.16 at 15:42, <[e-mail]> wrote:
>> On 05/08/16 16:35, Jan Beulich wrote:
>>>>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>>>> In general, the hooks provide flexibility when having to deal with
>>>> unforeseen cases, but their application should be rarely required (<
>>>> 10%)."
>>>
>>> But the greater flexibility of course comes with increased chances
>>> of screwing things up. I'm therefore still not entirely convinced that
>>> such XSAs wouldn't then better not be live patched.
>>
>> But this functionality is optional, right?  So although I tend to agree
>> with you, we can let the person / organization preparing the patch take
>> that risk if they want to?
> 
> That's a valid point, albeit I think patch producer and patch consumer
> might have different views, and the patch consumer may not know
> (or even know to care) whether (s)he's got handed a patch with or
> without use of hooks.

But the patch consumer probably doesn't know for sure if the patch was
even tested, or if it even fixes the things it was meant to fix.  The
producer / consumer relationship will always have an aspect of trust and
some mechanism for feedback.

I suppose there is a sort of "prisoner's dilemma" aspect to this though:
suppose provider A thinks that hooks are too dangerous, and doesn't
provide a hotfix for a certain XSA as a result, but provider B, seeing
an opportunity, ignores the risks and provides a hotpatch for their
product instead.  If the consumers can't evaluate the danger, provider A
will be pressured into providing a patch even though they don't think
it's a good idea.

So if this really is too dangerous to be used regularly, there is an
argument to be made not to accept the functionality.  (I'm not making an
argument either way at the moment.)

 -George

Jan Beulich Aug. 8, 2016, 3:15 p.m. UTC | #7

>>> On 08.08.16 at 16:10, <[e-mail]> wrote:
> On 08/08/16 15:01, Jan Beulich wrote:
>>>>> On 08.08.16 at 15:42, <[e-mail]> wrote:
>>> On 05/08/16 16:35, Jan Beulich wrote:
>>>>>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>>>>> In general, the hooks provide flexibility when having to deal with
>>>>> unforeseen cases, but their application should be rarely required (<
>>>>> 10%)."
>>>>
>>>> But the greater flexibility of course comes with increased chances
>>>> of screwing things up. I'm therefore still not entirely convinced that
>>>> such XSAs wouldn't then better not be live patched.
>>>
>>> But this functionality is optional, right?  So although I tend to agree
>>> with you, we can let the person / organization preparing the patch take
>>> that risk if they want to?
>> 
>> That's a valid point, albeit I think patch producer and patch consumer
>> might have different views, and the patch consumer may not know
>> (or even know to care) whether (s)he's got handed a patch with or
>> without use of hooks.
> 
> But the patch consumer probably doesn't know for sure if the patch was
> even tested, or if it even fixes the things it was meant to fix.  The
> producer / consumer relationship will always have an aspect of trust and
> some mechanism for feedback.
> 
> I suppose there is a sort of "prisoner's dilemma" aspect to this though:
> suppose provider A thinks that hooks are too dangerous, and doesn't
> provide a hotfix for a certain XSA as a result, but provider B, seeing
> an opportunity, ignores the risks and provides a hotpatch for their
> product instead.  If the consumers can't evaluate the danger, provider A
> will be pressured into providing a patch even though they don't think
> it's a good idea.
> 
> So if this really is too dangerous to be used regularly, there is an
> argument to be made not to accept the functionality.  (I'm not making an
> argument either way at the moment.)

Since both Konrad and Ross appear to be desiring hooks, and since
they're the maintainers, just me having a weak opinion in the other
direction to me means we ought to allow these. If other general
maintainers were considering this too dangerous too, the picture
might change.

Jan

Andrew Cooper Aug. 8, 2016, 4:07 p.m. UTC | #8

On 08/08/16 16:15, Jan Beulich wrote:
>>>> On 08.08.16 at 16:10, <[e-mail]> wrote:
>> On 08/08/16 15:01, Jan Beulich wrote:
>>>>>> On 08.08.16 at 15:42, <[e-mail]> wrote:
>>>> On 05/08/16 16:35, Jan Beulich wrote:
>>>>>>>> On 04.08.16 at 17:49, <[e-mail]> wrote:
>>>>>> In general, the hooks provide flexibility when having to deal with
>>>>>> unforeseen cases, but their application should be rarely required (<
>>>>>> 10%)."
>>>>> But the greater flexibility of course comes with increased chances
>>>>> of screwing things up. I'm therefore still not entirely convinced that
>>>>> such XSAs wouldn't then better not be live patched.
>>>> But this functionality is optional, right?  So although I tend to agree
>>>> with you, we can let the person / organization preparing the patch take
>>>> that risk if they want to?
>>> That's a valid point, albeit I think patch producer and patch consumer
>>> might have different views, and the patch consumer may not know
>>> (or even know to care) whether (s)he's got handed a patch with or
>>> without use of hooks.
>> But the patch consumer probably doesn't know for sure if the patch was
>> even tested, or if it even fixes the things it was meant to fix.  The
>> producer / consumer relationship will always have an aspect of trust and
>> some mechanism for feedback.
>>
>> I suppose there is a sort of "prisoner's dilemma" aspect to this though:
>> suppose provider A thinks that hooks are too dangerous, and doesn't
>> provide a hotfix for a certain XSA as a result, but provider B, seeing
>> an opportunity, ignores the risks and provides a hotpatch for their
>> product instead.  If the consumers can't evaluate the danger, provider A
>> will be pressured into providing a patch even though they don't think
>> it's a good idea.
>>
>> So if this really is too dangerous to be used regularly, there is an
>> argument to be made not to accept the functionality.  (I'm not making an
>> argument either way at the moment.)
> Since both Konrad and Ross appear to be desiring hooks, and since
> they're the maintainers, just me having a weak opinion in the other
> direction to me means we ought to allow these. If other general
> maintainers were considering this too dangerous too, the picture
> might change.

I find it funny that we are arguing over the safety of hooks like these
as part of a system expressly designed to overwrite arbitrary bits of
hypervisor code and data.

This is already a "you had better know exactly what you are doing" kind
of area.

There are a number of XSA cases where one-time data modification needs
to happen if live patching were to occur.

+1 introduce the hooks.  The worst that happens is that someone has yet
way to make things go wrong, but the best that happens is that more
issues can be correctly livepatched.

~Andrew

Konrad Rzeszutek Wilk Aug. 9, 2016, 6:01 p.m. UTC | #9

> >> > @@ -70,7 +71,11 @@ struct payload {
> >> >      unsigned int nsyms;                  /* Nr of entries in .strtab and 
> >> > symbols. */
> >> >      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
> >> >      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
> >> > -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> >> > +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
> >> > +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
> >> 
> >> These both seem like they want a const in the middle.
> > 
> > I did that initially and found out that the calling ->load_funcs[i] resulted
> > in _no_ code being called.
> > 
> > I did not dig in the assembler output to figure out what happend, let me
> > do that now.
> 
> Indeed - I can't see how a const modifier here could alter whether
> or not the pointed to function gets called.

I did a diff on the pre-assemble part (-S) with and without const on the above
structs. With 'const' we get:

        call    spin_debug_disable
 .LVL163:
-       .loc 1 1093 0
-       cmpl    $0, 324(%rbx)
-       je      .L112
-       movl    $0, %r12d
-.LVL164:
-.L113:
-       .loc 1 1094 0
-       mov     %r12d, %edx
-       movq    312(%rbx), %rax
-       call    *(%rax,%rdx,8)
-       .loc 1 1093 0
-       addl    $1, %r12d
-.LVL165:
-       cmpl    %r12d, 324(%rbx)
-       ja      .L113
-.LVL166:
-.L112:

being removed.

I thought it may have to do with the function not returning anything. But
even after making the typedef return it still omitted the call data->load_funcs[i]();

Jan Beulich Aug. 10, 2016, 9:46 a.m. UTC | #10

>>> On 09.08.16 at 20:01, <[e-mail]> wrote:
>> >> > @@ -70,7 +71,11 @@ struct payload {
>> >> >      unsigned int nsyms;                  /* Nr of entries in .strtab and 
>> >> > symbols. */
>> >> >      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
>> >> >      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
>> >> > -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
>> >> > +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
>> >> > +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
>> >> 
>> >> These both seem like they want a const in the middle.
>> > 
>> > I did that initially and found out that the calling ->load_funcs[i] resulted
>> > in _no_ code being called.
>> > 
>> > I did not dig in the assembler output to figure out what happend, let me
>> > do that now.
>> 
>> Indeed - I can't see how a const modifier here could alter whether
>> or not the pointed to function gets called.
> 
> I did a diff on the pre-assemble part (-S) with and without const on the above
> structs. With 'const' we get:
> 
>         call    spin_debug_disable
>  .LVL163:
> -       .loc 1 1093 0
> -       cmpl    $0, 324(%rbx)
> -       je      .L112
> -       movl    $0, %r12d
> -.LVL164:
> -.L113:
> -       .loc 1 1094 0
> -       mov     %r12d, %edx
> -       movq    312(%rbx), %rax
> -       call    *(%rax,%rdx,8)
> -       .loc 1 1093 0
> -       addl    $1, %r12d
> -.LVL165:
> -       cmpl    %r12d, 324(%rbx)
> -       ja      .L113
> -.LVL166:
> -.L112:
> 
> being removed.
> 
> I thought it may have to do with the function not returning anything. But
> even after making the typedef return it still omitted the call 
> data->load_funcs[i]();

Odd. I've tried this simple example:

typedef int fn_t(void);

struct s {
	unsigned n;
	fn_t**fn;
	fn_t*const*fnc;
	const fn_t**cfn;
};

int test1(const struct s*ps) {
	unsigned i;
	int rc = 0;

	for(i = 0; !rc && i < ps->n; ++i)
		rc = ps->fn[i]();

	return rc;
}

int test2(const struct s*ps) {
	unsigned i;
	int rc = 0;

	for(i = 0; !rc && i < ps->n; ++i)
		rc = ps->fnc[i]();

	return rc;
}

int test3(const struct s*ps) {
	unsigned i;
	int rc = 0;

	for(i = 0; !rc && i < ps->n; ++i)
		rc = ps->cfn[i]();

	return rc;
}

test1() and test2() get compiled identically. test3(), using the field
with the misplaced const, oddly enough gets compiled slightly
differently (and without a warning despite one would seem
warranted), yet the call doesn't get omitted. If, however, I change
the return type of fn_t to void, the function body of test3() ends
up empty, which is a compiler bug afaict, but which also suggests
that you've tried the variant with the misplaced const.

Jan

Konrad Rzeszutek Wilk Aug. 11, 2016, 1:20 a.m. UTC | #11

On Wed, Aug 10, 2016 at 03:46:49AM -0600, Jan Beulich wrote:
> >>> On 09.08.16 at 20:01, <[e-mail]> wrote:
> >> >> > @@ -70,7 +71,11 @@ struct payload {
> >> >> >      unsigned int nsyms;                  /* Nr of entries in .strtab and 
> >> >> > symbols. */
> >> >> >      struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
> >> >> >      struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
> >> >> > -    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> >> >> > +    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
> >> >> > +    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
> >> >> 
> >> >> These both seem like they want a const in the middle.
                                                      ^^^^^^
                                                      Right there
.. snip..
> Odd. I've tried this simple example:
.. snip..
> test1() and test2() get compiled identically. test3(), using the field
> with the misplaced const, oddly enough gets compiled slightly
> differently (and without a warning despite one would seem
> warranted), yet the call doesn't get omitted. If, however, I change
> the return type of fn_t to void, the function body of test3() ends
> up empty, which is a compiler bug afaict, but which also suggests
> that you've tried the variant with the misplaced const.

<sigh>

If I had read your email more carefuly (see above) I would not have
wasted your time on this! Sorry about that!

This 'const' business is quite interesting.

> 
> Jan
>

Jan Beulich Aug. 11, 2016, 8:52 a.m. UTC | #12

>>> On 10.08.16 at 11:46, <[e-mail]> wrote:
> Odd. I've tried this simple example:
> 
> typedef int fn_t(void);
> 
> struct s {
> 	unsigned n;
> 	fn_t**fn;
> 	fn_t*const*fnc;
> 	const fn_t**cfn;
> };
> 
> int test1(const struct s*ps) {
> 	unsigned i;
> 	int rc = 0;
> 
> 	for(i = 0; !rc && i < ps->n; ++i)
> 		rc = ps->fn[i]();
> 
> 	return rc;
> }
> 
> int test2(const struct s*ps) {
> 	unsigned i;
> 	int rc = 0;
> 
> 	for(i = 0; !rc && i < ps->n; ++i)
> 		rc = ps->fnc[i]();
> 
> 	return rc;
> }
> 
> int test3(const struct s*ps) {
> 	unsigned i;
> 	int rc = 0;
> 
> 	for(i = 0; !rc && i < ps->n; ++i)
> 		rc = ps->cfn[i]();
> 
> 	return rc;
> }
> 
> test1() and test2() get compiled identically. test3(), using the field
> with the misplaced const, oddly enough gets compiled slightly
> differently (and without a warning despite one would seem
> warranted), yet the call doesn't get omitted. If, however, I change
> the return type of fn_t to void, the function body of test3() ends
> up empty, which is a compiler bug afaict, but which also suggests
> that you've tried the variant with the misplaced const.

FTR: This is not a compiler bug, as specifically named undefined
in the C spec.

Jan

Ian Jackson Aug. 11, 2016, 10:56 a.m. UTC | #13

Jan Beulich writes ("Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case"):
> On 10.08.16 at 11:46, <[e-mail]> wrote:
> > Odd. I've tried this simple example:
> > 
> > typedef int fn_t(void);
...
> > 	const fn_t**cfn;

Ie,
        const int **cfn(void);

> > 	for(i = 0; !rc && i < ps->n; ++i)
> > 		rc = ps->cfn[i]();

From `(gcc-4)Function Attributes':

  `const'
       Many functions do not examine any values except their arguments,
       and have no effects except the return value.  Basically this is
       just slightly more strict class than the `pure' attribute below,
       since function is not allowed to read global memory.

       Note that a function that has pointer arguments and examines the
       data pointed to must _not_ be declared `const'.  Likewise, a
       function that calls a non-`const' function usually must not be
       `const'.  It does not make sense for a `const' function to return
       `void'.

       The attribute `const' is not implemented in GCC versions earlier
       than 2.5.  An alternative way to declare that a function has no
       side effects, which works in the current version and in some older
       versions, is as follows:

            typedef int intfn ();

            extern const intfn square;

       This approach does not work in GNU C++ from 2.6.0 on, since the
       language specifies that the `const' must be attached to the return
       value.

Ie, gcc has always treated a function marked const as having no
unexpected inputs and no side effects.

> > test1() and test2() get compiled identically. test3(), using the field
> > with the misplaced const, oddly enough gets compiled slightly
> > differently (and without a warning despite one would seem
> > warranted), yet the call doesn't get omitted. If, however, I change
> > the return type of fn_t to void, the function body of test3() ends
> > up empty, which is a compiler bug afaict, but which also suggests
> > that you've tried the variant with the misplaced const.

No, it is gcc realising that there is no point calling a function
which returns void and has no side effects

TBH I am inclined to agree that gcc should issue a warning for a
such a function.

> FTR: This is not a compiler bug, as specifically named undefined
> in the C spec.

In this case, the behaviour is defined by the GCC manual.

Ian.

Jan Beulich Aug. 11, 2016, 11:13 a.m. UTC | #14

>>> On 11.08.16 at 12:56, <[e-mail]> wrote:
> Jan Beulich writes ("Re: [Xen-devel] [PATCH v2 1/3] livepach: Add 
> .livepatch.hooks functions and test-case"):
>> On 10.08.16 at 11:46, <[e-mail]> wrote:
>> > Odd. I've tried this simple example:
>> > 
>> > typedef int fn_t(void);
> ...
>> > 	const fn_t**cfn;
> 
> Ie,
>         const int **cfn(void);
> 
>> > 	for(i = 0; !rc && i < ps->n; ++i)
>> > 		rc = ps->cfn[i]();
> 
> From `(gcc-4)Function Attributes':
> 
>   `const'
>        Many functions do not examine any values except their arguments,
>        and have no effects except the return value.  Basically this is
>        just slightly more strict class than the `pure' attribute below,
>        since function is not allowed to read global memory.
> 
>        Note that a function that has pointer arguments and examines the
>        data pointed to must _not_ be declared `const'.  Likewise, a
>        function that calls a non-`const' function usually must not be
>        `const'.  It does not make sense for a `const' function to return
>        `void'.
> 
>        The attribute `const' is not implemented in GCC versions earlier
>        than 2.5.  An alternative way to declare that a function has no
>        side effects, which works in the current version and in some older
>        versions, is as follows:
> 
>             typedef int intfn ();
> 
>             extern const intfn square;
> 
>        This approach does not work in GNU C++ from 2.6.0 on, since the
>        language specifies that the `const' must be attached to the return
>        value.
> 
> Ie, gcc has always treated a function marked const as having no
> unexpected inputs and no side effects.

Oh, I've always assumed that would be __attribute__((const))
only, but what you quote above proves me wrong.

Jan

Patch

diff --git a/docs/misc/livepatch.markdown b/docs/misc/livepatch.markdown
index 89c1050..e0fc5e4 100644
--- a/docs/misc/livepatch.markdown
+++ b/docs/misc/livepatch.markdown
@@ -340,6 +340,13 @@  When reverting a patch, the hypervisor iterates over each `livepatch_func`
 and the core code copies the data from the undo buffer (private internal copy)
 to `old_addr`.
 
+It optionally may contain the address of functions to be called right before
+being applied and after being reverted:
+
+ * `.livepatch.hooks.load` - an array of function pointers.
+ * `.livepatch.hooks.unload` - an array of function pointers.
+
+
 ### Example of .livepatch.funcs
 
 A simple example of what a payload file can be:
@@ -377,6 +384,22 @@  struct livepatch_func livepatch_hello_world = {
 
 Code must be compiled with -fPIC.
 
+### .livepatch.hooks.load and .livepatch.hooks.unload
+
+This section contains an array of function pointers to be executed
+before payload is being applied (.livepatch.funcs) or after reverting
+the payload. This is useful to prepare data structures that need to
+be modified patching.
+
+Each entry in this array is eight bytes.
+
+The type definition of the function are as follow:
+
+<pre>
+typedef void (*livepatch_loadcall_t)(void);  
+typedef void (*livepatch_unloadcall_t)(void);   
+</pre>
+
 ### .livepatch.depends and .note.gnu.build-id
 
 To support dependencies checking and safe loading (to load the
diff --git a/xen/arch/x86/test/xen_hello_world.c b/xen/arch/x86/test/xen_hello_world.c
index 422bdf1..e6a095b 100644
--- a/xen/arch/x86/test/xen_hello_world.c
+++ b/xen/arch/x86/test/xen_hello_world.c
@@ -4,14 +4,50 @@ 
  */
 
 #include "config.h"
+#include <xen/lib.h>
 #include <xen/types.h>
 #include <xen/version.h>
 #include <xen/livepatch.h>
+#include <xen/livepatch_payload.h>
 
 #include <public/sysctl.h>
 
 static char hello_world_patch_this_fnc[] = "xen_extra_version";
 extern const char *xen_hello_world(void);
+static unsigned int cnt;
+
+static void apply_hook(void)
+{
+    printk(KERN_DEBUG "Hook executing.\n");
+}
+
+static void revert_hook(void)
+{
+    printk(KERN_DEBUG "Hook unloaded.\n");
+}
+
+static void hi_func(void)
+{
+    printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
+};
+
+/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
+static void Z_check_fnc(void)
+{
+    printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
+    BUG_ON(cnt == 0 || cnt > 2);
+    cnt = 0; /* Otherwise if you revert, apply, revert the value will be 4! */
+}
+
+LIVEPATCH_LOAD_HOOK(apply_hook);
+LIVEPATCH_UNLOAD_HOOK(revert_hook);
+
+/* Imbalance here. Two load and three unload. */
+
+LIVEPATCH_LOAD_HOOK(hi_func);
+LIVEPATCH_UNLOAD_HOOK(hi_func);
+
+LIVEPATCH_UNLOAD_HOOK(Z_check_fnc);
 
 struct livepatch_func __section(".livepatch.funcs") livepatch_xen_hello_world = {
     .version = LIVEPATCH_PAYLOAD_VERSION,
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 5da28a3..f6dbd51 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -23,6 +23,7 @@ 
 #include <xen/wait.h>
 #include <xen/livepatch_elf.h>
 #include <xen/livepatch.h>
+#include <xen/livepatch_payload.h>
 
 #include <asm/event.h>
 
@@ -70,7 +71,11 @@  struct payload {
     unsigned int nsyms;                  /* Nr of entries in .strtab and symbols. */
     struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
     struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
-    char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
+    livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after */
+    livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. */
+    unsigned int n_load_funcs;           /* Nr of the funcs to load and execute. */
+    unsigned int n_unload_funcs;         /* Nr of funcs to call durung unload. */
+    char name[XEN_LIVEPATCH_NAME_SIZE];  /* Name of it. */
 };
 
 /* Defines an outstanding patching action. */
@@ -515,6 +520,27 @@  static int prepare_payload(struct payload *payload,
         }
     }
 
+    sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.load");
+    if ( sec )
+    {
+        if ( !sec->sec->sh_size ||
+             (sec->sec->sh_size % sizeof(*payload->load_funcs)) )
+            return -EINVAL;
+
+        payload->load_funcs = sec->load_addr;
+        payload->n_load_funcs = sec->sec->sh_size / sizeof(*payload->load_funcs);
+    }
+
+    sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.unload");
+    if ( sec )
+    {
+        if ( !sec->sec->sh_size ||
+             (sec->sec->sh_size % sizeof(*payload->unload_funcs)) )
+            return -EINVAL;
+
+        payload->unload_funcs = sec->load_addr;
+        payload->n_unload_funcs = sec->sec->sh_size / sizeof(*payload->unload_funcs);
+    }
     sec = livepatch_elf_sec_by_name(elf, ELF_BUILD_ID_NOTE);
     if ( sec )
     {
@@ -999,6 +1025,17 @@  static int apply_payload(struct payload *data)
 
     arch_livepatch_quiesce();
 
+    /*
+     * The hooks may call common code which expects spinlocks to be certain
+     * type, as such disable this temporarily.
+     */
+    spin_debug_disable();
+    for ( i = 0; i < data->n_load_funcs; i++ )
+        data->load_funcs[i]();
+    spin_debug_enable();
+
+    ASSERT(!local_irq_is_enabled());
+
     for ( i = 0; i < data->nfuncs; i++ )
         arch_livepatch_apply_jmp(&data->funcs[i]);
 
@@ -1025,6 +1062,17 @@  static int revert_payload(struct payload *data)
     for ( i = 0; i < data->nfuncs; i++ )
         arch_livepatch_revert_jmp(&data->funcs[i]);
 
+    /*
+     * The hooks may call common code which expects spinlocks to be certain
+     * type, as such disable this temporarily.
+     */
+    spin_debug_disable();
+    for ( i = 0; i < data->n_unload_funcs; i++ )
+        data->unload_funcs[i]();
+    spin_debug_enable();
+
+    ASSERT(!local_irq_is_enabled());
+
     arch_livepatch_revive();
 
     /*
diff --git a/xen/include/xen/livepatch_payload.h b/xen/include/xen/livepatch_payload.h
new file mode 100644
index 0000000..cebf568
--- /dev/null
+++ b/xen/include/xen/livepatch_payload.h
@@ -0,0 +1,49 @@ 
+/*
+ * Copyright (C) 2016 Citrix Systems R&D Ltd.
+ */
+
+#ifndef __XEN_LIVEPATCH_PAYLOAD_H__
+#define __XEN_LIVEPATCH_PAYLOAD_H__
+
+/*
+ * The following definitions are to be used in patches. They are taken
+ * from kpatch.
+ */
+typedef void livepatch_loadcall_t(void);
+typedef void livepatch_unloadcall_t(void);
+
+/*
+ * LIVEPATCH_LOAD_HOOK macro
+ *
+ * Declares a function pointer to be allocated in a new
+ * .livepatch.hook.load section.  This livepatch_load_data symbol is later
+ * stripped by create-diff-object so that it can be declared in multiple
+ * objects that are later linked together, avoiding global symbol
+ * collision.  Since multiple hooks can be registered, the
+ * .livepatch.hook.load section is a table of functions that will be
+ * executed in series by the livepatch infrastructure at patch load time.
+ */
+#define LIVEPATCH_LOAD_HOOK(_fn) \
+    livepatch_loadcall_t *__attribute__((weak)) \
+        livepatch_load_data_##_fn __section(".livepatch.hooks.load") = _fn;
+
+/*
+ * LIVEPATCH_UNLOAD_HOOK macro
+ *
+ * Same as LOAD hook with s/load/unload/
+ */
+#define LIVEPATCH_UNLOAD_HOOK(_fn) \
+    livepatch_unloadcall_t *__attribute__((weak)) \
+        livepatch_unload_data_##_fn __section(".livepatch.hooks.unload") = _fn;
+
+#endif /* __XEN_LIVEPATCH_PAYLOAD_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */