mbox series

[RESEND,RFC,0/6] Fork brute force attack mitigation (fbfam)

Message ID 20200910202107.3799376-1-keescook@chromium.org
Headers show
Series Fork brute force attack mitigation (fbfam) | expand

Message

Kees Cook Sept. 10, 2020, 8:21 p.m. UTC
[kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
 also visible at https://github.com/johwood/linux fbfam]

From: John Wood <john.wood@gmx.com>

The goal of this patch serie is to detect and mitigate a fork brute force
attack.

Attacks with the purpose to break ASLR or bypass canaries traditionaly use
some level of brute force with the help of the fork system call. This is
possible since when creating a new process using fork its memory contents
are the same as those of the parent process (the process that called the
fork system call). So, the attacker can test the memory infinite times to
find the correct memory values or the correct memory addresses without
worrying about crashing the application.

Based on the above scenario it would be nice to have this detected and
mitigated, and this is the goal of this implementation.

Other implementations
---------------------

The public version of grsecurity, as a summary, is based on the idea of
delay the fork system call if a child died due to a fatal error. This has
some issues:

1.- Bad practices: Add delays to the kernel is, in general, a bad idea.

2.- Weak points: This protection can be bypassed using two different
    methods since it acts only when the fork is called after a child has
    crashed.

    2.1.- Bypass 1: So, it would still be possible for an attacker to fork
	  a big amount of children (in the order of thousands), then probe
	  all of them, and finally wait the protection time before repeat
	  the steps.

    2.2.- Bypass 2: This method is based on the idea that the protection
	  doesn't act if the parent crashes. So, it would still be possible
	  for an attacker to fork a process and probe itself. Then, fork
	  the child process and probe itself again. This way, these steps
	  can be repeated infinite times without any mitigation.


This implementation
-------------------

The main idea behind this implementation is to improve the existing ones
focusing on the weak points annotated before. So, the solution for the
first bypass method is to detect a fast crash rate instead of only one
simple crash. For the second bypass method the solution is to detect both
the crash of parent and child processes. Moreover, as a mitigation method
it is better to kill all the offending tasks involve in the attack instead
of use delays.

So, the solution to the two bypass methods previously commented is to use
some statistical data shared across all the processes that can have the
same memory contents. Or in other words, a statistical data shared between
all the processes that fork the task 0, and all the processes that fork
after an execve system call.

These statistics hold the timestamp for the first fork (case of a fork of
task zero) or the timestamp for the execve system call (the other case).
Also, hold the number of faults of all the tasks that share the same
statistical data since the commented timestamp.

With this information it is possible to detect a brute force attack when a
task die in a fatal way computing the crashing rate. This rate shows the
milliseconds per fault and when it goes under a certain threshold there is
a clear signal that something malicious is happening.

Once detected, the mitigation only kills the processes that share the same
statistical data and so, all the tasks that can have the same memory
contents. This way, an attack is rejected.

The fbfam feature can be enabled, disabled and tuned as follows:

1.- Per system enabling: This feature can be enabled in build time using
    the config application under:

    Security options  --->  Fork brute force attack mitigation

2.- Per process enabling/disabling: To allow that specific applications can
    turn off or turn on the detection and mitigation of a fork brute force
    attack when required, there are two new prctls.

    prctl(PR_FBFAM_ENABLE, 0, 0, 0, 0)  -> To enable the feature
    prctl(PR_FBFAM_DISABLE, 0, 0, 0, 0) -> To disable the feature

    Both functions return zero on success and -EFAULT if the current task
    doesn't have statistical data.

3.- Fine tuning: To customize the detection's sensibility there is a new
    sysctl that allows to set the crashing rate threshold. It is accessible
    through the file:

    /proc/sys/kernel/fbfam/crashing_rate_threshold

    The units are in milliseconds per fault and the attack's mitigation is
    triggered if the crashing rate of an application goes under this
    threshold. So, the higher this value, the faster an attack will be
    detected.

So, knowing all this information I will explain now the different patches:

The 1/9 patch adds a new config for the fbfam feature.

The 2/9 and 3/9 patches add and use the api to manage the statistical data
necessary to compute the crashing rate of an application.

The 4/9 patch adds a new sysctl to fine tuning the detection's sensibility.

The 5/9 patch detects a fork brute force attack calculating the crashing
rate.

The 6/9 patch mitigates the attack killing all the offending tasks.

The 7/9 patch adds two new prctls to allow per task enabling/disabling.

The 8/9 patch adds general documentation.

The 9/9 patch adds an entry to the maintainers list.

This patch series is a task of the KSPP [1] and it is worth to mention
that there is a previous attempt without any continuation [2].

[1] https://github.com/KSPP/linux/issues/39
[2] https://lore.kernel.org/linux-fsdevel/1419457167-15042-1-git-send-email-richard@nod.at/

Any constructive comments are welcome.

Note: During the compilation these warnings were shown:

kernel/exit.o: warning: objtool: __x64_sys_exit_group()+0x18: unreachable instruction
arch/x86/kernel/cpu/mce/core.o: warning: objtool: mce_panic()+0x123: unreachable instruction
arch/x86/kernel/smpboot.o: warning: objtool: native_play_dead()+0x122: unreachable instruction
net/core/skbuff.o: warning: objtool: skb_push.cold()+0x14: unreachable instruction



John Wood (6):
  security/fbfam: Add a Kconfig to enable the fbfam feature
  security/fbfam: Add the api to manage statistics
  security/fbfam: Use the api to manage statistics
  security/fbfam: Add a new sysctl to control the crashing rate
    threshold
  security/fbfam: Detect a fork brute force attack
  security/fbfam: Mitigate a fork brute force attack

 fs/coredump.c           |   2 +
 fs/exec.c               |   2 +
 include/fbfam/fbfam.h   |  24 ++++
 include/linux/sched.h   |   4 +
 kernel/exit.c           |   2 +
 kernel/fork.c           |   4 +
 kernel/sysctl.c         |   9 ++
 security/Kconfig        |   1 +
 security/Makefile       |   4 +
 security/fbfam/Kconfig  |  10 ++
 security/fbfam/Makefile |   3 +
 security/fbfam/fbfam.c  | 279 ++++++++++++++++++++++++++++++++++++++++
 security/fbfam/sysctl.c |  20 +++
 13 files changed, 364 insertions(+)
 create mode 100644 include/fbfam/fbfam.h
 create mode 100644 security/fbfam/Kconfig
 create mode 100644 security/fbfam/Makefile
 create mode 100644 security/fbfam/fbfam.c
 create mode 100644 security/fbfam/sysctl.c

Comments

Jann Horn Sept. 10, 2020, 8:27 p.m. UTC | #1
On Thu, Sep 10, 2020 at 10:21 PM Kees Cook <keescook@chromium.org> wrote:
> Use the previous defined api to manage statistics calling it accordingly
> when a task forks, calls execve or exits.

You defined functions that return error codes in the previous patch,
but here you ignore the return values. That's a bad idea.

You should probably check the return value in execve() (and fail the
execution in the case where memory allocation fails), and make it so
that the other functions always succeed.
Jann Horn Sept. 10, 2020, 8:39 p.m. UTC | #2
On Thu, Sep 10, 2020 at 10:21 PM Kees Cook <keescook@chromium.org> wrote:
> [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> also visible at https://github.com/johwood/linux fbfam]
[...]
> The goal of this patch serie is to detect and mitigate a fork brute force
> attack.
>
> Attacks with the purpose to break ASLR or bypass canaries traditionaly use
> some level of brute force with the help of the fork system call. This is
> possible since when creating a new process using fork its memory contents
> are the same as those of the parent process (the process that called the
> fork system call). So, the attacker can test the memory infinite times to
> find the correct memory values or the correct memory addresses without
> worrying about crashing the application.

For the next version of this patchset, you may want to clarify that
this is intended to stop brute force attacks *against vulnerable
userspace processes* that fork off worker processes. I was halfway
through the patch series before I realized that.
Kees Cook Sept. 10, 2020, 11:58 p.m. UTC | #3
On Thu, Sep 10, 2020 at 01:21:01PM -0700, Kees Cook wrote:
> From: John Wood <john.wood@gmx.com>
> 
> The goal of this patch serie is to detect and mitigate a fork brute force
> attack.

Thanks for this RFC! I'm excited to get this problem finally handled in
the kernel. Hopefully the feedback is useful. :)
John Wood Sept. 11, 2020, 2:48 p.m. UTC | #4
Hi,

On Thu, Sep 10, 2020 at 04:58:29PM -0700, Kees Cook wrote:
> On Thu, Sep 10, 2020 at 01:21:01PM -0700, Kees Cook wrote:
> > From: John Wood <john.wood@gmx.com>
> >
> > The goal of this patch serie is to detect and mitigate a fork brute force
> > attack.
>
> Thanks for this RFC! I'm excited to get this problem finally handled in
> the kernel. Hopefully the feedback is useful. :)

Kees and Jann,

Thank you very much for your comments and advices. I'm a newbie in the
linux kernel development and this is my first attempt. So, I would prefer
to study all your comments before to reply since a big amount of terms
you expose are unknown to me.

In other words, a late reply to this serie comments is not a lack of
interest. Moreover, I think it would be better that I try to understand and
to implement your ideas before anything else.

My original patch serie is composed of 9 patches, so the 3 lasts are lost.
Kees: Have you removed them for some reason? Can you send them for review?

security/fbfam: Add two new prctls to enable and disable the fbfam feature
https://github.com/johwood/linux/commit/8a36399847213e7eb7b45b853568a53666bd0083

Documentation/security: Add documentation for the fbfam feature
https://github.com/johwood/linux/commit/fb46804541f5c0915f3f48acefbe6dc998815609

MAINTAINERS: Add a new entry for the fbfam feature
https://github.com/johwood/linux/commit/4303bc8935334136c6ef47b5e50b87cd2c472c1f

Is there a problem if I ask for some guidance (replying to this thread)
during the process to do my second patch series?

My goal is to learn as much as possible doing something useful for the
linux kernel.

Thanks a lot,
John Wood

> --
> Kees Cook
James Morris Sept. 12, 2020, 12:03 a.m. UTC | #5
On Thu, 10 Sep 2020, Kees Cook wrote:

> [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
>  also visible at https://github.com/johwood/linux fbfam]
> 
> From: John Wood <john.wood@gmx.com>

Why are you resending this? The author of the code needs to be able to 
send and receive emails directly as part of development and maintenance.
Kees Cook Sept. 12, 2020, 7:55 a.m. UTC | #6
On Fri, Sep 11, 2020 at 04:48:06PM +0200, John Wood wrote:
> In other words, a late reply to this serie comments is not a lack of
> interest. Moreover, I think it would be better that I try to understand and
> to implement your ideas before anything else.

Understood! :)

> My original patch serie is composed of 9 patches, so the 3 lasts are lost.
> Kees: Have you removed them for some reason? Can you send them for review?
> 
> security/fbfam: Add two new prctls to enable and disable the fbfam feature
> https://github.com/johwood/linux/commit/8a36399847213e7eb7b45b853568a53666bd0083
> 
> Documentation/security: Add documentation for the fbfam feature
> https://github.com/johwood/linux/commit/fb46804541f5c0915f3f48acefbe6dc998815609
> 
> MAINTAINERS: Add a new entry for the fbfam feature
> https://github.com/johwood/linux/commit/4303bc8935334136c6ef47b5e50b87cd2c472c1f

Oh, hm, I'm not sure where they went. I think they were missing from my
inbox when I saved your series from email. An oversight on my part;
apologies!

> Is there a problem if I ask for some guidance (replying to this thread)
> during the process to do my second patch series?

Please feel free! I'm happy to help. :)

> My goal is to learn as much as possible doing something useful for the
> linux kernel.

Sounds good; thanks!
Kees Cook Sept. 12, 2020, 7:56 a.m. UTC | #7
On Sat, Sep 12, 2020 at 10:03:23AM +1000, James Morris wrote:
> On Thu, 10 Sep 2020, Kees Cook wrote:
> 
> > [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> >  also visible at https://github.com/johwood/linux fbfam]
> > 
> > From: John Wood <john.wood@gmx.com>
> 
> Why are you resending this? The author of the code needs to be able to 
> send and receive emails directly as part of development and maintenance.

I wanted to flush it from my "review" TODO list, mainly.
John Wood Sept. 12, 2020, 9:36 a.m. UTC | #8
On Sat, Sep 12, 2020 at 12:56:18AM -0700, Kees Cook wrote:
> On Sat, Sep 12, 2020 at 10:03:23AM +1000, James Morris wrote:
> > On Thu, 10 Sep 2020, Kees Cook wrote:
> >
> > > [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> > >  also visible at https://github.com/johwood/linux fbfam]
> > >
> > > From: John Wood <john.wood@gmx.com>
> >
> > Why are you resending this? The author of the code needs to be able to
> > send and receive emails directly as part of development and maintenance.

I tried to send the full patch serie by myself but my email got blocked. After
get support from my email provider it told to me that my account is young,
and due to its spam policie I am not allow, for now, to send a big amount
of mails in a short period. They also informed me that soon I will be able
to send more mails. The quantity increase with the age of the account.

I hope that for the next version all works as expected.
Apologies.

> I wanted to flush it from my "review" TODO list, mainly.

Thanks Kees for the re-send and review.

> --
> Kees Cook

Regards,
John Wood
John Wood Sept. 12, 2020, 12:24 p.m. UTC | #9
On Sat, Sep 12, 2020 at 12:55:03AM -0700, Kees Cook wrote:
> On Fri, Sep 11, 2020 at 04:48:06PM +0200, John Wood wrote:
> > My original patch serie is composed of 9 patches, so the 3 lasts are lost.
> > Kees: Have you removed them for some reason? Can you send them for review?
> >
> > security/fbfam: Add two new prctls to enable and disable the fbfam feature
> > https://github.com/johwood/linux/commit/8a36399847213e7eb7b45b853568a53666bd0083
> >
> > Documentation/security: Add documentation for the fbfam feature
> > https://github.com/johwood/linux/commit/fb46804541f5c0915f3f48acefbe6dc998815609
> >
> > MAINTAINERS: Add a new entry for the fbfam feature
> > https://github.com/johwood/linux/commit/4303bc8935334136c6ef47b5e50b87cd2c472c1f
>
> Oh, hm, I'm not sure where they went. I think they were missing from my
> inbox when I saved your series from email. An oversight on my part;
> apologies!

I sent the full serie to Matthew Wilcox <willy@infradead.org> only, as he
wanted to help re-sending the full serie. Then I saw that only 6 patches
appeared in the linux-doc mailing list.

I can try to send the three pending patches in different stages (for example
one patch every 4 or 5 hours) to avoid blocking my email. I hope. Or I can
send the three pending patches only to the kernel-hardening mailing list
and you re-send to all the people involved. Or any other solution you
propose. It's up to you.

> > Is there a problem if I ask for some guidance (replying to this thread)
> > during the process to do my second patch series?
>
> Please feel free! I'm happy to help. :)

It's a pleasure working with you. Thanks a lot.

> --
> Kees Cook

Regards,
John Wood
Mel Gorman Sept. 12, 2020, 2:47 p.m. UTC | #10
On Sat, Sep 12, 2020 at 11:36:52AM +0200, John Wood wrote:
> On Sat, Sep 12, 2020 at 12:56:18AM -0700, Kees Cook wrote:
> > On Sat, Sep 12, 2020 at 10:03:23AM +1000, James Morris wrote:
> > > On Thu, 10 Sep 2020, Kees Cook wrote:
> > >
> > > > [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> > > >  also visible at https://github.com/johwood/linux fbfam]
> > > >
> > > > From: John Wood <john.wood@gmx.com>
> > >
> > > Why are you resending this? The author of the code needs to be able to
> > > send and receive emails directly as part of development and maintenance.
> 
> I tried to send the full patch serie by myself but my email got blocked. After
> get support from my email provider it told to me that my account is young,
> and due to its spam policie I am not allow, for now, to send a big amount
> of mails in a short period. They also informed me that soon I will be able
> to send more mails. The quantity increase with the age of the account.
> 

If you're using "git send-email" then specify --confirm=always and
either manually send a mail every few seconds or use an expect script
like

#!/bin/bash
EXPECT_SCRIPT=
function cleanup() {
	if [ "$EXPECT_SCRIPT" != "" ]; then
		rm $EXPECT_SCRIPT
	fi
}
trap cleanup EXIT

EXPECT_SCRIPT=`mktemp`
cat > $EXPECT_SCRIPT <<EOF
spawn sh ./SEND
expect {
	"Send this email"   { sleep 10; exp_send y\\r; exp_continue }
}
EOF

expect -f $EXPECT_SCRIPT
exit $?

This will work if your provider limits the rate mails are sent rather
than the total amount.
Ondrej Mosnacek Sept. 12, 2020, 8:48 p.m. UTC | #11
On Sat, Sep 12, 2020 at 4:51 PM Mel Gorman <mgorman@suse.de> wrote:
> On Sat, Sep 12, 2020 at 11:36:52AM +0200, John Wood wrote:
> > On Sat, Sep 12, 2020 at 12:56:18AM -0700, Kees Cook wrote:
> > > On Sat, Sep 12, 2020 at 10:03:23AM +1000, James Morris wrote:
> > > > On Thu, 10 Sep 2020, Kees Cook wrote:
> > > >
> > > > > [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> > > > >  also visible at https://github.com/johwood/linux fbfam]
> > > > >
> > > > > From: John Wood <john.wood@gmx.com>
> > > >
> > > > Why are you resending this? The author of the code needs to be able to
> > > > send and receive emails directly as part of development and maintenance.
> >
> > I tried to send the full patch serie by myself but my email got blocked. After
> > get support from my email provider it told to me that my account is young,
> > and due to its spam policie I am not allow, for now, to send a big amount
> > of mails in a short period. They also informed me that soon I will be able
> > to send more mails. The quantity increase with the age of the account.
> >
>
> If you're using "git send-email" then specify --confirm=always and
> either manually send a mail every few seconds or use an expect script
> like
>
> #!/bin/bash
> EXPECT_SCRIPT=
> function cleanup() {
>         if [ "$EXPECT_SCRIPT" != "" ]; then
>                 rm $EXPECT_SCRIPT
>         fi
> }
> trap cleanup EXIT
>
> EXPECT_SCRIPT=`mktemp`
> cat > $EXPECT_SCRIPT <<EOF
> spawn sh ./SEND
> expect {
>         "Send this email"   { sleep 10; exp_send y\\r; exp_continue }
> }
> EOF
>
> expect -f $EXPECT_SCRIPT
> exit $?
>
> This will work if your provider limits the rate mails are sent rather
> than the total amount.

...or you could keep it simple and just pass "--batch-size 1
--relogin-delay 10" to git send-email ;)
John Wood Sept. 13, 2020, 7:24 a.m. UTC | #12
Hi,

On Sat, Sep 12, 2020 at 10:48:39PM +0200, Ondrej Mosnacek wrote:
> On Sat, Sep 12, 2020 at 4:51 PM Mel Gorman <mgorman@suse.de> wrote:
> > On Sat, Sep 12, 2020 at 11:36:52AM +0200, John Wood wrote:
> > > On Sat, Sep 12, 2020 at 12:56:18AM -0700, Kees Cook wrote:
> > > > On Sat, Sep 12, 2020 at 10:03:23AM +1000, James Morris wrote:
> > > > > On Thu, 10 Sep 2020, Kees Cook wrote:
> > > > >
> > > > > > [kees: re-sending this series on behalf of John Wood <john.wood@gmx.com>
> > > > > >  also visible at https://github.com/johwood/linux fbfam]
> > > > > >
> > > > > > From: John Wood <john.wood@gmx.com>
> > > > >
> > > > > Why are you resending this? The author of the code needs to be able to
> > > > > send and receive emails directly as part of development and maintenance.
> > >
> > > I tried to send the full patch serie by myself but my email got blocked. After
> > > get support from my email provider it told to me that my account is young,
> > > and due to its spam policie I am not allow, for now, to send a big amount
> > > of mails in a short period. They also informed me that soon I will be able
> > > to send more mails. The quantity increase with the age of the account.
> > >
> >
> > If you're using "git send-email" then specify --confirm=always and
> > either manually send a mail every few seconds or use an expect script
> > like
> >
> > #!/bin/bash
> > EXPECT_SCRIPT=
> > function cleanup() {
> >         if [ "$EXPECT_SCRIPT" != "" ]; then
> >                 rm $EXPECT_SCRIPT
> >         fi
> > }
> > trap cleanup EXIT
> >
> > EXPECT_SCRIPT=`mktemp`
> > cat > $EXPECT_SCRIPT <<EOF
> > spawn sh ./SEND
> > expect {
> >         "Send this email"   { sleep 10; exp_send y\\r; exp_continue }
> > }
> > EOF
> >
> > expect -f $EXPECT_SCRIPT
> > exit $?
> >
> > This will work if your provider limits the rate mails are sent rather
> > than the total amount.

Yes, it seems to be what is happening.

> ...or you could keep it simple and just pass "--batch-size 1
> --relogin-delay 10" to git send-email ;)

Mel and Ondrej thanks a lot for the proposed solutions. I'm sure some of
your solutions will be used soon.

> --
> Ondrej Mosnacek
> Software Engineer, Platform Security - SELinux kernel
> Red Hat, Inc.

Regards,
John Wood