[v5,3/4] LSM: Define SELinux function to measure state and policy
diff mbox series

Message ID 20200730034724.3298-4-nramas@linux.microsoft.com
State Superseded
Headers show
Series
  • LSM: Measure security module data
Related show

Commit Message

Lakshmi Ramasubramanian July 30, 2020, 3:47 a.m. UTC
SELinux configuration and policy are some of the critical data for this
security module that needs to be measured. This measurement can be used
by an attestation service, for instance, to verify if the configuration
and policies have been setup correctly and that they haven't been tampered
with at runtime.

Measure SELinux configuration, policy capabilities settings, and the
loaded policy by calling the IMA hooks ima_measure_lsm_state() and
ima_measure_lsm_policy() respectively.

Sample measurement of SELinux state and hash of the policy:

10 e32e...5ac3 ima-buf sha256:86e8...4594 selinux-state-1595389364:287899386 696e697469616c697a65643d313b656e61626c65643d313b656e666f7263696e673d303b636865636b72657170726f743d313b6e6574776f726b5f706565725f636f6e74726f6c733d313b6f70656e5f7065726d733d313b657874656e6465645f736f636b65745f636c6173733d313b616c776179735f636865636b5f6e6574776f726b3d303b6367726f75705f7365636c6162656c3d313b6e6e705f6e6f737569645f7472616e736974696f6e3d313b67656e66735f7365636c6162656c5f73796d6c696e6b733d303
10 f4a7...9408 ima-ng sha256:8d1d...1834 selinux-policy-hash-1595389353:863934271

To verify the measurement check the following:

Execute the following command to extract the measured data
from the IMA log for SELinux configuration (selinux-state).

  grep -m 1 "selinux-state" /sys/kernel/security/integrity/ima/ascii_runtime_measurements | cut -d' ' -f 6 | xxd -r -p

The output should be the list of key-value pairs. For example,
 initialized=1;enabled=1;enforcing=0;checkreqprot=1;network_peer_controls=1;open_perms=1;extended_socket_class=1;always_check_network=0;cgroup_seclabel=1;nnp_nosuid_transition=1;genfs_seclabel_symlinks=0;

To verify the measured data with the current SELinux state:

 => enabled should be set to 1 if /sys/fs/selinux folder exists,
    0 otherwise

For other entries, compare the integer value in the files
 => /sys/fs/selinux/enforce
 => /sys/fs/selinux/checkreqprot
And, each of the policy capabilities files under
 => /sys/fs/selinux/policy_capabilities

For selinux-policy-hash, the hash of SELinux policy is included
in the IMA log entry.

To verify the measured data with the current SELinux policy run
the following commands and verify the output hash values match.

  sha256sum /sys/fs/selinux/policy | cut -d' ' -f 1

  grep -m 1 "selinux-policy-hash" /sys/kernel/security/integrity/ima/ascii_runtime_measurements | cut -d' ' -f 4

Signed-off-by: Lakshmi Ramasubramanian <nramas@linux.microsoft.com>
Suggested-by: Stephen Smalley <stephen.smalley.work@gmail.com>
Reported-by: kernel test robot <lkp@intel.com> # error: implicit declaration of function 'vfree'
Reported-by: kernel test robot <lkp@intel.com> # error: implicit declaration of function 'crypto_alloc_shash'
Reported-by: kernel test robot <lkp@intel.com> # sparse: symbol 'security_read_selinux_policy' was not declared. Should it be static?
---
 security/selinux/Makefile           |   2 +
 security/selinux/hooks.c            |   1 +
 security/selinux/include/security.h |  15 +++
 security/selinux/measure.c          | 150 ++++++++++++++++++++++++++++
 security/selinux/selinuxfs.c        |   3 +
 security/selinux/ss/services.c      |  71 +++++++++++--
 6 files changed, 233 insertions(+), 9 deletions(-)
 create mode 100644 security/selinux/measure.c

Comments

Stephen Smalley Aug. 3, 2020, 3:11 p.m. UTC | #1
On 7/29/20 11:47 PM, Lakshmi Ramasubramanian wrote:

> SELinux configuration and policy are some of the critical data for this
> security module that needs to be measured. This measurement can be used
> by an attestation service, for instance, to verify if the configuration
> and policies have been setup correctly and that they haven't been tampered
> with at runtime.
>
> Measure SELinux configuration, policy capabilities settings, and the
> loaded policy by calling the IMA hooks ima_measure_lsm_state() and
> ima_measure_lsm_policy() respectively.
>
> Sample measurement of SELinux state and hash of the policy:
>
> 10 e32e...5ac3 ima-buf sha256:86e8...4594 selinux-state-1595389364:287899386 696e697469616c697a65643d313b656e61626c65643d313b656e666f7263696e673d303b636865636b72657170726f743d313b6e6574776f726b5f706565725f636f6e74726f6c733d313b6f70656e5f7065726d733d313b657874656e6465645f736f636b65745f636c6173733d313b616c776179735f636865636b5f6e6574776f726b3d303b6367726f75705f7365636c6162656c3d313b6e6e705f6e6f737569645f7472616e736974696f6e3d313b67656e66735f7365636c6162656c5f73796d6c696e6b733d303
> 10 f4a7...9408 ima-ng sha256:8d1d...1834 selinux-policy-hash-1595389353:863934271
>
> To verify the measurement check the following:
>
> Execute the following command to extract the measured data
> from the IMA log for SELinux configuration (selinux-state).
>
>    grep -m 1 "selinux-state" /sys/kernel/security/integrity/ima/ascii_runtime_measurements | cut -d' ' -f 6 | xxd -r -p
>
> The output should be the list of key-value pairs. For example,
>   initialized=1;enabled=1;enforcing=0;checkreqprot=1;network_peer_controls=1;open_perms=1;extended_socket_class=1;always_check_network=0;cgroup_seclabel=1;nnp_nosuid_transition=1;genfs_seclabel_symlinks=0;
>
> To verify the measured data with the current SELinux state:
>
>   => enabled should be set to 1 if /sys/fs/selinux folder exists,
>      0 otherwise
>
> For other entries, compare the integer value in the files
>   => /sys/fs/selinux/enforce
>   => /sys/fs/selinux/checkreqprot
> And, each of the policy capabilities files under
>   => /sys/fs/selinux/policy_capabilities
>
> For selinux-policy-hash, the hash of SELinux policy is included
> in the IMA log entry.
>
> To verify the measured data with the current SELinux policy run
> the following commands and verify the output hash values match.
>
>    sha256sum /sys/fs/selinux/policy | cut -d' ' -f 1
>
>    grep -m 1 "selinux-policy-hash" /sys/kernel/security/integrity/ima/ascii_runtime_measurements | cut -d' ' -f 4
>
> Signed-off-by: Lakshmi Ramasubramanian <nramas@linux.microsoft.com>
> Suggested-by: Stephen Smalley <stephen.smalley.work@gmail.com>
> Reported-by: kernel test robot <lkp@intel.com> # error: implicit declaration of function 'vfree'
> Reported-by: kernel test robot <lkp@intel.com> # error: implicit declaration of function 'crypto_alloc_shash'
> Reported-by: kernel test robot <lkp@intel.com> # sparse: symbol 'security_read_selinux_policy' was not declared. Should it be static?

Possibly I'm missing something but with these patches applied on top of 
next-integrity, and the following lines added to /etc/ima/ima-policy:

measure func=LSM_STATE template=ima-buf
measure func=LSM_POLICY

I still don't get the selinux-state or selinux-policy-hash entries in 
the ascii_runtime_measurements file.  No errors during loading of the 
ima policy as far as I can see.
Lakshmi Ramasubramanian Aug. 3, 2020, 4:14 p.m. UTC | #2
On 8/3/20 8:11 AM, Stephen Smalley wrote:
> 
> Possibly I'm missing something but with these patches applied on top of 
> next-integrity, and the following lines added to /etc/ima/ima-policy:
> 
> measure func=LSM_STATE template=ima-buf
> measure func=LSM_POLICY
> 
> I still don't get the selinux-state or selinux-policy-hash entries in 
> the ascii_runtime_measurements file.  No errors during loading of the 
> ima policy as far as I can see.
> 

Could you please check if the following config is set?
CONFIG_IMA_QUEUE_EARLY_BOOT_DATA=y

Try changing /sys/fs/selinux/checkreqprot and check 
ascii_runtime_measurements file again?

Also, could you please check if
/sys/kernel/security/integrity/ima/policy contains LSM_STATE and 
LSM_POLICY entries?

  -lakshmi
Stephen Smalley Aug. 3, 2020, 8 p.m. UTC | #3
On Mon, Aug 3, 2020 at 12:14 PM Lakshmi Ramasubramanian
<nramas@linux.microsoft.com> wrote:
>
> On 8/3/20 8:11 AM, Stephen Smalley wrote:
> >
> > Possibly I'm missing something but with these patches applied on top of
> > next-integrity, and the following lines added to /etc/ima/ima-policy:
> >
> > measure func=LSM_STATE template=ima-buf
> > measure func=LSM_POLICY
> >
> > I still don't get the selinux-state or selinux-policy-hash entries in
> > the ascii_runtime_measurements file.  No errors during loading of the
> > ima policy as far as I can see.
> >
>
> Could you please check if the following config is set?
> CONFIG_IMA_QUEUE_EARLY_BOOT_DATA=y

Yes, I have that set.

> Try changing /sys/fs/selinux/checkreqprot and check
> ascii_runtime_measurements file again?

No change.  Likewise for changing enforce or running load_policy again.

> Also, could you please check if
> /sys/kernel/security/integrity/ima/policy contains LSM_STATE and
> LSM_POLICY entries?

Yes, it does.  However, I noticed that if I reduce the policy to only
contain those entries and no others and reboot, then I get
measurements.  Whereas if I append them to an existing policy like the
one below, they seem to be ignored:
dont_measure fsmagic=0x9fa0
dont_measure fsmagic=0x62656572
dont_measure fsmagic=0x64626720
dont_measure fsmagic=0x1021994
dont_measure fsmagic=0x858458f6
dont_measure fsmagic=0x73636673
measure func=BPRM_CHECK
measure func=MMAP_CHECK mask=MAY_EXEC
measure func=MODULE_CHECK uid=0
measure func=LSM_STATE template=ima-buf
measure func=LSM_POLICY

Also, I noticed the following in my dmesg output:
[   68.870715] ------------[ cut here ]------------
[   68.870715] WARNING: CPU: 2 PID: 1 at mm/page_alloc.c:4826
__alloc_pages_nodemask+0x627/0x700
[   68.870715] Modules linked in: 8139too crct10dif_pclmul
crc32_pclmul crc32c_intel ghash_clmulni_intel qxl serio_raw
drm_ttm_helper ttm drm_kms_helper virtio_console cec drm 8139cp
ata_generic mii pata_acpi floppy qemu_fw_cfg fuse
[   68.870715] CPU: 2 PID: 1 Comm: systemd Not tainted 5.8.0-rc2+ #44
[   68.870715] RIP: 0010:__alloc_pages_nodemask+0x627/0x700
[   68.870715] Code: ff ff 75 6c 48 8b 85 48 ff ff ff 4c 89 c2 44 89
e6 44 89 ff 41 c6 45 d0 00 49 89 45 b8 e8 41 e2 ff ff 49 89 c6 e9 9d
fc ff ff <0f> 0b e9 d4 fd ff ff 0f 0b e9 bc fc ff ff 0f 0b e9 f9 fd ff
ff e8
[   68.870715] RSP: 0000:ffff8881e82a7a18 EFLAGS: 00010246
[   68.870715] RAX: ffffed103d054f48 RBX: 1ffff1103d054f48 RCX: 0000000000000000
[   68.870715] RDX: 0000000000000000 RSI: 000000000000000b RDI: 0000000000000000
[   68.870715] RBP: ffff8881e82a7ae8 R08: ffffffffaa3fe2d5 R09: 0000000000000001
[   68.870715] R10: fffffbfff5a88f0f R11: 0000000000000001 R12: 00000000007eef6a
[   68.870715] R13: 0000000000040cc0 R14: 000000000000000b R15: ffffffffadde766b
[   68.870715] FS:  00007fdeb168c600(0000) GS:ffff8881e9800000(0000)
knlGS:0000000000000000
[   68.870715] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   68.870715] CR2: 00007fdeb17dd1d6 CR3: 00000001cc2d2002 CR4: 00000000003606e0
[   68.870715] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   68.870715] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   68.870715] Call Trace:
[   68.870715]  ? sched_clock_cpu+0xf5/0x110
[   68.870715]  ? __alloc_pages_slowpath.constprop.0+0x17a0/0x17a0
[   68.870715]  ? match_held_lock+0x2e/0x240
[   68.870715]  ? policy_nodemask+0x1a/0xa0
[   68.870715]  ? policy_node+0x56/0x60
[   68.870715]  kmalloc_order+0x25/0xc0
[   68.870715]  kmalloc_order_trace+0x1d/0x140
[   68.870715]  kmemdup+0x1a/0x40
[   68.870715]  ima_queue_data+0x61/0x370
[   68.870715]  ima_measure_lsm_data+0x32/0x60
[   68.870715]  selinux_measure_state+0x2b8/0x2bd
[   68.870715]  ? selinux_event_name+0xe0/0xe0
[   68.870715]  ? rcu_is_watching+0x39/0x50
[   68.870715]  security_load_policy+0x44c/0x8e0
[   68.870715]  ? mark_lock+0xa6/0xbd0
[   68.870715]  ? security_change_sid+0x90/0x90
[   68.870715]  ? mark_held_locks+0x3e/0xa0
[   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
[   68.870715]  ? asm_exc_page_fault+0x1e/0x30
[   68.870715]  ? lockdep_hardirqs_on+0xc5/0x1b0
[   68.870715]  ? asm_exc_page_fault+0x1e/0x30
[   68.870715]  ? copy_user_enhanced_fast_string+0xe/0x30
[   68.870715]  sel_write_load+0x157/0x260
[   68.870715]  vfs_write+0x135/0x290
[   68.870715]  ksys_write+0xb1/0x140
[   68.870715]  ? __ia32_sys_read+0x50/0x50
[   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
[   68.870715]  ? do_syscall_64+0x12/0xb0
[   68.870715]  do_syscall_64+0x52/0xb0
[   68.870715]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   68.870715] RIP: 0033:0x7fdeb2539497
[   68.870715] Code: Bad RIP value.
[   68.870715] RSP: 002b:00007fff6352b308 EFLAGS: 00000246 ORIG_RAX:
0000000000000001
[   68.870715] RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 00007fdeb2539497
[   68.870715] RDX: 00000000007eef6a RSI: 00007fdeb0de1000 RDI: 0000000000000004
[   68.870715] RBP: 0000000000000004 R08: 00007fdeb25d0040 R09: 00007fff6352b1a0
[   68.870715] R10: 0000000000000000 R11: 0000000000000246 R12: 00007fdeb0de1000
[   68.870715] R13: 00000000007eef6a R14: 000000000000000f R15: 0000000000000003
[   68.870715] irq event stamp: 23486085
[   68.870715] hardirqs last  enabled at (23486085):
[<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
[   68.870715] hardirqs last disabled at (23486084):
[<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
[   68.870715] softirqs last  enabled at (23486074):
[<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
[   68.870715] softirqs last disabled at (23486067):
[<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
[   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
Stephen Smalley Aug. 3, 2020, 8:29 p.m. UTC | #4
On 8/3/20 4:00 PM, Stephen Smalley wrote:

> On Mon, Aug 3, 2020 at 12:14 PM Lakshmi Ramasubramanian
> <nramas@linux.microsoft.com> wrote:
>> On 8/3/20 8:11 AM, Stephen Smalley wrote:
>>> Possibly I'm missing something but with these patches applied on top of
>>> next-integrity, and the following lines added to /etc/ima/ima-policy:
>>>
>>> measure func=LSM_STATE template=ima-buf
>>> measure func=LSM_POLICY
>>>
>>> I still don't get the selinux-state or selinux-policy-hash entries in
>>> the ascii_runtime_measurements file.  No errors during loading of the
>>> ima policy as far as I can see.
>>>
>> Could you please check if the following config is set?
>> CONFIG_IMA_QUEUE_EARLY_BOOT_DATA=y
> Yes, I have that set.
>
>> Try changing /sys/fs/selinux/checkreqprot and check
>> ascii_runtime_measurements file again?
> No change.  Likewise for changing enforce or running load_policy again.
>
>> Also, could you please check if
>> /sys/kernel/security/integrity/ima/policy contains LSM_STATE and
>> LSM_POLICY entries?
> Yes, it does.  However, I noticed that if I reduce the policy to only
> contain those entries and no others and reboot, then I get
> measurements.  Whereas if I append them to an existing policy like the
> one below, they seem to be ignored:
> dont_measure fsmagic=0x9fa0
> dont_measure fsmagic=0x62656572
> dont_measure fsmagic=0x64626720
> dont_measure fsmagic=0x1021994
> dont_measure fsmagic=0x858458f6
> dont_measure fsmagic=0x73636673
> measure func=BPRM_CHECK
> measure func=MMAP_CHECK mask=MAY_EXEC
> measure func=MODULE_CHECK uid=0
> measure func=LSM_STATE template=ima-buf
> measure func=LSM_POLICY
>
> Also, I noticed the following in my dmesg output:
> [   68.870715] ------------[ cut here ]------------
> [   68.870715] WARNING: CPU: 2 PID: 1 at mm/page_alloc.c:4826
> __alloc_pages_nodemask+0x627/0x700
> [   68.870715] Modules linked in: 8139too crct10dif_pclmul
> crc32_pclmul crc32c_intel ghash_clmulni_intel qxl serio_raw
> drm_ttm_helper ttm drm_kms_helper virtio_console cec drm 8139cp
> ata_generic mii pata_acpi floppy qemu_fw_cfg fuse
> [   68.870715] CPU: 2 PID: 1 Comm: systemd Not tainted 5.8.0-rc2+ #44
> [   68.870715] RIP: 0010:__alloc_pages_nodemask+0x627/0x700
> [   68.870715] Code: ff ff 75 6c 48 8b 85 48 ff ff ff 4c 89 c2 44 89
> e6 44 89 ff 41 c6 45 d0 00 49 89 45 b8 e8 41 e2 ff ff 49 89 c6 e9 9d
> fc ff ff <0f> 0b e9 d4 fd ff ff 0f 0b e9 bc fc ff ff 0f 0b e9 f9 fd ff
> ff e8
> [   68.870715] RSP: 0000:ffff8881e82a7a18 EFLAGS: 00010246
> [   68.870715] RAX: ffffed103d054f48 RBX: 1ffff1103d054f48 RCX: 0000000000000000
> [   68.870715] RDX: 0000000000000000 RSI: 000000000000000b RDI: 0000000000000000
> [   68.870715] RBP: ffff8881e82a7ae8 R08: ffffffffaa3fe2d5 R09: 0000000000000001
> [   68.870715] R10: fffffbfff5a88f0f R11: 0000000000000001 R12: 00000000007eef6a
> [   68.870715] R13: 0000000000040cc0 R14: 000000000000000b R15: ffffffffadde766b
> [   68.870715] FS:  00007fdeb168c600(0000) GS:ffff8881e9800000(0000)
> knlGS:0000000000000000
> [   68.870715] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   68.870715] CR2: 00007fdeb17dd1d6 CR3: 00000001cc2d2002 CR4: 00000000003606e0
> [   68.870715] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   68.870715] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   68.870715] Call Trace:
> [   68.870715]  ? sched_clock_cpu+0xf5/0x110
> [   68.870715]  ? __alloc_pages_slowpath.constprop.0+0x17a0/0x17a0
> [   68.870715]  ? match_held_lock+0x2e/0x240
> [   68.870715]  ? policy_nodemask+0x1a/0xa0
> [   68.870715]  ? policy_node+0x56/0x60
> [   68.870715]  kmalloc_order+0x25/0xc0
> [   68.870715]  kmalloc_order_trace+0x1d/0x140
> [   68.870715]  kmemdup+0x1a/0x40
> [   68.870715]  ima_queue_data+0x61/0x370
> [   68.870715]  ima_measure_lsm_data+0x32/0x60
> [   68.870715]  selinux_measure_state+0x2b8/0x2bd
> [   68.870715]  ? selinux_event_name+0xe0/0xe0
> [   68.870715]  ? rcu_is_watching+0x39/0x50
> [   68.870715]  security_load_policy+0x44c/0x8e0
> [   68.870715]  ? mark_lock+0xa6/0xbd0
> [   68.870715]  ? security_change_sid+0x90/0x90
> [   68.870715]  ? mark_held_locks+0x3e/0xa0
> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
> [   68.870715]  ? lockdep_hardirqs_on+0xc5/0x1b0
> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
> [   68.870715]  ? copy_user_enhanced_fast_string+0xe/0x30
> [   68.870715]  sel_write_load+0x157/0x260
> [   68.870715]  vfs_write+0x135/0x290
> [   68.870715]  ksys_write+0xb1/0x140
> [   68.870715]  ? __ia32_sys_read+0x50/0x50
> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
> [   68.870715]  ? do_syscall_64+0x12/0xb0
> [   68.870715]  do_syscall_64+0x52/0xb0
> [   68.870715]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [   68.870715] RIP: 0033:0x7fdeb2539497
> [   68.870715] Code: Bad RIP value.
> [   68.870715] RSP: 002b:00007fff6352b308 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000001
> [   68.870715] RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 00007fdeb2539497
> [   68.870715] RDX: 00000000007eef6a RSI: 00007fdeb0de1000 RDI: 0000000000000004
> [   68.870715] RBP: 0000000000000004 R08: 00007fdeb25d0040 R09: 00007fff6352b1a0
> [   68.870715] R10: 0000000000000000 R11: 0000000000000246 R12: 00007fdeb0de1000
> [   68.870715] R13: 00000000007eef6a R14: 000000000000000f R15: 0000000000000003
> [   68.870715] irq event stamp: 23486085
> [   68.870715] hardirqs last  enabled at (23486085):
> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
> [   68.870715] hardirqs last disabled at (23486084):
> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
> [   68.870715] softirqs last  enabled at (23486074):
> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
> [   68.870715] softirqs last disabled at (23486067):
> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---

I think one issue here is that systemd loads SELinux policy first, then 
IMA policy, so it doesn't know whether it needs to measure SELinux 
policy on first policy load, and another issue is that the policy is too 
large to just queue the policy data itself this way (or you need to use 
an allocator that can handle larger sizes).
Lakshmi Ramasubramanian Aug. 3, 2020, 8:37 p.m. UTC | #5
On 8/3/20 1:29 PM, Stephen Smalley wrote:
> On 8/3/20 4:00 PM, Stephen Smalley wrote:
> 
>> On Mon, Aug 3, 2020 at 12:14 PM Lakshmi Ramasubramanian
>> <nramas@linux.microsoft.com> wrote:
>>> On 8/3/20 8:11 AM, Stephen Smalley wrote:
>>>> Possibly I'm missing something but with these patches applied on top of
>>>> next-integrity, and the following lines added to /etc/ima/ima-policy:
>>>>
>>>> measure func=LSM_STATE template=ima-buf
>>>> measure func=LSM_POLICY
>>>>
>>>> I still don't get the selinux-state or selinux-policy-hash entries in
>>>> the ascii_runtime_measurements file.  No errors during loading of the
>>>> ima policy as far as I can see.
>>>>
>>> Could you please check if the following config is set?
>>> CONFIG_IMA_QUEUE_EARLY_BOOT_DATA=y
>> Yes, I have that set.
>>
>>> Try changing /sys/fs/selinux/checkreqprot and check
>>> ascii_runtime_measurements file again?
>> No change.  Likewise for changing enforce or running load_policy again.
>>
>>> Also, could you please check if
>>> /sys/kernel/security/integrity/ima/policy contains LSM_STATE and
>>> LSM_POLICY entries?
>> Yes, it does.  However, I noticed that if I reduce the policy to only
>> contain those entries and no others and reboot, then I get
>> measurements.  Whereas if I append them to an existing policy like the
>> one below, they seem to be ignored:
>> dont_measure fsmagic=0x9fa0
>> dont_measure fsmagic=0x62656572
>> dont_measure fsmagic=0x64626720
>> dont_measure fsmagic=0x1021994
>> dont_measure fsmagic=0x858458f6
>> dont_measure fsmagic=0x73636673
>> measure func=BPRM_CHECK
>> measure func=MMAP_CHECK mask=MAY_EXEC
>> measure func=MODULE_CHECK uid=0
>> measure func=LSM_STATE template=ima-buf
>> measure func=LSM_POLICY
>>
>> Also, I noticed the following in my dmesg output:
>> [   68.870715] ------------[ cut here ]------------
>> [   68.870715] WARNING: CPU: 2 PID: 1 at mm/page_alloc.c:4826
>> __alloc_pages_nodemask+0x627/0x700
>> [   68.870715] Modules linked in: 8139too crct10dif_pclmul
>> crc32_pclmul crc32c_intel ghash_clmulni_intel qxl serio_raw
>> drm_ttm_helper ttm drm_kms_helper virtio_console cec drm 8139cp
>> ata_generic mii pata_acpi floppy qemu_fw_cfg fuse
>> [   68.870715] CPU: 2 PID: 1 Comm: systemd Not tainted 5.8.0-rc2+ #44
>> [   68.870715] RIP: 0010:__alloc_pages_nodemask+0x627/0x700
>> [   68.870715] Code: ff ff 75 6c 48 8b 85 48 ff ff ff 4c 89 c2 44 89
>> e6 44 89 ff 41 c6 45 d0 00 49 89 45 b8 e8 41 e2 ff ff 49 89 c6 e9 9d
>> fc ff ff <0f> 0b e9 d4 fd ff ff 0f 0b e9 bc fc ff ff 0f 0b e9 f9 fd ff
>> ff e8
>> [   68.870715] RSP: 0000:ffff8881e82a7a18 EFLAGS: 00010246
>> [   68.870715] RAX: ffffed103d054f48 RBX: 1ffff1103d054f48 RCX: 
>> 0000000000000000
>> [   68.870715] RDX: 0000000000000000 RSI: 000000000000000b RDI: 
>> 0000000000000000
>> [   68.870715] RBP: ffff8881e82a7ae8 R08: ffffffffaa3fe2d5 R09: 
>> 0000000000000001
>> [   68.870715] R10: fffffbfff5a88f0f R11: 0000000000000001 R12: 
>> 00000000007eef6a
>> [   68.870715] R13: 0000000000040cc0 R14: 000000000000000b R15: 
>> ffffffffadde766b
>> [   68.870715] FS:  00007fdeb168c600(0000) GS:ffff8881e9800000(0000)
>> knlGS:0000000000000000
>> [   68.870715] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [   68.870715] CR2: 00007fdeb17dd1d6 CR3: 00000001cc2d2002 CR4: 
>> 00000000003606e0
>> [   68.870715] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
>> 0000000000000000
>> [   68.870715] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
>> 0000000000000400
>> [   68.870715] Call Trace:
>> [   68.870715]  ? sched_clock_cpu+0xf5/0x110
>> [   68.870715]  ? __alloc_pages_slowpath.constprop.0+0x17a0/0x17a0
>> [   68.870715]  ? match_held_lock+0x2e/0x240
>> [   68.870715]  ? policy_nodemask+0x1a/0xa0
>> [   68.870715]  ? policy_node+0x56/0x60
>> [   68.870715]  kmalloc_order+0x25/0xc0
>> [   68.870715]  kmalloc_order_trace+0x1d/0x140
>> [   68.870715]  kmemdup+0x1a/0x40
>> [   68.870715]  ima_queue_data+0x61/0x370
>> [   68.870715]  ima_measure_lsm_data+0x32/0x60
>> [   68.870715]  selinux_measure_state+0x2b8/0x2bd
>> [   68.870715]  ? selinux_event_name+0xe0/0xe0
>> [   68.870715]  ? rcu_is_watching+0x39/0x50
>> [   68.870715]  security_load_policy+0x44c/0x8e0
>> [   68.870715]  ? mark_lock+0xa6/0xbd0
>> [   68.870715]  ? security_change_sid+0x90/0x90
>> [   68.870715]  ? mark_held_locks+0x3e/0xa0
>> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
>> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
>> [   68.870715]  ? lockdep_hardirqs_on+0xc5/0x1b0
>> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
>> [   68.870715]  ? copy_user_enhanced_fast_string+0xe/0x30
>> [   68.870715]  sel_write_load+0x157/0x260
>> [   68.870715]  vfs_write+0x135/0x290
>> [   68.870715]  ksys_write+0xb1/0x140
>> [   68.870715]  ? __ia32_sys_read+0x50/0x50
>> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
>> [   68.870715]  ? do_syscall_64+0x12/0xb0
>> [   68.870715]  do_syscall_64+0x52/0xb0
>> [   68.870715]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>> [   68.870715] RIP: 0033:0x7fdeb2539497
>> [   68.870715] Code: Bad RIP value.
>> [   68.870715] RSP: 002b:00007fff6352b308 EFLAGS: 00000246 ORIG_RAX:
>> 0000000000000001
>> [   68.870715] RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 
>> 00007fdeb2539497
>> [   68.870715] RDX: 00000000007eef6a RSI: 00007fdeb0de1000 RDI: 
>> 0000000000000004
>> [   68.870715] RBP: 0000000000000004 R08: 00007fdeb25d0040 R09: 
>> 00007fff6352b1a0
>> [   68.870715] R10: 0000000000000000 R11: 0000000000000246 R12: 
>> 00007fdeb0de1000
>> [   68.870715] R13: 00000000007eef6a R14: 000000000000000f R15: 
>> 0000000000000003
>> [   68.870715] irq event stamp: 23486085
>> [   68.870715] hardirqs last  enabled at (23486085):
>> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
>> [   68.870715] hardirqs last disabled at (23486084):
>> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
>> [   68.870715] softirqs last  enabled at (23486074):
>> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
>> [   68.870715] softirqs last disabled at (23486067):
>> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
>> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
> 
> I think one issue here is that systemd loads SELinux policy first, then 
> IMA policy, so it doesn't know whether it needs to measure SELinux 
> policy on first policy load, and another issue is that the policy is too 
> large to just queue the policy data itself this way (or you need to use 
> an allocator that can handle larger sizes).
> 

The problem seems to be that a lock is held when the IMA hook to measure 
the LSM state is called. So memory allocation is not allowed, but the 
hook is doing an allocation. I'll address this - thanks for catching it.

I have the following CONFIGs enabled, but I still don't see the above 
issue on my machine.

Could you please let me know what CONFIG(s) to enable?

#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
CONFIG_PROVE_LOCKING=y
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_WW_MUTEX_SLOWPATH=y
CONFIG_DEBUG_RWSEMS=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_LOCKDEP=y
CONFIG_DEBUG_ATOMIC_SLEEP=y

I'll also try with the policy you have, check why the LSM policy is 
ignored when there are other policy entries.

thanks,
  -lakshmi
Stephen Smalley Aug. 3, 2020, 9:07 p.m. UTC | #6
On 8/3/20 4:37 PM, Lakshmi Ramasubramanian wrote:

> On 8/3/20 1:29 PM, Stephen Smalley wrote:
>> On 8/3/20 4:00 PM, Stephen Smalley wrote:
>>
>>> On Mon, Aug 3, 2020 at 12:14 PM Lakshmi Ramasubramanian
>>> <nramas@linux.microsoft.com> wrote:
>>>> On 8/3/20 8:11 AM, Stephen Smalley wrote:
>>>>> Possibly I'm missing something but with these patches applied on 
>>>>> top of
>>>>> next-integrity, and the following lines added to /etc/ima/ima-policy:
>>>>>
>>>>> measure func=LSM_STATE template=ima-buf
>>>>> measure func=LSM_POLICY
>>>>>
>>>>> I still don't get the selinux-state or selinux-policy-hash entries in
>>>>> the ascii_runtime_measurements file.  No errors during loading of the
>>>>> ima policy as far as I can see.
>>>>>
>>>> Could you please check if the following config is set?
>>>> CONFIG_IMA_QUEUE_EARLY_BOOT_DATA=y
>>> Yes, I have that set.
>>>
>>>> Try changing /sys/fs/selinux/checkreqprot and check
>>>> ascii_runtime_measurements file again?
>>> No change.  Likewise for changing enforce or running load_policy again.
>>>
>>>> Also, could you please check if
>>>> /sys/kernel/security/integrity/ima/policy contains LSM_STATE and
>>>> LSM_POLICY entries?
>>> Yes, it does.  However, I noticed that if I reduce the policy to only
>>> contain those entries and no others and reboot, then I get
>>> measurements.  Whereas if I append them to an existing policy like the
>>> one below, they seem to be ignored:
>>> dont_measure fsmagic=0x9fa0
>>> dont_measure fsmagic=0x62656572
>>> dont_measure fsmagic=0x64626720
>>> dont_measure fsmagic=0x1021994
>>> dont_measure fsmagic=0x858458f6
>>> dont_measure fsmagic=0x73636673
>>> measure func=BPRM_CHECK
>>> measure func=MMAP_CHECK mask=MAY_EXEC
>>> measure func=MODULE_CHECK uid=0
>>> measure func=LSM_STATE template=ima-buf
>>> measure func=LSM_POLICY
>>>
>>> Also, I noticed the following in my dmesg output:
>>> [   68.870715] ------------[ cut here ]------------
>>> [   68.870715] WARNING: CPU: 2 PID: 1 at mm/page_alloc.c:4826
>>> __alloc_pages_nodemask+0x627/0x700
>>> [   68.870715] Modules linked in: 8139too crct10dif_pclmul
>>> crc32_pclmul crc32c_intel ghash_clmulni_intel qxl serio_raw
>>> drm_ttm_helper ttm drm_kms_helper virtio_console cec drm 8139cp
>>> ata_generic mii pata_acpi floppy qemu_fw_cfg fuse
>>> [   68.870715] CPU: 2 PID: 1 Comm: systemd Not tainted 5.8.0-rc2+ #44
>>> [   68.870715] RIP: 0010:__alloc_pages_nodemask+0x627/0x700
>>> [   68.870715] Code: ff ff 75 6c 48 8b 85 48 ff ff ff 4c 89 c2 44 89
>>> e6 44 89 ff 41 c6 45 d0 00 49 89 45 b8 e8 41 e2 ff ff 49 89 c6 e9 9d
>>> fc ff ff <0f> 0b e9 d4 fd ff ff 0f 0b e9 bc fc ff ff 0f 0b e9 f9 fd ff
>>> ff e8
>>> [   68.870715] RSP: 0000:ffff8881e82a7a18 EFLAGS: 00010246
>>> [   68.870715] RAX: ffffed103d054f48 RBX: 1ffff1103d054f48 RCX: 
>>> 0000000000000000
>>> [   68.870715] RDX: 0000000000000000 RSI: 000000000000000b RDI: 
>>> 0000000000000000
>>> [   68.870715] RBP: ffff8881e82a7ae8 R08: ffffffffaa3fe2d5 R09: 
>>> 0000000000000001
>>> [   68.870715] R10: fffffbfff5a88f0f R11: 0000000000000001 R12: 
>>> 00000000007eef6a
>>> [   68.870715] R13: 0000000000040cc0 R14: 000000000000000b R15: 
>>> ffffffffadde766b
>>> [   68.870715] FS:  00007fdeb168c600(0000) GS:ffff8881e9800000(0000)
>>> knlGS:0000000000000000
>>> [   68.870715] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [   68.870715] CR2: 00007fdeb17dd1d6 CR3: 00000001cc2d2002 CR4: 
>>> 00000000003606e0
>>> [   68.870715] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
>>> 0000000000000000
>>> [   68.870715] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
>>> 0000000000000400
>>> [   68.870715] Call Trace:
>>> [   68.870715]  ? sched_clock_cpu+0xf5/0x110
>>> [   68.870715]  ? __alloc_pages_slowpath.constprop.0+0x17a0/0x17a0
>>> [   68.870715]  ? match_held_lock+0x2e/0x240
>>> [   68.870715]  ? policy_nodemask+0x1a/0xa0
>>> [   68.870715]  ? policy_node+0x56/0x60
>>> [   68.870715]  kmalloc_order+0x25/0xc0
>>> [   68.870715]  kmalloc_order_trace+0x1d/0x140
>>> [   68.870715]  kmemdup+0x1a/0x40
>>> [   68.870715]  ima_queue_data+0x61/0x370
>>> [   68.870715]  ima_measure_lsm_data+0x32/0x60
>>> [   68.870715]  selinux_measure_state+0x2b8/0x2bd
>>> [   68.870715]  ? selinux_event_name+0xe0/0xe0
>>> [   68.870715]  ? rcu_is_watching+0x39/0x50
>>> [   68.870715]  security_load_policy+0x44c/0x8e0
>>> [   68.870715]  ? mark_lock+0xa6/0xbd0
>>> [   68.870715]  ? security_change_sid+0x90/0x90
>>> [   68.870715]  ? mark_held_locks+0x3e/0xa0
>>> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
>>> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
>>> [   68.870715]  ? lockdep_hardirqs_on+0xc5/0x1b0
>>> [   68.870715]  ? asm_exc_page_fault+0x1e/0x30
>>> [   68.870715]  ? copy_user_enhanced_fast_string+0xe/0x30
>>> [   68.870715]  sel_write_load+0x157/0x260
>>> [   68.870715]  vfs_write+0x135/0x290
>>> [   68.870715]  ksys_write+0xb1/0x140
>>> [   68.870715]  ? __ia32_sys_read+0x50/0x50
>>> [   68.870715]  ? lockdep_hardirqs_on_prepare+0x100/0x260
>>> [   68.870715]  ? do_syscall_64+0x12/0xb0
>>> [   68.870715]  do_syscall_64+0x52/0xb0
>>> [   68.870715]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> [   68.870715] RIP: 0033:0x7fdeb2539497
>>> [   68.870715] Code: Bad RIP value.
>>> [   68.870715] RSP: 002b:00007fff6352b308 EFLAGS: 00000246 ORIG_RAX:
>>> 0000000000000001
>>> [   68.870715] RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 
>>> 00007fdeb2539497
>>> [   68.870715] RDX: 00000000007eef6a RSI: 00007fdeb0de1000 RDI: 
>>> 0000000000000004
>>> [   68.870715] RBP: 0000000000000004 R08: 00007fdeb25d0040 R09: 
>>> 00007fff6352b1a0
>>> [   68.870715] R10: 0000000000000000 R11: 0000000000000246 R12: 
>>> 00007fdeb0de1000
>>> [   68.870715] R13: 00000000007eef6a R14: 000000000000000f R15: 
>>> 0000000000000003
>>> [   68.870715] irq event stamp: 23486085
>>> [   68.870715] hardirqs last  enabled at (23486085):
>>> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
>>> [   68.870715] hardirqs last disabled at (23486084):
>>> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
>>> [   68.870715] softirqs last  enabled at (23486074):
>>> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
>>> [   68.870715] softirqs last disabled at (23486067):
>>> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
>>> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
>>
>> I think one issue here is that systemd loads SELinux policy first, 
>> then IMA policy, so it doesn't know whether it needs to measure 
>> SELinux policy on first policy load, and another issue is that the 
>> policy is too large to just queue the policy data itself this way (or 
>> you need to use an allocator that can handle larger sizes).
>>
>
> The problem seems to be that a lock is held when the IMA hook to 
> measure the LSM state is called. So memory allocation is not allowed, 
> but the hook is doing an allocation. I'll address this - thanks for 
> catching it.
>
> I have the following CONFIGs enabled, but I still don't see the above 
> issue on my machine.
>
The warning has to do with the memory allocation order being above the 
max order supported for kmalloc.  I think the problem is that 
ima_alloc_data_entry() is using kmemdup() to duplicate a payload of 
arbitrary size.  Policies on e.g. Fedora can be quite large, so you 
can't assume they can be allocated with kmalloc and friends.
Lakshmi Ramasubramanian Aug. 3, 2020, 10:08 p.m. UTC | #7
On 8/3/20 2:07 PM, Stephen Smalley wrote:

>>>> [   68.870715] irq event stamp: 23486085
>>>> [   68.870715] hardirqs last  enabled at (23486085):
>>>> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
>>>> [   68.870715] hardirqs last disabled at (23486084):
>>>> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
>>>> [   68.870715] softirqs last  enabled at (23486074):
>>>> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
>>>> [   68.870715] softirqs last disabled at (23486067):
>>>> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
>>>> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
>>>
>>> I think one issue here is that systemd loads SELinux policy first, 
>>> then IMA policy, so it doesn't know whether it needs to measure 
>>> SELinux policy on first policy load, and another issue is that the 
>>> policy is too large to just queue the policy data itself this way (or 
>>> you need to use an allocator that can handle larger sizes).
>>>
>>
>> The problem seems to be that a lock is held when the IMA hook to 
>> measure the LSM state is called. So memory allocation is not allowed, 
>> but the hook is doing an allocation. I'll address this - thanks for 
>> catching it.
>>
>> I have the following CONFIGs enabled, but I still don't see the above 
>> issue on my machine.
>>
> The warning has to do with the memory allocation order being above the 
> max order supported for kmalloc.  I think the problem is that 
> ima_alloc_data_entry() is using kmemdup() to duplicate a payload of 
> arbitrary size.  Policies on e.g. Fedora can be quite large, so you 
> can't assume they can be allocated with kmalloc and friends.
> 

Thanks for clarifying. Yes ima_alloc_entry() does use kmemdup to save 
the given buffer (to be measured) until IMA loads custom policy.

On my machine the SELinux policy size is about 2MB.

Perhaps vmalloc would be better than using kmalloc? If there are better 
options for such large buffer allocation, please let me know.

  -lakshmi
Stephen Smalley Aug. 4, 2020, 3:20 p.m. UTC | #8
On 8/3/20 6:08 PM, Lakshmi Ramasubramanian wrote:

> On 8/3/20 2:07 PM, Stephen Smalley wrote:
>
>>>>> [   68.870715] irq event stamp: 23486085
>>>>> [   68.870715] hardirqs last  enabled at (23486085):
>>>>> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
>>>>> [   68.870715] hardirqs last disabled at (23486084):
>>>>> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
>>>>> [   68.870715] softirqs last  enabled at (23486074):
>>>>> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
>>>>> [   68.870715] softirqs last disabled at (23486067):
>>>>> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
>>>>> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
>>>>
>>>> I think one issue here is that systemd loads SELinux policy first, 
>>>> then IMA policy, so it doesn't know whether it needs to measure 
>>>> SELinux policy on first policy load, and another issue is that the 
>>>> policy is too large to just queue the policy data itself this way 
>>>> (or you need to use an allocator that can handle larger sizes).
>>>>
>>>
>>> The problem seems to be that a lock is held when the IMA hook to 
>>> measure the LSM state is called. So memory allocation is not 
>>> allowed, but the hook is doing an allocation. I'll address this - 
>>> thanks for catching it.
>>>
>>> I have the following CONFIGs enabled, but I still don't see the 
>>> above issue on my machine.
>>>
>> The warning has to do with the memory allocation order being above 
>> the max order supported for kmalloc.  I think the problem is that 
>> ima_alloc_data_entry() is using kmemdup() to duplicate a payload of 
>> arbitrary size.  Policies on e.g. Fedora can be quite large, so you 
>> can't assume they can be allocated with kmalloc and friends.
>>
>
> Thanks for clarifying. Yes ima_alloc_entry() does use kmemdup to save 
> the given buffer (to be measured) until IMA loads custom policy.
>
> On my machine the SELinux policy size is about 2MB.
>
> Perhaps vmalloc would be better than using kmalloc? If there are 
> better options for such large buffer allocation, please let me know.

kvmalloc() can be used to select whichever one is most appropriate.
Stephen Smalley Aug. 4, 2020, 3:29 p.m. UTC | #9
On 8/4/20 11:20 AM, Stephen Smalley wrote:

> On 8/3/20 6:08 PM, Lakshmi Ramasubramanian wrote:
>
>> On 8/3/20 2:07 PM, Stephen Smalley wrote:
>>
>>>>>> [   68.870715] irq event stamp: 23486085
>>>>>> [   68.870715] hardirqs last  enabled at (23486085):
>>>>>> [<ffffffffaa419406>] _raw_spin_unlock_irqrestore+0x46/0x60
>>>>>> [   68.870715] hardirqs last disabled at (23486084):
>>>>>> [<ffffffffaa419443>] _raw_spin_lock_irqsave+0x23/0x90
>>>>>> [   68.870715] softirqs last  enabled at (23486074):
>>>>>> [<ffffffffaa8004f3>] __do_softirq+0x4f3/0x662
>>>>>> [   68.870715] softirqs last disabled at (23486067):
>>>>>> [<ffffffffaa601072>] asm_call_on_stack+0x12/0x20
>>>>>> [   68.870715] ---[ end trace fb02740ff6f4d0cd ]---
>>>>>
>>>>> I think one issue here is that systemd loads SELinux policy first, 
>>>>> then IMA policy, so it doesn't know whether it needs to measure 
>>>>> SELinux policy on first policy load, and another issue is that the 
>>>>> policy is too large to just queue the policy data itself this way 
>>>>> (or you need to use an allocator that can handle larger sizes).
>>>>>
>>>>
>>>> The problem seems to be that a lock is held when the IMA hook to 
>>>> measure the LSM state is called. So memory allocation is not 
>>>> allowed, but the hook is doing an allocation. I'll address this - 
>>>> thanks for catching it.
>>>>
>>>> I have the following CONFIGs enabled, but I still don't see the 
>>>> above issue on my machine.
>>>>
>>> The warning has to do with the memory allocation order being above 
>>> the max order supported for kmalloc.  I think the problem is that 
>>> ima_alloc_data_entry() is using kmemdup() to duplicate a payload of 
>>> arbitrary size.  Policies on e.g. Fedora can be quite large, so you 
>>> can't assume they can be allocated with kmalloc and friends.
>>>
>>
>> Thanks for clarifying. Yes ima_alloc_entry() does use kmemdup to save 
>> the given buffer (to be measured) until IMA loads custom policy.
>>
>> On my machine the SELinux policy size is about 2MB.
>>
>> Perhaps vmalloc would be better than using kmalloc? If there are 
>> better options for such large buffer allocation, please let me know.
>
> kvmalloc() can be used to select whichever one is most appropriate.

Other option would be for ima to compute and save the hash(es) of the 
payload and not the payload itself for later use.  I guess you won't 
know at that point which hash algorithm is desired?
Lakshmi Ramasubramanian Aug. 4, 2020, 3:57 p.m. UTC | #10
On 8/4/20 8:29 AM, Stephen Smalley wrote:

>>> Perhaps vmalloc would be better than using kmalloc? If there are 
>>> better options for such large buffer allocation, please let me know.
>>
>> kvmalloc() can be used to select whichever one is most appropriate.
> 
> Other option would be for ima to compute and save the hash(es) of the 
> payload and not the payload itself for later use.  I guess you won't 
> know at that point which hash algorithm is desired?
> 

I think IMA hash algorithm would be known at that point, but IMA policy 
is not loaded yet (which is why I need to queue up the buffer and 
process when policy is loaded).

I tried vmalloc and tested it with upto 16MB buffer (just made up a 
SELinux policy buffer of size 16MB) - that works fine.

I will try kvmalloc().

Also, I fixed the issue with LSM data not measured when using the IMA 
policy you had. Good catch.

Will post the updated patches today.

thanks,
  -lakshmi

Patch
diff mbox series

diff --git a/security/selinux/Makefile b/security/selinux/Makefile
index 4d8e0e8adf0b..83d512116341 100644
--- a/security/selinux/Makefile
+++ b/security/selinux/Makefile
@@ -16,6 +16,8 @@  selinux-$(CONFIG_NETLABEL) += netlabel.o
 
 selinux-$(CONFIG_SECURITY_INFINIBAND) += ibpkey.o
 
+selinux-$(CONFIG_IMA) += measure.o
+
 ccflags-y := -I$(srctree)/security/selinux -I$(srctree)/security/selinux/include
 
 $(addprefix $(obj)/,$(selinux-y)): $(obj)/flask.h
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index efa6108b1ce9..5521dfc1900b 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -7394,6 +7394,7 @@  int selinux_disable(struct selinux_state *state)
 	}
 
 	selinux_mark_disabled(state);
+	selinux_measure_state(state);
 
 	pr_info("SELinux:  Disabled at runtime.\n");
 
diff --git a/security/selinux/include/security.h b/security/selinux/include/security.h
index b0e02cfe3ce1..77f42d9b544b 100644
--- a/security/selinux/include/security.h
+++ b/security/selinux/include/security.h
@@ -222,16 +222,31 @@  static inline bool selinux_policycap_genfs_seclabel_symlinks(void)
 	return state->policycap[POLICYDB_CAPABILITY_GENFS_SECLABEL_SYMLINKS];
 }
 
+static inline bool selinux_checkreqprot(const struct selinux_state *state)
+{
+	return READ_ONCE(state->checkreqprot);
+}
+
 int security_mls_enabled(struct selinux_state *state);
 int security_load_policy(struct selinux_state *state,
 			 void *data, size_t len);
 int security_read_policy(struct selinux_state *state,
 			 void **data, size_t *len);
+int security_read_policy_kernel(struct selinux_state *state,
+				void **data, size_t *len);
 size_t security_policydb_len(struct selinux_state *state);
 
 int security_policycap_supported(struct selinux_state *state,
 				 unsigned int req_cap);
 
+#ifdef CONFIG_IMA
+extern void selinux_measure_state(struct selinux_state *selinux_state);
+#else
+static inline void selinux_measure_state(struct selinux_state *selinux_state)
+{
+}
+#endif
+
 #define SEL_VEC_MAX 32
 struct av_decision {
 	u32 allowed;
diff --git a/security/selinux/measure.c b/security/selinux/measure.c
new file mode 100644
index 000000000000..1583628d09d1
--- /dev/null
+++ b/security/selinux/measure.c
@@ -0,0 +1,150 @@ 
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Measure SELinux state using IMA subsystem.
+ */
+#include <linux/vmalloc.h>
+#include <linux/ktime.h>
+#include <linux/ima.h>
+#include "security.h"
+
+/*
+ * This function creates an unique name by appending the timestamp to
+ * the given string. This string is passed as "event name" to the IMA
+ * hook to measure the given SELinux data.
+ *
+ * The data provided by SELinux to the IMA subsystem for measuring may have
+ * already been measured (for instance the same state existed earlier).
+ * But for SELinux the current data represents a state change and hence
+ * needs to be measured again. To enable this, pass an unique "Event Name"
+ * to the IMA hook so that IMA subsystem will always measure the given data.
+ *
+ * For example,
+ * At time T0 SELinux data to be measured is "foo". IMA measures it.
+ * At time T1 the data is changed to "bar". IMA measures it.
+ * At time T2 the data is changed to "foo" again. IMA will not measure it
+ * (since it was already measured) unless the event name, for instance,
+ * is different in this call.
+ */
+static char *selinux_event_name(const char *name_prefix)
+{
+	char *event_name = NULL;
+	struct timespec64 curr_time;
+	int count;
+
+	ktime_get_real_ts64(&curr_time);
+	count = snprintf(NULL, 0, "%s-%lld:%09ld", name_prefix,
+			 curr_time.tv_sec, curr_time.tv_nsec);
+	count++;
+	event_name = kzalloc(count, GFP_KERNEL);
+	if (!event_name) {
+		pr_warn("%s: event name not allocated.\n", __func__);
+		return NULL;
+	}
+
+	snprintf(event_name, count, "%s-%lld:%09ld", name_prefix,
+		 curr_time.tv_sec, curr_time.tv_nsec);
+
+	return event_name;
+}
+
+static int read_selinux_state(char **state_str, int *state_str_len,
+			      struct selinux_state *state)
+{
+	char *buf, *str_fmt = "%s=%d;";
+	int i, buf_len, curr;
+
+	buf_len = snprintf(NULL, 0, str_fmt, "initialized", 0);
+	buf_len += snprintf(NULL, 0, str_fmt, "enabled", 0);
+	buf_len += snprintf(NULL, 0, str_fmt, "enforcing", 0);
+	buf_len += snprintf(NULL, 0, str_fmt, "checkreqprot", 0);
+
+	for (i = 0; i < __POLICYDB_CAPABILITY_MAX; i++) {
+		buf_len += snprintf(NULL, 0, str_fmt,
+				    selinux_policycap_names[i], 0);
+	}
+	++buf_len;
+
+	buf = kzalloc(buf_len, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	curr = snprintf(buf, buf_len, str_fmt,
+			"initialized", selinux_initialized(state));
+	curr += snprintf((buf + curr), (buf_len - curr), str_fmt,
+			 "enabled", !selinux_disabled(state));
+	curr += snprintf((buf + curr), (buf_len - curr), str_fmt,
+			 "enforcing", enforcing_enabled(state));
+	curr += snprintf((buf + curr), (buf_len - curr), str_fmt,
+			 "checkreqprot", selinux_checkreqprot(state));
+
+	for (i = 0; i < __POLICYDB_CAPABILITY_MAX; i++) {
+		curr += snprintf((buf + curr), (buf_len - curr), str_fmt,
+				 selinux_policycap_names[i],
+				 state->policycap[i]);
+	}
+
+	*state_str = buf;
+	*state_str_len = curr;
+
+	return 0;
+}
+
+void selinux_measure_state(struct selinux_state *state)
+{
+	void *policy = NULL;
+	char *event_name = NULL;
+	char *state_str = NULL;
+	size_t policy_len;
+	int state_str_len, rc = 0;
+	bool initialized = selinux_initialized(state);
+
+	rc = read_selinux_state(&state_str, &state_str_len, state);
+	if (rc) {
+		pr_warn("%s: Failed to read selinux state.\n", __func__);
+		return;
+	}
+
+	/*
+	 * Get an unique string for measuring the current SELinux state.
+	 */
+	event_name = selinux_event_name("selinux-state");
+	if (!event_name) {
+		pr_warn("%s: Event name for state not allocated.\n",
+			__func__);
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	rc = ima_measure_lsm_state(event_name, state_str, state_str_len);
+
+	kfree(event_name);
+	event_name = NULL;
+
+	if (rc)
+		goto out;
+
+	/*
+	 * Measure SELinux policy only after initialization is completed.
+	 */
+	if (!initialized)
+		goto out;
+
+	rc = security_read_policy_kernel(state, &policy, &policy_len);
+	if (rc)
+		goto out;
+
+	event_name = selinux_event_name("selinux-policy-hash");
+	if (!event_name) {
+		pr_warn("%s: Event name for policy not allocated.\n",
+			__func__);
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	rc = ima_measure_lsm_policy(event_name, policy, policy_len);
+
+out:
+	kfree(event_name);
+	kfree(state_str);
+	vfree(policy);
+}
diff --git a/security/selinux/selinuxfs.c b/security/selinux/selinuxfs.c
index 4781314c2510..6d46eaef5c92 100644
--- a/security/selinux/selinuxfs.c
+++ b/security/selinux/selinuxfs.c
@@ -173,6 +173,7 @@  static ssize_t sel_write_enforce(struct file *file, const char __user *buf,
 			from_kuid(&init_user_ns, audit_get_loginuid(current)),
 			audit_get_sessionid(current));
 		enforcing_set(state, new_value);
+		selinux_measure_state(state);
 		if (new_value)
 			avc_ss_reset(state->avc, 0);
 		selnl_notify_setenforce(new_value);
@@ -678,6 +679,8 @@  static ssize_t sel_write_checkreqprot(struct file *file, const char __user *buf,
 
 	fsi->state->checkreqprot = new_value ? 1 : 0;
 	length = count;
+
+	selinux_measure_state(fsi->state);
 out:
 	kfree(page);
 	return length;
diff --git a/security/selinux/ss/services.c b/security/selinux/ss/services.c
index ef0afd878bfc..3978c804c361 100644
--- a/security/selinux/ss/services.c
+++ b/security/selinux/ss/services.c
@@ -2182,6 +2182,7 @@  int security_load_policy(struct selinux_state *state, void *data, size_t len)
 		selinux_status_update_policyload(state, seqno);
 		selinux_netlbl_cache_invalidate();
 		selinux_xfrm_notify_policyload();
+		selinux_measure_state(state);
 		return 0;
 	}
 
@@ -2270,6 +2271,7 @@  int security_load_policy(struct selinux_state *state, void *data, size_t len)
 	selinux_status_update_policyload(state, seqno);
 	selinux_netlbl_cache_invalidate();
 	selinux_xfrm_notify_policyload();
+	selinux_measure_state(state);
 
 	rc = 0;
 	goto out;
@@ -2941,6 +2943,7 @@  int security_set_bools(struct selinux_state *state, u32 len, int *values)
 		selnl_notify_policyload(seqno);
 		selinux_status_update_policyload(state, seqno);
 		selinux_xfrm_notify_policyload();
+		selinux_measure_state(state);
 	}
 	return rc;
 }
@@ -3720,14 +3723,23 @@  int security_netlbl_sid_to_secattr(struct selinux_state *state,
 }
 #endif /* CONFIG_NETLABEL */
 
+static int security_read_policy_len(struct selinux_state *state, size_t *len)
+{
+	if (!selinux_initialized(state))
+		return -EINVAL;
+
+	*len = security_policydb_len(state);
+	return 0;
+}
+
 /**
- * security_read_policy - read the policy.
+ * security_read_selinux_policy - read the policy.
  * @data: binary policy data
  * @len: length of data in bytes
  *
  */
-int security_read_policy(struct selinux_state *state,
-			 void **data, size_t *len)
+static int security_read_selinux_policy(struct selinux_state *state,
+					void **data, size_t *len)
 {
 	struct policydb *policydb = &state->ss->policydb;
 	int rc;
@@ -3736,12 +3748,6 @@  int security_read_policy(struct selinux_state *state,
 	if (!selinux_initialized(state))
 		return -EINVAL;
 
-	*len = security_policydb_len(state);
-
-	*data = vmalloc_user(*len);
-	if (!*data)
-		return -ENOMEM;
-
 	fp.data = *data;
 	fp.len = *len;
 
@@ -3754,5 +3760,52 @@  int security_read_policy(struct selinux_state *state,
 
 	*len = (unsigned long)fp.data - (unsigned long)*data;
 	return 0;
+}
+
+/**
+ * security_read_policy - read the policy.
+ * @data: binary policy data
+ * @len: length of data in bytes
+ *
+ */
+int security_read_policy(struct selinux_state *state,
+			 void **data, size_t *len)
+{
+	int rc;
+
+	rc = security_read_policy_len(state, len);
+	if (rc)
+		return rc;
+
+	*data = vmalloc_user(*len);
+	if (!*data)
+		return -ENOMEM;
+
+	return security_read_selinux_policy(state, data, len);
+}
+
+/**
+ * security_read_policy_kernel - read the policy.
+ * @data: binary policy data
+ * @len: length of data in bytes
+ *
+ * Allocates kernel memory for reading SELinux policy.
+ * This function is for internal use only and should not
+ * be used for returning data to user space
+ *
+ */
+int security_read_policy_kernel(struct selinux_state *state,
+				void **data, size_t *len)
+{
+	int rc;
+
+	rc = security_read_policy_len(state, len);
+	if (rc)
+		return rc;
+
+	*data = vmalloc(*len);
+	if (!*data)
+		return -ENOMEM;
 
+	return security_read_selinux_policy(state, data, len);
 }