Security:Analysing security privileges of tizen services

From Tizen Wiki
Jump to: navigation, search

Target

This document should be read by Tizen service developers and maintainers (especially if the service runs as the privileged "root" user). A service is considered a non UI program running on the Tizen platform (a daemon) that allows access to one or more services or hardware for other applications. Some Tizen services run as the privileged "root" user, however all service developers are urged to consider running their software as non-privileged users, with fine grained permissions or capabilities granted, due to security issues.


Considerations

Tizen is designed to be a secure system. Its specification for mobile profile requires that there should not be any set-user-id binaries in the device. However, not only set-user binary, but every root process (service) is a potential security vulnerability. Generally, the less rights a process has, the more secure the system is. Still, without any rights, a service can not perform its tasks, that's why, for each process a compromise must be found. This compromise might be called a minimum set of rights (given by a process effective/file/real user, group ids and capabilities, process smack labels) and the environment (i.e file ownership, smack rules).

The goal of this document is to provide you with some knowledge how to analyse the service you maintain and find out which rights are needed and which are not. We do not provide the complete solution how to fix it, as it might be as it might be a project specific (i.e you should consider whether spiting a big service with a big set of rights into several smaller ones with small set of rights is possible). We give you some introduction into fixing techniques though.

Process

  • Check whether a service run as a root (real or effective uid)
  • Perform the analysis stage (needed resources and rights)
  • Change the capabilities, user id, groups, smack label of a process (binary) and system environment (i.e. owners/access rights to files) so that only needed rights are gained

Analysis stage

When analysing a process that's already running and providing a set of services, start with dropping all its privileges to a non-root user (how to choose non-root user name is described in Dac Policy chapter). You'll end up with a non-working service, but it will be easier to analyse what the service needs in terms of permissions.

Perparation of service run by systemd

To run a process under a different dedicated user with a set of linux capabilities, modify the .service file that describes this service. All services have their respective .service files in /var/lib/systemd/system directory. Service files are described in the systemd manual, the configuration options to change in the .service file are:

  • User - username that this process runs as.
  • Group - groupname that this process runs as.
  • SupplementaryGroups - Sets the supplementary Unix groups the processes are executed as.
  • WorkingDirectory - Takes an absolute directory path. Sets the working directory for executed processes.
  • CapabilityBoundingSet - Controls which capabilities to include in the capability bounding set for the executed process.

Once you have prepared the .service files, reload the systemd daemon (systemctl daemon-reload) and try starting your service, it will probably fail, but that's ok, this is a starting point only. To see what the service is reporting use the journalctl command (journalctl -xn).

Example .service

[Unit]
# A brief description of what the service does, will appear in the output from systemctl
Description=Tizen - Example service daemon

# We need networking and logging to be already up and running
After=network.target syslog.target

[Service]
# The user we run as
User=usersvc

# The group the user belongs to
Group=services

# Before our service starts, we execute a check to see if we can run at all
ExecStartPre=/usr/bin/validate-service

# Out main program starts here
ExecStart=/usr/sbin/service

# To reload our service, send it a HUP signal
ExecReload=/bin/kill -HUP $MAINPID

# We are process, to stop it we use kill()
KillMode=process

# When to restart the service
Restart=always

# What other targets need us
[Install]
WantedBy=multi-user.target

Real world example (security-server)

For a real example to run the security-server as a non-root user with once capability set (cap_mac_admin) we use the below .service file

[Unit]
Description=Start the security server

[Service]
Type=notify
User=secsrvr
Group=system
#let inherit cap_mac_admin from systemd
Capabilities=cap_mac_admin=i
#do not let to privilege escalation above cap_mac_admin
CapabilityBoundingSet=cap_mac_admin
SecureBits=keep-caps

ExecStart=/usr/bin/security-server
Sockets=security-server-data-share.socket
Sockets=security-server-get-gid.socket
Sockets=security-server-privilege-by-pid.socket
Sockets=security-server-app-privilege-by-name.socket
Sockets=security-server-cookie-get.socket
Sockets=security-server-cookie-check.socket
Sockets=security-server-password-check.socket
Sockets=security-server-password-set.socket
Sockets=security-server-password-reset.socket

[Install]
WantedBy=multi-user.target

Additionally we need to set special DAC permissions and capabilities on the /usr/bin/security-server binary:

Don't let anyone but root and members of UNIX group system to execute security-server

[root@localhost:~]# chmod o-x /usr/bin/security-server

Change the group ownership of security-server to system

[root@localhost:~]# chgrp system /usr/bin/security-server

Set the Linux capabilities for security server (cap_mac_admin). Note that we don't want to add cap_mac_admin to permitted set.

[root@localhost:~]# setcap cap_mac_admin+ie /usr/bin/security-server

Let's check how it looks like now:

[root@localhost:~]# ls -al /usr/bin/security-server 
-rwxr-xr-- 1 root system 314924 Mar 10  2014 /usr/bin/security-server
[root@localhost:~]# getcap /usr/bin/security-server 
/usr/bin/security-server = cap_mac_admin+ep

Now the process security-server runs as a non-privileged user. It will be able to control the MAC (SMACK) subsystem thanks to the capability cap_mac_admin, if we look at how the process runs now, we will notice it's effective capability set:

[root@localhost:/usr/lib/systemd/system]# id secsrvr
uid=10000(secsrvr) gid=1000(system) groups=1000(system),44(video)
[root@localhost:/usr/lib/systemd/system]# cat /proc/`pidof security-server`/status 
Name:	security-server
State:	S (sleeping)
Tgid:	18264
Pid:	18264
PPid:	1
TracerPid:	0
Uid:	10000	10000	10000	10000
Gid:	1000	1000	1000	1000
[...]
CapInh:	0000000200000000
CapPrm: 0000000200000000
CapEff:	0000000200000000
CapBnd:	0000000200000000
[...]

Preparation of standalone process

All services in Tizen should be started from systemd. If you are trying to test a program and are able to launch it from the command line, you can do that but remember that systemd might provide a different environment for the process later. make sure to run the process as the target user, use sudo or just login as the target user to the development operating system, this is a easy way to run [#strace] on your process and see the output on the console. Make sure to set the environment variables used by the process correctly when using sudo, to check what environment looks like use the set and the env commands

Files and directories

Start by checking if the process can access it's configuration files and create it's pid files, log files, unix sockets (if any). If the target process does not start use [#strace] to see what the process is failing on. Strace output will help you find failed open() syscalls and might lead you to the culprit. The audit subsystem can also log all failed open() and access() attempts, when dealing with multiple processes this can be more helpful.

For example, when running a process that can't access it's configuration file:

developer@AMDC2543:~$ id -a; strace -e trace=open /usr/sbin/zabbix_agentd 
[cut]
open("/lib/x86_64-linux-gnu/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/etc/zabbix/zabbix_agentd.conf", O_RDONLY) = -1 EACCES (Permission denied)
zabbix_agentd [5128]: cannot open config file [/etc/zabbix/zabbix_agentd.conf]: [13] Permission denied

Sockets and networking

When running as a normal non-root user you can't use network ports lower then 1024. To gain access to such ports you need a special capability or a special initialization procedure to gain access to that port first. If possible avoid using those low port numbers, they are well defined by IANA and using them is not recommended. You can check if your process if failing to acquire a low number port as it will fail in the bind() syscall

[r.kubiak@AMDC2543:~/devel]$ strace -e trace=bind ./echosrv 128
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=5761, si_uid=1354787703} ---
[OK] Start
[OK] socket created flags AF_INET and SOCK_STREAM, descriptor [3]
bind(3, {sa_family=AF_INET, sin_port=htons(128), sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EACCES (Permission denied)
[ERROR] bind(): Permission denied
+++ exited with 1 +++

Creating UNIX sockets requires write permissions to the directory the socket is created in, this causes a problem for the user that runs the process and poses a security threat to the system (if badly configured).

[r.kubiak@AMDC2543:~/devel]$ strace -e trace=bind ./echo-unix sockets/test
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=7931, si_uid=1354787703} ---
[INFO] using socket: sockets/test
bind(3, {sa_family=AF_FILE, sun_path="sockets/test"}, 110) = -1 EACCES (Permission denied)
[ERROR] bind(): Permission denied
+++ exited with 1 +++

Please notice that the directory that has the socket needs the execute permissions, for example:

[root@AMDC2543:/home/r.kubiak/devel]# ls -ald sockets
drwx----wx 2 root root 4096 Apr  2 17:20 sockets

[r.kubiak@AMDC2543:~/devel]$ strace -e trace=bind ./echo-unix sockets/test
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=8251, si_uid=1354787703} ---
[INFO] using socket: sockets/test
bind(3, {sa_family=AF_FILE, sun_path="sockets/test"}, 110) = 0

The above example runs fine

[root@AMDC2543:/home/r.kubiak/devel]# ls -ald sockets
drwx----w- 2 root root 4096 Apr  2 17:21 sockets

[r.kubiak@AMDC2543:~/devel]$ strace -e trace=bind ./echo-unix sockets/test
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=8284, si_uid=1354787703} ---
[INFO] using socket: sockets/test
bind(3, {sa_family=AF_FILE, sun_path="sockets/test"}, 110) = -1 EACCES (Permission denied)
[ERROR] bind(): Permission denied
+++ exited with 1 +++

This fails due to sockets directory permissions.

Tools

LSOF

List open files. An open file may be a regular file, a directory, a block special file, a character special file, an executing text reference, a library, a stream or a network file (Internet socket, NFS file or UNIX domain socket.) A specific file or all the files in a file system may be selected by path.

In addition to producing a single output list, lsof will run in repeat mode. In repeat mode it will produce output, delay, then repeat the output operation until stopped with an interrupt or quit signal. See the +|-r [t[m<fmt>]] option description for more information (useful for continues monitoring)

An extensive tutorial on using LSOF

To get information about a command running we can use lsof:

[root@localhost:/etc/audit]# lsof -c connmand
COMMAND   PID USER   FD      TYPE     DEVICE SIZE/OFF   NODE NAME
connmand 5737 root  cwd       DIR      179,5     4096      2 /
connmand 5737 root  rtd       DIR      179,5     4096      2 /
connmand 5737 root  txt       REG      179,5   711584  23268 /usr/sbin/connmand
connmand 5737 root  mem       REG      179,5   114832  11859 /usr/lib/libp11-kit.so.0.0.0
connmand 5737 root  mem       REG      179,5    73356   1841 /usr/lib/libz.so.1.2.7
connmand 5737 root  mem       REG      179,5   381524   2144 /usr/lib/libgmp.so.10.1.3
connmand 5737 root  mem       REG      179,5    72672   9247 /usr/lib/libhogweed.so.2.1
connmand 5737 root  mem       REG      179,5   165968   9076 /usr/lib/libnettle.so.4.3
connmand 5737 root  mem       REG      179,5    51992  11851 /usr/lib/libtasn1.so.3.2.0
connmand 5737 root  mem       REG      179,5   283860   2023 /usr/lib/libpcre.so.1.0.1
connmand 5737 root  mem       REG      179,5  1220936   1763 /usr/lib/libc-2.18.so
connmand 5737 root  mem       REG      179,5   229584   1825 /usr/lib/libgcc_s.so.1
connmand 5737 root  mem       REG      179,5    35276   1793 /usr/lib/librt-2.18.so
connmand 5737 root  mem       REG      179,5    15120   1769 /usr/lib/libdl-2.18.so
connmand 5737 root  mem       REG      179,5    75428   1791 /usr/lib/libresolv-2.18.so
connmand 5737 root  mem       REG      179,5   823808  11874 /usr/lib/libgnutls.so.28.11.5
connmand 5737 root  mem       REG      179,5    47208   5152 /usr/lib/libxtables.so.10.0.0
connmand 5737 root  mem       REG      179,5   113604   1789 /usr/lib/libpthread-2.18.so
connmand 5737 root  mem       REG      179,5   236888   1838 /usr/lib/libdbus-1.so.3.7.4
connmand 5737 root  mem       REG      179,5   970340   2028 /usr/lib/libglib-2.0.so.0.3600.4
connmand 5737 root  mem       REG      179,5    46844  23263 /usr/lib/connman/plugins/telephony.so
connmand 5737 root  mem       REG      179,5    22456   9223 /usr/lib/libsys-assert.so
connmand 5737 root  mem       REG      179,5   122456   1756 /usr/lib/ld-2.18.so
connmand 5737 root    0r      CHR        1,3      0t0   1059 /dev/null
connmand 5737 root    1w      CHR        1,3      0t0   1059 /dev/null
connmand 5737 root    2w      CHR        1,3      0t0   1059 /dev/null
connmand 5737 root    3u  a_inode        0,9        0   6319 [eventfd]
connmand 5737 root    4u  a_inode        0,9        0   6319 [signalfd]
connmand 5737 root    5u     unix 0xed600240      0t0 256556 socket
connmand 5737 root    6u     unix 0xee3ff180      0t0 257626 socket
connmand 5737 root    7r  a_inode        0,9        0   6319 inotify
connmand 5737 root    8r  a_inode        0,9        0   6319 inotify
connmand 5737 root    9u     IPv4     257634      0t0    UDP localhost.localdomain:domain 
connmand 5737 root   10u     IPv6     257639      0t0    UDP localhost6.localdomain6:domain 
connmand 5737 root   11u     IPv4     257643      0t0    TCP localhost.localdomain:domain (LISTEN)
connmand 5737 root   12u     IPv6     257647      0t0    TCP localhost6.localdomain6:domain (LISTEN)
connmand 5737 root   13u  netlink                 0t0 257653 ROUTE

This gives us information about network connections and opened files by connmand.

STRACE

Strace is used for more extensive debugging of applications. In many ways strace can provide more information about a process behavior and what it needs in terms of permissions and privileges. The simplest way to trace a process is to start it under strace, if you know that the process you are analyzing uses the fork() syscall, use the command line parameter -f to strace. Also remember that strace prints to STDERR by default so in order to save the output of strace make sure to transfer that stream to a file

# strace command 2> output.txt

for example:

# strace -f /usr/sbin/sshd
[cut]
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [1], 4) = 0
bind(3, {sa_family=AF_INET6, sin6_port=htons(22), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRINUSE (Address already in use)
rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
gettimeofday({946696984, 995441}, NULL) = 0
socket(PF_LOCAL, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 4
[even more output...]

Notice that bind() is failing with EADDRINUSE, that tells us that something is already using the port 22 (that's ok because we know another SSH daemon is listening). Let's use lsof from the point above to find what is using that port:

[root@localhost:~]# lsof -i :22
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd     1 root   13u  IPv6   9118      0t0  TCP *:ssh (LISTEN)
systemd     1 root   14u  IPv6 387265      0t0  TCP 192.168.129.3:ssh->host:38826 (ESTABLISHED)
sshd    11647 root    2u  IPv6 387265      0t0  TCP 192.168.129.3:ssh->host:38826 (ESTABLISHED)
sshd    11647 root    3u  IPv6 387265      0t0  TCP 192.168.129.3:ssh->host:38826 (ESTABLISHED)
sshd    11647 root    4u  IPv6 387265      0t0  TCP 192.168.129.3:ssh->host:38826 (ESTABLISHED)

Some useful optins for strace

  • -y prints information about descriptors, that means we'll get a path for each syscall instead of a number
# strace -y -f /usr/sbin/sshd
open("/proc/self/oom_score_adj", O_RDWR|O_LARGEFILE) = 3
fstat64(3</proc/11775/oom_score_adj>, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6eb9000
read(3</proc/11775/oom_score_adj>, "0\n", 1024) = 2
_llseek(3</proc/11775/oom_score_adj>, 0, [0], SEEK_SET) = 0
write(3</proc/11775/oom_score_adj>, "-1000\n", 6) = 6
close(3</proc/11775/oom_score_adj>)     = 0
munmap(0xb6eb9000, 4096)                = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
fcntl64(3<socket:[387836]>, F_GETFL)    = 0x2 (flags O_RDWR)
fcntl64(3<socket:[387836]>, F_SETFL, O_RDWR|O_NONBLOCK) = 0
  • -s limits the length of printed strings that occur in syscall parameters, the default limit is 32, sometimes we might need more to get some information
# strace -s 256 -y -f /usr/sbin/sshd
open("/etc/ssh/ssh_host_rsa_key", O_RDONLY|O_LARGEFILE) = 3
fstat64(3</etc/ssh/ssh_host_rsa_key>, {st_mode=S_IFREG|0600, st_size=1675, ...}) = 0
getuid32()                              = 0
fstat64(3</etc/ssh/ssh_host_rsa_key>, {st_mode=S_IFREG|0600, st_size=1675, ...}) = 0
read(3</etc/ssh/ssh_host_rsa_key>, "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAoR61d8QYtn6lF+eKRllseri2t+CsqKymxTV6b6pNjeutrfZV\nlP1EyNoy77VMbhVtEAJn/CMWVp9JQzpS0SCY9fcsA/z6euXYeyyED0/6FFGz51dq\nY0Uilg6t4rXEDiiZWclP+FUn8uT+mR9zw5lpcnY5F8rL7eXf2EL/ZTI2CkbepcyZ\nw4J6swJ0it6QIOakj9F6gJjAXL6fw"..., 1024) = 1024
read(3</etc/ssh/ssh_host_rsa_key>, "RqZnPHUqvjc/hQpT3Ldm1sCao3UNbc9/kSY/U+gDpZbhrEq\n8fLKHYznGPRCXbEQBx0MYDoB6ik3tHIM1paLv5ECgYBmOKqgEDqMDC/qcA7JZAGr\nK704t6QEhvf9RqWPM3FI8FTl/4N0XVW5COvvcxFh6FthJvwjl63wYIrj2XB6Z6JL\niJXHxR18na8/NopZgnke9+xapiKhtPNaYNU0X8d/6WyemXUH6JMjJnrlVgaMCHnh\npr6v+cCtTX1Qn"..., 1024) = 651
read(3</etc/ssh/ssh_host_rsa_key>, "", 373) = 0
read(3</etc/ssh/ssh_host_rsa_key>, "", 1024) = 0
close(3</etc/ssh/ssh_host_rsa_key>)     = 0
gettimeofday({946697507, 477767}, NULL) = 0
open("/etc/ssh/ssh_host_dsa_key", O_RDONLY|O_LARGEFILE) = 3
fstat64(3</etc/ssh/ssh_host_dsa_key>, {st_mode=S_IFREG|0600, st_size=672, ...}) = 0
getuid32()                              = 0
fstat64(3</etc/ssh/ssh_host_dsa_key>, {st_mode=S_IFREG|0600, st_size=672, ...}) = 0
read(3</etc/ssh/ssh_host_dsa_key>, "-----BEGIN DSA PRIVATE KEY-----\nMIIBvAIBAAKBgQCxheVuQguFUcH3hXobp0OzDIsZL3ai9E2kpSO9l9YDB8DMK1Y2\no3Li0Qby+7v9B2b47VmTVUwyywiOQ3C+cIJ46euwR44RHA/uw6TI8BbO3p39XwJD\nSnRzIj0gtXOeRCwwEAxo55KfS6hksiLcC77jy/JCDSduTxP5RGXIgODY1wIVAM4b\nWBLFL5AOHUjHt9L64p+3DvYZAoGBA"..., 1024) = 672
  • -e allows to filter out the calls we only want, the simplest way is to print only one syscall we are interested in for example open()
[root@localhost:~]# strace -s 256 -e trace=open -y -f /usr/sbin/sshd
open("/etc/ld.so.preload", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libsys-assert.so", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libpam.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPC, si_addr=0xb6dd76a8} ---
open("/dev/null", O_RDWR|O_LARGEFILE)   = 3
open("/etc/ssl/openssl.cnf", O_RDONLY|O_LARGEFILE) = 3
open("/etc/ssh/sshd_config", O_RDONLY|O_LARGEFILE) = 3
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 3
open("/etc/gai.conf", O_RDONLY|O_CLOEXEC) = 3
open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libnss_compat.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libnsl.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libnss_nis.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/lib/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ssh/ssh_host_rsa_key", O_RDONLY|O_LARGEFILE) = 3
open("/etc/ssh/ssh_host_dsa_key", O_RDONLY|O_LARGEFILE) = 3
open("/etc/ssh/ssh_host_ecdsa_key", O_RDONLY|O_LARGEFILE) = 3
Process 13668 attached
[pid 13668] open("/dev/null", O_RDWR <unfinished ...>
[pid 13667] +++ exited with 0 +++
<... open resumed> )                    = 3
open("/dev/tty", O_RDWR|O_NOCTTY|O_LARGEFILE) = -1 ENXIO (No such device or address)
open("/proc/self/oom_score_adj", O_RDWR|O_LARGEFILE) = 3
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 4
+++ exited with 255 +++

NMAP

NMAP is a port scanner by definition but thanks to the extensive options it has it can be used to achieve much much more. A basic run of NMAP on a M0 target would produce something like:

[root@localhost:~]# nmap -sS -A 127.0.0.1
Starting Nmap 6.40 ( http://nmap.org ) at 1999-12-31 20:04 PST
Nmap scan report for localhost.localdomain (127.0.0.1)
Host is up (0.000071s latency).
Not shown: 997 closed ports
PORT     STATE SERVICE    VERSION
22/tcp   open  ssh        OpenSSH 6.2 (protocol 2.0)
53/tcp   open  tcpwrapped
6000/tcp open  X11        X.Org (open)
No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).
TCP/IP fingerprint:
OS:SCAN(V=6.40%E=4%D=12/31%OT=22%CT=1%CU=41299%PV=N%DS=0%DC=L%G=Y%TM=386D7C
OS:FA%P=armv7l-tizen-linux-gnueabi)SEQ(SP=102%GCD=1%ISR=109%TI=Z%CI=I%II=I%
OS:TS=8)OPS(O1=MFFD7ST11NW7%O2=MFFD7ST11NW7%O3=MFFD7NNT11NW7%O4=MFFD7ST11NW
OS:7%O5=MFFD7ST11NW7%O6=MFFD7ST11)WIN(W1=AAAA%W2=AAAA%W3=AAAA%W4=AAAA%W5=AA
OS:AA%W6=AAAA)ECN(R=Y%DF=Y%T=40%W=AAAA%O=MFFD7NNSNW7%CC=Y%Q=)T1(R=Y%DF=Y%T=
OS:40%S=O%A=S+%F=AS%RD=0%Q=)T2(R=N)T3(R=N)T4(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%
OS:O=%RD=0%Q=)T5(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)T6(R=Y%DF=Y%T=4
OS:0%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T7(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%
OS:Q=)U1(R=Y%DF=N%T=40%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)IE(R=
OS:Y%DFI=N%T=40%CD=S)
Network Distance: 0 hops
Service Info: OS: Unix
OS and Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 27.58 seconds

NMAP can help you see what ports you have opened on what interfaces, it can also help to see how your service reacts to those types of scans. Nmap implements multiple techniques for scanning:

SCAN TECHNIQUES:
 -sS/sT/sA/sW/sM: TCP SYN/Connect()/ACK/Window/Maimon scans
 -sU: UDP Scan
 -sN/sF/sX: TCP Null, FIN, and Xmas scans
 --scanflags <flags>: Customize TCP scan flags
 -sI <zombie host[:probeport]>: Idle scan
 -sY/sZ: SCTP INIT/COOKIE-ECHO scans
 -sO: IP protocol scan
 -b <FTP relay host>: FTP bounce scan

It might be useful to see how the service reacts to them. Nmap is often used for penetration tests, with custom scripts and techniques it can do a lot of harm to the scanned systems, so testing a service for such attacks before it's deployed can be very helpful for a programmer.

SOCAT

SOCAT is network socket utility with support for UNIX sockets. It can read/write to any socket or stream thus can be very helpful in debugging applications that communicate over sockets. A very big list of examples with description is located here. A very useful example is in a situation when we want to eavesdrop on a socket and see what sort communication is flowing through:

// unix socket handling
// create a listening unix socket
$ rm -f /tmp/mysocket; socat UNIX-LISTEN:/tmp/mysocket -
// from another terminal, connect to this socket
$ socat UNIX:/tmp/mysocket -
// then transfer data bidirectionally

Apart from that wen can "inject" any data to already existing unix sockets on the system thus testing the running services. We can inject large amounts of data quickly without any need to write specialized tests. SOCAT can also act as a test server and respond to requests, this makes client development easy in cases when a server is not yet ready or unavailable.

TCPDUMP

Tcpdump is a network analysis tool, in special cases it can act as a "network sniffer", it can also record network traffic for later analysis. Tcpdump can help you track what's happening "on the wire" between services and applications. An extensive tutorial with a lot of example and all tcpdump options explaining is here.

INOTIFY-TOOLS

Inotify is a subsystem inside the linux kernel that let's you monitor (watch) filesystem changes in your applications. This subsystem is used extensively across the Tizen platform. The inotify-tools package can help you debug the usage of this subsystem. More examples and documentation here

inotifywait This command simply blocks for inotify events, making it appropriate for use in shell scripts. It can watch any set of files and directories, and can recursively watch entire directory trees.

[root@localhost:~]# inotifywait -m -r /opt/var/kdb
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/opt/var/kdb/db/menu_widget/ OPEN regionformat
/opt/var/kdb/db/menu_widget/ ACCESS regionformat
/opt/var/kdb/db/menu_widget/ CLOSE_NOWRITE,CLOSE regionformat
/opt/var/kdb/db/menu_widget/ OPEN regionformat_time1224
/opt/var/kdb/db/menu_widget/ ACCESS regionformat_time1224
/opt/var/kdb/db/menu_widget/ CLOSE_NOWRITE,CLOSE regionformat_time1224
/opt/var/kdb/db/menu_widget/ OPEN regionformat
/opt/var/kdb/db/menu_widget/ ACCESS regionformat
/opt/var/kdb/db/menu_widget/ CLOSE_NOWRITE,CLOSE regionformat

inotifywatch inotifywatch collects filesystem usage statistics and outputs counts of each inotify event.

[root@localhost:~]# inotifywatch -v -e access -e modify -t 60 -r /opt/var/kdb/db/*
Establishing watches...
Setting up watch(es) on /opt/var/kdb/db/account
OK, /opt/var/kdb/db/account is now being watched.
Setting up watch(es) on /opt/var/kdb/db/ail
OK, /opt/var/kdb/db/ail is now being watched.
[...]
OK, /opt/var/kdb/db/usb is now being watched.
Total of 87 watches.
Finished establishing watches, now collecting statistics.
Will listen for events for 60 seconds.
total  access  filename
3      3       /opt/var/kdb/db/menu_widget/


Linux capabilities

Special privileges for non-root processes

If your process needs to use special privileges normally not available for non-root users like TCP ports below 1024, files in root-owned locations. You can use the LINUX Capabilities to get those privileges and still run your process as a non privileged user. A detailed overview of linux capabilities can be found in the man page. For testing purposes a special [#link kernel module] is provided to trace what capabilities each process requests.

In most cases you'll need only one or two capabilities to get the extra privileges you need. To be able to bind to a low port (below 1024) you need the capability CAP_NET_BIND_SERVICE, to create raw sockets (for example when using ping) you'll need CAP_NET_RAW, to bypass the DAC subsystem (gain access to files not available to the running user) you can use CAP_DAC_OVERRIDE.

Remember to keep the capabilities list as short as possible, less capabilities you assign to your process makes your process more secure. Before you apply capabilities to your daemon you should read topic False Boundaries and Arbitrary Code Execution

Testing capabilities

Before deploying your process with systemd you can test how it will behave with the new capabilities set. To set a capability for a binary in linux use setcap, to list all capabilities assigned to a binary in linux [getcap], if you want to see what capabilities are assigned to an already running process and you know it's PID have a look at the /proc/$PID/status file and look for the CapInh, CapPrm, CapEff, CapBnd fields in it

CapInh                      bitmap of inheritable capabilities
CapPrm                      bitmap of permitted capabilities
CapEff                      bitmap of effective capabilities
CapBnd                      bitmap of capabilities bounding set

To decode/encode capability bitmasks and execute misc operation use the capability shell wrapper capsh

For example

[root@AMDC2543:/home/r.kubiak/devel]# setcap cap_net_bind_service+ep echo-tcpip
[root@AMDC2543:/home/r.kubiak/devel]# getcap echo-tcpip
echo-tcpip = cap_net_bind_service+ep

Let's test a simple program that binds to a port that is passed as the first argument:

developer@AMDC2543:~$ /sbin/getcap ./echosrv # No capabilites set for this binary
developer@AMDC2543:~$ ./echosrv 99
[OK] Start
[OK] socket created flags AF_INET and SOCK_STREAM, descriptor [3]
[ERROR] bind(): Permission denied

This example fails, we are a user and we can't bind to port 99

Now as root let's add the special capability

[root@AMDC2543:/home/developer]# setcap cap_net_bind_service+ep echosrv

And run the program again

developer@AMDC2543:~$ /sbin/getcap echosrv 
echosrv = cap_net_bind_service+ep
developer@AMDC2543:~$ ./echosrv 99
[OK] Start
[OK] socket created flags AF_INET and SOCK_STREAM, descriptor [3]
[OK] bound to port [99]
[OK] Listen succeeded on socket
[OK] Finish

Success!!

Notice we didn't use strace, this is because the capabilities are added to the echosrv binary and not the strace binary, so they won't work. For details about this consult the linux capability documentation.

Linux capabilities analysis kernel module

This module allows you to find what capabilities a process needs. This is very helpful when we need to drop root privileges from some process and run it with a different UID/GID. The simplest way to use that module would be to run the target process with the new UID/GID and register all CAP requests from the module. Adding those capabilities to the process one by one using setcap you should be able to run the process as a non-root user.

Remember that you should find the smallest set of capabilities to run the process, if you add too many capabilities to a process there won't be much difference between running it as root. Try to minimize the capabilities needed. Typical capabilities that involve UNIX permissions and file access should be resolved on the configuration/process levels, directories/files should have proper permissions for the process to access them without root privileges and without additional CAPs. Once that's done the set of capabilities should be much smaller. Capabilities like CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FOWNER should be avoided. It should be possible to replace those capabilities with process level configuration (for example writing log files to a directory that's owned by the user that the process runs under, writing the pid file to a directory that the process runs under etc.)

A list of all capabilities currently available with descriptions is here

DAC policy (When/How to add new user and groups?)

The system user and group policy and system users assignment is described in the following article on Tizen wiki . When you need a new system daemon running with non-root UID follow the appropriate Tizen procedure.

Good practices for developers

  • Follow specified guidelines for writing services in tizen
  • Build your program with additional security flags like, also always check if those flags do not impact your program:
CFLAGS="-fPIE -fstack-protector-all -D_FORTIFY_SOURCE=2" 
LDFLAGS="-Wl,-z,now -Wl,-z,relro"

Low level service common code

All new services should implement certain low level tasks in the same way. When specified each part can be extended by a developer (for example adding new command line options).

Kernel auditing

External audit tutorial

Configuration

The default kernel configuration for Tizen enables auditing but only in a limited manner, to audit syscalls you need to re-configure the kernel and rebuild it. Below is a configuration for the 3.10 kernel that enables full auditing support, the config has been used with commit: 8bb430b75ff8ada5e3b666169f391badd5d84ffd, this config also disables SMACK.

Kernel_Config

To enable auditing on linux, the kernel needs to be passed a command line parameter "audit=1" for M0 targets (and any other hardware that uses a DTB) this can be done in the .dts file. For example:

[r.kubiak@AMDC2543:~/tizen/sources/linux-3.10]$ pwd
/home/r.kubiak/tizen/sources/linux-3.10
[r.kubiak@AMDC2543:~/tizen/sources/linux-3.10]$ cat arch/arm/boot/dts/exynos4412-trats2.dts | grep bootargs
		bootargs = "console=ttySAC2,115200N8 root=/dev/mmcblk0p5 rootwait earlyprintk panic=5 audit=1 no_console_suspend=1";

This entry will cause the audit=1 parameter to be embedded in the uImage file created, no need to access the u-boot bootloader.

The runtime configuration for audit is located in /etc/audit. auditd.conf defines how the daemon works, how fast it writes the collected logs to disk among other things. The default configuration just writes all logs to /var/log/audit/audit.log. To defines the auditing rules you need to edit the file /etc/audit/audit.rules (or create a new file in /etc/audit/rules.d directory). The initial contents of this file contains no rules:

[root@localhost:/etc/audit]# cat audit.rules 
# This file contains the auditctl rules that are loaded
# whenever the audit daemon is started via the initscripts.
# The rules are simply the parameters that would be passed
# to auditctl.
# First rule - delete all
-D
# Increase the buffers to survive stress events.
# Make this bigger for busy systems
-b 320
# Feel free to add below this line. See auditctl man page

An example set of rules

-w /etc/shadow -k pam
-w /etc/passwd -k pam
-w /etc/ssh/sshd_config -k ssh
-a exit,always -F path=/usr/sbin/connmand -k networking
-a exit,always -F path=/usr/sbin/wpa_supplicant -k networking

The first 3 rules set up "watches" with their respective labels "pam" and "ssh". A watch is a audit specific selector for filesystem objects. A watch will inform you about any change to the object, or about a specific set of changes if you wish to filter those. The next two lines set up rules for monitoring two processes "connmand" and "wpa_supplicant" a label for those rules is set as "networking".

Running and usage

The audit subsystem in Linux provides an interface to observe the running OS for security events. It is designed to be a analysis tool not a debugging tool. Running audit in a real production system without proper configuration will cause significant performance degradation, also the amount of logs the audit subsystem generates might fill the disk space very quickly.

The provided RPM starts the auditd daemon via systemd. Auditd writes it's log file to /var/log/audit directory, those files are rotated automatically. Log files are plain text and can be analyzed with any CLI tools (grep, awk, sed), however the audit subsystem provides specialized tools for searching and formatting logs.

For searching the logged data use ausearch, a short extract from the man page is below:

ausearch is a tool that can query the audit daemon logs based for events based on different search criteria. The ausearch utility can also take input from stdin as long as the input is the raw log data. Each commandline option given forms an "and" statement. For example, searching with -m and -ui means return events that have both the requested type and match the user id given. An exception is the -n option; multiple nodes are allowed in a search which will return any matching node.
It should also be noted that each syscall excursion from user space into the kernel and back into user space has one event ID that is unique. Any auditable event that is triggered during this trip share this ID so that they may be correlated.
Different parts of the kernel may add supplemental records. For example, an audit event on the syscall "open" will also cause the kernel to emit a PATH record with the file name. The ausearch utility will present all records that make up one event together. This could mean that even though you search for a specific kind of record, the resulting events may contain SYSCALL records.
Also be aware that not all record types have the requested information. For example, a PATH record does not have a hostname or a loginuid.

The full manual with examples and field selectors can be found here.

Another tool for reading the logged data from audit is aureport:

aureport is a tool that produces summary reports of the audit system logs. The aureport utility can also take input from stdin as long as the input is the raw log data. The reports have a column label at the top to help with interpretation of the various fields. Except for the main summary report, all reports have the audit event number. You can subsequently lookup the full event with ausearch -a event number. You may need to specify start & stop times if you get multiple hits. The reports produced by aureport can be used as building blocks for more complicated analysis.

Below we will analyze the lines we entered in audit.rules files

First to search for a label "networking" that we defined for two rules in our file we should execute::

[root@localhost:/etc/audit]# ausearch -k networking
type=PATH msg=audit(946691470.260:16872): item=1 name=(null) inode=1756 dev=b3:05 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=PATH msg=audit(946691470.260:16872): item=0 name="/usr/sbin/connmand" inode=23268 dev=b3:05 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=CWD msg=audit(946691470.260:16872):  cwd="/"
type=EXECVE msg=audit(946691470.260:16872): argc=2 a0="/usr/sbin/connmand" a1="-n"
type=SYSCALL msg=audit(946691470.260:16872): arch=40000028 syscall=11 per=800000 success=yes exit=0 a0=118910 a1=136c88 a2=136e98 a3=cd070 items=2 ppid=1 pid=5737 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0  ses=4294967295 tty=(none) comm="connmand" exe="/usr/sbin/connmand" key="networking"

This information will tell you only when those binaries where executed. For a list of "pam" events execute:

[root@localhost:/etc/audit]# ausearch -k pam
----
time->Fri Dec 31 17:51:15 1999
type=PATH msg=audit(946691475.710:16876): item=0 name="/etc/passwd" inode=8918 dev=b3:05 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=CWD msg=audit(946691475.710:16876):  cwd="/"
type=SYSCALL msg=audit(946691475.710:16876): arch=40000028 syscall=5 per=800000 success=yes exit=6 a0=b6cd8fdc a1=80000 a2=1b6 a3=1b6 items=1 ppid=2518 pid=5756 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 ses=4294967295 tty=(none) comm="sh" exe="/usr/bin/bash" key="pam"
----
time->Fri Dec 31 17:51:17 1999
type=PATH msg=audit(946691477.890:16880): item=0 name="/etc/passwd" inode=8918 dev=b3:05 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=CWD msg=audit(946691477.890:16880):  cwd="/"
type=SYSCALL msg=audit(946691477.890:16880): arch=40000028 syscall=5 per=800000 success=yes exit=6 a0=b6c34fdc a1=80000 a2=1b6 a3=1b6 items=1 ppid=5762 pid=5765 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 ses=4294967295 tty=(none) comm="sh" exe="/usr/bin/bash" key="pam"

Of course you can use any select, field defined in the manual for ausearch and you can log any syscall or a set of syscalls you wish or need for your purposes. You can later search the results using the exit status or any other logged field. The system is very flexible.