The Optimization of Automotive Fast Boot
- 1 Scope
- 2 Contributer
- 3 Backgrounds
- 4 Kernel Optimization
- 4.1 Initial performance and target
- 4.2 Optimization tracing
- 4.3 Optimization Result
- 5 User Space Optimization
- 6 Performance of Fast Boot
James Jia; Mikko Ylinen; Neo Fang; Xin Cao;
Target & Purpose of Fast Boot
For most Automotive systems, it requires 5 seconds to complete cold booting, which is very critical for optimizing the procedure of system booting. After analyzing the procedure of system cold boot, we can find it consists the following steps:
- HW platform initialization, ~100ms;
- EFI/BIOS, ~900ms;
- Bootloader, ~50ms;
- Kernel, ~1s;
- User space and desktop showing up, ~3s.
Parts of optimization
This wiki page discusses the methods and results of kernel and user space optimizations. In the other words, we need to shorten the booting time kernel + Userspace < 4s.
Hardware platform influences
During our optimization works, we found different HW configurations will bring different performance data. For example, VTC1010 will be slower than MinnowMax ; and the types of SSD bring ~100ms differences of performance.
Initial performance and target
Code base of kernel: 3.14.19; Original performance: ~1.3 secs; Target: <1 sec.
All the optimization ideas are from the kernel booting log. From the log, we can figure out which devices or frameworks consume too much time. At the same time, boot-chart can also help you to check the IO/CPU consumption in each time slot.
Remove unnecessary kernel modules and features
Every document about kernel optimization will suggest to remove the unnecessary features in kernel configure. However, after remove some obviously unused features, we can not remove any more. Because for a system in developement, it's hard to evaluate and predict the side-effects of removing a feature.
Fix the errors in kernel logs
After checking the booting logs, we will find several errors. Errors hide optimization chances. Because in kernel booting up procedure, if initialize some devices failed, kernel will choose to wait for a while. Hence, fixing the errors will avoid the waiting. For example, please check the bug of TC-2195
Parallel SCSI/ATA initialization
SCSI/ATA devices(SSD) initialization and partition mounting requires long time I/O waiting. Normally, kernel will initialize SSD in the last step. At that time, the CPU is in low usage, and time is wasted for IO waiting. Promote the priority of SCSI/ATA devices will improve the performance obviously(~200ms improvement). For details, please check the bug of TC-2215
Parallel DRM/I915 driver
The graphics drivers require long time to initialize. As the same idea as SCSI/ATA, we can postpone the initialization of i915. Wei Xing provides the optimization patch for Tizen IVI.
fbcon can be disabled and improve the performance greatly. ~200ms saved. About howto disable fbcon, please check the document under kernel source code Documentation/fb/fbcon.txt
Disable serial port console
Serial port number is 5 by default. Eliminate the serial port count or disable it completely will save 100ms booting time. But be attention, the modification will impact the serial port usage, which might bring side-effects for our system development.
The following methods will not bring obvious improvement or not fully tested.
- If the error log "EFI : ioremap of 0xE00000 failed" exists, add "vmalloc=256m" in the kernel command line, which will improve 10ms.
- Shorten the waiting time of ATA devices. By default, the waiting time is 100ms, eliminate it to 5ms to save time. Normally, 5ms is enough to wait a SSD to be ready, but this method is not fully tested and might bring device detection failure issue.
- Asyn device detection. This method is still in designing.
With the optimization methods(excluding the MISC methods), we tested the performance under MinnowMax and VTC1010.
- VTC1010: 713ms.
- MinnowMax: 690ms.
The performances of both platforms match the optimization target(<1s).
User Space Optimization
Initial performance and target
The target of fastboot is to make Weston desktop display on monitor within 3 secs. So, it needs to make the user space processes start up as early as possible.
Tizen release version: tizen-3.0.m14.3-ivi_20141028.3;
Original performance: 4.5 secs;
Target: <3 secs.
From the systemd log and the analysis of its unit files, we can trace down the unit files load and run sequence, use Bootchartd to figure out the services(according to CPU utilization) which compete for CPU with Weston when it starts up.
Remove useless systemd and Tizen specific services
With the analysis of systemd unit files, we find that the specific service files, such as systemd-readahead-collect.service, are not available in current Tizen. And they still waste time because systemd tries to load, queue and start them. So disabling these services will save time when Tizen boots up(~900ms improvement).
The unit files, such as dbus.socket and dbus.service, are prerequisite for systemd and Weston. Change their boot sequences in systemd to start as early as possible during Tizen boot-up. For more details about systemd and the mechanism of service boot-up sequence, please refer: systemd
Postpone the other service files
Weston and the related services is not the whole story. The CPU competition between Weston process and the other services can also slow down Weston desktop display on monitor. So we should guarantee the priority of Weston process, which can occupy CPU as mush as possible when it's starting. Therefore, delay the other services until Weston complete starting up(~500ms improvement). After adjusting the boot-up sequence, we get the following boot chart:
- VTC1010: 3.067s, close to the fast boot target.
- MinnowMax: 3.2s, 200ms time gap is caused of low CPU performance.
Performance of Fast Boot
The performance of Fast Boot is based on two platforms: VTC1010 and MinnowMax.
BIOS Version: MV11A109 (Release Date: 07/30/2014)
CPU: Intel(R) Atom(TM) CPU E3827 @ 1.74GHz
Monitor: DELL 13" VGA Monitor
BIOS Version: MNW2MAX1.I32.0073.R02.1409261435
CPU: Intel(R) Atom(TM) CPU E3825 @ 1.33GHz
Monitor: Giantec 11.6" FHD LCD Monitor
Current performance data
With the optimizations of kernel and user space, we tested 10 times of the boot up data(bootloader+kernel+weston desktop showing), and get the following average data of Fast Boot on VTC1010 and MinnowMax.
- VTC1010: 2.8s
- MinnowMax: 3.0s
Considering the target, ~4s, the optimizations achieves ~1s improvement.