Early Camera

From Tizen Wiki
Jump to: navigation, search


For IVI devices, it’s a fundamental requirement that a user should immediately see the rear video in the IVI’s screen after putting the reverse gear, no matter what the IVI box’s current state is. It is also one of the most popular ticked box when buyers are scrolling down the options list. Usually the time consuming should be less than 2 seconds, which means in the worst case our silicon should boot into OS and bring up the camera within very short time. This document introduces how to minimize the cost of every boot stage to achieve the 2S target, including boot loader, kernel and camera drivers’ optimization.

  • Target of feature implementation

Firmware + Tizen must turn on the rear view camera and audio within 2 seconds

  • Platform of feature testing

VTC1010 + Tizen IVI image

Feature design and implementation

According to the latest performance testing of Tizen IVI start up(which is optimized), we find that bootloader+kernel+userspace = ~ 3 seconds. Considering add bios time in, it requires about 4 seconds to complete IVI system starting up. Hence, it is impossible to achieve the rear camera target(2 seconds) if implementing in user space. So, it is necessary to ramp up the rear cameras in kernel side.



As the diagram shows, the Early Camera feature consists of the following components:

  1. V2g_bridge is the newly created kernel mode component. It plays the role of camera application in the kernel mode.
  2. Direct rendering manager (DRM) framework: DRM and i915.
  3. Video for Linux* framework: V4L2.
  4. Camera driver: UVC, PCIe* cameras, or MIPI-CSI cameras.
  5. GPIO: Reverse GPIO signal will trigger/shutdown early camera. For current HW limitation, this component is in TODO list.

They are all kernel drivers. New APIs are added in these modules to support early camera in kernel mode. Here is a general block diagram:

Working flow chart


In this diagram, v2g_bridge module is created to build connection with v4l2 module and drm/i915 module. And the major idea is to mmap the v4l2 buffers to SpriteC graphics plane, hence early camera will not have any conflicts with weston start up.

DRM module

  • To avoid DRM_MASTER competition issue, we will export drm APIs directly to v2g_bridge module rather than calling DRM_IOCTL_DEF.
  • Sprite C plane will be used to show camera video, and the framebuffer is used by weston.
  • Two new APIs drm_open_kernel and drm_release_kernel are provided for early camera feature to open/release drm file. When the early camera working, weston is also using drm APIs to master the graphics driver. Hence, the inode of drm file should be shared by early camera and weston.

V4L2 module

  • The camera devices and related drivers should be initialized ASAP. Therefore, USB_VIDEO_CLASS and VIDEO_V4L2 are set as builtin modules.
  • To minimize changes to the V4L2 module, the following V4L2 APIs are exported for v2g_bridge module:
  1. v4l2_open_kernel
  2. v4l2_release_kernel
  • To detect camera devices, we need to implement camera device enumerations.
  1. v4l2_enumerate_camera. This function will wait the signal of video device register, and check v4l2 camera list. 5 secs time out will be triggered if no available camera devices connected.

v2g_bridge(video to graphics) implementation

v2g_bridge has the following functions:

  • v2g_control is a thread to enumerate V4L2 cameras, open the drm file, control state transferring, and create a thread named v2g_streamd;
  • v2g_streamd is a thread to stream V4L2 cameras and mmap the video to spritec plane.
  • Initializes and releases the related resources.

For the state transferring between v2g_control and v2g_streamd, please check the following diagram.


Feature performance testing

How to enable/disable early camera

I have defined module parameters for enabling/disabling early camera. For testing and demo, a timer is created to show early camera function in the module. The duration of timer is 15 seconds.

  • Add v2g_bridge.v2g_enable=1 in kernel command line to enable the feature.
  • Add v2g_bridge.v2g_debug=1 in kernel command line to enable debug messages.

If using a non-VGA monitor, please disable the unavailable display ports:

  • VTC1010, add "video=HDMI-A-1:d video=VGA-1:d" in kernel command line;
  • MinnowMax, add "video=VGA-1:d video=DP-1:d" in kernel command line;

How to measure the accurate time consuming

Because the current bios of IVI platform does not catch the boot requirement(<1s), we measure the total time usage of bootloader+kernel+camera_video_show. In VTC1010, please follow the steps to measure:

  1. power on vtc1010, wait ~20 sec for bios log appearing. Start to capture video(with your mobile phone).
  2. keep videoing until the camera video showing on the monitor.
  3. use a video analyzer to check the accurate time stamp of the 1st frame(bios log disappeared) and last frame(early camera showing). The delta of these two time stamps are the accurate time usage value of bootloader+kernel+camera_video_show.

Performance of camera

According to our previous experience on Bayley Bay board, if applied eDP panel and PCIe cameras, the early camera speed can match the requirement(bios+kernel+rear camera < 2s). Currently, we only have normal desktop monitors and USB cameras, which will slower than eDP+PCIe. But it is enough for feature implementation and demo

UVC camera

  • 2 seconds(bootloader+kernel+rear camera) without kernel optimization;
  • 1.7 seconds(bootloader+kernel+rear camera) after applied some kernel optimization patches(which are under review);

PCIe camera

Most of PCIe camera drivers, such as cx23885, only support videobuffer, which does not provide any interfaces of DMA. Hence, in current kernel versions(3.14), to integrate PCIe cameras into early camera framework, it requires porting the camera drivers on videobuffer2. But fortunately, linuxtv.org has completed the porting work, and it will be integrated in kernel main branch.

MIPI-CSI camera