Porting Guide/Graphics and UI

From Tizen Wiki
Jump to: navigation, search

The following figure shows an overview of the Tizen UI and graphics architecture.

Graphicsarchitecture.png


Tizen provides high performance 3D graphics as a component of UI and graphics. There are hardware-accelerated OpenGL ES (Open Graphics Library Embedded System) and EGL (Embedded-System Graphics Library) for 3D applications, such as 3D games.

OpenGL ES is an application programming interface (API) for advanced 3D graphics, targeted to handheld and embedded devices. To overcome device constraints, such as limited processing capabilities and memory availability, it provides a subset of the functionality in OpenGL. Embedded system-specific features were added to enhance rendering efficiency, such as precision qualifiers to the shading language from OpenGL ES 2.0.

OpenGL

The following figure shows an overview of the interface for Tizen high performance 3D graphics library with OpenGL.

Opengl.png

EGL is a "glue" layer between OpenGL ES and the underlying native platform window system - X Window system. EGL communicates with X Window to get information from the application window. And it creates a drawing surface and manages rendering context and resources.

3D Graphics Components

OpenGL ES is a software interface to graphics hardware. It is designed as a hardware-independent interface to be used for many different hardware platforms. OpenGL ES is a subset of the OpenGL designed for embedded systems. It accepts primitives, such as points, lines, and triangles, and converts them into pixels using a graphics pipeline, known as the OpenGL state machine.

EGL is a (mostly) platform-independent API and an interface between rendering APIs, such as OpenGL ES or OpenVG and an underlying native platform window system. It communicates with X Window system, and creates drawing surfaces and manages rendering contexts. The EGL interface has the following components:

  • EGLDisplay (X Display): Encapsulates all of the system dependencies for interfacing with the native windowing system.
  • EGLSurface: Encapsulates the rendering destinations (window surface, pixmap surface), which are tied to X Window and X Pixmap.
  • EGLContext: Encapsulates the OpenGL ES rendering context, which holds the state of GL server and client.

The following figure shows a brief overview of EGL interface.

Egl interface.png

Porting

3D graphics vendors must provide these porting requirements for OpenGL ES/EGL:

OpenGL ES
  • The required version of OpenGL ES: 1.1 and 2.0
  • The driver must support the following extensions to OpenGL ES 1.1:
GL_OES_framebuffer_object
GL_OES_blend_subtract
GL_OES_blend_func_separate
GL_OES_matrix_palette
GL_OES_draw_texture
GL_OES_texture_cube_map
GL_OES_query_matrix
GL_OES_point_size_array
  • The driver must support the following extensions to OpenGL ES 2.0:
GL_OES_EGL_image
GL_EXT_texture_format_BGRA8888
GL_OES_get_program_binary
GL_OES_texture_npot
GL_OES_fragment_precision_high
GL_OES_rgb8_rgba8
GL_OES_depth24
GL_OES_vertex_half_float
GL_OES_texture_float
GL_OES_compressed_ETC1_RGB8_texture
GL_OES_packed_depth_stencil
GL_OES_standard_derivatives
GL_OES_element_index_uint
GL_OES_mapbuffer
GL_EXT_multi_draw_arrays
GL_OES_vertex_array_object
GL_IMG_texture_compression_pvrtc
GL_OES_read_format
GL_EXT_multisampled_render_to_texture
  • You need some tools for generating a binary shader on Linux.
EGL
  • The minimum version required for EGL is 1.4.
  • EGL must support DRI2, as the common 3D applications use DRI2 based EGL and the 3D composite window manager usually needs a DRI2 based EGL.
FBDEV based EGL (optional)
  • The FBDEV must consist of a triple buffer.
  • It must be performed with page flipping instead of copying the buffer.
  • To avoid a tearing problem, perform with LCD vsync signal. Usually, the compositor enables vsync using the eglSwapInterval() API.
DRI2 based EGL (mandatory)
  • DRI2 based EGL must be performed with the X11 DRI2 protocol.
  • It must support texture from pixmap (using the EGL_KHR_image_pixmap and GL_OES_EGL_image).
GPU synchronization API
  • To avoid flickering, use an internal EGL API, which is used to wait for completion of the buffer copy command.
Limitation of the EGL surface
  • The FBDEV must consist of a triple buffer.
  • The driver must support the following extensions to EGL 1.4:
EGL_KHR_image
EGL_KHR_image_base
EGL_KHR_gl_texture_2D_image
EGL_KHR_gl_texture_cubemap_image
EGL_KHR_gl_renderbuffer_image
EGL_KHR_image_pixmap
EGL_KHR_lock_surface
EGL_KHR_fence_sync
EGL_KHR_reusable_sync
EGL_IMG_context_priority
Note
A method to allow the CPU to read and write the texture memory area of the GPU is needed.
Header and library name

The header files and libraries must be provided according to the Khronos API Implementers Guide. The following table gives a brief overview about the EGL library path and the respective header files.

Library path Files
/usr/include/EGL egl.h

eglext.h

eglplatform.h

/usr/include/GLES gl.h

glext.h

glplatform.h

/usr/include/GLES2 gl2.h

gl2ext.h

gl2platform.h

/usr/include/KHR khrplatform.h
/usr/lib libEGL.so

libGLESv1_CM.so

libGLESv2.so

Note
You may need to implement some memory management techniques for sharing memory between your CPU and the graphics processing unit.

X Server

The X server is an X Window system display server that provides a basis for the graphical user interface (GUI) and rich input capabilities for your system. It creates a hardware abstraction layer where software is written to use a generalized set of commands, allowing for device independence and reuse of programs on any system that implements X.

Input Driver

The following figure shows the relations between the X Window system and input drivers.

Input.png

X Window system

The X Window system provides the interface for the manipulation of resources, such as windows, pixmap, and gc. X clients send the requests to the X server for drawing something or for manipulating windows. The X server accepts the clients’ requests and sends back replies to the requests. In addition, the X server sends X events when input events from input devices are pending or when the X server encounters something that the X client may be interested in, such as configure or exposure events.

X server

The X server reads events from the internal event queue and makes and sends the X events to the X clients.

X input driver

X input driver reads the input event stream from each input device node, makes X internal events, and put the events into the X server's internal event queue. For doing this, the X input driver mainly uses the interfaces implemented on the xf86 DDX layer inside X server.

evdev driver

Implements for reading the input event stream originated from kernel evdev subsystem and for creating X internal events. Supports devices, such as keyboard, mouse, touchpad, and touch screen.

evdev multi-touch driver

Implements for reading the MT protocol event stream originated from kernel evdev subsystem, for interpreting them and for creating X internal events. Supports touch screen devices.

Porting OAL Interface

The following section provides a brief understanding about X server and X input driver.

Evdev module

The evdev contains the following drivers:

  • Driver for absolute pointing device like touch-screen.
  • Driver for relative pointing device like mouse.
  • Driver for key (board) device, including HW key and BT/USB keyboard.

Each driver sets up each input device, and depends on the X server's configuration.

  • After the device setup is done, the device reads input events from the device node, creates internal events, and puts them in the input event queue inside X server.
  • To use the X input driver, the module must be loaded and must be registered as an input driver by the X server.
  • Module, driver, and device must provide the following information:
    • Module: module information.
    • Driver: driver information.
    • Device: read() for reading input events, proc() for controlling the device status.
For information about the data structures and functions for each module and device, see http://www.x.org/wiki/Development/Documentation/XorgInputHOWTO.
The basic module information is stored inside XF86ModuleData (xf86Module.h) and is provided to the X server.

Configuration

ServerFlags
  • AllowEmptyInput
true means that X server is started without any keyboard or mouse driver.
  • AutoAddDevices
false means that hot plugged input devices are not supported.
  • AutoEnableDevices
true means that the DevicePresenceNotify event is sent to all X clients by default.
InputClass
  • Identifier
Sets the unique input device name.
  • Driver
Sets the driver for device.
  • MatchDevicePath
Sets the device node path to be handled by the device. A wildcard can be used.
  • Option
Sets known options and custom options to be used inside driver for setting up devices or for enabling or disabling certain feature. Integer, Boolean, and string type can be used.
  • Checks that the found device matches one of device types (MatchIsTouchScreen, MatchIsTablet, MatchIsPointer, MatchIsKeyboard) and sets the type for the device if matched.
Tizen Extension for Evdev Multi-touch Driver
  • Provides 1 legacy single touch protocol
  • Provides 2 kinds of MT protocol:
    • MT protocol A (which includes Tracking ID)
    • MT protocol B
  • To implement support for a new touch protocol:
    • Implement a new touch protocol inside the driver.
    • Clone the evdev multi-touch driver and implement a new touch protocol inside the driver.
    • Convert from a new protocol to an existing protocol, which is supported by the evdev multi-touch driver.

Video Driver

The following figure illustrates the X server and the X video driver.

Video.png

X Window System

X Window system provides the interface for the manipulation of resources, such as windows, pixmap, and gc. The X clients send the requests to X server for drawing something or for manipulating windows. The X server accepts the clients’ requests and sends back replies to the requests. In addition, the X server sends X events to the client when input events from devices are pending or when X server encounters events that the X client may be interested, such as configure or exposure events.

X server

X server gets the requests from X clients and delivers them to the xf86 porting layer. Input events are processed by the X server and the X server sends them to the X clients.

X video driver

The X video driver is the driver library, which contains the implementation to support the xf86 DDX layer to display something on a screen.

  • Driver Module:
Implementation of a driver module for xf86 DDX.
  • Mode Setting:
Implements the devices to display images on it.
  • EXA:
Implements Graphic acceleration Architecture. Handles memory allocation and exa draw primitives.
  • DRI2:
Implements the DRI2 (Direct Rendering Infrastructure ver.2).
  • XV:
XFree86 offers the X Video Extension (XV), which allows clients to treat video as any other primitive and Put video into drawable mode. By default, the extension reports no video adapters as being available because the DDX layer has not been initialized.
Mode Setting

Kernel module for Display mode setting.

Functionality
  • Crtc control and management
  • Connector control and management
  • Encoder control and management
  • Framebuffer control and management
MemMgr

Kernel module for graphic memory management.

Functionality
  • Control by mode setting
  • Control by GPU
  • Shared memory among processes
Device Driver

Video controller hardware to control the kernel side of the module.

Porting OAL Interface

The X video driver must implement the following driver module and extensions, using the interface provided by the xf86 DDX and X extensions in the X server.

Driver Module
  • XF86ModuleData
    • A symbol data which the X server finds when the X server load drivers.
    • The name of data is defined in the modnameModuleData format, where <modname> is the name of the X video driver.

The following example defines the XF86ModuleData of the sec_drv.so driver: 

_X_EXPORT XF86ModuleData secModuleData = {&SECVersRec, SECSetup, NULL};

For xf86 header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86Module.h.

To implement a driver module, see http://www.x.org/releases/X11R7.6/doc/xorg-server/DESIGN.txt.

ScrnInfoPtr
  • The data structure must be initialized during the initialization of X video driver.
  • The probe functions are called when initializing the X server. The function pointers are mapped in the probe function of the X video driver.
  • The variables of the ScrnInfoPtr structure are set up by the function pointers of ScrnInfoPtr.
  • The ScrnInfoPtr structure has the following function pointers: 
xf86ProbeProc *Probe
xf86PreInitProc *PreInit
xf86ScreenInitProc *ScreenInit
xf86SwitchModeProc *SwitchMode
xf86AdjustFrameProc *AdjustFrame
xf86EnterVTProc *EnterVT
xf86LeaveVTProc *LeaveVT
xf86ValidModeProc *ValidMode

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86str.h.

Mode Setting
  • xf86CrtcConfigPtr
    • The data structure contains the information of outputs, crtcs, and the configuration of outputs and crtcs.
    • The function pointers, xf86CrtcConfigFuncsRec *funcs, have to be set in the X video driver.
    • The function pointers of the xf86CrtcConfigFuncsRec are:
Bool (*resize)(ScrnInfoPtr scrn, int width, int height);

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.

  • xf86CrtcPtr
    • The data structure contains the information of a crtc.
    • The X video driver creates the number of instances of crtcs as the number of hardware crtcs.
    • The function pointers, xf86CrtcFuncsRec *funcs, have to be set in the X video driver.
    • The function pointers of the xf86CrtcFuncsRec are:
Bool (*set_mode_major)(xf86CrtcPtr crtc, DisplayModePtr mode, Rotation rotation, int x, int y);
void (*set_cursor_colors) (xf86CrtcPtr crtc, int bg, int fg);
void (*set_cursor_position) (xf86CrtcPtr crtc, int x, int y);
void (*show_cursor) (xf86CrtcPtr crtc);
void (*hide_cursor) (xf86CrtcPtr crtc);
void (*load_cursor_argb) (xf86CrtcPtr crtc, CARD32 *image);
void (*destroy) (xf86CrtcPtr crtc);

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.

  • xf86OutputPtr
    • The data structure contains the output information.
    • The X video driver creates the number of instances of outputs as the number of hardware connectors.
    • The function pointers, xf86OutputFuncsRec *funcs, have to be set in the X video driver.
    • The function pointers of the xf86OutputFuncsRec are:
void (*create_resources)(xf86OutputPtr output);
Bool (*set_property)(xf86OutputPtr output, Atom property, RRPropertyValuePtr value);
Bool (*get_property)(xf86OutputPtr output, Atom property);
xf86OutputStatus (*detect)(xf86OutputPtr output);
int (*mode_valid)(xf86OutputPtr output, DisplayModePtr pMode);
DisplayModePtr (*get_modes)(xf86OutputPtr output);
void (*destroy) (xf86OutputPtr output);

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.

EXA
  • ExaDriverPtr
    • The data structure contains the information on the X graphic acceleration.
    • It contains the functionality of memory allocation of the pixmap private and EXA drawing primitives.
    • It contains the function pointers and the variables to be implemented by the X video driver.
    • The function pointers and variables of the ExaDriverPtr are:
int exa_major, exa_minor;
CARD8 *memoryBase;
unsigned long offScreenBase;
unsigned long memorySize;
int pixmapOffsetAlign;
int pixmapPitchAlign;
int flags;
int maxX;
int maxY;
void (*WaitMarker) (ScreenPtr pScreen, int marker);
Bool (*PrepareAccess)(PixmapPtr pPix, int index);
void (*FinishAccess)(PixmapPtr pPix, int index);
void *(*CreatePixmap)(ScreenPtr pScreen, int size, int align);
void (*DestroyPixmap)(ScreenPtr pScreen, void *driverPriv);
Bool (*ModifyPixmapHeader)(PixmapPtr pPixmap, int width, int height, int depth, int bitsPerPixel, int devKind, pointer pPixData);
Bool (*PixmapIsOffscreen)(PixmapPtr pPix);
Bool (*PrepareSolid) (PixmapPtr pPixmap, int alu, Pixel planemask, Pixel fg);
void (*Solid) (PixmapPtr pPixmap, int x1, int y1, int x2, int y2);
void (*DoneSolid) (PixmapPtr pPixmap);
Bool (*PrepareCopy) (PixmapPtr pSrcPixmap, PixmapPtr pDstPixmap, int dx, int dy, int alu, Pixel planemask);
void (*Copy) (PixmapPtr pDstPixmap, int srcX, int srcY, int dstX, int dstY, int width, int height)
void (*DoneCopy) (PixmapPtr pDstPixmap);
Bool (*CheckComposite) (int op, PicturePtr pSrcPicture, Pictureptr pMaskPicture, PicturePtr pDstPicture);
Bool (*PrepareComposite) (int op, , PicturePtr pSrcPicture, Pictureptr pMaskPicture, PicturePtr pDstPicture, PixmapPtr pSrc, PixmapPtr, pMask, PixmapPtr pDst);
void (*Composite) (PixmapPtr pDst, int srcX, int srcY, int maskX, int maskY, int dstX, int dstY, int width, int height);
void (*DoneComposite) (PixmapPtr pDst);
Bool (*UploadToScreen) (PixmapPtr pDst, int x, int y, int w, int h, char *src, int src_pitch);
Bool (*DownloadFromScreen)(PixmapPtr pSrc, int x, int y, int w, int h, char *dst, int dst_pitch);

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=exa/exa.h.

DRI2
  • DRI2InfoPtr
    • The data structure for DRI2 (Direct Rendering Infrastructure ver.2)
    • The structure manages and controls the DRI2 buffers.
    • It contains the function pointers and the variables to be implemented by the X video driver.
    • The function pointers and variables of the DRI2InfoPtr are:
const char *driverName;
const char *deviceName;
int fd;
DRI2CreateBufferProcPtr CreateBuffer;
DRI2DestroyBufferProcPtr DestroyBuffer;
DRI2CopyRegionProcPtr CopyRegion;
DRI2ScheduleSwapProcPtr ScheduleSwap;
DRI2GetMSCProcPtr GetMSC;
DRI2ScheduleWaitMSCProcPtr ScheduleWaitMSC;
DRI2AuthMagicProcPtr AuthMagic;
DRI2ReuseBufferNotifyProcPtr ReuseBufferNotify;

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg- server.git;a=blob;f=hw/xfree86/dri2/dri2.h.

XV
  • XF86VideoAdaptor
    • XFree86 offers the X video extension, which allows clients to treat video as any other primitive and put video into drawables. By default, the extension reports no video adaptors as being available because the DDX layer has not been initialized. The driver can initialize the DDX layer by filling out 1 or more XF86VideoAdaptorRecs, as described later in this document and passing a list of XF86VideoAdaptorPtr pointers to the following function:
    Bool xf86XVScreenInit(ScreenPtr pScreen, XF86VideoAdaptorPtr *adaptPtrs, int num)
    • After doing this, the extension reports video adaptors as per the availability, provided the data in their respective XF86VideoAdaptorRecs was valid. The xf86XVScreenInit() function copies data from the structure passed to it, so that the driver can free it after the initialization. At the moment, the DDX supports only rendering into window drawables. Pixmap rendering is supported after a sufficient survey of suitable hardware is completed. For more information, see http://www.xfree86.org/current/DESIGN16.html#64.
    • It contains the function pointers and the variables to be implemented by the X video driver.
    • The function pointers and variables of the XF86VideoAdaptor are:
flags
name
nEncodings, pEncodings
nFormats, pFormats
nPorts, pPortPrivates
nAttributes, pAttributes
nImages, pImages
typedef int (*PutVideoFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, 
RegionPtr clipBoxes, pointer data, DrawablePtr pDraw )
typedef int (*PutStillFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, 
RegionPtr clipBoxes, pointer data, DrawablePtr pDraw )
typedef int (*GetVideoFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, 
RegionPtr clipBoxes, pointer data, DrawablePtr pDraw )
typedef int (*GetStillFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, 
RegionPtr clipBoxes, pointer data, DrawablePtr pDraw )
typedef void (*StopVideoFuncPtr)(ScrnInfoPtr pScrn, pointer data, Bool Exit)
typedef int (*SetPortAttributeFuncPtr)(ScrnInfoPtr pScrn, Atom attribute, INT32 value, pointer data)
typedef int (*GetPortAttributeFuncPtr)(ScrnInfoPtr pScrn, Atom attribute, INT32*value, pointer data)
typedef void (*QueryBestSizeFuncPtr)(ScrnInfoPtr pScrn, Bool motion,short vid_w, short vid_h, short drw_w, short drw_h, unsigned int *p_w, unsigned int *p_h, 
pointer data)
typedef int (*PutImageFuncPtr)( ScrnInfoPtr pScrn, short src_x, short src_y, short drw_x, short drw_y, short src_w, short src_h, short drw_w, short drw_h, int 
image, unsigned char* buf, short width, short height, Bool Sync, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw )
typedef int (*QueryImageAttributesFuncPtr)(ScrnInfoPtr pScrn, int image, unsigned short *width, unsigned short *height, int *pitches, int *offsets)

For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86xv.h.

References

OpenGL ES 1.X, 2.0 and EGL 1.4 API specifications are supported by Tizen. OpenGL ES and EGL APIs are managed by the Khronos group. For more information about OpenGL ES API and prototype, see the Khronos website.

For information about the Xorg Input driver, see http://www.x.org/wiki/Development/Documentation/XorgInputHOWTO.

For the Khronos API Implementers Guide, see http://www.khronos.org/registry/implementers_guide.pdf

X server
Base open source: xorg-server
Base version: 1.12.99.905 (Some patches have been added to 1.12.2)
Base URL: http://cgit.freedesktop.org/xorg/xserver/snapshot/xserver-6619f5c0e1086b57888ff7146e8ed5897b50d440.tar.gz
Xorg configuration
http://www.x.org/releases/X11R7.7/doc/man/man5/xorg.conf.5.xhtml
https://wiki.archlinux.org/index.php/Xorg
http://linux.die.net/man/5/xorg.conf
Evdev driver
Base open source: xf86-input-evdev
Base version: 2.3.2
Base URL: http://cgit.freedesktop.org/xorg/driver/xf86-input evdev/snapshot/xf86-input-evdev-2.3.2.tar.gz
Evdev multi-touch driver
Base open source: xf86-input-evdev
Base version: 2.3.1
Base URL: http://cgit.freedesktop.org/xorg/driver/xf86-input-evdev/snapshot/xf86-input-evdev-2.3.1.tar.gz


Kernel configurations for input devices
Configuration Purpose
CONFIG_INPUT

CONFIG_INPUT_MISC

CONFIG_INPUT_EVDEV

For supporting the input device
CONFIG_INPUT_EVDEV For the evdev input system
CONFIG_INPUT_UINPUT For enabling input emulation
CONFIG_INPUT_TOUCHSCREEN For enabling the touch screen
CONFIG_INPUT_KEYBOARD For enabling keyboard devices such as gpio-key and touch-key
CONFIG_KEYBOARD_GPIO For gpio-key
CONFIG_INPUT_MOUSEDEV

CONFIG_INPUT_MOUSE_PSAUX

For mouse devices
Retrieved from "https://wiki.tizen.org/index.php?title=Porting_Guide/Graphics_and_UI&oldid=24450"