Porting Guide/Graphics and UI
The following figure shows an overview of the Tizen UI and graphics architecture.
Tizen provides high performance 3D graphics as a component of UI and graphics. There are hardware-accelerated OpenGL ES (Open Graphics Library Embedded System) and EGL (Embedded-System Graphics Library) for 3D applications, such as 3D games.
OpenGL ES is an application programming interface (API) for advanced 3D graphics, targeted to handheld and embedded devices. To overcome device constraints, such as limited processing capabilities and memory availability, it provides a subset of the functionality in OpenGL. Embedded system-specific features were added to enhance rendering efficiency, such as precision qualifiers to the shading language from OpenGL ES 2.0.
Contents
OpenGL
The following figure shows an overview of the interface for Tizen high performance 3D graphics library with OpenGL.
EGL is a "glue" layer between OpenGL ES and the underlying native platform window system - X Window system. EGL communicates with X Window to get information from the application window. And it creates a drawing surface and manages rendering context and resources.
3D Graphics Components
OpenGL ES is a software interface to graphics hardware. It is designed as a hardware-independent interface to be used for many different hardware platforms. OpenGL ES is a subset of the OpenGL designed for embedded systems. It accepts primitives, such as points, lines, and triangles, and converts them into pixels using a graphics pipeline, known as the OpenGL state machine.
EGL is a (mostly) platform-independent API and an interface between rendering APIs, such as OpenGL ES or OpenVG and an underlying native platform window system. It communicates with X Window system, and creates drawing surfaces and manages rendering contexts. The EGL interface has the following components:
- EGLDisplay (X Display): Encapsulates all of the system dependencies for interfacing with the native windowing system.
- EGLSurface: Encapsulates the rendering destinations (window surface, pixmap surface), which are tied to X Window and X Pixmap.
- EGLContext: Encapsulates the OpenGL ES rendering context, which holds the state of GL server and client.
The following figure shows a brief overview of EGL interface.
Porting
3D graphics vendors must provide these porting requirements for OpenGL ES/EGL:
- OpenGL ES
- The required version of OpenGL ES: 1.1 and 2.0
- The driver must support the following extensions to OpenGL ES 1.1:
GL_OES_framebuffer_object GL_OES_blend_subtract GL_OES_blend_func_separate GL_OES_matrix_palette GL_OES_draw_texture GL_OES_texture_cube_map GL_OES_query_matrix GL_OES_point_size_array
- The driver must support the following extensions to OpenGL ES 2.0:
GL_OES_EGL_image GL_EXT_texture_format_BGRA8888 GL_OES_get_program_binary GL_OES_texture_npot GL_OES_fragment_precision_high GL_OES_rgb8_rgba8 GL_OES_depth24 GL_OES_vertex_half_float GL_OES_texture_float GL_OES_compressed_ETC1_RGB8_texture GL_OES_packed_depth_stencil GL_OES_standard_derivatives GL_OES_element_index_uint GL_OES_mapbuffer GL_EXT_multi_draw_arrays GL_OES_vertex_array_object GL_IMG_texture_compression_pvrtc GL_OES_read_format GL_EXT_multisampled_render_to_texture
- You need some tools for generating a binary shader on Linux.
- EGL
- The minimum version required for EGL is 1.4.
- EGL must support DRI2, as the common 3D applications use DRI2 based EGL and the 3D composite window manager usually needs a DRI2 based EGL.
- FBDEV based EGL (optional)
- The FBDEV must consist of a triple buffer.
- It must be performed with page flipping instead of copying the buffer.
- To avoid a tearing problem, perform with LCD
vsync
signal. Usually, the compositor enablesvsync
using theeglSwapInterval()
API.
- DRI2 based EGL (mandatory)
- DRI2 based EGL must be performed with the X11 DRI2 protocol.
- It must support texture from pixmap (using the
EGL_KHR_image_pixmap
andGL_OES_EGL_image
).
- GPU synchronization API
- To avoid flickering, use an internal EGL API, which is used to wait for completion of the buffer copy command.
- Limitation of the EGL surface
- The FBDEV must consist of a triple buffer.
- The driver must support the following extensions to EGL 1.4:
EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_image_pixmap EGL_KHR_lock_surface EGL_KHR_fence_sync EGL_KHR_reusable_sync EGL_IMG_context_priority
Note |
---|
A method to allow the CPU to read and write the texture memory area of the GPU is needed. |
- Header and library name
The header files and libraries must be provided according to the Khronos API Implementers Guide. The following table gives a brief overview about the EGL library path and the respective header files.
Library path | Files |
---|---|
/usr/include/EGL
|
egl.h
|
/usr/include/GLES
|
gl.h
|
/usr/include/GLES2
|
gl2.h
|
/usr/include/KHR
|
khrplatform.h
|
/usr/lib
|
libEGL.so
|
Note |
---|
You may need to implement some memory management techniques for sharing memory between your CPU and the graphics processing unit. |
X Server
The X server is an X Window system display server that provides a basis for the graphical user interface (GUI) and rich input capabilities for your system. It creates a hardware abstraction layer where software is written to use a generalized set of commands, allowing for device independence and reuse of programs on any system that implements X.
Input Driver
The following figure shows the relations between the X Window system and input drivers.
- X Window system
The X Window system provides the interface for the manipulation of resources, such as windows, pixmap, and gc. X clients send the requests to the X server for drawing something or for manipulating windows. The X server accepts the clients’ requests and sends back replies to the requests. In addition, the X server sends X events when input events from input devices are pending or when the X server encounters something that the X client may be interested in, such as configure or exposure events.
- X server
The X server reads events from the internal event queue and makes and sends the X events to the X clients.
- X input driver
X input driver reads the input event stream from each input device node, makes X internal events, and put the events into the X server's internal event queue. For doing this, the X input driver mainly uses the interfaces implemented on the xf86 DDX layer inside X server.
- evdev driver
Implements for reading the input event stream originated from kernel evdev subsystem and for creating X internal events. Supports devices, such as keyboard, mouse, touchpad, and touch screen.
- evdev multi-touch driver
Implements for reading the MT protocol event stream originated from kernel evdev subsystem, for interpreting them and for creating X internal events. Supports touch screen devices.
Porting OAL Interface
The following section provides a brief understanding about X server and X input driver.
- Evdev module
The evdev contains the following drivers:
- Driver for absolute pointing device like touch-screen.
- Driver for relative pointing device like mouse.
- Driver for key (board) device, including HW key and BT/USB keyboard.
Each driver sets up each input device, and depends on the X server's configuration.
- After the device setup is done, the device reads input events from the device node, creates internal events, and puts them in the input event queue inside X server.
- To use the X input driver, the module must be loaded and must be registered as an input driver by the X server.
- Module, driver, and device must provide the following information:
- Module: module information.
- Driver: driver information.
- Device:
read()
for reading input events,proc()
for controlling the device status.
- For information about the data structures and functions for each module and device, see http://www.x.org/wiki/Development/Documentation/XorgInputHOWTO.
- The basic module information is stored inside XF86ModuleData (
xf86Module.h
) and is provided to the X server.
Configuration
ServerFlags
AllowEmptyInput
true
means that X server is started without any keyboard or mouse driver.
AutoAddDevices
false
means that hot plugged input devices are not supported.
AutoEnableDevices
true
means that theDevicePresenceNotify
event is sent to all X clients by default.
InputClass
- Identifier
- Sets the unique input device name.
- Driver
- Sets the driver for device.
MatchDevicePath
- Sets the device node path to be handled by the device. A wildcard can be used.
- Option
- Sets known options and custom options to be used inside driver for setting up devices or for enabling or disabling certain feature. Integer, Boolean, and string type can be used.
- Checks that the found device matches one of device types (
MatchIsTouchScreen
,MatchIsTablet
,MatchIsPointer
,MatchIsKeyboard
) and sets the type for the device if matched.
Tizen Extension for Evdev Multi-touch Driver
- Provides 1 legacy single touch protocol
- Provides 2 kinds of MT protocol:
- MT protocol A (which includes Tracking ID)
- MT protocol B
- To implement support for a new touch protocol:
- Implement a new touch protocol inside the driver.
- Clone the evdev multi-touch driver and implement a new touch protocol inside the driver.
- Convert from a new protocol to an existing protocol, which is supported by the evdev multi-touch driver.
Video Driver
The following figure illustrates the X server and the X video driver.
- X Window System
X Window system provides the interface for the manipulation of resources, such as windows, pixmap, and gc. The X clients send the requests to X server for drawing something or for manipulating windows. The X server accepts the clients’ requests and sends back replies to the requests. In addition, the X server sends X events to the client when input events from devices are pending or when X server encounters events that the X client may be interested, such as configure or exposure events.
- X server
X server gets the requests from X clients and delivers them to the xf86 porting layer. Input events are processed by the X server and the X server sends them to the X clients.
- X video driver
The X video driver is the driver library, which contains the implementation to support the xf86 DDX layer to display something on a screen.
- Driver Module:
- Implementation of a driver module for xf86 DDX.
- Mode Setting:
- Implements the devices to display images on it.
- EXA:
- Implements Graphic acceleration Architecture. Handles memory allocation and exa draw primitives.
- DRI2:
- Implements the DRI2 (Direct Rendering Infrastructure ver.2).
- XV:
- XFree86 offers the X Video Extension (XV), which allows clients to treat video as any other primitive and Put video into drawable mode. By default, the extension reports no video adapters as being available because the DDX layer has not been initialized.
- Mode Setting
Kernel module for Display mode setting.
- Functionality
- Crtc control and management
- Connector control and management
- Encoder control and management
- Framebuffer control and management
- MemMgr
Kernel module for graphic memory management.
- Functionality
- Control by mode setting
- Control by GPU
- Shared memory among processes
- Device Driver
Video controller hardware to control the kernel side of the module.
Porting OAL Interface
The X video driver must implement the following driver module and extensions, using the interface provided by the xf86 DDX and X extensions in the X server.
- Driver Module
XF86ModuleData
- A symbol data which the X server finds when the X server load drivers.
- The name of data is defined in the
modnameModuleData
format, where<modname>
is the name of the X video driver.
The following example defines the XF86ModuleData of the sec_drv.so
driver:
_X_EXPORT XF86ModuleData secModuleData = {&SECVersRec, SECSetup, NULL};
For xf86 header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86Module.h.
To implement a driver module, see http://www.x.org/releases/X11R7.6/doc/xorg-server/DESIGN.txt.
- ScrnInfoPtr
- The data structure must be initialized during the initialization of X video driver.
- The probe functions are called when initializing the X server. The function pointers are mapped in the probe function of the X video driver.
- The variables of the
ScrnInfoPtr
structure are set up by the function pointers ofScrnInfoPtr
. - The
ScrnInfoPtr
structure has the following function pointers:
xf86ProbeProc *Probe xf86PreInitProc *PreInit xf86ScreenInitProc *ScreenInit xf86SwitchModeProc *SwitchMode xf86AdjustFrameProc *AdjustFrame xf86EnterVTProc *EnterVT xf86LeaveVTProc *LeaveVT xf86ValidModeProc *ValidMode
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86str.h.
- Mode Setting
xf86CrtcConfigPtr
- The data structure contains the information of outputs, crtcs, and the configuration of outputs and crtcs.
- The function pointers,
xf86CrtcConfigFuncsRec *funcs
, have to be set in the X video driver. - The function pointers of the
xf86CrtcConfigFuncsRec
are:
Bool (*resize)(ScrnInfoPtr scrn, int width, int height);
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.
xf86CrtcPtr
- The data structure contains the information of a crtc.
- The X video driver creates the number of instances of crtcs as the number of hardware crtcs.
- The function pointers,
xf86CrtcFuncsRec *funcs
, have to be set in the X video driver. - The function pointers of the
xf86CrtcFuncsRec
are:
Bool (*set_mode_major)(xf86CrtcPtr crtc, DisplayModePtr mode, Rotation rotation, int x, int y); void (*set_cursor_colors) (xf86CrtcPtr crtc, int bg, int fg); void (*set_cursor_position) (xf86CrtcPtr crtc, int x, int y); void (*show_cursor) (xf86CrtcPtr crtc); void (*hide_cursor) (xf86CrtcPtr crtc); void (*load_cursor_argb) (xf86CrtcPtr crtc, CARD32 *image); void (*destroy) (xf86CrtcPtr crtc);
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.
xf86OutputPtr
- The data structure contains the output information.
- The X video driver creates the number of instances of outputs as the number of hardware connectors.
- The function pointers,
xf86OutputFuncsRec *funcs
, have to be set in the X video driver. - The function pointers of the
xf86OutputFuncsRec
are:
void (*create_resources)(xf86OutputPtr output); Bool (*set_property)(xf86OutputPtr output, Atom property, RRPropertyValuePtr value); Bool (*get_property)(xf86OutputPtr output, Atom property); xf86OutputStatus (*detect)(xf86OutputPtr output); int (*mode_valid)(xf86OutputPtr output, DisplayModePtr pMode); DisplayModePtr (*get_modes)(xf86OutputPtr output); void (*destroy) (xf86OutputPtr output);
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/modes/xf86Crtc.h.
- EXA
ExaDriverPtr
- The data structure contains the information on the X graphic acceleration.
- It contains the functionality of memory allocation of the pixmap private and EXA drawing primitives.
- It contains the function pointers and the variables to be implemented by the X video driver.
- The function pointers and variables of the
ExaDriverPtr
are:
int exa_major, exa_minor; CARD8 *memoryBase; unsigned long offScreenBase; unsigned long memorySize; int pixmapOffsetAlign; int pixmapPitchAlign; int flags; int maxX; int maxY; void (*WaitMarker) (ScreenPtr pScreen, int marker); Bool (*PrepareAccess)(PixmapPtr pPix, int index); void (*FinishAccess)(PixmapPtr pPix, int index); void *(*CreatePixmap)(ScreenPtr pScreen, int size, int align); void (*DestroyPixmap)(ScreenPtr pScreen, void *driverPriv); Bool (*ModifyPixmapHeader)(PixmapPtr pPixmap, int width, int height, int depth, int bitsPerPixel, int devKind, pointer pPixData); Bool (*PixmapIsOffscreen)(PixmapPtr pPix); Bool (*PrepareSolid) (PixmapPtr pPixmap, int alu, Pixel planemask, Pixel fg); void (*Solid) (PixmapPtr pPixmap, int x1, int y1, int x2, int y2); void (*DoneSolid) (PixmapPtr pPixmap); Bool (*PrepareCopy) (PixmapPtr pSrcPixmap, PixmapPtr pDstPixmap, int dx, int dy, int alu, Pixel planemask); void (*Copy) (PixmapPtr pDstPixmap, int srcX, int srcY, int dstX, int dstY, int width, int height) void (*DoneCopy) (PixmapPtr pDstPixmap); Bool (*CheckComposite) (int op, PicturePtr pSrcPicture, Pictureptr pMaskPicture, PicturePtr pDstPicture); Bool (*PrepareComposite) (int op, , PicturePtr pSrcPicture, Pictureptr pMaskPicture, PicturePtr pDstPicture, PixmapPtr pSrc, PixmapPtr, pMask, PixmapPtr pDst); void (*Composite) (PixmapPtr pDst, int srcX, int srcY, int maskX, int maskY, int dstX, int dstY, int width, int height); void (*DoneComposite) (PixmapPtr pDst); Bool (*UploadToScreen) (PixmapPtr pDst, int x, int y, int w, int h, char *src, int src_pitch); Bool (*DownloadFromScreen)(PixmapPtr pSrc, int x, int y, int w, int h, char *dst, int dst_pitch);
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=exa/exa.h.
- DRI2
DRI2InfoPtr
- The data structure for DRI2 (Direct Rendering Infrastructure ver.2)
- The structure manages and controls the DRI2 buffers.
- It contains the function pointers and the variables to be implemented by the X video driver.
- The function pointers and variables of the
DRI2InfoPtr
are:
const char *driverName; const char *deviceName; int fd; DRI2CreateBufferProcPtr CreateBuffer; DRI2DestroyBufferProcPtr DestroyBuffer; DRI2CopyRegionProcPtr CopyRegion; DRI2ScheduleSwapProcPtr ScheduleSwap; DRI2GetMSCProcPtr GetMSC; DRI2ScheduleWaitMSCProcPtr ScheduleWaitMSC; DRI2AuthMagicProcPtr AuthMagic; DRI2ReuseBufferNotifyProcPtr ReuseBufferNotify;
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg- server.git;a=blob;f=hw/xfree86/dri2/dri2.h.
- XV
XF86VideoAdaptor
- XFree86 offers the X video extension, which allows clients to treat video as any other primitive and put video into drawables. By default, the extension reports no video adaptors as being available because the DDX layer has not been initialized. The driver can initialize the DDX layer by filling out 1 or more
XF86VideoAdaptorRecs
, as described later in this document and passing a list ofXF86VideoAdaptorPtr
pointers to the following function:
Bool xf86XVScreenInit(ScreenPtr pScreen, XF86VideoAdaptorPtr *adaptPtrs, int num)
- After doing this, the extension reports video adaptors as per the availability, provided the data in their respective
XF86VideoAdaptorRecs
was valid. Thexf86XVScreenInit()
function copies data from the structure passed to it, so that the driver can free it after the initialization. At the moment, the DDX supports only rendering into window drawables. Pixmap rendering is supported after a sufficient survey of suitable hardware is completed. For more information, see http://www.xfree86.org/current/DESIGN16.html#64. - It contains the function pointers and the variables to be implemented by the X video driver.
- The function pointers and variables of the
XF86VideoAdaptor
are:
- XFree86 offers the X video extension, which allows clients to treat video as any other primitive and put video into drawables. By default, the extension reports no video adaptors as being available because the DDX layer has not been initialized. The driver can initialize the DDX layer by filling out 1 or more
flags name nEncodings, pEncodings nFormats, pFormats nPorts, pPortPrivates nAttributes, pAttributes nImages, pImages typedef int (*PutVideoFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw ) typedef int (*PutStillFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw ) typedef int (*GetVideoFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw ) typedef int (*GetStillFuncPtr)( ScrnInfoPtr pScrn, short vid_x, short vid_y, short drw_x, short drw_y, short vid_w, short vid_h, short drw_w, short drw_h, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw ) typedef void (*StopVideoFuncPtr)(ScrnInfoPtr pScrn, pointer data, Bool Exit) typedef int (*SetPortAttributeFuncPtr)(ScrnInfoPtr pScrn, Atom attribute, INT32 value, pointer data) typedef int (*GetPortAttributeFuncPtr)(ScrnInfoPtr pScrn, Atom attribute, INT32*value, pointer data) typedef void (*QueryBestSizeFuncPtr)(ScrnInfoPtr pScrn, Bool motion,short vid_w, short vid_h, short drw_w, short drw_h, unsigned int *p_w, unsigned int *p_h, pointer data) typedef int (*PutImageFuncPtr)( ScrnInfoPtr pScrn, short src_x, short src_y, short drw_x, short drw_y, short src_w, short src_h, short drw_w, short drw_h, int image, unsigned char* buf, short width, short height, Bool Sync, RegionPtr clipBoxes, pointer data, DrawablePtr pDraw ) typedef int (*QueryImageAttributesFuncPtr)(ScrnInfoPtr pScrn, int image, unsigned short *width, unsigned short *height, int *pitches, int *offsets)
For detailed header information, see http://source.tizen.org/git/?p=pkgs/xorg/server/xorg-server.git;a=blob;f=hw/xfree86/common/xf86xv.h.
References
OpenGL ES 1.X, 2.0 and EGL 1.4 API specifications are supported by Tizen. OpenGL ES and EGL APIs are managed by the Khronos group. For more information about OpenGL ES API and prototype, see the Khronos website.
For information about the Xorg Input driver, see http://www.x.org/wiki/Development/Documentation/XorgInputHOWTO.
For the Khronos API Implementers Guide, see http://www.khronos.org/registry/implementers_guide.pdf
- X server
- Base open source:
xorg-server
- Base version: 1.12.99.905 (Some patches have been added to 1.12.2)
- Base URL: http://cgit.freedesktop.org/xorg/xserver/snapshot/xserver-6619f5c0e1086b57888ff7146e8ed5897b50d440.tar.gz
- Xorg configuration
- http://www.x.org/releases/X11R7.7/doc/man/man5/xorg.conf.5.xhtml
- https://wiki.archlinux.org/index.php/Xorg
- http://linux.die.net/man/5/xorg.conf
- Evdev driver
- Base open source:
xf86-input-evdev
- Base version: 2.3.2
- Base URL: http://cgit.freedesktop.org/xorg/driver/xf86-input evdev/snapshot/xf86-input-evdev-2.3.2.tar.gz
- Evdev multi-touch driver
- Base open source:
xf86-input-evdev
- Base version: 2.3.1
- Base URL: http://cgit.freedesktop.org/xorg/driver/xf86-input-evdev/snapshot/xf86-input-evdev-2.3.1.tar.gz
- Kernel configurations for input devices
Configuration | Purpose |
---|---|
CONFIG_INPUT
|
For supporting the input device |
CONFIG_INPUT_EVDEV
|
For the evdev input system |
CONFIG_INPUT_UINPUT
|
For enabling input emulation |
CONFIG_INPUT_TOUCHSCREEN
|
For enabling the touch screen |
CONFIG_INPUT_KEYBOARD
|
For enabling keyboard devices such as gpio-key and touch-key |
CONFIG_KEYBOARD_GPIO
|
For gpio-key |
CONFIG_INPUT_MOUSEDEV
|
For mouse devices |