SWAP Python tool API

From Tizen Wiki
Jump to: navigation, search

This page represents SWAP Python tools API description.

Python tool API description

Config files description

SWAP Pyton tool sources contain an example of tool configuration files, located at <swap-manager>/src/cli/example/. The folder contains several config files:

File Content description
emulator.py 
# Provides target device/emulator configuration.

api_version = '1.0'
# Used to verify compatibility of different config files;

connection = {
# Dictionary, containing data about the connection with the target;

    'ip': "127.0.0.1", 
# target IP address;

    'type': "sdb",
# target connection type. Possible values:
# ssh - for SSH connection;
# sdb - for SDB connection;
# alone - for standalone profiling;

    'type_info': {
        'target': 'emulator',
    },
# information about selected connection type, differs for different type:
# for sdb:
# target - target type, emulator, device and serial are supported;
# for ssh:
# params - SSH connection parameters;
# username - SSH username.
}
example_conf.py 
# Provides application's profiling session configuration.

api_version = '1.0'
# Used to verify compatibility of different config files;

sampling = {
# Turns on SWAP sampling feature and contains its options;

    "period": 50,
# Period of sampling;
}                                                                        

features = [
# Features, that doesn't require additional options, are turned on by including to this list. Available features are:
    'memory',          # memory API profiling feature for target binaries;
    'file',            # file API profiling feature for target binaries;
    'network',         # network API profiling feature for target binaries;
    'thread',          # thread API profiling feature for target binaries;
    'opengl',          # OpenGL API profiling feature;
    'memory_for_all',  # memory API profiling for all process binaries;
    'file_for_all',    # file API profiling for all process binaries;
    'network_for_all', # network API profiling for all process binaries;
    'context_switch',  # context switch profiling;
    'lsan'             # Leak Sanitizer feature;
]

app = {
# Dictionary that contains application profiling data and options;

    'app_info': {
# Dictionary that specifies application for profiling;

        'type': 'NATIVE',
# Application type, possible values are:
# NATIVE - for Tizen native applications;
# COMMON - for console applications (isn't supported by SWAP);
# RUNNING - for already running applications (isn't supported by SWAP);
# WEB - for Tizen web applications (doesn't supported now);

        'id': 'org.example.filemanager',
# The second parameter depends on the type value:
# for NATIVE:
# id - Tizen native application ID;
# for COMMON:
# path - path where application binary is located;
# for RUNNING:
# pid - PID of the target process. There is a special case for 'pid': -1 - this causes system-wide instrumentation for all processes;
# id - Tizen ID of the target application;
# path - path where application binary is located;
# for WEB:
# id - Tizen web application ID.

    },
    'function_inst': {
# Dictionary that contains information about binary instrumentation probes;

        '/opt/usr/globalapps/org.example.filemanager/bin/filemanager': [
# Each key of a dictionary is a path to binary for profiling;

            ('main', 'xxxx', 'd'),
        ],
# Each value is an array for the key binary that contains tuples of the following format:
# function name in '' or address;
# string that describes arguments of a target function, one character for each argument. The following values are accepted:
# b - for bool value;
# c - for char value;
# f - for float value;
# d - for int value;
# x - for long value;
# p - for pointer value;
# w - for double value;
# s - for C-string;
# character that describes returned value. Accepted all characters for arguments and more:
# v and n - for function returns void

Python API and usage description

SWAP Python tool example also contains tools API usage example in example.py file:

File Content description
example.py 
#!/usr/bin/python2

import time
from swap_cli import swap
# To use SWAP Python tool swap module should be imported from swap_cli folder.

def simple_cb(data):
    pass

# load configs
c = swap.Client('example/emulator.py')
c.load_config(['example/example_conf.py'])
# SWAP Python tool interface is provided via Client class object. Client class object constructor takes path to the target config as an argument, so the user should have one Client for each target. To provide profiling session configuration, load_config() method is used, it takes profiling session config file as an argument.

c.set_event_cb('MSG_PROCESS_INFO', simple_cb)
# After necessary configurations are loaded, some additional preparations can be done, for example, in this example callback is set on MSG_PROCESS_INFO - it will be called when MSG_PROCESS_INFO will be received.

# instrumentation
c.start('/tmp/outdir')
# Profiling is started by start() method, it takes output folder path as an argument. In an output folder, you can find trace and another additional data (LSan reports) after profiling.

time.sleep(60)
c.stop()
# Profiling is finished by calling stop() method.

The whole SWAP Python tool API is described below:

API name Arguments Return value Description 
class Client
        
(constructor) (target conf)
 target_conf
string path to a file, that contains target device/emulator config (see above description of emulator.py for details) 		object of Client class The class provides all SWAP Python tool functionality, which is implemented as methods of the class.
The class constructor takes target configuration file path as an argument, so, if several different targets are used for profiling, several Client class objects should be created. Configuration file description and details can be found above as a description of emulator.py. 
get_version()
 None string that matches SWAP Python tool version Returns SWAP Python tool version. 
load_config(path)
 path
string, path to a file that contains profiling session configuration (see above description of example_conf.py for details)		 None This function is used to load application's profiling data, i.e. which application is a target one, which features should be used.

example_conf.py can be used as an example of this configuration file, its description above covers all of the possible meanings. 

set_event_callback(event, callback)
 event
string, that matches message name
callback
pointer to a function that will be called when event message is received 		None This function allows registering user-defined callback on any received SWAP message.

Event name depends on a protocol that is being used and can be received from protocol description. For example, protocol 4.2 supports the following event names:

  • MSG_PROCESS_INFO
  • MSG_TERMINATE
  • MSG_ERROR
  • MSG_SAMPLE
  • MSG_SYSTEM
  • MSG_FUNCTION_ENTRY
  • MSG_FUNCTION_EXIT
  • MSG_SYSCALL_ENTRY
  • MSG_SYSCALL_EXIT
  • MSG_FILE_FUNCTION_ENTRY
  • MSG_FILE_FUNCTION_EXIT
  • MSG_PROCESS_STATUS_INFO
  • MSG_CONTEXT_SWITCH_ENTRY
  • MSG_CONTEXT_SWITCH_EXIT
  • MSG_PROCESS_MAP
  • MSG_PROCESS_UNMAP
  • MSG_WEB_SAMPLING
  • MSG_APP_SETUP_STAGE
  • MSG_WEB_APP_SETUP_STAGE
  • MSG_FBI
  • MSG_UI_HIERARCHY
  • MSG_PROBE_MEMORY
  • MSG_PROBE_UIEVENT
  • MSG_PROBE_RESOURCE
  • MSG_PROBE_LIFECYCLE
  • MSG_PROBE_SCREENSHOT
  • MSG_PROBE_THREAD
  • MSG_PROBE_SYNC
  • MSG_PROBE_NETWORK
  • MSG_PROBE_GL
  • MSG_LSAN

To get description which message stands for which event, please refer to the protocol description.
callback is a pointer to the callback function which should be called when event message is received. The callback function should have the following signature:
callback(data)
where data is message payload. Payload has a protocol- and message-specific format, so, please refer to the protocol description for details. 

start(outdir)
 outdir
string with path to an output directory where trace, LSan reports, necessary target info etc. will be located. The directory will be created if it doesn't exist.		 None It establishes connection between SWAP Python tool on host and SWAP on target and starts profiling target application. After workflow has returned from start() SWAP is supposed to be executed on target.

Trace is saved to <outdir>/trace.bin file. 

is_alive()
 None bool - True if SWAP has returned response, False otherwise Function checks if connection with target SWAP works and if SWAP responds on host's requests. 
stop()
 None None Stops tracing, closes connection with the target SWAP.

Trace Parser API description

Besides SWAP trace, Trace Parser also requires some additional data about gathered trace: trace protocol version, target device/emulator info about CPU cores count and count of energy devices located on target (its top energy consumers, like LCD, CPU, WiFI and so on). Optionally, API map list can be added from the target: it provides additional data about gathered API functions.
User can specify trace data manually or, which is most preferable, take it from output directory of SWAP tool with the trace. File named session_data.json contains all the necessary data for tracing.
API map list file can also be found at the output of SWAP tool, it's named api_map_list.

Protocol versions

Trace Parser supports the following protocol versions:

  • 3.0
  • 4.0
  • 4.1
  • 4.2

Protocol version should be chosen the same as one on the target where the trace was gathered. For detailed information and protocol description, please refer to Protocol Description located at swap-manager/docs/protocol.rst or at swap-manager README file.
If the user hasn't specified any protocol version, 4.2 protocol version will be used as default.

Output types

Trace Parser supports the following output types:

Type Short Description
text t Output in human-readable text format
python p Output is formatted like python module
json j Output in JSON format
csv c Output in comma-separated value format

The user can specify the type by its name (Type column) or short name (Short column).

Text output format

msg_id = 0x1                            //message ID number in hex format
seq_num = 0                             //message sequence number
sec = 1510680912                        //second when event has occurred
usec = 436919901                        //microsecond when event has occurred
payload_len = 8617                      //length of message specific data
...                                     // Rest is message specific information
=============================           // Such line separates messages

Python output format

trace_info = {                          # Contains data about parsed trace
    'protocol_version': 42              # Trace's protocol version
}
trace obj = [                           # Array that contains all trace messages. Each message is a dictionary
    {
        'msg_id': 0x1,                  # Message ID number in hex
        'seq_num': 0,                   # Message sequence number
        'sec': 1510680912,              # Second when event has occurred
        'usec': 436919901,              # Microsecond when event has occurred
        'payload_len': 8617,            # Length of message specific data
        ...,                            # Message specific data
    },
    ...                                 # Other messages
]

JSON output format

{
"trace_info" : {                        # Contains data about parsed trace
    "protocol_version" : 42             # Trace's protocol version
},
"trace_obj" : [                         # Contains all trace messages
    {
        "msg_id" : 1,                   # Message ID number
        "seq_num': 0,                   # Message sequence number
        "sec": 1510680912,              # Second when event has occurred
        "usec": 436919901,              # Microsecond when event has occurred
        "payload_len": 8617,            # Length of message specific data
        ...                             # Message specific data
    },
    ...                                 # Other messages
]

CSV output format

Message ID Sequence number Second Microsecond Length Message specific data
0x1, 0, 1510680912, 436919901, 8617, ...

Arguments description

Trace Parser supports the following command-line arguments:

Argument Options Description
--help, -h Print usage
--input, -i input file path Path to trace file. If not specified, trace is taken from stdio. Optional
--output, -o output file path Path to output file. If not specified, data outputted to stdout. Optional
--session, -s session data file Path to session_data.json file. If not specified, target data is taken from command line arguments or default. Optional
--api_map, -a API map list file Path to api_map_list file. Optional
--cpu_num, -c CPUs number Number of CPUs at the target device. If not specified, is taken from session_data.json or default. Optional
--devs_num, -d energy devices number Number of energy devices on target. If not specified, is taken from session_data.json or default. Optional
--version, -v protocol version Protocol version. If not specified, is taken from session_data.json or default. Optional
--type, -t output type Output format

You can run Trace Parser by running the following command in a directory, which contains swap_parser executable and libspp.so library. For example, with session_data.json and api_map_list specified: 

$ ./swap_parser -i trace.bin -o trace.txt -s session_data.json -a api_map_list -t text

This command will read trace.bin, parse it into text format and output to trace.txt. The same command, if target info data is specified manually: 

$ ./swap_parser -i trace.bin -o trace.txt -c 4 -d 5 -v 4.2 -a api_map_list -t text

If user wants to use default data, it will be like the following: 

$ ./swap_parser -i trace.bin -o trace.txt -t text

Or the minimum command, that expects trace at stdin and outputs it to stdout: 

$ ./swap_parser -t text

For Development and support guide see attached File:MAINTAIN.pdf.

Advanced usage examples

All the examples below expects that the preparation work has been already done and that current directory is <swap-manager>/src/cli/.

Function profiling on emulator, storing data in human-readable format

  • Creating target configuration file, saved as target_conf.py:
api_version = '1.0'

connection = {
        'ip': "127.0.0.1",

        'type': "sdb",
        'type_info': {
                'target': 'emulator',
        },
}
  • Creating profiling session configuration file, saved as session_conf.py:
api_version = '1.0'

env = {
        'apps_path': "/opt/usr/globalapps/",
}

features = [
]

app = {
    'app_info': {
        'type': 'NATIVE', 'id': 'org.example.filemanager',
    },

    'function_inst': {
        '/opt/usr/globalapps/org.example.filemanager/bin/filemanager': [
            ('main', 'xxxx', 'd'),
        ],
        '/lib/libc.so.6': [
            ('malloc', 'd', 'p'),
        ],
    },
}
  • Creating execution script, saved as run_script.py:
#!/usr/bin/python2

import time
from swap_cli import swap

# load configs
c = swap.Client('target_conf.py')
c.load_config(['session_conf.py'])

# instrumentation
c.start('/tmp/outdir')
time.sleep(60)
c.stop()
  • Execute run_script.py and wait till it finished;
  • Parse binary trace to human-readable format:
$ trace_parser/bin/swap_parser -i /tmp/outdir/trace.bin -o /tmp/parsed_trace.txt -t t -s /tmp/outdir/session_info.json
  • Now parsed trace is stored at /tmp/parsed_trace.txt.

Memory profiling on device, storing data in Python module format

  • Modifying created target_conf.py:
api_version = '1.0'

connection = {
        'ip': "127.0.0.1",

        'type': "sdb",
        'type_info': {
                'target': 'device',
        },
}
  • Modifying created session_conf.py:
api_version = '1.0'

env = {
        'apps_path': "/opt/usr/globalapps/",
}

features = [
    'memory'
]

app = {
    'app_info': {
        'type': 'NATIVE', 'id': 'org.example.filemanager',
    },
}
  • Execute run_script.py and wait till it finished;
  • Parse binary trace to Python module format:
$ trace_parser/bin/swap_parser -i /tmp/outdir/trace.bin -o /tmp/parsed_trace.py -t p -s /tmp/outdir/session_info.json
  • Now parsed trace is located at /tmp/parsed_trace.py, it can be imported at any Python script with the code like, depending on the Python script location:
import parsed_trace

Counting malloc()'s for the application on device

  • Modifying session_conf.py:
api_version = '1.0'

env = {
        'apps_path': "/opt/usr/globalapps/",
}

features = [
]

app = {
    'app_info': {
        'type': 'NATIVE', 'id': 'org.example.filemanager',
    },

    'function_inst': {
        '/lib/libc.so.6': [
            ('malloc', 'd', 'p'),
        ],
    },
}
  • Modifying execution script run_script.py:
#!/usr/bin/python2

import time
from swap_cli import swap

malloc_cnt = 0

def malloc_cb(data):
    malloc_cnt += 1

# load configs
c = swap.Client('target_conf.py')
c.load_config(['session_conf.py'])
c.set_event_cb('MSG_FUNCTION_ENTRY', malloc_cb)

# instrumentation
c.start('/tmp/outdir')
time.sleep(60)
c.stop()

print "Malloc count = " + malloc_cnt
  • Execute run_script.py and wait till it finished
  • At the end, malloc()'s count is printed in stdout.
Retrieved from "https://wiki.tizen.org/index.php?title=SWAP_Python_tool_API&oldid=28744"