Rootstrap configuration guide

From Tizen Wiki
Jump to: navigation, search


Tester image creation, configuration guide


Testing is performed in a KVM Virtual machine. A running VM has two disk images mounted:

  • HDA contains system. There is one HDA per target system.
  • HDB contains /home with project code and run script. All targets share one image.

To prepare a tester session, the Jenkins job loop-mounts HDB, copies tester scripts to HDB, un-mounts HDB, starts kvm-qemu. In the VM session, a script /home/build/run is started after bootup using a crontab entry. Testin script output redirects to tty of VM, comes to Jenkins console. Tester script performs shutdown after testing is complete. Control returns to Jenkins job which loop-mounts HDB image again and retrieves test reports. At the end of session log there will be tail of dmesg and syslog.

How to start kvm session manually

Sometimes it is needed to run/update seed images manually. For that, use similar cmd what automated tester uses, but remember to replace MAC address and VNC display number with some unique values, otherwise manual session may conflict with automated session if running in same host (I usually run manual sessions in debug worker tworker-179). Note about VNC number: starting with version 1.22.1 of tester scripts, vnc number is picked initially as Jenkins slot number, but if such a socket (5900+vncnum) is already in use, another number +20 is tried. That ensures vnc socket does not overlap and existing qemu process will not block a new one.


cp kvm-seed-hda-Fedora_19-x86_64-debug hda2
chmod 644 hda2
tar SxfO - < kvm-seed-hdb.1G.tar > hdb2
qemu-kvm -name f20-64 -cpu core2duo -M pc -m 2048 -hda hda2 -hdb hdb2 -net nic,macaddr=52:54:78:87:86:85 -net user -vnc :43

HDA and HDB image files

HDA and HDB are sparse files. User should take care of not losing sparseness in copy and transfer. cp preserves sparseness, but rsync by default loses sparseness unless -S option is used.

For new target, always start with creation of new sparse 6G image. Attempt to re-use of previous images will make spareseness worse.

Steps to prepare HDB image for tester home

Note: Only needed once, as all targets share same hdb.

qemu-img create -f raw hdb 20G

Initialize a new hdb

In KVM session: create partition (can use parted), format eft4 FS on it: (assuming the HDB disk is /dev/sdb)

parted /dev/sdb
 mklabel msdos
 mkpart primary 1 -1
mkfs.ext4 /dev/sdb1
mount /dev/sdb1 /home

Steps to prepare a HDA image for a tester target VM

Create empty 6 GB disk image for system

qemu-img create -f raw hda 6G 

Start KVM machine for installing new distro from DVD image to "hda"

Note: install system tries to guess needed swap size based on RAM size (at least Ubuntu does this) so do not use large RAM for installation, because we dont want to spend too big part of 6GB for swap.

qemu-kvm -name Fedora16 -M pc -m 1024 -boot d -hda hda -drive file=Fedora-16-x86_64-DVD.iso,media=cdrom -vnc :22

From desktop, connect to VNC display with VNC viewer

examples: - xfvnc, using Jenkins worker IP and display as :22 - gvncviewer

Notes for DVD and distro type selection:

  • Fedora: select Base Environment=Minimal Install, Add-Ons=Standard, review/modify disk partitions, partition scheme:Standard
  • Fedora 22/32bit: can not start X server, I used text based install
  • Ubuntu: server-ARCH.iso (not desktop, it has no option for small system); use F4: "minimal VM". Later in install: components: Base system, OpenSSH server.
  • Fedora-18,19, CentOS_6: resize partitions, root:4616 MB swap:1024MB (or default will make big swap and small rootfs)
  • openSUSE: set screen resolution to 800x600, using F3; Desktop Selection: Other-Minimal Server Selection, Installation Settings: revert Firewall (disabled) and ssh (enabled)

After install, reboot, and start KVM without DVD

  • Ubuntu: login as "ubuntu" (was created duing installation), sudo bash, passwd -- to set root passwd and being able to login as root
  • Ubuntu 12.xx,13.xx: service whoopsie stop; apt-get remove whoopsie libwhoopsie0

Before wiping repos config: Install utils that are used inside tester VM

Tester VM will not have repos configured, but after installation, repos config is needed to perform 1st update and to install utils. Point to local mirror, or config proxy, etc.

yum update
yum install which python-nose python-coverage python-mock pylint make
Fedora 22 note: yum is replaced with dnf:
dnf install which python-nose python-coverage python-mock pylint make
yum update
Centos_6: yum install openssh-clients net-tools
Centos_7: yum install net-tools
yum install python-nose python-coverage python-mock pylint make
Debian, Ubuntu
apt-get update
apt-get upgrade
apt-get install debianutils python-nose python-coverage python-mock pylint make
zypper ref
zypper up
#Note: zypper proposes rm of base-conflicts, so no need to type it
zypper rm patterns-openSUSE-minimal_base-conflicts 
# (otherwise does not let install python-pylint)

zypper in util-linux python-nose python-coverage python-mock pylint make
openSUSE 13.2:
zypper in util-linux python-nose python-coverage python-mock python3-pylint make e2fsprogs

openSUSE: can switch graphical boot screen off: yast -- System-Boot Loader -- Options

Clear repositories config

Starting with version 1.35.0 of tools-testing package, tester VMs do not have any repo config in them. Repo config is created dynamically by tester scripts, based on /etc/tools-tester.d/base-repos-ENV.conf which is installed on all worker hosts from package tools-testing-settings-ENV Jobs that use HDA in snapshot mode (regular tester jobs), can leave created repo config there, it will not be saved. Job that should leave usable state behind, (like packages update job), needs to clear repo config before performing shutdown.

Cleaning of repo config after creation of new VM from distro:

  • Fedora, CentOS
rm -f /etc/yum.repos.d/*
  • Centos

Same as Fedora. Note that CentOS repo config uses EPEL from Fedora repos.

CentOS_6 wants to rename ethX with every next unique MAC we use. To stop it: /etc/udev/rules.d/70-persistent-net.rules: modify eth0 line to be:

SUBSYSTEM="net", ACTION="add", DRIVERS=="?*", KERNELS=="0000:00:03.0", NAME="eth0"

Note: KERNELS value comes from pci address, check: ls -l /sys/class/net

remove /lib/udev/rules.d/75-persistent-net-generator.rules which would overwrite above 70-persistent-net.rules file

CentOS_7: no need to change udev stuff

  • Debian, Ubuntu
rm -f /etc/apt/sources.list
  • OpenSUSE
rm -f $/etc/zypp/repos.d/*

Common to all: Add user build without homedir creation, uid=777 needs to be same as jenkins in Tester

useradd -M -u 777 build 

Enable automatic start of /home/build/run

During history of otctools testing, several methods of starting testing have been tried. It is problematic to keep multiple methods (systemd vs init-scripts vs ubuntu config style). Under heavy load, problems arise because of startup race and order of events:

  • slow startup of DHCP iface causes testing fail if tester starts too early;

if-up hooks were tried, but slow mount of /home directory still caused failures;

Finally, current unified method seems to work on all targets. There is entry in root crontab:

@reboot /root/runtester > /dev/ttyS0 2>&1

/root/runtester checks for tester start conditions:

- iface up and usable i.e. default route is set to
- /home is mounted

If these do not come true during 300s, we reboot for retry, and there is limit set to reboot count. Reboot in kvm session will restart VM without returning control to Jenkins.

/root/runtester: This script is now part of tools-testing git repo.

Add second image mount as /home in /etc/fstab

/dev/sdb1   /home   ext4   defaults  1  1

Shutdown VM, boot with two disks to verify /home gets mounted from sdb1

qemu-kvm -name "Ubuntu-11" -M pc -m 2048 -hda hda -hdb hdb -vnc :0

In hdb seed image (only needed once, hdb is re-used)

  • Create git identity for user build: /home/build/.gitconfig:
        email =
        name = Jenkins Worker
  • Create /home/build/reports directory owned by user build.

Shut down KVM, rename hda as seed

Note that images mgmt happens on Tester-179 and seed images are named as -debug. Deploy scripts will handle renaming and replicating to other workers.

mv hda kvm-seed-hda-$REPO-$ARCH-debug

Example name: kvm-seed-hda-Centos_6-x86_64-debug

hdb is shared by all, so this is only needed for first time:

mv hdb kvm-seed-hdb

For trials, and any sessions run manually, should copy kvm-hdb-seed to hdb and discard it later.

Useful offsets of rootfs in HDA image for loop mounting

In many cases, start of VM is not needed for changes in rootfs, loop mount can be used instead. Here are notes for loop mounting.

# CentOS_6: cant mount, uses LVM

# Debian 7, Ubuntu 12.04, 12.10, 13.10 : rootfs starts 2048, *512=1048576
# mount -o loop,offset=1048576 -t ext4 -v hda tmpmount/

# Fedora 19: rootfs partition starts 3074048, *512=1573912576
# mount -o loop,offset=1573912576 -t ext4 -v hda tmpmount/

# Fedora 20: rootfs partition starts 2283520, *512=1169162240
# mount -o loop,offset=1169162240 -t ext4 -v hda tmpmount/

# openSUSE 12.1-x86_64, 12.2-x86_64, 12.3 : rootfs partition starts 1044480, *512=534773760
# mount -o loop,offset=534773760 -t ext4 -v hda tmpmount/

# openSUSE 12.1-i586: rootfs partition starts 2072576, *512=1061158912
# mount -o loop,offset=1061158912 -t ext4 -v hda tmpmount/

# openSUSE 12.2-i586 : rootfs partition starts 1814528, *512=929038336
# mount -o loop,offset=929038336 -t ext4 -v hda tmpmount/

# openSUSE 13.1-i586: rootfs partition starts 2072576, *512=1061158912
# mount -o loop,offset=1061158912 -t ext4 -v hda tmpmount/

# openSUSE 13.1-x86_64: rootfs partition starts 2056192, *512=1052770304
# mount -o loop,offset=1052770304 -t ext4 -v hda tmpmount/

Additional config and fixes in KVM Tester images

Remove other mailers, install sendmail (old)

In several kvm seed images we had to remove postfix pkg, because cats-* packages require sendmail and this installation conflict is not automatically handled. Check that no other mailers like postfix, exim are installed, safer is to install sendmail in seed image.

Ubuntu: log.c assert panic in

August 2013: Ubuntu images kernel cmdline got appended "--no-log" because sometimes there was panic when booting up:

vi /etc/default/grub
# locate line: GRUB_CMDLINE_LINUX_DEFAULT and add --no-log to its value
OpenSUSE 12.1 and 12.2: loop module added to /etc for loading on boot

2013-11-19: mic testing failed with GPT parted error, reason seemed to be that kernel loop module was not loaded after boot. Solution: add file /etc/module-load.d/loop.conf with single line "loop" in it.

Fedora-19,20: increase loop devices max count

Added /etc/modprobe.d/loop.conf with single line:

options loop max_loop=32
Fedora 19,20; remove some systemd-started services and packages we do not use (optional)
yum remove mdmonitor nfs remote-fs smartd mcelog rngd
Fedora 19,20 remove fingerprint packages as the service always fails to start:
yum remove fprintd fprintd-pam libfprint