summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
Diffstat (limited to 'documentation')
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml414
1 files changed, 414 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index 7f59c39f2d..2dbc3d14b8 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -4252,6 +4252,420 @@
4252 </section> 4252 </section>
4253 </section> 4253 </section>
4254 4254
4255 <section id="running-and-writing-tests-for-a-qemu-image">
4256 <title>Running and Writing Tests for a QEMU Image</title>
4257
4258 <para>
4259 The OpenEmbedded build system makes available a series of automated
4260 tests for QEMU images.
4261 These tests are commands that run on the target system over
4262 <filename>ssh</filename> and are written in Python, thus making
4263 use of the <filename>unittest</filename> module.
4264 </para>
4265
4266 <para>
4267 This section describes how you set up the environment to use these
4268 tests, run available tests, and write and add your own tests.
4269 You can the same information in a different form and with example
4270 log files of test runs on the
4271 <ulink url='https://wiki.yoctoproject.org/wiki/Image_tests'>Image Tests</ulink>
4272 Wiki page.
4273 </para>
4274
4275 <section id="qemu-image-enabling-tests">
4276 <title>Enabling Tests</title>
4277
4278 <para>
4279 In order to run tests, you need to do the following:
4280 <itemizedlist>
4281 <listitem><para><emphasis>Ensure you run the test
4282 automatically with no interaction and under
4283 <filename>sudo</filename>:</emphasis>
4284 To accomplish this, you must do one of the
4285 following:
4286 <itemizedlist>
4287 <listitem><para>Add
4288 <filename>NOPASSWD</filename> for your user
4289 in <filename>/etc/sudoers</filename> either for
4290 ALL commands or just for
4291 <filename>runqemu-ifup</filename>.
4292 You must provide the full path as that can
4293 change if you are using multiple clones of the
4294 source repository.
4295 <note>
4296 On some distributions, you also need to
4297 comment out "Defaults requiretty" in
4298 <filename>/etc/sudoers</filename>.
4299 </note></para></listitem>
4300 <listitem><para>Manually configure a tap interface
4301 for your system.</para></listitem>
4302 <listitem><para>Run as root the script in
4303 <filename>scripts/runqemu-gen-tapdev</filename>,
4304 which should generate a list of tap devices.
4305 List generation is usually done in environments
4306 similar to AutoBuilder.</para></listitem>
4307 </itemizedlist></para></listitem>
4308 <listitem><para><emphasis>Set up the
4309 <filename>DISPLAY</filename> variable:</emphasis>
4310 You need to set this variable so that you have an X
4311 server available (e.g. start
4312 <filename>vncserver</filename> for a headless machine).
4313 </para></listitem>
4314 <listitem><para><emphasis>Be sure your host's firewall
4315 accepts incoming connections from
4316 192.168.7.0/24:</emphasis>
4317 Some of the tests (in particular smart tests) start a
4318 HTTP server on a random high number port, which is
4319 used to serve files to the target.
4320 The smart module serves
4321 <filename>${DEPLOY_DIR}/rpm</filename> so it can run
4322 smart channel commands. That means your host's firewall
4323 must accept incoming connections from 192.168.7.0/24,
4324 which is the default class used for
4325 <filename>tap0</filename> devices by
4326 <filename>runqemu</filename>.</para></listitem>
4327 <listitem><para><emphasis>Have your QEMU image built:</emphasis>
4328 The QEMU image on which you want to run tests needs to
4329 be built.
4330 For example, the following command would build the
4331 <filename>core-image-sato</filename> image when
4332 <filename>MACHINE</filename> is set to "qemu".
4333 <literallayout class='monospaced'>
4334 bitbake core-image-sato
4335 </literallayout></para></listitem>
4336 <listitem><para><emphasis>Globally inherit
4337 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink></emphasis>
4338 Edit your <filename>local.conf</filename> file as
4339 follows:
4340 <literallayout class='monospaced'>
4341 INHERIT += "testimage"
4342 </literallayout></para></listitem>
4343 </itemizedlist>
4344 </para>
4345
4346 <note>
4347 If you built your QEMU image using <filename>rm_work</filename>,
4348 most of the tests will fail with errors because they rely on
4349 <filename>${WORKDIR}/installed_pkgs.txt</filename>.
4350 </note>
4351 </section>
4352
4353 <section id="qemu-image-running-tests">
4354 <title>Running Tests</title>
4355
4356 <para>
4357 After setting up to run the tests, use BitBake to start
4358 the standard suite of tests:
4359 <literallayout class='monospaced'>
4360 bitbake core-image-sato -c testimage
4361 </literallayout>
4362 </para>
4363
4364 <para>
4365 Once you start the tests, the following happens:
4366 <itemizedlist>
4367 <listitem><para>A copy of the root filesystem is written
4368 to <filename>${WORKDIR}/testimage</filename>.
4369 </para></listitem>
4370 <listitem><para>The image is booted under QEMU using the
4371 standard <filename>runqemu</filename> script.
4372 </para></listitem>
4373 <listitem><para>A default timeout of 500 seconds occurs
4374 to allow for the boot process to reach the login prompt.
4375 You can change the timeout period by setting
4376 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink>
4377 in the <filename>local.conf</filename> file.
4378 </para></listitem>
4379 <listitem><para>Once the boot process is reached and the
4380 login prompt appears, the tests run.
4381 The full boot log is written to
4382 <filename>${WORKDIR}/testimage/qemu_boot_log</filename>.
4383 </para></listitem>
4384 <listitem><para>Each test module loads in the order found
4385 in <filename>TEST_SUITES</filename>.
4386 You can find the full output of the commands run over
4387 <filename>ssh</filename> in
4388 <filename>${WORKDIR}/testimgage/ssh_target_log</filename>.
4389 </para></listitem>
4390 <listitem><para>If no failures occur, the task running the
4391 tests ends successfully.
4392 You can find the output from the
4393 <filename>unittest</filename> in the task log at
4394 <filename>${WORKDIR}/temp/log.do_testimage</filename>.
4395 </para></listitem>
4396 </itemizedlist>
4397 </para>
4398
4399 <para>
4400 All test files reside in
4401 <filename>meta/lib/oeqa/runtime</filename> in the
4402 <link linkend='source-directory'>Source Directory</link>.
4403 Tests (also referred to as modules) have a one-to-one
4404 relationship with the filenames in <filename>runtime</filename>.
4405 Modules can have multiple classes and test methods and are
4406 usually grouped together by the area tested (e.g tests for
4407 <filename>systemd</filename> reside in
4408 <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
4409 </para>
4410
4411 <para>
4412 You can add tests to any layer provided you place them in the
4413 proper area and you extend
4414 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
4415 in the <filename>local.conf</filename> file.
4416 Be sure that tests reside in
4417 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename>.
4418 <note>
4419 Be sure that module names do not collide with module names
4420 provided by the Yocto Project.
4421 </note>
4422 </para>
4423
4424 <para>
4425 You can change the set of tests run by appending or overriding
4426 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>
4427 variable in <filename>local.conf</filename>.
4428 Each name in <filename>TEST_SUITES</filename> represents a
4429 required test for the image.
4430 Module skipping is not allowed even if a test is not suitable
4431 for an image (e.g. running the <filename>rpm</filename> tests on
4432 an image without <filename>rpm</filename>).
4433 Appending "auto" to <filename>TEST_SUITES</filename> causes the
4434 build system to try to run all tests that are suitable for the
4435 image (i.e. the build system decides whether to run each test).
4436 </para>
4437
4438 <para>
4439 The order you list tests in <filename>TEST_SUITES</filename>
4440 is important.
4441 The order influences test dependencies.
4442 Consequently, tests that depend on other tests should be added
4443 after the test on which they depend.
4444 For example, since <filename>ssh</filename> depends on the
4445 <filename>ping</filename> test, <filename>ssh</filename>
4446 needs to come after <filename>ping</filename> in the list.
4447 The test class provides no re-ordering or dependency handling.
4448 <note>
4449 Each module can have multiple classes with multiple test
4450 methods.
4451 And, Python <filename>unittest</filename> rules apply.
4452 </note>
4453 </para>
4454
4455 <para>
4456 The following list summarizes how to run the tests:
4457 <itemizedlist>
4458 <listitem><para>Run the default set of tests simply by
4459 running the following:
4460 <literallayout class='monospaced'>
4461 bitbake &lt;qemu_image&gt; -c testimage
4462 </literallayout></para></listitem>
4463 <listitem><para>The default tests for the image are defined
4464 as:
4465 <literallayout class='monospaced'>
4466 DEFAULT_TEST_SUITES_pn-&lt;qemu_image&gt; = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
4467 </literallayout></para></listitem>
4468 <listitem><para>Add your own test to the list of the
4469 by using the following:
4470 <literallayout class='monospaced'>
4471 TEST_SUITES_append = " mytest"
4472 </literallayout></para></listitem>
4473 <listitem><para>Run a specific list of tests as follows:
4474 <literallayout class='monospaced'>
4475 TEST_SUITES = "test1 test2 test3"
4476 </literallayout>
4477 Remember, order is important.
4478 Be sure to place a test that is dependent on another test
4479 later in the order.</para></listitem>
4480 </itemizedlist>
4481 </para>
4482 </section>
4483
4484 <section id="qemu-image-writing-new-tests">
4485 <title>Writing New Tests</title>
4486
4487 <para>
4488 As mentioned previously, all new test files need to be in the
4489 proper place for the build system to find them.
4490 New tests can go in the
4491 <link linkend='source-directory'>Source Directory</link> at
4492 <filename>meta/lib/oeqa/runtime</filename>.
4493 Alternatively, a layer can add its own tests in
4494 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename> as long
4495 as you extend <filename>BBPATH</filename>.
4496 Just remember that filenames need to map directory to test
4497 (module) names and you not use module names that collide with
4498 existing core tests.
4499 </para>
4500
4501 <para>
4502 To create a new test, start by copying an existing module
4503 (e.g. <filename>syslog.py</filename> or
4504 <filename>gcc.py</filename> are good ones to use).
4505 Test modules can use code from
4506 <filename>meta/lib/oeqa/utils</filename>, which are helper
4507 classes.
4508 </para>
4509
4510 <note>
4511 Structure shell commands such that you rely on them and they
4512 return a single code for success.
4513 Be aware that sometimes you will need to parse the output.
4514 See the <filename>df.py</filename> and
4515 <filename>date.py</filename> modules for examples.
4516 </note>
4517
4518 <para>
4519 You will notice that all test classes inherit
4520 <filename>oeRuntimeTest</filename>, which is found in
4521 <filename>meta/lib/oetest.py</filename>.
4522 This base class offers some helper attributes, which are
4523 described in the following sections:
4524 </para>
4525
4526 <section id='qemu-image-writing-tests-class-methods'>
4527 <title>Class Methods</title>
4528
4529 <para>
4530 Class methods are as follows:
4531 <itemizedlist>
4532 <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis>
4533 Returns "True" if <filename>pkg</filename> is in the
4534 installed package list of the image, which is based
4535 on
4536 <filename>${WORKDIR}/installed_pkgs.txt</filename>
4537 that is generated during the
4538 <filename>do.rootfs</filename> task.
4539 </para></listitem>
4540 <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis>
4541 Returns "True" if the feature is in
4542 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
4543 or
4544 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>.
4545 </para></listitem>
4546 <listitem><para><emphasis><filename>restartTarget(params)</filename>:</emphasis>
4547 Restarts the QEMU image optionally passing
4548 <filename>params</filename> to the
4549 <filename>runqemu</filename> script's
4550 <filename>qemuparams</filename> list (e.g "-m 1024" for
4551 more memory).</para></listitem>
4552 </itemizedlist>
4553 </para>
4554 </section>
4555
4556 <section id='qemu-image-writing-tests-class-attributes'>
4557 <title>Class Attributes</title>
4558
4559 <para>
4560 Class attributes are as follows:
4561 <itemizedlist>
4562 <listitem><para><emphasis><filename>pscmd</filename>:</emphasis>
4563 Equals "ps -ef" if <filename>procps</filename> is
4564 installed in the image.
4565 Otherwise, <filename>pscmd</filename> equals
4566 "ps" (busybox).
4567 </para></listitem>
4568 <listitem><para><emphasis><filename>tc</filename>:</emphasis>
4569 The called text context, which gives access to the
4570 following attributes:
4571 <itemizedlist>
4572 <listitem><para><emphasis><filename>d</filename>:</emphasis>
4573 The BitBake data store, which allows you to
4574 use stuff such as
4575 <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>.
4576 </para></listitem>
4577 <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis>
4578 Used internally.
4579 The tests do not need these.
4580 </para></listitem>
4581 <listitem><para><emphasis><filename>filesdir</filename>:</emphasis>
4582 The absolute path to
4583 <filename>meta/lib/oeqa/runtime/files</filename>,
4584 which contains helper files for tests meant
4585 for copying on the target such as small
4586 files written in C for compilation.
4587 </para></listitem>
4588 <listitem><para><emphasis><filename>qemu</filename>:</emphasis>
4589 Provides access to the
4590 <filename>QemuRunner</filename> object,
4591 which is the class that boots the image.
4592 The <filename>qemu</filename> attribute
4593 provides the following useful attributes:
4594 <itemizedlist>
4595 <listitem><para><emphasis><filename>ip</filename>:</emphasis>
4596 The machine's IP address.
4597 </para></listitem>
4598 <listitem><para><emphasis><filename>host_ip</filename>:</emphasis>
4599 The host IP address, which is only
4600 used by smart tests.
4601 </para></listitem>
4602 </itemizedlist></para></listitem>
4603 <listitem><para><emphasis><filename>target</filename>:</emphasis>
4604 The <filename>SSHControl</filename> object,
4605 which is used for running the following
4606 commands on the image:
4607 <itemizedlist>
4608 <listitem><para><emphasis><filename>host</filename>:</emphasis>
4609 Used internally.
4610 The tests do not use this command.
4611 </para></listitem>
4612 <listitem><para><emphasis><filename>timeout</filename>:</emphasis>
4613 A global timeout for commands run on
4614 the target for the instance of a
4615 test.
4616 The default is 300 seconds.
4617 </para></listitem>
4618 <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis>
4619 The single, most used method.
4620 This command is a wrapper for:
4621 <filename>ssh root@host "cmd"</filename>.
4622 The command returns a tuple:
4623 (status, output), which are what
4624 their names imply - the return code
4625 of 'cmd' and whatever output
4626 it produces.
4627 The optional timeout argument
4628 represents the number of seconds the
4629 test should wait for 'cmd' to
4630 return.
4631 If the argument is "None", the
4632 test uses the default instance's
4633 timeout period, which is 300
4634 seconds.
4635 If the argument is "0", the test
4636 runs until the command returns.
4637 </para></listitem>
4638 <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis>
4639 <filename>scp localpath root@ip:remotepath</filename>.
4640 </para></listitem>
4641 <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis>
4642 <filename>scp root@host:remotepath localpath</filename>.
4643 </para></listitem>
4644 </itemizedlist></para></listitem>
4645 </itemizedlist></para></listitem>
4646 </itemizedlist>
4647 </para>
4648 </section>
4649
4650 <section id='qemu-image-writing-tests-instance-attributes'>
4651 <title>Instance Attributes</title>
4652
4653 <para>
4654 A single instance attribute exists, which is
4655 <filename>target</filename>.
4656 The <filename>target</filename> instance attribute is
4657 identical to the class attribute of the same name, which
4658 is described in the previous section.
4659 This attribute exists as both an instance and class
4660 attribute so tests can use
4661 <filename>self.target.run(cmd)</filename> in instance
4662 methods instead of
4663 <filename>oeRuntimeTest.tc.target.run(cmd)</filename>.
4664 </para>
4665 </section>
4666 </section>
4667 </section>
4668
4255 <section id="platdev-gdb-remotedebug"> 4669 <section id="platdev-gdb-remotedebug">
4256 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title> 4670 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title>
4257 4671
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-10-28 11:03:26 +0000