diff options
| -rw-r--r-- | documentation/test-manual/test-manual-intro.xml | 79 | ||||
| -rw-r--r-- | documentation/test-manual/test-manual-understand-autobuilder.xml | 21 |
2 files changed, 45 insertions, 55 deletions
diff --git a/documentation/test-manual/test-manual-intro.xml b/documentation/test-manual/test-manual-intro.xml index 9464d332c2..8e2c7cd874 100644 --- a/documentation/test-manual/test-manual-intro.xml +++ b/documentation/test-manual/test-manual-intro.xml | |||
| @@ -214,52 +214,41 @@ | |||
| 214 | Tests map into the codebase as follows: | 214 | Tests map into the codebase as follows: |
| 215 | <itemizedlist> | 215 | <itemizedlist> |
| 216 | <listitem><para> | 216 | <listitem><para> |
| 217 | <emphasis>bitbake-selftest</emphasis>: <itemizedlist> | 217 | <emphasis>bitbake-selftest</emphasis>: </para> |
| 218 | <listitem> | 218 | <para>These tests are self-contained and test BitBake as well as its APIs, which |
| 219 | <para>These tests are self-contained and test BitBake as well as its | 219 | include the fetchers. The tests are located in |
| 220 | APIs, which include the fetchers. The tests are located in | 220 | <filename>bitbake/lib/*/tests</filename>. </para> |
| 221 | <filename>bitbake/lib/*/tests</filename>. </para> | 221 | <para>From within the BitBake repository, run the following: |
| 222 | </listitem> | 222 | <literallayout class="monospaced"> |
| 223 | <listitem> | ||
| 224 | <para>From within the BitBake repository, run the following: | ||
| 225 | <literallayout class="monospaced"> | ||
| 226 | $ bitbake-selftest | 223 | $ bitbake-selftest |
| 227 | </literallayout> | 224 | </literallayout> |
| 228 | </para> | 225 | </para> |
| 229 | </listitem> | 226 | <para>To skip tests that access the Internet, use the |
| 230 | <listitem> | 227 | <filename>BB_SKIP_NETTEST</filename> variable when running |
| 231 | <para>To skip tests that access the Internet, use the | 228 | "bitbake-selftest" as follows: |
| 232 | <filename>BB_SKIP_NETTEST</filename> variable when running | 229 | <literallayout class="monospaced"> |
| 233 | "bitbake-selftest" as follows: | ||
| 234 | <literallayout class="monospaced"> | ||
| 235 | $ BB_SKIP_NETTEST=yes bitbake-selftest | 230 | $ BB_SKIP_NETTEST=yes bitbake-selftest |
| 236 | </literallayout>The | 231 | </literallayout></para> |
| 237 | default output is quiet and just prints a summary of what was | 232 | <para>The default output is quiet and just prints a summary of what was run. To |
| 238 | run. To see more information, there is a verbose | 233 | see more information, there is a verbose |
| 239 | option:<literallayout> | 234 | option:<literallayout class="monospaced"> |
| 240 | $ bitbake-selftest -v | 235 | $ bitbake-selftest -v |
| 241 | </literallayout></para> | 236 | </literallayout></para> |
| 242 | <para>Use this option when you wish to skip tests that access the | 237 | <para>Use this option when you wish to skip tests that access the network, which |
| 243 | network, which are mostly necessary to test the fetcher modules. | 238 | are mostly necessary to test the fetcher modules. To specify individual test |
| 244 | To specify individual test modules to run, append the test | 239 | modules to run, append the test module name to the "bitbake-selftest" |
| 245 | module name to the "bitbake-selftest" command. For example, to | 240 | command. For example, to specify the tests for the bb.data.module, run: |
| 246 | specify the tests for the bb.data.module, run: | 241 | <literallayout class="monospaced"> |
| 247 | <literallayout class="monospaced"> | ||
| 248 | $ bitbake-selftest bb.test.data.module | 242 | $ bitbake-selftest bb.test.data.module |
| 249 | </literallayout>You | 243 | </literallayout>You |
| 250 | can also specify individual tests by defining the full name and | 244 | can also specify individual tests by defining the full name and module plus |
| 251 | module plus the class path of the test, for example: | 245 | the class path of the test, for example: |
| 252 | <literallayout> | 246 | <literallayout class="monospaced"> |
| 253 | $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override | 247 | $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override |
| 254 | </literallayout></para> | 248 | </literallayout></para> |
| 255 | </listitem> | 249 | <para>The tests are based on <ulink |
| 256 | <listitem> | 250 | url="https://docs.python.org/3/library/unittest.html">Python |
| 257 | <para>The tests are based on <ulink | 251 | unittest</ulink>. </para></listitem> |
| 258 | url="https://docs.python.org/3/library/unittest.html">Python | ||
| 259 | unittest</ulink>. </para> | ||
| 260 | </listitem> | ||
| 261 | </itemizedlist> | ||
| 262 | </para></listitem> | ||
| 263 | <listitem><para> | 252 | <listitem><para> |
| 264 | <emphasis>oe-selftest</emphasis>: <itemizedlist> | 253 | <emphasis>oe-selftest</emphasis>: <itemizedlist> |
| 265 | <listitem> | 254 | <listitem> |
| @@ -297,12 +286,12 @@ | |||
| 297 | </literallayout> | 286 | </literallayout> |
| 298 | For example, the following command would run the tinfoil getVar | 287 | For example, the following command would run the tinfoil getVar |
| 299 | API | 288 | API |
| 300 | test:<literallayout> | 289 | test:<literallayout class="monospaced"> |
| 301 | $ oe-selftest -r tinfoil.TinfoilTests.test_getvar | 290 | $ oe-selftest -r tinfoil.TinfoilTests.test_getvar |
| 302 | </literallayout>It | 291 | </literallayout>It |
| 303 | is also possible to run a set of tests. For example the | 292 | is also possible to run a set of tests. For example the |
| 304 | following command will run all of the tinfoil | 293 | following command will run all of the tinfoil |
| 305 | tests:<literallayout> | 294 | tests:<literallayout class="monospaced"> |
| 306 | $ oe-selftest -r tinfoil | 295 | $ oe-selftest -r tinfoil |
| 307 | </literallayout></para> | 296 | </literallayout></para> |
| 308 | </listitem> | 297 | </listitem> |
| @@ -408,7 +397,7 @@ | |||
| 408 | <title><filename>bitbake-selftest</filename></title> | 397 | <title><filename>bitbake-selftest</filename></title> |
| 409 | 398 | ||
| 410 | <para>A simple test example from <filename>lib/bb/tests/data.py</filename> is: | 399 | <para>A simple test example from <filename>lib/bb/tests/data.py</filename> is: |
| 411 | <literallayout> | 400 | <literallayout class="monospaced"> |
| 412 | class DataExpansions(unittest.TestCase): | 401 | class DataExpansions(unittest.TestCase): |
| 413 | def setUp(self): | 402 | def setUp(self): |
| 414 | self.d = bb.data.init() | 403 | self.d = bb.data.init() |
| @@ -438,7 +427,7 @@ | |||
| 438 | builds. Rather than directly using Python's unittest, the code wraps most of the | 427 | builds. Rather than directly using Python's unittest, the code wraps most of the |
| 439 | standard objects. The tests can be simple, such as testing a command from within the | 428 | standard objects. The tests can be simple, such as testing a command from within the |
| 440 | OE build environment using the following | 429 | OE build environment using the following |
| 441 | example:<literallayout> | 430 | example:<literallayout class="monospaced"> |
| 442 | class BitbakeLayers(OESelftestTestCase): | 431 | class BitbakeLayers(OESelftestTestCase): |
| 443 | def test_bitbakelayers_showcrossdepends(self): | 432 | def test_bitbakelayers_showcrossdepends(self): |
| 444 | result = runCmd('bitbake-layers show-cross-depends') | 433 | result = runCmd('bitbake-layers show-cross-depends') |
| @@ -487,7 +476,7 @@ | |||
| 487 | under QEMU. As a result, they are assumed to be running in a target image | 476 | under QEMU. As a result, they are assumed to be running in a target image |
| 488 | environment, as opposed to a host build environment. A simple example from | 477 | environment, as opposed to a host build environment. A simple example from |
| 489 | <filename>meta/lib/oeqa/runtime/cases/python.py</filename> contains the | 478 | <filename>meta/lib/oeqa/runtime/cases/python.py</filename> contains the |
| 490 | following:<literallayout> | 479 | following:<literallayout class="monospaced"> |
| 491 | class PythonTest(OERuntimeTestCase): | 480 | class PythonTest(OERuntimeTestCase): |
| 492 | @OETestDepends(['ssh.SSHTest.test_ssh']) | 481 | @OETestDepends(['ssh.SSHTest.test_ssh']) |
| 493 | @OEHasPackage(['python3-core']) | 482 | @OEHasPackage(['python3-core']) |
| @@ -514,7 +503,7 @@ | |||
| 514 | <para>These tests are run against built extensible SDKs (eSDKs). The tests can assume | 503 | <para>These tests are run against built extensible SDKs (eSDKs). The tests can assume |
| 515 | that the eSDK environment has already been setup. An example from | 504 | that the eSDK environment has already been setup. An example from |
| 516 | <filename>meta/lib/oeqa/sdk/cases/devtool.py</filename> contains the | 505 | <filename>meta/lib/oeqa/sdk/cases/devtool.py</filename> contains the |
| 517 | following:<literallayout> | 506 | following:<literallayout class="monospaced"> |
| 518 | class DevtoolTest(OESDKExtTestCase): | 507 | class DevtoolTest(OESDKExtTestCase): |
| 519 | @classmethod | 508 | @classmethod |
| 520 | def setUpClass(cls): | 509 | def setUpClass(cls): |
| @@ -548,7 +537,7 @@ | |||
| 548 | <para>These tests are run against built SDKs. The tests can assume that an SDK has | 537 | <para>These tests are run against built SDKs. The tests can assume that an SDK has |
| 549 | already been extracted and its environment file has been sourced. A simple example | 538 | already been extracted and its environment file has been sourced. A simple example |
| 550 | from <filename>meta/lib/oeqa/sdk/cases/python2.py</filename> contains the | 539 | from <filename>meta/lib/oeqa/sdk/cases/python2.py</filename> contains the |
| 551 | following:<literallayout> | 540 | following:<literallayout class="monospaced"> |
| 552 | class Python3Test(OESDKTestCase): | 541 | class Python3Test(OESDKTestCase): |
| 553 | def setUp(self): | 542 | def setUp(self): |
| 554 | if not (self.tc.hasHostPackage("nativesdk-python3-core") or | 543 | if not (self.tc.hasHostPackage("nativesdk-python3-core") or |
| @@ -571,7 +560,7 @@ | |||
| 571 | <para>The performance tests usually measure how long operations take and the resource | 560 | <para>The performance tests usually measure how long operations take and the resource |
| 572 | utilisation as that happens. An example from | 561 | utilisation as that happens. An example from |
| 573 | <filename>meta/lib/oeqa/buildperf/test_basic.py</filename> contains the | 562 | <filename>meta/lib/oeqa/buildperf/test_basic.py</filename> contains the |
| 574 | following:<literallayout> | 563 | following:<literallayout class="monospaced"> |
| 575 | class Test3(BuildPerfTestCase): | 564 | class Test3(BuildPerfTestCase): |
| 576 | 565 | ||
| 577 | def test3(self): | 566 | def test3(self): |
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.xml b/documentation/test-manual/test-manual-understand-autobuilder.xml index c96713c1dc..a04006605f 100644 --- a/documentation/test-manual/test-manual-understand-autobuilder.xml +++ b/documentation/test-manual/test-manual-understand-autobuilder.xml | |||
| @@ -20,7 +20,7 @@ | |||
| 20 | <filename>config.json</filename> file within the | 20 | <filename>config.json</filename> file within the |
| 21 | <filename>yocto-autobuilder-helper</filename> repository. The targets are defined in | 21 | <filename>yocto-autobuilder-helper</filename> repository. The targets are defined in |
| 22 | the ‘overrides’ section, a quick example could be qemux86-64 which looks | 22 | the ‘overrides’ section, a quick example could be qemux86-64 which looks |
| 23 | like:<literallayout> | 23 | like:<literallayout class="monospaced"> |
| 24 | "qemux86-64" : { | 24 | "qemux86-64" : { |
| 25 | "MACHINE" : "qemux86-64", | 25 | "MACHINE" : "qemux86-64", |
| 26 | "TEMPLATE" : "arch-qemu", | 26 | "TEMPLATE" : "arch-qemu", |
| @@ -32,7 +32,7 @@ | |||
| 32 | }, | 32 | }, |
| 33 | </literallayout>And | 33 | </literallayout>And |
| 34 | to expand that, you need the “arch-qemu” entry from the “templates” section, which looks | 34 | to expand that, you need the “arch-qemu” entry from the “templates” section, which looks |
| 35 | like:<literallayout> | 35 | like:<literallayout class="monospaced"> |
| 36 | "arch-qemu" : { | 36 | "arch-qemu" : { |
| 37 | "BUILDINFO" : true, | 37 | "BUILDINFO" : true, |
| 38 | "BUILDHISTORY" : true, | 38 | "BUILDHISTORY" : true, |
| @@ -52,11 +52,12 @@ | |||
| 52 | } | 52 | } |
| 53 | }, | 53 | }, |
| 54 | </literallayout>Combining | 54 | </literallayout>Combining |
| 55 | these two entries you can see that “qemux86-64” is a three step build where the “bitbake | 55 | these two entries you can see that “qemux86-64” is a three step build where the |
| 56 | BBTARGETS” would be run, then “bitbake SANITYTARGETS” for each step; all for | 56 | <filename>bitbake BBTARGETS</filename> would be run, then <filename>bitbake |
| 57 | MACHINE=”qemx86-64” but with differing SDKMACHINE settings. In step 1 an extra variable | 57 | SANITYTARGETS</filename> for each step; all for |
| 58 | is added to the <filename>auto.conf</filename> file to enable wic image | 58 | <filename>MACHINE=”qemx86-64”</filename> but with differing SDKMACHINE settings. In |
| 59 | generation.</para> | 59 | step 1 an extra variable is added to the <filename>auto.conf</filename> file to enable |
| 60 | wic image generation.</para> | ||
| 60 | <para>While not every detail of this is covered here, you can see how the templating | 61 | <para>While not every detail of this is covered here, you can see how the templating |
| 61 | mechanism allows quite complex configurations to be built up yet allows duplication and | 62 | mechanism allows quite complex configurations to be built up yet allows duplication and |
| 62 | repetition to be kept to a minimum.</para> | 63 | repetition to be kept to a minimum.</para> |
| @@ -284,18 +285,18 @@ | |||
| 284 | substitution of the paths. The Helper script repository includes a | 285 | substitution of the paths. The Helper script repository includes a |
| 285 | <filename>local-example.json</filename> file to show how you could override these | 286 | <filename>local-example.json</filename> file to show how you could override these |
| 286 | from a separate configuration file. Pass the following into the environment of the | 287 | from a separate configuration file. Pass the following into the environment of the |
| 287 | Autobuilder:<literallayout> | 288 | Autobuilder:<literallayout class="monospaced"> |
| 288 | $ ABHELPER_JSON="config.json local-example.json" | 289 | $ ABHELPER_JSON="config.json local-example.json" |
| 289 | </literallayout>As | 290 | </literallayout>As |
| 290 | another example, you could also pass the following into the | 291 | another example, you could also pass the following into the |
| 291 | environment:<literallayout> | 292 | environment:<literallayout class="monospaced"> |
| 292 | $ ABHELPER_JSON="config.json <replaceable>/some/location/</replaceable>local.json" | 293 | $ ABHELPER_JSON="config.json <replaceable>/some/location/</replaceable>local.json" |
| 293 | </literallayout>One | 294 | </literallayout>One |
| 294 | issue users often run into is validation of the <filename>config.json</filename> files. | 295 | issue users often run into is validation of the <filename>config.json</filename> files. |
| 295 | A tip for minimizing issues from invalid json files is to use a Git | 296 | A tip for minimizing issues from invalid json files is to use a Git |
| 296 | <filename>pre-commit-hook.sh</filename> script to verify the JSON file before | 297 | <filename>pre-commit-hook.sh</filename> script to verify the JSON file before |
| 297 | committing it. Create a symbolic link as | 298 | committing it. Create a symbolic link as |
| 298 | follows:<literallayout> | 299 | follows:<literallayout class="monospaced"> |
| 299 | $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit | 300 | $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit |
| 300 | </literallayout></para> | 301 | </literallayout></para> |
| 301 | </section> | 302 | </section> |
