summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
Commit message (Collapse)AuthorAgeFilesLines
...
* oeqa: adding selftest-hello and use it to speed up testsThomas Roos2023-05-221-9/+9
| | | | | | | | | | | | | | Adding a selftest version of hello world to run it in tests where no download is necessary. Also using this in several tests to speed them up. Using the -native version wherever possible will also speed up tests a lot. [YOCTO #11142] (From OE-Core rev: c3f26b63934888df0e3cd563c1c2804eb78a368e) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: avoid leaking server_socketMikko Rapeli2023-05-221-0/+1
| | | | | | | | | | | close() the server_socket before qemusock replaces it. (From OE-Core rev: be7bbbeeb44158b181d1abc5a4d7871a9045fdd6) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: support serial console login via qemu stdoutMikko Rapeli2023-05-221-11/+19
| | | | | | | | | | | | | | | | runqemu script works with qemu machines which provide login and serial console to the qemu process stdout. Add the same support to qemurunner so that testing with testimage.bbclass is possible. Default qemu machines provide serial console boot logs and login via socket to qemu process but I don't see a reason why qemu process stdout should not be supported too since they work with runqemu as well. (From OE-Core rev: a3d4b80c5b4cce933c759d023c75b8671c56fe12) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk/assimp: run only when zlib is in the SDKAlexander Kanavin2023-05-051-0/+3
| | | | | | | | | | | | | | Otherwise assimp will silently fall back to a vendored copy of zlib which will fail with -D_TIME_BITS=64 due to https://github.com/madler/zlib/pull/764 This was exposed by multilib mips core-image-minimal SDKs, where the default 64 bit sysroot has zlib, but 32 bit sysroot does not. (From OE-Core rev: c0fb603c9e26e91388320c02842b42cc7b091d6c) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* zvariant: add ptest feature for zvariant test suiteFrederic Martinsons2023-05-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Below is the output of run-ptest script under qemu root@qemux86-64:~# /usr/lib/zvariant/ptest/run-ptest running 37 tests test framing_offset_size::tests::framing_offset_size_bump ... ok$<2> test owned_value::tests::from_value ... ok$<2> test object_path::unit::owned_from_reader ... ok$<2> test str::tests::from_string ... ok$<2> test signature::tests::signature_slicing ... ok$<2> test str::tests::test_ordering ... ok$<2> test owned_value::tests::map_conversion ... ok$<2> test owned_value::tests::serde ... ok$<2> test tests::enums ... ok$<2> test tests::derive ... ok$<2> test tests::f64_value ... ok$<2> test tests::i16_value ... ok$<2> test tests::fd_value ... ok$<2> test tests::i32_value ... ok$<2> test tests::i8_value ... ok$<2> test tests::i64_value ... ok$<2> test tests::ip_addr ... ok$<2> test tests::issue_59 ... ok$<2> test tests::issue_99 ... ok$<2> test tests::array_value ... ok$<2> test tests::object_path_value ... ok$<2> test tests::dict_value ... ok$<2> test tests::signature_value ... ok$<2> test tests::serialized_size ... ok$<2> test tests::struct_byte_array ... ok$<2> test tests::struct_ref ... ok$<2> test tests::str_value ... ok$<2> test tests::option_value ... ok$<2> test tests::struct_value ... ok$<2> test tests::struct_with_hashmap ... ok$<2> test tests::u16_value ... ok$<2> test tests::u32_value ... ok$<2> test tests::unit ... ok$<2> test tests::u8_value ... ok$<2> test tests::unit_fds ... ok$<2> test tests::value_value ... ok$<2> test tests::recursion_limits ... ok$<2> test result: ok$<2>. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s root@qemux86-64:~# (From OE-Core rev: 912bbec9fe44f22ab70c3553af6cb699543b8411) Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/distrodata: clean up exception lists in recipe maintainers testAlexander Kanavin2023-05-051-3/+3
| | | | | | | | | | | | | | | | | Specifically: - add missing maintainer.inc entries for initramfs-module-*, systemd-machine-units and target-sdk-provides-dummy and drop them from exception list. - remove rust from exception list for unbuildable-by-default recipes as it is now buildable. - add missing maintainer.inc entry for libx11-compose-data and cve-update-nvd2-native; as they are also unbuildable by default, they needs to be in exception list as well. (From OE-Core rev: e9158b191c1cfc16f97abed6c05891aa84fe9463) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes: Default to https git protocol where possibleRichard Purdie2023-05-052-4/+6
| | | | | | | | | | The recommendation from server maintainers is that the https protocol is both faster and more reliable than the dedicated git protocol at this point. Switch to it where possible. (From OE-Core rev: 139102a73d4151f4748b4a861bd4ab28dda7dab7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: Make returning no test results a failureRichard Purdie2023-05-032-3/+9
| | | | | | | | | | | | | Ensure that even if a ptests results section is empty, the log parser adds that empty section. Then ensure that empty sections trigger warnings. This means if a ptest suddently stops returning any results, we notice and see warnings about it. This has gone unnoticed on the autobuilder far too many times so is very much worth highlighting as a regression. We shouldn't have empty ptests. (From OE-Core rev: 5ad0cf57b41ec7f44647a03bc568d0b24906cc8d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/metadata.py: Fix running oe-selftest running with no distro setThomas Roos2023-04-201-3/+3
| | | | | | | | | | | | This will use default values when no distribution is set. [YOCTO #15086] (From OE-Core rev: 888fe63b46efceeff08dbe8c4f66fec33d06cb7a) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Use SSTATE_DIR of parent build dirThomas Roos2023-04-201-0/+10
| | | | | | | | | | | This will configure the build-st/conf/local.conf with the SSTATE_DIR of the parent build dir to speed up the build, test, dev process. (From OE-Core rev: 71dca60ddcf4370ff7e91de6ddf8a7d4cf76db50) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: Add test for --part-name argumentDit Kozmaj2023-04-131-0/+23
| | | | | | | | | | | | | | | | | | | Add test for wic --part-name argument in .wks file. Test three different cases: - Set only the --part-name argument. - Set only the --label argument. In this case the GPT partition label is set to the value of --label for compatibility reasons. - Set both. In this case the code has been changed to set the GPT partition label to the --part-name value. The test uses a test_gpt_partition_name.wks file created for this purpose. (From OE-Core rev: 06af6795cf6f079362303bddb39a14459e228e72) Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa ping.py: fail test if target IP address has not been setMikko Rapeli2023-04-061-0/+1
| | | | | | | | | | | It is possible to call exported tests with --target-ip set to ":22" where IP address is not set at all. Detect this case and fail the test instead of calling ping without an IP address. (From OE-Core rev: 17c995c53775b8cee279ca4ced916092067e1195) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa ping.py: avoid busylooping failing ping commandMikko Rapeli2023-04-061-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use a sleep on error path before trying again. For example when oeqa runtime tests are executed without setting target IP address correctly, the tests are drowning logs with messages: 2023-04-04 07:19:24,985 - runtime - INFO - test_ping (ping.PingTest.test_ping) ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ... 2023-04-04 07:19:55,002 - runtime - INFO - ... FAIL 2023-04-04 07:19:55,002 - runtime - INFO - Traceback (most recent call last): File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/runtime/cases/ping.py\", line 23, in test_ping output += proc.communicate()[0].decode('utf-8') ^^^^^^^^^^^^^^^^^^ File \"/usr/lib/python3.11/subprocess.py\", line 1194, in communicate stdout = self.stdout.read() ^^^^^^^^^^^^^^^^^^ File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/core/decorator/oetimeout.py\", line 18, in _timeoutHandler raise OEQATimeoutError(\"Timed out after %s \" oeqa.core.exception.OEQATimeoutError: Timed out after 30 seconds of execution (From OE-Core rev: 87ec75710b5cd7b3f35d886003844d62d3182b54) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: efibootpartition.py: fix QEMU_USE_KVM usageMartin Jansa2023-04-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | * if you don't have QEMU_USE_KVM in local.conf it fails with: 2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE 2023-03-12 18:06:29,594 - oe-selftest - INFO - ... ERROR 2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, in test_boot_efi if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]): ~~~~~~~^^^^^^^^^^^^^^^^ KeyError: 'QEMU_USE_KVM' [YOCTO #12937] (From OE-Core rev: 7c32ca2f91beb98769c89470b37f06cb4b99aebb) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/bblogging: uncomment python stdout checksMark Asselstine2023-04-051-12/+6
| | | | | | | | | | | | Since bitbake commit 81a58647b2f4fc0a2589b2978fc9d81b2bfe6aec [bitbake: build: Make python output print to stdout when running with -v (verbose)] we no longer need to comment out the python stdout checks. (From OE-Core rev: 67886a8473c511c8ab3db2e4587cc5a070979d11) Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: clean up deprecated backslash expansionRoss Burton2023-04-033-5/+8
| | | | | | | (From OE-Core rev: 77085a05240c3f8226b9f2199c977f2555807789) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: runqemu: use better error message when asserts failMartin Jansa2023-04-031-3/+6
| | | | | | | | | | | | * It was showing whole log and that the runqemu command failed, but not where the log file is, nor why it thinks the runqemu failed [YOCTO #12937] (From OE-Core rev: ad4b72e6524114a3bdee10cab71f194ea143cd24) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: runqemu: better check for ROOTFS: in the logMartin Jansa2023-04-031-1/+2
| | | | | | | | | | | | | | | | | | * it was searching for line like this: ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4] but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs" or with my pending changes the line looks like this: ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4] and test was failing. * Check for whole line starting with ROOTFS: and ending just with .ext4 [YOCTO #12937] (From OE-Core rev: 7a0f622f23aff2c4eeca0606e7682931eb53287a) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: imagefeatures.py: don't mix tabs and spaces for indentationMartin Jansa2023-04-031-4/+4
| | | | | | | | | | * introduced in: https://git.openembedded.org/openembedded-core/commit/?id=96d4486df6d870ef19e2055b026729e66bc118f3 (From OE-Core rev: 143f624116802579e7a6804c8d4b562f931880c8) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta-selftest: provide a recipe for zvariantFrederic Martinsons2023-04-011-0/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This recipe is for showing a "real world" example of a crate that depends on some git repositories. Usually, this kind of crate is built within a global workspace (here it is the zbus project) and so doesn't need a Cargo.lock on its own. For the sake of the demonstration, I had to tweak things a little to be able to compile zvariant in standalone (no relative path in dependency, no symlink to LICENSE provide a Cargo.lock) The use case where the crate had some git repository in dependency is very common for "private" crate that are not aimed to be published on crates.io. When the project grow bigger, it is common to have a bin and multiple lib developped in parallel, and these libs are surely on a git repostitory. A test case have been also added to check for: - the previous patch about git subpath parameter and devtool - the correctness of overriding dependencies (first patch of the series) (From OE-Core rev: 409e045f96f69877de6f36ed14c5c19a9cb74eaf) Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: whitespace and indentation cleanupsEnrico Jörns2023-03-314-16/+16
| | | | | | | (From OE-Core rev: 4922221d1259e2f78233f17bb901cdac5b9aa520) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: fix undefined TimeoutExpiredEnrico Jörns2023-03-311-1/+1
| | | | | | | (From OE-Core rev: 19ae3f4440b09d4cfe4bc589d859d476168541bd) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: limit precision of timing debugging outputEnrico Jörns2023-03-311-7/+7
| | | | | | | | | | | | | * There is no need to be that precise. It just irritates in the logs. * There is also no point in printing plain time.time() value a single time while only using formatted printout everywhere else, thus remove it. * Use %d for printing integer times (From OE-Core rev: f0988cb8cf8d03708490cca4eba345492ef78d52) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: replace hard-coded user 'root' in debug outputEnrico Jörns2023-03-311-2/+2
| | | | | | | | | | | | Since login user is configurable with boot_patterns['send_login_user'], unconditionally using 'root' in the debug message can be confusing. Also fix the debug message to say 'Logged in' instead of 'Logged'. (From OE-Core rev: 900e3d42b918b5a33d8d952b3a8078fbe72ba98f) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: remove unused importsEnrico Jörns2023-03-311-3/+0
| | | | | | | (From OE-Core rev: 0c00b5cdd57c3d9c47d4780d6627c74221911b72) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: fix usage of undefined EPIPEEnrico Jörns2023-03-311-1/+2
| | | | | | | (From OE-Core rev: 1cb17e37e275794e26debed2d171c2394d8bb945) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetcontrol: remove unused importsEnrico Jörns2023-03-311-4/+0
| | | | | | | (From OE-Core rev: 46b42edbf814d15be3a18283018ede2d4f817bc6) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetcontrol: do not set dump_host_cmds redundantlyEnrico Jörns2023-03-311-1/+1
| | | | | | | | | This is already set above, thus simply use the set value from there. (From OE-Core rev: 2f6571ffb23b1a0b5bd6a1212fc108e536f04539) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetcontrol: fix misspelled RuntimeErrorEnrico Jörns2023-03-311-1/+1
| | | | | | | (From OE-Core rev: 92ffc35052768c753a89b4839c70db87072437a2) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/cases/runqemu: update importsEnrico Jörns2023-03-311-2/+2
| | | | | | | | | | 'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is actually used. (From OE-Core rev: 50f766233e83528eebeca92877277374c7193530) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/loader: Ensure module names don't contain uppercase charactersRichard Purdie2023-03-311-0/+3
| | | | | | | | | | | Python modules aren't supposed to have uppercase characters in their names according to python conventions. We have regexs in the code which work on that assumption too. Rather than showing errors under some filtering situations, make it clear and error if a problematic name is seen. (From OE-Core rev: 7964a6ee54e881a6128707a7f2a42eec2ed63881) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/imagefeatures: set a test for mutliubi in test_image_fstypesRomuald Jeanne2023-03-301-5/+20
| | | | | | | | | | | | | Allow 'multiubi' fstype image. Set 'MULTIUBI_BUILD' with two entries to allow configuration for two different '.ubifs'. Configure 'MKUBIFS_ARGS_*' and 'UBINIZE_ARGS_*' for both configurations. (From OE-Core rev: 96d4486df6d870ef19e2055b026729e66bc118f3) Signed-off-by: Romuald Jeanne <romuald.jeanne@st.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: eSDK rename to esdkMartin Jansa2023-03-301-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * I was hit by oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers running all tests except only this selected one: poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK 2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers'] 2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build .. 2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name) this is caused by _built_modules_dict(modules) function which filters out eSDK.oeSDKExtSelfTest.test_install_libraries_headers based on the regexp and then it runs all loaded tests, because modules are empty the initial regexp and comment from 2017: https://git.openembedded.org/openembedded-core/commit/?id=80db3d999ae26d298d9d5418a32b11a4f27af9d5 # Assumption: package and module names do not contain upper case # characters, whereas class names do m = re.match(r'^([^A-Z]+)(?:\.([A-Z][^.]*)(?:\.([^.]+))?)?$', module) might still be valid, but it was loosened in 2018 to accept upper case in module: https://git.openembedded.org/openembedded-core/commit/?id=1ecf48fd286a77078451b67879a44f9c9dc7a894 Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all. Then skipping the not matching modules was added later in 2018: https://git.openembedded.org/openembedded-core/commit/?id=f2042bf3638ed4edfb167e7f7d4be6da60997ead and regexp was updated again in 2020 not to accept upper case in modules: https://git.openembedded.org/openembedded-core/commit/?id=ad81ea90a815389e45ff302a85151724c71f71c3 oeqa/core/loader: refine regex to find module test case in format <module name>.<class name>.<test case name> this is clear when test cases is only 3 item deep. but confused when it is 4 item deep, eg, oelib.types.TestList.test_list_nosep I'm afraid that changing this regexp again to accept eSDK will break someone's favorite test case, renaming eSDK looks much safer option There is only 1 such case in poky: $ oe-selftest --list-modules | grep INFO.- | sed 's/^.*INFO - //g' | grep -v '^[a-z_\.]*$' Listing all available test modules: eSDK Most modules are just a-z (52x), then oelib.<foo> (6x) and 7 modules with underscore '_'. (From OE-Core rev: 173ef4397b5be19f72385b1a0d892a1fa8979d53) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/cases/package.py: adding unittest for package rename conflictsFawzi KHABER2023-03-301-0/+7
| | | | | | | | | | | This Unittest tries to rename a package, using an already used name and fails on do_package. Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr> (From OE-Core rev: c3fe173d5196506d89aa464ba56aabcf581a60db) Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Fix false negative version issueGeoffrey GIRY2023-03-301-0/+19
| | | | | | | | | | | | | | | NVD DB store version and update in the same value, separated by '_'. The proposed patch check if the version from NVD DB contains a "_", ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison. [YOCTO #14127] Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr> (From OE-Core rev: 7d00f6ec578084a0a0e5caf36241d53036d996c4) Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducible: Split different packages from missing packages ↵Jose Quaresma2023-03-281-3/+7
| | | | | | | | | | | | | output When we provide sstate_targets recipes they are reported as missing which is not quite correct because they are proveided to cut build/debugging time. (From OE-Core rev: fd70a564d6946fa460638dd04ce2daecf4566cf3) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: OESelftestTestContext: convert relative to full path when ↵Jose Quaresma2023-03-281-0/+10
| | | | | | | | | | | | | | | | | | | | | newbuilddir is provided Relative paths in BBLAYERS only works when the new build dir are on the same ascending directory node: . ├── build ├── build-st It works because they share the same ascending relative directory node. So use the full path when the argument newbuilddir is provided to make the oe-selftest work everywere regardless of the location chosen. (From OE-Core rev: 2e022c1977bc1006c00a87e08a2dca5b69db4801) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqs/selftest: OESelftestTestContext: replace the os.environ after ↵Jose Quaresma2023-03-281-5/+5
| | | | | | | | | | | | subprocess.check_output No function change but is needed to simplify the next patch. (From OE-Core rev: 193d8f0d6383f54c61f90d58eeb8f41580c2360a) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: loader.py: show warning when skipping selected module and abort if all ↵Martin Jansa2023-03-222-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | are skipped * skipped modules were triggering an ERROR before: poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers imagefeatures.ImageFeatures.test_image_gen_debugfs -K -B /OE/build/poky/build-eSDK 2023-03-13 15:07:53,430 - oe-selftest - ERROR - Not found eSDK.oeSDKExtSelfTest.test_install_libraries_headers in loaded test cases * but didn't show the reason why it wasn't loaded and more importantly -r was ignored when all selected modules were silently skipped * add a warning when skipping some module and abort if some modules were selected, but all ended being skipped: poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK 2023-03-13 15:11:51,028 - oe-selftest - WARNING - module 'eSDK.oeSDKExtSelfTest.test_install_libraries_headers' was skipped from selected modules, because it doesn't match with module name assumptions: package and module names do not contain upper case characters, whereas class names do 2023-03-13 15:11:51,028 - oe-selftest - ERROR - All selected modules were skipped, this would trigger selftest with all tests and -r ignored. * I was hit by this in oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers which is skipped due to upper case characters in module name and selftest started to run all tests (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name as first from 529) poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK 2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers'] 2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build .. 2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name) I'll rename eSDK to esdk in next commit to avoid this. * also fix small typo in context I've noticed when debugging this (From OE-Core rev: 15229c9abaf4cc398c31d97d171b9fac57141873) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and manifest ↵Martin Jansa2023-03-221-42/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | as well * these cases were correctly respecting IMAGE_LINK_NAME in most tests * the only exception was relatively wide glob for manifest: "test-empty-image-*.manifest" * and even wider glob for -dbg: "*-dbg.rootfs.tar.bz2" * replace them with the exact filename we expect for given image * be aware that gzip won't accept the symlink in IMAGE_LINK_NAME causing: 2023-03-13 08:58:23,845 - oe-selftest - INFO - ... FAIL 2023-03-13 08:58:23,845 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 124, in test_bmap self.assertTrue(runCmd('gzip -t %s' % gzip_path)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) AssertionError: Command 'gzip -t /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz' returned non-zero exit status 1: gzip: skipping: /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz is a symbolic link * and "*-dbg.rootfs.tar.bz2" doesn't work if IMAGE_NAME_SUFFIX is changed to anything else than ".rootfs" or moved into IMAGE_LINK_NAME (like I plan in future changes where this will need to be updated again). Luckily we can use the symlink which currently doesn't have IMAGE_NAME_SUFFIX nor the DATETIME, so we don't need to search for it with glob, e.g. currently: core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2 core-image-minimal-qemux86-64-dbg.tar.bz2 -> core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2 [YOCTO #12937] (From OE-Core rev: 39285e981343930e41afe4eb8f2db675a85d54c2) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: minidebuginfo.py respect IMAGE_LINK_NAMEMartin Jansa2023-03-221-7/+8
| | | | | | | | | | | | | * use IMAGE_LINK_NAME instead of hardcoding core-image-minimal-${MACHINE} assumption [YOCTO #12937] (From OE-Core rev: 9f8ffe22d9bb7cd93b9bc9ece917a48b27ab22d3) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: gdbserver.py: respect IMAGE_LINK_NAMEMartin Jansa2023-03-221-4/+5
| | | | | | | | | | | | | * use IMAGE_LINK_NAME instead of hardcoding core-image-minimal-${MACHINE} assumption [YOCTO #12937] (From OE-Core rev: 3ee65ecdf841d74e2767d620e7f4a62e3a8863dc) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and KERNEL_FIT_LINK_NAMEMartin Jansa2023-03-221-18/+16
| | | | | | | | | | | | | | | | | | | | | | | * use these variables instead of assuming: "fitImage-its-%s-%s-%s" % (image_type, machine, machine) notice that these files have no extension and the -machine suffix is duplicated (once from core-image-minimal and once from kernel-fitimage.bbclass:kernel_do_deploy:append through KERNEL_FIT_LINK_NAME which is MACHINE by default: KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}" KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}" [YOCTO #12937] (From OE-Core rev: d6e52428ca25d9b6ec2a52406f2d1859d05867d3) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: runqemu: respect IMAGE_LINK_NAMEMartin Jansa2023-03-221-4/+6
| | | | | | | | | | | | | | * don't assume that every built image is named: <recipe>-<machine>.<suffix> [YOCTO #12937] (From OE-Core rev: d3c1c983d062f0731610f4a6984126ec6dab8caf) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: respect IMAGE_LINK_NAME also in test_rawcopy_plugin_qemuMartin Jansa2023-03-221-1/+5
| | | | | | | | | | | | | | | | * this one is more tricky, because the test_rawcopy_plugin.wks.in file is used while building core-image-minimal-mtdutils, but the image filename inside wks.in is from core-image-minimal, so we cannot just let bitbake expand IMAGE_LINK_NAME, use separate variable set in the same config fragment IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL [YOCTO #12937] (From OE-Core rev: 8864ee825fdc52fe7a8cf93876c81c639fba66f4) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: respect IMAGE_LINK_NAMEMartin Jansa2023-03-221-19/+19
| | | | | | | | | | | | | * use IMAGE_LINK_NAME instead of hardcoding core-image-minimal-${MACHINE} assumption [YOCTO #12937] (From OE-Core rev: d6151aeb1e8f6162113b1123fa0f162536e48a44) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting SSTATE_MIRRORMartin Jansa2023-03-221-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * with my build/conf/local.conf: SSTATE_DIR = "/OE/build/poky/build/sstate-cache" these devtool tests will first set own SSTATE_DIR and the original one set as SSTATE_MIRROR: 2023-03-11 11:51:46,837 - oe-selftest - INFO - test_devtool_update_recipe_append (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append) 2023-03-11 11:51:46,846 - oe-selftest - DEBUG - Appending to: /OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/conf/selftest.inc SSTATE_DIR = "/OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/sstate_devtool" SSTATE_MIRRORS += "file://.* file:////OE/build/poky/build/sstate-cache/PATH" * but that unfortunately leads to a warning from sanity.bbclass about SSTATE_MIRRORS without matching BB_HASHSERVE, because BB_HASHSERVE is set to "auto" by default these tests failing with: 2023-03-11 11:55:39,610 - oe-selftest - INFO - ====================================================================== 2023-03-11 11:55:39,610 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_append_git (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append_git) 2023-03-11 11:55:39,610 - oe-selftest - INFO - ---------------------------------------------------------------------- 2023-03-11 11:55:39,611 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1118, in test_devtool_update_recipe_append_git self.assertNotIn('WARNING:', result.output) AssertionError: 'WARNING:' unexpectedly found in 'NOTE: Starting bitbake server...\nWARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.\nLoading cache...done.\nLoaded 0 entries from dependency cache.\nParsing recipes...done.\nParsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 52 skipped, 0 masked, 0 errors.\n\nSummary: There was 1 WARNING message.\nINFO: Updating SRCREV in recipe mtd-utils-selftest_git.bb\nNOTE: Writing append file /tmp/devtoolqa1m2lh02v/layer/recipes-devtools/mtd/mtd-utils-selftest_git.bbappend' ---------------------------------------------------------------------- * just setting BB_HASHSERVE to empty doesn't work, because then we would need to disable OEEquivHash as well as it fails with: ERROR: OEEquivHash requires BB_HASHSERVE to be set (From OE-Core rev: 96d4392ee9c5c3674e5c4c4512f527a2ca6765e4) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa rtc.py: skip if read-only-rootfsMikko Rapeli2023-03-141-3/+5
| | | | | | | | | | | hwclock command fails on read-only-rootfs: AssertionError: 1 != 0 : Failed to reset RTC time, output: hwclock: cannot open /etc/adjtime: Read-only file system (From OE-Core rev: 017bf8c160f6ab67d9f8e8d9e30b15bf84f73807) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/sstate: Split classes to allow more parallelismRichard Purdie2023-03-131-4/+8
| | | | | | | | | | | By splitting the single class into multiple classes, it allows more parallelism in the execution. These tests don't have interdependencies that benefit from sstate reuse or anything like that so this makes sense to improve overall test execution time if/where possible. (From OE-Core rev: 9dbc2f4ebbde47b1c997948a9690ffb8eb29b552) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/sstate: Move common code to base classRichard Purdie2023-03-131-99/+98
| | | | | | | | | Move the other common shares test functions to the base class to improve the code structure. (From OE-Core rev: de3e6f85c5537a3571ffbe2326b73f2c2526bce2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-05 07:41:39 +0000