summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
Commit message (Collapse)AuthorAgeFilesLines
...
* oeqa/sdk/case: Ensure DL_DIR is populated with artefacts if usedRichard Purdie2024-06-131-3/+6
| | | | | | | | | | | Where we're using DL_DIR in sdk archive to try and cache testing artefacts, copy into the cache so that it gets populated and this doesn't have to be done manually. Currently we're making a lot of repeat requests to github as this wasn't being populated. (From OE-Core rev: a3284958a2cc6c90a5fac26976bddc23f821c972) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: add RECIPE_UPDATE_EXTRA_TASKS testTim Orling2024-06-131-0/+46
| | | | | | | | | | | | | | | | | | | | Add test_devtool_upgrade_recipe_update_extra_tasks test case to test upgrade of python3-guessing-game from v0.1.0 to v0.2.0 which will exercise the update_crates task during the upgrade. Add python3-guessing-game_git.bb.upgraded and python3-guessing-game-crates.inc.upgraded which are the 0.2.0 variants. Check that the new recipe file has the expected differences. Check that the new -crates.inc file has the expected differences, which should be reproducible because of Cargo.lock. (From OE-Core rev: d14368bc775cbf5142c1312dfc2076e328381aef) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/spdx: Fix for SPDX_VERSION additionRichard Purdie2024-06-121-1/+2
| | | | | | | | Update the test for the addition of SPDX_VERSION to the deploy path. (From OE-Core rev: 055e8c21908127722abad9e992d6408d8697a119) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftests: add test_minidebuginfo_qemuEtienne Cordonnier2024-06-071-0/+16
| | | | | | | | | | test_systemd_coredump_minidebuginfo was getting skipped in CI, because the feature isn't enabled per default in any image at the moment. Add this selftest so that test_systemd_coredump_minidebuginfo gets executed in CI. (From OE-Core rev: 377603886f0d975ec23f32ee462693d4e3370aaf) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core: remove duplicate 'os' importEnrico Jörns2024-06-061-1/+0
| | | | | | | (From OE-Core rev: f1362fe26cc8adc09d15ea2a854af6590728c683) Signed-off-by: Enrico Jörns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: extend docstring for runqemu context managerEnrico Jorns2024-06-061-1/+17
| | | | | | | | | | Usage and arguments are not necessarily self-explaining, thus add a bit of documentation here. (From OE-Core rev: 62343333d50b75ad04a4220bcf0990e0a89996a0) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: make minidebuginfo test work with coreutilsEtienne Cordonnier2024-06-061-1/+2
| | | | | | | | | The test was failing with core-image-full-cmdline which uses sleep from coreutils instead of sleep from busybox. (From OE-Core rev: 8497edf0c56da34ea7b7509b8fcd46fcba2fd0e3) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ltp: Drop groups that were removed in new versionRichard Purdie2024-06-041-1/+1
| | | | | | | | | The io, cap_bounds and filecaps groups were removed in latest version of ltp. Tweak the test class to match. (From OE-Core rev: d5460f15cbc56e52cbbc09785a777278208da7f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: fix regression in minidebuginfo testEtienne Cordonnier2024-05-311-1/+2
| | | | | | | | | | The builtin command "sleep" was getting executed instead of the busybox binary. Hence "pidof sleep" was empty and the test was failing. (From OE-Core rev: 3188f3f0718b8a59bc9e3cae4cdf74e82bdb86c3) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pinentry: update 1.2.1 -> 1.3.0Alexander Kanavin2024-05-301-1/+1
| | | | | | | | | | License-update: formatting, FSF address (From OE-Core rev: 7e2b529fff1aee883d95c3cf97300c8d8b5830d3) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/rust: correctly form the PATH environment variableAlexander Kanavin2024-05-281-3/+2
| | | | | | | | | | | | | | | | | | There were two issues here: - the path to native python was not included, so the test suite was running with host python, causing failures due to host python being given native libraries: https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/1711/steps/12/logs/stdio (rust recipe uses native python throughout) - tmp/hosttools was listead ahead of native-sysroot/usr/bin (From OE-Core rev: aec96b5aec8d6b4a5ae8a719ca05bb727a6197cb) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: build the default set of toolsAlexander Kanavin2024-05-281-0/+2
| | | | | | | | | | | | | | | | | | | | | | Setting it explicitly replaces rust's default choice which is rustdoc (needed for example in selftests and otherwise expected to be present in typical rust installations): https://github.com/rust-lang/rust/blob/master/config.example.toml#L320 This addresses some of the rust selftest failures but not all. Help is appreciate to restore the selftest. Unfortunately, this also breaks rust reproducibility (or rather exposes that it was never properly fixed, as explained here: https://lists.openembedded.org/g/openembedded-core/message/199288 ) (From OE-Core rev: 58eaf2ee6c0809bf0a0d3c1d177e62bda7241651) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual: remove obsolete CROPS and Eclipse manually testing scriptsRoss Burton2024-05-282-616/+0
| | | | | | | | | | | | | The Eclipse plugin is very dead, we don't need to carry the manual test guide for it. CROPS has its own automated testing, nobody is doing the testing by hand. (From OE-Core rev: f74a634f199782ce5fa3023d02079c1534bc4aeb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: add test for modifying recipes using go.bbclassOla x Nilsson2024-05-281-0/+19
| | | | | | | | | | | | | go.bbclass uses a special do_unpack function that causes the git root to be different from S. Verify that it unpacks as expected. [ YOCTO #15483 ] (From OE-Core rev: fab0c737b95b8d0c0bbf58336bc308776c956406) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/sstatetests: move exception list for cache objects to the parent classAlexander Kanavin2024-05-281-13/+13
| | | | | | | | | | | This is useful not just in cdn/local cache tests, but also in the build bundle test, and so should be applied by default. (From OE-Core rev: 37f0ba71868ef1907883ab751c82ba222fbc87da) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/sstatetests: separate sstate presence check into its own classAlexander Kanavin2024-05-281-2/+3
| | | | | | | | | | | This allows reusing the check function in tests other than CDN/local mirror ones, such as sstate bundle testing. (From OE-Core rev: 8a4ab4c063f307b480d98c6aac1e76cde0b9e99e) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: remove unused imports from test casesRoss Burton2024-05-225-5/+2
| | | | | | | | | Remove redundant imports. (From OE-Core rev: b6223259458578d8b967aff11d3263dfed496708) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: rename test casesRoss Burton2024-05-225-7/+7
| | | | | | | | | | | Instead of having a test called eg "assimp", rename it to "cmake" as the point of the test is to verify that CMake works. This should make it clearer what the tests are actually exercising. (From OE-Core rev: ccf7fdc0e5b6df218b319f972cd5ba142c06c243) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk/assimp: Upgrade and fix for gcc 14Richard Purdie2024-05-211-4/+4
| | | | | | | | | To enable this test to work with gcc 14, pass the option to make warnings non-fatal. Also upgrade to version 5.4.1 from 5.3.1. (From OE-Core rev: c3df6287ae26dc9d7f11eb7e26fdbcaefe4dfead) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdkext/devtool: replace use of librdfaRoss Burton2024-05-211-4/+3
| | | | | | | | | | | librdfa hasn't had a commit for a decade now and as such has problems with modern compilers (specifically gcc 14.1). Switch the recipe creation test to something much simpler that we also control: dbus-wait. (From OE-Core rev: d801de1f702d8d0def55011b5b6ad39d85f978f1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes: Ensure S is set to a valid directoryRichard Purdie2024-05-211-2/+2
| | | | | | | | | | | | Several recipes have S pointing at a directory that does not exist. Set S in these cases to somethig valid making the metadata and recipe behaviour more consistent. Tweak one of the QA test diff offsets to match the changed recipe. (From OE-Core rev: 22f1f5849a9a3bf287dbe8933546e52e39ddc86e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Drop oe-local-files and simplifyRichard Purdie2024-05-211-22/+16
| | | | | | | | | | | | | | | | | | | | The only real reason for oe-local-files was to support S = WORKDIR. With changes to drop support for that, it makes sense to simplify devtool and to try and make both the code and the processes/workflows simpler. This patch drops support for S = WORKDIR, removes oe-local-files and then updates the test cases to match this new situation. At the code level, we assume we can always now track code changes using git and that things committed into git are handled as patches (as before) but delta against HEAD is saved as specific file level changes to the recipe. One test is disabled as it is no longer approproate. It is being keped until we can make WORKDIR != UNPACKDIR at which point it should be revisited. (From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes: Start WORKDIR -> UNPACKDIR transitionRichard Purdie2024-05-213-14/+14
| | | | | | | | | Replace references of WORKDIR with UNPACKDIR where it makes sense to do so in preparation for changing the default value of UNPACKDIR. (From OE-Core rev: 1f18b9a512800860d5153d89eb82b56388efad6f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/cases/runtime_test: Exclude centos-9 from virgl testsRichard Purdie2024-05-171-1/+1
| | | | | | | | | Similarly to centos 8, centos 9 doesn't support the render device we need for this test. (From OE-Core rev: c2be3afabf84f287c90b61ae2509728a6634fb8f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/debuginfod: use localpkgfeed to speed server startupRoss Burton2024-05-151-6/+8
| | | | | | | | | | | | | | | | Sometimes the debuginfod selftest fails due to a timeout, because it spends too long scanning a huge deploy directory that due to what tests were ran previously can contain 30K packages. The test only needs a subset of the feed, so use the new localpkgfeed class to construct a minimal feed before running the test. [ YOCTO #14937 ] (From OE-Core rev: 855376f518b28248ccd82ef5b2e89e6a8c970542) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: Fix for usrmerge in DISTRO_FEATURESRichard Purdie2024-05-091-0/+2
| | | | | | | | | If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would fail. Improve the test so it works in both cases. (From OE-Core rev: afa211746a2aa1993a54cc5a5e1937679341da8e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/recipetool: Fix for usrmerge in DISTRO_FEATURESRichard Purdie2024-05-091-5/+11
| | | | | | | | | If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would fail. Improve the test so it works in both cases. (From OE-Core rev: 68a27d307a7042e242c49cf3d069469f40e09902) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "oeqa/selftest/devtool: fix test_devtool_add_git_style2"Alexandre Truong2024-04-301-1/+1
| | | | | | | | | | | | | | | | This reverts commit ab6d3e3d645ffc343f434bf731339fa237df027a from poky repository. The previous reverted commit was a workaround. The fix "oeqa/selftest/devtool: fix _test_devtool_add_git_url" tackle the issue. So, the workaround is not needed anymore. (From OE-Core rev: 731f47ecfd8ad6558aac629806810789c623986b) Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr> Reviewed-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: fix _test_devtool_add_git_urlAlexandre Truong2024-04-301-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is a follow-up to the bug#15466. As a reminder, the bug was about devtool’s submodule detection checking for HEAD when a version is being passed. As Vincent Kriek pointed out: the --version that is being passed to devtool is only used for setting the PV value in the recipe. To take into account the tag, we add --srcrev to the command: devtool add --srcrev v3.1.0 --version v3.1.0 mbedtls git://git@github.com/ARMmbed/mbedtls.git;protocol=https Changes to _test_devtool_add_git_url have been made to take into account the srcrev. srcrev will be passed as an optional parameter because the test_devtool_add_git_style1 does not need the srcrev contrary to test_devtool_add_git_style2 Fixes [YOCTO #15466] (From OE-Core rev: a8686f3641e4407dee3d807898ffd620e2732b78) Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr> Reported-by: Alexandre Truong <alexandre.truong@smile.fr> Suggested-by: Vincent Kriek <vincent@coelebs.dev> Reviewed-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: add test for updating local files into another layerJulien Stephan2024-04-251-2/+18
| | | | | | | | | | | | We don't have a test to check if we can correctly devtool update-recipe/finish into another layer. So update the existing test_devtool_update_recipe_local_files to also check the updates into another layer. (From OE-Core rev: bd44c895d36e246a25c7a6e40bf9f4089dc7a297) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: selftest: context: run tests serially if testtools/subunit modules are ↵Julien Stephan2024-04-231-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | not found If testtools and/or subunit modules are not found we get the following backtrace (example for testtools): NOTE: Starting bitbake server... Traceback (most recent call last): File "<..>/poky/scripts/oe-selftest", line 60, in <module> ret = main() File "<..>/poky/scripts/oe-selftest", line 47, in main results = args.func(logger, args) File "<..>/poky/meta/lib/oeqa/selftest/context.py", line 391, in run rc = self._internal_run(logger, args) File "<..>/poky/meta/lib/oeqa/selftest/context.py", line 377, in _internal_run rc = self.tc.runTests(**self.tc_kwargs['run']) File "<..>/poky/meta/lib/oeqa/selftest/context.py", line 161, in runTests return super(OESelftestTestContext, self).runTests(processes, skips) File "<..>/poky/meta/lib/oeqa/core/context.py", line 91, in runTests result = self.runner.run(self.prepareSuite(self.suites, processes)) File "<..>/poky/meta/lib/oeqa/selftest/context.py", line 154, in prepareSuite from oeqa.core.utils.concurrencytest import ConcurrentTestSuite File "<..>/poky/meta/lib/oeqa/core/utils/concurrencytest.py", line 22, in <module> import testtools ModuleNotFoundError: No module named 'testtools' Fix this by adding a custom callback on -j/--num-processes parameter to check testtools and subunit modules. Fallback to serial testing if missing. This strategy is already used in sdk/context.py (From OE-Core rev: 35284404473b2c2d9f69594582868ed66ef3525e) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: Do not use -l option with dfKhem Raj2024-04-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | -l option is specific to df provided by coreutils, if df applet from busybox is used then it does not work and fails like below Fixes df: invalid option -- 'l' BusyBox v1.36.1 () multi-call binary. Usage: df [-PkmhT] [-t TYPE] [FILESYSTEM]... DEBUG: [Command returned '1' after 0.71 seconds] DEBUG: Command: df -hl Status: 1 Output: df: invalid option -- 'l' it seems worth a compromise to show remote mounted filesystems if any during ptests and it works with both df implementations (From OE-Core rev: 75cc1ea4348a2294fdc5ab20530fcff27056ff06) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: fix test_devtool_add_git_style2Alexandre Truong2024-04-041-1/+1
| | | | | | | | | | | | | | | | | | | | | The problem is the following: AssertionError: 'gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master' != 'git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master' Mbedlts made changes to their repository, adding a sub-module, thus the assert triggers an error with the url: git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master https://github.com/Mbed-TLS/mbedtls/commit/456a54da8ef44c8b725b524567c08cffec6a7214 was the upstream change. To fix the issue, the url has been changed to: gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master (From OE-Core rev: 9ac737fbe05c85ec8333b396ce2f89de6654916f) Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr> Reviewed-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/overlayfs: test read-only rootfsBaruch Siach2024-04-041-6/+28
| | | | | | | | | | | Use the read-only squashfs filesystem to test the read-only case. (From OE-Core rev: 1a61da196f014dc34a998d3e46f03abc79ebc84d) Signed-off-by: Baruch Siach <baruch@tkos.co.il> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sstatetests: Fix race issueRichard Purdie2024-03-311-2/+3
| | | | | | | | | | | | Under some load conditions, the result event can come back before the command complete event. If that happens, the code would hang indefinitely. Rework the code to wait for both events and avoid the hang. (From OE-Core rev: 8cfc94a4404c54bc73eab9f98d9da1f84c2135ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/sstatetests: run CDN check twice, ignoring errors the first timeAlexander Kanavin2024-03-301-1/+5
| | | | | | | | | | | | | | | The current CDN isn't able to serve all objects on first request, and it was suggested to work around that by trying again: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15335#c16 Once CDN moves to a better location this can probably be reverted. [YOCTO #15335] (From OE-Core rev: 0db3466303f56736d2fa9a105435eaaadb385082) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatetests.py: Add testing for correct sstate permissionsEilís 'pidge' Ní Fhlannagáin2024-03-071-1/+27
| | | | | | | | | | | | This patch adds to run_test_sstate_creation so that it also tests that sstate directories don't accidentally pickup umask permissions from the user upon creation. [RP: Python style tweaking] (From OE-Core rev: 7d6eb828e97ad3f27d94efdccd920fb2aef36743) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Fix dbus-wait timeout and loop conditionalRichard Purdie2024-03-071-4/+5
| | | | | | | | | | | | | | | The dbus-wait command returns a timeout after 60s but reports "success", detect this. Unfortunately it does effectively break the test as the signal is nearly never being correctly detected since it was already sent. For that reason comment out the code instead too. Also fix the loop conditional as the logic was incorrect and it was looping indefinitely when an image match didn't occur. (From OE-Core rev: 89c930e9e4b38b116edcba59e88621a39f8bda67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Add screenshot sample logic/timeout/dbus-waitEilís 'pidge' Ní Fhlannagáin2024-03-071-38/+41
| | | | | | | | | | | | This patch uses dbus-wait to wait for matchbox to be up. Once that happens, it sets a timeout of 60 seconds and takes a screenshot and compares it, every 2 seconds. If diff=0 it passes. If the timeout ends, it fails. (From OE-Core rev: 287b4f0a8244f7214f6a1aaa84ef16cc528f8326) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Exclude qemuriscv64Eilís 'pidge' Ní Fhlannagáin2024-03-071-1/+1
| | | | | | | | | | | Excluding riscv64 due to mouse rather than a touchscreen which adds a moving cursor, so the diff ends up > 0. Need to fix the image to use the touchscreen rather than mouse input. (From OE-Core rev: 7f7032c7613abd62ce510c98211c75fc7c5e7090) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Mask out the mouse panel icon for nowRichard Purdie2024-03-071-2/+2
| | | | | | | | | | The mouse panel icon can move when the time changes between 3 and 4 digits. Ideally we'd fix the alignment of the clock on the panel but to get the tests working, increase the size of the mask for now. (From OE-Core rev: 8ec02142ab98741749281bdb29d5973c96b839d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Various code improvements and fixesRichard Purdie2024-03-071-43/+48
| | | | | | | | | | | | | | | | * Allow tools to be found from the host PATH so that imagemagick from a buildtools tarball/sdk can work * Reformat the code to have imports at the start of the file and have more standard formatting and whitespace * Always save copies of the images, the space imapct is negligle compared to the debug win * Write the images to ${T} * Use bb.utils.mkdirhier() instead of more complex code * Restrict the tests to images containing matchbox-desktop (From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Proof of concept for screenshot testcasesEilís 'pidge' Ní Fhlannagáin2024-03-071-0/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This takes the work rburton did on image screenshot testing and expands it. Right now this works with most of the qemu based machines except for - qemuppc64 - qemuarmv5 - qemuriscv32 - qemuloongarch64 See "Known Issues" further down. This test takes a screendump of a qemu image, blanks out the clock and compares it to an image we have on record. If the diff is exact, the test passes. If not, it stores the image in build/failed-images and fails out. In order to enable this test, you will need meta-openembedded/meta-oe in your bblayers.conf for imagemagick and the following in local.conf: IMAGE_CLASSES += "testimage" TEST_SUITES = "login" TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot " Known Issues ------------ The main issue is that I've yet to find a gating factor that would allow me to tell when the qemu instance is fully up and rendered. I've tried a few tactics here, (dbus-wait, qmp) but for now a disgusting time.sleep(30) is there. You can replicate this by running qemumips. The screen load takes forever, but you even see it on qemux86 where the Home and Workspace Switch icons will sometimes take a while to fully load. Eventually I'm going to have to take multiple screenshots and compare them, but then you get into the issue where the question is, is the diff greater than 0 because it hasn't fully loaded or something is actually incorrect. There are the issues I know about: - runqemu qemuppc64 comes up blank. - qemuarmv5 comes up with multiple heads but sending "head" to screendump. seems to create a png with a bad header. - qemuriscv32 and qemuloongarch64 don't work with testimage apparently? - qemumips64 is missing mouse icon. - qemumips takes forever to render and is missing mouse icon. - qemuarm and qemuppc return incorrect width - All images have home and screen flipper icons not always rendered fully at first. The sleep seems to help this out some, depending on machine load. (From OE-Core rev: dc7cefbaccde50df6c4396e66d50659a45e00631) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Co-authored-by: Ross Burton <ross.burton@arm.com> Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: Fix error on calls to run_monitorEilís 'pidge' Ní Fhlannagáin2024-03-071-2/+2
| | | | | | | | | | | | | | | | | | | A change in QEMUMonitorProtocol.cmd() requires that we either pass in kwargs instead of an argument dict or move to cmd_raw() cmd() was renamed to cmd_raw() (and command() was renamed to cmd()) See: https://github.com/qemu/qemu/commit/37274707f6f3868fae7e0055d9a703006fc142d0 https://github.com/qemu/qemu/commit/684750ab4f8a3ad69512b71532408be3ac2547d4 My concern with this patch is that I haven't seen this come up with utils/dump.py which also uses QemuMonitor's run_monitor. If it is occuring, this should fix issues there as well (From OE-Core rev: 9665d38ab60c1c3b27887c2b1a6396f13a1b33ea) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* minicom: upgrade 2.8 -> 2.9Alexander Kanavin2024-03-071-2/+3
| | | | | | | | | | | Adjust selftest to git-add the directory with newly added patches, as the new minicom recipe has no default patches, and thus no directory with them (and the selftest assumed it does). (From OE-Core rev: 1fb2aa3f242ef20f8edfb518164b629258a04dd4) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/gotoolchain: set GOPROXYJose Quaresma2024-03-031-0/+1
| | | | | | | | | | | | Since go-1.21 GOPROXY needs to be set explicitly, otherwise it fails with: - GOPROXY list is not the empty string, but contains no entries This fixes the selftest (From OE-Core rev: c491d967858c01fead21495f44f1a9f8cdf8e833) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/lib/utils/postactions: fix host disk usage stats retrievalAlexis Lothoré2024-02-291-1/+1
| | | | | | | | | | | | | | | | | | The recently introduced postactions module can raise, on failing ptests, the following warning: WARNING: core-image-ptest-glib-2.0-1.0-r0 do_testimage: Can not get host disk usage: [Errno 2] No such file or directory: '/usr/bin/df' The issue is likely not happening because of df absence (to be confirmed amongst the variety of workers) but because of the wrong path. Fix it by letting subprocess search for df, passing only the binary name. To make it work, we also have to reset the environment, otherwise the environment configured before running bitbake will be used, and search will fail. (From OE-Core rev: da7cc5def2839a0e15d07244f858847479c12caa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest devtool: ide-sdk testsAdrian Freihofer2024-02-271-9/+33
| | | | | | | | | | Improve the GDB related tests. Verify GDB finds the correct source files. (From OE-Core rev: 67eed460c0bf18d23f2c9180f195417895acfd55) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: testimage: add host disk usage stat as post actionAlexis Lothoré2024-02-271-1/+12
| | | | | | | | | | | | | | Since the target under test can be a virtualized guest, when some tests fail because of disk usage (see [1]), also fetch disk usage statistics from host to allow checking whether a host disk space saturation could affect running tests. [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220 (From OE-Core rev: 2ab3a0935b1e7a016402f149da1fc01b38d7af55) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: add target disk usage stat as post actionAlexis Lothoré2024-02-271-1/+16
| | | | | | | | | | | | | In order to debug issues related to disk space (see [1]), add a failed tests post action to retrieve disk usage on the target. Rely on the test context object to run the corresponding command onto the target [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220 (From OE-Core rev: 637e216b0e5191571270aa07e1f50a6e41a8c08f) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-07 23:03:53 +0000