summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* selftest/container: fix IMAGE_FEATURES assignmentRoss Burton2024-11-061-3/+0
| | | | | | | | | | | | | | | | | | | | Assigning IMAGE_FEATURES="" in local.conf doesn't actually do anything useful, as bitbake.conf has IMAGE_FEATURES += EXTRA_IMAGE_FEATURES after the include of local.conf. In this test case, this results in IMAGE_FEATURES still using EXTRA_IMAGE_FEATURES, so the image contains the files installed by the post-install-logging feature. As the intention here is that container-test-image is as minimal as possible, move the IMAGE_FEATURES assignment into the image itself so it actually works, and update the expected file list to remove the post-install-logging files. (From OE-Core rev: c65349e1238ddc6634dfa4759c57e6168a355200) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducibility: restart bitbake for each buildYoann Congal2024-11-051-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since [0], the reproducibility test tries to save logs of failed builds. Using the memory persistent bitbake prevent this by concatenating logs in the single persistent instance (See [YOCTO #15641]). So, force a BitBake server restart for each build to ensure a new log file is created. This fixes this error (seen on AB [1]): 2024-11-02 08:16:00,816 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail. 2024-11-02 08:16:00,819 - oe-selftest - INFO - ... ERROR 2024-11-02 08:16:00,819 - oe-selftest - INFO - Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 321, in test_reproducible_builds self.copy_file(variables["BB_CONSOLELOG"], os.path.join(save_dir, "bitbake-%s.log" % name)) File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 216, in copy_file shutil.copyfile(source, dest) File "/usr/lib/python3.11/shutil.py", line 256, in copyfile with open(src, 'rb') as fsrc: ^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-python/build-st/tmp/log/cooker/qemux86-64/20241102071141.log' [0]: https://git.openembedded.org/openembedded-core/commit/?id=c78cc753843d4199443052e8deb0c9c3b7e4b580 [1]: https://valkyrie.yoctoproject.org/#/builders/87/builds/17/steps/40/logs/stdio (From OE-Core rev: e89bbc00ba16574d719b199c01ffbf37646f4f54) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runCmd: print stderr when that is a separate streamAlexander Kanavin2024-11-011-0/+2
| | | | | | | | | | | | | runCmd by default merges stderr into stdout, and only needs to print stdout when errors occur. When stderr is requested as a separate stream, and an error occurs, stderr is discarded, obscuring useful error messages. This changes the output to include both streams. (From OE-Core rev: 72c747b37ccdd486ddae06e3d0a99fb2b93643ba) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: wic_sector_size add wic-tools pathVince Chang2024-11-011-44/+48
| | | | | | | | | | Add wic-tools to the PATH to avoid failures when running the wic_sector_size test case on a build host that doesn’t have parted. (From OE-Core rev: 013dcdf75669421bc38d699263cb1e8d5b95d398) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: remove 2>/dev/null to help fail diagnosticVince Chang2024-11-011-21/+23
| | | | | | | | | | | | | | The wic test case previously used '2>/dev/null' to suppress error messages. This commit updates the code to use 'stderr=subprocess.PIPE' when calling runCmd(). Refer: https://lists.openembedded.org/g/openembedded-core/topic/109308684 (From OE-Core rev: bd26d999a0ba1107ee5629a8e238f4fe945e9be5) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: add "Inactive-Upstream"Katariina Lounento2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The list of valid statuses (`upstream_status_literal_valid_status`) was missing "Inactive-Upstream", which caused patchtest to fail the test test_patch.TestPatch.test_upstream_status_presence_format for patches containing lines like: +Upstream-Status: Inactive-Upstream [lastrelease: 2013 lastcommit: 2013] with the error: FAIL: test Upstream-Status presence: Upstream-Status is in incorrect format (test_patch.TestPatch.test_upstream_status_presence_format) "Inactive-Upstream" is documented in the Yocto Project and OpenEmbedded Contributor Guide [1]: Inactive-Upstream [lastcommit: when (and/or) lastrelease: when] The upstream is no longer available. This typically means a defunct project where no activity has happened for a long time — measured in years. To make that judgement, it is recommended to look at not only when the last release happened, but also when the last commit happened, and whether newly made bug reports and merge requests since that time receive no reaction. It is also recommended to add to the patch description any relevant links where the inactivity can be clearly seen. `upstream_status_nonliteral_valid_status` only seems to be used in logging and the value was copied verbatim from the aforementioned documentation. After this change all upstream status options documented in [1] are covered. [1] https://docs.yoctoproject.org/5.0.3/contributor-guide/recipe-style-guide.html#patch-upstream-status (From OE-Core rev: a39e6e6854b60fd10f21c6c3f1e7d676e12ba9ee) Signed-off-by: Katariina Lounento <katariina.lounento@vaisala.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: fix malformed cve status description with : charactersPeter Marko2024-11-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When CPE is not provided and character ":" is in cve status description, current code takes only last part of split function. This works only if there is no ":" in description, otherwise it drops the other split parts. Do a new split of the original string to take the whole description unchanged. This fixes following entries from world build of poky+meta-oe+meta-python: tiff-4.6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2015-7313 CVE_STATUS: fixed-version: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue description: //security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue corrected: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue gnupg-2.5.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2022-3219 CVE_STATUS: upstream-wontfix: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993 description: //dev.gnupg.org/T5993 corrected: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35325 CVE_STATUS: upstream-wontfix: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303 description: //github.com/yaml/libyaml/issues/303 corrected: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35326 CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 description: //github.com/yaml/libyaml/issues/302 corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35328 CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 description: //github.com/yaml/libyaml/issues/302 corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 cpio-2.15-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-7216 CVE_STATUS: disputed: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html description: //lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html corrected: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html openssh-9.9p1-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-51767 CVE_STATUS: upstream-wontfix: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. description: //bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. corrected: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. cups-2.4.10-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2021-25317 CVE_STATUS: not-applicable-config: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply. description: root, so this doesn't apply. corrected: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply. unzip-1_6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2008-0888 CVE_STATUS: fixed-version: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source description: //bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source corrected: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source syslog-ng-4.7.0-r0 do_cve_check: CVE_STATUS with 6 parts for CVE-2022-38725 CVE_STATUS: cpe-incorrect: cve-check wrongly matches cpe:2.3:a:oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 description: syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 corrected: cve-check wrongly matches cpe:2.3:a:oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 (From OE-Core rev: cc33dd9176726cb4b2d2f142ed1bc655da8e0a9f) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: do not skip cve status description after :Peter Marko2024-11-011-1/+1
| | | | | | | | | | | | | | | | Correct maxsplit parameter from 5 to 4 to not drop text if description contains ":". Example: >>> "detail: cpe:vendor:product:description:cont".split(':', 5) ['detail', ' cpe', 'vendor', 'product', 'description', 'xxx'] >>> "detail: cpe:vendor:product:description:cont".split(':', 4) ['detail', ' cpe', 'vendor', 'product', 'description:xxx'] (From OE-Core rev: 3c4d8ca41ac0b429af92bf0ea84f1dfd0cda9e1f) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package: remove PACKAGE_SNAP_LIB_SYMLINKSRoss Burton2024-11-011-20/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was added to OpenEmbedded in 2011[1] for the micro distro[2] and subsequently ported to oe-core[3]. This feature aims to improve runtime link speed by removing intermediate symlinks, i.e. libstdc++.so.6 is not a symlink but the actual library. However, there are several issues here: - The meta-micro distribution has been unmaintained since 2012[4] - The original creator of this function has confirmed that it is not in use anymore because the renaming doesn't affect the -dev packages, so on-target development or use of SDKs isn't possible. Whilst the latter is possible to fix, the performance impact of removing a single symlink resolution is negliable at best and the packaging code is already very complex. So, remove this functionality as it is known to be broken, isn't tested, and removing it reduces complexity. [1] https://git.openembedded.org/openembedded/commit/?id=cf7114179ead8ddff8f66e84d630811920ac9add [2] https://git.openembedded.org/openembedded/tree/conf/distro/micro.conf [3] https://git.openembedded.org/openembedded-core/commit/?id=600dbb7cb384c2290af38b993a9bea3a4dfc4494 [4] https://git.openembedded.org/meta-micro/ (From OE-Core rev: 9bfdf4c9eff16804588942a13ac7484059197f99) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/sstatetests: run CDN mirror check only onceAlexander Kanavin2024-11-011-2/+0
| | | | | | | | | | | The first no-fail check was an attempt to work around the old CDN's instability (and it didn't really help); it should not be necessary with the new CDN, and only delays a-full completion. (From OE-Core rev: 7f75c42b7fcf60a9ca58d3ded9047df675d76dc2) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Add SPDX 3.0 include source case for work-shareHongxu Jia2024-11-011-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | Build gcc and check gcc-14.2.0/README in objset is available $ oe-selftest -r spdx.SPDX30Check.test_gcc_include_source ... 2024-10-26 01:24:57,063 - oe-selftest - INFO - test_gcc_include_source (spdx.SPDX30Check.test_gcc_include_source) 2024-10-26 01:28:24,204 - oe-selftest - INFO - The spdxId of gcc-14.2.0/README in gcc.spdx.json is http://spdx.org/spdxdocs/gcc-f2eaeb0d-b54b-53ba-899a-8c36c21139bf/88d5068ffd41e5ea6b4e0dd390b23bf499bb2b6674a41e09eaf2a887eced16c8/sourcefile/42 2024-10-26 01:28:26,369 - oe-selftest - INFO - ... ok 2024-10-26 01:28:33,315 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-26 01:28:33,316 - oe-selftest - INFO - Ran 1 test in 216.457s 2024-10-26 01:28:33,316 - oe-selftest - INFO - OK 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS: 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS - spdx.SPDX30Check.test_gcc_include_source: PASSED (209.31s) 2024-10-26 01:28:45,260 - oe-selftest - INFO - SUMMARY: 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 216.457s 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) (From OE-Core rev: ccd6dde301dc8c45c8f901ebd4676b488d638b08) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-{2.2,3.0}: support SPDX include source for work-share directoryHongxu Jia2024-11-011-26/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Originally, while SPDX_INCLUDE_SOURCES = "1" [1], there is bug in scan for gcc, libgcc in which the sources locates in work-share directory. Copy source from ${WORKDIR} to ${SPDXWORK} did not satisfy the situation while ${S} was not included in ${WORKDIR} This commit aim to support SPDX include source for work-share directory 1. If is_work_shared_spdx, Copy source from ${S} to ${SPDXWORK}, normally the dest dir in ${SPDXWORK} has the same basename dir of ${S}; but for kernel source, rename basename dir 'kernel-source' to ${BP} (${BPN}-${PV}) 2. For SPDX source copy, do hard link copy to save copy time 3. Move do_patch to no work shared situation along with do_unpack 4. Tweak task do_create_spdx dependencies to assure the patched source in work share is ready for SPDX source copy 5. Remove bb.data.inherits_class('kernel', d) from is_work_shared_spdx, the kernel source locates in 'work-shared', test kernel.bbclass is not necessary [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES (From OE-Core rev: 64454b1956a9b50d6c89a3f3d7c594c1272cb289) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-{2.2,3.0}: fix do_create_spdx dependency while spdx include sourcesHongxu Jia2024-11-011-0/+4
| | | | | | | | | | | | | | | | Call function ${@create_spdx_source_deps(d)} or ${create_spdx_source_deps(d)} along with addtask not working, use task do_create_spdx flag 'depends' to instead Move function create_spdx_source_deps to spdx-common.bbclass for both of create-spdx-2.2.bbclass and create-spdx-3.0.bbclass (From OE-Core rev: dda4b7ccd416ef370634babd150e944f2aa9116e) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/spdx30_tasks.py: improve debug log in add_package_filesHongxu Jia2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | While SPDX_INCLUDE_SOURCES = "1" [1], there are mess of `Adding file' in debug log ''' DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/sources-unpack/COPYING to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/gettext-0.22.5/.tarball-version to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native ''' Summary the total number other than print for each file. ''' DEBUG: Added 7201 files to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/f5e0e04913ac4c595be791fc001d545a77519ed6ee8c743deef721ca0898bc94/document/gettext-minimal-native ''' [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES (From OE-Core rev: a2f1498f3db44f34599b86221b688e1abf08a3c7) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/sbom30.py: correct typoHongxu Jia2024-11-011-1/+1
| | | | | | | | | | | The isinstance expected 2 arguments (From OE-Core rev: 038dcdca760404a44e0bcef389b4b60903f14a1f) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/sbom30.py: correct python list comprehensionHongxu Jia2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | The python list comprehension is not right for list: >>> license_text_map = {'LicenseRef-FSF-Unlimited': 'http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/8d31e22acc4a8979f24dc24042692fb548fc8fc8d85d775ddac406abb122ceea/license-text/FSF-Unlimited'} >>> license_text = ((k, license_text_map[k]) for k in sorted(license_text_map.keys())) >>> print(license_text) <generator object <genexpr> at 0x7f8575173270> >>> [(k,v) for k, v in license_text] [] Change the () to [] to make it a list instead of a generator expression. (From OE-Core rev: e13c54645bf716ddfb19606fd212cc349b570ad4) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducibility: continue testing in case of build failureYoann Congal2024-11-011-9/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current reproducibility test stops all build tasks when a single task fails (default BitBake behavior). This means that a single build failure prevents the reproducibility of other packages from being tested, which is not ideal. To address this, we now use the --continue option of BitBake during test builds, allowing the build process to proceed even when some tasks fail. The failure cases are handled as gracefully as possible. In the event of a build failure: - The entire reproducibility test will be considered a failure. - The complete BitBake log will be saved in the "saved output" directory to facilitate debugging. On the autobuilder, this log should be available at https://autobuilder.yocto.io/pub/repro-fail/. - The last 20 lines of the log, which should show the failing recipes, will be displayed in the oe-selftest console. - If BitBake fails to create the deployment directory, it will be manually created to allow the comparison process to proceed. Here is what the output looks like when testing reproducibility of bash, a failing recipe (hello-fail) and a non-reproducible recipe (hello-norepro): <snip> 2024-10-01 23:09:33,977 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds) 2024-10-01 23:10:39,093 - oe-selftest - INFO - Non-reproducible packages will be copied to <snip>/yocto/repro_output/oe-reproducible-20241001-ng43_hzd 2024-10-01 23:10:39,094 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... 2024-10-01 23:18:12,378 - oe-selftest - ERROR - Bitbake failed! but keep going... Log: 2024-10-01 23:18:12,379 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1: 2024-10-01 23:18:12,379 - oe-selftest - INFO - 2024-10-01 23:18:12,379 - oe-selftest - INFO - ... (last 20 lines of output) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running task 976 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_package_qa) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 342 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx_setscene) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 343 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_spdx_setscene) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build) 2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 841 didn't need to be rerun and 1 failed. 2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Generating JSON CVE summary 2024-10-01 23:18:12,380 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/log/cve/cve-summary.json 2024-10-01 23:18:12,380 - oe-selftest - INFO - 2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: 1 task failed: 2024-10-01 23:18:12,380 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile 2024-10-01 23:18:12,380 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.3221257 2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code. 2024-10-01 23:18:12,380 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail. 2024-10-01 23:18:12,382 - oe-selftest - INFO - Building reproducibleB-extended (sstate NOT allowed)... 2024-10-01 23:46:58,451 - oe-selftest - ERROR - Bitbake failed! but keep going... Log: 2024-10-01 23:46:58,463 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1: 2024-10-01 23:46:58,463 - oe-selftest - INFO - 2024-10-01 23:46:58,463 - oe-selftest - INFO - ... (last 20 lines of output) 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: Running task 978 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx) 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_deb: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_rpm: Succeeded 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build) 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 2 didn't need to be rerun and 1 failed. 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Generating JSON CVE summary 2024-10-01 23:46:58,464 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/log/cve/cve-summary.json 2024-10-01 23:46:58,464 - oe-selftest - INFO - 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: 1 task failed: 2024-10-01 23:46:58,464 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile 2024-10-01 23:46:58,464 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.4136075 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There were 5 WARNING messages. 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code. 2024-10-01 23:46:58,467 - oe-selftest - ERROR - reproducibleB-extended build failed. Trying to compute built packages differences but the test will fail. 2024-10-01 23:46:58,481 - oe-selftest - INFO - Checking deb packages for differences... 2024-10-01 23:46:58,586 - oe-selftest - INFO - Reproducibility summary for deb: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,588 - oe-selftest - INFO - Checking ipk packages for differences... 2024-10-01 23:46:58,658 - oe-selftest - INFO - Reproducibility summary for ipk: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,659 - oe-selftest - INFO - Checking rpm packages for differences... 2024-10-01 23:46:58,691 - oe-selftest - INFO - Reproducibility summary for rpm: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,692 - oe-selftest - INFO - Running diffoscope 2024-10-01 23:46:59,765 - oe-selftest - INFO - ... FAIL 2024-10-01 23:46:59,766 - oe-selftest - INFO - Traceback (most recent call last): File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds self.fail('\n'.join(fails)) AssertionError: Bitbake reproducibleA failure Bitbake reproducibleB-extended failure The following deb packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb The following ipk packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk The following rpm packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm 2024-10-01 23:46:59,769 - oe-selftest - INFO - ====================================================================== 2024-10-01 23:46:59,770 - oe-selftest - INFO - FAIL: test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds) 2024-10-01 23:46:59,770 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-01 23:46:59,770 - oe-selftest - INFO - Traceback (most recent call last): File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds self.fail('\n'.join(fails)) AssertionError: Bitbake reproducibleA failure Bitbake reproducibleB-extended failure The following deb packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb The following ipk packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk The following rpm packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm 2024-10-01 23:46:59,770 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-01 23:46:59,770 - oe-selftest - INFO - Ran 1 test in 2246.039s 2024-10-01 23:46:59,770 - oe-selftest - INFO - FAILED 2024-10-01 23:46:59,770 - oe-selftest - INFO - (failures=1) 2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS: 2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (2245.79s) 2024-10-01 23:47:03,203 - oe-selftest - INFO - SUMMARY: 2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2246.040s 2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0) => Test failed but hello-norepro is displayed as non-reproducible. The testresult.json contains: { "oeselftest_debian-12_qemux86-64_20240930000424": { "configuration": { <snip> }, "result": { "reproducible": { "files": { "package_deb": { "different": [ { "reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb", "test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb" } ], "different_excluded": [], "missing": [], "same": [ <snip> { "reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb", "test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb" }, <snip> ] }, "package_ipk": { <same as deb> }, "package_rpm": { <same as deb> } } }, "reproducible.ReproducibleTests.test_reproducible_builds": { "duration": 2146.5671875476837, "log": "Traceback (most recent call last):\n File \"<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py\", line 380, in test_reproducible_builds\n self.fail('\\n'.join(fails))\nAssertionError: Bitbake reproducibleA failure\nBitbake reproducibleB-extended failure\nThe following deb packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nThe following ipk packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk\nThe following rpm packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm\n", "status": "FAILED" }, "reproducible.rawlogs": { "log": "DIFFERENT: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nSAME: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb\n<snip>" } } } } => "reproducible.ReproducibleTests.test_reproducible_builds".status is correctly "FAILED" but the reproducibility of bash and hello-norepro is tested. (From OE-Core rev: c78cc753843d4199443052e8deb0c9c3b7e4b580) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Update the BB_HASHSERVE_UPSTREAMJose Quaresma2024-10-312-2/+2
| | | | | | | (From OE-Core rev: ec7b024725fc93b7fef881d41343c8b77d97a57c) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: Remove passing test cases from the exclude list in the Rust OE self-test.Deepesh Varatharajan2024-10-291-3/+0
| | | | | | | | | | | | | | | | | There were some rust tests which were FAILing only for mips architectures. Hence, they were added to the exclusion list to avoid the failures when running rust oe-selftest for mips architectures.The current yocto framework disabled the rust oe-selftest for mips architectures.Hence, these tests have been added back to the test framework. The rust-1.79 has been successfully tested on X86, X86_64, ARM, AARCH64 and PPC Architectures with the current test inclusions. (From OE-Core rev: e1ec1ad75feb1321228bf3b52251e1aecfdb8746) Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: add test for WIC_SECTOR_SIZEVince Chang2024-10-291-0/+50
| | | | | | | | | | Add test for WIC_SECTOR_SIZE=4096. Verified it on ext4 file system. (From OE-Core rev: 69a1f18ce18d67881d7379ccc19ca001a35acb68) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* do_package/sstate/sstatesig: Change timestamp clamping to hash output onlyRichard Purdie2024-10-281-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code was changing the timestamps of the files in the do_package output, particularly the files added for debug sources. This was to do two things: a) make do_package sstate more reproducible b) ensure better hash equivalence matching Unfortuately the debug source files are hardlinks into the source tree for efficiency so touching these, touches a lot of files in ${B} and ${S}. This causes unpredictable effects if compile is run again for example, or could cause compiling in the install task. The hash equivalence matching is of key importance but we can mimic that using clamping of the file timestamps in the depsig output used to generate the hashes. This patch drops the global timestamp clamping, instead allowing the files to retain their creation timestamps into sstate. This makes do_package sstate slightly less reproducibile. We could clamp the sstate timestamps but that would lead to two different sets of timestamps depending on whether the data came from sstate or not. I'd prefer to have consistent code behaviour, rather than differing behavhour depending on whether data came from sstate or not. If we wanted to have reproducibiliy and fix the "corruption" of S/B and have consistent codepaths, the only other option would be two copies of the sources, which could end up huge and seems the least desireable option. This patch therefore drops the timestamp clamping in the sstate files and tweaks the depsig data generation to clamp the timestamps for do_package instead since this seems the best compromise. I validated that rpm/deb/ipk files still generate correctly as before. (From OE-Core rev: 475759fdab7200488b2a568b2ba1aa31a456d113) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/package: Copy locale licenseJoshua Watt2024-10-251-0/+4
| | | | | | | | | | | When creating split locales, copy the license from LICENSE:${PN}-locale if set, otherwise leave it unspecified (which will result in falling back to LICENSE) (From OE-Core rev: 34b844d3a920574d81affbdd2096ffd7dea14067) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/package: Check incompatible licenses at packaging timeJoshua Watt2024-10-251-3/+3
| | | | | | | | | | | | | | | | | | | Instead of checking for incompatible licenses in the anonymous python and setting '_exclude_incompatible-', (re)check all the packages in populate_packages(). This ensures that all packages are processed, even dynamically generated ones. The use of the '_exclude-incompatible-' variable set in base.bbclass has been the mechanism used for per-packages licenses since it was added as a feature (although with different names for the variable throughout history). However, since this misses dynamic packages, calling oe.license.skip_incompatible_package_licenses() a second time on the actual final package set is a better solution. (From OE-Core rev: 1816c5a5e724a7f907c9afe4a47b6699ea081ace) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/license: Move package license skip to libraryJoshua Watt2024-10-251-0/+39
| | | | | | | | | | Moves the code that skips packages with incompatible licenses to the library code so that it can be called in other locations (From OE-Core rev: 86eb409e3c1b30110869ec5a0027ae2d48bbfe7f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes-global/license: Move functions to library codeJoshua Watt2024-10-251-0/+163
| | | | | | | | | | | | | | Moves several of the functions in license.bbclass to be library code New function dependencies were manually verified using bitbake-dumpsigs to ensure that bitbake identified the same dependencies even though they are now in library code (although the new function names mean that the task hashes still change) (From OE-Core rev: 0333e04e353991260c5f67a72f80f3ab9dcf526a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible: fix typoGaël PORTAY2024-10-251-1/+1
| | | | | | | | | | This fixes a typo in comment. (From OE-Core rev: ab861003e819389849889ba2d4e60e40cd7a7ed9) Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest wic.py: apply build config to runqemu tooMikko Rapeli2024-10-251-1/+1
| | | | | | | | | | | | | Build configuration is applied to bitbake build command but removed before calling runqemu. Thus disabling KVM support on aarc64 host was not effective. Note that this pattern is used in a lot of tests. KVM gets enabled via CI scripts. (From OE-Core rev: 7a9fbf509fec2e149fd5702552bc4d819969ab1f) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest wic.py: fix missing ext4 imageMikko Rapeli2024-10-251-0/+4
| | | | | | | | | | | | test_rawcopy_plugin fails if machine does not build ext4 images by default. Make the ext4 image build explicit in the test. Fixes test on genericarm64 machine which defaults to wic image only. (From OE-Core rev: 0344b9e7949ef854b09deb0874a45cde2861f55b) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest wic.py: support UKIs via uki.bbclassMikko Rapeli2024-10-251-12/+108
| | | | | | | | | | | | | | | | | Use label to detect rootfs since UKI with kernel command line is generated before rootfs is generated by wic. Adapt wic tests to build and boot uki.bbclass generated UKIs. Keeping one UKI test in wic.py and rest of the UKI features are tested with dedicated uki.py test. Add plain non-UKI systemd-boot tests to wic suite for aarch64 and x86. (From OE-Core rev: 3f94256b977637d4276f82db7c20b8b5e57b9d86) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest wic.py: add TEST_RUNQEMUPARAMS to runqemuMikko Rapeli2024-10-251-6/+10
| | | | | | | | | | | | | | | | To support "slirp" networking on shared build machines instead of tun/tap devices. Users can set TEST_RUNQEMUPARAMS = "slirp" in their build/conf/local.conf to run selftests using "slirp" networking. The same works for testimage.bbclass and oeqa runtime tests. (From OE-Core rev: 4974ec71367492ce314da63c359ccf99acfca882) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest efibootpartition.py: remove systemd-boot from grub-efi testMikko Rapeli2024-10-251-2/+2
| | | | | | | | | | | The test is actually using grub-efi not systemd-boot so remove it completely. systemd-boot will be tested via uki.py tests. (From OE-Core rev: 542ea853bb194629d23e1db9c05e25181bfce145) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest efibootpartition.py: add TEST_RUNQEMUPARAMS to runqemuMikko Rapeli2024-10-251-3/+4
| | | | | | | | | | | | TEST_RUNQEMUPARAMS variable is used to add runqemu parameters like "slirp" networking. Support this also in selftests so that "slirp" networking can be used instead of the tun/tap devices setup which is easier to work with on shared build machines. (From OE-Core rev: 74e44e63378e1c08bb547a0a04428d88753b5040) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa selftest uki.py: add tests for uki.bbclassMikko Rapeli2024-10-251-0/+142
| | | | | | | | | | | | | Tests builds and boots qemu into uki binary with systemd and sysvinit. Due to depedency to x86 specific ovmf UEFI firmware, tests are specific to x86 curently. UEFI firmware for ARM can be generated via qemuarm64-secureboot machine in meta-arm and similar tests on qemu will pass. (From OE-Core rev: 8a3cb17876dbcaf07696a4bcd454e2f9a444fb1b) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/buildoptions: Use continue option in source mirrors testRichard Purdie2024-10-251-1/+1
| | | | | | | | | It is helpful to see all the issues in the source mirror test so use the continue option. (From OE-Core rev: f52e7ddc6ca5ce03cbcdf3cc15fc04f3ff31cd5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/rust: remove unused imports or codeRoss Burton2024-10-241-5/+1
| | | | | | | | | Remove unused imports, or disabling code which has been commented out. (From OE-Core rev: e0ff4813b1cf4df0d851c857d57fb88d7db51bdd) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/rust: skip on all MIPS platformsRoss Burton2024-10-241-5/+3
| | | | | | | | | | | | | | | | | | | As per "The rustc book"[1], mips*-unknown-linux-* are "tier 3" targets: Tier 3 targets are those which the Rust codebase has support for, but which the Rust project does not build or test automatically, so they may or may not work. We already skip qemumips in this selftest, but we're now also seeing failures with qemumips64 so refactor the test to skip all machines where the architecture is mips or mips64. [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html (From OE-Core rev: 177e268811c04260923ac4b16fa047315304add0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: add test_commit_message_user_tagsTrevor Gamblin2024-10-234-0/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | This test makes patchtest check to ensure that there aren't any GitHub-style user account names being tagged in the commit message, e.g. it should catch lines like: "fix added by @threexc" This is desired so that if (for example) we add upstream changelogs in recipe upgrade commit messages verbatim, we don't end up subscribing any associated maintainers to our repo mirrors' updates by accident. There is a small possibility of a false positive with this test, where if someone is mentioning Python decorators in their commit message (or similar syntax from other languages), it will fail when it should pass. However, having this test in place to guard against username inclusion is more important that the occasional false positive for that reason. With this addition, a failure will look like: |FAIL: test commit message user tags: Mbox includes one or more GitHub-style username tags. Ensure that any "@" symbols are stripped out of usernames (test_mbox.TestMbox.test_commit_message_user_tags) (From OE-Core rev: 8278d82d8203212bb159eb3805d4a5617c5370df) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust-target-config: Update the data layout for x86-64 targetDeepesh Varatharajan2024-10-221-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | The data layout for x86-64 target was different in rust from llvm. It resulted in the following data layout error: -------------------------------------------------------------------------------------- error: data-layout for target `x86_64-linux`, `e-m:e-i64:64-f80:128-n8:16:32:64-S128`, differs from LLVM target's `x86_64-unknown-linux-gnu` default layout, `e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128` -------------------------------------------------------------------------------------- The issue was reported in meta-rust layer as: https://github.com/meta-rust/meta-rust/pull/442 Hence, the data layout for x86-64 is modified as per the llvm sources which fixes the issue. After running the testsuite with the modified data layout; it was observed that the "tests/codegen/i128-x86-align.rs" passed. The modified data layout fixed the scalar pair which resolved the testcase failure. Hence, the testcase has been removed from the exclude list. (From OE-Core rev: 96f11d299b32a32086eb3d890c88eb7ce42057b0) Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: Add different PCI BAR errors with qemu 9.1.0Richard Purdie2024-10-221-0/+11
| | | | | | | | | With qemu 9.1.0 we see parselogs failures with mips and mips64. We've long since ignored these errors so update to continue to do so with the new form. (From OE-Core rev: 4f3c299e7a282e953d9adb7549c30015df317171) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* spdx30: Link license and build by aliasJoshua Watt2024-10-152-22/+31
| | | | | | | | | | | | | The license information and Build created by do_create_spdx are changed to be referenced by their link alias instead of the actual SPDX ID. This fixes a case where do_create_package_spdx would pull these from mismatching sstate, and then the SPDX IDs would be unresolved when assembling the final document (From OE-Core rev: c0fcdc72a7c8fca86a874d1b04298fe9e500c796) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Don't used cachedpath for os.lstat()Richard Purdie2024-10-141-0/+4
| | | | | | | | | | | The functions behave slightly differently to the functions they're caching and the use in insane.bbclass isn't compatible. For now, to avoid build failures, switch back to the stat calls. We may be able to improve cachedpath or change the call sites. (From OE-Core rev: fa771ae887ab5152f043748cf3419735831bcf7b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ssh: Fix incorrect timeout fixRichard Purdie2024-10-141-2/+2
| | | | | | | | | I'd meant to change the command timeout in the previous change, fix the correct one. (From OE-Core rev: bb991988cb23be2c8947171726ada321f27e6eed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Clean up serial_lock handlingRichard Purdie2024-10-141-5/+6
| | | | | | | | | | Avoid "RuntimeError: release unlocked lock" since the lock shouldn't be locked even in the error path. Add a try/finally path to ensure this. (From OE-Core rev: b0732ee009ca47580d1d2ad75334f4aa50e6efd5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/cases: add basic barebox testsEnrico Jorns2024-10-111-0/+44
| | | | | | | | | | | | | | | | | | | This adds basic tests for qemuarm and qemuarm64. So far, barebox fails to run properly under KVM for the same reasons u-boot fails to run. A patch series to address the problem was submitted by Ahmad Fatoum after debugging the oe-selftest failures for this series: https://lore.kernel.org/barebox/20241009060511.4121157-1-a.fatoum@pengutronix.de/ For now, simply disable KVM for these tests. (From OE-Core rev: 9284ceb4d32a51c77792d9009bba400d0b17d731) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/cases: add basic u-boot testEnrico Jorns2024-10-111-0/+43
| | | | | | | | | | | | | | | | | | This adds basic tests for qemuarm and qemuarm64. So far, U-Boot fails to run properly under KVM since this requires some special care with instructions used for MMIO accesses. Reported upstream by Ahmad Fatoum after debugging the oe-selftest failures for this series: https://lore.kernel.org/u-boot/2211f9f0-cd7d-4b55-860d-a34c04877e7b@pengutronix.de/ For now, simply disable KVM for these tests. (From OE-Core rev: cadbd937e6358f9811f3ba7cf20cc50f8edcd844) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: support passing custom boot patterns to runqemuEnrico Jorns2024-10-113-5/+7
| | | | | | | | | | | | | This allows defining non-standard patterns from QEMU tests just as they are already supported by the testimage.bbclass. Will allow testing non-Linux shells in QEMU, too (e.g. a U-Boot shell or another bootloader shell). (From OE-Core rev: f9e6c3deee5a4159a09396b625a7327f6c204df5) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: support ignoring vt100 escape sequencesEnrico Jorns2024-10-111-1/+3
| | | | | | | | | | | | | | | | | | | | If we talk to terminals that like colors, we need to ignore the vt100 escape sequences when matching strings. An unprocessed barebox console prompt would e.g. look like: ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m where we cannot match for something like "barebox@ARM QEMU virt64:/". The same applies to colored Linux terminal output of course. The "\x1b\[" from the regex catches the standard start of ANSI escape sequence while the rest catches the actual command code executed. (From OE-Core rev: 33bbe4cb040f890121681865fbcf28bc8213a170) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* barebox: add initial supportMarco Felsch2024-10-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds the support for the barebox bootloader to oe-core. The recipe is inspired by meta-ptx [1] but is a major rework of the one found there. Barebox comes with a wide range of supported architectures and follows the concepts of Linux in various aspects like the driver model, the shell, or the virtual file system. This not only eases porting Linux drivers but also makes barebox a developer-friendly and feature-rich bootloader alternative [2]. For barebox (like for the kernel or other bootloaders) it is quite likely that people will not just build the original recipe but need to adapt it, point to custom repositories, apply patch stacks, COMPATIBLE_MACHINE etc. They may also choose to have different recipe names for different variants. Having only a single .bb file and requiring to copy or .bbappend it is inconvenient and results in unnecessary code duplication. Therefore, the base support for building barebox is encapsulated in barebox.bbclass (like kernel.bbclass for the kernel). Adds barebox to maintainers.inc but excludes it from the maintainers check since with the current check mechanism barebox would be skipped (and making the check fail) due to not being the PREFERRED_PROVIDER for virtual/bootloader. [1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox [2] https://www.barebox.org/demo/?graphic=0 (From OE-Core rev: 5c69f5626278a6e9756188a5771b18075380f52d) Signed-off-by: Marco Felsch <m.felsch@pengutronix.de> Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package-manager: skip processing installed-pkgs with empty globsClaus Stovgaard2024-10-111-39/+37
| | | | | | | | | | | | | | | | We can skip processing the installed-pkgs file if globs is empty. This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY returns an empty string. If globs is an empty string the result from processing with empty glob in oe-pkgdata-util will always be 0 packages to install. Instead of return early on this we just skip and still generate the locale archive if needed. (From OE-Core rev: 160c45c83d5addf01e4834cf896af871bd6fca7f) Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/elf: don't regenerate machine data on every callRoss Burton2024-10-111-131/+133
| | | | | | | | | | | | | | | | Every time oe.elf.machine_dict() is called a large dictionary is created and returned. However, the "arch" package_qa test will call this method for every binary in a package, which results in a lot of dictionary creation. Concrete exmaple: in running ltp:do_package_qa, the arch test takes 25% of the runtime, and opitimising the machine_dict() call to cache the generated dictionary reduces the runtime from 57s to 44s. (From OE-Core rev: dca4455aa4f870bad900a0214c4e95e09a055295) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>