summaryrefslogtreecommitdiffstats
path: root/meta/classes
Commit message (Collapse)AuthorAgeFilesLines
* spdx: add option to include only compiled sourcesDaniel Turull2025-07-071-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | When SPDX_INCLUDE_COMPILED_SOURCES is enabled, only include the source code files that are used during compilation. It uses debugsource information generated during do_package. This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled. As example, when used with the default config with linux-yocto, the spdx size is reduced from 156MB to 61MB. Tested with bitbake world on oe-core. (From OE-Core rev: c6a2f1fca76fae4c3ea471a0c63d0b453beea968) Adapted to existing files for create-spdx-2.2 CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> CC: Joshua Watt <JPEWhacker@gmail.com> (From OE-Core rev: a2866934e58fb377a73e87576c8594988a63ad1b) Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check.bbclass: Fix symlink handling also for text filesNiko Mauno2025-05-191-2/+3
| | | | | | | | | | | | | | | | | | | | | | While backporting commit 7543e4e24a8b ("cve-check.bbclass: Mitigate symlink related error") I failed to address the fact that in Scarthgap this bbclass still generates also the text CVE file, in addition to the JSON file. In order to avoid the error that occurred with JSON files from occurring now with CVE files: ERROR: core-image-base-1.0-r0 do_image_complete: Recipe core-image-base is trying to install files into a shared area when those files already exist. The files and the manifests listing them are: /home/poky/build/tmp/deploy/images/qemux86-64/.cve (matched in manifest-qemux86_64-core-image-minimal.image_complete) Please adjust the recipes so only one recipe provides a given file. update the symlink handling for the text file variant too. (From OE-Core rev: 81e702c85c62782dbedd5ca2a5e6569e6339dd60) Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check.bbclass: Mitigate symlink related errorNiko Mauno2025-03-151-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to Yocto reference manual, in description of the IMAGE_LINK_NAME variable, it is said that It is possible to set this to "" to disable symlink creation, however, you also need to set :term:`IMAGE_NAME` to still have a reasonable value e.g.:: IMAGE_LINK_NAME = "" IMAGE_NAME = "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_VERSION_SUFFIX}" However, when using following additions in local.conf file: INHERIT += "cve-check" IMAGE_LINK_NAME = "" IMAGE_NAME = "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_VERSION_SUFFIX}" the implicit symlink creation in cve_check_write_rootfs_manifest leads to following build failure $ bitbake core-image-minimal core-image-base ... ERROR: core-image-base-1.0-r0 do_image_complete: Recipe core-image-base is trying to install files into a shared area when those files already exist. The files and the manifests listing them are: /home/poky/build/tmp/deploy/images/qemux86-64/.json (matched in manifest-qemux86_64-core-image-minimal.image_complete) Please adjust the recipes so only one recipe provides a given file. Mitigate the issue by creating the symlink only in case IMAGE_LINK_NAME has not been set to empty string. (From OE-Core rev: 64bfec359bd909761ce0a6a716286d938ed162d1) (From OE-Core rev: 35e210e3ebe21f1f4466760076b7140618af536e) Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve_check: Use a local copy of the database during buildsRichard Purdie2024-11-091-3/+4
| | | | | | | | | | | Rtaher than trying to use a sqlite database over NFS from DL_DIR, work from a local copy in STAGING DIR after fetching. (From OE-Core rev: 57de6545695ac11816d670959d9e63666de08e3d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 03596904392d257572a905a182b92c780d636744) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check: add support for cvss v4.0Peter Marko2024-11-091-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | https://nvd.nist.gov/general/news/cvss-v4-0-official-support CVSS v4.0 was released in November 2023 NVD announced support for it in June 2024 Current stats are: * cvss v4 provided, but also v3, so cve-check showed a value sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 != 0.0; 2069 * only cvss v4 provided, so cve-check did not show any sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 = 0.0; 260 (From OE-Core rev: 358dbfcd80ae1fa414d294c865dd293670c287f0) (From OE-Core rev: 290407b3785bce2d22212a7ab9d3a349c8935cc0) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrcMark Hatle2024-09-251-1/+2
| | | | | | | | | | | | | | | | While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. (From OE-Core rev: cc24c32795e6894387a6e7ebc9b1d9f4215621f0) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 02e262c291c0b2066132b4cb2ca5fda8145284a9) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usageRichard Purdie2024-09-251-20/+19
| | | | | | | | | | | | | | | | | | We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of postfuncs. Finally get around to doing that which should make the buildhistory code a little more readable. Unfortunately ordering the buildhistory function calls after the sstate ones is difficult without coding that into the sstate class. This patch does that to ensure everything functions as expected until we can find a better way. This is still likely preferable than the generic sstate postfuncs support since the function flow is much more readable. (From OE-Core rev: 466c505b779dec2ba790f4e6cde7fbb35037f4ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c9e2a8fa2f0305ef1247ec405555612326f798f8) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* buildhistory: Restoring files from preserve listPedro Ferreira2024-09-251-0/+23
| | | | | | | | | | | | | | | This fix will ensure that, when we activate feature `BUILDHISTORY_RESET`, files marked to keep on feature `BUILDHISTORY_PRESERVE` will indeed exist is buildhistory final path since they are moved to buildhistory/old but not restored at any point. (From OE-Core rev: 93ee5b0ee71a51daba9a332e8dba93d78a849677) Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9f68a45aa238ae5fcdfaca71ba0e7015e9cb720e) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* buildhistory: Fix intermittent package file list creationPedro Ferreira2024-09-251-6/+3
| | | | | | | | | | | | | | | | | | | | The directory that buildhistory_list_pkg_files writes to during do_package is created by do_packagedata so a clean buildhistory doesn't have files-in-package written during the first build since packagedata happens after do_package. Ensure the output package folder is created to avoid missing files-in-package.txt files. Also it ensures that in case of `find` fails we leave with a hard error instead of hiding the error on the for loop. (From OE-Core rev: eb94b09a9183e0b0d9cfc45287e0967ae185c099) Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8de9b8c1e199896b9a7bc5ed64967c6bfbf84bea) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHSMark Hatle2024-08-061-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a create-spdx-* classes is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib. When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as: ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found for package libilp32-libgcc-dbg, False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12: sstate:libilp32-libgcc::14.1.0:r0::12: Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-* to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations. (From OE-Core rev: 5c1ce317fff6df6818f72d93197e5ec59ad4c462) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f1499c36c1054fc90f7b7268cc95285f2eca72f7) spdx-3.0 items are not application and were removed. spdx-common.bbclass item was moved into create-sdpx-2.2.bbclass. Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* archiver.bbclass: Fix work-shared checking for kernel recipesBenjamin Szőke2024-08-011-1/+2
| | | | | | | | | | | | | | Source dir can be a symbolic link in some BSP's linux kernel recipe which points to work-shared path (like linux-fslc in meta-freescale). Change to use os.path.realpath() in order to get real path of source dir. (From OE-Core rev: 340dc094df5eda4a3834a90578b331d9edcffa94) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9191aa685418af32f003e067ef7c5737a271e3a5) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* multilib.bbclass: replace deprecated e.data with dKai Kang2024-08-012-56/+56
| | | | | | | | | | | | | | | | Replace deprecated e.data with d in multilib.bbclass and multilib_global.bbclass. Remove event check in function multilib_virtclass_handler_vendor in multilib_global.bbclass. The function flag 'eventmask' has been set with 'bb.event.ConfigParsed', so no need to check the event any more. (From OE-Core rev: 01a84c3db74e8cc4923016c312a484a252fcfee0) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8d87662d676a2b9ef921cb49ea6edc07b37410e2) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check: Introduce CVE_CHECK_MANIFEST_JSON_SUFFIXAleksandar Nikolic2024-08-011-2/+4
| | | | | | | | | | | | | | | The variable contains the suffix of the CVE JSON manifest file. By default, this variable is set to 'json', so the current behavior is not changed, but enables developers to use some other suffix, e.g., cve.json (similar to spdx.json). (From OE-Core rev: 0cb103430d0505a3cd135e727379489bc3fe6e46) Signed-off-by: Aleksandar Nikolic <an010@live.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d99eee76923659c0b95bf9ef415ae5d44f736d01) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* classes/create-spdx-2.2: Fix SPDX Namespace Prefixjoshua Watt2024-07-031-1/+1
| | | | | | | | | | | | | | According to the SPDX documentation, it should be "spdxdocs" not "spdxdoc" [YOCTO #15398] (From OE-Core rev: 9abcb18014020804738dfc7d278d7097679f4d19) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d1b25413ced62dc2927dae57b8d16e67d15dc220) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check: Log if CVE_STATUS set but not reported for componentSimone Weiß2024-02-241-0/+3
| | | | | | | | | | | Log if the CVE_STATUS is set for a CVE, but the cve is not reported for a component. This should hopefully help to clean up not needed CVE_STATUS settings. (From OE-Core rev: 013d531a84fa08b6ae8a47bdf3ba1fa8f18ba270) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd.bbclass: Fix order of postinst-useradd-*Piotr Łobacz2024-02-241-26/+45
| | | | | | | | | | | | | | | | | | | | | | | | | postinst-useradd-* haven't been running in order of dependency. This patch is reworked from Piotr Łobacz's patch and fixes: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904 basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-* scripts with any order. Sometimes this means, for example a useradd postinst will attempt to run without the corresponding group postinst causing errors. This patch ensures that we first run groupadd, then useradd and then group membership. [RP: Tweaked to avoid removing previous fixes and for whitespace/style issues Also ensure the scripts are changed to execute with -e to highlight errors] (From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> Signed-off-by: Jan Górski <j.gorski@welotec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd.bbclass: Fix missing space when appending vardeps.Siong W.LIM2024-02-191-1/+1
| | | | | | | (From OE-Core rev: c4658401f1331026dc47a859dd665aab74918bba) Signed-off-by: Siong <wslim87@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multilib_global.bbclass: fix parsing error with no kernel module splitChen Qi2024-02-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | The problem could be reproduced with the following settings: MACHINE = "qemux86-64" KERNEL_SPLIT_MODULES = "0" require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32" The error message is as below: bb.data_smart.ExpansionError: Failure expanding variable KERNEL_VERSION_PKG_NAME, expression was ${@legitimize_package_name(d.getVar('KERNEL_VERSION'))} which triggered exception TypeError: expected string or bytes-like object The variable dependency chain for the failure is: KERNEL_VERSION_PKG_NAME -> RPROVIDES:kernel-modules This is because multilib_virtclass_handler_global function in multilib_global.bbclass deletes KERNEL_VERSION. So we need to handle such situation. We'll also need to delete KERNEL_VERSION_PKG_NAME to avoid this parsing error. (From OE-Core rev: 43dd497bc161ac44faecfdff052db03679dbb4f8) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDSAlexander Kanavin2024-01-271-0/+1
| | | | | | | | | | | | | | | | Otherwise, PACKAGE_WRITE_DEPS would contain non-multilib variants of dependencies even when building multilib items, resulting in sysroots being populated with entirely wrong versions of them. This hasn't been noticed until now through sheer (bad) luck, I think, except in the cpio recipe, but the previous commit shows that the issues did occur, quietly. Every other recipe in oe-core and meta-oe does not prepend the multilib prefix. (From OE-Core rev: 234965cb88ccfa9c3a357928f7155b119044e8fc) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sysroot user management postinsts: run with /bin/sh -e to report errors when ↵Alexander Kanavin2024-01-271-1/+1
| | | | | | | | | | | | | | | | | | | they happen This exposes the following failure in a multilib setup, when everything up to do_package_write_rpm is in sstate, but do_package_write_rpm is not (there's a similar fail for lib64-man-db, and the failures themselves will be fixed separately in the next commit): Exception: subprocess.CalledProcessError: Command '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot/usr/bin/postinst-base-passwd' returned non-zero exit status 1. Subprocess output: install: cannot stat '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/share/base-passwd/passwd.master': No such file or directory (From OE-Core rev: 5ffa333db28bc5d8e440c983fdf95589d332461d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* externalsrc: fix task dependency for do_populate_licJulien Stephan2024-01-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | do_populate_lic dependencies are defined inside license.bbclass such as: addtask populate_lic after do_patch before do_build but externalsrc deletes the do_patch task, so the only dependency left for do_populate_lic is "before do_build" On a devtool context, when doing devtool modify, sources are extracted inside build/workspace/sources/${BPN}/ and local files inside build/workspace/sources/${BPN}/oe-local-files When building the recipe after a devtool modify, do_unpack is called again to unpack (possibly modified) local files from build/workspace/sources/${BPN}/oe-local-files into ${WORKDIR}. Since the only left dependency for do_populate_lic is do_build, the do_populate_lic can be called BEFORE do_unpack. Most of the time this is not a problem, because license files are generally located inside ${S}, which corresponds to build/workspace/sources/${BPN} (and is already unpacked), but this can lead to an issue if recipe sets LIC_FILES_CHKSUM to look for files in ${WORKDIR} (example from init-ifupdown_1.0.bb): LIC_FILES_CHKSUM = "file://${WORKDIR}/copyright;md5=3dd6192d306f582dee7687da3d8748ab" So devtool modify init-ifupdown && bitbake init-ifupdown gives the following error: WARNING: init-ifupdown-1.0-r0 do_populate_lic: Could not copy license file <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright to <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/license-destdir/qemux86_64/init-ifupdown/copyright: [Errno 2] No such file or directory: '<...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright' ERROR: init-ifupdown-1.0-r0 do_populate_lic: QA Issue: init-ifupdown: LIC_FILES_CHKSUM points to an invalid file: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright [license-checksum] ERROR: init-ifupdown-1.0-r0 do_populate_lic: Fatal QA errors were found, failing task. ERROR: Logfile of failure stored in: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/temp/log.do_populate_lic.838584 ERROR: Task (<...>/poky/meta/recipes-core/init-ifupdown/init-ifupdown_1.0.bb:do_populate_lic) failed with exit code '1' Fix this by forcing the do_populate_lic task to run after do_unpack (From OE-Core rev: ea6a0cccdd274534809df62a0a196bf83489a1e5) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Unlink vendor dir laterVyacheslav Yurkov2024-01-191-1/+1
| | | | | | | | | | Vendor directory might still be required during install stage, so defer the removal until later stage. (From OE-Core rev: f4538e3884a0cfa07e16e6a2c986271cc7322dd8) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Handle modules from the same repoVyacheslav Yurkov2024-01-191-5/+4
| | | | | | | | | | | Take into account module version when populating vendor directory, because a module with the same URL but with a different version tag could be used as an indirect dependency. (From OE-Core rev: 8f6320c0858941b2441e290ef3586b48c2700cd1) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Reference local modulesVyacheslav Yurkov2024-01-191-1/+17
| | | | | | | | | | | Create symlinks for local modules, which are usually not referenced in the SRC_URI, but still expected to be found in the vendor directory during the build. (From OE-Core rev: 16da5d9ad448aafd8b5fd63480727bd1b09ec9f1) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/recipes: Switch to use inherit_deferRichard Purdie2024-01-181-1/+1
| | | | | | | | | | | | | | | | | Now that bitbake supports the use of inherit_defer, switch all conditional (variable based) inherits to use this instead. This leads to more a more deterministic user experience since there is no longer an immediate expansion and later changes to the variables in question (e.g. a bbappend) are accounted for. This patch tries to ensure the behaviour before/after remains as unchanged as it reasonably can, e.g. by always inherting populate_sdk_base. native and nativesdk continue to need to be inherited last, hence being used with inherit_defer in a handful of very specific cases. (From OE-Core rev: 451363438d38bd4552d5bcec4a92332f5819a5d4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* chrpath.bbclass: fix Darwin supportEtienne Cordonnier2024-01-151-1/+1
| | | | | | | | | | | | | | | Ported from the meta-darwin layer: The call to out.split("\n") expects a string, thus the parameter text=True is needed (otherwise Popen returns a bytes object). Note that "text" is just a more readable alias for universal_newlines. (From OE-Core rev: 0abaa7bf7f7d9a5ac96e6fdbe99334cb2fb0e4db) Signed-off-by: Dominik Schnitzer <dominik@snap.com> Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Drop tarfile module to improve performanceRobert Yang2023-12-231-12/+16
| | | | | | | | | | | | | | | | | | | * The tarfile module doesn't support xz options or environment varible XZ_DEFAULTS, this makes do_ar_patched incrediblely slow when the file is large, for example, chromium-x11 is about 3GB: - "bitbake chromium-x11 -car_patched" hasn't been done after 3 hours on my host, I checked the partial tar.xz file is only 1.5GB, so maybe more than 6 hours is required to complete the task. - Now only less than 4 minutes is needed on the same host. * Need add xz to HOSTTOOLS when archiver.bbclass is enabled and compression is xz. (From OE-Core rev: 6548354f049b173e8d443bc547d35c9d9fc05259) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-2.2: combine spdx can try to write before dir creationJeremy A. Puhlman2023-12-171-1/+1
| | | | | | | | | | | | | | | | | | | On occasion a file is attmpeded to be opened prior to the creation of the spdx_workdir. Create the directory before the open, just in case. File: '/build/layers/poky/meta/classes/create-spdx-2.2.bbclass', lineno: 1081, function: combine_spdx 1077: ) 1078: 1079: image_spdx_path = spdx_workdir / (rootfs_name + ".spdx.json") 1080: *** 1081: with image_spdx_path.open("wb") as f: 1082: doc.to_json(f, sort_keys=True, indent=get_json_indent(d)) (From OE-Core rev: bb9f2a9c0ff5dcdeaf1a0beb6a614d0d022a2481) Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Improve work-shared checkingRobert Yang2023-12-131-4/+2
| | | | | | | | | | | | There are other recipes except the listed ones which use work-shared, improve the checking to make other recipes such as llvm-project-source work with do_ar_patched. (From OE-Core rev: 5fbb4ca8da4f4f1ea426275c45634802dcb5a575) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd: Fix useradd do_populate_sysroot dependency bugRichard Purdie2023-12-101-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | If a task is adde which has a dependency on the do_populate_sysroot task of the recipe, it will cause it to be installed into the sysroot (similar to do_addto_recipe_sysroot). This fails since the postinst script is an overlapping file: Exception: FileExistsError: [Errno 17] File exists: 'tmp/sysroots-components/all/useraddbadtask/usr/bin/postinst-useradd-useraddbadtask' -> 'tmp/work/all-poky-linux/useraddbadtask/1.0/recipe-sysroot/usr/bin/postinst-useradd-useraddbadtask' The copy written out at do_prepare_recipe_sysroot time is just for debug so rename it, meaning there are no longer overlapping files and the installation can be successful, removing the error. [YCOTO #14961] With the bug fixed, enable the test. (From OE-Core rev: 564339afb73fc52a66c1a08437587cad1c4d46e7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd: Fix issues with useradd dependenciesEilís 'pidge' Ní Fhlannagáin2023-12-081-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | If recipe A requires the useradd actions of recipe B we need to ensure that recipe B is part of the recipe A dependancy chain. In order to do that, we introduce USERADD_DEPENDS. This makes sure that the do_populate_sysroot_setscene of recipe B exists for recipe A in case of a missing TMPDIR. This requires changes made in runqueue.py by RP. This commit along with the runqueue fixes effects: Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens) Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails Bug 13279 - Make sure users/groups exist for package_write_* tasks Bug 15084 - For some reason using of same user in two recipes does not work properly I've included the start of self-testing for useradd by adding tests for 13419 (which ends up testing 13904, 13279, 15084 by virtue of them all having the same root cause) (From OE-Core rev: b47f2352376bd16b7e7087b4dab143403e67e094) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "cve-check: Modify judgment processing using "=" in version comparison"Ross Burton2023-12-051-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change introduced a warning if version comparisons failed, but this is far too common an issue in data that we don't control, so this shouldn't cause a warning: WARNING: automake-native-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2011-0715 WARNING: automake-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2003-0577 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-0982 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-1284 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2006-3355 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2007-0578 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2007-0578 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2009-1301 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2009-1301 This reverts commit a1989e4197178c2431ceca499e0b4876b233b131. (From OE-Core rev: c7c7dbdd5474002cfd9ec24864e77a0df2b790ea) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Modify judgment processing using "=" in version comparisonMatsunaga-Shinji2023-12-021-0/+1
| | | | | | | | | | | | | | | | Judgment processing of vulnerable using "=" compares characters as strings rather than numbers, and misjudges "cases that do not match in strings but do match in numbers" as "Patched". (e.g. PV = "1.2.0" and Vulnerabilities Affected Versions (registered with NVD) = "1.2") Therefore, if the comparison operator used in the judgment processing of vulnerable is "=", add numeric comparison processing. (From OE-Core rev: a1989e4197178c2431ceca499e0b4876b233b131) Signed-off-by: Shinji Matsunaga <shin.matsunaga@fujitsu.com> Signed-off-by: Shunsuke Tokumoto <s-tokumoto@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: tag all submodulesJulien Stephan2023-12-011-0/+3
| | | | | | | | | | | | In the case of a repository with submodules, we need to add the "devtool-base" and "devtool-patched" tag on all submodules in order to properly detect the added/removed/modified patches (From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd_base: Fix sed command line for passwd-expireAdam Johnston2023-11-211-1/+1
| | | | | | | | | | | | | | | | | | | | | A previous commit tried to add the --follow-symlinks option to the perform_passwd_expire function in useradd_base.bbclass, however it used a single -. This is interpreted as --file=ollow-symlinks which results in... sed: couldn't open file ollow-symlinks: No such file or directory and... ERROR: <image name>: passwd --expire operation did not succeed. Fix by adding the missing - (From OE-Core rev: 3c0deafcfcea3f610c7dd9a2d2884a16fbfe0497) Signed-off-by: Adam Johnston <adam.johnston@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sed -i destroys symlinksJoakim Tjernlund2023-11-201-1/+1
| | | | | | | | | | | | If /etc/passwd is a symlink, sed -i on same file will replace the symlink with a new file. Prevent that by adding --follow-symlinks option to sed (From OE-Core rev: 6ec004b2e7b4342465af8e5e6cc66041834821a0) Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* go-vendor: Minor style tweaksRichard Purdie2023-11-081-6/+2
| | | | | | | | | Drop a len() usage that isn't needed, drop a comment that isn't needed and use bb.fatal() to end the task with an error. (From OE-Core rev: 1b9cf9cc2a52902e1181d2dac2ba9f2c3172835a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Add go-vendor classLukas Funke2023-11-061-0/+200
| | | | | | | (From OE-Core rev: d61bdf392e10140671ca56f2a2b0dc824be8ab80) Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: sort the package list in the JSON reportRoss Burton2023-10-261-0/+2
| | | | | | | | | | | | | | The JSON report generated by the cve-check class is basically a huge list of packages. This list of packages is, however, unsorted. To make things easier for people comparing the JSON, or more specifically for git when archiving the JSON over time in a git repository, we can sort the list by package name. (From OE-Core rev: e9861be0e5020830c2ecc24fd091f4f5b05da036) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* externalsrc.bbclass: Support specifying patterns in CONFIGURE_FILESPeter Kjellerstedt2023-10-061-2/+4
| | | | | | | | | | | This allows, e.g., *.cmake to be added to CONFIGURE_FILES to make the do_configure task depend on changes to any cmake file. (From OE-Core rev: 09873b3fb24a00cfbd73282d29e4c5821774f579) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Show error if document is not foundJoshua Watt2023-09-291-0/+2
| | | | | | | | | | | As in other places, print a more helpful error if a SPDX document is not found when assembling documents for the final SPDX archive. (From OE-Core rev: 0a3f7e0b41552e113347d6f5edefe84fd4e47fdf) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: add CVSS vector string to CVE database and reportsAntoine Lubineau2023-09-281-1/+4
| | | | | | | | | | | This allows building detailed vulnerability analysis tools without relying on external resources. (From OE-Core rev: 048ff0ad927f4d37cc5547ebeba9e0c221687ea6) Signed-off-by: Antoine Lubineau <antoine.lubineau@easymile.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx: Ensure it is clear where the message comes fromRichard Purdie2023-09-261-1/+1
| | | | | | | | | | This message can appear in do_rootfs and the users tend to think this is some kind of packaging issue. Add SPDX to the message so users like me think in the right direction. (From OE-Core rev: ddcf15134b5ec4bf4c728156461c2e45194d71ca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* spdx: use TOOLCHAIN_OUTPUTNAME for SDK filename prefixDenys Dmytriyenko2023-09-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Replace SDK_NAME with TOOLCHAIN_OUTPUTNAME when naming SDK host and target SPDX archives. In most cases TOOLCHAIN_OUTPUTNAME is almost identical to SDK_NAME, but sometimes custom meta-toolchain recipes have some variations and populate_sdk_base.bbclass already uses TOOLCHAIN_OUTPUTNAME to name corresponding host and target manifests: SDK_HOST_MANIFEST = "${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.host.manifest" SDK_TARGET_MANIFEST = "${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.target.manifest" To match the naming convention, also use TOOLCHAIN_OUTPUTNAME here for naming SPDX archives as well, resulting in: ${TOOLCHAIN_OUTPUTNAME}-host.spdx.tar.zst ${TOOLCHAIN_OUTPUTNAME}-target.spdx.tar.zst (From OE-Core rev: 38158670256e2ce803aa45f609108d29523b3135) Signed-off-by: Denys Dmytriyenko <denys@konsulko.com> Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx/sbom: Ensure files don't overlap between machinesRichard Purdie2023-09-221-11/+30
| | | | | | | | | | | | | | | | | | | | | | Currently the by-id and by-namespace SPDX files are created without reference to PACKAGE_ARCH. This means that for two machines using a common package architecture (e.g. genericx86-64 and qqemux86-64), there would be overlapping files. This means that the build of one can remove files from the other leading to build failures. An example would be: MACHINE=qemux86-64 bitbake core-image-minimal MACHINE=genericx86-64 bitbake core-image-minimal MACHINE=qemux86-64 bitbake linux-yocto -c clean MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs To fix this, add PACKAGE_ARCH to the path used for the files and use a search path based upon PACKAGE_ARCHS to access them. (From OE-Core rev: b2db10e966438071d00d2057b84d5f347613d841) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multilib: fix SSTATE_ARCHS for multilib usageRichard Purdie2023-09-201-0/+3
| | | | | | | | | | | | | | When building multilibs, we need to inject the multilib sstate pkgarch into SSTATE_ARCHS so the list forms a complete search path. Add a tweak to do this. PACKAGE_ARCH defaults to TUNE_PKGARCH so this is equivalent and just guards against recipes changing the value which may have other unwanted side effects. (From OE-Core rev: 37126ffc7ccbd3df57ebbd8e581d158f03bb3b4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Add extra debugging for missing package filesJoshua Watt2023-09-131-1/+2
| | | | | | | | | | In a effort to debug YOCTO #15185, improve the logging when a package file cannot be found in the list of SPDX files. (From OE-Core rev: 35d827d44d65380bfb7f6e1d0a0ee63516f822a8) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: Drop ';' delimiter from ROOTFS/IMAGE*COMMAND variablesRichard Purdie2023-09-095-20/+20
| | | | | | | | | | | | | Originally these were shell functions but they have long since been processed by bb.build.exec_func(). Since we no longer need shell syntax, we can drop the ';' delimiters and just use a space separated string. This cleans up the variable and quietly removes any stray ';' that do happen to still make it in. (From OE-Core rev: c3365dfd9ddd7fbe70b62e0f11166e57a8ca6f73) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes/classes/scripts: Drop SRCPV usage in OE-CoreRichard Purdie2023-08-241-2/+0
| | | | | | | | Now that SRCPV isn't needed we can simplify things in a few places... (From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base/package: Move source revision information from PV to PKGVRichard Purdie2023-08-241-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Source control information being present in PV used to be a hard requirement for bitbake to operate correctly. Now that hashes are a required part of task stamps, this requirement no longer exists. This means we can defer the hash pieces to PKGV and simplify PV. Use new bitbake fetcher API to inject the source revisions directly into the hash allowing removal of some horrible code from base.bbclass and avoiding any hardcoding about how SRCREV may or may not be used. Use that API to object the string to append to PKGV and append that directly. The user visible effect of this change is that PV will no longer have revision information in it and this will now be appended to PV through PKGV when the packages are written. Since PV is used in STAMP and WORKDIR, users will see small directory naming and stamp naming changes. This will mean that sstate reuse through hash equivalence where the source revision changes but the output does not will become possible as the sstate naming will become less specific and no longer contain the revision. The SRCPV variable will no longer be needed in PV and is effectively now just a null operation. Usage can be removed over time. (From OE-Core rev: a8e7b0f932b9ea69b3a218fca18041676c65aba0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-06 21:43:38 +0000