summaryrefslogtreecommitdiffstats
path: root/meta/classes-global/sstate.bbclass
Commit message (Collapse)AuthorAgeFilesLines
* do_package/sstate/sstatesig: Change timestamp clamping to hash output onlyRichard Purdie2024-12-091-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code was changing the timestamps of the files in the do_package output, particularly the files added for debug sources. This was to do two things: a) make do_package sstate more reproducible b) ensure better hash equivalence matching Unfortuately the debug source files are hardlinks into the source tree for efficiency so touching these, touches a lot of files in ${B} and ${S}. This causes unpredictable effects if compile is run again for example, or could cause compiling in the install task. The hash equivalence matching is of key importance but we can mimic that using clamping of the file timestamps in the depsig output used to generate the hashes. This patch drops the global timestamp clamping, instead allowing the files to retain their creation timestamps into sstate. This makes do_package sstate slightly less reproducibile. We could clamp the sstate timestamps but that would lead to two different sets of timestamps depending on whether the data came from sstate or not. I'd prefer to have consistent code behaviour, rather than differing behavhour depending on whether data came from sstate or not. If we wanted to have reproducibiliy and fix the "corruption" of S/B and have consistent codepaths, the only other option would be two copies of the sources, which could end up huge and seems the least desireable option. This patch therefore drops the timestamp clamping in the sstate files and tweaks the depsig data generation to clamp the timestamps for do_package instead since this seems the best compromise. I validated that rpm/deb/ipk files still generate correctly as before. (From OE-Core rev: 9df0bf5775ec96773e77c489ba6a443037d65ab6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 475759fdab7200488b2a568b2ba1aa31a456d113) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usageRichard Purdie2024-09-251-1/+4
| | | | | | | | | | | | | | | | | | We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of postfuncs. Finally get around to doing that which should make the buildhistory code a little more readable. Unfortunately ordering the buildhistory function calls after the sstate ones is difficult without coding that into the sstate class. This patch does that to ensure everything functions as expected until we can find a better way. This is still likely preferable than the generic sstate postfuncs support since the function flow is much more readable. (From OE-Core rev: 466c505b779dec2ba790f4e6cde7fbb35037f4ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c9e2a8fa2f0305ef1247ec405555612326f798f8) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* sstate.bbclass: Add _SSTATE_EXCLUDEDEPS_SYSROOT to vardepsexcludeMark Hatle2024-05-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When using tinfoil to control the build, multiple commands (serially) could trigger an error such as: When reparsing ....bb:do_package, the basehash value changed from ... to .... The metadata is not deterministic and this needs to be fixed. ERROR: The following commands may help: ERROR: $ bitbake esw-conf -cdo_package -Snone ERROR: Then: ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff However following these commands it was not able to be reproduced. Forcing bitbake to dump the signatures and then running bitbake-diffsigs showed that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but was blank is a different version. Upon inspecting the code in sstate.bbclass, one usage (without the _) is already excludes, the leading _ version is used as a cache, only if set but is not actually required to be defined. So ignoring the value should work properly. (From OE-Core rev: 4138f0a048149195d39347d97f8155199b6cf0cb) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4ec704ed6a1cfaf0a6c20f2038e7192e361ef590) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* abi_version/sstate: Switch to a new version for the upcoming releaseRichard Purdie2024-04-161-1/+1
| | | | | | | | | | | In testing websocket hashequivalence, corrupted sstate was injected into the autobuilder extensively. With the new release/LTS, being able to clearly differentiate between old and new sstate is probably desireable anyway so bump the appropriate versions. (From OE-Core rev: 7f107c180f592be29f57f580c60a6adbbebd7714) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate.bbclass: Only sign packages at the time of their creationTobias Hagelborn2024-02-181-46/+94
| | | | | | | | | | | | | | | | | The purpose of the change is to never sign a package not created by the build itself. sstate_create_package is refactored into Python and re-designed to handle signing inside the function. Thus, the signing should never apply to existing sstate packages. The function is therefore renamed into sstate_create_and_sign_package. The creation of the archive remains in a separate shellscript function. Co-authored-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> (From OE-Core rev: ba223f8fff19ea59440d56cf3fe46200f3f71e22) Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes-global/sstate: Fix variable typoJoshua Watt2023-12-241-1/+1
| | | | | | | | | | The vardepsexclude was subtly wrong in that it referenced STATE_MANMACH when the actual variable name is SSTATE_MANMACH. (From OE-Core rev: ec5054396f7fafea2a071d2695ae111fc585d6e6) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Fix dir ownership issues in SSTATE_DIRRichard Purdie2023-12-151-2/+7
| | | | | | | | | | | | | | | | | | | We currently use mkdir -p to create missing parent directories within SSTATE_DIR. Reading the man page for mkdir mentions that parent directories are created with the current umask, *not* the mode passed upon the commandline. We could fix this by setting and resetting the umask but since we already have decent python code able to do this, move to using that injecting a python function into the chain of functions already present. This should help fix the occasional sstate directory creation with the wrong permissions. [YOCTO #14385] (From OE-Core rev: ae642a4b038c6946e6c8aa9778bf09099d938a31) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Remove unneeded code from setscene_depvalid() related to useraddRichard Purdie2023-12-081-5/+0
| | | | | | | | | With recent changes to runqueue in bitbake, this horrible hack is no longer needed and we can drop it. (From OE-Core rev: f7f50b6013f7b636fe46448b1cbe1473de891527) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Ensure sstate searches update file mtimeRichard Purdie2023-11-081-10/+2
| | | | | | | | | | | | | | | Commands like "bitbake XXX -S printdiff" search for sstate files but don't download them. This means that local files aren't touched as the download code would do, meaning the sstate cleanup scripts can delete them. This can then lead to obtuse build failures. Have the search code touch local files in the same way as the main code paths would to avoid these files disappearing. Move the function to a common touch() function in lib/oe instead of duplicating code. (From OE-Core rev: a27fc0bd5706ab5b9c68a0271fcf57377a678cdf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Stop allowing overlapping symlinks from sstateRichard Purdie2023-09-221-24/+6
| | | | | | | | | | | | | | | | When originally implemented, overlapping symlinks in DEPLOY_DIR were common. That is no longer the case and these overlapping links are causing bugs in other areas (e.g. bug 14123). Therefore start showing errors for overlapping symlinks in shared areas. Whilst here, fix a broken file reference in the grep command to match current file layouts and update the message shown to users to match current times. Most of the message content is obsolete now due to other advances and changes in the way the staging code now works. (From OE-Core rev: 15f98f72e1730fed5164fcd539737a7a7b96a1a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* license/license_image: Fix license file layout to avoid overlapping filesRichard Purdie2023-09-201-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently DEPLOY_DIR/licenses is added to SSTATE_ALLOW_OVERLAP_FILES. This leads to bugs since when one MACHINE_ARCH recipes is cleaned, it removes the files for another which then results in later build failures as license files disappear. The solution is to include SSTAGE_PKGARCH in the path names to the license files. That does mean a search has to be used to find the correct license files for a given PN but that can be done via SSTATE_ARCHS. The implication for other tools is the layout has changed so tools will need to adapt to the new paths. The benefit is no more strange build failures such as from patterns like: MACHINE=qemux86-64 bitbake core-image-minimal MACHINE=genericx86-64 bitbake core-image-minimal MACHINE=qemux86-64 bitbake linux-yocto -c clean MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs [YOCTO #14123] For anyone finding this commit, I'd question how much people should be relying on this code for tooling and suggest the SPDX manifests should be the preferred data format going forward anyway. (From OE-Core rev: 1a4ab9fc26659507e678e87312b514e8ea515673) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multilib: fix SSTATE_ARCHS for multilib usageRichard Purdie2023-09-201-1/+2
| | | | | | | | | | | | | | When building multilibs, we need to inject the multilib sstate pkgarch into SSTATE_ARCHS so the list forms a complete search path. Add a tweak to do this. PACKAGE_ARCH defaults to TUNE_PKGARCH so this is equivalent and just guards against recipes changing the value which may have other unwanted side effects. (From OE-Core rev: 37126ffc7ccbd3df57ebbd8e581d158f03bb3b4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Fix nativesdk entry in SSTATE_ARCHSRichard Purdie2023-09-201-1/+1
| | | | | | | | | | PACKAGE_ARCH gets refined by each target so this value isn't valid in all contexts. Tweak to use underlying variables to build it so that it remains valid in wider contexts and works with new usage in sdpx and license classes. (From OE-Core rev: ca90379147cb75755d205670781d8922161e9317) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes/classes/scripts: Drop SRCPV usage in OE-CoreRichard Purdie2023-08-241-4/+0
| | | | | | | | Now that SRCPV isn't needed we can simplify things in a few places... (From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* abi_version/sstate: Handle pkgconfig output changes and bump output versionsRichard Purdie2023-03-261-1/+1
| | | | | | | | | | | | | A recent package.py change has highlighted some corruption issues with -dev pkgconfig package dependencies. Bump the output versions to trigger a rebuild and ensure everything is consistent. Take the opportunity to also drop all HASHEQUIV_HASH_VERSION entries since the main version is changing. (From OE-Core rev: f45ddfbf007de858327eef0ffefd5840ef4c69b8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Add check_for_interrupts() call functionality added in bitbakeRichard Purdie2023-02-231-0/+3
| | | | | | | | | | | | | Add in the new 'check' calls which mean that the code can exit from longer running code loops if requested by the user. This covers sstate checks and sstate manifest cleanup code which currently couldn't be interrupted by the user from the UI. Increase the minimum bitbake version which brings in this API. (From OE-Core rev: cb6c50689175668b66ffbe09ff8f250ba9a9034b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate.bbclass: Fetch non-existing local .sig files if neededTobias Hagelborn2023-02-021-2/+3
| | | | | | | | | | | | | | | | | | For the case of a global shared state cache and a local sstate cache with a mix of signed and un-signed entries, there is a case where a .sig is missing locally, but may exist in the global sstate cache. For this case, do not just fail sstate fetch, but rather backfill the .sig file from the global sstate cache. If this case is detected, re-run the fetch operation to get the global .sig file (if it exists). (From OE-Core rev: d5ea1a5396bf9fd4303cae46bc0e042be8de8d67) Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: show progress bar againEnrico Scholz2022-11-221-1/+5
| | | | | | | | | | | | | | | | | | | Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f) broke the | Checking sstate mirror object availability: ... progress bar because the removed 'thread_worker' was still referenced in an asynchronous function. As the result of the future is never read, the resulting backtrace is silently discarded. Replace the information given to 'ProcessProgress' by a counter. (From OE-Core rev: eacd52216815e0c75e5f257b6db792afd8b1b6df) Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Allow optimisation of do_deploy_archives task dependenciesJose Quaresma2022-11-141-1/+1
| | | | | | | | | | | do_deploy_archives tasks don't need their dependencies so we can optimistion this as we do for some other tasks. (From OE-Core rev: 3dd9f6e398844380d3765c54d35afe0d2ccf82e7) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Fix sstate file mirroring checksum issueRichard Purdie2022-09-071-0/+1
| | | | | | | | | | | After a recent fix in bitbake, the checksums may be checked for local file fetches at an earlier code point. The underlying recipe checksums won't match sstate values so ensure these aren't set via SRC_URI flags when reconfiguring the fetch for sstate. (From OE-Core rev: 315cfebe77a46c868f307919d3540046a6cb8587) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: Update classes to match new bitbake class scope functionalityRichard Purdie2022-08-121-0/+1364
Move classes to classes-global or classes-recipe as appropriate to take advantage of new bitbake functionality to check class scope/usage. (From OE-Core rev: f5c128008365e141082c129417eb72d2751e8045) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-07 11:35:42 +0000