summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* bitbake: fetch/git: always fetch lfs when creating shallow tarballPeter Marko2025-04-101-3/+2
| | | | | | | | | | | | | | | | | | Before fast shallow tarballs the shallow tarball with lfs worked only if the original .../downloads/git2/... directory existed. Once it was not there, do_unpack on shallow tarball without lfs files failed due to disabled network. It was fixed for fast shallow tarballs and this commit fixes the other scenarios, too. This can happen when such shallow tarball is put to mirror or if someone does cleanup to reclaim disk space. (Bitbake rev: 4e6fb31a88448cdacf4e9f84cf0a8a8035f84d63) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: tests/fetch: Fix git PREMIRRORONLY testJulian Haller2025-04-101-3/+4
| | | | | | | | | | | | Using a shallow clone to simulate an outdated git mirror tarball does not work in the intended way. A shallow clone already contains the latest commit which can hide certain fetcher behavior. Simulate an outdated mirror tarball, as the test titles indicate, by removing the newer commits from the mirror. (Bitbake rev: a51ee01f0a586fefd5a4061f4a1ca6cbf81b7046) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch: Fix BB_FETCH_PREMIRRORONLY for git mirror tarballsJulian Haller2025-04-101-0/+4
| | | | | | | | | | | | | When invoking the original git fetcher after downloading a mirror tarball, BB_FETCH_PREMIRRORONLY is ignored. This leads to git fetch commands targeting the upstream source being executed silently. Ensure setting BB_NO_NETWORK before invoking the original fetcher. While this was only observed for git, setting this in general for all fetcher types makes sense at this location. (Bitbake rev: 1b1321f2b60c0a66159e3f20c6befcb0b3ccc4c7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Handle srcrevs for annotated tags in tag checkRichard Purdie2025-04-032-1/+13
| | | | | | | | | | | | | | If SRCREV points at an annotated tag, the comparision code can fail as the resolved tag might not be the same sha. Handle this by also resolving the SRCREV. We only need to do this if they don't match in the first place for a minor performance win. Also add a test for this. (Bitbake rev: 136c06e251de68ed64355ec6b47a522ff3a372e3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Update version to 2.12.0 for releaseRichard Purdie2025-03-291-1/+1
| | | | | | (Bitbake rev: 5b4e20377eea8d428edf1aeb2187c18f82ca6757) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add a test for download paths without a proper filenameRoss Burton2025-03-272-0/+3534
| | | | | | | | | | | | | | | | | For example the miniupnpd recipe has a SRC_URI like this: http://miniupnp.tuxfamily.org/files/download.php?file=${BP}.tar.gz In this case the path is /files/download.php, which isn't useful when the latest_upstream logic bails early if there is no version in the path. The logic now also checks in the downloadfilename, so add a test that this works as expected. (Bitbake rev: fffbf5d5e1c8556cddf0794e0b303bb0106747a0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: support setting PV in the wget fetcherRoss Burton2025-03-271-1/+2
| | | | | | | | | Some code paths in latest_versionstring() need PV to be set correctly. (Bitbake rev: 0a9f90ff658e09feda63b398ec35715a65ff6193) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: use a namedtuple for the wget test dataRoss Burton2025-03-271-22/+24
| | | | | | | | | | Use a named tuple so the test can access named members instead of just accessing the data via index, which is harder to understand. (Bitbake rev: 4b15652c84b06f0506c757e2647875a9b1cc7bfe) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/tests: use subtests when looping in a test caseRoss Burton2025-03-271-32/+35
| | | | | | | | | | Marking the test iterations as subtests means that when one fails, it can identify clearly which iteration has failed. (Bitbake rev: 52c55e681332d7cdbe06f3c9d9c8d77cb0cb93f6) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: use long arguments for clarityRoss Burton2025-03-271-5/+5
| | | | | | | | | | | It's best practise to use long-form arguments in scripts where the conciseness of short arguments is less useful than in an interactive terminal. (Bitbake rev: 54039bc9d169871af6d36578df4c21bff296f6de) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: don't conditionalise --continueRoss Burton2025-03-271-6/+1
| | | | | | | | | | | | If the target file exists, we pass --continue so that wget will try to resume the download instead of starting from the beginning. However if the file doesn't exist --continue has no effect, so there's no need to conditionalise the use of the argument. (Bitbake rev: a5ee50d4d2b2e6c00abe1551382afd9799345dbf) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: consider downloadfilename when checking for upstreamRoss Burton2025-03-271-1/+5
| | | | | | | | | | | | | | | | | | | | | latest_versionstring() currently looks at just the end of the URI when guessing what the filename to look for is, but this doesn't work if the URL filename is not simple. For example, miniupnpd has a SRC_URI of: http://miniupnp.tuxfamily.org/files/download.php?file=${BP}.tar.gz;downloadfilename=${BP}.tar.gz The filename component of this is "download.php", which causes the heuristics in latest_versionstring() to exit early. Instead, if the downloadfilename is set then use that, as it's often the actual filename that we're after. (Bitbake rev: 2d5f135e997d13fabda0ad266fd5c928ee33f487) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gomod: Fix mirroring problemChristian Lindeberg2025-03-272-13/+31
| | | | | | | | | | | | Build the 'downloadfilename' parameter by replacing path separators in the module path like the git fetcher builds the mirror tar ball name. Copy the downloaded file in the fetcher's unpack method like the crate fetcher instead of calling the base fetcher's unpack method. (Bitbake rev: 7762cea087597019460d66b04268757bd46befdf) Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump version to 2.9.2Richard Purdie2025-03-251-1/+1
| | | | | | | | | After the fetcher revisions changes, we need a new version marker to match this with in OE-Core. (Bitbake rev: 8cc976e2792fdde3900729f3b09dd18ab640b5e8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "bblayers/query: Fix using "removeprefix" string method"Richard Purdie2025-03-241-2/+1
| | | | | | | | This reverts commit 004cfdec1c865f2351bbac99acb3d63bfef9d380. Now we have python 3.9 as a minimum we can do this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: Drop multiple branch/revision support for single git urlsRichard Purdie2025-03-205-281/+139
| | | | | | | | | | | | | | | | | We used to use this for bare clones where a single git url could handle multiple revisions (which implied multiple branches). We don't use this any more and I doubt we'd want to go back to it. If we remove it, we can simplfy the looping in the code which seems desireable. This patch does change the warning for missing branch parameters to a error. The message has hinted about that for long enough. Some test cases are removed since they are no longer needed. (Bitbake rev: 2515fbd10824005fa7f34e87706000c079920366) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Bump minimum python version requirement to 3.9Richard Purdie2025-03-201-2/+2
| | | | | | | | | | | | | Since Ubuntu 20.04 is heading to EoL and that we're having other issues on that platform needing buildtools anyway, we're about to lose the last python 3.8 platform we were supporting. Bump the minimum version to 3.9 since there are many developers wanting access to newer python features. (Bitbake rev: 0d2e682d00dfc2ec776b3a89f25202db98024895) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Fix typo in npm testRichard Purdie2025-03-191-1/+1
| | | | | | (Bitbake rev: 79b04f61236117d310c12c1b1378ae63afb931ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add git tag verification testsRichard Purdie2025-03-191-0/+59
| | | | | | | | Add tests for git tag verification in both standard and shallow clones. (Bitbake rev: f47127066d67e2ad80974fa1e7c0fcc7409161af) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Ensure tags work with shallow clonesRichard Purdie2025-03-191-0/+2
| | | | | | | | | | If we want to validate a tag when using shallow clones, we need to ensure the tag is imported to the clone. Adding to extra_refs allows this to happen ensuring tag validation in shallow clones works. (Bitbake rev: 03945475886b8d31c7eccc80fe594ab5306bcf5d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Rework tag parameter handlingRichard Purdie2025-03-193-14/+17
| | | | | | | | | | | | | | | | | | | | | | Currently bitbake disallows tag parameters along with revision parameters. This isn't great since quite often, we'd like to verify that a given revision does match some tag. At the same time we don't want to or need to access the network to verify this, which normally a tag would require. Rework the code so that tag and revisions can both be specified together. Verify that any tag specified matches the revision in use at unpack time. This means we can start requiring people to put tags in git SRC_URIs when revisions are used, making review a little easier that it isn't some random revision. The test that is dropped looks like a different test but the comment is a copy and paste error. The SRCREV/rev mismatch test remains, this removes the rev and tag set test. (Bitbake rev: d591d7633fe8d739ec00395920e44910b0b77e27) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add missing branch parameter to testsRichard Purdie2025-03-191-23/+23
| | | | | | (Bitbake rev: fd01e8e3a5a757d5f506095fc1ac4e45d888ae78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: Track data during finalizeDataYoann Congal2025-03-181-1/+2
| | | | | | | | | | | | finalizeData changes the value of variables, so, it needs to have data tracking enabled to get a full and accurate variable value history. Fixes [YOCTO #15652] (Bitbake rev: 228979b506f7d0e24c643440bcc7becf684da283) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: Refactor temporary data tracking in a context managerYoann Congal2025-03-181-7/+18
| | | | | | | | | | | | | | | A new context manager Tinfoil._data_tracked_if_enabled() is introduced to replace the following structure: if self.tracking: self.run_command('enableDataTracking') # Code that need data tracking if self.tracking: self.run_command('disableDataTracking') (Bitbake rev: 0fea4555d2143c6b23a79d3d5cf791103a68141b) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Restore escape quoting for the git url when usedPatrik Nordvall2025-03-182-1/+5
| | | | | | | | | | | | | | | | | | | | | | This fixes a bug where escapes in the url path would not be properly restored for the git commands in the git fetcher. For example, a space which is encoded as '%20' was not properly encoded before the clone command. e.g. SRC_URI="git://git.openembedded.org/bitbake%20example/bitbake;protocol=https" resulted in git clone 'https://git.openembedded.org/bitbake example/bitbake' instead of git clone 'https://git.openembedded.org/bitbake%20example/bitbake' (Bitbake rev: be48024253b93215bb110cd1d05925e789ec9680) Signed-off-by: Patrik Nordvall <patrik.nordvall95@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure module dependency changes invalidate the base ↵Richard Purdie2025-03-131-0/+5
| | | | | | | | | | | | | | | | | config cache Changing module files was changing the tash hashes but it was not invalidating the parse cache, leading to tashhash mismatch errors during builds. Add information from modulecode_deps to the configuration hash used for cache invalidation to avoid this and trigger reparses when function library code changes. [YOCTO #15795] (Bitbake rev: a121db3d8d28420c36369237b8bb11c2d0aaf5f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add `gc-mark-stream` command for batch hash markingAlexandre Marques2025-03-133-0/+93
| | | | | | | | | | | | | | | | | | | | | | | | Implements the `gc-mark-stream` command to allow for marking equivalence entries in batch, by making use of stream mode communication to the server. The aim of this is to improve efficiency by reducing the impact of latency when marking a high volume of hash entries. Example usage of the new `gc-mark-stream` command: ``` $ cat << HASHES | \ ./bin/bitbake-hashclient --address "ws://localhost:8688/ws" gc-mark-stream "alive" unihash f37918cc02eb5a520b1aff86faacbc0a38124646 unihash af36b199320e611fbb16f1f277d3ee1d619ca58b taskhash a1117c1f5a7c9ab2f5a39cc6fe5e6152169d09c0 method oe.sstatesig.OEOuthashBasic HASHES ``` (Bitbake rev: c84715f28cd36666ea07a179d91b8c32ea0df8e7) Signed-off-by: Alexander Marques <c137.marques@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: Fix command complete event orderingRichard Purdie2025-03-111-2/+2
| | | | | | | | | | | | The command complete event triggers the UI to start new commands. We should send this once we've cleaned up existing commands since on very slow systems, this could otherwise cause failures. There is a 30s timeout on incomming commands but slow systems have been known to hit this on poorly configured I setups. (Bitbake rev: d202298b3d989c39bc532b69c410e2e78b082a11) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add debug for parsing being completeRichard Purdie2025-03-111-0/+1
| | | | | | | | | | We have a "parsing started" event in the cooker deamon log but we don't currently log the corresponding "parsing complete" event which is confusing. Add this so that the logs are more logical. (Bitbake rev: 1aa491c1f1211bf9faab712c321b66629fb7be66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm: Adding npmrc support for private registry authenticationEric Meyers2025-03-091-0/+15
| | | | | | | | | (Bitbake rev: 5fa6137b6d98544766f3152b874e67d04fafb88f) Signed-off-by: Eric Meyers <eric.meyers@arthrex.com> Cc: Geoff Parker <geoffrey.parker@arthrex.com> Cc: Chuck Wolber <chuckwolber@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: Skipping typing when inspecting Python modulesPedro Ferreira2025-03-091-2/+10
| | | | | | | | | | | | If a custom python module is added thru BBIMPORTS and it uses typing(Any,Tuple,Union...), codeparser will fail because inspect.py raises TypeError exception if the object is a built-in module, class, or function. (Bitbake rev: 0ecfd0b8540220633e71d24cd73cc5306863ae3c) Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Verify regular expression "URLs" that contain a ?Peter Kjellerstedt2025-03-071-3/+6
| | | | | | | | | | | | A regular expression "URL" in PREMIRRORS and MIRRORS may contain a ? as part of the regular expression. Make sure this does not cause problems. (Bitbake rev: 5af7fe4473cd7e75d4eb7f8b93c499bd157ff156) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Partial revert decodeurl() to not use the URI classStefan Herbrechtsmeier2025-03-071-3/+51
| | | | | | | | | | | | | | | | | | | | | | | | This partial reverts commit a5d569c94700f04b8193c6bccae5af619931b00f which changes decodeurl() to use the URI class to parse the URL instead of doing it itself. While reusing code is generally a good idea, using urllib.parse.urlparse() (which the URI class does) to parse the regular expression "URLs" that are used in PREMIRRORS and MIRRORS does not work. A regular expression URL containing https?://... would be silently ignored, while a URL using a negative lookahead such as git://(?!internal\.git\.server).*/.* would result in a cryptic error: Exception: re.error: missing ), unterminated subpattern at position 0 The problem is that urllib.parse.urlparse() treats the ? as the start of URL parameters and thus stops parsing whatever part of the URL it was parsing. Restore the old function and use it in the PREMIRRORS and MIRRORS code. (Bitbake rev: f8a7712754e6d0199a0d227fca288307b935368d) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: clear expand_cache in _setvar_update_overridevarsChen Qi2025-03-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At the end of this function, self.overrides is cleared, which means we'll construct a new self.overrides after this call. And a new self.overrides will give out different expand_cache, so the old expand_cache should also be cleared to avoid any wrong value. Currently, there's a problem revealed by recent recipe specific virtual provider patch. If we enable multilib and set "OVERRIDES:prepend" in local.conf, things don't work. Here's the error message: ERROR: Nothing PROVIDES 'lib32-gcc-cross-x86_64' Below are reproduce steps: 1. Add in local.conf the following lines: MACHINE ?= "qemux86-64" require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32" OVERRIDES:prepend = "some-override:" (Note that using :append and :remove also reproduces the issue.) 2. bitbake -n lib32-sysstat (bitbake -n core-image-minimal also reproduces the issue) The expandWithRefs calls getVar, which fills expand_cache. So when setting OVERRIDES:prepend, this will fill the expand_cache. When overridevars are updated, if we don't clear expand_cache, we'll retrieve wrong values. Previously, things happened to work because there's a call to expand PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc, which in turn expands TARGET_VENDOR. Now what we expand is PREFERRED_PROVIDER_ virtual/cross-cc, so the problem is revealed. (Bitbake rev: 842e94234c9b9faf4e902721f516071f643e49c9) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: simple clean upChen Qi2025-03-061-3/+1
| | | | | | | | | | | | When var matches __setvar_regexp__, which has the value as below, __setvar_regexp__ = re.compile(r'(?P<base>.*?)(?P<keyword>:append|:prepend|:remove)(:(?P<add>[^A-Z]*))?$') the keyword will not be __doc__ or __module__ and var will always contain ":". (Bitbake rev: 0af396917fe176df7b1ed0bd7c7267507ed12e03) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: remove unnecessary ? from __expand_var_regexp__Chen Qi2025-03-061-1/+1
| | | | | | | | | | The non-greedy modifier suffix ? is not necessary here because } is not in the character set [a-zA-Z0-9\-_+./~:]. (Bitbake rev: 80735719d0002c02b704a49604ddd82c461203a2) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/cooker.py: produce an error when BB_HASHSERVE_UPSTREAM ↵Alexander Kanavin2025-03-061-2/+8
| | | | | | | | | | | | | | | | cannot be used due to missing python module Previously if the websockets module was not installed bitbake would only print a warning and continue, resulting in a degraded user experience due to inability to use the configured sstate server. Let's consider that as fatal misconfiguration, so that users can address the issue properly and not wonder why builds are taking forever. (Bitbake rev: cfba2a9fca9dfa3b05ec9040fe0cb8143ac04af7) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add an additional test case to check whether the fast ↵Stefan Koch2025-03-061-0/+12
| | | | | | | | | fetch is shallow (Bitbake rev: 16f1961e077c525ccfc12496a3deca944df89fc6) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Adapt test cases for fast shallow fetchesStefan Koch2025-03-061-19/+12
| | | | | | | | | | | | - Address the absence of an initial full bare clone - Utilize the initial shallow clone - Modify existing test cases for this behavior - Remove incompatible test cases (Bitbake rev: 599fedacd7782dcb52825c22200f35344c102548) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gitsm: Unpack even when `ud.clonedir` is not availableStefan Koch2025-03-061-1/+4
| | | | | | | (Bitbake rev: 13d76361ec37faecd84e7b81da22ada7d4e0ba90) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Add support for fast initial shallow fetchStefan Koch2025-03-061-29/+85
| | | | | | | | | | | | | | | | | | | | | | | | When `ud.shallow == 1`: - Prefer an initial shallow clone over an initial full bare clone, while still utilizing any already existing full bare clones. - If the Git error "Server does not allow request for unadvertised object" occurs, the initial full bare clone is fetched automatically. This may happen if the Git server does not allow the request or if the Git client has issues with this functionality, especially with the Git client from Ubuntu 20.04. This improves: - Resolve timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminate the need to use an HTTPS tarball `SRC_URI` to reduce data transfer. - Allow SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. (Bitbake rev: 457288b2fda86fd00cdcaefac616129b0029e1f9) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests: Fix kirkstone testRichard Purdie2025-03-051-2/+2
| | | | | | | | After the fixture changes, the tests need to be tweaked unfotunately. (Bitbake rev: 708abd1a8060684127acc7ce4142f05865005750) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/fixtures: Add walnascar release informationRichard Purdie2025-03-053-28/+90
| | | | | | (Bitbake rev: 44f732ae1b2d812577a9004f8e15c4ebcdfd1d74) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: Add script to compare toaster fixtures to project release dataRichard Purdie2025-03-051-0/+38
| | | | | | | | | | | | We need to validate the toaster fixtures information against our current release data to ensure it is correct when we release. Add a script we can use to do this which the autobuilder can run to validate things. [YOCTO #15516] (Bitbake rev: 5b2d79ed505bbfa2fb2d355935e75199b7f2c37e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: Verify mcdepends are validMark Hatle2025-03-034-1/+14
| | | | | | | | | | | | | | | In order to avoid a potentially confusing backtrace, check that the mcdepend is valid when we add it. Add a test case to ensure invalid configurations are caught and trigger an error. [RP: Reworked test case to simplify and improve code] (Bitbake rev: ff523497270f37b484b44a4445c2194791bcb6ff) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/utils: Avoid deadlock from lock_timeout() and recursive eventsRichard Purdie2025-03-032-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've been seeing intermittent failures on Ubuntu 22.04 in oe-selftest which were problematic to debug. The failure was inside lock_timeout and once that was identified and the backtrace obtained, the problem becomes clearer: File "X/bitbake/lib/bb/server/process.py", line 466, in idle_thread_internal retval = function(self, data, False) File "X/bitbake/lib/bb/command.py", line 123, in runAsyncCommand self.cooker.updateCache() File "X/bitbake/lib/bb/cooker.py", line 1629, in updateCache self.parser = CookerParser(self, mcfilelist, total_masked) File "X/bitbake/lib/bb/cooker.py", line 2141, in __init__ self.bb_caches = bb.cache.MulticonfigCache(self.cfgbuilder, self.cfghash, cooker.caches_array) File "X/bitbake/lib/bb/cache.py", line 772, in __init__ loaded += c.prepare_cache(progress) File "X/bitbake/lib/bb/cache.py", line 435, in prepare_cache loaded = self.load_cachefile(progress) File "X/bitbake/lib/bb/cache.py", line 516, in load_cachefile progress(cachefile.tell() + previous_progress) File "X/bitbake/lib/bb/cache.py", line 751, in progress bb.event.fire(bb.event.CacheLoadProgress(current_progress, cachesize), File "X/bitbake/lib/bb/event.py", line 234, in fire fire_ui_handlers(event, d) File "X/bitbake/lib/bb/event.py", line 210, in fire_ui_handlers _ui_handlers[h].event.send(event) File "X/bitbake/lib/bb/cooker.py", line 117, in send str_event = codecs.encode(pickle.dumps(event), \'base64\').decode(\'utf-8\') File "/usr/lib/python3.10/asyncio/sslproto.py", line 320, in __del__ _warn(f"unclosed transport {self!r}", ResourceWarning, source=self) File "/usr/lib/python3.10/warnings.py", line 109, in _showwarnmsg sw(msg.message, msg.category, msg.filename, msg.lineno, File "X/bitbake/lib/bb/main.py", line 113, in _showwarning warnlog.warning(s) File "/usr/lib/python3.10/logging/__init__.py", line 1489, in warning self._log(WARNING, msg, args, **kwargs) File "/usr/lib/python3.10/logging/__init__.py", line 1624, in _log self.handle(record) File "/usr/lib/python3.10/logging/__init__.py", line 1634, in handle self.callHandlers(record) File "/usr/lib/python3.10/logging/__init__.py", line 1696, in callHandlers hdlr.handle(record) File "/usr/lib/python3.10/logging/__init__.py", line 968, in handle self.emit(record) File "X/bitbake/lib/bb/event.py", line 778, in emit fire(record, None) File "X/bitbake/lib/bb/event.py", line 234, in fire fire_ui_handlers(event, d) File "X/bitbake/lib/bb/event.py", line 197, in fire_ui_handlers with bb.utils.lock_timeout(_thread_lock): File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "X/bitbake/lib/bb/utils.py", line 1888, in lock_timeout bb.server.process.serverlog("Couldn\'t get the lock for 5 mins, timed out, exiting. %s" % traceback.format_stack()) or put in simpler terms, whilst sending an event(), an unrelated warning message happens to be triggered from asyncio: /usr/lib/python3.10/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f0e797d3100> which triggers a second event() which can't be sent as we're already in the critcal section and already hold the lock. That warning is due to the version of asyncio used on Ubuntu 22.04 with python 3.10 and that comined with timing issues explains why we don't see it on other python versions or distros. We can't handle the second event as the lock is there to serialise the events. Instead, we queue the event and then process the queue later. Add a new version of lock_timeout which allows us to handle the situation more gracefully. (Bitbake rev: 2c590ff1aff89d23b25ce808650f200013a1e6af) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Add signal blocking for lock_timeoutRichard Purdie2025-03-031-0/+2
| | | | | | | | | | We never want to exit whilst holding these locks as it deadlocks all python threads. Add signal blocking around the lock critical part so a signal shouldn't cause such an exit. (Bitbake rev: a097755c671e2b530dea6200a94b39fa9dca246c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Tweak lock_timeout logicRichard Purdie2025-03-031-1/+1
| | | | | | | | | | We should really try and take the lock in the try/finally block so that in some rare cases such as badly timed interrupt/signal, we always release the lock. (Bitbake rev: a9eb8bf7174b6962b5ba07192fe95b8c7112d9d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Print information about lock issue before exitingRichard Purdie2025-03-031-0/+1
| | | | | | (Bitbake rev: cdf6c51a064f8f335c3262b7f102618996f1a229) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: remove mention of OE_EXTRA_IMPORTSRoss Burton2025-02-271-1/+1
| | | | | | | | | | | | This is an OpenEmbedded-specific variable that is no longer used[1]. [1] oe-core 1f56155e ("base: Switch to use addpylib directive and BB_GLOBAL_PYMODULES") (Bitbake rev: 34bfa55a3c14f5d1d6bb3e31b747906fa7cb99c3) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-23 03:48:08 +0000