summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2/wget: Keep query parameters in URL during checkstatusPhilippe-Alexandre Mathieu46 min.1-1/+4
| | | | | | | | | | | | | | | | | | | | When recreating the uri in wget's checkstatus method, we only use the scheme, netloc and path. This completely strips the query parameters from the final URI and potentially breaks the checking functionality from certain fetchers. This is the case for the Azure storage fetcher, as it requires a SAS token that is formatted as a series of query parameters. The error manifests itself when using a private storage account as a PREMIRROR or SSTATE_MIRROR (since regular SRC_URI won't run the checkstatus). This problem is present in scarthgap, but wasn't in kirkstone. CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> (Bitbake rev: 093fec5f019cc0a181782961452a8ccb9a19459d) Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add 'install-buildtools' commandAlexander Kanavin46 min.1-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This basically calls install-buildtools from oe-core/poky, but it ensures via command line parameters that the installation location is stable and the downloads are preserved for reproducibility: $ bin/bitbake-setup install-buildtools Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf ====== Buildtools archive is downloaded into /home/alex/bitbake-builds/yocto-master-testing/buildtools-downloads/20250319141333 and its content installed into /home/alex/bitbake-builds/yocto-master-testing/buildtools ... (output from install-buildtools script) ====== It also detects when buildtools are already installed, and will direct users what to do: ====== alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup install-buildtools Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf Buildtools are already installed in /home/alex/bitbake-builds/yocto-master-testing/buildtools. If you wish to use them, you need to source the the environment setup script e.g. $ . /home/alex/bitbake-builds/yocto-master-testing/buildtools/environment-setup-x86_64-pokysdk-linux You can also re-run bitbake-setup install-buildtools with --force option to force a reinstallation ====== (Bitbake rev: 37d99d94b8d76d66673529b7118ea86a923d6d98) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add tests to bitbake-selftestAlexander Kanavin46 min.1-0/+251
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Run like this: alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-selftest -v bb.tests.setup test_setup (bb.tests.setup.BitbakeSetupTest.test_setup) ... ok ---------------------------------------------------------------------- Ran 1 test in 9.223s OK The test does a basic run-through of init, then status/update on an unchanged configuration, then status/update on a configuration changed via new commits to the test layer, then status/update on configuration changed via the top level json config file. Note that nothing whatsoever is fetched from the network; the test relies entirely on synthetic data contained inside itself, including minimal stubs for oe-setup-build and bitbake-config-build. This data is used to create temporary git repositories then clone them via local filesystem URIs. Later on this can be supplemented by an oe-selftest that tests bitbake-setup against real config files in the official configuration repository and real layers, templates and fragments. (Bitbake rev: 12676b84fca45bece686b8da2feec75ea2f191c2) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: sigen: Avoid bitbake abort with EOFerror from an incomplete ↵HEADmasterPedro Ferreira49 min.1-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | hashserv communication The issue itself is sporadic but aborts the build with an uncaught exception. The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_report_unihash(d) 0003: File: '(...)/poky/meta/classes/sstate.bbclass', lineno: 882, function: sstate_report_unihash 0878: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) 0879: 0880: if report_unihash: 0881: ss = sstate_state_fromvars(d) *** 0882: report_unihash(os.getcwd(), ss['task'], d) 0883:} 0884: 0885:# 0886:# Shell function to decompress and prepare a package for installation File: '(...)/poky/lib/bb/siggen.py', lineno: 651, function: report_unihash 0647: method = self.method 0648: if tid in self.extramethod: 0649: method = method + self.extramethod[tid] 0650: *** 0651: data = self.client().report_unihash(taskhash, method, outhash, unihash, extra_data) 0652: new_unihash = data['unihash'] 0653: 0654: if new_unihash != unihash: 0655: hashequiv_logger.debug('Task %s unihash changed %s -> %s by server %s' % (taskhash, unihash, new_unihash, self.server)) File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 139, function: wrapper 0135: pass 0136: 0137: def _get_downcall_wrapper(self, downcall): 0138: def wrapper(*args, **kwargs): *** 0139: return self.loop.run_until_complete(downcall(*args, **kwargs)) 0140: 0141: return wrapper 0142: 0143: def _add_methods(self, *methods): File: '/usr/lib/python3.9/asyncio/base_events.py', lineno: 642, function: run_until_complete 0638: future.remove_done_callback(_run_until_complete_cb) 0639: if not future.done(): 0640: raise RuntimeError('Event loop stopped before Future completed.') 0641: *** 0642: return future.result() 0643: 0644: def stop(self): 0645: """Stop running the event loop. 0646: File: '(...)/poky/lib/hashserv/client.py', lineno: 70, function: report_unihash 0066: m["taskhash"] = taskhash 0067: m["method"] = method 0068: m["outhash"] = outhash 0069: m["unihash"] = unihash *** 0070: return await self.invoke({"report": m}) 0071: 0072: async def report_unihash_equiv(self, taskhash, method, unihash, extra={}): 0073: await self._set_mode(self.MODE_NORMAL) 0074: m = extra.copy() File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 104, function: invoke 0100: async def proc(): 0101: await self.socket.send_message(msg) 0102: return await self.socket.recv_message() 0103: *** 0104: return await self._send_wrapper(proc) 0105: 0106: async def ping(self): 0107: return await self.invoke({"ping": {}}) 0108: File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 82, function: _send_wrapper 0078: async def _send_wrapper(self, proc): 0079: count = 0 0080: while True: 0081: try: *** 0082: await self.connect() 0083: return await proc() 0084: except ( 0085: OSError, 0086: ConnectionError, File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 70, function: connect 0066: await self.socket.send("") 0067: 0068: async def connect(self): 0069: if self.socket is None: *** 0070: self.socket = await self._connect_sock() 0071: await self.setup_connection() 0072: 0073: async def close(self): 0074: if self.socket is not None: File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 55, function: connect_sock 0051: import websockets 0052: 0053: async def connect_sock(): 0054: try: *** 0055: websocket = await websockets.connect(uri, ping_interval=None) 0056: except (OSError, asyncio.exceptions.TimeoutError, websockets.InvalidHandshake, websockets.InvalidURI) as exc: 0057: raise ConnectionError("Could not connect to websocket: %s" % exc) from exc 0058: return WebsocketConnection(websocket, self.timeout) 0059: File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 444, function: __await_impl__ 0440: async with asyncio_timeout(self.open_timeout): 0441: for _ in range(MAX_REDIRECTS): 0442: self.connection = await self.create_connection() 0443: try: *** 0444: await self.connection.handshake(*self.handshake_args) 0445: except asyncio.CancelledError: 0446: self.connection.close_transport() 0447: raise 0448: except Exception as exc: File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 104, function: handshake 0100: # receiving a response, when the response cannot be parsed, or when the 0101: # response fails the handshake. 0102: 0103: if self.protocol.handshake_exc is not None: *** 0104: raise self.protocol.handshake_exc 0105: 0106: def process_event(self, event: Event) -> None: 0107: """ 0108: Process one incoming event. File: '/usr/local/lib/python3.9/dist-packages/websockets/client.py', lineno: 315, function: parse 0311: 0312: def parse(self) -> Generator[None]: 0313: if self.state is CONNECTING: 0314: try: *** 0315: response = yield from Response.parse( 0316: self.reader.read_line, 0317: self.reader.read_exact, 0318: self.reader.read_to_eof, 0319: ) File: '/usr/local/lib/python3.9/dist-packages/websockets/http11.py', lineno: 238, function: parse 0234: 0235: try: 0236: status_line = yield from parse_line(read_line) 0237: except EOFError as exc: *** 0238: raise EOFError("connection closed while reading HTTP status line") from exc 0239: 0240: try: 0241: protocol, raw_status_code, raw_reason = status_line.split(b" ", 2) 0242: except ValueError: # not enough values to unpack (expected 3, got 1-2) Exception: EOFError: connection closed while reading HTTP status line (Bitbake rev: 5ba7c2f0797a72536a81f57276d4e5c75f23011c) Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Remove multiprocessingpoolJoshua Watt49 min.1-16/+0
| | | | | | | | | | This API is no longer used and bitbake has moved beyond Python 2.7.3 as the minimum version, so remove it. (Bitbake rev: 0eb7b5dd512ed8d8b77b5779858b9fbd99edb4a4) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Use a "fork" multiprocessing contextJoshua Watt49 min.7-9/+35
| | | | | | | | | | | | | | | | | Python 3.14 changes the default multiprocessing context from "fork" to "forkserver"; however bitbake heavily relies on "fork" to efficiently pass data to the child processes. As such, make "fork" context in the bb namespace and use it in place of the normal multiprocessing module. Note that multiprocessing contexts were added in Python 3.4, so this should be safe to use even before Python 3.14 [YOCTO #15858] (Bitbake rev: 62be9113d98fccb347c6aa0a10d5c4ee2857f8b6) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: Switch u-boot based test to use our own mirrorRichard Purdie25 hours1-1/+1
| | | | | | | | | The upstream servers are having issues so switch to our own shadow copy of the repo. (Bitbake rev: e910c7cd24fd366d6756641cd599c4efeb492e2a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Split profile reports into separate filesRichard Purdie43 hours2-13/+27
| | | | | | | | | Use a more logical name for the profile reports and put each report into a separate file since people struggle to discover them currently. (Bitbake rev: a8145c84e0899285a5e6a809f1515118b002b106) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Optimise signal/sigmask performanceRichard Purdie43 hours1-4/+13
| | | | | | | | | | | | | Running "time bitbake -pP idle" with a valid cache shows around 800,000 calls to enum creation from python's signal.py. We don't care about this overhead and it adversely affects cache load time quite badly. Try and use _signal directly, falling back to signal, which avoids this overhead we don't need and makes cache loading much faster. (Bitbake rev: ee5fce67ce35b025c68aa61e2e758903269ee346) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Add an option to specify what to profileRichard Purdie6 days3-6/+8
| | | | | | | | | | | | | | | | | | Starting with python 3.12, profiling now stays enabled over threads yet you can't extract the profile data in the threads themselves, which makes it difficult to use for our use case. Our main loop starts the idle loop which starts the parsing threads and this means we can't profile in the main loop and the parsing threads or the idle loop at the same time due to this. Add options to the commandline so you can specify which piece of bitbake you want to enable profiling for. This allows some profiling with python 3.12 onwards rather than crashing. (Bitbake rev: 09f29a4968841ee5070f70277ba8c253bb14f017) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Fix an event duplication raceRichard Purdie6 days1-2/+5
| | | | | | | | | | | | It is possible for multple bitbake threads to empty ui_queue in parallel leading to duplicate console messages and much confusion when debuging. Use the lock to extract the queue data which means only one thread will processing, removing the duplicate out of order messages. (Bitbake rev: 945095602e40d54efb8de494218f4a2b25c9969f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/process/utils: Create profiling common function to remove ↵Richard Purdie6 days3-49/+31
| | | | | | | | | | | | | | code duplication We have code duplication in the way we handle profiling of code sections. Create a common function in utils which covers this. The main loop and idle loop profile files were also reversed. Fix this and the naming, removing a couple of unused variables containing the profile log names in the process too. (Bitbake rev: b4f6bae97ac9607420fc49fd4c9e957d89c9a5f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Avoid file not found traceback in logsRichard Purdie6 days1-1/+4
| | | | | | | | | If the server is quickly stopped, we see tracebacks in the locks due to the file not existing. Hide these as they're not errors. (Bitbake rev: a7e1a07e9ef7e6f6a1bcaf567d7916a8ee1ef087) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/tests: fix typo in testGyorgy Sarvari9 days1-1/+1
| | | | | | | | | | | The test behavior did not change visibly though. "bitbake-selftest bb.tests.runqueue" passes completely, just like before. (Bitbake rev: 1751aed08f8472f20fcfbadbb09d35f951904952) Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Use shared counter for processing parser jobsJoshua Watt2025-07-091-12/+18
| | | | | | | | | | | | | | Instead of pre-partitioning which jobs will go to which parser processes, pass the list of all jobs to all the parser processes (efficiently via fork()), then used a shared counter of the next index in the list that needs to be processed. This allows the parser processes to run independently of needing to be feed by the parent process, and load balances them much better. (Bitbake rev: 373c4ddaf0e8128cc4f7d47aefa9860bd477a00f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add better parse debugRichard Purdie2025-07-071-1/+1
| | | | | | | | | If parsing ends early and unexpectedly, add some internal values to better understand why/how it failed. (Bitbake rev: 775f9720a17c9f3d6815d42c733ab5aaaa53749c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Try and avoid parsing hangsRichard Purdie2025-07-071-9/+15
| | | | | | | | | | | | | | | | | | | | | | We sometimes see hangs in parsing during automated testing. It appears that SIGINT was sent to the underlying processes which see KeyboardInterrupt but they're stuck trying to write into the results pipe. The SIGINT was probably from some kind of parsing failure which doens't happen often, hence the hang being rare (in the incompatible license selftests from OE). This patch: * sets a flag to indicate exit upon SIGINT so the exit is more graceful and a defined exit path * empties the results queue after we send the quit event * empties the results queue after the SIGINT for good measure * increases the 0.5s timeout to 2s since we now have some very slow to parse recipes due to class extensions (ptests) This should hopefully make the parsing failure codepaths more robust. (Bitbake rev: 5b533370595f83b87e480bace3e0b42c9ba61e22) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump version to 2.15.1Richard Purdie2025-07-011-1/+1
| | | | | | (Bitbake rev: f68b513c38fa33c89236efbaab2674a25983d5e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Refactor filemode variable conversion to a functionRichard Purdie2025-07-012-0/+28
| | | | | | | | | | | We have other places in the code where we need to take filemode/mask information from a bitbake variable and turn it into a real python number. Turn this internal code into public API in bb.utils and add some tests for it. (Bitbake rev: d89e30fb2fb15b09f2cb95c4e5aa9f749ca257ea) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: add wait_for decorator and build_file_sync() helperRoss Burton2025-07-011-1/+134
| | | | | | | | | | | | | | | | | The bitbake worker/server IPC is asynchronous, but tinfoil only has functionality to wait for a response on the build_targets() call. Extract the bulk of the "wait for events and handle errors" logic to a standalone wait_for wrapper, which is the build_targets code without the extra_events or event_callback arguments (for now). Then use this to create a build_file_sync() helper that just wraps the existing build_file() with @wait_for. (Bitbake rev: bacd125a9da66cd205f6ba2ab17930b976e82150) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Add tag to shallow clone tarball nameRichard Purdie2025-06-261-2/+2
| | | | | | | | | | | It makes sense to allow the shallow clone mirror tarball name to include the tag name so that tags can be added to existing urls and the tarballs will be maintained correctly. The code already allows this to be done easily just by moving the tag handling code. (Bitbake rev: 68fce3be14e4dd801661f4ef302d229fb16a04b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Improve shallow clone tag fetchingRichard Purdie2025-06-261-1/+1
| | | | | | | | | | Currently, tags are fetched as just a name, which works but means they're not seen as tags by git commands like git describe. Instead, fetch them as refs/tags/XXX which means such commands then work correctly. (Bitbake rev: c1f30ad61f5e55beb377451887bbbc5cb569f2e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests: Fix automated testing failuresRichard Purdie2025-06-212-4/+6
| | | | | | | | | | The display window scrolling mechanism we were using was failing in one case and needed in another. After trying various approaches, this one with window.scrollTo seems to work and fixes the test failures. (Bitbake rev: f1175778108a559e1dd0d55cb68f42816c86f393) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: add support for 'built-in' fragmentsAlexander Kanavin2025-06-202-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When reviewing proposed fragments to add settings for DISTRO and MACHINE, RP noted that such fragments only add clutter and overhead, and there's no need to maintain them as separate files. Rather when bitbake sees 'fragmentvar/fragmentvalue' it can expand that into FRAGMENTVAR = "fragmentvalue". To achieve that, 'addfragments' directive is extended with a parameter that sets the name of the variable that holds definitions of such built-in fragments, for example like this: "machine:MACHINE distro:DISTRO" Then each enabled fragment name is matched against these definitions and the respective variable is set, e.g. 'machine/qemuarm' would match 'machine:MACHINE' and result in MACHINE set to 'qemuarm'. This happens before any fragment files are looked up on disk, and no such lookup happens if there was a match, which should prevent possible misuse of the feature. So the builtin fragment definition is also an allowlist for them. Please also see the patches for oe-core that show an application of the feature. (Bitbake rev: 3b9d7bea915dc7f10e845854f1dae325743f9456) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: allow overriding default unpack directory with a variableAlexander Kanavin2025-06-201-1/+1
| | | | | | | | | | | This will allow setting the variable in oe-core such that it matches S, which in turn will allow dropping S from almost all git:// fetching recipes, and not having to set it in newly written ones. (Bitbake rev: 378db0fdd95f8704fccd852452555bfddcad384d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.15.0Richard Purdie2025-06-161-1/+1
| | | | | | | | | Update version to 2.15.0 for the development series and features needed for toolchain selection in OE. (Bitbake rev: c2f29c9475c4b9cdd12af1f8610f2675f8fdd964) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/BBHandler: Add support for BB_DEFER_BBCLASSESRichard Purdie2025-06-162-6/+12
| | | | | | | | | Add support for automatically promoting class inherits to deferred inherits by listing them in the BB_DEFER_BBCLASSES variable. (Bitbake rev: 8e741b2e885a12d119788d04aa4efcd724dd6bfa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Add event for deferred inheritsRichard Purdie2025-06-162-0/+13
| | | | | | | | | | | | | | | Now that deferred inherits are extension specific, we can pass this list to an event, which our metadata can use to set class overrides earlier (as an example). There are limitations to this, the list of classes is unexpanded and recursive classes are not visible. There isn't much that can be done about this, the ones we are interested in would usually be visible at the top level (such as class extensions). (Bitbake rev: 205d461c05fc7b4a7c81039af3bc3fd71cbb982c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast: Change deferred inherits to happen per recipeRichard Purdie2025-06-161-9/+11
| | | | | | | | | | | | | | | | | | | | | | | | Currently deferred inherits get processed once for all class extensions as a minor speed optimisation. Unfortunately this limits our options for being able to report deferred classes to our code. There are two challenges with using our deferred classes in OE at present. One is that PACKAGECONFIG values don't work well with class overrides like class-native if there are deferred classes based on PACKAGECONFIG, such as python support. The second is that toolchain selection is proving problematic to implement due to interactions between the toolchain deferred inherit, the class extensions and class overrides being very late. By changing deferred inherits to be recipe extension specific, we open the way to generate events and "peek" at where things will end up, allowing the class overrides to be set earlier. The class extension code is updated to use a deferred inherit for the class extension inheriting so that it is still inherited last. (Bitbake rev: 29277cf4d88eb4dfa9572851177d009eab5afd0c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/az: Add sanity check and clarify documentationRobbin Van Damme2025-06-021-0/+4
| | | | | | | | | | | | AZ_SAS token should be prefixed with a question mark. Add a sanity check for this and fix the documentation. [YOCTO #15882] (Bitbake rev: 22011765202514600314732b97f1bb938e21f585) Signed-off-by: Robbin Van Damme <robbinvandamme@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/buildtest: Switch to new CDNRichard Purdie2025-05-291-1/+1
| | | | | | | | | The project is switching the way handle our CDN provision of sstate objects, update the URL accordingly. (Bitbake rev: 3372524cb961d95993b27fe4a8d794cdb7255e09) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add test case to check shallow cloning using `PREMIRRORS`Koch, Stefan2025-05-281-0/+27
| | | | | | | (Bitbake rev: 6e1434d93d489aa4bab07777a7a9dc58ba0ca5a7) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure a valid symlink in `PREMIRRORS` case when using ↵Stefan Koch2025-05-282-1/+16
| | | | | | | | | | | | shallow cloning - Since `ud.path` contains in that case the `PREMIRRORS` prefix path, this change ensures that a correct symlink is set up. (Bitbake rev: 37ed18e45aa17406162efc5ee3ddb2d6b33d07b9) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Move the `ensure_symlink()` function into the `FetchMethod` ↵Stefan Koch2025-05-281-19/+17
| | | | | | | | | class (Bitbake rev: d934763f91b9bda85f273a44f12a04a3ca19fc28) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add support for .debs containing uncompressed data tarballsIsaac True2025-05-281-2/+2
| | | | | | | | | | | Some .deb files contain uncompressed data tarballs which do not have an additional file extension after `.tar`. Add support for such cases while preserving the existing behaviour. (Bitbake rev: e3834deb427ceb1d1ae9a96c1e0dec990d7f3805) Signed-off-by: Isaac True <isaac.true@emlix.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Optimize loop in scenequeue data constructionBenjamin Szőke2025-05-271-2/+1
| | | | | | | | | | | | In scenequeue data generation loop, progress bar update in each cycle cause a significant loss of speed. Remove progress bar update in for loop. It was become quite fast (faster 30 times, about) without progress bar update. (Bitbake rev: f775e53f1196e3e5d0ded277c061c5c268b3b28d) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Fix deprecation warning about sqlite adapterJoshua Watt2025-05-191-0/+17
| | | | | | | | | | | | The default adapters for sqlite datetime are deprecated as of Python 3.12, so implement our own. [YOCTO #15333] (Bitbake rev: 38a1d715bf58acbc9cb21eed413b3542c81cf15a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests: Update after release numbering changesRichard Purdie2025-05-161-1/+1
| | | | | | (Bitbake rev: f06112286ad141f0d5e4ae75a29f83cdd564c2ec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: pass failed task logs through the log infrastructureYoann Congal2025-05-161-3/+3
| | | | | | | | | | | | | By switching from print() to bb.plain() to print failing task logs, we allow them to be saved in BB_CONSOLELOG. Fixes [YOCTO #15798]: This allows AB reproducibility test to save the full log of the failing tasks and helps debugging. (Bitbake rev: d3b3ad32da7c7ebf61814fc807f8667a37aa149b) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/fixtures: Add whinlatter, drop styheadRichard Purdie2025-05-153-34/+97
| | | | | | | | | Update the fixtures generator table and update the fixtures removing styhead and adding whinlatter. (Bitbake rev: 72b1dee84df641cb24bc726f2dfe8f1cf1555247) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/tests/fetch: add a test case to ensure git shallow fetch ↵Chen Qi2025-05-151-0/+13
| | | | | | | | | | | | | | | | | | works for tag containing slash Add a test case to ensure git shallow fetch succeeds for SRC_URI with tag containing slash. For example, we want to succeed for SRC_URI like below: SRC_URI = "git://salsa.debian.org/debian/debianutils.git;protocol=https;branch=master;tag=debian/${PV}" See the following link for more information: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15862 (Bitbake rev: 919d4cf6e688e67229c46d30c84d523b21936377) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: fix shallow clone for tag containing slashChen Qi2025-05-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | If a tag contains slash, e.g., debian/5.22, then shallow clone fails because it's using a wrong ref. To reproduce the issue, add the following lines in local.conf: BB_GIT_SHALLOW = "1" BB_GENERATE_SHALLOW_TARBALLS = "1" And then run 'bitbake debianutils -c fetch'. What the original os.path.basename(ref) wanted to do is to remove the strings such as refs/heads/. So we do it explitly to fix this issue. Fixes: [YOCTO #15862] (Bitbake rev: c6d6999f1ed01e7445b8f177a888038edacf555c) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Test gitsm with LFSPhilip Lorenz2025-05-081-11/+111
| | | | | | | | | | | | | | Add a test case to verify that the gitsm fetcher properly handles repositories storing objects with LFS. The test case verifies that LFS objects are fetched on the initial clone but also ensures that consecutive updates extend the original clone with any newly referenced LFS objects. (Bitbake rev: 2a8722ddd155596862029f6ea34e1e92c77e0b7f) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix LFS object checkout in submodulesPhilip Lorenz2025-05-081-6/+5
| | | | | | | | | | | | | | | Skipping smudging prevents the LFS objects from replacing their placeholder files when `git submodule update` actually checks out the target revision in the submodule. Smudging cannot happen earlier as the clone stored in `.git/modules` is bare. This should be fine as long as all LFS objects are available in the download cache (which they are after the other fixes are applied). (Bitbake rev: d270e33a07c50bb9c08861cf9a6dc51e1fd2d874) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix incorrect lfs parametrization for submodulesPhilip Lorenz2025-05-081-1/+1
| | | | | | | | | | | The existing code would pass `True` or `False` to the git fetcher. As the fetcher expects `lfs` to be set to `1` this always lead to LFS fetching being disabled. (Bitbake rev: 5e487a5a096400271ed1e29b0df72903f2304e49) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Use git-lfs fetch to download objectsPhilip Lorenz2025-05-081-24/+1
| | | | | | | | | | | | | | | | | | | Its not clear which exact git-lfs versions failed to work with bare repositories, however git-lfs 2.13.2 which is shipped by Debian 10 (i.e. the oldest supported distribution by scarthgap) shows no issue when fetching into a bare repository. Switch to git-lfs fetch which in turn eliminates issues seen when using the gitsm fetcher with submodules utilizing lfs. In these scenarios, fetching of LFS objects did not actually happen as the gitsm fetcher parametrizes the to be fetched repositories with `bareclone=1` which in turn means that the target revision was never checked out (and therefore no LFS objects were fetched). (Bitbake rev: 501d3317df5780a5b0e3c2562b26ed276c1dbe90) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Simplify git LFS detectionPhilip Lorenz2025-05-081-11/+1
| | | | | | | | | | | | Its unclear why this function does not operate on the desired source revision to begin with (which really should be the decider on whether a particular source revision uses LFS or not). Simplify the decision logic by always checking the `.gitattributes` file of the target revision. (Bitbake rev: b3faa0ce5c0a6945f26b5b303a7f38c00d132397) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Check for git-lfs existence before using itPhilip Lorenz2025-05-082-31/+68
| | | | | | | | | | | | | So far, existence of `git-lfs` was only checked during unpacking. As the binary is also used in earlier steps also check for its existence there. Additionally, factor out the LFS existence check into a dedicated function and call it wherever git-lfs is used for the first time. (Bitbake rev: 5818367db9b261b7e07c347d38044e6cba8f9727) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Move commonly used imports to topPhilip Lorenz2025-04-291-17/+2
| | | | | | | | | | Avoid multiple import statements for anything that is used more than once. Additionally, drop no longer used imports. (Bitbake rev: 7c74310440f4d6ec47cf5bacf597e18308b3bb20) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Clean up no longer used name parameterPhilip Lorenz2025-04-291-4/+4
| | | | | | | | | There's no need to pass `name` when it is no longer used. (Bitbake rev: b132d35dee643e270e3e6dd536dcc90334a0111c) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-23 11:23:34 +0000