summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2/git: verify if local clone contains tagGyorgy Sarvari2025-08-192-3/+51
| | | | | | | | | | | | | | | | | | | | In case a recipe specifies a git SRC_URI along with revision and tag, but only the revision is present in the local clone without the tag (because it was tagged after it was cloned), then unpacking fails with the following error: ... rev-list -n 1 1.0 failed with exit code 128, output:\nfatal: ambiguous argument \'1.0\': unknown revision or path not in the working tree This happens because the during the download step only the revision's presence is verified to decide if the repository needs to be updated. To avoid this, check also if the tag is present in the local repository, when the "tag" tag is specified. (Bitbake rev: 546b347b4d3d82c01ecc99f45296f66e44638adc) Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-ref-variables: update BB_TASK_IONICE_LEVELPeter Marko2025-08-191-2/+2
| | | | | | | | | | | | | | | | | Provided example does not work due to two problems. CFQ scheduler was removed in kernel 4.20 (per [1]) Replace it with BFQ scheduler. Also fix typo "queu" -> "queue". [1] https://en.wikipedia.org/wiki/I/O_scheduling (Bitbake rev: 6716853e35ebc2e1523210a83b483cdacb600295) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: print() was eliminated from all loops for better performance.Benjamin Szőke2025-08-191-7/+7
| | | | | | | | | | | Refactoring prints, print() functions were eliminated from all loops and it uses "\n".join(...) in a single print() call for better performance. (Bitbake rev: c32c3d9b83818661e12f3e437563ab4e1fa05e15) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Use 40 Hz refresh rate (FPS) for footer update.Benjamin Szőke2025-08-191-4/+20
| | | | | | | | | | Refresh footer in 40 Hz to avoid heavy print() flooding but keep it fluent for human eyes. (Bitbake rev: c36efdf642d858c6997819744d00a3c1965c6417) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Use a StringIO buffer for update footer.Benjamin Szőke2025-08-191-10/+26
| | | | | | | | | | | Optimize printing in footer update with use a StringIO buffer and it prints content to terminal in a single call in every cycle. (Bitbake rev: 32ba622d78f20b231f30f848379b4bbc3d7414da) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progressbar: Add self._fd_console to use for self._handle_resize()Benjamin Szőke2025-08-191-2/+9
| | | | | | | | | | Introduce self._fd_console as a dedicated attribute of self._handle_resize(). (Bitbake rev: f8c76eb89d52b1c28407f0b52dfe4318faa47cd2) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Update version to 2.15.2Richard Purdie2025-08-122-2/+2
| | | | | | (Bitbake rev: 7ad404fd1aa0a48978c0351c5f52f17b8992c8b8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: use enum to represent runQueue state to improve readabilityChris Laplante2025-08-121-44/+46
| | | | | | | (Bitbake rev: e61265570f5415d647666891d806836437330219) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/runqueue: remove unused 'd' arg from check_for_interruptsChris Laplante2025-08-122-17/+17
| | | | | | | (Bitbake rev: e0c5e94cc14dd6b1cbde491073c1f9407c97e428) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: remove some redundant control flowChris Laplante2025-08-121-4/+1
| | | | | | | | | | | Prior to commit aa84a900e ("cooker: Ensure delays are accurately transfered to the idle loops from runqueue"), this was necessary. But now retval is returned directly. (Bitbake rev: c2eb4336fe10b1bf8bbc6291c32ca362840f39e1) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Add bb.utils.explode_deps as a filter functionJoshua Watt2025-08-121-0/+2
| | | | | | | | | Marks bb.utils.explode_deps as callable from filter functions (Bitbake rev: bb07003641e76de994482f7835a432f20297af96) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add setVarFilter function to implement variabl filteringRichard Purdie2025-08-121-0/+19
| | | | | | | | | | | | | | | | Adds a new setVarFilter() API to the data store allowing filters to be applied to variables. Note that filters are applied to the non-override part of the variable name so a filter set against RDEPENDS would apply against RDEPENDS:${PN} and friends. The filter function is applied before returning the final variable value. (Bitbake rev: a9471c10d1de039474ddb4738abd286b928d82f4) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add filter supportJoshua Watt2025-08-123-0/+231
| | | | | | | | | | | | | | Add the python API for applying filters to a string and being able to register functions as filters. Filter functions are pure functions where an input is translated into an output and there are no external data accesses. This means translations can be cached as they won't change. (Bitbake rev: 7d25d7511ca14213eea78ee739d260295cfa4045) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ignore KeyErrors during shutdownRichard Purdie2025-08-041-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When parsing failures occur, trap KeyError to avoid these kinds of tracebacks (from a parsing error in meta-ti). bb.parse.ParseError: ParseError at /srv/pokybuild/yocto-worker/check-layer-nightly/build/meta-ti/meta-ti-bsp/recipes-kernel/linux/linux-ti-mainline_git.bb:6: Could not inherit file classes/${KERNEL_BASE_CLASS}.bbclass During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/command.py", line 123, in runAsyncCommand self.cooker.updateCache() File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 1639, in updateCache if not self.parser.parse_next(): File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2314, in parse_next self.shutdown(clean=False, eventmsg=str(exc)) File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2209, in shutdown read_results() File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2195, in read_results self.result_queue.get(timeout=0.25) File "/usr/lib/python3.10/multiprocessing/queues.py", line 122, in get return _ForkingPickler.loads(res) File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cache.py", line 338, in __setstate__ setattr(self, key, self._restore(state[key], pid)) File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cache.py", line 318, in _restore ret[dep] = map[mapnum] KeyError: 156 (Bitbake rev: 750c68ee7ee3f3d4518348e4c948243504880770) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: ignore comments in .gitattributes when detecting LFSGyorgy Sarvari2025-07-311-1/+1
| | | | | | | | | | | | | | | | | | | | Fixes [YOCTO 15917] When fetching a git repository the .gitattributes file is scanned, checking if LFS support is required for the repository. This scan checks if the word "lfs" is present in the file, however the used regex doesn't account for comments, which makes some repositories[1] be to misidentified as requiring LFS support (which fails fetching, in case lfs support isn't installed on the build host). To avoid it, change the used regex to ignore lines starting with "#". [1]: e.g. https://github.com/MicrochipTech/cryptoauthlib (Bitbake rev: 7917a758fc328747116c7899e689171bd0efc883) Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-ref-variables: document BB_DEFER_BBCLASSESAntonin Godard2025-07-312-0/+22
| | | | | | | | | | | Document the BB_DEFER_BBCLASSES variable added in 8e741b2e885a ("ast/BBHandler: Add support for BB_DEFER_BBCLASSES"). (Bitbake rev: 3f9eba26b29db526fed263a7e70b824988e17656) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-intro: Mention existence of three supported ↵Robert P. J. Day2025-07-311-0/+12
| | | | | | | | | | | | | "classes" directories Early in Section 1, mention that BitBake supports three related classes directories. (Bitbake rev: 0d5201e904b0e6f4dd9f7bb116dda508ec09b78a) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-intro: Add examples to show general wildcardingRobert P. J. Day2025-07-311-0/+2
| | | | | | | | | | | Add a couple more examples to the wildcarding section to make sure readers understand it's not just a single character wildcard. (Bitbake rev: 572062ba1f0a2953a62ef1974e35134fcb462f5e) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-ref-variables: fixupsAntonin Godard2025-07-311-11/+11
| | | | | | | | | | | | Fix typos reported by Quentin here: https://lore.kernel.org/r/cee995db-4c3d-4bf0-813d-ce3f5f8f92e3@cherry.de https://lore.kernel.org/r/73497b5e-721e-4e5f-beb4-c56ab9178c9f@cherry.de Cc: Quentin Schulz <quentin.schulz@cherry.de> (Bitbake rev: add8a7ac093d60e731a638c25ec15efcc3988781) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: clarify significance of "include_all" directiveRobert P. J. Day2025-07-281-30/+39
| | | | | | | | | | | | Rewrite the include and include_all sections to drive home the fact that the include_all directive is relevant in only very specific cases, and not something developers should expect to use in normal operation. (Bitbake rev: 4b3bfe70d02cc1c11972357e2dc595acc75056e5) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: Keep query parameters in URL during checkstatusPhilippe-Alexandre Mathieu2025-07-281-1/+4
| | | | | | | | | | | | | | | | | | | | When recreating the uri in wget's checkstatus method, we only use the scheme, netloc and path. This completely strips the query parameters from the final URI and potentially breaks the checking functionality from certain fetchers. This is the case for the Azure storage fetcher, as it requires a SAS token that is formatted as a series of query parameters. The error manifests itself when using a private storage account as a PREMIRROR or SSTATE_MIRROR (since regular SRC_URI won't run the checkstatus). This problem is present in scarthgap, but wasn't in kirkstone. CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> (Bitbake rev: 096301250455e2a83bdd818a56317c62436c9981) Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: sigen: Avoid bitbake abort with EOFerror from an incomplete ↵Pedro Ferreira2025-07-231-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | hashserv communication The issue itself is sporadic but aborts the build with an uncaught exception. The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_report_unihash(d) 0003: File: '(...)/poky/meta/classes/sstate.bbclass', lineno: 882, function: sstate_report_unihash 0878: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) 0879: 0880: if report_unihash: 0881: ss = sstate_state_fromvars(d) *** 0882: report_unihash(os.getcwd(), ss['task'], d) 0883:} 0884: 0885:# 0886:# Shell function to decompress and prepare a package for installation File: '(...)/poky/lib/bb/siggen.py', lineno: 651, function: report_unihash 0647: method = self.method 0648: if tid in self.extramethod: 0649: method = method + self.extramethod[tid] 0650: *** 0651: data = self.client().report_unihash(taskhash, method, outhash, unihash, extra_data) 0652: new_unihash = data['unihash'] 0653: 0654: if new_unihash != unihash: 0655: hashequiv_logger.debug('Task %s unihash changed %s -> %s by server %s' % (taskhash, unihash, new_unihash, self.server)) File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 139, function: wrapper 0135: pass 0136: 0137: def _get_downcall_wrapper(self, downcall): 0138: def wrapper(*args, **kwargs): *** 0139: return self.loop.run_until_complete(downcall(*args, **kwargs)) 0140: 0141: return wrapper 0142: 0143: def _add_methods(self, *methods): File: '/usr/lib/python3.9/asyncio/base_events.py', lineno: 642, function: run_until_complete 0638: future.remove_done_callback(_run_until_complete_cb) 0639: if not future.done(): 0640: raise RuntimeError('Event loop stopped before Future completed.') 0641: *** 0642: return future.result() 0643: 0644: def stop(self): 0645: """Stop running the event loop. 0646: File: '(...)/poky/lib/hashserv/client.py', lineno: 70, function: report_unihash 0066: m["taskhash"] = taskhash 0067: m["method"] = method 0068: m["outhash"] = outhash 0069: m["unihash"] = unihash *** 0070: return await self.invoke({"report": m}) 0071: 0072: async def report_unihash_equiv(self, taskhash, method, unihash, extra={}): 0073: await self._set_mode(self.MODE_NORMAL) 0074: m = extra.copy() File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 104, function: invoke 0100: async def proc(): 0101: await self.socket.send_message(msg) 0102: return await self.socket.recv_message() 0103: *** 0104: return await self._send_wrapper(proc) 0105: 0106: async def ping(self): 0107: return await self.invoke({"ping": {}}) 0108: File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 82, function: _send_wrapper 0078: async def _send_wrapper(self, proc): 0079: count = 0 0080: while True: 0081: try: *** 0082: await self.connect() 0083: return await proc() 0084: except ( 0085: OSError, 0086: ConnectionError, File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 70, function: connect 0066: await self.socket.send("") 0067: 0068: async def connect(self): 0069: if self.socket is None: *** 0070: self.socket = await self._connect_sock() 0071: await self.setup_connection() 0072: 0073: async def close(self): 0074: if self.socket is not None: File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 55, function: connect_sock 0051: import websockets 0052: 0053: async def connect_sock(): 0054: try: *** 0055: websocket = await websockets.connect(uri, ping_interval=None) 0056: except (OSError, asyncio.exceptions.TimeoutError, websockets.InvalidHandshake, websockets.InvalidURI) as exc: 0057: raise ConnectionError("Could not connect to websocket: %s" % exc) from exc 0058: return WebsocketConnection(websocket, self.timeout) 0059: File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 444, function: __await_impl__ 0440: async with asyncio_timeout(self.open_timeout): 0441: for _ in range(MAX_REDIRECTS): 0442: self.connection = await self.create_connection() 0443: try: *** 0444: await self.connection.handshake(*self.handshake_args) 0445: except asyncio.CancelledError: 0446: self.connection.close_transport() 0447: raise 0448: except Exception as exc: File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 104, function: handshake 0100: # receiving a response, when the response cannot be parsed, or when the 0101: # response fails the handshake. 0102: 0103: if self.protocol.handshake_exc is not None: *** 0104: raise self.protocol.handshake_exc 0105: 0106: def process_event(self, event: Event) -> None: 0107: """ 0108: Process one incoming event. File: '/usr/local/lib/python3.9/dist-packages/websockets/client.py', lineno: 315, function: parse 0311: 0312: def parse(self) -> Generator[None]: 0313: if self.state is CONNECTING: 0314: try: *** 0315: response = yield from Response.parse( 0316: self.reader.read_line, 0317: self.reader.read_exact, 0318: self.reader.read_to_eof, 0319: ) File: '/usr/local/lib/python3.9/dist-packages/websockets/http11.py', lineno: 238, function: parse 0234: 0235: try: 0236: status_line = yield from parse_line(read_line) 0237: except EOFError as exc: *** 0238: raise EOFError("connection closed while reading HTTP status line") from exc 0239: 0240: try: 0241: protocol, raw_status_code, raw_reason = status_line.split(b" ", 2) 0242: except ValueError: # not enough values to unpack (expected 3, got 1-2) Exception: EOFError: connection closed while reading HTTP status line (Bitbake rev: 5ba7c2f0797a72536a81f57276d4e5c75f23011c) Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Remove multiprocessingpoolJoshua Watt2025-07-231-16/+0
| | | | | | | | | | This API is no longer used and bitbake has moved beyond Python 2.7.3 as the minimum version, so remove it. (Bitbake rev: 0eb7b5dd512ed8d8b77b5779858b9fbd99edb4a4) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Use a "fork" multiprocessing contextJoshua Watt2025-07-237-9/+35
| | | | | | | | | | | | | | | | | Python 3.14 changes the default multiprocessing context from "fork" to "forkserver"; however bitbake heavily relies on "fork" to efficiently pass data to the child processes. As such, make "fork" context in the bb namespace and use it in place of the normal multiprocessing module. Note that multiprocessing contexts were added in Python 3.4, so this should be safe to use even before Python 3.14 [YOCTO #15858] (Bitbake rev: 62be9113d98fccb347c6aa0a10d5c4ee2857f8b6) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-fetching: update the Git fetcher tag ↵Antonin Godard2025-07-221-4/+6
| | | | | | | | | | | | description After commit d591d7633fe8 ("fetch/git: Rework tag parameter handling"), update the description of the tag= parameter for the Git fetcher. (Bitbake rev: 85b31a55d114a1430868233d56573b470fef8908) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: Switch u-boot based test to use our own mirrorRichard Purdie2025-07-221-1/+1
| | | | | | | | | The upstream servers are having issues so switch to our own shadow copy of the repo. (Bitbake rev: e910c7cd24fd366d6756641cd599c4efeb492e2a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Split profile reports into separate filesRichard Purdie2025-07-212-13/+27
| | | | | | | | | Use a more logical name for the profile reports and put each report into a separate file since people struggle to discover them currently. (Bitbake rev: a8145c84e0899285a5e6a809f1515118b002b106) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Optimise signal/sigmask performanceRichard Purdie2025-07-211-4/+13
| | | | | | | | | | | | | Running "time bitbake -pP idle" with a valid cache shows around 800,000 calls to enum creation from python's signal.py. We don't care about this overhead and it adversely affects cache load time quite badly. Try and use _signal directly, falling back to signal, which avoids this overhead we don't need and makes cache loading much faster. (Bitbake rev: ee5fce67ce35b025c68aa61e2e758903269ee346) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/bitbake-user-manual-fetching: remove 'rev' default valueAntonin Godard2025-07-181-2/+2
| | | | | | | | | | | Remove rev's "master" default value. Mention that it must match SRCREV, if set. Reported-by: Quentin Schulz <quentin.schulz@cherry.de> (Bitbake rev: 2519b317e4afb1686f907274715a9e2b9c6457f4) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Add an option to specify what to profileRichard Purdie2025-07-174-7/+9
| | | | | | | | | | | | | | | | | | Starting with python 3.12, profiling now stays enabled over threads yet you can't extract the profile data in the threads themselves, which makes it difficult to use for our use case. Our main loop starts the idle loop which starts the parsing threads and this means we can't profile in the main loop and the parsing threads or the idle loop at the same time due to this. Add options to the commandline so you can specify which piece of bitbake you want to enable profiling for. This allows some profiling with python 3.12 onwards rather than crashing. (Bitbake rev: 09f29a4968841ee5070f70277ba8c253bb14f017) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Fix an event duplication raceRichard Purdie2025-07-171-2/+5
| | | | | | | | | | | | It is possible for multple bitbake threads to empty ui_queue in parallel leading to duplicate console messages and much confusion when debuging. Use the lock to extract the queue data which means only one thread will processing, removing the duplicate out of order messages. (Bitbake rev: 945095602e40d54efb8de494218f4a2b25c9969f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/process/utils: Create profiling common function to remove ↵Richard Purdie2025-07-173-49/+31
| | | | | | | | | | | | | | code duplication We have code duplication in the way we handle profiling of code sections. Create a common function in utils which covers this. The main loop and idle loop profile files were also reversed. Fix this and the naming, removing a couple of unused variables containing the profile log names in the process too. (Bitbake rev: b4f6bae97ac9607420fc49fd4c9e957d89c9a5f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Avoid file not found traceback in logsRichard Purdie2025-07-171-1/+4
| | | | | | | | | If the server is quickly stopped, we see tracebacks in the locks due to the file not existing. Hide these as they're not errors. (Bitbake rev: a7e1a07e9ef7e6f6a1bcaf567d7916a8ee1ef087) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/tests: fix typo in testGyorgy Sarvari2025-07-141-1/+1
| | | | | | | | | | | The test behavior did not change visibly though. "bitbake-selftest bb.tests.runqueue" passes completely, just like before. (Bitbake rev: 1751aed08f8472f20fcfbadbb09d35f951904952) Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Use shared counter for processing parser jobsJoshua Watt2025-07-091-12/+18
| | | | | | | | | | | | | | Instead of pre-partitioning which jobs will go to which parser processes, pass the list of all jobs to all the parser processes (efficiently via fork()), then used a shared counter of the next index in the list that needs to be processed. This allows the parser processes to run independently of needing to be feed by the parent process, and load balances them much better. (Bitbake rev: 373c4ddaf0e8128cc4f7d47aefa9860bd477a00f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add better parse debugRichard Purdie2025-07-071-1/+1
| | | | | | | | | If parsing ends early and unexpectedly, add some internal values to better understand why/how it failed. (Bitbake rev: 775f9720a17c9f3d6815d42c733ab5aaaa53749c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Try and avoid parsing hangsRichard Purdie2025-07-071-9/+15
| | | | | | | | | | | | | | | | | | | | | | We sometimes see hangs in parsing during automated testing. It appears that SIGINT was sent to the underlying processes which see KeyboardInterrupt but they're stuck trying to write into the results pipe. The SIGINT was probably from some kind of parsing failure which doens't happen often, hence the hang being rare (in the incompatible license selftests from OE). This patch: * sets a flag to indicate exit upon SIGINT so the exit is more graceful and a defined exit path * empties the results queue after we send the quit event * empties the results queue after the SIGINT for good measure * increases the 0.5s timeout to 2s since we now have some very slow to parse recipes due to class extensions (ptests) This should hopefully make the parsing failure codepaths more robust. (Bitbake rev: 5b533370595f83b87e480bace3e0b42c9ba61e22) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump version to 2.15.1Richard Purdie2025-07-012-2/+2
| | | | | | (Bitbake rev: f68b513c38fa33c89236efbaab2674a25983d5e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Refactor filemode variable conversion to a functionRichard Purdie2025-07-013-5/+30
| | | | | | | | | | | We have other places in the code where we need to take filemode/mask information from a bitbake variable and turn it into a real python number. Turn this internal code into public API in bb.utils and add some tests for it. (Bitbake rev: d89e30fb2fb15b09f2cb95c4e5aa9f749ca257ea) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: add wait_for decorator and build_file_sync() helperRoss Burton2025-07-011-1/+134
| | | | | | | | | | | | | | | | | The bitbake worker/server IPC is asynchronous, but tinfoil only has functionality to wait for a response on the build_targets() call. Extract the bulk of the "wait for events and handle errors" logic to a standalone wait_for wrapper, which is the build_targets code without the extra_events or event_callback arguments (for now). Then use this to create a build_file_sync() helper that just wraps the existing build_file() with @wait_for. (Bitbake rev: bacd125a9da66cd205f6ba2ab17930b976e82150) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual-ref-variables: improve BB_PRESSURE_* docAntonin Godard2025-06-261-6/+64
| | | | | | | | | | | | | | | | The current default value for BB_PRESSURE_MAX_CPU is 500, which is really low for most systems. Provide a value of 15000 which limits the load average without being extremely slow either. Provide similar values for BB_PRESSURE_MAX_IO and BB_PRESSURE_MAX_MEMORY. Mention that these should be adjusted depending on the need, and warn about the potential spam of messages when the value is too low. (Bitbake rev: 09baa527dafca4bdf56a9189f6b3f7512886cbfb) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual-ref-variables: document ↵Antonin Godard2025-06-261-0/+5
| | | | | | | | | | | | BB_GIT_DEFAULT_DESTSUFFIX This variable was recently added with 378db0fdd95f ("fetch2/git: allow overriding default unpack directory with a variable"). (Bitbake rev: a569be0d369cb8b9457beedfeb75b984f751ba5a) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Add tag to shallow clone tarball nameRichard Purdie2025-06-261-2/+2
| | | | | | | | | | | It makes sense to allow the shallow clone mirror tarball name to include the tag name so that tags can be added to existing urls and the tarballs will be maintained correctly. The code already allows this to be done easily just by moving the tag handling code. (Bitbake rev: 68fce3be14e4dd801661f4ef302d229fb16a04b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Improve shallow clone tag fetchingRichard Purdie2025-06-261-1/+1
| | | | | | | | | | Currently, tags are fetched as just a name, which works but means they're not seen as tags by git commands like git describe. Instead, fetch them as refs/tags/XXX which means such commands then work correctly. (Bitbake rev: c1f30ad61f5e55beb377451887bbbc5cb569f2e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: correctly reference UNPACKDIR when describing ↵Alexander Kanavin2025-06-261-14/+7
| | | | | | | | | | | unpacking Also, drop an obsolete S = WORKDIR/git note. (Bitbake rev: 3884b386cda649030f59e3040dd08e6aa2ec50cf) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests: Fix automated testing failuresRichard Purdie2025-06-212-4/+6
| | | | | | | | | | The display window scrolling mechanism we were using was failing in one case and needed in another. After trying various approaches, this one with window.scrollTo seems to work and fixes the test failures. (Bitbake rev: f1175778108a559e1dd0d55cb68f42816c86f393) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: add support for 'built-in' fragmentsAlexander Kanavin2025-06-203-5/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When reviewing proposed fragments to add settings for DISTRO and MACHINE, RP noted that such fragments only add clutter and overhead, and there's no need to maintain them as separate files. Rather when bitbake sees 'fragmentvar/fragmentvalue' it can expand that into FRAGMENTVAR = "fragmentvalue". To achieve that, 'addfragments' directive is extended with a parameter that sets the name of the variable that holds definitions of such built-in fragments, for example like this: "machine:MACHINE distro:DISTRO" Then each enabled fragment name is matched against these definitions and the respective variable is set, e.g. 'machine/qemuarm' would match 'machine:MACHINE' and result in MACHINE set to 'qemuarm'. This happens before any fragment files are looked up on disk, and no such lookup happens if there was a match, which should prevent possible misuse of the feature. So the builtin fragment definition is also an allowlist for them. Please also see the patches for oe-core that show an application of the feature. (Bitbake rev: 3b9d7bea915dc7f10e845854f1dae325743f9456) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: allow overriding default unpack directory with a variableAlexander Kanavin2025-06-201-1/+1
| | | | | | | | | | | This will allow setting the variable in oe-core such that it matches S, which in turn will allow dropping S from almost all git:// fetching recipes, and not having to set it in newly written ones. (Bitbake rev: 378db0fdd95f8704fccd852452555bfddcad384d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.15.0Richard Purdie2025-06-162-2/+2
| | | | | | | | | Update version to 2.15.0 for the development series and features needed for toolchain selection in OE. (Bitbake rev: c2f29c9475c4b9cdd12af1f8610f2675f8fdd964) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-getvar: skip info output of bitbake for quiethongxu2025-06-161-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Calling oe-debuginfod in a build failed: ... $ oe-debuginfod |Getting sysroot... |Error: NOTE: Reconnecting to bitbake server... |NOTE: Retrying server connection (#1)... (18:55:53.009687) |path-to-build/tmp/work/x86_64-linux/elfutils-native/0.192/recipe-sysroot-native doesn't exist. |Have you run 'bitbake elfutils-native -caddto_recipe_sysroot'? ... The script oe-debuginfod calls bitbake-getvar to get sysroot, the output of bitbake-getvar was mixed with info output of bitbake ... NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection (#1)... (18:55:53.009687) ... Set logger level to logging.WARNING to skip info output for quiet (Bitbake rev: 873c524e1a33846df8f34b7c87b298349277b3d5) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-09-02 21:05:15 +0000