summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
Commit message (Collapse)AuthorAgeFilesLines
...
* bitbake: asyncrpc: Remove ClientPoolJoshua Watt2024-05-312-76/+2
| | | | | | | | | | | | Batching support on the client side has proven to be a much more effective way of dealing with server latency than multiple client connections and is also much nicer on the server, so drop the client pool support from asyncrpc and the hash server (Bitbake rev: 6f80560f1c7010d09fe5448fdde616aef8468102) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Drop client pool supportJoshua Watt2024-05-311-38/+15
| | | | | | | | | | Drops support for client pools, since batching support in the client code has proven to be much more effective (Bitbake rev: 85dafaf8e070459f7de7bfb37300d8b60a27002e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop older python version compatibility codeRichard Purdie2024-05-314-29/+6
| | | | | | | | | | | | | | cooker: We can call multiprocessing close() unconditionally and tweak a comment give 3.8 is now the minimum version. lib/bb: We can drop the logger addition code only needed before 3.6 asyncrpc/hashserv: Since the minimum version is 3.8, we can drop the conditional code. (Bitbake rev: 16f4386400f88ba50605307961c248bef09895c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Improve handling errors during parsing when profilingRichard Purdie2024-05-311-4/+5
| | | | | | | | | We've seeing profiling tracebacks when parse errors occur during profiling. Try and avoid these but not processing invalid profiles. (Bitbake rev: 171bd9dd575307fbd61b5179ad86131d76add067) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: server: Add support for SO_REUSEPORTJoshua Watt2024-05-301-7/+27
| | | | | | | | | | | | | | SO_REUSEPORT is a socket option that allows multiple servers to listen on the same TCP port, and the kernel will automatically load balance the connections between them. This is particularly helpful for the hash server since it runs in a single thread. To take advantage of a multi-core server, multiple servers can be started in parallel with this option (up to 1 per CPU) and the kernel will load balance between them. (Bitbake rev: d72d5a7decb489e2af0ebc43cfea0ca3e4353e9b) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Enable batching of unihash queriesJoshua Watt2024-05-301-3/+6
| | | | | | | | | | Uses the batching API of the client to reduce the effect of latency when making multiple queries to the server (Bitbake rev: a54734b4ac2ddb3bce004e576cf74d6ad6caf62a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb: Use namedtuple for Task dataJoshua Watt2024-05-282-18/+39
| | | | | | | | | | | | Task dependency data is becoming unwieldy with the number of indices it contains. Convert it to use a named tuple instead, which allows members to be indexed by a named property or an index (which allows it to retain backward compatibility). (Bitbake rev: 26446cca4d22734c3f1b328a205c169dadb7e494) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Improve rehash get_unihash parallelismRichard Purdie2024-05-281-2/+10
| | | | | | | | | Improve the rehash code to query unihashes in parallel since this is more efficient on slower links. (Bitbake rev: c1949d5350342eaaf6ab988d7bfba99496d55523) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Process unihashes in parallel at initRichard Purdie2024-05-282-9/+10
| | | | | | | | | | | | | | Improve the runqueue init code to call unihash queries in parallel since this is faster and more efficient, particularly on slower links with longer round trip times. The call to the function from cooker is unneeded since that function calls prepare() and hence this functionality will already have run, so drop that obsolete call. (Bitbake rev: 721c97a115a7a4bf21955be79391bd6e0099f40e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Allow rehash loop to exit in case of interruptsRichard Purdie2024-05-281-1/+3
| | | | | | | | | | The initial hash serve loop exits in the case where interrupts are present but probably checks a bit too often. Tweak that and also allow the slow rehash loop to break on interrupt, improving bitbake Ctrl+C response. (Bitbake rev: 4534365591fd17bcc2b684900863b67bc69519ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Add timing warnings around slow loopsRichard Purdie2024-05-281-0/+22
| | | | | | | | | | | | | With hashserve enabled, there are two slow paths/loops, one at initial runqueue generation and also during the rehash process when new outhashes are found. Add timing information at the hashserve log level for when these loops take longer than 30s or 60s overall. This will leave evidence in the logs when things are running particularly slowly. (Bitbake rev: 6c357ede08e0b2a93bdaad2c1d631994faf2b784) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ui/buildinfohelper: Add exception treatment to fix missing target_fileMarlon Rodriguez Garcia2024-05-281-1/+4
| | | | | | | | | | | Based on the discution on https://lists.yoctoproject.org/g/toaster/message/6157 in some cases the value for Target_file could be missing and is needed to bypass it to finish build. (Bitbake rev: c60f6d20911632b41473f7c8577949be2f99ad80) Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/npmsw: The fetcher shouldn't have any knowledge of SRichard Purdie2024-05-231-1/+1
| | | | | | | | | | | | | I don't know why there is hardcoded knowledge of S in the fetcher but there shouldn't be and the OE unpack changes highlight this doing things it shouldn't. Drop the S reference and use rootdir which is the only place it should be touching. (Bitbake rev: 84f102954e10a3390fca9c26d5c3c639e952a2c9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests.codeparser: add tests for shell expansionsAntonin Godard2024-05-211-0/+40
| | | | | | | | | | | Tests quotes around `` and $() expansions, nested and multiple expansions, and that escaped quotes are treated as characters by the parser. (Bitbake rev: d98130cb4d500c495bc692c56dde3e019f36320a) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: remove redundant list conversionAntonin Godard2024-05-211-1/+1
| | | | | | | (Bitbake rev: 89712949de9476e4674864a8dcd6862fefe92eae) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: support shell substitutions in quotesAntonin Godard2024-05-211-7/+22
| | | | | | | | | | | | | | | | | | | | | | | | | The current shell substitution mechanism only works without quotes. For example: var1=$(cmd1 ...) Will work and add `cmd1` to the correspondind `run.do_*` file. However, although quite common, this syntax is not supported: var1="$(cmd1 ...)" This commit adds this feature by adding a step to process_words() to check whether we are dealing with quotes first, and by iterating on what's between them to detect new shell substitution candidates. These candidates are tested and parsed like before in the next step. The original `part` being part of the candidates means the syntax var1=$(cmd1 ...) is still valid. (Bitbake rev: f56e1a37b2ba1773ed308043d7eb073cc2e6c06e) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure generateTaskDepTreeData fails for NoProviderRichard Purdie2024-05-171-3/+3
| | | | | | | | | | If an invalid provider is requested, error out early rather than trying to build partial runqueue data structures as the taskdep UI will have exited after seeing the bad provider. (Bitbake rev: a478087998cb794cc4e31189b3ce07973d3949bc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Improve/fix cache invalidation via mtimeRichard Purdie2024-05-161-4/+8
| | | | | | | | | | | | | | | | | | We have been seeing obscure failures in devtool, particularly on newer autobuilder workers where it appears the cache is assumed to be valid when it shouldn't be. We're using the 'seconds' granulation mtime field which is not really a good way of telling if a file has changed. We can switch to the "ns" version which is better however also add in inode number and size as precautions. We already have all this data and tuples are fast so there isn't really any cost to do so. This hopefully fixes [YOCTO #15318]. (Bitbake rev: d9e5d313c79500e3c70ab9c3239b6b2180194f67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc/client: Fix websockets minimum version for python 3.10Richard Purdie2024-05-161-1/+3
| | | | | | | | | | | | python 3.10 support is only available in websockets 10.0 and later: https://github.com/python-websockets/websockets/commit/08d8011132ba038b3f6c4d591189b57af4c9f147 Update the version for this case. This avoids failures on Ubuntu 22.04. (Bitbake rev: 0e4767c4a880408750e1a6855270c5a4eef8383d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: update to version 2.9.1Richard Purdie2024-05-081-1/+1
| | | | | | | | This allow the use of new siggen API (Bitbake rev: e53503546990adeab67b6d044fcce59dc5a3f455) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Check websockets versionjoshua Watt2024-05-081-0/+22
| | | | | | | | | | | Checks that the minimum version of the websockets module is present, and if not raises an ImportError. This allows the user to get earlier feedback if using websockets is going to succeed (Bitbake rev: 330ea6914aad65dc8b34c986c44779820c392f03) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle ImportError for websocketsjoshua Watt2024-05-081-1/+2
| | | | | | | | | | | | Handles ImportError when creating a hash equivalence to ping the server. This notifies user earlier with a more precise error if websockets can't be used, and also prevents passing a known bad upstream value to the local server (Bitbake rev: aa80b3cfc5d16dfba13ca7fb9b78bae179ce3b74) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen/runqueue: Report which dependencies affect the taskhashjoshua Watt2024-05-082-4/+6
| | | | | | | | | | | | | Report which task dependencies in BB_TASKDEPDATA are included in the taskhash. This allows tasks to identify which tasks dependencies may change without the task re-running. Knowing this information is important for tasks that want to transfer information from dependencies (such as SPDX) (Bitbake rev: a313b4f07727e8187526157ba039911c3f73dd46) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/svn: Fix mirroring issue with svnKari Sivonen2024-05-082-1/+12
| | | | | | | | | | | | Add return false to supports_checksum for svn fetcher which fhis fixes MIRROR usage for svn uris. Also add a testcase. [YOCTO #15473] (Bitbake rev: 21cfc7ae9a19f39ac8904e1c3466e7e499ac523f) Signed-off-by: Kari Sivonen <kari.sivonen@live.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Handle conflict between cwd and cleandirsRichard Purdie2024-05-081-0/+2
| | | | | | | | | | If the cwd of the task is also a cleandirs, you would see warnings from bitbake about being unable to obtain cwd during the task execution. Tweak the code to detect this and avoid the warnings. (Bitbake rev: 6c7fd60c10955b0f23f64b25b5b4e154eb22a8f8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/crate: add upstream latest version check functionAlexander Kanavin2024-05-082-0/+25
| | | | | | | | | | | This is actually rather easy: crate web API provides a json with all the versions, for example: https://crates.io/api/v1/crates/cargo-c/versions (Bitbake rev: f6c2755db9a1f88c8534193b420fa31d135945e6) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Use hash client to ping upstream serverjoshua Watt2024-05-021-4/+3
| | | | | | | | | | | | | | The cooker attempts to connect to the upstream hash equivalent server to warn the user early if it is misconfigured. However, this was making the assumption that it was a raw TCP connection and failed when attempting to use a websocket upstream server. Fix this by creating an hash client and using the ping API to check the server instead of using a raw socket. (Bitbake rev: 5e84c13a6c594ed34c341849806657ddda206714) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gcp: Add missing runfetchcmd importSven Schwermer2024-04-201-0/+1
| | | | | | | | | This adds the missing import. This bug was introduced with 1ab1d36c. (Bitbake rev: 97ffe14311407f6e705ec24b70870ab32f0637b9) Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.9.0 development version postreleaseRichard Purdie2024-04-161-1/+1
| | | | | | (Bitbake rev: 67a1aa8dbb3cb3a30fa7d697431ebb30323e4f28) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.8.0Richard Purdie2024-04-161-1/+1
| | | | | | (Bitbake rev: c86466d51e8ff14e57a734c1eec5bb651fdc73ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Capture SSL environment for hashserverJoshua Watt2024-04-161-24/+70
| | | | | | | | | | | | | | Now that the bitbake hash server supports SSL connections, we need to capture a few environment variables which can affect the ability to connect via SSL. Note that the variables are only put in place to affect the environment while actually invoking the server [RP: Tweak to use BB_ORIGENV as well] [RP: Tweak to handle os.environ restore correctly] (Bitbake rev: 0bacf6551821beb8915513b120ae672ae8eb1612) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Handle unclosed functions correctlyRichard Purdie2024-04-141-1/+7
| | | | | | | | | | | | | | | | | A function accidentally defined as: somefunction() { : } which is unclosed due to the space at the end, would currently silently cause breakage. Have the parser throw and error for this. [YOCTO #15470] (Bitbake rev: a7dce72da6be626734486808f1b731247697e638) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: include parse_address from hashservMichael Opdenacker2024-04-141-0/+23
| | | | | | | | | | | | | | Moving the code and related definitions from hashserv/__init__.py to asyncrpc/client.py, allowing this function to be used in other asyncrpc clients. (Bitbake rev: b67bb05e431414866b8e8c6a4c88d20b9cdb44a3) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Suggested-by: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftpsRob Woolley2024-04-111-1/+4
| | | | | | | | | | | | | | | Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Rob Woolley <rob.woolley@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add support for hashserve credentialsJoshua Watt2024-03-231-5/+16
| | | | | | | | | | Adds support for hashserver credentials to be specified in the SignatureGenerator (Bitbake rev: 741bef3755fde7bae1386aad575ea704d9fe0969) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Install Git LFS in local repository configDerek Erdmann2024-03-221-1/+1
| | | | | | | | | | | | | | | | | Git uses a lock file to prevent concurrent modifications to the global config, so if unpack tasks for different recipes try to run "git lfs install" simultaneously the operation can fail: error: could not lock config file /home/build/.gitconfig: File exists exit status 255 Run `git lfs install --force` to reset Git configuration. Adding "--local" sets the smudge and clean filters in the local repository's config instead of modifying the user's global config. (Bitbake rev: 328ca4de8422be514fa0d0c9e3cfd36bb9d3e9a7) Signed-off-by: Derek Erdmann <derek.erdmann@sonos.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: better estimate number of available cpusFelix Moessbauer2024-03-221-1/+4
| | | | | | | | | | | | | When running in a cgroup which is limited to a subset of cpus (via cpuset.cpus), cpu_count() should return the number of cpus that can be used instead of the number of cpus the system has. This also aligns the semantics with nproc. (Bitbake rev: a029bfe96c6542f178720c72a772b7ede9898118) Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix misleading "no output" msgViswanath Kraleti2024-03-221-1/+4
| | | | | | | | | | | When a command is run with a non-null log, errors are only output to the log and are not returned in the exception. In that case direct users to that logfile instead of telling the command had no output. (Bitbake rev: 944fe0a77932a5559e01ae6035c4bffa5185ea6a) Signed-off-by: Viswanath Kraleti <quic_vkraleti@quicinc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: handle URIs with single-valued query parametersRoss Burton2024-03-222-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Whilst typically the URI query is a list of key-value pairs, that's not actually required by the URI specification. For example: http://example.com/foo?bar is a valid query, but this will result in the fetcher raising an exception: File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__ self.query = self._param_str_split(urlp.query, "&") File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]: ValueError: not enough values to unpack (expected 2, got 1) In this case the query is just "bar", but the fetcher is trying to split it into a key-value pair. The URI object exposes the parsed query explicitly as a dictionary of key-value pairs, so we have to be a little creative here: if a value is None then it isn't a key-value pair, but a bare key. Fix this by handling elements without the deliminator in _param_str_split() (by assigning the value to None), and handle a None value when formatting the query in _param_str_join(). (Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: support NO_COLORPeter A. Bigot2024-03-222-2/+2
| | | | | | | | | | | | | Red text on a black background can make it difficult for people with visual impairments to read the text of error messages. Respect the presence of a non-empty NO_COLOR environment variable as an indication the user doesn't want colorization to be automatically enabled. See: https://no-color.org/ (Bitbake rev: d9986c54cd3d67ed1f7cb636b17696c8d0d4db85) Signed-off-by: Peter A. Bigot <pab@pabigot.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch2/git: Escape parentheses in git src namePhilippe Rivest2024-03-221-1/+1
| | | | | | | | | | | | | | | | | | FIXES [YOCTO #15404] When using git fetcher on a repo with parentheses in its URL, the invocation of the git clone command will fail. The clone directory is not quoted thus the shell will return an error and won't execute the command. (Bitbake rev: b5624ee5643d881afa004571a096a189ab5389b5) Cc: Philippe Rivest <privest@genetec.com> (Bitbake rev: 12f9738577934ad7c99f0770f1392a9d6050e7d6) Signed-off-by: Philippe Rivest <technophil98@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: allow '=' in environment variable valuesdavid d zuhn2024-03-221-1/+1
| | | | | | | | | | | | Limit the split to key & value (2 items) instead of the n items one can get if there are '=' characters in the value. Fixes [YOCTO #15447] (Bitbake rev: 86315961829ab1d137a0265cc246c44d3929e1fb) Signed-off-by: david d zuhn <david.zuhn@sonos.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: improve descriptions of '-S printdiff'Alexander Kanavin2024-03-221-1/+3
| | | | | | | (Bitbake rev: becf88c2250a47102c8d36ad8b40839e0bfa9137) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: remove BB_ENV_PASSTHROUGH from preserved_envvars()Michael Opdenacker2024-03-051-1/+0
| | | | | | | | | | | | | preserved_envvars() is used when the BB_ENV_PASSTHROUGH environment variable is not set. Therefore, its code shouldn't return this variable. (Bitbake rev: 0a33b560233b983456178541603ab96fea22238b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Reported-by: Robert P. J. Day <rpjday@crashcourse.ca> Tested-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Avoid clean upon failureRichard Purdie2024-03-032-2/+12
| | | | | | | | | | | | | | Currently when git fetches fail, it destroys all the existing local clone data. For large repositories this can introduce long build delays when for example, you just typo'd the git revision hash. The git fetcher should be able to recover most directories so when the fetch is for a git repo, avoid removing things unless clean is explicitly called (e.g. a -c cleanall task). (Bitbake rev: 1b3cd039fe19b24bd4be9a0202a98cdcbb0e9443) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: taskexp_ncurses: fix execution example in introductory commentUlrich Ölmann2024-02-281-1/+1
| | | | | | | | | | | | | | Drop the ".py" file extension from the "-u" option's argument that has been overlooked while applying the original patch (see [1]) to make the example work. While at it sort the recipes' names consistently with respect to what is found in the self-test examples below. [1] https://lore.kernel.org/bitbake-devel/6f2645a7c4db2ae149d387544d2b94209cfed3f4.camel@linuxfoundation.org/ (Bitbake rev: 1f4d517b7a0389f78d1f791135f8dc9120e9912b) Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add support for server headersJoshua Watt2024-02-272-3/+46
| | | | | | | | | | | | Adds support for asyncrpc servers to send connection headers to clients on connection. Since this is a breaking protocol change, clients must opt-in to expect headers from the server, corresponding to a version bump in the client protocol. (Bitbake rev: 1cb2b8be6cc5269553f549285592e47b7d29db03) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure that git LFS objects are availablePhilip Lorenz2024-02-232-6/+90
| | | | | | | | | | | | | | | | | | | | | The current implementation only performs a git lfs fetch alongside of a regular git fetch. This causes issues when the downloaded revision is already part of the fetched repository (e.g. because of moving back in history or the updated revision already being part of the repository at the time of the initial clone). Fix this by explicitly checking whether the required LFS objects are available in the downloade directory before confirming that a downloaded repository is up-to-date. This issue previously went unnoticed as git lfs would silently fetch the missing objects during the `unpack` task. With network isolation turned on, this no longer works, and unpacking fails. (Bitbake rev: cfae1556bf671acec119a6c8bbc4b667a856b9ae) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Add support for BB_LOADFACTOR_MAXRichard Purdie2024-02-231-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Some ditros don't enable /proc/pressure and it tends to be those which we see bitbake timeout issues on, seemingly as load gets too high and the bitbake processes don't get scheduled in for minutes at a time. Add support for stopping running extra tasks if the system load average goes above a certain threshold by setting BB_LOADFACTOR_MAX. The value used is scaled by CPU number, so a value of 1 would be when the load average equals the number of cpu cores of the system, under one only starts tasks when the load average is below the number of cores. This means you can centrally set a value such as 1.5 which will then scale correctly to different sized machines with differing numbers of CPUs. The pressure regulation is probably more accurate and responsive, however our graphs do show singificant load spikes on some workers and this patch is aimed at trying to avoid those. Pressure regulation is used where available in preference to this load factor regulation when both are set. (Bitbake rev: 14a27306f6dceb4999c2804ccae5a09cc3d8dd49) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "bitbake: wget.py: always use the custom user agent"Alexander Kanavin2024-02-201-4/+3
| | | | | | | | | | | | | | | | | | This reverts commit 987ab2a446aab235d6e86e97c05f25fb800d7acc. There's been a report that this breaks downloads from Jfrog Artifactory as self.user_agent is set to 'Mozilla Firefox', and when Artifactory sees that, it sends a response tailored for showing in an interactive browser (which in my opinion it has every right to). If we're using wget, we should say so via wget's default; handling uncooperative servers should be done on per-recipe basis, and ideally with tickets to admins of those servers. (Bitbake rev: feef5cd12e877f42ffcace168d44b0e6eb80a907) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-07 01:47:17 +0000