summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
Commit message (Collapse)AuthorAgeFilesLines
...
* bitbake: runqueue: Remove tie between rqexe and starts_workerRichard Purdie2023-12-171-15/+8
| | | | | | | | | | | We've been moving to try and separate several pieces of runqueue. Allow start_worker to operate separately to rqexe since they don't need to be tied. This allows rqexe to be available to print_diff for future improvements. (Bitbake rev: 834e452243ff2eea6e8e2e7f4935b5233ffb4b00) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Fix mkdir with PosixPathRichard Purdie2023-12-171-1/+1
| | | | | | | | | Avoid: Exception: AttributeError: 'PosixPath' object has no attribute 'find' (Bitbake rev: 0b37fe89ba12549109905b6d0e6d07d342162436) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: Added new feature to import eventlogs from command line ↵Marlon Rodriguez Garcia2023-12-122-1/+87
| | | | | | | | | | | | | | | | | | | | | | | | into toaster using replay functionality Added a new button on the base template to access a new template. Added a model register the information on the builds and generate access links Added a form to include the option to load specific files Added jquery and ajax functions to block screen and redirect to build page when import eventlogs is trigger Added a new button on landing page linked to import build page, and set min-height of buttons in landing page for uniformity Removed test assertion to check command line build in content, because new button contains text Updated toaster_eventreplay to use library Fix test in test_layerdetails_page Rebased from master This feature uses the value from the variable BB_DEFAULT_EVENTLOG to read the files created by bitbake Exclude listing of files that don't contain the allvariables definitions used to replay builds This part of the feature should be revisited. Over a long period of time, the BB_DEFAULT_EVENTLOG will exponentially increase the size of the log file and cause bottlenecks when importing. (Bitbake rev: ab96cafe03d8bab33c1de09602cc62bd6974f157) Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Do not create directories with ${ in the namePavel Zhukov2023-12-081-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases ${ may not be expanded in the WORKDIR (one of the cases is undefined variable) and causes cryptic failures [1]. Guard this by erroring out if directory name contains ${ Fixes: [Yocto #15255] [1] ERROR: x-native-1.0+${SRCPV}-r0 do_deploy_source_date_epoch: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_hardcode_path(d) 0003: File: '/home/mischief/src/poky/meta/classes-global/sstate.bbclass', lineno: 654, function: sstate_hardcode_path 0650: bb.note("Removing hardcoded paths from sstate package: '%s'" % (sstate_hardcode_cmd)) 0651: subprocess.check_output(sstate_hardcode_cmd, shell=True, cwd=sstate_builddir) 0652: 0653: # If the fixmefn is empty, remove it.. *** 0654: if os.stat(fixmefn).st_size == 0: 0655: os.remove(fixmefn) 0656: else: 0657: bb.note("Replacing absolute paths in fixmepath file: '%s'" % (sstate_filelist_relative_cmd)) 0658: subprocess.check_output(sstate_filelist_relative_cmd, shell=True) Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/mischief/src/poky/build/tmp/work/core2-64-poky-linux/x-native/1.0+${SRCPV}-r0/sstate-build-deploy_source_date_epoch/fixmepath' (Bitbake rev: e91c256ec076e70cf8a18e369fe7862e50618c48) Signed-off-by: Pavel Zhukov <pavel@zhukoff.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Move to version 2.6.1 to mark runqueue changesRichard Purdie2023-12-061-1/+1
| | | | | | (Bitbake rev: 651a6dcf6f8ff33a4e9290a37c23e4f243974ac3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/toaster: Fix assertEquals deprecation warningsRichard Purdie2023-12-061-1/+1
| | | | | | | | | Fix: DeprecationWarning: Please use assertEqual instead (Bitbake rev: dd990ea6843685927954101feb729f3faa3a16d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Improve inter setscene task dependency handlingRichard Purdie2023-12-061-13/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The way the code currently handles dependencies between setscene tasks is fairly poor, basically by deleting chunks of dependencies and adding reversed dependency relationships. This was once the best way to handle things but now a lot of the surrounding code has changed and this approach is suboptimal and can be improved. This change firstly adds debug logging for "hard" setscene task dependencies since previously the codepaths were missing from logs making them very hard to read. The changes to the setscene dependency graph are removed entirely this these altered graphs were a significant source of problems. Instead, if a hard dependency is run into, we mark the hard dependency as buildable and defer the task until the hard dependencies are met. The code now also skips the check_dependencies() code for hard dependencies since previously that code was having to list all possible hard dependencies. We don't need to do that as we can safely assume hard dependencies are required. With these changes to runqueue's behaviour, we stand some chance of being able to fix other bugs in OE-Core related to useradd for example. (Bitbake rev: 367789b53c1c22ec26e0f4836cdf2bdd9c7d84fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure GCP fetcher checks if file exists before download.Charlie Johnston2023-12-021-2/+5
| | | | | | | | | | | | | | | | | The GCP fetcher was calling bb.fetch2.check_network_access with "gsutil stat" as the command, but then never actually ran that command to check if the file exists. In cases where the file did not exist in a gs:// premirror, this would lead to an unhandled exception from do_fetch when the GCP python API tried to perform the download. This change resolves that issue by adding a runfetchcmd to call gsutil. (Bitbake rev: 1ab1d36c0af6fc58a974106b61ff4d37da6cb229) Signed-off-by: Charlie Johnston <charlie.johnston@loftorbital.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: tests: Fix duplicate test_underscore_override testRichard Haar2023-12-021-10/+10
| | | | | | | | | Found a duplicate test, added _2 suffix to one, 74 tests now pass up from 73. (Bitbake rev: ae2a19dadb4f3065b8731a61f45f29e6a70af402) Signed-off-by: Richard Haar <rh@richhaar.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Avoid eventlog variable listing lockupsRichard Purdie2023-11-271-25/+11
| | | | | | | | | | | | | | | If the event log is enabled and parsing the metadata triggers log messages, the event code and deadlock. Iterating the variables inside the event handling code causes this. SOURCE_DATE_EPOCH triggers a python function which calls bb.debug() and can trigger a lockup as one example. Move the code around and add it to the BuildStarted events explictly. This does mean runs without builds no longer get variables added to the eventlog however we can look into a more targetted version of data if/as/where neded. (Bitbake rev: 4135a617ae16d509362b5bf56378139cdc0876d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ui/ncurses: Add missing function call to avoid tracebackRichard Purdie2023-11-271-0/+3
| | | | | | | | | The ncurses UI wasn't working due to a missing function call. Add it to avoid a traceback when starting builds. (Bitbake rev: db8f36b69a68de2179e5685cf24a42ec10d68257) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/aws: forward env-vars used in gitlab-ci K8sFelix Moessbauer2023-11-231-0/+2
| | | | | | | | | | | | | | | | | | | This patch adds the following variables to the allow-list, which are used in the "IAM roles for AWS when using the GitLab chart": - AWS_ROLE_ARN - AWS_WEB_IDENTITY_TOKEN_FILE These variables are set in the CI job environment and are needed to access the sstate cache artifacts in a connected S3 bucket. [1] https://docs.gitlab.com/charts/advanced/external-object-storage/aws-iam-roles.html Reported-by: Zhi Bin Dong <zhibin.dong@siemens.com> (Bitbake rev: c534526ea73805ee7cc16f3168b05ece10e0c03c) Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: git: add missing destsuffix and subpath parameters in ↵Julien Stephan2023-11-231-0/+8
| | | | | | | | | | | | docstrings Docstring for git fetcher is missing destsuffix and subpath parameters, so add them (Bitbake rev: 7cea6c2efefbb0f789f654c04a61d9ba770c148b) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch2: git: Update Git-LFS download and testsDesone Burns2023-11-232-28/+47
| | | | | | | | | | | | | | | | | | | | When downloading a Git repository containing an LFS, the Git hooks are not always initialized correctly to perform the download. This change updates the Git downloader to run the "git lfs install" command in order to "smudge" the LFS files. The tests have also been updated to do checks for situations in which git lfs is not installed, as the application was required to be installed for any of the tests to run previously. The Git LFS functionality was working to some degree previously, but this change also updates the fetcher to allow LFS downloads for nobranch SRC_URIs. (Bitbake rev: 05f8529fb439db93d85a892704b6f2f0ac0c9217) Signed-off-by: Desone Burns II <dburns@seegrid.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache: Simplify virtualfn2realfn()Peter Kjellerstedt2023-11-231-4/+2
| | | | | | | | | Also correct the description of variant2virtual(). (Bitbake rev: d766e9bd22ec6392fbf1694eea5032b9d09f1949) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Be consistent with what type bb_data representsPeter Kjellerstedt2023-11-231-6/+5
| | | | | | | | | | | | | | | | In _parse_recipe(), bb_data was initially a datastore object, but then after the call to bb.parse.handle() it was all of a sudden turned into a dict of variant:datastore pairs. At the same time, parseRecipeVariants() used bb_data for datastore objects, while parseRecipe() used it for the dicts of variant:datastore pairs. Change these functions to consistently use "bb_data" for datastore objects and use "datastores" for the dicts of variant:datastore pairs. (Bitbake rev: ed0f94c037476efd465db87726b2480406076beb) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: Make parseRecipeFile() handle virtual recipes correctlyPeter Kjellerstedt2023-11-231-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running `devtool search gcc` would result in errors like this for virtual recipes: ERROR: When reparsing .../meta/recipes-devtools/gcc/libgcc-initial_13.2.bb:do_populate_sysroot, the basehash value changed from b1cd809ed98cef9db0fb1b17d34c4083e739c336f9d5619b89715b0294d81af5 to 44c2f92781dc4a20e98b7bb4724e204e64b101905fa75e71241a574b725997dc. The metadata is not deterministic and this needs to be fixed. ERROR: The following commands may help: ERROR: $ bitbake libgcc-initial -cdo_populate_sysroot -Snone ERROR: Then: ERROR: $ bitbake libgcc-initial -cdo_populate_sysroot -Sprintdiff The reason was the newly introduced :layer-<layername> override, which is used, e.g., in meta/classes-global/insane.bbclass to add the patch-status QA test only for the meta layer: ERROR_QA:append:layer-core = " patch-status" When tinfoil parsed the recipes using the parseRecipeFile() function, it failed to properly identify the layername for virtual recipes, which resulted in the error above. The correct thing to do is to make parseRecipeFile() call bb.cache.virtualfn2realfn() to convert the virtual filename into a real filename and virtual class. (Bitbake rev: da2aed134412f5de04d7b540f92d735983ad0108) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Avoid sideeffects for autorev from getAllKeysWithFlagsRichard Purdie2023-11-231-0/+7
| | | | | | | | | | | | | If we expand the variable AUTOREV in OE-Core, it triggers side effects in the fetcher. The situation isn't ideal and needs improvement but this breaks and is blocking enabling BB_DEFAULT_EVENTLOG. Hack around the issue for now so we unblock things until we can work out a better plan for how to improve AUTOREV support. (Bitbake rev: cb9b6530f3d12c56a8b48847af2e7461924205d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add support for BB_DEFAULT_EVENTLOGRichard Purdie2023-11-231-8/+17
| | | | | | | | | | Currently it is only possible to specify an eventlog on the bitbake commandline. Add a variable that can be used in bitbake.conf so that we can log data by default more easily. (Bitbake rev: ee174b231897a53cdde0f68769518342e53210cf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: utils: remove spaces on empty linesJulien Stephan2023-11-201-8/+8
| | | | | | | | (Bitbake rev: 979ff8b4228e4c6f90985e9a3faa5fc9f3922218) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Refactor StaleSetSceneTasks event out of ↵Richard Purdie2023-11-161-18/+18
| | | | | | | | | | | | | | build_scenequeue_data The event at the end of the function doesn't really belong there, move it out so the function becomes cleaner. It also avoids having to pass a number of parameters around, another sign this is the correct thing to do. This continues refactoring started a while ago to improve and modularise the code. (Bitbake rev: f712b927a84e8b8deaa8c907e8f9bcdec681f2aa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Move 'cantskip' into sqdataRichard Purdie2023-11-161-7/+7
| | | | | | | | | Conceptually, this belongs in sqdata so this is the next step in refactoring this code, continuing what was started a while ago. (Bitbake rev: 2cf267cb657cd1864db43955cc010abba7a64093) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add option to set log level when running as a processJoshua Watt2023-11-142-2/+5
| | | | | | | | | | | | | | | | When running an asyncrpc server as a subprocess, it is often desired to run it with a lower logging level since the normal logging of clients connecting and disconnecting is not desired. As such, add an option to set the logging level of the server when running as a subprocess and set the level to WARNING when starting a local hashserver or PRserver (Bitbake rev: 61dac7b99ad6d2a858f85d8ed1b5524d558be6c8) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: client: Add disconnect APIJoshua Watt2023-11-091-1/+7
| | | | | | | | | | Adds an API to explicitly disconnect a client. This can be useful for testing the auto-reconnect behavior of clients (Bitbake rev: cb2fec7eaca20608faf4b8ff11ec8590ac7f2229) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: client: Prevent double closing of loopJoshua Watt2023-11-091-4/+6
| | | | | | | | | | Invalidate the loop in the client close() call so that it is not closed twice (which is an error in the asyncio code) (Bitbake rev: ef22f8ee82c242383625f078baafb629e45dad7e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add InvokeErrorJoshua Watt2023-11-094-4/+22
| | | | | | | | | | | | | Adds support for Invocation Errors (that is, errors raised by the actual RPC call instead of at the protocol level) to propagate across the connection. If a server RPC call raises an InvokeError, it will be sent across the connection and then raised on the client side also. The connection is still terminated on this error. (Bitbake rev: 50ee68175e7cf20a32bfbb176db2c47d7859da04) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add SQLalchemy backendJoshua Watt2023-11-091-2/+9
| | | | | | | | | | | Adds an SQLAlchemy backend to the server. While this database backend is slower than the more direct sqlite backend, it easily supports just about any SQL server, which is useful for large scale deployments. (Bitbake rev: e0b73466dd7478c77c82f46879246c1b68b228c0) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Prefix log messages with client infoJoshua Watt2023-11-091-3/+18
| | | | | | | | | | Adds a logging adaptor to the asyncrpc clients that prefixes log messages with the client remote address to aid in debugging (Bitbake rev: f4d64ce73c2449c008ff5d9b32376a2893ef7195) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add context manager APIJoshua Watt2023-11-091-0/+13
| | | | | | | | | | | | | | | | Adds context manager API for the asyncrcp client class which allow writing code that will automatically close the connection like so: with hashserv.create_client(address) as client: ... Rework the bitbake-hashclient tool and PR server to use this new API to fix warnings about unclosed event loops when exiting (Bitbake rev: d01d684a0f6398270fe35ed59b7d28f3fd9b7e41) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add websocket connection implementationJoshua Watt2023-11-093-2/+106
| | | | | | | | | | | | | | | Adds support to the hash equivalence client and server to communicate over websockets. Since websockets are message orientated instead of stream orientated, and new connection class is needed to handle them. Note that websocket support does require the 3rd party websockets python module be installed on the host, but it should not be required unless websockets are actually being used. (Bitbake rev: 56dd2fdbfb6350a9eef43a12aa529c8637887a7e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Abstract socketsJoshua Watt2023-11-095-228/+298
| | | | | | | | | | | | | Rewrites the asyncrpc client and server code to make it possible to have other transport backends that are not stream based (e.g. websockets which are message based). The connection handling classes are now shared between both the client and server to make it easier to implement new transport mechanisms (Bitbake rev: 2aaeae53696e4c2f13a169830c3b7089cbad6eca) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix errors when using -S printdiffRichard Purdie2023-11-091-2/+2
| | | | | | | | | | | | | | | | | When running printdiff, messages like: "Can't find a task we're supposed to have written out?" might be shown. This happens when hashequivalence is enabled and is due to the use of "hash" when "unihash" should be used (which defaults back to hash). Changing this fixed various errors oe-selftest highlighted. Also print the task ID when erroring about a missing task as it aids debugging. (Bitbake rev: e7783bbe0559abb91f5038cf83575c0f25138af2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix runall option for setscene tasksRichard Purdie2023-11-091-0/+1
| | | | | | | | | | | If --runall is used with setscene tasks, it will still skip them if it can. By marking the tasks as targets, this avoids that skipping and means --runall deploy_source_date_epoch works as expected for example. (Bitbake rev: 8c6cd1e8dd224bc1cc92736cbb7e113fd1fb5aae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: add header with length of messageEtienne Cordonnier2023-11-061-12/+22
| | | | | | | | | | | | | | | | | | | | | | | | | The IPC mechanism between runqueue.py and bitbake-worker is currently not scalable: The data is sent with the format <tag>pickled-data</tag>, and bitbake-worker has no information about the size of the message. Therefore, the bitbake-worker is calling select() and read() in a loop, and then calling "self.queue.find(b"</" + item + b">")" for each chunk received. This does not scale, because queue.find has a linear complexity relative to the size of the queue, and workerdata messages get very big e.g. for builds which reference a lot of files in SRC_URI. The number of chunks varies, but on my test system a lot of chunks of 65536 bytes are sent, and each iteration takes 0.1 seconds, making the transfer of the "workerdata" data very slow (on my test setup 35 seconds before this fix, and 1.5 seconds after this fix). This commit adds a 4 bytes header after <tag>, so that bitbake-worker knows how many bytes need to be received, and does not need to constantly search the whole queue for </tag>. (Bitbake rev: 595176d6be95a9c4718d3a40499d1eb576b535f5) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: fix lfs fetch with destsuffix paramRagesh Nair2023-10-231-2/+2
| | | | | | | | | | | | | | | Pre-fetching of LFS blobs now also works if a destsuffix parameter is supplied. This also fixes issues with Yocto Kirkstone (and newer) builds, where fetching the LFS blobs is retried unsuccessfully during do_unpack, as network access is blocked for any task other than do_fetch. (Bitbake rev: e411dc07d032be6811d0393c50a06fc28e669b24) Signed-off-by: Ragesh Nair <ragesh.nair@duagon.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: add missing 'import os'Chris Laplante2023-10-231-0/+1
| | | | | | | | (Bitbake rev: 302969885d37a76edec3aa79181e98f8d7e28021) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: set has 'add', not 'append' methodChris Laplante2023-10-231-1/+1
| | | | | | | | | | Discovered via pylint (Bitbake rev: 1a353cda696b7f59386ad2d78a57005b90a37da4) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: clarify that 'closest' signature means 'most recent' ↵Alexander Kanavin2023-10-231-1/+1
| | | | | | | | | | | | | | | | | (and not closest in its content) This is printed by 'bitbake -S printdiff' and more accurately reflects what the code does. Most of the time the most recent item should be what the user wants to see when debugging sstate misses, but there could also be printdiff-all (print differences with all matching sstate objects), or printdiff-N (N most recent, and not just the latest). (Bitbake rev: 315f48f965a54da5cec92908d91aa61c2d450add) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Fix find_bbfiles string endswith callBELHADJ SALEM Talel2023-10-231-1/+1
| | | | | | | | (Bitbake rev: 5f742591b251b6a5302ab07fe6b809c2863c3c70) Signed-off-by: Talel BELHAJSALEM <bhstalel@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: Add tests to cover multiple branch/name parametersPavel Zhukov2023-10-231-6/+69
| | | | | | | | | | | Create repository with few branches and test if fetcher can work with such repository as PREMIRROR (Bitbake rev: a1737610e5d5b61e126ec3632d7f27b337a87818) Signed-off-by: Pavel Zhukov <pavel@zhukoff.net> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: replace deprecated ast.Str and 's'Chris Laplante2023-10-171-2/+2
| | | | | | | | | These have been deprecated since 3.8 (Bitbake rev: 5419a8473d6d4cd1d01537de68ad8d72cf5be0b2) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add API for upstream source tracingAlberto Pianon2023-10-157-0/+91
| | | | | | | | | | | | | | | | | This patch adds an API to bb.fetch2 to enable users to plug in an unpack tracer that can trace each source file back to its corresponding upstream source url, even when multiple upstream sources are combined together in the same unpack directory. This may be required for software composition analysis, license compliance, and detailed SBoM generation. This patch provides only the needed hooks in bb.fetch2 code and a dummy abstract class defining the API; users may load their own unpack tracer class by setting the BB_UNPACK_TRACER_CLASS config parameter. (Bitbake rev: 05051152cc42acc52bcf9af9a696f632fac4307f) Signed-off-by: Alberto Pianon <alberto@pianon.eu> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: selftest/fetch: Ensure top level directory timestamp doesn't break testRichard Purdie2023-10-061-5/+7
| | | | | | | | | | The infrastructure changes highlighed an issue where the top level directory mtime stamp isn't included in the tarballs. This isn't an issue the test is meant to check for so ignore it. (Bitbake rev: 5a1e5b8dc55d6bb4d93bac3492f8c43ff957e712) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: add a new function to retrieve task signaturesJulien Stephan2023-09-283-0/+45
| | | | | | | | | | | | adding a new command in cooker to compute and get task signatures this commit also add the associated command and event needed to get the signatures using tinfoil (Bitbake rev: 05c15162de90c41dad67e37a95ec9fdb440a7864) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Adds vscode devcontainer supportMatthias Schnelte2023-09-261-0/+1
| | | | | | | | | | | | | | | | Visual studio code is injecting a git credential helper into every dev container. This helper is forwarding the git credentials from the host into the container. In order for this helper to work the REMOTE_CONTAINERS_IPC needs to be available in the environment for all git operations that require authentication. (Bitbake rev: 612bdf09416ff96d08b22cf92b077516d23d0963) Signed-off-by: Matthias Schnelte <develop@schnelte.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/lib: spawn server/worker using the current Python interpreterRoss Burton2023-09-262-3/+5
| | | | | | | | | | | | | | | | | The user may have invoked ./bin/bitbake using a different Python interpreter than whatever python3 is on $PATH (for example, explicitly using a different version). However, as the server and workers are spawned directly they'll use the hashbang and thus a different Python. We also ensure that argv[0] is set to sys.executable instead of 'bitbake-server' or 'bitbake-worker', so that sys.executable is set to the right value inside the child. Without this the server won't be able to start any workers. (Bitbake rev: b44d5d2a53d3082c8ce94e09c0cf833e33e25aec) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: Do not fail when logging is disabled and full config is usedPeter Kjellerstedt2023-09-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | If Tinfoil is initialized with setup_logging = False and Tinfoil.prepare() is called with config_only = False, then it fails because self.localhandlers is only initialized when setup_logging = True. This is seen with, e.g., `bitbake-getvar -q -r busybox MACHINE`: Traceback (most recent call last): File ".../bitbake/bin/bitbake-getvar", line 41, in <module> tinfoil.prepare(quiet=2) File ".../bitbake/lib/bb/tinfoil.py", line 390, in prepare for handler in self.localhandlers: AttributeError: 'Tinfoil' object has no attribute 'localhandlers'. Did you mean: 'oldhandlers'? (Bitbake rev: 616101ddb630e2c9975022068b52a87c4cf647f6) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: Update debug variable referenceRichard Purdie2023-09-221-1/+1
| | | | | | | | | The code has changed and the debug message didn't work. Fix it. The output is still incredibly useful. (Bitbake rev: f1fa4bb3066e2bbaff0b69088ba5c6c6c597b93d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker/runqueue: Avoid unnecessary bytes object copiesEtienne Cordonnier2023-09-221-2/+2
| | | | | | | | | | | | | | | | | | declaring queue=b"" creates an object of types bytes(). bytes() is an immutable object, and therefore doing "self.queue = self.queue + r" creates a new object containing "self.queue" concatenated with "r". On my test setup, we are passing 180MB of data of "workerdata" to the bitbake-worker, so those copies significantly slow down the initialization of the bitbake-worker. Rather use bytearray() which a mutable type, and use extend() to avoid copies. In my test setup, byterray.extend() is 10.000 times faster than copying the queue, for a queue size of 180MB. (Bitbake rev: 2302b5316091dff189e6c3f546341b2274ed9d0a) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Add missing dependency handling of remove operatorInsu Park2023-09-202-0/+27
| | | | | | | | | | | | | | | | | A recipe variable handles its dependencies even on the "contains" variables within the "inline Python expressions" like bb.utils.filter(). And it also handles those in the append operator correctly, but the problem is that it does not so in the remove operator. Fix it by adding the missing dependencies every time the remove operator has been handled. Also add a test case to check if the override operators handle dependencies correctly. (Bitbake rev: b90520eedb1dbc7f6a3928d089fe74fafb864eb5) Signed-off-by: Insu Park <insu0.park@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-05 01:44:40 +0000