summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: server: Fix early parsing errors preventing zombie bitbakeJoshua Watt2021-07-201-1/+1
| | | | | | | | | | | | | | | | | If the client process never sends cooker data, the server timeout will be 0.0, not None. This will prevent the server from exiting, as it is waiting for a new client. In particular, the client will disconnect with a bad "INHERIT" line, such as: INHERIT += "this-class-does-not-exist" Instead of checking explicitly for None, check for a false value, which means either 0.0 or None. (Bitbake rev: 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add piping compression libraryJoshua Watt2021-07-205-0/+338
| | | | | | | | | | | Adds a library that implements file-like objects (similar to gzip.GzipFile) that can stream to arbitrary compression programs. This is utilized to implement a LZ4 and zstd compression API. (Bitbake rev: 61c3acd058ea018696bd284b3922d0b458838d05) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart/parse: Allow ':' characters in variable/function namesRichard Purdie2021-07-204-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is becomming increasingly clear we need to find a way to show what is/is not an override in our syntax. We need to do this in a way which is clear to users, readable and in a way we can transition to. The most effective way I've found to this is to use the ":" charater to directly replace "_" where an override is being specified. This includes "append", "prepend" and "remove" which are effectively special override directives. This patch simply adds the character to the parser so bitbake accepts the value but maps it back to "_" internally so there is no behaviour change. This change is simple enough it could potentially be backported to older version of bitbake meaning layers using the new syntax/markup could work with older releases. Even if other no other changes are accepted at this time and we don't backport, it does set us on a path where at some point in future we could require a more explict syntax. I've tested this patch by converting oe-core/meta-yocto to the new syntax for overrides (9000+ changes) and then seeing that builds continue to work with this patch. (Bitbake rev: 0dbbb4547cb2570d2ce607e9a53459df3c0ac284) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/s3: allow to use credentials from environment variablesAdam Romanek2021-07-111-1/+4
| | | | | | | | | | | | | | | | | Previously access to AWS S3 was expected to be preconfigured and credentials to be stored in ~/.aws/credentials. With this change one can use Bitbake s3 fetcher without AWS credentials stored permanently as above, just with them exported as the following environment variables: - AWS_ACCESS_KEY_ID, - AWS_SECRET_ACCESS_KEY. - AWS_DEFAULT_REGION. (Bitbake rev: 01825699044c42e87e485e6c64cc1dd9b6f87f48) Signed-off-by: Adam Romanek <romanek.adam@gmail.com> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: providers: replace newly added logger.warn() with logger.warning()Denys Dmytriyenko2021-07-101-3/+3
| | | | | | | | | | | | | | Commits https://git.openembedded.org/bitbake/commit/?id=78cd63285713fde59506eb2e71a7b7ee59a594ff and https://git.openembedded.org/bitbake/commit/?id=5cbf6d95fc1009e78e7d0745a49e0bf418b37abb added few calls to logger.warn(), which is deprecated and instead should use logger.warning(): https://lists.openembedded.org/g/bitbake-devel/topic/82742194#12377 (Bitbake rev: a28ba2d31cd3aa557d4977e9376c5d01cd863e9a) Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: run gc in foreground to avoid race with tarAdam Romanek2021-07-101-1/+1
| | | | | | | | | | | | | | | | | | | | It looks like git gc can interrupts the package creation when BB_GENERATE_MIRRORS_TARBALL is in use. Log excerpts: tar -czf TOPDIR/../downloads/git2_bitbucket.name-hidden.git.tar.gz . failed with exit code 1, output: tar: ./objects/pack/pack-89a1d76f6c08f53172ef1d02ff851d90564362c4.pack: file changed as we read it tar: ./objects/pack/pack-b4a48ada355d333630fdf6b4f67205b7c264dc2c.idx: File removed before we read it Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. (Bitbake rev: a8d8cb847063862d1a7998963dd8b767ff73d877) Signed-off-by: Adam Romanek <romanek.adam@gmail.com> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual: fix erroneous statement in glossary introMichael Opdenacker2021-06-061-4/+0
| | | | | | | | | | | This is the BitBake glossary. Remove an erroneous statement probably coming from the introduction to the YP/OE glossary. (Bitbake rev: 398a1686176c695d103591089a36e25173f9fd6e) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: docs: Add BB_HASHSERVE definition to glossaryMichael Opdenacker2021-06-061-0/+10
| | | | | | | (Bitbake rev: 0ab3255f8e9d9c8c8aa4788504ab06a50d1bb1f2) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: warn on setting noexec/nostamp/fakeroot flag to any value ↵Chris Laplante2021-06-061-0/+5
| | | | | | | | | | | | | | | | | | | besides '1' Currently setting those flags to even the empty string "" causes it to be set, which is contrary to the documentation. In a future version of BitBake, we'd like to change the behavior so that setting the flag to "" does not set it. This will allow conditionally setting noexec, using variable expansion or inline Python. I found no places in poky or meta-openembedded where this warning would trigger. [YOCTO #13808] (Bitbake rev: 1e7655c4f765ba7b4791c4cca048a69bf8d9c93d) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: user-manual: ref-manual: remove mentions to ↵Quentin Schulz2021-06-063-22/+0
| | | | | | | | | | | | | | | | | | | | | | BB_SETSCENE_VERIFY_FUNCTION2 This variable was removed in July 2019 with commit 5deaa5df730a "runqueue: Drop unused BB_SETSCENE_VERIFY_FUNCTION2". There's no replacement, so let's update the documentation to not mention this variable anymore. This was found by running: git grep -hoP '^ :term:`\K\w+(?=`)' doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst | \ xargs -I{} sh -c \ "grep -Rq --include='*.py' --include='*.conf' --include='*.bbclass' --include='*.bb' --include='*.bbappend' --include='*.inc' \ -w -E {}'_*(_[a-z]+[0-9a-z]*[a-z]+)*' || \ echo {}" (Bitbake rev: c2c0f5126c7c784bfd7a08f127e161a58c6b5d12) Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: user-manual: remove mentions to BBVERSIONSQuentin Schulz2021-06-062-42/+9
| | | | | | | | | | | | | | | | | | | | BBVERSIONS support was dropped in November 2016 in commit 0bb188f01e39 "ast: remove BBVERSIONS support". Let's update the documentation to reflect this. This was found by running: git grep -hoP '^ :term:`\K\w+(?=`)' doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst | \ xargs -I{} sh -c \ "grep -Rq --include='*.py' --include='*.conf' --include='*.bbclass' --include='*.bb' --include='*.bbappend' --include='*.inc' \ -w -E {}'_*(_[a-z]+[0-9a-z]*[a-z]+)*' || \ echo {}" (Bitbake rev: 376e20c1aaff197020cdb68d309b3b22f19dafe8) Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Reduce verbosityPaul Barker2021-06-061-4/+4
| | | | | | | (Bitbake rev: cc803609167b4c399ab768d9e131d618c086a4f2) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add ping methodPaul Barker2021-06-062-1/+11
| | | | | | | | | | This method is needed to support startup of the prservice. As it is so generic we can add it to the common asyncrpc module. (Bitbake rev: 25ccd697ea76f66b813be2296866b2d3405b079c) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add check for empty SRC_URI hash stringScott Weaver2021-06-061-1/+4
| | | | | | | | | | | | | | | | | | | | | | No error was being reported when the hash string was set to empty. For example: SRC_URI[md5sum] = "" On a related note (not a bug): Because whitespace in the string will result in a checksum mismatch, the error message was updated to make it a little clearer why the error was thrown. For example: SRC_URI[md5sum] = " " or SRC_URI[md5sum] = " 209f8326f5137d8817a6276d9577a2f1" Now creates a message like this: File: '/home/scott/yocto-cache/downloads/rsync-3.2.3.tar.gz' has md5 checksum '209f8326f5137d8817a6276d9577a2f1' when ' 209f8326f5137d8817a6276d9577a2f1' was expected [YOCTO #14232] (Bitbake rev: a13510d0028e234ea2f4744b0d0c38558395c70f) Signed-off-by: Scott Weaver <weaverjs@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Handle error in heartbeat funciton in OOM caseRichard Purdie2021-05-181-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've seen cases where an OOM error causes bitbake server to hang: 9171 02:21:09.127810 Command Completed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/bin/bitbake-server", line 51, in <module> bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 550, in execServer server.run() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 108, in run ret = self.main() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 242, in main ready = self.idle_commands(.1, fds) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 370, in idle_commands bb.event.fire(heartbeat, self.cooker.data) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 216, in fire fire_class_handlers(event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 123, in fire_class_handlers execute_handler(name, handler, event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 93, in execute_handler ret = handler(event) File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 182, in defaultrun_buildstats write_host_data(os.path.join(bsdir, "host_stats"), e, d, "interval") File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 160, in write_host_data output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child restore_signals, start_new_session, preexec_fn) OSError: [Errno 12] Cannot allocate memory We need to wrap the calls in the same high level wrapper as idle function calls and trigger an exit upon an unhandled exception. (Bitbake rev: 74042b5b89d5a170013fc1a327ce3a6530fbf7d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserver/client.py: drop unused importsArmin Kuster2021-05-141-3/+0
| | | | | | | (Bitbake rev: 9dc77266085c605b108641a9d76ac4dbdc064c34) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv/server.py: drop unused importsArmin Kuster2021-05-141-7/+0
| | | | | | | | | remove unused vars. (Bitbake rev: 3287d28a506f67abd192799e61ef28e74ce7002d) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: when checking latest versions, consider all numerical ↵Alexander Kanavin2021-05-141-1/+1
| | | | | | | | | | | | | | directories Previously the regex was maching x.y, but wasn't matching x, which is a problem e.g. here: https://download.gnome.org/sources/epiphany/ (the new gnome version scheme adds 40-series at the end). (Bitbake rev: c03101576f447263ea38e8464210d3a3a2c27226) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Switch to post release version number 1.51.0Richard Purdie2021-05-062-2/+2
| | | | | | (Bitbake rev: 97a64d12f70eb02f1d35b4ffefb291b80ca8c425) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bin/bitbake-getvar: Add a new command to query a variable value ↵Richard Purdie2021-05-061-0/+48
| | | | | | | | | | | | | | | | | | | (with history) We've talked about having this for long enough. Add a command which queries a single variable value with history. This saves "bitbake -e | grep" and avoids the various pitfalls that has. It also provides a neat example of using tinfoil to make such a query. Parameters to limit the output to just the value, to limit to a variable flag and to not expand the output are provided. [YOCTO #10748] (Bitbake rev: 4c1881b620e885f55d7772f8626b8a76c2828333) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/svn: Fix parsing revision of SVN repos with redirectsHarald Brinkmann2021-05-061-1/+1
| | | | | | | | | | svn was printing a message when encountering HTTP redirects. This confused the revision parser. (Bitbake rev: a944a335f8f4c4fe5df55f3d7d8e757bd2835146) Signed-off-by: Harald Brinkmann <Harald.Brinkmann@detectomat.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Handle requests in main threadPaul Barker2021-05-061-123/+36
| | | | | | | | | | | The prserver process is cleanly separated from the main bitbake process so requests can be handled in the main thread. This removes the need for a request queue and a separate request handling thread. (Bitbake rev: 6b09415bed6b5e7c12aaf39b677d9ef72844e233) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Extract daemonization from PRServer classPaul Barker2021-05-061-103/+72
| | | | | | | | | | | The code to start the prservice process as a daemon is extracted out of the PRServer class and simplified. This makes the PRServer class easier to modernise as it no longer needs to worry about process management. (Bitbake rev: 39c7c158c52157b18f5ccbbd673e3298e6402f52) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Use multiprocessing to auto start prserverPaul Barker2021-05-061-4/+8
| | | | | | | | | | | | | | | | | We can use the modern multiprocessing support in Python instead of manually using fork to start the prserver process. To do this we need to set up the signal handlers for the prserver process in the work_forever function (which is now used as the main function for this process). The old code to start the prserver process using fork is not removed in this commit as it is tightly intertwined with the daemonization code which will be refactored in a following commit. (Bitbake rev: b3da56240c0f92efab1c0b293738c35c0f1ee6ab) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add bb.utils.rename() helper function and use for renamingDevendra Tewari2021-05-063-6/+18
| | | | | | | | | | | | | | | | | | | | os.rename can fail for example an incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than trying to fix every call site, add a wrapper in bb.utils for renames. We can then handle cross device failures and fall back to shutil.move. The reason os.rename is still used is because shutil.move is too slow for speed sensitive sections of code. [YOCTO #14301] (Bitbake rev: c5c4e49574ab2a65e06298a0a77bb98b041cf56b) Signed-off-by: Devendra Tewari <devendra.tewari@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Handle deferred task rehashing in multiconfig buildsRichard Purdie2021-05-011-4/+10
| | | | | | | | | | | | If the hash of a task changes and that hash is a deferred task (e.g. a multiconfig build), we need to ensure that the hash change propagates through to all the tasks else the build will run multiple copies of the task, sometimes with oddly differing results as the outhashes of native tasks built in differing locations can confuse things. (Bitbake rev: 2db571324f755edc4981deecbcfdf0aaa5a97627) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix multiconfig deferred task sstate validity caching issueRichard Purdie2021-05-011-4/+4
| | | | | | | | | | | | | | | | | | | | | We were testing the validity of deferred tasks setscene status "up front" which is very unlikely to succeed and leads to cache invalidation issues. With the change to rebuild the deferred task list, this status becomes out of sync. The result was tasks being executed when they should not have been leading to extra work for the build unnecessarily. Instead, don't process validity status for deferred tasks and assume their data will become available. If it doesn't, this will now result in a build error as the setscene task will fail and the main task will run instead. In theory we could try and track the state changes in the deferred list and re-test validity then but I'm not sure it is worth the effort when the other code path and errors in setscene tasks will give a pretty good idea of what is happening anyway. (Bitbake rev: edcafac13b3b241b6687419e59018d21811507a1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Re-enable command line logging levelsJoshua Watt2021-05-011-0/+8
| | | | | | | | | | | | The "-l" command line options to enable specific logging domains wasn't working with the switch to structured logging because they were only being used to set the legacy logging domains. Fix this by implementing the logic to parse the user options into the logging configuration. (Bitbake rev: 005fc7a8c588d0b0bca382469645cbf481ad8e30) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/s3: Add progress handler for S3 cp commandPrzemyslaw Gorszkowski2021-05-011-1/+40
| | | | | | | | Adds progress support for fetchers from S3. (Bitbake rev: 90d31b2d5a81e5f41fe95907c78fd2f5f36e39ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: LineFilterProgressHandler - Handle parsing line which ↵Przemyslaw Gorszkowski2021-05-011-2/+5
| | | | | | | | | | | ends with CR only S3 commands need to handle different CR only line endings, update the handler to cope with this. (Bitbake rev: 3f7b9c1b429a4c68240e80832a8ef93ee210e5ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Add connect functionPaul Barker2021-04-271-0/+3
| | | | | | | | | | | This function abstracts the setup of a PR service client connection so that openembedded-core doesn't need to be updated any time the details are changed. (Bitbake rev: d892287b31f81b075983ba500be265f75b53df64) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Drop unused dump_db methodPaul Barker2021-04-271-24/+0
| | | | | | | (Bitbake rev: ecb7bf34eac02ff58dbc27b3768ceaf4adb1c9cd) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Drop obsolete python version checkPaul Barker2021-04-271-4/+0
| | | | | | | | | Bitbake no longer supports Python 2 so this version check is obsolete. (Bitbake rev: 45eb6c6e124e507012df9c288f1fbde0e7899e5d) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Refactor to use asyncrpcPaul Barker2021-04-272-306/+41
| | | | | | | | | | The asyncrpc module can now be used to provide the json & asyncio based RPC system used by hashserv. (Bitbake rev: 5afb9586b0a4a23a05efb0e8ff4a97262631ae4a) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Common implementation of RPC using json & asyncioPaul Barker2021-04-273-0/+394
| | | | | | | | | | | | | | The hashserv module implements a flexible RPC mechanism based on sending json formatted messages over unix or tcp sockets and uses Python's asyncio features to build an efficient message loop on both the client and server side. Much of this implementation is not specific to the hash equivalency service and can be extracted into a new module for easy re-use elsewhere in bitbake. (Bitbake rev: 4105ffd967fa86154ad67366aaf0f898abf78d14) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Use generic ConnectionErrorPaul Barker2021-04-273-17/+12
| | | | | | | | | | The Python built-in ConnectionError type can be used instead of a custom HashConnectionError type. This will make code refactoring simpler. (Bitbake rev: 8a796c3d6d99cfa8ef7aff0ae55bb0f23bbbeae1) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual: fix typo left over from Sphinx migrationNicolas Dechesne2021-04-261-2/+2
| | | | | | | | | | Fixes d99760cc687c (sphinx: last manual round of fixes/improvements) Reported-by: Michal Piechowski <m.z.piechowski@gmail.com> (Bitbake rev: 00ce48919de720639eda2b6f7065a82b641e5167) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual: code insertion simplification over two linesMichael Opdenacker2021-04-261-6/+3
| | | | | | | | | | | | | | | This simplifies paragraphs ending with a colon and followed by code insertion. Automatically substituted through the command: sed -i -z "s/:\n\s*::/::/g" file.rst This generates identical HTML output. (Bitbake rev: 51c80fc3497eecc8e50194fe1ff8069b59f03eda) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: bitbake-user-manual: simplify colon usageMichael Opdenacker2021-04-266-193/+193
| | | | | | | | | | - This replaces instances of ": ::" by "::", which generates identical HTML output (Bitbake rev: fd8ce4dcaff3aae395f9945fb0a3be54905e1727) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: layerindex: Add --fetchdir parameter to layerindex-fetchDiego Sueiro2021-04-231-4/+11
| | | | | | | | | | | | Introduce --fetchdir parameter to layerindex-fetch enabling users to choose the directory to fetch the layers different from BBLAYERS_FETCH_DIR. [YOCTO #14347] (Bitbake rev: 784a904faffac723ddf58ba765b9dd11ac068de5) Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: layerindex: Fix bitbake-layers layerindex-show-depends commandDiego Sueiro2021-04-231-0/+1
| | | | | | | | | | | | | | | | | | | | | | | Running 'bitbake-layers layerindex-show-depends meta-filesystems' fails with: ``` Traceback (most recent call last): File "<...>/poky/bitbake/bin/bitbake-layers", line 93, in <module> ret = main() File "<...>/poky/bitbake/bin/bitbake-layers", line 86, in main return args.func(args) File "<...>/poky/bitbake/lib/bblayers/layerindex.py", line 209, in do_layerindex_show_depends self.do_layerindex_fetch(args) File "<...>/poky/bitbake/lib/bblayers/layerindex.py", line 182, in do_layerindex_fetch args.shallow) AttributeError: 'Namespace' object has no attribute 'shallow' ``` Initialize the shallow attribute to fix it. (Bitbake rev: 71f095c147fe6aa7b5e6272002e0498cf9494256) Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/user-manual-fetching: Remove basepath unpack parameter docsRichard Purdie2021-04-201-3/+0
| | | | | | | | | | | | The code for this was removed in 2016 in commit e659a3b0c2771679057ee3e13cd42e6c62383ff2. Nobody seems to have missed it so remove the documentation so we match the code. [YOCTO #13449] (Bitbake rev: 76bf42ea41a28b19d0377c2e548b0a59119fdf67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: Fix crash when using git LFS and submodulesNiels Avonds2021-04-201-1/+1
| | | | | | | | | | | | | | | | Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Niels Avonds <niels@codebits.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-server: Remove now unneeded codeRichard Purdie2021-04-201-2/+0
| | | | | | | | | With the previous patch this code is now pointless as we'd have hit a TypeError before now. (Bitbake rev: 6301a99055c79d89b715f72182cd0ef1b781b89a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-server: ensure server timeout is a floatRoss Burton2021-04-202-2/+2
| | | | | | | | | | | | | | | | | | bitbake-server is spawned by process.py and passes the arguments it is given to ProcessServer. There's some type confusion here: bitbake-server is called with a string representation of the timeout, which may be None. If the timeout is not set, pass 0 instead of None. Inside bitbake-server a ProcessServer is created which expects the timeout to be a float not a string, so always float() the value. [ YOCTO #14350 ] (Bitbake rev: c93ae1f861208f6d39fd15c84fbcd0e2b54331f5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil/data_smart: Allow variable history emit() to function remotelyRichard Purdie2021-04-182-0/+16
| | | | | | | | | | We can't access the emit() function of varhistory currently as the datastore parameter isn't handled correctly, nor is the output stream. Add a custom wrapper for this function which handles the two details correctly. (Bitbake rev: ba0fa084ccd2b1ade96425d158fd31e49e42f286) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add tests for local and remote "noshared" git fetchingHenning Schild2021-04-181-0/+38
| | | | | | | (Bitbake rev: e0267fe43bda208856af939b17e39beb9e5586c3) Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: deduplicate local git testing codeHenning Schild2021-04-181-28/+6
| | | | | | | | | Purely cosmetic change that probably improves the code. (Bitbake rev: 9c0733f0062f3cf19514c891cc06c9a8e0db429b) Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for disabling shared clones on unpackHenning Schild2021-04-181-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | By default the unpacker will create a "shared" clone when cloning from the DL_DIR to the WORKDIR. This patch introduces an option to control that behaviour. Imagine some recipe steps are executed in a namespace that is different from the one your downloader and unpacker ran in. (chroot) Because a "shared" clone has an absolute reference to its "alternate" you now have to make that "alternate" visible in that new namespace (chroot) at the exact place. With this patch you can unpack "noshared" and get a stand-alone copy. This copy will also work if the "alternate" is not visible or existant. The switch is a global bitbake switch and will affect all git urls. Build systems that need "noshared" most likely need it for everything they do with git. (Bitbake rev: 6ae6f1865d5e666ebc670f70b7401a7b41648102) Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: tests/fetch: remove write protected files tooMikko Rapeli2021-04-181-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For some reason several git-annex files in Debian 10 buster are read-only and removing them with "rm -rf" fails. Fixes test failures like: $ bitbake-selftest ... rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied EE..................................ssss.sssssssssssssss.sssss....................................................................................................... ====================================================================== ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex fetcher, ud = self.fetch_shallow(uri) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow bb.utils.remove(ud.clonedir, recurse=True) File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path)) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1. Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod the whole temporary directory which should cover any directory name differences between different git-annex versions. Fixes tests failing after chmod call: Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/ (Bitbake rev: 7729ef2983c72867e99fad82d671069ba5cb32b2) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-07 16:30:11 +0000