summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch/wget: Increase timeout to 100s from 30sRichard Purdie2024-11-211-2/+2
| | | | | | | | | | Testing shows the worst case CDN response time can be up to 100s. The wget fetcher is used for accessing sstate from the CDN so increase our timeouts there to match our worst case repsonse times. (Bitbake rev: 7aca591529e115bc277f93811d8c586630acc8c3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker/cooker: Increase default pipe sizeRichard Purdie2024-11-211-1/+1
| | | | | | | | | | | | | | | | The default pipe size is 64kb on builds, which can be inefficient for larger log files from workers. Increase the pipe size to 512kb since build systems have decent amounts of memory and this is a more efficient way of batching the data. Tweak the default read sizes to match the pipe size for efficiency. Since the contstant is only present in python 3.10 onwards, add some compatibility code. (Bitbake rev: 69c14e46600ba5ae9703f67704ab2548875ae6d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Drop support for BB_DANGLINGAPPENDS_WARNONLYRichard Purdie2024-11-211-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | DANGLINGAPPENDS_WARNONLY is a very poorly designed interface and is commonly abused. The challenge is that once it is set, by any layer, it applies everywhere. Some layers rely on this to get notification they need to update bbappend files and having the behaviour change from inclusion of an antisocial layer is not good. In addition, showing warnings as an accepted thing on the console devalues them and trains the user to ignore them. I want to steer us away from this mindset. We could extend the functionality and make it apply only to certain layers, or only to certain appends but then we've basically re-invented BBMASK. Given all the above, we should drop support for BB_DANGLINGAPPENDS_WARNONLY and direct anyone with issues to BBMASK instead. https://lists.openembedded.org/g/openembedded-architecture/message/2029 [YOCTO #14870] (Bitbake rev: fca9c9e3cb6f8e9f99bf51dc5e8a8d83f4c84c69) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: add new "finalizeData" APIYoann Congal2024-11-182-0/+14
| | | | | | | | | | | | Create a new tinfoil API "finalizeData" which does key expantion and runs the anonymous functions to allow bitbake-getvar to have the same output as "bitbake -e". (Bitbake rev: 63db64ca98564420a378f14c337074ae06da8c63) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: fix needconfig for revalidateCachesYoann Congal2024-11-071-1/+1
| | | | | | | | | | Fixes what looked like a type since parseConfiguration.needconfig is already set just after parseConfiguration() definition. (Bitbake rev: d371df029ece3b9e600a89d08337c437a8ddbf63) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Decrease idle/main loop frequencyRichard Purdie2024-11-071-2/+2
| | | | | | | | | | The idle and main loops have socket select calls to know when to execute. This means we can increase the normal timeout frequency since this is just a fall back and have some small efficiency gains. (Bitbake rev: 8d8e17af8619c976819170c9d5d9a686a666c317) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Don't send heartbeats when no idle functionsRichard Purdie2024-11-071-1/+1
| | | | | | | | | If there are no idle functions present, don't sent heartbeat events. These are only meant to happen while builds are active. (Bitbake rev: 9a2d5e63b07c3912838781776c61f0f1ac9640e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Merge a function to simplfy codeRichard Purdie2024-11-071-26/+16
| | | | | | | | | Keeping this code separate just makes the code harder to understand, merge them. (Bitbake rev: e5ac26a0e1779df1da3277bf48899c8f7642f1f8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Avoid dumpsigs idle loop blockingRichard Purdie2024-11-071-29/+44
| | | | | | | | | | | | | | | | | | | | | We're seeing some failures on hosts where slow "idle" loop iterations are causing bitbake server timeouts. These seem to happen particularly in the dump_signatures() function within runqueue. That isn't entirely surprising since it creates a pool of threads to execute work an at best can take around 10s to execture and return control backto the main loop. On a slow system, it is understandable this can take longer, particularly as these functions are creating large chunks of IO. Since the work is being done in threads, we can launch them, return to idle and check on the results periodically as they complete. This should hopefully address some of the remaining timeout issues we see on the autobuilder in oe-selftest sstate tests. (Bitbake rev: e66f1b643b4b77404ba31f2704cda5af9bf00a57) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/compress: use lz4 instead of lz4cJustin Bronder2024-11-052-4/+4
| | | | | | | | | | | | | | | | | | | lz4c has been considered deprecated by upstream since at least 2018 [1] and has been disabled by default recently [2]. openSUSE Tumbleweed is no longer packaging the deprecated version and others will likely follow. Luckily, we're not using any legacy-specific options and, going back as far as Ubuntu 16.04/Fedora 25, both lz4 and lz4cli are installed as part of the same package 1. https://github.com/lz4/lz4/pull/553 2. https://github.com/lz4/lz4/pull/1479 (Bitbake rev: 907472034b344e4eb73cfd43059a413469f52e1c) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Sort pn-buildlistRobert Yang2024-11-051-4/+5
| | | | | | | | | So that we can compare the different pn-buildlist easily. (Bitbake rev: 529043117a7c62feb45bc891658a412cc8dd7e3f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Use our own mirror of mobile-broadband-provider to ↵Richard Purdie2024-10-291-1/+1
| | | | | | | | | | | | decouple from gnome gitlab GNOME gitlab has occasional downtime which impacts bitbake-selftest and causes autobuilder failures. Switch to our own mirror for test purposes to avoid those issues. (Bitbake rev: 91e268b11ed683bd197026f9b36001f6d54ee05c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Use our own mirror of sysprof to decouple from gnome ↵Richard Purdie2024-10-281-1/+1
| | | | | | | | | | | | gitlab GNOME gitlab has occasional downtime which impacts bitbake-selftest and causes autobuilder failures. Switch to our own mirror for test purposes to avoid those issues. (Bitbake rev: 008808755ed6cfeb6c41273e69ce718f0833c26c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git: Clean broken symlinkRobert Yang2024-10-281-1/+1
| | | | | | | | | | | The file may be a symlink when mirror from local disk, and the symlink will be broken when the linked file is removed, use os.path.islink() to check and remove the broken symlink. (Bitbake rev: d0deb5fe6e7a84b07b49ef685b209729d2fdc780) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git: Clean shallow mirror tarballRobert Yang2024-10-281-0/+5
| | | | | | | | | | | | | | | Fixed: BB_GIT_SHALLOW = "1" BB_GENERATE_SHALLOW_TARBALLS = "1" $ bitbake utfcpp -cfetch && bitbake utfcpp -ccleanall The downloads/gitsmshallow_github.com.nemtrif.*.tar.gz won't be cleaned without this fix. (Bitbake rev: bab7a8970a0237a9d24217685a595e76a1336c07) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: Add clean functionRobert Yang2024-10-281-0/+13
| | | | | | | | | | | | Fixed: $ bitbake utfcpp -cfetch && bitbake utfcpp -ccleanall The downloads/git2/github.com.nemtrif.ftest won't be cleaned without this fix. (Bitbake rev: 79f25fc5c1b8d0e08540f4aa07875309f5325f47) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: Remove downloads/tmpdir when failedRobert Yang2024-10-281-3/+5
| | | | | | | | | The tmpdir such as downloads/tmplp3cnemv won't be removed without this fix. (Bitbake rev: 2ba8d3214759142afc11f0a88d80eb30a8bcde3a) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: Add call_process_submodules() to remove duplicated codeRobert Yang2024-10-281-28/+14
| | | | | | | | | There are 14 lines can be removed, and can make it easy to maintain. (Bitbake rev: 0ea2c1ac079d63349407a69172ff80cd9acc7252) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/layerdetails: Drop unwanted test ↵Richard Purdie2024-10-241-29/+9
| | | | | | | | | | | | skipping/exception handling We really don't want to be skipping a test like this, drop the exception handling code as in my testing we thankfully no longer seem to be seeing it. If we do again we need to fix the underlying issue. (Bitbake rev: b67547f890dc7f347ec05a97cbfb22d3703998cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/utils: Handle WebDriverExceptionRichard Purdie2024-10-241-5/+7
| | | | | | | | | | | | Fix the exception handling in the cancel builds function. This involved adding WebDriverException which sometimes occurs but also correcting the other exception handlers to continue to increment the timeout to avoid test hangs. (Bitbake rev: e111a2bd4f7a8a4dc2c63e94e91ac6cacca95af8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/helper: Add wait for jquery to completeRichard Purdie2024-10-241-0/+2
| | | | | | | | | | Most of the tests that click on buttons need the DOM to stablise, including any running JQuery code before the test can proceed. Add calls to do this whenever we're about to click on an element. (Bitbake rev: 0eb206b355248e2a874a62baec30025652f2a5a8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/project_page: Use ↵Richard Purdie2024-10-241-53/+40
| | | | | | | | | | | | | | | | | | | wait_until_element_clickable before click calls Switch the clickable() calls to use the new element_clickable() function which accepts a finder labmda function. This means if the element doesn't yet exist, the code can rebuild the query and try again once a small amount of time has elapsed. There were a ton of timing related races around these element interactions and this seemed to be the most robust way to address the issues. The change also makes some of the elements slightly more specific so the code can work effectively. (Bitbake rev: 38643aadbb5a960004b886cf7709beaf2fc96652) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/util: Avoid test hangsRichard Purdie2024-10-241-1/+1
| | | | | | | | | | If the element never exists, the timeout is never incremented and the test hangs indefinitely. Fix the exception handling to avoid that and allow the timeout to happen. (Bitbake rev: 9eabe923d457bbce65227da4cd71c275c32108e6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser: Add various fixes for test failures encounteredRichard Purdie2024-10-244-7/+13
| | | | | | | | | | | | Fixes for the browser tests which: * Add/improve waiting for specific elements * Waiting for elements to be interactable, not just visible * Hiding notification windows * Using more specific asserts to obtain improved error messages (Bitbake rev: 0c3f9bc2b9516d2af391f2e081280eb7fa62a05e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/helper: Add wait_until_element_clickable funcitonRichard Purdie2024-10-241-0/+7
| | | | | | | | Add a variant of the clickable function that accepts an element parameter. (Bitbake rev: a34b2451a0e3baaf3b2fd2eb351847ff6a5ee033) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/helper: Improve wait_until_clickable ↵Richard Purdie2024-10-241-11/+3
| | | | | | | | | | | | | | | exception handling Our own Wait() class allows exception handling which this form of wrapper does not. Switch the code to use our Wait() class to allow retrying upon encountering those exceptions (such as an element not being present yet). The displayed and visible test is what Selenium would be doing internally, there is no JS reprensetation of clickable directly. (Bitbake rev: 8266a01b750b3758badeee8fb3a1acfa72c17a93) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/utils: Drop unused code from ↵Richard Purdie2024-10-241-6/+1
| | | | | | | | | | | | | wait_until_build_cancelled The code modifying the build object requires direct database access which is potentially problematic. It was being run on a database which was being reset after changes anyway so the code effectively did nothing. Remove it as it is unneeded. (Bitbake rev: 2e3fa624eb3972aef91d877337479eeb0f3dd365) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/helpers: Drop remains of polling/sleep callsRichard Purdie2024-10-241-10/+7
| | | | | | | | | | | | | | | | Drop the remaining poll parameters from the helpers code along with the remaining sleep call since the tests no longer depend on this. This has the nice benefit of significantly speeding up the toaster test runs (45 minutes down to 12 minutes overall). If a parameter is needed, it should be the timeout, not the polling frequency. (Bitbake rev: 6de912e4f278ffd694fb2258482081dc3bc61c7a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functiona/project_page_tab_config: Improve waits and ↵Richard Purdie2024-10-241-20/+32
| | | | | | | | | | | | | drop polling Drop the poll parameters and make the waits much more specific for the requirements of the tests. This includes looping waiting for a list of layer elements as that code was previously particularly fragile. (Bitbake rev: cf6b8e8aa5484110a41377ba42b3fdd9d6efd877) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functiona/project_page_tab_config: Switch to using ↵Richard Purdie2024-10-241-123/+90
| | | | | | | | | | | | | | | | | library create_project function Switch this test module to use the common project creation code which contains race fixes. That code requires the database access wrapper be dropped and we no longer have ordering constraints. There is one test that does require database access. Move this to a separate class and allow database access there. Use ordering constraints to allow them to run after the main code. They depend on the project creation from the other class which isn't ideal but good enough for now. (Bitbake rev: fa10ba2a8749415d8f06cfc15c228c6eb7df1bcf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser/helpers: Add not visible wait functionRichard Purdie2024-10-241-0/+7
| | | | | | | | | In some cases we want to wait until some element is not visible. Add such a function helper. (Bitbake rev: cede6519d25112037264550126b109903220b58c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functiona/project_page: Improve waits and drop pollingRichard Purdie2024-10-241-13/+40
| | | | | | | | | | | | | | These changes: * Drop the poll parameters * Add waits where needed or make them more specific * Add calls to close notification boxes since they can obscure other page elements * In one case wait for a "Loading" message to be replaced (Bitbake rev: 2bde969486a4761fa0e6f9590c9fa08d385666b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functiona/project_page: Drop pointless test covered ↵Richard Purdie2024-10-241-9/+0
| | | | | | | | | | | elsewhere The project creation test is well covered in another module so just drop this one as there isn't any extra value added here. (Bitbake rev: d26414f191c73fb5330d10d97e240150a3fa1ee8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functiona/project_page: Switch to using library ↵Richard Purdie2024-10-241-76/+42
| | | | | | | | | | | | | | | | | create_project function Switch this test module to use the common project creation code which contains race fixes. That code requires the database access wrapper be dropped and we no longer have ordering constraints. There are two tests that do require database access. Move these to a separate class and allow database access there. Use ordering constraints to allow them to run after the main code. They depend on the project creation from the other class which isn't ideal but good enough for now. (Bitbake rev: e441bfe98ac41d48692ffbaeec90a9c780337fa4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/project_config: Fix the wait calls to ↵Richard Purdie2024-10-241-26/+21
| | | | | | | | | | | avoid polling Drop poll calls within the module, adding wait calls for elements of the page where testing showed it was needed. (Bitbake rev: 26746eae31f126ff87e678da66dcc976e412c89b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/basic: Improve the projects table ↵Richard Purdie2024-10-242-7/+14
| | | | | | | | | | | selection to a common function The project page selection code is race prone. Create a common function to resolve the race issue and use it from all the call sites rather than duplicate code. (Bitbake rev: f2bd615b97a6ff3944fa9c1d89a0ea996a12943d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional/helpers: Drop unneeded poll values and sleepRichard Purdie2024-10-242-3/+2
| | | | | | | | | | Drop the remaining poll values and the sleep within the main wait function as the races this was working around now seem to be resolved with the preceeding patches. (Bitbake rev: 91e54cbacc57316a4f2ca766afac81e7f3a54718) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser: Drop unneeded poll parametersRichard Purdie2024-10-247-20/+20
| | | | | | | | | In my testing I wasn't able to see issues needing the poll parameter in the browser tests. Drop those. (Bitbake rev: 3be5e7c78129abf059476416279718156fc666b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/browser: Increase the default wait timeoutRichard Purdie2024-10-241-1/+1
| | | | | | | | | | | | Rather than hardcoded poll and sleep values, we want an overall long timeout and to drop all the other 'sleeping' code. To facilitate this, increase the overall timeout so we can then start to remove the other values. (Bitbake rev: 69a479383f9a8cae6600ce51b12b62be3a15ea38) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/function/basic: Make element detection more specificRichard Purdie2024-10-241-23/+32
| | | | | | | | | | | | | The tests would race and potentially fail as the elements on the page being waited for were not specific enough. Add suitable elements to wait for and drop the remainder of the poll parameters since these are no longer needed since we no longer need to sleep for things to appear. (Bitbake rev: 9db2fd8534610ad8db90a7a696fa7b748b497d15) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/function/basic: Improve exception handling for error ↵Richard Purdie2024-10-241-49/+59
| | | | | | | | | | | | readability The 'fail' calls are useful for find() failures but not for wrapping existing asserts where they hide more useful error messages. Rework the code flow to give more readable error reporting. (Bitbake rev: 1a7a9af418874d3112aa6f148b30e5a4d3d176f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/project_config: Reuse common project creation codeRichard Purdie2024-10-241-46/+4
| | | | | | | | | | Switch this test module to use the common project creation code which contains race fixes. That code requires the database access wrapper be dropped and we no longer have ordering constraints. (Bitbake rev: 653233a1c798d71800b41d3321203a48aa0337d8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional_basic: Reuse common project creation codeRichard Purdie2024-10-241-21/+1
| | | | | | | | | | Switch this test module to use the common project creation code which contains race fixes. That code requires the database access wrapper be dropped and we no longer have ordering constraints. (Bitbake rev: f015d96e28d89fd0aef76ff127ebde8613125a66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional: Tweak the project creation function for ↵Richard Purdie2024-10-242-9/+17
| | | | | | | | | | | | | | | better reuse Tweak the function name and return the created project ID. Allow the release_name test to be skipped with a value of None. Add an extra wait call to ensure the create project button is ready before clicking it. (Bitbake rev: d20c6e3c67298c5db33b220a2e4a554e6f2c4418) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional: Move _create_test_new_project to base ↵Richard Purdie2024-10-242-69/+72
| | | | | | | | | | | class as helper The test project creation is needed by mutliple tests. Move it to the base class as a helper function for reuse. (Bitbake rev: abe483b98b38e95b4ad42629a67fb5d243c94094) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests/functional: Improve project creation testsRichard Purdie2024-10-242-8/+22
| | | | | | | | | | | | | | | | | | | | | Mixing database access and access via a running server is fraught with danger and problems. The "django_db" marker means the transactions are dropped at the end of the test but the transactions made via the webapi remain so the database ends up confused at best. Drop the database accesses and use the server API. This means slightly abusing the typeahead to get lists of projects in the database. Add code to delete a project if it already exists. This allows tests to re-run against an existing database. Deletion is done using the server API but this means handling CSRF tokens. Add requests module requirement to requirements file since the project creation code now uses requests. (Bitbake rev: 738270c53a08ddc95400de70f3dd8c08b2940182) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: set User-Agent to 'bitbake/version' in checkstatus()Alexander Kanavin2024-10-221-6/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This eliminates the last usage of 'fake mozilla' in bitbake, and it's then truthful everywhere about presenting itself, or wget (when that is used). I understand this will make people nervous so I want to provide an extended decription. 1. How was this tested? - bitbake-selftest -k FetchCheckStatusTest (tests a few hardcoded URIs, all passed) - bitbake -k -c checkuri world (runs checkstatus() over all recipes in oe-core, and all passed again - this hopefully goes a long way to reassure everyone that hosts around the world and various CDNs typically do not have a problem with user-agent strings they haven't seen before or bitbake user-agent specifically) 2. What about that removed cloudflare comment? I digged into git history, and I think it is not fully accurate. First, 'fake mozilla' agent is used only for checkstatus() - in actual fetching with wget it is not. And that has not been a problem for anyone. Second, here's how the comment occured. Usage of 'fake mozilla' was introduced here: https://git.yoctoproject.org/poky/commit/?h=master&id=ab26fdae9e5ae56bb84196698d3fa4fd568fe903 At that point it did not have to be specifically 'mozilla', the commit message indicates that any User-Agent would have been ok. Mozilla was simply copied from upstream version check for convenience. Later on, the string was updated to a more recent Mozilla: https://git.yoctoproject.org/poky/commit/?h=master&id=9f123238261a68e37cec634782e9320633cac5d4 The claim in the added comment become something else: that User-Agent *must* a browser, without evidence or tests. Even though it demonstrably doesn't have to be - wget is ok. 3. What if someone has a server that is ok with wget agent, but not ok with bitbake agent? Please see point one. It's not impossible but I think it's highly unlikely. I do think we should rather tell servers the truth, and learn where the actual issues are. Then we can consider options - whether that would be pretending to be wget, or allowing user-agent to be configured. We should also add such servers to bitbake-selftest so we know what they are. (Bitbake rev: 234f9e810494394527f59fdf22eb86435d046d53) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget.py: use default wget user-agent in upstream version checksAlexander Kanavin2024-10-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has been set to various mozilla versions from this point onwards without explanation or specific examples: https://git.yoctoproject.org/poky/commit/?h=master-next&id=ba06cc8943d021f406c3d292ce415b77319ca5c6 However, sourceforge has recently banned the current 'fake mozilla' version we've been supplying, resulting in broken version checks: alex@Zen2:/srv/storage/alex/yocto/build-test-sourceforge$ devtool check-upgrade-status gptfdisk ... --2024-10-12 11:53:23-- https://sourceforge.net/projects/gptfdisk/files/gptfdisk/ Resolving sourceforge.net (sourceforge.net)... 172.64.150.145, 104.18.37.111, 2606:4700:4400::6812:256f, ... Connecting to sourceforge.net (sourceforge.net)|172.64.150.145|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2024-10-12 11:53:24 ERROR 403: Forbidden. gptfdisk 1.0.10 UNKNOWN_BROKEN Alexander Kanavin <alex.kanavin@gmail.com> On the other hand, using wget's default user-agent works just fine: --2024-10-12 12:16:06-- https://sourceforge.net/projects/gptfdisk/files/gptfdisk/ Resolving sourceforge.net (sourceforge.net)... 104.18.37.111, 172.64.150.145, 2606:4700:4400::6812:256f, ... Connecting to sourceforge.net (sourceforge.net)|104.18.37.111|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] (Bitbake rev: 138117dfc05248dcb48707c838c8c1cc09f12d07) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster/tests: Replace assertTrue for more specific assertsRichard Purdie2024-10-142-91/+91
| | | | | | | | | | | | assetTrue is a poor choice for "x in y" since assertIn gives much more useful output upon failure. Change such inserts to assertIn or assertEqual to make errors easier to debug. (Bitbake rev: dde78e0ff8af872fdc5cdf5354174fc713141102) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add GitHub Codespaces supportOliver Kästner2024-10-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GitHub Codespaces allow users to use GitHub-hosted pre-configured Docker containers to work on GitHub repositories in VS Code within the browser or using a locally running instance of VS Code [1]. Users can define access permissions for Codespaces containers, allowing them to clone or work with other private repositories [2]. To do this, a git credential helper is injected into the container at /.codespaces/bin/gitcredential_github.sh, which contains the following lines: #!/bin/sh echo protocol=https echo host=github.com echo path= echo username=PersonalAccessToken echo password=$GITHUB_TOKEN The `GITHUB_TOKEN` environment variable is automatically set when the Codespaces container is created. If authorized by the user, it has the specified permissions. It is thus required to be able to fetch private git repos from a Codespace container, so add it to `FETCH_EXPORT_VARS` to make it available for invocations of git. [1] https://docs.github.com/en/codespaces/overview [2] https://docs.github.com/en/codespaces/managing-your-codespaces/managing-repository-access-for-your-codespaces (Bitbake rev: 71ca83bdd541f70737a3a85f05e40c222611af2f) Signed-off-by: Oliver Kästner <git@oliver-kaestner.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-08-03 20:18:10 +0000