summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/utils.py
Commit message (Collapse)AuthorAgeFilesLines
* lib/oe/utils: use multiprocessing from bbMartin Jansa2025-08-141-1/+1
| | | | | | | | | | | | | | | | Fixes build with python-3.14 It was added to bitbake in 62be9113d98fccb347c6aa0a10d5c4ee2857f8b6 and oe-core now requires latest bitbake already, so we can use this. [YOCTO #15858] (From OE-Core rev: 75b39bf4b0cbf0884ce5c7209f3ae9420a86316e) Signed-off-by: Martin Jansa <martin.jansa@gmail.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: extract method for parallel_make without d contextPeter Marko2025-08-111-8/+18
| | | | | | | | | | | | | | | oeqa does not have general access to d variable context and needs to determine parallel make settings. Extract the code from parallel_make into reusable parallel_make_value. Also correct function description of return value from None to empty string. (From OE-Core rev: c8670e9c7db565401412dad979c2ee53a586b59d) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sanity/utils: Directly use gcc, not BUILD_CCRichard Purdie2025-07-031-11/+6
| | | | | | | | | The test/helper is written assuming gcc, so just call that and stop accessing BUILD_CC which may be set to clang. (From OE-Core rev: 0a165a93693a293f08cb0d7e2dfa1016803a917a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* uninative/utils: Drop workarounds for gcc 4.8/4.9Richard Purdie2025-07-031-26/+0
| | | | | | | | | We require at least gcc 8.0 in sanity.bbclass so drop the 4.8/4.9 special case handling in uninative. (From OE-Core rev: 552e037bf598ac523f35b69d2dafc99e5ba59c5f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe: Move vardepexclude entries alongside functionsRichard Purdie2025-06-161-1/+4
| | | | | | | | | Now we have decorators that can do this, move the variable dependencies exclusions alongside the code that needs them for maintainability. (From OE-Core rev: e522169c5f95de6fc74b43672573700d8eb8e082) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/package/utils: Improve multiprocess_launch argument passingRichard Purdie2025-02-281-1/+3
| | | | | | | | | | The current code for multiple argument passing is horrible. Tweak the multiprocess_launch function to only convert to a tuple if it isn't already one, which means we can then use function arguments in a standard way. (From OE-Core rev: 7c99f90079e722764ebdc30e8d0e781454b3a51a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.bbclass: Drop support for ImageQAFailed exceptions in image_qaPeter Kjellerstedt2024-09-301-13/+0
| | | | | | | | | | | | | | | | | | | | After commit 905e224849fbbed1719e0add231b00e2d570b3b4 (image_qa: fix error handling), any unexpected exceptions in do_image_qa() would result in a variable being set, but never used, effectively hiding the error. Since image_qa now calls oe.qa.exit_if_errors(), remove the support for oe.utils.ImageQAFailed and instead rely on the called functions to call oe.qa.handle_error() themselves. This matches what do_package_qa() does. Also update the description of do_image_qa() to explain that the called functions are expected to call oe.qa.handle_error() themselves. [ YOCTO #15601 ] (From OE-Core rev: 0c3e111c965af2bc56533633c376b70b7fa5e1de) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image_qa: fix error handlingLouis Rannou2024-09-171-1/+1
| | | | | | | | | | | | | | | | Make ImageQAFailed inherit BBHandledException so exceptions raised in tests are catched when the actual test function is executed by bb.utils.better_exec. Change the do_image_qa tasks so errors are handled with oe.qa.handle_error. Add some comment to explain this requires to list the test in ERROR_QA or WARN_QA. [YOCTO #14807] https://bugzilla.yoctoproject.org/show_bug.cgi?id=14807 (From OE-Core rev: 905e224849fbbed1719e0add231b00e2d570b3b4) Signed-off-by: Louis Rannou <louis.rannou@non.se.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Ensure sstate searches update file mtimeRichard Purdie2023-11-081-0/+12
| | | | | | | | | | | | | | | Commands like "bitbake XXX -S printdiff" search for sstate files but don't download them. This means that local files aren't touched as the download code would do, meaning the sstate cleanup scripts can delete them. This can then lead to obtuse build failures. Have the search code touch local files in the same way as the main code paths would to avoid these files disappearing. Move the function to a common touch() function in lib/oe instead of duplicating code. (From OE-Core rev: a27fc0bd5706ab5b9c68a0271fcf57377a678cdf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Refactor to make multiprocess_launch callable without dAdrian Freihofer2023-09-121-3/+9
| | | | | | | | | | | This is a preparation for making the strip_execs function callable from devtool without going via tinfoil and a bitbake server process. (From OE-Core rev: af8ee73cdef90b83556a7ac5e139a08108706486) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: Drop ';' delimiter from ROOTFS/IMAGE*COMMAND variablesRichard Purdie2023-09-091-5/+5
| | | | | | | | | | | | | Originally these were shell functions but they have long since been processed by bb.build.exec_func(). Since we no longer need shell syntax, we can drop the ';' delimiters and just use a space separated string. This cleans up the variable and quietly removes any stray ';' that do happen to still make it in. (From OE-Core rev: c3365dfd9ddd7fbe70b62e0f11166e57a8ca6f73) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: Add copyright statements to files without oneRichard Purdie2022-08-121-0/+2
| | | | | | | | | Where there isn't a copyright statement, add one to make it explicit. Also add license identifiers as MIT if there isn't one. (From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: remove the ThreadedPoolJose Quaresma2022-07-151-64/+0
| | | | | | | | | | | | | | | The ThreadedPool in OE-core is mainly because python2 doesn't have threaded pools but python2 is dead for some time now and python3 have a ThreadPoolExecutor. The only local in OE-core where this ThreadedPool is in use is on the sstate.bbclass that is ported to the python3 ThreadPoolExecutor. (From OE-Core rev: aa8fd5e7c2a1125895accfd55ce9320819a10959) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Improve internal variable namingSaul Wold2022-03-101-2/+2
| | | | | | | | | Update internal variable names to improve the terms used. (From OE-Core rev: f408068e5d7998ae165f3002e51bc54b380b8099) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* prelink: Drop support for itRichard Purdie2022-02-051-11/+0
| | | | | | | | | | | | | | Prelink is being dropped by glibc in 2.36. It already causes issues with binary corruption, has a number of open bugs and is of questionable benefit without disabling load address randomization and PIE executables. We disabled it by default a while back but left people able to use it. We would be unable to maintain it alone without glibc support so remove the remaining pieces. (From OE-Core rev: 23c0be78106f1d1e2bb9c724174a1bb8c56c2469) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: by default cap cpu_count() to 64 coresRoss Burton2021-11-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Larger systems may have large numbers of cores, but beyond a certain point they can't all be used for compiling: whilst purely compute-intensive jobs can be parallelised to hundreds of cores, operations such as compressing (needs lots of RAM) or compiling (lots of I/O) don't scale linearly. For example, the Marvel ThunderX2 has 32 cores, each capable of executing four threads, and can be configured with two sockets, making 256 CPUs according to Linux. Zstd using 256 threads has been seen to fail to allocate memory during even small recipes such as iso-codes. Add a default cap of 64 CPUs to the cpu_count() method so that extreme parallisation is limited. 64 is high enough that meaningful gains beyond it are unlikely, but high enough that most systems won't be effected. (From OE-Core rev: 765d0f25ce48636b1838a5968e2dc15de2127428) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: allow naming threads in ThreadedPoolRoss Burton2021-11-251-6/+5
| | | | | | | | | | When looking at logs involving thread pools it is useful if the threads can be named. (From OE-Core rev: 18342945b021608794d83ecf567afd43f4379b24) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: log exceptions in ThreadedWorker functionsRoss Burton2021-10-181-1/+2
| | | | | | | | | | | | | | If the function a ThreadedWorker is executing raises an exception, don't use print() as that mostly disappears. Instead, output it to the logger. This is done using bb.mainlogger.debug directly instead of bb.debug() as this allows us to pass the exception instance directly, which is then incorporated into the log stream. (From OE-Core rev: 2f1ea25c222b344dd8b784b2bc73a6540ab30274) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: Label packages providing per-file dependencies in depends.dotAndres Beltran2021-09-011-2/+30
| | | | | | | | | | | | | | | | Currently, depends.dot includes per-file dependencies but not the packages providing those files. This makes it hard to obtain all package dependencies by just looking at depends.dot. Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of their values to the package providing the component. Include runtime packages as dependencies in depends.dot, together with the component provided by the package as a label. (From OE-Core rev: 2ba33093017574bbe29eeba699eb90628614d03a) Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: add directory size functionRoss Burton2021-03-281-0/+31
| | | | | | | | | | | | | | | | | | | For the purpose of image construction using du on a rootfs directory isn't entirely satisfactory. Bare "du" will report the actual disk usage so file systems which can compress the data will report less than the actual space required. Using "du --apparent-size" will report the actual space used, but as this simply sums the bytes used for content across an entire file system can result in significant under-reporting due to block size overhead. Attempt to solve these problems by implementing our own function to calculate how large a rootfs will be. This function handles hardlinks correctly but rounds up all sizes to multiples of the block size (currently, 4KB is the hard-coded block size). (From OE-Core rev: 6ca53ad7b26ee2b4e6d2c121c6f6d6eed7f6b56f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Return empty string in parallel_makeTomasz Dziendzielski2021-01-161-1/+1
| | | | | | | | | | | | | In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make function and if PARALLEL_MAKE is set to empty string then this variable is exported as "None" causing cmake to fail with: "'CMAKE_BUILD_PARALLEL_LEVEL' environment variable invalid number 'None' given." (From OE-Core rev: 2f790ded554a52ac18d1c28002142f9c62abec8b) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: respect scheduler affinity in cpu_count()Ross Burton2020-09-301-2/+1
| | | | | | | | | | | | | | | cpu_count() returns multiprocessing.cpu_count() but that is simply returns os.cpu_count() so we could use that directly. However this returns the number of CPUs on the host, not the number of usable CPUs on the host. If the user is using scheduler affinity then the number of usable CPUs may be less, so when determining how many cores we can use check the affinity instead. (From OE-Core rev: e2e8ccbe410b2f38bcd9525982b2261cf71aab60) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils.py: add get_host_compiler_version()Tim Orling2020-04-011-0/+31
| | | | | | | | | | | | | | Add helper function to get the host compiler and version. Do not assume compiler is gcc. NOTE: cannot set env to d.getVar("PATH") as that does not contain the session PATH which was set by environment-setup-... which breaks the install-buildtools use-case (From OE-Core rev: 88712929354ff9c876bb1e48b6f15c33af5f2bbc) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: fix gcc 10 version detectionCharles-Antoine Couret2020-03-291-1/+1
| | | | | | | | | | | | Utils can not detect GCC 10 correctly due to wrong regex. It generates this error "ERROR: Can't get compiler version from gcc --version output" Sub-version numbers should be 1 or more digits instead of 1 only. (From OE-Core rev: 186fe4a3d390a52b87282c3e694ce3251e45ee78) Signed-off-by: Charles-Antoine Couret <charles-antoine.couret@mind.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: allow to set a lower bound on returned cpu_count()André Draszik2020-03-061-2/+3
| | | | | | | | | | | | | | | | | | | This will be needed for making xz compression more deterministic, as xz archives are created differently in single- vs multi-threaded modes. This means that due to bitbake's default of using as many threads as there are cores in the system, files compressed with xz will be different if built on a multi-core system compared to single-core systems. Allowing cpu_count() here to return a lower bound, will allow forcing xz to always use multi-threaded operation. (From OE-Core rev: c6dda204e15c5b27559f7617b18a7b3ce604e2f8) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cmake: prefer CMAKE_BUILD_PARALLEL_LEVELDaniel McGregor2020-02-041-4/+7
| | | | | | | | | | | | cmake 3.12 introduced this environment variable. Prefer it to passing PARALLEL_MAKE and PARALLEL_MAKEINST on the cmake command line, because it gets passed to second stage cmake invocations while command-line arguments do not (for example, multi-stage clang builds) (From OE-Core rev: cdd44c93f02bb8cc2fa773e13c8ce36e3da23921) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Remove bb.build.FuncFailedRichard Purdie2019-08-061-1/+1
| | | | | | | | | | | | | Whilst seemingly a good idea, this exception doesn't really serve any purpose that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't pythonic. Its not used in many places, lets clean up those and remove usage of it entirely. It may ultimately be dropped form bitbake entirely. (From OE-Core rev: efe87ce4b2154c6f1c591ed9d8f770c229b044ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-091-0/+4
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils/multiprocess_launch: Improve failing subprocess outputRichard Purdie2019-04-261-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Output before this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.: Traceback (most recent call last): File "/media/build1/poky/meta/lib/oe/utils.py", line 272, in run ret = self._target(*self._args, **self._kwargs) File "/media/build1/poky/meta/classes/package_ipk.bbclass", line 230, in ipk_write_pkg shell=True) File "/usr/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Note how stdout/stderr from the failing command isn't shown. After this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:Foobar *** Error: Package name Foobar contains illegal characters, (other than [a-z0-9.+-]) opkg-build: Please fix the above errors and try again. We suddenly get a much more usable error message. The traceback is supressed as its distracting from the real problem in this case. Ideally python itself would handle this but it doesn't so we have to wrap the exception. We already do this in bitbake itself for the same reason. (From OE-Core rev: 09276dc76a8bda237b0b0b6d117a1980ae9dbfcc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Make prune_suffix prune a suffixAndre Rosa2019-04-091-3/+3
| | | | | | | | | | | ... instead of replacing a substring that could happen more than once and not only when it ends with it. Do the same for the prefix. See related https://github.com/openembedded/bitbake/pull/24 . There it stops replacing sufixes once first one is matched but not here. (From OE-Core rev: 610ac84170f8a91cc3321edfc336a9e39f24ebe3) Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: added sh_quote() functionEnrico Scholz via Openembedded-core2019-04-041-0/+3
| | | | | | | | | | | This function is a wrapper around "shlex.quote()" and can be used in "${@...}" context where shlex (or pipes, which provides similar functionality) is unavailable. (From OE-Core rev: 127141f5023a7e3fc3963dc7d76cfce9067a9e8a) Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Fix hang in multiprocess_launch()Richard Purdie2019-02-171-0/+4
| | | | | | | | | | | | | | If large results values are returned by the subprocesses, we can hit a deadlock where the subprocess is trying to write data back to the parent, the pipe is full and the parent is waiting for the child to exit. Avoid this by calling the update() method which would trigger reading a result from the child, avoiding the deadlock. The issue is described in https://bugs.python.org/issue8426 (From OE-Core rev: 0035e8066ecbbff94d6a1994a9f72b1368d660d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: add -src to system package name blacklistRoss Burton2019-02-151-1/+1
| | | | | | | | | | | oe.utils.packages_filter_out_system() returns PACKAGES after removing "system" packages but it doesn't handle ${PN}-src as generated by PACKAGE_DEBUG_SPLIT_STYLE=debug-with-srcpkg. (From OE-Core rev: 162632d3d1e40c83ed9c5d49a026edf3912860a0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-2/+2
| | | | | | | | | | | | | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. Note that some show up as: """ meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.   """ where the problem isn't on 1293 in package.bbclass but in some _prepend to a package.bbclass function in a different file like mesa.inc, often from do_package_split() calls. (From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ccache.bbclass: Refactor it to make it more reliableRobert Yang2019-01-141-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous ccache.bbclass has the following problems: - It uses host's ccache for native recipes, but this may not work on some hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always build failures (m4-native failed at do_configure, and others will also be failed if I disable CCACHE for m4-native) - native/nativesdk/cross/crosssdk recipes use host's ccache, but target uses ccache-native, this may confuse user. - The target recipes may use both host's ccache and ccache-native, this may cause unexpected problems and be hard to debug. This is because ccache-native is in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when rebuild target recipes, and then it would use hosttools/ccache, but the previous ccache files were generated by ccache-native. - Target recipes can't use ccache when no ccache is installed on the host: CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}" After refactored: All types recipes (native, target and others) will use ccache-native except ccache-native itself, host's cache won't be used any more. It is more reliable now, which will work everywhere when ccache-native can be built. And now we need use "CCACHE_DISABLE = '1'" to disable ccache for the recipe rather than "CCACHE = ''" since we set CCACHE in anonymous function, and d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we can't check whether CCACHE is set or not in anonymous function since it is always set. Use CCACHE_DISABLE to disable it would be more clear. (From OE-Core rev: b25271b65262f70d849a4861da216c9be6c54d53) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Set stderr for host_gcc_version()Robert Yang2018-12-141-1/+2
| | | | | | | | | | | | | | | | | | | Fixed: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc $ rm -fr tmp/hosttools/ && bitbake -p [snip] ERROR: Error running gcc --version: It didn't print the error message, now it is: ERROR: Error running gcc --version: ccache: error: Could not find compiler "gcc" in PATH For the error itself, it is because ccache is not in my HOSTTOOLS, so this is an expected error. (From OE-Core rev: 91955caae584b4f75118e04411851b1a3d783fec) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Improve multiprocess_lauch exception handlingRichard Purdie2018-12-051-2/+3
| | | | | | | | | | | | | | We've seen a cryptic: "ERROR: Fatal errors occurred in subprocesses, tracebacks printed above" message from oe-selftest with no other traceback information. Improve the traceback logging to try and give a better indication of any errors that is ocurring. (From OE-Core rev: 521dd3d00979a27b6932e58d5497de68abac26e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: add eol to format_pkg_list()grygorii tertychnyi via Openembedded-core2018-10-111-1/+7
| | | | | | | | | | | | | | | | | Append '\n' to the non-empty formatted string before return. If you write it to the (manifest) file, it will ensure file ends with a newline. Many GNU utilities have problems processing the last line of a file if it is not '\n' terminated. E.g. if the last line is not terminated by a newline character, then "read" will read it but return false, leaving the broken partial line in the read variable(s). It can also break or adversely affect some text processing tools, that operate on the file. (From OE-Core rev: ee4d0c879713ba50dc6cc3300f44647faebee2e0) Signed-off-by: grygorii tertychnyi <gtertych@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils.py: Add vartrue()Robert Yang2018-09-131-0/+8
| | | | | | | | | | It can be used to simplify code like: "${@['iffalse', 'iftrue'][var]}" (From OE-Core rev: fc5a5af7bc3619f575988a75efc0c4fe15478b2d) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* populate_base_sdk: Stop running gcc --version all the timeRichard Purdie2018-09-051-1/+4
| | | | | | | | | | Running 'gcc --version' for every image recipe is slow and increases parsing time/resource usage for no good reason. Only compute the value in when we're really running the task/function. (From OE-Core rev: bf49316bb9913b7c89de64d6a194be31aa66e16b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Fix get_multilib_datastore to work for original tuneRichard Purdie2018-08-231-0/+3
| | | | | | | | | | | Currently the original datastore returned by this function doesn't always work as the tune isn't set back to the original. Fix it to work like all_multilib_tune_list() in utils.bbclass and correct the data returned. (From OE-Core rev: 4e1dc858fbf671ef27089a2b9bcdc965fe19d698) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Drop now unused multiprocess_execRichard Purdie2018-07-241-34/+0
| | | | | | (From OE-Core rev: f3b30def2cd1c9ede7630489c3949a45b6eba6ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Add multiprocess_launch API and testcaseRichard Purdie2018-07-241-0/+70
| | | | | | | | | | | | | | | | | | | | | The current methods of spawning processes for parallel execution have issues around collection of results or exceptions. Take the code from package_ipk/deb, make it generic, add a results collection mechanism, fix the exception handling and for it into a standard library function. Also add a test case which tests both the success and failure modes of operation to stop this functionality regressiing again. In particular, compared to multiprocess_exec, this fork off the parent approach means we can pass in the datastore and functions work in the same scope as the parent. This removes some of the complexities found trying to scale multiprocess_exec to wider use. (From OE-Core rev: 88f0c214e593a45566df5131bda4c946f5ccc8c2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Improve get_multilib_datastoreRichard Purdie2018-07-021-3/+9
| | | | | | | | | | | | | Currently this function assumes that no multilib is applied and that we're applying a multilib. This means if we're in multilib context and want the non-multilib context we can't obtain it (and no other function exists for this either). Improve the function to allow this to be requested. (From OE-Core rev: 295c5a3d19834a2fac255346d0a373449cfdd776) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: Fix parallel_make limitJoshua Watt2018-02-241-1/+1
| | | | | | | | | | | parallel_make_argument() was incorrectly taking the maximum of the limit and the calculated value instead of the minimum. (From OE-Core rev: 45205be547967c84dff96403c3a6825a62e3ca6a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: add parallel make helpersJoshua Watt2018-02-161-0/+43
| | | | | | | | | | | | | | | | | | | The code to extract the integer number of parallel build threads and construct a new argument from them has started to be copied in multiple locations, so create two new helper utilities to aid recipes. The first helper (parallel_make()) extracts the integer number of parallel build threads from PARALLEL_MAKE. The second (parallel_make_argument()) does the same and then puts the result back into a format string, optionally clamping it to some maximum value. Additionally, rework the oe-core recipes that were manually doing this to use the new helper utilities. (From OE-Core rev: ccd1142d22b31ed85d8823b1bc9e11ccfd72b61f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Make get_multilib_datastore available from lib/oeRichard Purdie2018-01-041-0/+8
| | | | | | | | | Currently we can't access this function from lib/oe as its a class function. Move it to allow such access. (From OE-Core rev: b241a666f2867ffa425f6d43763d7c3c17941dcf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: remove param_bool()Ross Burton2017-12-021-11/+0
| | | | | | | | | | This function is not used by any classes or recipes that I can find, so lets delete it. (From OE-Core rev: a7cd9d1183be603777fc9c8c448281fe01224f7b) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Handle exceptions in multiprocess_execRichard Purdie2017-09-021-8/+13
| | | | | | | | | | | | Currently exceptions that happen in pool commands are ignored. Any errors would be printed on the console but everything else is silent. Switch to use pool.map_async which allows for an error_callback which we can use to detect exceptions and make sure these errors are handled. (From OE-Core rev: 7f2f9b3ff011b340b5d23bb7c47b12c357dc9f02) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: helper function for optional include filesPatrick Ohly2017-06-121-0/+40
| | | | | | | | | | | | | | | | | | | | | | The main intention is to provide easy-to-use and read helper functions for including files only when certain distro features are set. Functionally they are the same as bb.utils.contains and bb.utils.contains_any. Distro features are part of the base configuration and thus safe to use for conditional inclusion in recipes and bbappends, in contrast to recipe variables which might still change during parsing. Therefore the check is limited to DISTRO_FEATURES. This is the reason for having this in OE-core instead of bitbake. Default values are set so that no redundant parameters have to be passed for conditional includes. As a secondary usage, the functions can also be used in boolean checks. (From OE-Core rev: 13024ce5aae453769b546d5fbe533443aec3d6fd) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-10-06 14:05:59 +0000