summaryrefslogtreecommitdiffstats
path: root/scripts/lib/recipetool/create.py
Commit message (Collapse)AuthorAgeFilesLines
* recipetool: create_npm: reuse license utilsEnguerrand de Ribaucourt2024-08-231-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | create_npm.py duplicated the logic for matching licenses from files and also finding them. This patch refactors the code to reuse the license utils. This will make the code more maintainable and also align both behaviors. For instance, some licenses weren't matched properly because the duplicate logic did not support the difference in format in the md5 tables for COMMON_LICENSE_DIR and licenses.csv. This is also faster since the license files were being read twice. The result is slightly more accurate since the utils have better implementations, and I was able to reuse the logic for the root PN package, as well as the base LICENSE variable. I chose to extract generate_common_licenses_chksums into create.py since it can be considered a general utility function to allow other recipetool creators to refer to COMMON_LICENSE_DIR files. I updated the wording in the code when appropriate. v3: - added commit - this replaces the commit that added all the COMMON_LICENSE_DIR md5 to licenses.csv (From OE-Core rev: 7bc18bed63b94689890bcde63402d7cc1cedffa9) Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: split guess_license functionEnguerrand de Ribaucourt2024-08-231-12/+22
| | | | | | | | | | | | | | | The npm recipetool handler redefines the license code the could be unified. In order to do this refactoring, extract the bits we'll need into separate functions. guess_license() is renamed to find_licenses() and is split into find_license_files() and match_licenses(). (From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2) Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/devtool: Update to work correctly with UNPACKDIRRichard Purdie2024-05-221-1/+1
| | | | | | | | | | Tweak recipetool and devtool to correctly use UNPACKDIR. This allows some simplification of the code. This patch makes things basically work but there are likely deeper improvements that can be made now that WORKDIR != UNPACKDIR. (From OE-Core rev: d2eeaa88b27a2875c419591d1d91bcc85d7b129c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: python 3.12 regexAdrian Freihofer2024-02-131-6/+6
| | | | | | | | | | | | | | | | | | All the regexes throw a warning like this: WARNING: scripts/lib/recipetool/create_buildsys.py:140: SyntaxWarning: invalid escape sequence '\s' proj_re = re.compile('project\s*\(([^)]*)\)', re.IGNORECASE) Python 3 interprets string literals as Unicode strings, and therefore \s is treated as an escaped Unicode character which is not correct. Declaring the RegEx pattern as a raw string instead of unicode is required for Python 3. (From OE-Core rev: 24b0ba00d4f0b4d9834f7693ecb6032dfc534a80) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: Only include the expected SRC_URI checksumsPeter Kjellerstedt2023-12-081-1/+3
| | | | | | | | | | | | Rather than including all SRC_URI checksums, include the ones that are expected. These are the same as are output if no checksums are included when building the recipe. (From OE-Core rev: c2af83eb5e8573480179b6c0bcce50606b547099) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: Improve identification of licensesPeter Kjellerstedt2023-12-081-46/+45
| | | | | | | | | | | | Rather than having a static list of crunched MD5 checksums for some of the most common licenses, calculate it for all common licenses. This should improve the identification of license text variantions. (From OE-Core rev: 377f9513dc56e9b8e5f5813c1535be0206756949) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create_buildsys_python: add pypi supportJulien Stephan2023-12-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Today, we can use devtool/recipetool to create recipes for python projects using the github url or the direct release tarball of the project, but the create_buildsys_python plugin doesn't support the pypi class, since we cannot know from the extracted source if the package is available on pypi or not. By implementing the new optional process_url callback, we can detect that the url is a pypi one (i.e 'https://pypi.org/project/<package>') and retrieve the release tarball location. Also detect if the url points to a release tarball hosted on "files.pythonhosted.iorg" (i.e https://files.pythonhosted.org/packages/...) In both cases, adds the pypi class, remove 'S' and 'SRC_URIxxx' variables from the created recipe as they will be handled by the pypi class and add the PYPI_PACKAGE variable This helps to produce cleaner recipes when package is hosted on pypi. If the url points to a github url or a release tarball not coming from "files.pythonhosted.org", the created recipe is the same as before. One can also use the newly added "--no-pypi" switch to NOT inherit from pypi class on matching url, to keep legacy behaviour. To create a recipe for a pypi package, one can now use one of the new following syntax (using recipetool create / devtool add): * recipetool create https://pypi.org/project/<package> * recipetool create https://pypi.org/project/<package>/<version> * recipetool create https://pypi.org/project/<package> --version <version> or the old syntax: * recipetool create https://files.pythonhosted.org/packages/<...> (From OE-Core rev: 097a43846cd99a7d74d004efc57f583ce78970a4) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: add new optional process_url callback for pluginsJulien Stephan2023-12-061-22/+32
| | | | | | | | | | | | | | | | | | | | | | | Add a new process_url callback that plugins can optionally implement if they which to handle url. Plugins can implement this callback for example, to: * transform the url * add special variables using extravalues * add extra classes * ... If a plugin handles the url, it must append 'url' to the handled list and must return the fetchuri No functional changes expected for plugins non implementing this optional callback (From OE-Core rev: 3e5ce351e7dbe283562bf1db1f2ce3b121c49b53) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: add trailing newlinesJulien Stephan2023-12-061-3/+5
| | | | | | | | | | | | | | create_recipe() function relies on oe.recipeutils.patch_recipe_lines() which relies on bb.utils.edit_metadata(). edit_metada expect lines to have trailing newlines, so add it to each lines before calling patch_recipe_lines, otherwise edit_metadata will not be able to squash blank line if there are two consecutive blanks after a removal (From OE-Core rev: 31b27cc76a1b669e2b126c332608fd772c124d69) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Ignore *.go files while scanning for licensesLukas Funke2023-11-061-1/+1
| | | | | | | (From OE-Core rev: 1765acc8bdc08e392daa8b77add8b4ce1a9e70b6) Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/devtool: Ensure server knows about changed filesRichard Purdie2023-09-181-0/+1
| | | | | | | | | Bitbake has changed to require notification when metadata changes in the middle of tinfoil sessions. Add the required function calls at the places metadata is changed. (From OE-Core rev: e5574163ab49a8f51b2b34fd37acfd1cad9b7595) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes/classes/scripts: Drop SRCPV usage in OE-CoreRichard Purdie2023-08-241-1/+1
| | | | | | | | Now that SRCPV isn't needed we can simplify things in a few places... (From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Fix inherit in created -native* recipesYoann Congal2023-06-281-0/+4
| | | | | | | | | | native and nativesdk classes are special and must be inherited last : put them at the end of the gathered classes to inherit. (From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Do not use mutable default arguments in PythonStefan Herbrechtsmeier2022-04-191-2/+2
| | | | | | | | | | | | | | | | | Remove mutable default arguments in Python because they can lead to all sorts of nasty and horrible bugs. https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/ Revert `recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []` and instead check fallback_licenses before use. (From OE-Core rev: 99dee60b8db557f54783bf0f61098587badc683c) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/devtool: Further SPDX identifier cleanupsRichard Purdie2022-03-021-5/+5
| | | | | | | | | | | | Some of these are hard to know what to do with since the original source files for the checksum aren't present. The safe option is to use "-only" as often the main license is ambiguous and the source files themselves determine the "or-later" possibility. The "-only" option therefore is realistically what we need to use in this code. (From OE-Core rev: 2b0cbafc7854de0308a624b17b8aaba704b031d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Use SPDX license identifiersPeter Kjellerstedt2022-03-011-6/+6
| | | | | | | | | | | | There are still a couple of cases where the license may be set as, e.g., "GPL" or "GPL-2.0" since there is not enough information to decide the actual SPDX license. It is then assumed that the developer will have to correct the information. (From OE-Core rev: e7df51f8d2361e9fe2d67669d2e17f0a5d01004e) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Fix circular reference in SRC_URISaul Wold2022-02-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating a new recipe.bb file for a binary, don't use BP which includes the version information, instead use BPN which is just the name base Package Name. Since PB is not specified, it takes the default: PV = "1.0+git${SRCPV}" But SRCPV is defined in terms of the SRC_URI, which leads to infinite recursion (traceback below). Here are the pertinent variables which cause the recursion: SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}" BP = "${BPN}-${PV}" PV = "1.0+git${SRCPV}" SRCPV = "${@bb.fetch2.get_srcrev(d)}" def get_srcrev(d, method_name='sortable_revision'): # ... trimmed scms = [] fetcher = Fetch(d.getVar('SRC_URI').split(), d) # ... trimmed [YOCTO #14040] (From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Separate licenses with & operatorStefan Herbrechtsmeier2021-12-201-9/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Separate licenses with & operator since it should be satisfied most use cases and it is a reasonable assumption that all the licenses apply. Furthermore flat, split and sort the licenses to minimize license string changes. Separate package licenses with & operator: -LICENSE:${PN} = "MIT ISC" +LICENSE:${PN} = "ISC & MIT" Respect | and brackets in LICENSE: -LICENSE = "BSD-3-Clause & (ISC & | & MIT)" +LICENSE = "BSD-3-Clause & (ISC | MIT)" Sort licenses: -LICENSE = "MIT & BSD-3-Clause & ISC" +LICENSE = "BSD-3-Clause & ISC & MIT" Remove duplicates: -LICENSE = "MIT & ISC & MIT" +LICENSE = "ISC & MIT" (From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Sort output of guess_license function to be deterministicStefan Herbrechtsmeier2021-12-201-1/+1
| | | | | | | | | | Sort the output of guess_license function by license file to be deterministic and to support self-testing. (From OE-Core rev: ff2a7520813b3f775f96879d1905222e08b8c83f) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Set master branch only as fallbackStefan Herbrechtsmeier2021-12-091-8/+7
| | | | | | | | | | | | | The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)' sets the branch= parameter too early to master and thereby breaks the -B/--srcbranch option. ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other (From OE-Core rev: 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: handle GitLab URLs like we do GitHubRoss Burton2021-12-031-1/+1
| | | | | | | | | | GitHub URLs are automatically transformed to git: fetches, so handle GitLab URLs too. (From OE-Core rev: 651fb951819840fe122458ddbd852ee6c7ec0455) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Manual git url branch additionsRichard Purdie2021-10-301-0/+3
| | | | | | | | | | Following the scripted conversion adding branches to git:// SRC_URI entries, add the remaining references, mainly in the selftests and recipetool. (From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Simplify common source files skip in guess_licenseStefan Herbrechtsmeier2021-10-231-1/+2
| | | | | | | | (From OE-Core rev: 189c0297632968900715d7a4a5edcdf3a56e25f5) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Rework crunch_license to recognize more variantsStefan Herbrechtsmeier2021-10-141-24/+97
| | | | | | | | (From OE-Core rev: 6c70f5bd2acd036c21be5b03df89d3a8f381a822) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Add support for linenumbers to licenses.csvStefan Herbrechtsmeier2021-10-141-4/+8
| | | | | | | | | | | | Add support for linenumbers (begin and end lines) to licenses.csv. Add an optional linenumbers parameter to get_license_md5sums to support different use cases. (From OE-Core rev: d5c4979669f125e73c24dcc73fa3c4f3787bbb62) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Add logger info for missing license entriesStefan Herbrechtsmeier2021-10-141-0/+3
| | | | | | | | (From OE-Core rev: 0548a5d8eeee682a6e250ddc1886279f52747db2) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: ignore empty license filesStefan Herbrechtsmeier2021-10-141-2/+3
| | | | | | | | (From OE-Core rev: a00720344d1996db9e4afff1c974b3158fb8dae7) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Change default paramter fallback_licenses of function ↵Stefan Herbrechtsmeier2021-10-141-1/+1
| | | | | | | | | | split_pkg_licenses from None to [] (From OE-Core rev: ab06305bf9a8ee2250aae8043e37119df2fbfc15) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Skip common source files in guess_licenseStefan Herbrechtsmeier2021-10-141-0/+2
| | | | | | | | (From OE-Core rev: a930d39787dd77e10dfa7b7297af5fa04ca731ea) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Move license md5sums into CSV filesStefan Herbrechtsmeier2021-10-141-37/+15
| | | | | | | | (From OE-Core rev: 631f6b7d1856e0e4f2313703fa1162d05e5deea6) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package/scripts: Fix FILES_INFO handlingRichard Purdie2021-08-171-2/+2
| | | | | | | | | | | | | | | There is a long standing bug where FILES_INFO isn't written into pkgdata with a package suffix. This means if the files are read into the datastore as intended, the last one "wins". Fix this to work as intended. Most of the call sites using the data need to be updated to handle this and the overrides change correctly. Also fix some other problematic references noticed along the way. (From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Convert to new override syntaxRichard Purdie2021-08-021-2/+2
| | | | | | | | | | | | This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". (From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/recipetool/create.py: fix regex stringsTim Orling2020-07-121-5/+5
| | | | | | | | | | | Python now expects regex strings to be prepended with r. Silence pylint/autopep8 and similar warnings by identifying these regex patterns as... regex patterns. (From OE-Core rev: 0eae98a369d80340e48dc690d09a1364cde97973) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: fix SRCBRANCH not being passed to paramsTuomas Salokanto2020-06-171-0/+1
| | | | | | | | | | | | When explicitly passing a branch using --srcbranch in 'devtool add' or 'recipetool create', the branch name is not included in the params of bb.fetch2.encodeurl and default 'master' branch is used instead. (From OE-Core rev: 0424df825f1e509faf6cd44403c0736bb91b57c3) Signed-off-by: Tuomas Salokanto <tuomas.salokanto@gmail.com> Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create: npm: remove the 'noverify' url parameterJean-Marie LEMETAYER2020-01-271-2/+0
| | | | | | | | | | | This commit removes the 'noverify' parameter which was added to the url to fix warnings with the shrinkwrap / lockdown file generation. This is not needed anymore with the new npm fetcher. (From OE-Core rev: 401a9818c633373854d3c7dec032b9f455f2b2b4) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: npm: rename npm command line optionsJean-Marie LEMETAYER2020-01-271-5/+3
| | | | | | | | | | This commit renames the '--fetch-dev' option into '--npm-dev' as it is a npm only option. (From OE-Core rev: 2b75cc848ceebee4067788a621299bfd5fb62231) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix devtool upgrade with reproducible_builds classPaul Eggleton2019-11-211-3/+1
| | | | | | | | | | | | | | | | | | If the reproducible_build class is inherited then there may be a "source-date-epoch" subdirectory in a fetched source tree; devtool upgrade was not expecting that in the upgraded source. Take a small snippet of code from recipetool create which already handles this, and make it a shared function that can be used in both places. Additionally, fix an assumption that the source is always in a subdirectory in the cleanup code that blocked debugging this. [YOCTO #13635] (From OE-Core rev: 0d642861cd9cf034b8d4951433980addc215d4fd) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create: Fix to work with reproducible_buildsRichard Purdie2019-11-211-1/+1
| | | | | | | | | | The extra diretory created by reproducibile_builds was confusing recipetool and hence devtool. Ignore this directory to restore the correct behaviour and allow it to work with the class. (From OE-Core rev: 188ba30ef290efab6d5a9b95fba8e991b779ab33) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: add MD5 hash for the line-wrapped MPL-1.1 licenseRoss Burton2019-07-111-0/+1
| | | | | | | (From OE-Core rev: d2755523d339a4eba7a034bdef5563657609f563) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Build pkgdata specific to the current recipeRichard Purdie2019-06-301-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | This switches the code to build pkgdata specific to the current recipe which means that its filtered to the recipes dependencies and can perform better as we can drop the lockfile. It uses a similar method to the staging code to do this, using BB_TASKDEPDATA to construct a list of packagedata task output which this recipe should "see". The original pkgdata store is left unaltered so existing code works. The lock file was there to prevent files disappearing as they were read or as directories were listed. Since we have a copy of the data and only access output from completed tasks (as per their manifests), we can remove the lock. The lock was causing starvation issues on systems with parallelism. There was also a potential determinism problem as the current code could "see" data from recipes which it doesn't depend upon. [YOCTO #13412] (From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-091-11/+1
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan2019-05-081-1/+1
| | | | | | | | | | | | | | | | | | | | If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") (From OE-Core rev: 8e4b301573071ea04911b0df6533c25bc4596b93) Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logging: use warning instead warnChen Qi2018-07-261-2/+2
| | | | | | | | | | | | | | | | The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ (From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: fix port number parsing issueMing Liu2018-04-031-2/+4
| | | | | | | | | | | | | | | | | A flaw was found when I run: $ recipetool create "ssh://git@xxx.xxx:7999/xxx.git" the url turned out to be: "git://git@xxx.xxx/7999/xxx.git;protocol=ssh" after parsing, the port number was parsed as part of the path, this is definitely wrong and lead to fetching failures. This issue could be fixed in reformat_git_uri, by filtering out port numbers when formatting ":". (From OE-Core rev: 4290e04b69360b5e1da9f37166015e30f66cb335) Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: fix conflict between SRCREV and tagChang Rebecca Swee Fun2017-12-181-4/+4
| | | | | | | | | | | | | | | | | | | | | | If you specify 'tag=' for a git URL and passed to recipetool create, you will get into Bitbake expansion error shown below: ----- snip ----- $ devtool add --version 2.4.2 mbedtls "git://github.com/ARMmbed/mbedtls;tag=mbedtls-2.4.2" ... bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Conflicting revisions (abeccb9dbd7e19ae91ac50e1edd3803111c5f9b6 from SRCREV and mbedtls-2.4.2 from the url) found, please specify one valid value ----- snip ----- Assuming the tag is valid, we should get the tag commit hash and drop the usage of 'tag=' from SRC_URI. By using a commit hash corresponding to the tag will prevent bitbake from accessing remote repository in order to expand SRCPV. (From OE-Core rev: 53f8effa3eb07dc7035ff9933e7918318f242579) Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: show a warning for github archive URLsPaul Eggleton2017-11-111-0/+3
| | | | | | | | | | | | | | github archive URLs are not guaranteed to be stable [1] and thus we should show a warning if a user specifies one to recipetool create (or devtool add). [1] http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142519.html (From OE-Core rev: 7e84a777aa924a237b4e604120ebf8a4b3ba53b2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: ignore incidental kernel module sourcePaul Eggleton2017-11-111-2/+4
| | | | | | | | | | | | | | | | | If the source tree happens to contain a kernel module as an example, a test or under a "contrib" directory then we shouldn't be picking it up and making the determination that the entire thing is a kernel module. An example that triggered this is zstd, which ships a kernel module under contrib/linux-kernel: https://github.com/facebook/zstd (From OE-Core rev: c2b3154158d4bb0855daa56477393341139d4cf9) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: pass absolute source tree path to pluginsPaul Eggleton2017-11-111-2/+2
| | | | | | | | | | We shouldn't be passing a relative path to the plugins if that's what's been specified on the recipetool command line. (From OE-Core rev: 949067384c5166058ebc76f931cc492dad1db645) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard: set a preferred provider when adding a new recipe with devtoolJuan M Cruz Alcaraz2017-09-131-0/+3
| | | | | | | | | | | | | | | | | | | | A recipe added with "devtool add" requires to be able to take precedence on recipes previously defined with PREFERRED_PROVIDER. By adding the parameter "--provides" to "devtool add" it is possible to specify an element to be provided by the recipe. A devtool recipe can override a previous PREFERRED_PROVIDER using the layer configuration file in the workspace. E.g. devtool add my-libgl git@git://my-libgl-repository --provides virtual/libgl [YOCTO #10415] (From OE-Core rev: adeea2fe6895898a5e6006e798898f0f5dabd890) Signed-off-by: Juan M Cruz Alcaraz <juan.m.cruz.alcaraz@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: detect Eclipse licensesPaul Eggleton2017-08-311-1/+5
| | | | | | | | | Add detection of EPL 1.0 and EDL 1.0 license files. (From OE-Core rev: 41e7580991f8ad77a57eb7fd292e39f1583109f6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-21 13:40:12 +0000