| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
| |
The infrastructure changes highlighed an issue where the top level directory
mtime stamp isn't included in the tarballs. This isn't an issue the test is
meant to check for so ignore it.
(Bitbake rev: 5a1e5b8dc55d6bb4d93bac3492f8c43ff957e712)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A recipe variable handles its dependencies even on the "contains"
variables within the "inline Python expressions" like bb.utils.filter().
And it also handles those in the append operator correctly, but the
problem is that it does not so in the remove operator.
Fix it by adding the missing dependencies every time the remove
operator has been handled.
Also add a test case to check if the override operators handle
dependencies correctly.
(Bitbake rev: b90520eedb1dbc7f6a3928d089fe74fafb864eb5)
Signed-off-by: Insu Park <insu0.park@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The runtaskdep data in siginfo files was written out with full paths to
the bb files, matching bitbake's internal "unique key" ID for recipes/tasks.
When originally implemented this made sense.
Over time, the main use for the data in siginfo files has become to match
against other siginfo files to debug changes of hash calcuations. The
recipename data is not useful for this as the siginfo filenames use PN
instead which can often be derived from the recipe filename but not always.
It is time to throw away the 'tid' data format and switch over the use a
hybrid PN form which includes the multiconfig. That can be easily stripped
off in the find_siginfo code in oe-core.
The other purpose of having a sortable dependency ID is retained and the
multiconfig needs to be included to allow the taskhashes to be processed
and calculated correctly. PN is meant to be unique between recipes, only
one would ever be built so using PN in this location is fine.
The one risk of this change is there isn't any compatibility to the old
format. I'm not convinced we should spend time complicating the code
with it. This change will change the taskhashes everywhere so the only
mixing of old and new siginfo files will be either through hash equivalence
or through users using the tool against old and new info files manually
which will give some weird output but it should be clear they're in
different formats as there would be large paths from the old files not
present in the new ones.
We have options to add backwards compatibility if some issue is found
to need that.
(Bitbake rev: 637933e2e5a59228a8d17aae4160551cab5f2f61)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Update test_python_snippet_function_reference to not require the 'func'
flag, now that we know the real function will be returned for context
functions without the flag.
(Bitbake rev: 83f41281ec3d9b4327ffc8e2312e1fb8f53cbf02)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This test makes sure that '${@eval()}' calls the eval builtin, even if an 'eval' variable is defined in the metadata.
(Bitbake rev: e9150447738a48f772240874b3512b08e982b19b)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
This ensures that any change to the presence of builtins in inline
python execs will be noticed.
(Bitbake rev: ee22d3d51c60db2da97422b2be1e42239b7a2324)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
This is a test for an issue seen long ago, to avoid regressions, where a
reference to a def'd function in the metadata would return the string
value from the metadata rather than the function in inline python.
(Bitbake rev: 9f7cb22febd557817c164e25a93f5660e9c06358)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Set the maxsplit value to match the expected number of variables.
This also avoids an unnecessary split as the parameters are in the form 'key=value'
and the 'value' could contain the '=' character.
(Bitbake rev: 3b17a7ed9bf6cd2808946c2d9c3ed9961af11f19)
Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By default, pytest captures all stdout and exposes it using its built-in
fixtures (capsys, caplog etc), so stdout does not support getvalue().
To support running tests using both unittest and pytest, use assertLogs
to capture logging and assert on the log output instead.
(Bitbake rev: 2d28caa01bab9540d2bbaf713ae3e5c563d003f5)
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
git config returns an error code if user.email or user.name are not set,
so bb.process.run raises an ExecutionError, failing the test.
Improve the logic by catching the ExectionError and using this to set a
default value.
Change a direct invocation of 'git init' to use self.git_init so that
it follows the code path above.
Remove the related comment from the README now that git sets up the user
details automatically.
(Bitbake rev: 0c7079d0609a13713b890a9675abd9fba032f199)
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
TestCase.assertRaisesRegexp was renamed to assertRaisesRegex in Python
3.2, so rename to fix a warning during test execution.
(Bitbake rev: 9bc2ca3197bff7a5ef6e0ece6bfef0cda68b808e)
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
pytest test collection attempts to collect all classes containing
"Test". TestTimeout is not a test class so add the __test__ = False
attribute to indicate this.
(Bitbake rev: b5455b98093a3eaf122ebe3d29c3bef949fd2440)
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If you have a host where the git config for initial branch
name is something else than 'master' the unittest will fail
as they assume the default branch name is 'master'. Fix this
by explicitly set the intial branch name at 'git init'.
(Bitbake rev: 5bcd213c23da30a84baba88b775b1740e6bb77d0)
Signed-off-by: Peter Bergin <peter.bergin@windriver.com>
Signed-off-by: Peter Bergin <peter@berginkonsult.se>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The crate fetcher handles a crate with a name that matches the recipe's
name specially by placing the unpacked code in the current directory
(which typically is ${S}) rather than together with the sources for the
other crates. This broke when the URI names for all crates were changed
recently to include the version in the name.
Correct the crate fetcher to test against ${BP} instead of ${BPN}.
Also add a test case to the selftests to avoid this breaking again.
[Yocto #15012]
(Bitbake rev: 9af52b748357d0c843ce2507ce4d119fd9c37008)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
The recommendation from server maintainers is that the https protocol
is both faster and more reliable than the dedicated git protocol at this point.
Switch to it where possible.
(Bitbake rev: 879b830fcc60a022682ef22f58c04e7e07d03c63)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update regex pattern to allow variable flag name with a single character.
Regression tests have also been updated in `bb.parse` and
`bin/bitbake-selftest -k ParseTest` has been successfully executed.
Eliminate a trailing space as well.
(Bitbake rev: bb9e523291a3cad6e1596ee6a1e715b5e5feba8f)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is common for rust packages to depend on different versions of the
same crate. E.g.
| crate://crates.io/windows_x86_64_msvc/0.42.2 \
| crate://crates.io/windows_x86_64_msvc/0.48.0 \
Identification only by the plain crate name makes the sha256sum
ambiguous
| SRC_URI[windows_x86_64_msvc.sha256sum] = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
| SRC_URI[windows_x86_64_msvc.sha256sum] = "1a515f5799fe4961cb532f983ce2b23082366b898e52ffbce459c86f67c8378a"
and requires lot of manual work to fix the SRC_URI list.
Making the 'name' property unique by appending the version allows
direct copy & paste of reported sha256sum errors to complete the
crates list.
(Bitbake rev: ae2efb05196f9e29ef56ad9a84e2eae5fbdd8030)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change brings checksum verification of each crate
in a recipe, e.g
| SRC_URI += " \
| crate://crates.io/aho-corasick/0.7.20 \
| crate://crates.io/atomic-waker/1.1.0 \
| crate://crates.io/cc/1.0.79 \
| "
|
| SRC_URI[aho-corasick.sha256sum] = "cc936419f96fa211c1b9166887b38e5e40b19958e5b895be7c1f93adec7071ac"
| SRC_URI[atomic-waker.sha256sum] = "debc29dde2e69f9e47506b525f639ed42300fc014a3e007832592448fa8e4599"
| SRC_URI[cc.sha256sum] = "50d30906286121d95be3d479533b458f87493b30a4b5f79a607db8f5d11aa91f"
That will require to move the checksum initialization
after the possible call to urldata_init method in order
for the crate fetcher to parse the url.
Another way of doing could have been implementing a decodeurl
method that would have been specific for each fetcher class.
(Bitbake rev: 4920686c13dd66f9bfa4f7dd38d6e955f153eeec)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch enables the usage of the '@' character in variable flag
names.
One use case of variable flags is to assign the network namespaces of
some systemd services/targets to configure other parts of the build
process of some system. The filenames of systemd services/targets might
contain the '@' character if they are template unit files that can take
in a single parameter/argument and be instanced multiple times, as
indicated by systemd's official manual page.
The '@' character is disallowed as the first character in a variable
flag name. Imposing more restrictions on the first character is a
compromise to make parsing easier and to allow for more options in the
future to extend the syntax.
This patch is successfully verified by creating a custom BitBake recipe
that sets and unsets the value of a variable flag with the '@' character
in its name and ensuring that no ParseError is being thrown. Regression
tests have also been added to `bb.parse`.
`bin/bitbake-selftest` has also been successfully executed and all tests
passed.
(Bitbake rev: 00f9ab2cacfbd2a63b6b4959cf5401babae7e32a)
Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
For readability of following patches, rename this internal variable to
allow for others in a similar format.
(Bitbake rev: 450b4a00895ed1f9396a8ff859dc1cc0eccc838f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This allow to have classic fetch parameters
(like destsuffix, sha256, name ...) not being
considered by crate fetcher itself (and so mess
up its download)
Moreover, it allow to overload the name of the downloaded
crate (maybe usefull if there is a naming clash between
two crates coming from different repositories)
(Bitbake rev: 278bd2f1758b8af97552af8d23d16ffb5127a131)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
>From the npm-install documentation [1] the CLI provides a set of
short forms when the install fetches from git. These include
"github:"
example: npm install github:mygithubuser/myproject
"gist:"
example: npm install gist:101a11beef
"gitlab:"
example: npm install gitlab:mygitlabuser/myproject
"bitbucket:"
example: npm install bitbucket:mybitbucketuser/myproject
Commit 1d8af6aed0a9 [fetch2: npmsw: Add support for github prefix in
npm shrinkwrap version] by Stefan Herbrechtsmeier added support for
the "github:" but the others would marked as 'Unsupported dependency'.
The other prefixes are added in this commit, along with extending the
tests to cover some of these.
However, there is one more short form for github which npm-install
allows which forgoes the prefix altogether.
example: npm install mygithubuser/myproject
Unfortunately this format is a bit problematic as it lacks any easily
identifiable 'marker' to match against, and it could be either the
github short form or install from folder format. Experimentation shows
that the folder format requires a leading './' or '/', so we use this
to rule out the ambiguity.
If this approach to folder and github formats disambiguation is
incorrect it won't matter anyways as the folder format is unrecognized
by the code as-is and thus with this change or without, things would
fail.
Since we have to be less strict in the check for git operations we
move it to be the last install format which we check, such that the
less ambiguous formats can be sorted out first.
[1] https://docs.npmjs.com/cli/v9/commands/npm-install
[Yocto #14236]
(Bitbake rev: 0ac6f6cb5d807919ed13a8b7bb3fb551b79c5a71)
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added tests that verify that git-lfs works with an actual
real git-lfs server. This was not previously the case because
the repo in the test was a simulation of git-lfs but not
a real git lfs repo.
The 2 added tests are almost the same but test that the
git lfs file checkout is successfult with or without the
lfs=1 flag. The lfs=1 URI parameter is a quirk that triggers
2 different code paths for git lfs.
lfs=1, when used on git lfs repositories triggers the git lfs
downloading at the fetch bare stage.
lfs query parameter unset triggers the git lfs downloading only
on checkout as an implicit behavior of git. This leads to possible
network access on the unpack stage and outside the DL_DIR.
lfs=0 actually disables git-lfs functionality even if supported.
(Bitbake rev: d2be7f7f652360f13cd66d0850f3e19ffe2afb0a)
Signed-off-by: Paulo Neves <paulo@myneves.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Not restoring the mocked _find_git_lfs leads to other tests
failing.
(Bitbake rev: 70f848631450bd723c223227c21c60e815ee033d)
Signed-off-by: Paulo Neves <paulo@myneves.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
optimisation
Currently the codeparser cache ends up being extended for every parse run
since there are values in the functions such as the result of os.getpid()
from LOGFIFO in OE-Core.
Digging into that issue, there are also lots of similar but different
functions being parsed where the change might just be a path to WORKDIR,
a change in PN or PV or something like DATE/TIME.
There is no reason we have to use these changing values when computing the
dependenies of the functions. Even with a small tweak like:
BB_HASH_CODEPARSER_VALS = "LOGFIFO=/ T=/ WORKDIR=/ DATE=1234 TIME=1234 PV=0.0-1 PN=nopn"
the cache is reduced from ~4.6MB, increasing by ~300kb for every parse run
to around 1.3MB and remaining static for oe-core and meta-oe. In my local
build, admittedly heavily experimented with, the cache had grown to 120MB.
The benefits of doing this are:
* faster load time for bitbake since the cache is smaller to read from disk
and load into memory
* being able to skip saving the cache upon shutdown
* lower memory footprint for bitbake
* faster codeparser data lookups (since there is less data to search)
We only use these special values when passing code fragments to the codeparser
to parse so the real variable values should otherwise be used in the hash data.
The overall effect of this change, combined with others to avoid saving unchanged
cache files can be ~2s on a ~16s parse on my local system and results in a more
responsive feeling bitbake. It also allows parsing performance to be investigated
more consistently.
(Bitbake rev: f24bbaaddb36f479a59a958e7fc90ef454c19473)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
While the parameter is not required it allows testing of possible
regression in fetcher code when parameter specified and mirrors are
used.
(Bitbake rev: 1a21918049091a6d77426dbf8868ffdc14ba1003)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Basic test for mercurial functionality including PREMIRRORONLY with
disabled network
(Bitbake rev: c0e349a5fc92e9c90afc53800c7e83495393ff79)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the move to a server idle thread, we always need threading. The
existing accessor functions could end up turning this off!
I was going to hold the lock whilst changing it, check if the value
was already set, cache the result and also fix the event code to always
release the lock with a try/finally.
Instead, disable the existing functions and use a with: block
to handle the lock, keeping things much simpler.
(Bitbake rev: 645c9d3b50e55f69b222cc338373cdfd91d524ce)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I've been seeing event handlers where 'd' seems to disappear half way through
event handler execution. This is problematic when multiple threads are active
since this code assumes single threading.
The easiest fix is to change the handler function calls to contain d as a
parameter as we do elsewhere for other functions. This will break any non-text
handlers but I was only able to spot one of those in runqueue. It will also
break handlers than call functions that assume 'd' is in the global namespace
but those failures should be obvious and we can fix those to pass d around.
This solution avoids manipulating builtins which was always a horrible thing
to do anyway and solves the issue without needing locking, thankfully.
(Bitbake rev: 1e12f0a4b592dacd006d370ec29cd71d2a44312e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moving code into python modules is a very effective way to reduce parsing
time and overhead in recipes. The downside has always been that any
dependency information on which variables those functions access is lost
and the hashes can therefore become less reliable.
This patch adds parsing of the imported module functions and that dependency
information is them injected back into the hash dependency information.
Intermodule function references are resolved to the full function
call names in our module namespace to ensure interfunction dependencies
are correctly handled too.
(Bitbake rev: 605c478ce14cdc3c02d6ef6d57146a76d436a83c)
(Bitbake rev: 91441e157e495b02db44e19e836afad366ee8924)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
All siggens in common use should now support multiconfig, drop the
compatibility code.
(Bitbake rev: b36545b4df6d935ed312ff407d4e0474c3ed8d1a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When obtaining latest upstream versions, the code needs
to check if the existing tarball is in a versioned directory
(e.g. component-name/x.y/component-name-x.y.z.tar.gz) and
if it is, it needs to first obtain the list of all
such versioned directories and then check all of them by going
one step up in the directory hierarchy.
Existing code was returning a correct match when the component
name did not have numbers, e.g. a check on 'source/epiphany/43/'
would return 43, but was stopping too soon when the component
name itself had numbers ('source/libxml2/2.10/' would return libxml2).
This change ensures the last match is taken instead of the first.
Also, adjust the fetcher tests to check that versioned directories
are correctly traversed in this case (e.g. the step to go one level
up is taken and a new tarball is discovered in a different versioned
directory).
(Bitbake rev: b6601be22c6d776327acdcd1fa931400f41ac786)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[YOCTO #14917]
Attempting to use a dictionary in a python code snippet for variable
assignment results in an error. For example attempting something such
as
IDX = "green"
VAL = "${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]}"
produces the error
expansion of VAL threw ExpansionError: Failure expanding variable
VAL, expression was ${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]}
which triggered exception SyntaxError: '{' was never closed (Var
<VAL>, line 1)
The existing __expand_python_regexp__, "\${@.+?}", will match the
first close curly bracket encountered, resulting in incomplete and
un-parsable code, and thus produce the error. We can correct this by
allowing a single depth of nested curly brackets in
__expand_python_regexp__ by using "\${@(?:{.*?}|.)+?}", which will
match up to and including the matching close curly bracket to the
open, '${@', curly bracket, even if there are one or more singly
nested curly brackets present. This change allows the usecase
described above to function.
This change can't be made on its own though. The old regex would, in
an obscure way, handle the case where a python snippet contained an
unexpandable variable. Since the unexpandable variable is in curly
brackets it would cause incomplete/un-parsable python code and thus
remain unparsed. So something like
VAL = "${@d.getVar('foo') + ${unsetvar}}"
would remain unparsed as the close curly bracket in "${unsetvar}"
would match and terminate the snippet prematurely. This quirk resulted
in the proper handling of python snippets with unexpanded
variables. With the change to __expand_python_regexp__ the full
snippet will match and be parsed, but to match the old/correct
behavior we would not want to parse it until ${unsetvar} can be
expanded. To ensure the old/correct behavior for python snippets with
unexpanded variables remains in place we add a check for unexpanded
variables in the python snippets before running them.
This handling of unparsed variables brings two benefits. The first we
now have an explicit check visible to all for unexpanded variables
instead of a somewhat hidden behavior. The second is that if there are
multiple python snippets the old behavior would run the code for each
but a single snippet with unexpanded variables would mean all snippets
would remain unparsed, meaning more and repeated processing at a later
time.
For example:
"${@2*2},${@d.getVar('foo') ${unsetvar}}"
old behavior would give:
"${@2*2},${@d.getVar('foo') ${unsetvar}}"
new behavior will give:
"4,${@d.getVar('foo') ${unsetvar}}"
The old behavior would calculate '2*2' but toss the result when the
second snippet would fail to parse resulting in future recalculations
(or fetching from cache), while the new behavior avoids this.
(Bitbake rev: 94e49b9b9e409c29eb04603b1305d96ebe661a4b)
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
CVE-2022-39253 in git meant file:// urls within submodules were disabled. Add
a parameter to the commands in the tests to allow this to continue to work.
(Bitbake rev: 209f7ba352b60722830157054e3fc56cb9c693eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a test for special characters in user and password to qualify
decodeurl() inspired by a bug report describing that '=' signs in a
password was problematic.
Add a second test to qualify decodeurl() as related to the change in
commit 628c4bf6c89b [fetch2/__init__: handle @ in package names].
Relates to [YOCTO #14476]
(Bitbake rev: ee04cf09c7022168c035affa654773652a49793e)
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
| |
(Bitbake rev: b0506480baa9bcf3ef645b0aed5a07ad9950245c)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The class and exception aim to test rare cases there deadlocks are
possible.
Can be used in context managers:
with Timeout(<value>):
do_deadlock()
(Bitbake rev: c5fcdd804d422f959a189b270d72123a50e74da6)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We have some confusion for users since some classes are meant to work
in the configuration space (or "globally") and some are meant to be
selected by recipes individually.
The cleanest way I could find to clarify this is to create "classes-global"
and "classes-recipe" directories which contain the approproate classes and
have bitbake switch scope between them at the appropriate point during
parsing. The existing "classes" directory is always searched as a fallback.
Once a class is moved to a specific directory, it will no longer be found
in the incorrect context. A good example from OE is that
INHERIT += "testimage"
will no longer work but
IMAGE_CLASSES += "testimage"
will, which makes the global scope cleaner by only including it where it
is useful and intended to be used (images).
(Bitbake rev: f33ce7e742f46635658c400b82558cf822690b5e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Where copyright headers were not present, add them to make things
clear.
(Bitbake rev: 1aa338a216350a2751fff52f866039343e9ac013)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A long time ago, we made DL_DIR a final fallback for the local fetcher.
Since then we added checksum support and task hashes and the world
has changed. There were warnings added some time ago if this fallback
triggers and it is now time to drop it entirely.
The original use case was for sstate however the sstate code now sets
FILESPATH correctly so DL_DIR is no longer needed.
There have been a few small bugs exposed by this change, missing mkdir
calls and some minor test issues that needed tweaks. In general this
simplifies and improves the fetcher code flow though.
This completes a cleanup that ensures local files are correctly covered
at parse time which ensures rebuilds and reparses happen at the right
times.
(Bitbake rev: 3e1444e536c71d3885ef6b9d428807163c309640)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the local fetcher was not able to find the file anywhere but it
was included in the SRC_URI for checksumming just make it a fatal
error. Ensure a list of searched locations is included too to match
the runtime error that would have resulted.
(Bitbake rev: 5e3b2ad90d9cd0f248b1cb740637caa24442d101)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently if you trigger one of the comment errors, the newline characters
are stripped and the line numbers are incorrect. In one case it prints
the empty line which is also unhelpful.
Rework the code around these errors so the line numbers are correct
and the lines in question are more clearly displayed complete with newlines
so the user can more clearly see the error.
I also added a couple of simplistic test cases to ensure that errors
are raised by the two known comment format errors.
[YOCTO #11904]
(Bitbake rev: 712da71b24445c814d79a206ce26188def8fce0a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With PREMIRRORS set and BB_NO_NETWORK = "1" bitbake should not try to
fetch into non-initialized git directory if tarball is broken (or not in
gzip format)
[Yocto 14822]
(Bitbake rev: c9aaca3dd2dfdf4a291d6e1f6263037e0f54b4b6)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Existing test uses fake local repo for better performance. Adding
test which uses real (but still small) project hosted on
yoctoproject.org.
(Bitbake rev: 247f3536a691cdaa3f96768d1b42653b1da9ae84)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Function "git" is defined in super class FetcherTest already.
(Bitbake rev: bfed4ad2d50bcd635bb2ef99977e03e879fc79a2)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Basic test to cover functionality of BB_FETCH_PREMIRRORONLY using
local git repository. Local repository has been chosen to allow easy
manipulation with the repo to simulate behaviour reported in
[Yocto 13233]
(Bitbake rev: 773e0815ba0c2183afcb169cda525b7625e60e42)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Fix an issue where two tests have the same name with one overwriting the
other.
(Bitbake rev: da812d938fd79e2cc7bdf355ccf5b0f9ead684c4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change the owner information in the mirror tarballs generated using
BB_GENERATE_MIRROR_TARBALLS="1". This is an extension of commit
0178ab83, which used the original pokybuild:user information, but failed
to clean up the numerical user and group ids. Now set the more canonical
values of oe:oe and 0:0.
(Bitbake rev: 37437115d3fb1a9f5d8ed7356a0fc01a408e4f8c)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
CC: Marek Vasut <marex@denx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
contains items
Adding bb.utils.filter('WARN_QA', 'patch-fuzz', d) when WARN_QA is in
BB_BASEHASH_IGNORE_VARS or in vardepsexclude should not add a dependency
on WARN_QA.
Fix it and add some tests.
(Bitbake rev: 6aecc2fe51a52020f6f13be08449e18d42e7a6b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using BB_GENERATE_MIRROR_TARBALLS="1" to generate mirror tarballs
of git repositories, they leaked local information: username, group and
time of the last fetch. Remove all these by setting fixed information:
* uname = pokybuild
* gname = users
* mtime = committer time of newest commit in repo
The username and group value were taken from the archives available on
the downloads.yoctoproject.org mirror. The modification time is chosen
so it still retains some relationship to the contents of the archive.
(Bitbake rev: 0178ab83e6312e97e528aa8c5e12105f5165d896)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|