summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/utils
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/utils/command: simplify tap detectionAdrian Freihofer7 days1-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplify the code by removing the fallback to ifconfig if the ip command is not available. ip commands are nowadays available on all host machines. The transition from ifconfig to ip has taken place long time ago e.g. for the runqemu-gen-tapdevs script. This also fixes the detection of tap devices if the tap devices are not named tap0, tap1, etc. but have a different name, e.g. foo0, foo1 which is the case if the OE_TAP_NAME environment variable is set. Some examples: $ ip tuntap show mode tap $ sudo ./scripts/runqemu-gen-tapdevs 1000 2 Creating 2 tap devices for GID: 1000... Creating tap0 Creating tap1 ... $ ip tuntap show mode tap tap0: tap persist group 1000 tap1: tap persist group 1000 $ sudo ./scripts/runqemu-gen-tapdevs 1000 0 Note: Destroying pre-existing tap interface tap0... Note: Destroying pre-existing tap interface tap1... $ ip tuntap show mode tap $ sudo OE_TAP_NAME=foo ./scripts/runqemu-gen-tapdevs 1000 2 Creating 2 tap devices for GID: 1000... Creating foo0 Creating foo1 ... $ ip tuntap show mode tap foo0: tap persist group 1000 foo1: tap persist group 1000 $ sudo OE_TAP_NAME=foo ./scripts/runqemu-gen-tapdevs 1000 0 Note: Destroying pre-existing tap interface foo0... Note: Destroying pre-existing tap interface foo1... $ ip tuntap show mode tap (From OE-Core rev: 6459ea7c019bcb7a486d286dd964eeeeab99c37d) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: devtool: split tap detection into functionAdrian Freihofer7 days1-0/+16
| | | | | | | | | | Make the check for tap devices available as a function which can be used by other tests as well. (From OE-Core rev: ad8f3a8d959a245301118cf7b850f1a0ab567f01) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/command: fast-path get_bb_var()Ross Burton2025-06-201-1/+14
| | | | | | | | | | | | | | | get_bb_var() currently end up calling 'bitbake -e' and parsing the whole output. However if postconfig isn't set then we can speed this up by just calling bitbake-getvar. The complication with failing bitbake-getvar calls is because we need to be careful to return None instead of the empty string when the variable doesn't exist. (From OE-Core rev: fafe77879aa6225aa8b5187ff590bb4998cbf987) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/utils/sshcontrol: correct condition for ending the select() loopAlexander Kanavin2025-06-091-1/+1
| | | | | | | | | | | | This was set backwards; per https://docs.python.org/3/library/subprocess.html#subprocess.Popen.returncode a return code of None indicates the process is still running, and so the code entered a busyloop that ended on timeout 5 minutes later, lengthening selftests significantly. (From OE-Core rev: a6690deffd7ddbce0e784701ea3fdbb84313b009) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/subprocesstweak: clean up __str__()Ross Burton2025-06-051-9/+4
| | | | | | | | | | Call super().__str__ to get the bulk of the string representation, and we don't need to guard on output/strerr existing as they always set. (From OE-Core rev: 2adcac16dd26fd054ea779cc4e7aa32282d9bdde) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Fix a bug with fork/exit handlingRichard Purdie2025-03-031-6/+9
| | | | | | | | | | | | | If you send this forked process a SIGTERM, it will execute all of the parent's exit code leading to two sets of console/exit output which is extremely confusing. Wrap the code in a try/finally to ensure we always call os._exit() to avoid this. I spent far too long trying to work out the crazy console output from this. (From OE-Core rev: 652e40bfae24b8e23bbf7a7f35d900d2ab8d0f92) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Convert from ifconfig to use ipRichard Purdie2025-02-201-1/+1
| | | | | | | | ifconfig is obsolete and being removed, convert to use ip instead. (From OE-Core rev: 3b2fb477750606976359884b18c33a37832e5a78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/metadata: Add commit_time to branch metadata being savedRichard Purdie2025-02-091-0/+5
| | | | | | | | | | | | As well as commit counts, it is helpful to know when metadata dates from. Store the unix timestamp for commits in a commit_time field alongside the commit count. This is useful for performance graph analysis and saves having to recompute the data. (From OE-Core rev: 56d1bc3f8f45d2f9c8ca0319c429cec562a16384) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sshcontrol: Handle empty readsRichard Purdie2025-01-251-2/+4
| | | | | | | | | | | | | | Looking at some of the autobuilder failures, it seems that somehow empty reads might be possible despite not being EOF. Tweak the code to be a little more robust in handling this. In theory this shouldn't be possible but python does handle signals a bit differently (e.g. transparrently retrying syscalls for EINTR) so adding this check and a bit of code safety at least rules out this problem. (From OE-Core rev: 9d91a5674c515a43ae76d8615f72e5e2dc16c961) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rpm: replace use of rpm2cpio with rpm2archiveAlexander Kanavin2024-12-121-5/+5
| | | | | | | | | | | | | | rpm2cpio has been deprecated upstream, so this prepares for its eventual removal. rpm2archive produces a tar archive which can be uncompressed with tar executable from the host. (From OE-Core rev: ed824d3fb23f0c89d8dfdacb2c4ef0b7c21a5144) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Fix stack trace generation in exception handlerMathieu Dubois-Briand2024-12-091-2/+4
| | | | | | | | | | | | | | | | | | Qemurunner exception handling code currently formats the stack trace using traceback.format_exception(), with parameters introduced in python 3.10. This will fail on platforms with an older python version. Change this to the old parameter order, still supported in current python versions. https://docs.python.org/3/library/traceback.html#traceback.format_exception Fixes [YOCTO #15675] (From OE-Core rev: 5f9ecf5f210e967594069f172728fd5b4d5b4f1d) Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/gitarchive: Fix syntax warningRichard Purdie2024-12-051-1/+1
| | | | | | | | | | | The backslash characters cause syntax warnings, mark the strings are raw to avoid this. (From OE-Core rev: f717f61a37ed83618d054fc4017b5f5386fb2e3c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: Return tag name and improve exclude handlingRichard Purdie2024-11-231-1/+3
| | | | | | | | | | | Tweak the gitarchive exclude handling not to error if excluded files don't match. Also return the tagname created so that other code can then use it. (From OE-Core rev: 1adba3430faffdf6217b6a00533a3b48a9388abc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu/testimage: Ensure GNU tar is in test imagesRichard Purdie2024-11-211-1/+2
| | | | | | | | | | | | | | | | We've seen cases where a ptest (strace) has a 43GB sparse file in the test directory. busybox tar doesn't work well with this. The resulting 1.4GB archive takes hours to extract too. Ensure tar is added to our full images and use the sparse option to collect files for debugging. This stops crazy build hangs. Since tar is GPLv3, we have to exclude it from that test code. We don't boot any of those images so the debug collection code is safe there, at least for now. (From OE-Core rev: fefeb919696b6ac76f0997acfb0f612203ef7f1b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runCmd: print stderr when that is a separate streamAlexander Kanavin2024-11-011-0/+2
| | | | | | | | | | | | | runCmd by default merges stderr into stdout, and only needs to print stdout when errors occur. When stderr is requested as a separate stream, and an error occurs, stderr is discarded, obscuring useful error messages. This changes the output to include both streams. (From OE-Core rev: 72c747b37ccdd486ddae06e3d0a99fb2b93643ba) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Clean up serial_lock handlingRichard Purdie2024-10-141-5/+6
| | | | | | | | | | Avoid "RuntimeError: release unlocked lock" since the lock shouldn't be locked even in the error path. Add a try/finally path to ensure this. (From OE-Core rev: b0732ee009ca47580d1d2ad75334f4aa50e6efd5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: support passing custom boot patterns to runqemuEnrico Jorns2024-10-112-3/+4
| | | | | | | | | | | | | This allows defining non-standard patterns from QEMU tests just as they are already supported by the testimage.bbclass. Will allow testing non-Linux shells in QEMU, too (e.g. a U-Boot shell or another bootloader shell). (From OE-Core rev: f9e6c3deee5a4159a09396b625a7327f6c204df5) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: support ignoring vt100 escape sequencesEnrico Jorns2024-10-111-1/+3
| | | | | | | | | | | | | | | | | | | | If we talk to terminals that like colors, we need to ignore the vt100 escape sequences when matching strings. An unprocessed barebox console prompt would e.g. look like: ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m where we cannot match for something like "barebox@ARM QEMU virt64:/". The same applies to colored Linux terminal output of course. The "\x1b\[" from the regex catches the standard start of ANSI escape sequence while the rest catches the actual command code executed. (From OE-Core rev: 33bbe4cb040f890121681865fbcf28bc8213a170) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: Fix archive retrieval from targetMathieu Dubois-Briand2024-09-201-0/+1
| | | | | | | | | | | | | A previous patch broke archive retrieval because of an undeclared variable. Declare the archive_name variable as expected. (From OE-Core rev: 39d78fa53fb98515af6dc72c60017e5335993372) Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org> CC: Alexis Lothoré <alexis.lothore@bootlin.com> CC: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: do not uncompress retrieved archive on hostAlexis Lothoré2024-08-131-1/+2
| | | | | | | | | | | | | | | | | | | | | | | Current postaction module executes a remote tar command, pipe it in a SSH connection, and uncompress the raw stream with another tar command. With this command, the whole artifacts tree is directly available on the host executing the test, but it is not very convenient if we want to download the whole retrieved ptests directory. Stop uncompressing the retrieved ptests archive onto host, just save the archive as it is. The new output then looks like the following: tmp/log/oeqa-artefacts └── oeqa-target-artefacts-20240812-juzqdb80 ├── host_disk_usage.txt ├── target_disk_usage.txt └── tests_artifacts.tar.gz Suggested-By: Alexandre Belloni <alexandre.belloni@bootlin.com> (From OE-Core rev: f90894d996c8a8f980e46c87b7968b176793b3fe) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: fix exception handlingAlexis Lothoré2024-08-131-1/+1
| | | | | | | | | | | | | | | The current exception handler in list_and_fetch_failed_tests_artifacts expects a non-exisiting variable and then fail to display the original exception message since it raises a new one. The issue has been introduced with commit 6e80b2ab660e ("oeqa/utils/postactions: transfer whole archive over ssh instead of doing individual copies"). Now that tests artifacts are now handled individually, there's no point of trying to print individual names in the exception. (From OE-Core rev: a50e72bb64fb8b0d14c23164eaeeabd9c271ac19) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: Separate artefact collection from test result collectionRichard Purdie2024-07-182-29/+32
| | | | | | | | | | | | | | | Combining the test result collection and artefact collection hasn't worked out well as the data has different life cycles, the artefacts can be large and we need to be able to clean them up on a different timescale. Separate them out to be controlled by a separate variable, OEQA_ARTEFACT_DIR. Also rework the code to inject a directory with a date/time and random component to allow builds to run in parallel. Pass function arguments to avoid re-reading variables. (From OE-Core rev: e1cf7e94c3fcbe7dbc29e4286f0e1014b95964a9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/postactions: Allow artifact collection to be skippedRichard Purdie2024-07-101-0/+5
| | | | | | | | | | | | | It does not always make sense to collect artifacts and data from the target on failure, e.g. if testing firmware or if the target is not running an SSH server. Allow this by setting TESTIMAGE_FAILED_QA_ARTIFACTS to an empty value. (From OE-Core rev: 7036f814baa6550e0b1ac2db37fcd51960ee9e48) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: transfer whole archive over ssh instead of doing ↵Alexis Lothoré2024-07-081-10/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | individual copies Fixes [YOCTO 15536] The postactions retrieval actions currently rely on scp executed individually on any file or directory expanded from TESTIMAGE_FAILED_QA_ARTIFACTS. Unfortunately, symlinks are not preserved with this mechanism, which lead to big storage space consumption. Things may go even worse if those symlinks create some circular chains. This mechanism then needs to be updated to preserve symlinks instead of following them during copy. There are multiple ways to do it: - create a local archive on the target and execute scp on this file - use rsync instead of scp for all files - create an archive and pipe it to ssh instead of storing it onto the target The first solution may create pressure on targets storage space, while the second assumes that rsync is installed on the target, which may not be true. So the third one is a compromise: tar is very likely present, at least through busybox, and no disk space is used on the target. Replace the current per-file scp call by a single call to tar run on the target. Retrieve the generated compressed archive directly from SSH output, and feed it to another tar process but on host, to uncompress and extract it at the same place as before. (From OE-Core rev: 4aeb10aa38efc6768928fbb74985e36e972b8e46) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: extend docstring for runqemu context managerEnrico Jorns2024-06-061-1/+17
| | | | | | | | | | Usage and arguments are not necessarily self-explaining, thus add a bit of documentation here. (From OE-Core rev: 62343333d50b75ad04a4220bcf0990e0a89996a0) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/postactions: Do not use -l option with dfKhem Raj2024-04-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | -l option is specific to df provided by coreutils, if df applet from busybox is used then it does not work and fails like below Fixes df: invalid option -- 'l' BusyBox v1.36.1 () multi-call binary. Usage: df [-PkmhT] [-t TYPE] [FILESYSTEM]... DEBUG: [Command returned '1' after 0.71 seconds] DEBUG: Command: df -hl Status: 1 Output: df: invalid option -- 'l' it seems worth a compromise to show remote mounted filesystems if any during ptests and it works with both df implementations (From OE-Core rev: 75cc1ea4348a2294fdc5ab20530fcff27056ff06) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: Fix error on calls to run_monitorEilís 'pidge' Ní Fhlannagáin2024-03-071-2/+2
| | | | | | | | | | | | | | | | | | | A change in QEMUMonitorProtocol.cmd() requires that we either pass in kwargs instead of an argument dict or move to cmd_raw() cmd() was renamed to cmd_raw() (and command() was renamed to cmd()) See: https://github.com/qemu/qemu/commit/37274707f6f3868fae7e0055d9a703006fc142d0 https://github.com/qemu/qemu/commit/684750ab4f8a3ad69512b71532408be3ac2547d4 My concern with this patch is that I haven't seen this come up with utils/dump.py which also uses QemuMonitor's run_monitor. If it is occuring, this should fix issues there as well (From OE-Core rev: 9665d38ab60c1c3b27887c2b1a6396f13a1b33ea) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/lib/utils/postactions: fix host disk usage stats retrievalAlexis Lothoré2024-02-291-1/+1
| | | | | | | | | | | | | | | | | | The recently introduced postactions module can raise, on failing ptests, the following warning: WARNING: core-image-ptest-glib-2.0-1.0-r0 do_testimage: Can not get host disk usage: [Errno 2] No such file or directory: '/usr/bin/df' The issue is likely not happening because of df absence (to be confirmed amongst the variety of workers) but because of the wrong path. Fix it by letting subprocess search for df, passing only the binary name. To make it work, we also have to reset the environment, otherwise the environment configured before running bitbake will be used, and search will fail. (From OE-Core rev: da7cc5def2839a0e15d07244f858847479c12caa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: testimage: add host disk usage stat as post actionAlexis Lothoré2024-02-271-1/+12
| | | | | | | | | | | | | | Since the target under test can be a virtualized guest, when some tests fail because of disk usage (see [1]), also fetch disk usage statistics from host to allow checking whether a host disk space saturation could affect running tests. [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220 (From OE-Core rev: 2ab3a0935b1e7a016402f149da1fc01b38d7af55) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: add target disk usage stat as post actionAlexis Lothoré2024-02-271-1/+16
| | | | | | | | | | | | | In order to debug issues related to disk space (see [1]), add a failed tests post action to retrieve disk usage on the target. Rely on the test context object to run the corresponding command onto the target [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220 (From OE-Core rev: 637e216b0e5191571270aa07e1f50a6e41a8c08f) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/postactions: isolate directory creation in dedicated actionAlexis Lothoré2024-02-271-6/+10
| | | | | | | | | | | | | | | In order to be able to create actions that could store new files during failed test post actions, we need to split artifacts directory creation from artifacts retrieval. Create a new dedicated action to create artifacts main directory so we can add actions creating files in this new directory, without worrying about actions order if at least this action is set first. (From OE-Core rev: 5d796586a9342f4f984494a5b493dbaf77af7026) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: create a list of failed test post actionsAlexis Lothoré2024-02-271-0/+68
| | | | | | | | | | | | | | | | | | | | | testimage is able to detect whenever a test run leads to some tests failing, and execute some actions in this case. The only action currently defined in such case is to retrieve artifacts from the target under test, as listed in TESTIMAGE_FAILED_QA_ARTIFACTS In order to be able to add multiple actions, define a central function to gather all "post actions" to run whenever a test has failed (run_failed_tests_post_actions). This function contains a table listing all functions to be called whenever a test fails. Any function in this table will be provided with bitbake internal data dictionary ("d") and the current runtime testing context ("tc"). Isolate all this feature in a dedicated postactions.py file inherited by testimage. This patch does not bring any functional change. (From OE-Core rev: c01aa8df0613a103859b4431d3cc5056b2fef1b8) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa: share get_json_result_dir helperAlexis Lothoré2024-02-271-0/+7
| | | | | | | | | | | | | Multiple places in oeqa need to get the log output path, and redefine a small helper to accomplish this Define this helper in lib/oeqa/utils/__init__.py and import it wherever needed to allow using it. (From OE-Core rev: 01b1a6a5a4e7cede4d23a981b5144ae9c8306274) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Handle rare shutdown raceRichard Purdie2024-01-301-2/+6
| | | | | | | | | | | | | | The pid file can disappear when qemu is shutting down leading to a file not found race before it is read. Tweak the code to handle this and fix a rare but annoying race error case. [YOCTO #15036] (From OE-Core rev: 8c07aac9d55f92fe5fbe3cab9f006efecf266328) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: more cleanups for output blockingAlex Bennée2023-12-211-8/+12
| | | | | | | | | | | | | | | | | If we are only tracking stdout and are not using self.readsock we end up throwing an exception blocking further action from the thread. Fix this by checking self.readsock is not None first. While we are at it split even into fd, event to make things clearer and handle the fail path of stringify_event by echoing the hex value of the unknown flag. (From OE-Core rev: 5e58737c66090fe009ec49296f3e7d687eb05766) Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Mikko Rapeli <mikko.rapeli@linaro.org> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Improve handling of serial port output blockingRichard Purdie2023-12-201-29/+69
| | | | | | | | | | | Similar to stdout in the previous commit, we need to ensure serial output if written is read and put somewhere, else qemu might block on writes to the serial port leading to hangs in the kernel. Use our existing logging thread to log data when run_serial is not in use. (From OE-Core rev: 05761282ba31e4ba3594f7321e2162d01fe12a5f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Improve stdout logging handlingRichard Purdie2023-12-201-22/+32
| | | | | | | | | | | We need to ensure we read from and log the output from qemu stdout as otherwise the buffers can fill and block, leading qemu to hang. Use our existing logging thread to do this. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> (From OE-Core rev: a9c46ee014ef1e6436b39fdd4fd15d15388ea795) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: remove unused importRoss Burton2023-12-131-1/+0
| | | | | | | | | | | Removes unused logging import (From OE-Core rev: 1f21509a2fadb66888589e9946b34dddf5becc72) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Drop newlines serial workaroundRichard Purdie2023-10-151-12/+0
| | | | | | | | | Drop the newlines serial workaround, it doesn't seem to fix things enough of the time to be useful. (From OE-Core rev: 6cd57b2d148c5de7839d52d8cec359e4a10e0cd6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: ensure tag matches regex before getting its fieldsAlexis Lothoré2023-10-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Whenever we ask gitarchive to retrieve test results for specific revisions, we first do a "large" search in get_tags, which uses glob patterns with git ls-remote, and then we filter received tags with a regex to parse the tags fields. Currently gitarchive assumes that all tags returned by get_tags will match the regex. This assumption is wrong (for example searching "master-next" in get_tags may return some tags like "abelloni/master-next), and leads then to exception when we try to retrieve tags fields: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module> sys.exit(main()) ^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main ret = args.func(args, logger) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs fields, runs = get_test_runs(log, repo, tag_name, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs groups = m.groupdict() Fix this exception by merely skipping those additionals tags which won't match the regex (From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: fix tag pattern searchingAlexis Lothoré2023-10-111-1/+1
| | | | | | | | | | | | | | | | Whenever we ask gitarchive to search for tags, we can provide it with a pattern (containing glob patterns). However, when searching for example for tags matching branch master-next, it can find more tags which does not correspond exactly to branch master-next (e.g. abelloni/master-next tags will match). Prevent those additional tags from being fetched by gitarchive by using a more specific pattern: prefix user-provided pattern with "refs/tags" (From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Ensure we retry after BrokenPipeErrorRichard Purdie2023-10-111-1/+1
| | | | | | | | | | | | If the BrokenPipeError occurs when writing to the serial port to wake it up, defer the write and try again (which will happen on the 5s timeout of the select call). Why it should return ESHUTDOWN and then work later I'm not sure but it does appear to make it work. For now we need 'working' QA tests whilst the issue is debugged. (From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Add extra logging when console doesn't appearRichard Purdie2023-10-101-0/+10
| | | | | | | | | | | | | If the console doesn't appear, breifly sleep and try an extra read. This is a useful debugging trick which we may as well preserve in the code. We're already failing at this point so extra data is useful. This means if we perturb things on the other port, we may gain useful logging insight. (From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu/qemurunner: Use nodelay with tcp serial connectionsRichard Purdie2023-10-101-0/+1
| | | | | | | | | This disables Nagle's algorithm for our tcp serial connections which may be causing data transfer issues. (From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Add newlines serial workaroundRichard Purdie2023-10-101-0/+12
| | | | | | | | | | | We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes leading to failures in CI. Add a workaround of sending some newlines as a way of unblocking the kernel/release issues whilst we try and work out how to get to the bottom of the issue. (From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: detect login prompt without without utf-8 conversionMikko Rapeli2023-10-081-2/+1
| | | | | | | | | | | In case utf-8 conversion of the serial console data is flaky due to bad characters in the stream. (From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Show both the login console log and all logging upon failureRichard Purdie2023-10-041-3/+2
| | | | | | | | | It is unclear when things fail which output was on which serial port. Improve the output to show the last lines of both data to improve debugging. (From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Log the second serial console as well as the firstRichard Purdie2023-10-041-6/+3
| | | | | | | | | | To aid debugging, always log the second serial console as well as the first to a seperate log file. This should make it clearer what happened when we see test failures. (From OE-Core rev: 372cd58fe672726900af80f3e65a108984da2750) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Use backslashreplace with utf8 to make invalid characters clearRichard Purdie2023-10-041-2/+2
| | | | | | | | | Switch to the backslashreplace error handling when decoding strings so that invalid characters are clear in the stream to improve debugging. (From OE-Core rev: 8ec86d46f736ec3e625b741e97545377c79414ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Skip conversion from/to binary data for logfileRichard Purdie2023-10-041-4/+3
| | | | | | | | | | | | | There is no point in decoding binary data only to encode it again risking conversion issues. Write the raw data to the log file as binary and skip the conversion. Also always update self.msg even if a logfile isn't specified to improve logging/debug. (From OE-Core rev: b5d67471dddd198723c4f711747783a33e8e5987) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-22 00:18:06 +0000