devtool: new ide-sdk plugin

The new devtool ide plugin provides the eSDK and configures an IDE to
work with the eSDK. In doing so, bitbake should be used to generate the
IDE configuration and update the SDK, but it should no longer play a
role when working on the source code. The work on the source code should
take place exclusively with the IDE, which, for example, calls cmake
directly to compile the code and execute the unit tests from the IDE.

The plugin works for recipes inheriting the cmake or the meson bbclass.
Support for more programming languages and build tools may be added in
the future.

There are various IDEs that can be used for the development of embedded
Linux applications. Therefore, devtool ide-sdk, like devtool itself,
supports plugins to support IDEs.
VSCode is the default IDE for this first implementation. Additionally,
some generic helper scripts can be generated with --ide none instead of
a specific IDE configuration. This can be used for any IDE that
supports calling some scripts.

There are two different modes supported:

- devtool modify mode (default):
  devtool ide-sdk configures the IDE to manage the build-tool used by the
  recipe (e.g. cmake or meson). The workflow looks like:
    $ devtool modify a-recipe
    $ devtool ide-sdk a-recipe a-image
    $ code "$BUILDDIR/workspace/sources/a-recipe"
    Work in VSCode, after installing the proposed plugins

  Deploying the artifacts to the target device and running a remote
  debugging session is supported as well.
  This first implementation still calls bitbake and devtool to copy the
  binary artifacts to the target device. In contrast to compiling,
  installation and copying must be performed with the file rights of the
  target device. The pseudo tool must be used for this. Therefore
  bitbake -c install a-recipe && devtool deploy-target a-recipe
  are called by the IDE for the deployment. This might be improved later
  on.

  Executing the unit tests out of the IDE is supported via Qemu user if
  the build tool supports that. CMake (if cmake-qemu.bbclass is
  inherited) and Meson support Qemu usermode.

- Shared sysroots mode: bootstraps the eSDK with shared sysroots for
  all the recipes passed to devtool ide-sdk. This is basically a wrapper
  for bitbake meta-ide-support && bitbake build-sysroots. The workflow
  looks like:
    $ devtool ide-sdk --share-sysroots a-recipe another-recipe
    vscode where/the/sources/are
  If the IDE and the build tool support it, the IDE gets configured to
  offer the cross tool-chain provided by the eSDK. In case of VSCode and
  cmake a cmake-kit is generated. This offers to use the cross
  tool-chain from the UI of the IDE.

Many thanks to Enguerrand de Ribaucourt for testing and bug fixing.

(From OE-Core rev: 3f8af7a36589cd05fd07d16cbdd03d6b3dff1f82)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 438 insertions, 0 deletions

diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
new file mode 100644
index 0000000000..b2193130d2
--- /dev/null
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -0,0 +1,438 @@
+#
+# Copyright (C) 2023-2024 Siemens AG
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+"""Devtool ide-sdk IDE plugin for VSCode and VSCodium"""
+
+import json
+import logging
+import os
+import shutil
+from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, get_devtool_deploy_opts
+
+logger = logging.getLogger('devtool')
+
+
+class GdbCrossConfigVSCode(GdbCrossConfig):
+    def __init__(self, image_recipe, modified_recipe, binary):
+        super().__init__(image_recipe, modified_recipe, binary, False)
+
+    def initialize(self):
+        self._gen_gdbserver_start_script()
+
+
+class IdeVSCode(IdeBase):
+    """Manage IDE configurations for VSCode
+
+    Modified recipe mode:
+    - cmake: use the cmake-preset generated by devtool ide-sdk
+    - meson: meson is called via a wrapper script generated by devtool ide-sdk
+
+    Shared sysroot mode:
+    In shared sysroot mode, the cross tool-chain is exported to the user's global configuration.
+    A workspace cannot be created because there is no recipe that defines how a workspace could
+    be set up.
+    - cmake: adds a cmake-kit to .local/share/CMakeTools/cmake-tools-kits.json
+             The cmake-kit uses the environment script and the tool-chain file
+             generated by meta-ide-support.
+    - meson: Meson needs manual workspace configuration.
+    """
+
+    @classmethod
+    def ide_plugin_priority(cls):
+        """If --ide is not passed this is the default plugin"""
+        if shutil.which('code'):
+            return 100
+        return 0
+
+    def setup_shared_sysroots(self, shared_env):
+        """Expose the toolchain of the shared sysroots SDK"""
+        datadir = shared_env.ide_support.datadir
+        deploy_dir_image = shared_env.ide_support.deploy_dir_image
+        real_multimach_target_sys = shared_env.ide_support.real_multimach_target_sys
+        standalone_sysroot_native = shared_env.build_sysroots.standalone_sysroot_native
+        vscode_ws_path = os.path.join(
+            os.environ['HOME'], '.local', 'share', 'CMakeTools')
+        cmake_kits_path = os.path.join(vscode_ws_path, 'cmake-tools-kits.json')
+        oecmake_generator = "Ninja"
+        env_script = os.path.join(
+            deploy_dir_image, 'environment-setup-' + real_multimach_target_sys)
+
+        if not os.path.isdir(vscode_ws_path):
+            os.makedirs(vscode_ws_path)
+        cmake_kits_old = []
+        if os.path.exists(cmake_kits_path):
+            with open(cmake_kits_path, 'r', encoding='utf-8') as cmake_kits_file:
+                cmake_kits_old = json.load(cmake_kits_file)
+        cmake_kits = cmake_kits_old.copy()
+
+        cmake_kit_new = {
+            "name": "OE " + real_multimach_target_sys,
+            "environmentSetupScript": env_script,
+            "toolchainFile": standalone_sysroot_native + datadir + "/cmake/OEToolchainConfig.cmake",
+            "preferredGenerator": {
+                "name": oecmake_generator
+            }
+        }
+
+        def merge_kit(cmake_kits, cmake_kit_new):
+            i = 0
+            while i < len(cmake_kits):
+                if 'environmentSetupScript' in cmake_kits[i] and \
+                        cmake_kits[i]['environmentSetupScript'] == cmake_kit_new['environmentSetupScript']:
+                    cmake_kits[i] = cmake_kit_new
+                    return
+                i += 1
+            cmake_kits.append(cmake_kit_new)
+        merge_kit(cmake_kits, cmake_kit_new)
+
+        if cmake_kits != cmake_kits_old:
+            logger.info("Updating: %s" % cmake_kits_path)
+            with open(cmake_kits_path, 'w', encoding='utf-8') as cmake_kits_file:
+                json.dump(cmake_kits, cmake_kits_file, indent=4)
+        else:
+            logger.info("Already up to date: %s" % cmake_kits_path)
+
+        cmake_native = os.path.join(
+            shared_env.build_sysroots.standalone_sysroot_native, 'usr', 'bin', 'cmake')
+        if os.path.isfile(cmake_native):
+            logger.info('cmake-kits call cmake by default. If the cmake provided by this SDK should be used, please add the following line to ".vscode/settings.json" file: "cmake.cmakePath": "%s"' % cmake_native)
+        else:
+            logger.error("Cannot find cmake native at: %s" % cmake_native)
+
+    def dot_code_dir(self, modified_recipe):
+        return os.path.join(modified_recipe.srctree, '.vscode')
+
+    def __vscode_settings_meson(self, settings_dict, modified_recipe):
+        if modified_recipe.build_tool is not BuildTool.MESON:
+            return
+        settings_dict["mesonbuild.mesonPath"] = modified_recipe.meson_wrapper
+
+        confopts = modified_recipe.mesonopts.split()
+        confopts += modified_recipe.meson_cross_file.split()
+        confopts += modified_recipe.extra_oemeson.split()
+        settings_dict["mesonbuild.configureOptions"] = confopts
+        settings_dict["mesonbuild.buildFolder"] = modified_recipe.b
+
+    def __vscode_settings_cmake(self, settings_dict, modified_recipe):
+        """Add cmake specific settings to settings.json.
+
+        Note: most settings are passed to the cmake preset.
+        """
+        if modified_recipe.build_tool is not BuildTool.CMAKE:
+            return
+        settings_dict["cmake.configureOnOpen"] = True
+        settings_dict["cmake.sourceDirectory"] = modified_recipe.real_srctree
+
+    def vscode_settings(self, modified_recipe):
+        files_excludes = {
+            "**/.git/**": True,
+            "**/oe-logs/**": True,
+            "**/oe-workdir/**": True,
+            "**/source-date-epoch/**": True
+        }
+        python_exclude = [
+            "**/.git/**",
+            "**/oe-logs/**",
+            "**/oe-workdir/**",
+            "**/source-date-epoch/**"
+        ]
+        settings_dict = {
+            "files.watcherExclude": files_excludes,
+            "files.exclude": files_excludes,
+            "python.analysis.exclude": python_exclude
+        }
+        self.__vscode_settings_cmake(settings_dict, modified_recipe)
+        self.__vscode_settings_meson(settings_dict, modified_recipe)
+
+        settings_file = 'settings.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), settings_file, settings_dict)
+
+    def __vscode_extensions_cmake(self, modified_recipe, recommendations):
+        if modified_recipe.build_tool is not BuildTool.CMAKE:
+            return
+        recommendations += [
+            "twxs.cmake",
+            "ms-vscode.cmake-tools",
+            "ms-vscode.cpptools",
+            "ms-vscode.cpptools-extension-pack",
+            "ms-vscode.cpptools-themes"
+        ]
+
+    def __vscode_extensions_meson(self, modified_recipe, recommendations):
+        if modified_recipe.build_tool is not BuildTool.MESON:
+            return
+        recommendations += [
+            'mesonbuild.mesonbuild',
+            "ms-vscode.cpptools",
+            "ms-vscode.cpptools-extension-pack",
+            "ms-vscode.cpptools-themes"
+        ]
+
+    def vscode_extensions(self, modified_recipe):
+        recommendations = []
+        self.__vscode_extensions_cmake(modified_recipe, recommendations)
+        self.__vscode_extensions_meson(modified_recipe, recommendations)
+        extensions_file = 'extensions.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), extensions_file, {"recommendations": recommendations})
+
+    def vscode_c_cpp_properties(self, modified_recipe):
+        properties_dict = {
+            "name": modified_recipe.recipe_id_pretty,
+        }
+        if modified_recipe.build_tool is BuildTool.CMAKE:
+            properties_dict["configurationProvider"] = "ms-vscode.cmake-tools"
+        elif modified_recipe.build_tool is BuildTool.MESON:
+            properties_dict["configurationProvider"] = "mesonbuild.mesonbuild"
+        else:  # no C/C++ build
+            return
+
+        properties_dicts = {
+            "configurations": [
+                properties_dict
+            ],
+            "version": 4
+        }
+        prop_file = 'c_cpp_properties.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), prop_file, properties_dicts)
+
+    def vscode_launch_bin_dbg(self, gdb_cross_config):
+        modified_recipe = gdb_cross_config.modified_recipe
+
+        launch_config = {
+            "name": gdb_cross_config.id_pretty,
+            "type": "cppdbg",
+            "request": "launch",
+            "program": os.path.join(modified_recipe.d, gdb_cross_config.binary.lstrip('/')),
+            "stopAtEntry": True,
+            "cwd": "${workspaceFolder}",
+            "environment": [],
+            "externalConsole": False,
+            "MIMode": "gdb",
+            "preLaunchTask": gdb_cross_config.id_pretty,
+            "miDebuggerPath": modified_recipe.gdb_cross.gdb,
+            "miDebuggerServerAddress": "%s:%d" % (modified_recipe.gdb_cross.host, gdb_cross_config.gdbserver_port)
+        }
+
+        # Search for header files in recipe-sysroot.
+        src_file_map = {
+            "/usr/include": os.path.join(modified_recipe.recipe_sysroot, "usr", "include")
+        }
+        # First of all search for not stripped binaries in the image folder.
+        # These binaries are copied (and optionally stripped) by deploy-target
+        setup_commands = [
+            {
+                "description": "sysroot",
+                "text": "set sysroot " + modified_recipe.d
+            }
+        ]
+
+        if gdb_cross_config.image_recipe.rootfs_dbg:
+            launch_config['additionalSOLibSearchPath'] = modified_recipe.solib_search_path_str(
+                gdb_cross_config.image_recipe)
+            src_file_map["/usr/src/debug"] = os.path.join(
+                gdb_cross_config.image_recipe.rootfs_dbg, "usr", "src", "debug")
+        else:
+            logger.warning(
+                "Cannot setup debug symbols configuration for GDB. IMAGE_GEN_DEBUGFS is not enabled.")
+
+        launch_config['sourceFileMap'] = src_file_map
+        launch_config['setupCommands'] = setup_commands
+        return launch_config
+
+    def vscode_launch(self, modified_recipe):
+        """GDB Launch configuration for binaries (elf files)"""
+
+        configurations = [self.vscode_launch_bin_dbg(
+            gdb_cross_config) for gdb_cross_config in self.gdb_cross_configs]
+        launch_dict = {
+            "version": "0.2.0",
+            "configurations": configurations
+        }
+        launch_file = 'launch.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), launch_file, launch_dict)
+
+    def vscode_tasks_cpp(self, args, modified_recipe):
+        run_install_deploy = modified_recipe.gen_install_deploy_script(args)
+        install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty
+        tasks_dict = {
+            "version": "2.0.0",
+            "tasks": [
+                {
+                    "label": install_task_name,
+                    "type": "shell",
+                    "command": run_install_deploy,
+                    "problemMatcher": []
+                }
+            ]
+        }
+        for gdb_cross_config in self.gdb_cross_configs:
+            tasks_dict['tasks'].append(
+                {
+                    "label": gdb_cross_config.id_pretty,
+                    "type": "shell",
+                    "isBackground": True,
+                    "dependsOn": [
+                        install_task_name
+                    ],
+                    "command": gdb_cross_config.gdbserver_script,
+                    "problemMatcher": [
+                        {
+                            "pattern": [
+                                {
+                                    "regexp": ".",
+                                    "file": 1,
+                                    "location": 2,
+                                    "message": 3
+                                }
+                            ],
+                            "background": {
+                                "activeOnStart": True,
+                                "beginsPattern": ".",
+                                "endsPattern": ".",
+                            }
+                        }
+                    ]
+                })
+        tasks_file = 'tasks.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), tasks_file, tasks_dict)
+
+    def vscode_tasks_fallback(self, args, modified_recipe):
+        oe_init_dir = modified_recipe.oe_init_dir
+        oe_init = ". %s %s > /dev/null && " % (modified_recipe.oe_init_build_env, modified_recipe.topdir)
+        dt_build = "devtool build "
+        dt_build_label = dt_build + modified_recipe.recipe_id_pretty
+        dt_build_cmd = dt_build + modified_recipe.bpn
+        clean_opt = " --clean"
+        dt_build_clean_label = dt_build + modified_recipe.recipe_id_pretty + clean_opt
+        dt_build_clean_cmd = dt_build + modified_recipe.bpn + clean_opt
+        dt_deploy = "devtool deploy-target "
+        dt_deploy_label = dt_deploy + modified_recipe.recipe_id_pretty
+        dt_deploy_cmd = dt_deploy + modified_recipe.bpn
+        dt_build_deploy_label = "devtool build & deploy-target %s" % modified_recipe.recipe_id_pretty
+        deploy_opts = ' '.join(get_devtool_deploy_opts(args))
+        tasks_dict = {
+            "version": "2.0.0",
+            "tasks": [
+                {
+                    "label": dt_build_label,
+                    "type": "shell",
+                    "command": "bash",
+                    "linux": {
+                        "options": {
+                            "cwd": oe_init_dir
+                        }
+                    },
+                    "args": [
+                        "--login",
+                        "-c",
+                        "%s%s" % (oe_init, dt_build_cmd)
+                    ],
+                    "problemMatcher": []
+                },
+                {
+                    "label": dt_deploy_label,
+                    "type": "shell",
+                    "command": "bash",
+                    "linux": {
+                        "options": {
+                            "cwd": oe_init_dir
+                        }
+                    },
+                    "args": [
+                        "--login",
+                        "-c",
+                        "%s%s %s" % (
+                            oe_init, dt_deploy_cmd, deploy_opts)
+                    ],
+                    "problemMatcher": []
+                },
+                {
+                    "label": dt_build_deploy_label,
+                    "dependsOrder": "sequence",
+                    "dependsOn": [
+                        dt_build_label,
+                        dt_deploy_label
+                    ],
+                    "problemMatcher": [],
+                    "group": {
+                        "kind": "build",
+                        "isDefault": True
+                    }
+                },
+                {
+                    "label": dt_build_clean_label,
+                    "type": "shell",
+                    "command": "bash",
+                    "linux": {
+                        "options": {
+                            "cwd": oe_init_dir
+                        }
+                    },
+                    "args": [
+                        "--login",
+                        "-c",
+                        "%s%s" % (oe_init, dt_build_clean_cmd)
+                    ],
+                    "problemMatcher": []
+                }
+            ]
+        }
+        if modified_recipe.gdb_cross:
+            for gdb_cross_config in self.gdb_cross_configs:
+                tasks_dict['tasks'].append(
+                    {
+                        "label": gdb_cross_config.id_pretty,
+                        "type": "shell",
+                        "isBackground": True,
+                        "dependsOn": [
+                            dt_build_deploy_label
+                        ],
+                        "command": gdb_cross_config.gdbserver_script,
+                        "problemMatcher": [
+                            {
+                                "pattern": [
+                                    {
+                                        "regexp": ".",
+                                        "file": 1,
+                                        "location": 2,
+                                        "message": 3
+                                    }
+                                ],
+                                "background": {
+                                    "activeOnStart": True,
+                                    "beginsPattern": ".",
+                                    "endsPattern": ".",
+                                }
+                            }
+                        ]
+                    })
+        tasks_file = 'tasks.json'
+        IdeBase.update_json_file(
+            self.dot_code_dir(modified_recipe), tasks_file, tasks_dict)
+
+    def vscode_tasks(self, args, modified_recipe):
+        if modified_recipe.build_tool.is_c_ccp:
+            self.vscode_tasks_cpp(args, modified_recipe)
+        else:
+            self.vscode_tasks_fallback(args, modified_recipe)
+
+    def setup_modified_recipe(self, args, image_recipe, modified_recipe):
+        self.vscode_settings(modified_recipe)
+        self.vscode_extensions(modified_recipe)
+        self.vscode_c_cpp_properties(modified_recipe)
+        if args.target:
+            self.initialize_gdb_cross_configs(
+                image_recipe, modified_recipe, gdb_cross_config_class=GdbCrossConfigVSCode)
+            self.vscode_launch(modified_recipe)
+            self.vscode_tasks(args, modified_recipe)
+
+
+def register_ide_plugin(ide_plugins):
+    ide_plugins['code'] = IdeVSCode