summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* progress: always show done messageKuang-che Wu2024-10-241-9/+4
| | | | | | | | | | | | The done message was omitted if the task is shorter than 0.5s. This might confuse users. Bug: b/371638995 Change-Id: I3fdd2cd8daea16d34fba88457d09397fff71af15 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440222 Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* subcmds: reduce multiprocessing serialization overheadKuang-che Wu2024-10-2311-172/+228
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Follow the same approach as 39ffd9977e to reduce serialization overhead. Below benchmarks are tested with 2.7k projects on my workstation (warm cache). git tracing is disabled for benchmark. (seconds) | v2.48 | v2.48 | this CL | this CL | | -j32 | | -j32 ----------------------------------------------------------- with clean tree state: branches (none) | 5.6 | 5.9 | 1.0 | 0.9 status (clean) | 21.3 | 9.4 | 19.4 | 4.7 diff (none) | 7.6 | 7.2 | 5.7 | 2.2 prune (none) | 5.7 | 6.1 | 1.3 | 1.2 abandon (none) | 19.4 | 18.6 | 0.9 | 0.8 upload (none) | 19.7 | 18.7 | 0.9 | 0.8 forall -c true | 7.5 | 7.6 | 0.6 | 0.6 forall -c "git log -1" | 11.3 | 11.1 | 0.6 | 0.6 with branches: start BRANCH --all | 21.9 | 20.3 | 13.6 | 2.6 checkout BRANCH | 29.1 | 27.8 | 1.1 | 1.0 branches (2) | 28.0 | 28.6 | 1.5 | 1.3 abandon BRANCH | 29.2 | 27.5 | 9.7 | 2.2 Bug: b/371638995 Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* sync: reduce multiprocessing serialization overheadKuang-che Wu2024-10-232-87/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: - Manifest object is large (for projects like Android) in terms of serialization cost and size (more than 1mb). - Lots of Project objects usually share only a few manifest objects. Before this CL, Project objects were passed to workers via function parameters. Function parameters are pickled separately (in chunk). In other words, manifests are serialized again and again. The major serialization overhead of repo sync was O(manifest_size * projects / chunksize) This CL uses following tricks to reduce serialization overhead. - All projects are pickled in one invocation. Because Project objects share manifests, pickle library remembers which objects are already seen and avoid the serialization cost. - Pass the Project objects to workers at worker intialization time. And pass project index as function parameters instead. The number of workers is much smaller than the number of projects. - Worker init state are shared on Linux (fork based). So it requires zero serialization for Project objects. On Linux (fork based), the serialization overhead is O(projects) --- one int per project On Windows (spawn based), the serialization overhead is O(manifest_size * min(workers, projects)) Moreover, use chunksize=1 to avoid the chance that some workers are idle while other workers still have more than one job in their chunk queue. Using 2.7k projects as the baseline, originally "repo sync" no-op sync takes 31s for fetch and 25s for checkout on my Linux workstation. With this CL, it takes 12s for fetch and 1s for checkout. Bug: b/371638995 Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921 Tested-by: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* Fix incremental syncs for prjs with submodulesKaushik Lingarkar2024-10-181-1/+3
| | | | | | | | | | | | | | | | | | | When performing an incremental sync (re-running repo init with an updated manifest revision) with --fetch-submodules or sync-s=true, there is an attempt to get a list of all projects (including submodules) before projects are actually fetched. However, we can only list submodules of a project if we have already fetched its revision. Instead of throwing an error when we don't have the revision, assume there are no submodules for that project. In the sync cmd, we already update the list of projects to include submodules after fetching superprojects. Change-Id: I48bc68c48b5b10117356b18f5375d17f9a89ec05 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439761 Commit-Queue: Kaushik Lingarkar <kaushik.lingarkar@linaro.org> Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org>
* sync: Always use WORKER_BATCH_SIZEv2.48Josip Sokcevic2024-10-073-27/+43
| | | | | | | | | | | | | | | | | | | With 551285fa35ccd0836513e9cf64ee8d3372e5e3f4, the comment about number of workers no longer stands - dict is shared among multiprocesses and real time information is available. Using 2.7k projects as the baseline, using chunk size of 4 takes close to 5 minutes. A chunk size of 32 takes this down to 40s - a reduction of rougly 8 times which matches the increase. R=gavinmak@google.com Bug: b/371638995 Change-Id: Ida5fd8f7abc44b3b82c02aa0f7f7ae01dff5eb07 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438523 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* project: Handle git sso auth failures as repo exitJosip Sokcevic2024-10-032-0/+19
| | | | | | | | | | | | | | | | If a user is not authenticated, repo continues execution and it will likely result in more of the same errors being printed. A user is also likely to SIGTERM the process resulting in more errors. This change stops repo sync if any of repositories can't be fetched to Git authentcation using sso helper. We could extend this to all Git authentication Change-Id: I9e471e063450c0a51f25a5e7f12a83064dfb170c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438522 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* superproject: Remove notice about betaJosip Sokcevic2024-10-031-12/+0
| | | | | | | | | | | It's been the default for Android for over a year now, and it's no longer useful notice. Change-Id: I53c6f1e7cee8c1b2f408e67d3a6732db3b272bee Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438521 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* project: run fetch --refetch on unable to not parse commitJosip Sokcevic2024-10-031-1/+4
| | | | | | | | | | | | | | | Similarly to e59e2ae757623e64f625a9cdadf1c2010ef82b34, handle missing gc'ed commits by running `git fetch --refetch`. R=jojwang@google.com Bug: b/360889369 Bug: b/371000949 Change-Id: I108b870b855d3b9f23665afa134c6e35f7cd2830 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438461 Reviewed-by: Joanna Wang <jojwang@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* Disable git terminal prompt during fetch/cloneJosip Sokcevic2024-09-262-6/+22
| | | | | | | | | | | | | | | | | | | | git fetch operation may prompt user to enter username and password. This won't be visible to user when repo sync operation since stdout and stderr are redirected. If that happens, user may think repo is doing work and likely won't realize it's stuck on user's input. This patch disables prompt for clone and fetch operations, and repo will fail fast. R=gavinmak@google.com Bug: b/368644181 Change-Id: I2efa88ae66067587a00678eda155d861034b9127 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438001 Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* init: add --manifest-upstream-branchKaushik Lingarkar2024-09-263-2/+28
| | | | | | | | | | | | | | | When a sha1 is provided to '--manifest-branch', the ref which is expected to contain that sha1 can be provided using the new '--manifest-upstream-branch' option. This is useful with '--current-branch' to avoid having to sync all heads and tags, or with a commit that comes from a non-head/tag ref (like a Gerrit change ref). Change-Id: I46a3e255ca69ed9e809039e58b0c163e02af94ef Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436717 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Kaushik Lingarkar <kaushik.lingarkar@linaro.org> Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
* man: regenerate man pagesKaushik Lingarkar2024-09-252-2/+10
| | | | | | | | | Change-Id: Icf697eda7d5dcdc87854ad6adf607353c7ba5ac2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437941 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* project: Copy and link files even with local branchesBrian Norris2024-09-191-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the winding maze that constitutes Sync_LocalHalf(), there are paths in which we don't copy-and-link files. Examples include something like: cd some/project/ repo start head . # do some work, make some commit, upload that commit to Gerrit [[ ... in the meantime, someone addes a <linkfile ...> for some/project/ in the manifest ... ]] cd some/project/ git pull --rebase repo sync In this case, we never hit a `repo rebase` case, which might have saved us. Instead, the developer is left confused why some/project/ never had its <linkfile>s created. Notably, this opens up one more corner case in which <linkfile ... /> or <copyfile ... /> could potentially clobber existing work in the destination directory, but there are existing cases where that's true, and frankly, those seem like bigger holes than this new one. Change-Id: I394b0e4529023a8ee319dc25d03d513a19251a4a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437421 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Brian Norris <briannorris@google.com> Commit-Queue: Brian Norris <briannorris@google.com>
* sync: include TARGET_RELEASE when constructing smart sync target.v2.47Navil2024-09-121-0/+13
| | | | | | | | | | | | | | | | | | | | | | When using the smart sync option, we try to construct the target that was "lunched" from the TARGET_PRODUCT and TARGET_BUILD_VARIANT envvars. However, an android target is now made of three parts, {TARGET_PRODUCT}-{TARGET_RELEASE}-{TARGET_BUILD_VARIANT}. I am leaving the option of creating a target if a TARGET_RELEASE is not specified in case there are other consumers who depend on that option. BUG=b:358101714 TEST=./run_tests TEST=smart sync on android repo and manually inspecting smart_sync_override.xml Change-Id: I556137e33558783a86a0631f29756910b4a93d92 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436977 Tested-by: Yiwei Zhang <yiwzhang@google.com> Reviewed-by: Yiwei Zhang <yiwzhang@google.com> Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
* color: fix have_fg not re assign to truebright2024-09-123-0/+83
| | | | | | | | | | | | | | | | In method _parse the value of this variable 'have_fg ' is always False, Maybe reassign it to True is lost. I guess the author’s original intention was: if set some value in gitconfig file(for ex: text = black red ul), the first is bg color, the second is fg color, and the last one is attr. Change-Id: I372698fe625db4c1fdaa94ea7f193a80a850ecb9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/425997 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Bright Ma <mmh1989@foxmail.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* tox.ini: Make the lint and format environments run black for all codePeter Kjellerstedt2024-09-121-2/+2
| | | | | | | | | | This matches the extra files specified in run_tests. Change-Id: Ic8999383a17b3ec7ae27322323ea44eeaa40c968 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/434998 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* tox.ini, constraints.txt: Lock the version of black to <24Peter Kjellerstedt2024-09-122-0/+4
| | | | | | | | | | | The formatting produced by black versions before 24 matches the current formatting of the code. Change-Id: I045f22d2f32a09d4683867293e81512f2abd1036 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/434997 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* project: run fetch --refetch onacould not parse commitJosip Sokcevic2024-09-121-0/+11
| | | | | | | | | | | | | | | | git may gc reachable objects in partial clone repository due to a bug (report: https://lore.kernel.org/git/20240802073143.56731-1-hanyang.tony@bytedance.com/ ). Until git is properly patched and released, force --refetch iff "could not parse commit" is part of git output. --refetch will will ensure that gc'ed objects are retrieved. Bug: b/360889369 Change-Id: I0fc911c591060f859235dcd8d019881106f0858e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437017 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Sam Saccone <samccone@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* Add a `--rebase` option to sync commandJeroen Dhollander2024-08-302-3/+21
| | | | | | | | | | | | | | | | Previously repo would abort a sync if there were published changes not merged upstream. The --rebase option allows the modification of published commits. This is a copy of http://go/grev/369694 with the merge conflicts resolved. Bug: 40014610 Change-Id: Idac8199400346327b530abea33f1ed794e5bb4c2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/435838 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Jeroen Dhollander <jeroendh@google.com> Commit-Queue: Jeroen Dhollander <jeroendh@google.com>
* sync: Fix git command for aborting rebase being called incorrectly.Erik Elmeke2024-08-281-1/+1
| | | | | | | | | | | | The argument list was incorrectly destructured so only the first element of the list was considered a git-cmd, split up by each character in the string. Change-Id: Idee8a95a89a7da8b8addde07135354fc506c2758 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/435839 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com> Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
* project: ignore more curl failure modesv2.46Mike Frysinger2024-07-021-4/+13
| | | | | | | | | | | | | Current clone bundle fetches from Google storage results HTTP/404 and curl exiting 56. This is basically WAI, so stop emitting verbose error output whenever that happens. Also add a few more curl exit statuses based on chromite history, and document them. Change-Id: I3109f8a8a19109ba9bbd62780b40bbcd4fce9b76 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/432197 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* logging: Fix log formatting with colored outputShik Chen2024-07-022-2/+39
| | | | | | | | | | | | | | | | | | | | The log message is already formatted before being passed to the colorer. To avoid the exception "TypeError: not enough arguments for format string", we should use the `nofmt_colorer` instead. This bug occurs only when the formatted string still contains '%' character. The following snippet can reproduce the bug: ``` from repo_logging import RepoLogger RepoLogger(__name__).error("%s", "100% failed") ``` Change-Id: I4e3977b3d21aec4e0deb95fc1c6dd1e59272d695 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/432017 Tested-by: Shik Chen <shik@google.com> Commit-Queue: Shik Chen <shik@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* upload: add a --topic option for setting topic explicitlyMike Frysinger2024-07-013-13/+24
| | | | | | | | | | | | | | Let people specify the exact topic when uploading CLs. The existing -t option only supports setting the topic to the current local branch. Add a --topic-branch long option to the existing -t to align it a bit better with --hashtag & --hashtag-branch. Change-Id: I010abc4a7f3c685021cae776dd1e597c22b79135 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/431997 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* sync: Abort rebase in progress if force-checkout is setErik Elmeke2024-05-231-1/+31
| | | | | | | | | | | | | This will make "repo sync -d --force-checkout" more reliable in CI automation, as there are fewer things in the way that may need manual intervention. Bug: b/40015382 Change-Id: I8a79971724a3d9a8e2d682b7a0c04deda9e34177 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/423317 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com> Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
* ssh: Set git protocol version 2 on SSH ControlMasterErik Elmeke2024-05-161-1/+57
| | | | | | | | | | | | | | | | | | | | | | | | | According to https://git-scm.com/docs/protocol-v2#_ssh_and_file_transport, when using SSH, the environment variable GIT_PROTOCOL must be set when establishing the connection to the git server. Normally git does this by itself. But in repo-tool where the SSH connection is managed by the repo-tool, it must be passed in explicitly instead. Under some circumstances of environment configuration, this caused all repo sync commands over ssh to always use git protocol version 1. Even when git was configured to use version 2. Using git protocol v2 can significantly improve fetch speeds, since it uses server side filtering of refs, reducing the amount of unneccessary objects to send. Change-Id: I6d4c3b7300a6090d707480b1a638ed03622fa71a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411362 Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
* upload: drop check for uncommitted local changesMike Frysinger2024-05-141-31/+0
| | | | | | | | | | | | | git push, like most git commands, does not warn or otherwise prompt users when there are local uncommitted changes. To simplify the upload logic, and to harmonize repo upload with git push as a more git-esque flow, stop checking/warning/prompting the user here too. Change-Id: Iee18132f0faad0881f1a796cb58821328e04b694 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/425337 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* git: raise hard version to 1.9.1Mike Frysinger2024-05-012-11/+3
| | | | | | | | | | | | | | | | | | Debian 7 Wheezy went EOL in May 2018. We don't need to carry support for that anymore as there have been 5 major releases since. Ubuntu Precise went EOL in Apr 2019 (including the extended support phase). That means we can bump the required git version from 1.7.9 to 1.9.1. git-1.7.9 was released in 2012 while git-1.9.1 was released in 2014. So that shouldn't be a problem either. And we've been warning people using git versions older than 1.9.1 for 3 years now that they need to upgrade. Change-Id: Ifbbf72f51010b0a944c2785895d1b605333f9146 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415637 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* release: update-hooks: helper for automatically syncing hooksMike Frysinger2024-04-234-33/+166
| | | | | | | | | | | | These hooks are maintained in other projects. Add a script to automate their import so people don't send us changes directly, and we can try to steer them to the correct place. Change-Id: Iac0bdb3aae84dda43a1600e73107555b513ce82b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/422177 Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* gitc: delete a few more dead referencesMike Frysinger2024-04-182-8/+0
| | | | | | | | Change-Id: I1da6f2ee799c735a63ac3ca6e5abd1211af10433 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/419217 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* man: regenerate man pagesMike Frysinger2024-04-187-236/+51
| | | | | | | | Change-Id: I8d9dcb37f315d4208b7c8005206ae939dad79a3e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/419197 Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* Remove platform_utils.realpathKaiyi Li2024-03-272-25/+12
| | | | | | | | | | ... since it's just a simple wrapper of os.path.realpath now. Change-Id: I7433e5fe09c64b130f06e2541151dce1961772c9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416637 Tested-by: Kaiyi Li <kaiyili@google.com> Reviewed-by: Greg Edelston <gredelston@google.com> Commit-Queue: Kaiyi Li <kaiyili@google.com>
* Fix drive mounted directory on WindowsKaiyi Li2024-03-271-21/+1
| | | | | | | | | | | | | | On my Windows machine, I mount drive D: to the directory C:\src. The old implementation returns the incorrect 'C:\\??\\Volume{ad2eb15e-f293-4d48-a448-54757d95a97c}' result, which breaks the repo init command. With the use of os.path.realpath, it can return 'D:\\' correctly. Change-Id: Ia5f53989055125cb282d4123cf55d060718aa1ff Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416580 Reviewed-by: Greg Edelston <gredelston@google.com> Tested-by: Kaiyi Li <kaiyili@google.com> Commit-Queue: Kaiyi Li <kaiyili@google.com>
* git_command: unify soft/hard versions with requirements.jsonMike Frysinger2024-03-214-25/+19
| | | | | | | | | | | Use the requirements logic in the wrapper to load versions out of the requirements.json file to avoid duplicating them in git_command.py. Change-Id: Ib479049fc54ebc6f52c2c30d1315cf1734ff1990 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415617 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: Check if dotgit exists w/out symlink checkv2.45Josip Sokcevic2024-03-201-1/+1
| | | | | | | | | | | | | | | | | | | | os.path.exists returns false on a broken symlink. This is not what repo needs when checking if a project is setup properly. For example, if src/foo/.git can't be resolved, repo tries to create symlink and that results in FileExistsError. Use lexists which returns True even if symlink is broken. That will force path where repo checks where symlink is pointing to and will fix it to the correct location. Bug: b/281746795 Change-Id: Id3f7dc3a3cb6499d02ce7335eca992ddc7deb645 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415197 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: George Engelbrecht <engeg@google.com> Reviewed-by: Greg Edelston <gredelston@google.com>
* git: raise soft version to 2.7.4Mike Frysinger2024-03-202-6/+11
| | | | | | | | | | | | | | | | | git-1.9.1 was released in 2014 while git-2.7.4 was released in 2016. Debian Stretch was released in 2017 and Ubuntu Xenial was released in 2016 which are plenty old at this point. Both of those include at least git-2.7.4. We will start warning users of Debian Jessie (released in 2015 & EOL in 2020) and Ubuntu Trusty (released in 2014 & EOL Apr 2024) that they will need to upgrade. Change-Id: I6be3809bc45968fdcb02cad3f7daf75ded1bb5b1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415137 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* git: raise hard version to 1.7.9Mike Frysinger2024-03-202-7/+6
| | | | | | | | | | | | | | | | | | Debian 6 Squeeze went EOL in Feb 2016. We don't need to carry support for that anymore as there have been 6 major releases since. That means we can bump the required git version from 1.7.2 to 1.7.9. Ubuntu Precise shipped with the latter. git-1.7.2 was released in 2010 while git-1.7.9 was released in 2012. So that shouldn't be a problem either. And we've been warning people using git versions older than 1.9.1 for 3 years now that they need to upgrade. Change-Id: I7712f110ea158297b489b8379b112c6700b21a46 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415097 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* docs: release: add recent git/python/ssh/debian infoMike Frysinger2024-03-201-5/+23
| | | | | | | | Change-Id: I744360b1bfc816e94b3511f0130abb2c08dedd42 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415117 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* main: Stringify project name in error_infoJosip Sokcevic2024-03-152-3/+3
| | | | | | | | | | | | | | | | | | If a project can't be removed from checkout due to uncommitted changes present, error.project is type of Project and not a string (as it is in some cases). Project is not JSON serializable, resulting in exception within exception handler: TypeError: Object of type Project is not JSON serializable This change casts project to string as a defensive mechanism. It also passes project name instead of project object. Change-Id: Ie7b782d73dc3647975755d5a3774d16ea6cd5348 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/413877 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* ssh: Print details if running the command failsv2.44Sebastian Schuberth2024-03-111-2/+6
| | | | | | | | | Change-Id: I87adbdd5fe4eb2709c97ab4c21b414145acf788b Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/392915 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Tuan Vo Hung <vohungtuan@gmail.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* upload: Fix patchset description destinationJosip Sokcevic2024-03-081-0/+1
| | | | | | | | | Bug: 308467447 Change-Id: I8ad598d39f5fdb24d549d3277ad5fedac203581b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412477 Reviewed-by: George Engelbrecht <engeg@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* sync: Fix sorting for nested projectsJosip Sokcevic2024-03-082-13/+42
| | | | | | | | | | | | | | | | | | | The current logic to create checkout layers doesn't work in all cases. For example, let's assume there are three projects: "foo", "foo/bar" and "foo-bar". Sorting lexicographical order is incorrect as foo-bar would be placed between foo and foo/bar, breaking layering logic. Instead, we split filepaths based using path delimiter (always /) and then use lexicographical sort. BUG=b:325119758 TEST=./run_tests, manual sync on chromiumos repository Change-Id: I76924c3cc6ba2bb860d7a3e48406a6bba8f58c10 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412338 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: George Engelbrecht <engeg@google.com>
* sync: introduce --force-checkoutv2.43Josip Sokcevic2024-03-072-5/+31
| | | | | | | | | | | | | | In some cases (e.g. in a CI system), it's desirable to be able to instruct repo to force checkout. This flag passes --force flag to `git checkout` operations. Bug: b/327624021 Change-Id: I579edda546fb8147c4e1a267e2605fcf6e597421 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411518 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Reviewed-by: George Engelbrecht <engeg@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* upload: Add support for setting patchset descriptionSergiy Belozorov2024-03-043-0/+48
| | | | | | | | | Bug: 308467447 Change-Id: I7abcbc98131b826120fc9ab85d5b889f90db4b0a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/355968 Tested-by: Sergiy Belozorov <sergiyb@chromium.org> Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
* sync: Introduce git checkout levelsv2.42Josip Sokcevic2024-02-272-9/+81
| | | | | | | | | | | | | | | | | | | | | If a repo manifest is updated so that project B is placed within a project A, and if project A had content in new B's location in the old checkout, then repo sync could break depending on checkout order, since B can't be checked out before A. This change introduces checkout levels which enforces right sequence of checkouts while still allowing for parallel checkout. In an example above, A will always be checked out first before B. BUG=b:325119758 TEST=./run_tests, manual sync on ChromeOS repository Change-Id: Ib3b5e4d2639ca56620a1e4c6bf76d7b1ab805250 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410421 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Greg Edelston <gredelston@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* git_command: Return None from GetEventTargetPath() if set to empty stringPeter Collingbourne2024-02-261-0/+2
| | | | | | | | | | | | If trace2.eventTarget was set to the empty string, match git behavior and don't write a trace. Bug: 319673783 Change-Id: I02b3884ad97551f8a9d7363c2cbe6b0adee6f73e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410518 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Peter Collingbourne <pcc@google.com>
* trace: Save trace2 sid in REPO_TRACE fileJosip Sokcevic2024-02-221-4/+7
| | | | | | | | | | | | | | git-trace2 events contain additional information what git is doing under the hood, and repo doesn't have visibility into. Instead of relying on timestamp information to match REPO_TRACE with git-trace2 events, add SID information into REPO_TRACE. Change-Id: I37672a3face81858072c7a3ce34ca3379199dab5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410280 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* subcmds: sync: Remove deprecated _AUTO_GCGreg Edelston2024-02-201-15/+0
| | | | | | | | | | | | | Opportunistic cleanup. It looks like this deprecated feature was slated for deletion nearly a year ago. Bug: None Test: ./run_tests Change-Id: I0bd0c0e6acbd1eaee1c0b4945c79038257d22f44 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410198 Reviewed-by: Yiwei Zhang <yiwzhang@google.com> Commit-Queue: Greg Edelston <gredelston@google.com> Tested-by: Greg Edelston <gredelston@google.com>
* launcher: Set shebang to python3Josip Sokcevic2024-02-071-2/+2
| | | | | | | | | | | | | | | | | | | | Some (most?) Linux repos don't have /usr/bin/python, unless python-is-python3 is installed. While package owners can adjust shebang, we have seen an increase in number of bugs filed as extra steps are required. Per PEP-0394, python3 is acceptable and should be available if python3 is supported. We no longer support python2, and repo no longer works with python2, so this change makes that explicit. Bug: 40014585 Change-Id: I9aed90fd470ef601bd33bd596af3df69da69ef5d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407497 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Jason Chang <jasonnc@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Максим Паймушкин <maxim.paymushkin@gmail.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* sync: ensure RepoChangedException propagatedJason Chang2024-02-061-1/+1
| | | | | | | | | | | | | Prior to this change RepoChangedException would be caught and re-rasied as a different exception. This would prevent RepoChangedException handler from running in main.py Bug: b/323232806 Change-Id: I9055ff95d439d6ff225206c5bf1755cc718bcfcc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407144 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* stop passing project to UpdateManifestErrorYiwei Zhang2024-02-021-3/+1
| | | | | | | | | | | | | | | | | UpdateManifestError inherits from RepoExitError which inherits from BaseRepoError. None of them takes project as kwargs causing the error like "UpdateManifestError() takes no keyword arguments" in b/317183321 [1]: https://gerrit.googlesource.com/git-repo/+/449b23b698d7d4b13909667a49a0698eb495eeaa/error.py#144 Bug: b/317183321 Change-Id: I64c3dc502027f9dda56a0824f2712364b4502934 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/398997 Commit-Queue: Yiwei Zhang <yiwzhang@google.com> Tested-by: Yiwei Zhang <yiwzhang@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Jason Chang <jasonnc@google.com>
* project: Rename if deletion failsv2.41Josip Sokcevic2024-01-251-10/+22
| | | | | | | | | | | | | If a project contains files not owned by the current user, remove will fail. In order to ensure repo sync continues to work, rename the affected project instead, and let user know about it. Bug: 321273512 Change-Id: I0779d61fc67042308a0226adea7d98167252a5d3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/404372 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-29 06:01:32 +0000