summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* project: use --netrc-optional instead of --netrcv2.54stableGavin Mak2025-04-101-1/+8
| | | | | | | | | | | | | | | | | Some users are reporting a "curl: (26) .netrc error: no such file" message on sync caused by an change to curl behavior. See https://github.com/curl/curl/issues/16163. Use --netrc-optional which was introduced in curl version 7.9.8 released in 2002. Bug: 409354839 Change-Id: I8365c6e806968a4ee765a7e023b4bced30489c20 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/467026 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
* launcher: switch command quoting to shlex.quoteMike Frysinger2025-04-102-9/+10
| | | | | | | | | | | Minor fix, but just in case, provides properly quoted commands for people to copy & paste. Change-Id: Ia9fce5c0df9f51cbed9d49861adcf6821251e46f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/466821 Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* sync: Warn about excessive job countsGavin Mak2025-04-091-0/+20
| | | | | | | | | | | | | | | Warn users if the effective job count specified via `-j`, `--jobs-network`, or `--jobs-checkout` exceeds a threshold (currently 100). This encourages users to use more reasonable values. Bug: 406868778 Bug: 254914814 Change-Id: I116e2bbaf3dc824c04d1b2fbe52cf9ca5be77b9a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/466801 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
* run_tests: fix running when cwd is not the rootMike Frysinger2025-04-031-5/+13
| | | | | | | | | | | | | | If you try running this from a subdir, then most of the tests fail because they assume they're running from the top of the source tree. Change all the tests to actually run there. For example: cd docs && ../run_tests Change-Id: I92e17476393a108e56b58e049193b9fd72c5b7ba Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464841 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* run_tests: enable Python 3.8 CI coverageMike Frysinger2025-04-022-0/+90
| | | | | | | | Change-Id: I507da20d3b7234e9f2a22d7654a6405b362eebaf Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464541 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* Fix EROFS error when root fs is mounted read-onlyEgor Duda2025-04-021-0/+6
| | | | | | | | | | | | | repo attempts to create /etc/.repo_gitconfig.json file, and fails if root file system is mounted read-only. Removing non-existing file on read-only filesystem results in EROFS instead of ENOENT. Bug: 401018409 Change-Id: I64edc0567fb88649f3fd8cacb65a8780744640d4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/458821 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Egor Duda <egor.duda@gmail.com> Commit-Queue: Egor Duda <egor.duda@gmail.com>
* pager: drop unused global varsMike Frysinger2025-04-011-1/+1
| | | | | | | | | | | We use global when we need to write to a variable, not read it. This function only reads, so drop the keyword. Change-Id: Iee91998fba67fd3e8ebaf2f4a79f95032f70b1c0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464501 Reviewed-by: Scott Lee <ddoman@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* launcher: change RunError to subprocess.CalledProcessErrorMike Frysinger2025-04-012-9/+6
| | | | | | | | | | | Since we require Python 3.6 now in the launcher, swap out our custom RunError class for the standard subprocess one. Change-Id: Id0ca17c40e22ece03e06366a263ad340963f979d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/464401 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Scott Lee <ddoman@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* launcher: change collections.namedtuple to typing.NamedTupleMike Frysinger2025-03-281-6/+11
| | | | | | | | | | | Since we require Python 3.6 now in the launcher, switch to NamedTuple so we get better documentation & typing information. Change-Id: Ic58fdc07db02fc49166eccbbc3e527f474973424 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/463721 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* tox: sync black settings with run_testsMike Frysinger2025-03-272-1/+3
| | | | | | | | | | We updated run_tests to use black-25, so update tox too. Change-Id: I7ee6471fbc78825bd2dbc8c1f8dab9dc10460852 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/463601 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Scott Lee <ddoman@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* black: update to v25Mike Frysinger2025-03-255-36/+36
| | | | | | | | | | Requires a little reformatting in the tree. Change-Id: Iaa40fe0dfca372c49c04cc26edccb5f7b0c2a8ad Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/462883 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* run_tests: move test filtering to pytest markersMike Frysinger2025-03-254-14/+13
| | | | | | | | | | | | Move the test disable logic even closer to the exact test that's disabled. This way people updating tests have a better chance of seeing they'll get reduced coverage in the CQ. Change-Id: I57c1a073a844019798b27e14d742fd32925d9ae8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/462882 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* run_tests: move CQ test skips hereMike Frysinger2025-03-251-1/+30
| | | | | | | | | | | Our recipes have been disabling a bunch of tests. To increase visibility, and to make it easier to test changes, move that logic to this script. Change-Id: I3894f047715177c0f1d27a2fe4c3490972dab204 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/462881 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* run_tests: run all tests all the timeMike Frysinger2025-03-251-1/+3
| | | | | | | | | | | Using a generator w/all() causes the code to exit on the first error. We really want to see all errors all the time, so use sum() instead. Change-Id: Ib1adb8de199db9fe727d4b49c890b4d5061e9e6b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/462901 Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Scott Lee <ddoman@google.com>
* launcher: change RunResult to subprocess.CompletedProcessMike Frysinger2025-03-242-13/+10
| | | | | | | | | | | Since we require Python 3.6 now in the launcher, swap out our custom RunResult class for the standard subprocess one. Change-Id: Idd8598df37c0a952d3ef828df6e250cab03c6589 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/462341 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* Sync: Fix full submodule sync while shallow specifiedv2.53Albert Akmukhametov2025-03-131-8/+20
| | | | | | | | | | | | | | | | | | | | Git allows to clone submodules as shallow clone [1]. On the other hand, when repo synchronize a projcet with submodules inside, it ignores the shallow parameter. When a project contains submodules, project.py parses the .gitmodules file for URL and path. This parsing does not consider the shallow option. Consequently, this parameter is not propgated to newly created Project instance for that submodule. [1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtshallow Change-Id: I54fc9c69ae1b8e3cda2801202e3f0c7693b718d2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/454261 Tested-by: Albert Akmukhametov <alb.02057@gmail.com> Commit-Queue: Albert Akmukhametov <alb.02057@gmail.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Никита Сказкоподателев (Nask) <skazkopodatelev@gmail.com>
* sync: Refresh index before updating repoJosip Sokcevic2025-03-131-0/+2
| | | | | | | | | | | | | If the repo index is stale, reset --keep will refuse to reset workspace. An index can be stale if there are any modifications to file node, including mtime, atime, ownership changes, etc. Bug: b/375423099 Change-Id: Ibef03d9d8d2babbb107041707281687342ab7a77 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/460022 Commit-Queue: Josip Sokcevic <sokcevic@chromium.org> Tested-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Scott Lee <ddoman@google.com>
* gc: Add tags to remote pack listv2.52Josip Sokcevic2025-02-051-0/+2
| | | | | | | | | | | | | | | | | | | | If tags are omitted from the remote pack list, they must be present in local pack. However, local packs don't have promisor objects, meaning that all blobs must be available locally, and therefore all missing blobs will be downloaded during rev-list phase. Git downloads those sequentially, by invokving fetch operation (rev-list/fetch). Instead of downloading tags' blobs, instruct Git to include all tags in remote rev-list operation. This change was tested with `git fsck --all`. R=yiwzhang@google.com Bug: b/392732561 Change-Id: Id94a40aebbe4f084c952329583d559d296db1a11 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/451422 Reviewed-by: Yiwei Zhang <yiwzhang@google.com> Tested-by: Josip Sokcevic <sokcevic@chromium.org> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
* Activate submodulesKaushik Lingarkar2025-02-041-0/+22
| | | | | | | | | | | | | | This change moves further towards ensuring Git can understand repo's submodules. 'submodule init' is used to make the submodules active[1]. [1] https://git-scm.com/docs/gitsubmodules#_active_submodules Change-Id: I0c20ff1991101fc5be171e566d8fb644aab47200 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/446182 Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com> Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Mike Frysinger <vapier@google.com>
* Use 'gitfile' in submodule checkoutsKaushik Lingarkar2025-02-041-7/+42
| | | | | | | | | | | | | | | | | | This change takes another step towards ensuring Git can understand repo's submodules to some extent. Replace the old '.git' symlink with gitfile[1] pointing to the bare checkout of the submodule. This is required for Git's 'recurse submodules' opts to work with repo's submodules as '.git' is expected to be writable by Git when recursing over submodules. [1] https://git-scm.com/docs/gitrepository-layout#_description Change-Id: I52d15451768ee7bd6db289f4d2b3be5907370d42 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/446181 Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
* Update internal filesystem layout for submodulesKaushik Lingarkar2025-02-043-2/+34
| | | | | | | | | | | | | | Change the bare checkout directory for submodules from 'subprojects' to 'modules'. Git expects bare submodule checkouts to be in the 'modules' directory. If old subproject directories are found, they will be migrated to the new modules directory. This change is the first step in ensuring Git can understand repo's submodules to some extent. Change-Id: I385029f1bb55d040616d970d6ffb4bb856692520 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/444881 Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
* Revert "sync: skip network half on repo upgrade"Josip Sokcevic2025-01-311-1/+1
| | | | | | | | | | | | | | | This reverts commit 61224d01fa29bcf54dd6d521e09e09a8c0da77fe. Reason for revert: the manifest will be updated during in the post-upgrade process, and that can result in a missing object in LocalHalf, since NetworkHalf is not skipped. Bug: b/392979411 Change-Id: I8a46e5b54093ed78285c8b30f000bb08a8244179 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/450181 Tested-by: Josip Sokcevic <sokcevic@chromium.org> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Scott Lee <ddoman@google.com>
* docs: smart-sync: split out & expand detailsMike Frysinger2025-01-303-20/+131
| | | | | | | | | | | | The existing documentation on smart-sync behavior is a bit light on details, and out of date wrt what the code actually does. Start a dedicated document and fill it out more. Change-Id: I1a8a3ac6edf9291d72182ad55db865035d9b683e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/450002 Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
* run_tests: update to python 3.11 & pytest 8.3.4Mike Frysinger2025-01-301-17/+11
| | | | | | | | Change-Id: Iffe45d85a54dc380cdd37bbbbe64b058eacad0a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/449901 Reviewed-by: Josip Sokcevic <sokcevic@chromium.org> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* git_config: prefer XDG config locationflexagoon2025-01-231-0/+14
| | | | | | | | | | | | | Currently, repo ignores the XDG path for the git config file, and creates a new one in the user's home directory. This commit changes the behavior to prefer the XDG path if it exists, which matches git behavior and avoids littering the home directory. Bug: 40012443 Change-Id: Icd3ec6db6b0832f47417bbe98ff9461306b51297 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/448385 Tested-by: lmaor xenix <25misha52@gmail.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* sync: Recover from errors during read-treev2.51Josip Sokcevic2025-01-161-5/+12
| | | | | | | | | | | | | | | | | | | | | When repo is initializing a git repository, it calls `git read-tree`. During such operation, git is restoring workspace based on the current index. However, some things can go wrong: a user can run out of disk space, or, in case of partial clone, user may no longer reach the remote host. That will leave affected repository in a bad state with partially checked out workspace. The follow up repo sync won't try to fix such state. This change removes .git symlink, which will force the next `repo sync` to redo Git repository setup. Bug: b/363171216 Bug: b/390161127 Change-Id: I57db4b6cae0ef21826dc7cede4d3bf02cfc3d955 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447801 Reviewed-by: Scott Lee <ddoman@google.com> Tested-by: Josip Sokcevic <sokcevic@chromium.org> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
* gc: Add repack optionJosip Sokcevic2025-01-141-2/+160
| | | | | | | | | | | | | When a repository is partially cloned, no longer needed blobs are never removed. To reclaim some of disk space, allow user to pass --repack which affects only repositories with filter=blob:none and if projects are not shared. Change-Id: I0608172c9eff82fb8a6b6ef703eb109fedb7a6cc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447722 Commit-Queue: Josip Sokcevic <sokcevic@chromium.org> Tested-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Scott Lee <ddoman@google.com>
* gc: extract deletion from Execute methodJosip Sokcevic2025-01-141-7/+14
| | | | | | | | Change-Id: Icef4f28fbdb9658892611def7589f5eba43c952c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447721 Reviewed-by: Scott Lee <ddoman@google.com> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org> Tested-by: Josip Sokcevic <sokcevic@chromium.org>
* sync: Handle KeyboardInterrupt during checkoutJosip Sokcevic2025-01-081-0/+2
| | | | | | | | | | | KeyboardInterrupt is handled during NetworkHalf. This patch handles KeyboardInterrupt during LocalHalf. Bug: b/372069163 Change-Id: I26847f7ca3cdf1fe57b265b4f6b18cc8102d2921 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447401 Tested-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Gavin Mak <gavinmak@google.com>
* sync: skip network half on repo upgradeJosip Sokcevic2024-12-181-1/+1
| | | | | | | | | | | | | | | When repo upgrades itself, it will restart itself and rerun sync command. At that point, we know that network half is already done and we can just proceed with local half. R=ddoman@google.com Bug: b/377567091 Change-Id: I77205b1f2df19891597347d55283a617de3c6634 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/446201 Reviewed-by: Scott Lee <ddoman@google.com> Tested-by: Josip Sokcevic <sokcevic@chromium.org> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
* gc: Introduce new command to remove old projectsv2.50.1Josip Sokcevic2024-12-184-2/+193
| | | | | | | | | | | | | | | | | | | When projects are removed from manifest, they are only removed from worktree and not from .repo/projects and .repo/project-objects. Keeping data under .repo can be desired if user expects deleted projects to be restored (e.g. checking out a release branch). Android has ongoing effort to remove many stale projects and this change allows users to easily free-up their disk space. Bug: b/344018971 Bug: 40013312 Change-Id: Id23c7524a88082ee6db908f9fd69dcd5d0c4f681 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/445921 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@chromium.org>
* sync: Delete symlinks relative to client topdirv2.50.0v2.50Josip Sokcevic2024-12-161-1/+4
| | | | | | | | | | | | | If repo sync is invoked outside the repo root, and the latest manifest removes symlinks, repo incorrectly tries to remove symlink - it starts from `cwd` instead of the repo root. Bug: b/113935847 Bug: 40010423 Change-Id: Ia50ea70a376e38c94389880f020c80da3c3f453c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/445901 Tested-by: Josip Sokcevic <sokcevic@chromium.org> Reviewed-by: Gavin Mak <gavinmak@google.com>
* Add smoke test for subcmd forallFredrik de Groot2024-12-111-0/+156
| | | | | | | | | | | | | | | | | | | After some refactoring earlier, the forall command was broken briefly, returning after only one run instead of after all projects. This test, albeit simple in nature, would have caught that. Due to the somewhat demanding nature of forall, a lot more setup was needed than expected but seems to do its job now so hopefully it catches similar stuff in the future. Change-Id: I51e161ff0e7e31a65401211c376f319bda504532 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/445461 Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Fredrik de Groot <fredrik.de.groot@haleytek.com>
* Remove gitc support from repoJosip Sokcevic2024-12-037-168/+12
| | | | | | | | | | gitc is no longer available. Change-Id: I0cbfdf936832f2cdd4876104ae3cc5a6e26154e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/444841 Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* branches: Escape percent signs in branch namesJosip Sokcevic2024-12-031-1/+4
| | | | | | | | | | | | | | If a branch name contains a percent sign, it will be interpreted as a placeholder and color.py will fail to format it. To avoid this, escape the percent signs prior to calling Coloring method. Bug: b/379090488 Change-Id: Id019c776bbf8cbed5c101f2773606f1d32c9e057 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/443801 Reviewed-by: Scott Lee <ddoman@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* sync: Do not fail to sync a manifest with no projectsPeter Kjellerstedt2024-11-261-1/+1
| | | | | | | | | | | | | | | | Since commit 454fdaf1191c87e5c770ab865a911e10e600e178 (v2.48), syncing a manifest without any projects would result in: Repo command failed: RepoUnhandledExceptionError Number of processes must be at least 1 Bug: 377546300 Change-Id: Iaa2f6a3ac64542ad65a19c0eef449f53c09cae67 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/443442 Reviewed-by: Erik Elmeke <erik@haleytek.corp-partner.google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
* Fix event log command event hierarchy.Josip Sokcevic2024-11-223-18/+16
| | | | | | | | | | | | command should be cmd_name, to match what git is emitting. This also fixes arguments, so that only relevant arguments are passed instead of the entire sys.args, which will contain wrapper information Change-Id: Id436accfff511292ec2c56798fffb2306dda38fc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/443741 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* Make repo installation work without .gitJosip Sokcevic2024-11-181-4/+5
| | | | | | | | | | | Some tools like jj and cog will not have .git. This change makes it possible to run all repo commands in such setups. Change-Id: I7f3845dc970fbaa731c31e0aa48355a4b56ed3a6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/442821 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* sync: fix connection error on macOSv2.49.3Kuang-che Wu2024-11-061-0/+11
| | | | | | | | | | | | | | | | With a large number of sync workers, the sync process may fail on macOS due to connection errors. The root cause is that multiple workers may attempt to connect to the multiprocessing manager server at the same time when handling the first job. This can lead to connection failures if there are too many pending connections, exceeding the socket listening backlog. Bug: 377538810 Change-Id: I1924d318d076ca3be61d75daa37bfa8d7dc23ed7 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/441541 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* upload: Return correct tuple values in _ProcessResultsv2.49.2Josip Sokcevic2024-10-311-2/+2
| | | | | | | | | | | | | | | Incorrect tuple values were returned with http://go/grev/440221 - instead of returning (Project, ReviewableBranch), _ProcessResults was returning (int, ReviewableBranch). R=jojwang@google.com Bug: 376731172 Change-Id: I75205f42fd23f5ee6bd8d0c15b18066189b42bd9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/441121 Reviewed-by: Sam Saccone <samccone@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* worktree: Do not try to fix relative pathsv2.49.1Allen Webb2024-10-301-18/+23
| | | | | | | | | | | | | --worktree was broken with incorrect paths in the .git files whenever the local copy of git populated gitdir with relative paths instead of absoulte paths. Bug: 376251410 Change-Id: Id32dc1576315218967de2a9bfe43bf7a5a0e7aa6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440801 Commit-Queue: Allen Webb <allenwebb@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Allen Webb <allenwebb@google.com>
* forall: Fix returning results earlyJosip Sokcevic2024-10-301-3/+3
| | | | | | | | | | | | | rc should be returned only after all results are processed. R=jojwang@google.com Bug: b/376454189 Change-Id: I8200b9954240dd3e8e9f2ab82494779a3cb38627 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440901 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Joanna Wang <jojwang@google.com>
* Use full name of the revision when checking dest-branchv2.49joehsu2024-10-281-2/+5
| | | | | | | | | | | | | | | | | | The manifest usually doesn't sepecify the revision with the full name (e.g. refs/heads/REV). However, when checking if the name of the merge branch, full name is used on the merge branch. The CL use full name of revision when comparing it with the merge branch. Bug: b/370919047 Test: repo upload on a project with `dest-branch` set Change-Id: Ib6fa2f7246beb5bae0a26a70048a7ac03b6c5a2f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438401 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Joe Hsu <joehsu@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* Add REPO_SKIP_SELF_UPDATE check in syncFredrik de Groot2024-10-282-1/+6
| | | | | | | | | | | | | | | | | | | The command _PostRepoFetch will try to self update during repo sync. That is beneficial but adds version uncertainty, fail potential and slow downs in non-interactive scenarios. Conditionally skip the update if env variable REPO_SKIP_SELF_UPDATE is defined. A call to selfupdate works as before, meaning even with the variable set, it will run the update. Change-Id: Iab0ef55dc3d3db3cbf1ba1f506c57fbb58a504c3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439967 Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* manifest: add optional base check on remove and extendFredrik de Groot2024-10-283-0/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds an optional, built-in checker for guarding against patches hanging on wrong base revisions, which is useful if a lower layer of the manifest changes after a patch was done. When adding a patch with a new revision using extend-project or remove-project/project: C---D---E patches in project bla / A---B project bla in manifest state 1 <extend-project name="bla" revision="E" base-rev="B"> If project bla gets updated, in a new snap ID or by a supplier or similar, to a new state: C---D---E patches in project bla / A---B---F---G project bla in manifest state 2 Parsing will fail because revision of bla is now G, giving the choice to create a new patch branch from G and updating base-rev, or keeping previous branch for some reason and only updating base-rev. Intended for use in a layered manifest with hashed revisions. Named refs like branches and tags also work fine when comparing, but will be misleading if a branch is used as base-rev. Change-Id: Ic6211550a7d3cc9656057f6a2087c505b40cad2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436777 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* [event_log] Stop leaking semaphore resourcesJosip Sokcevic2024-10-241-2/+10
| | | | | | | | | | | | With the global state and fork, we are left with uncleaned resources. Isolate mulitprocessing.Value in a function so we stop the leak. Bug: 353656374 Change-Id: If50bb544bda12b72f00c02bc1d2c0d19de000b88 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440261 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* progress: always show done messageKuang-che Wu2024-10-241-9/+4
| | | | | | | | | | | | The done message was omitted if the task is shorter than 0.5s. This might confuse users. Bug: b/371638995 Change-Id: I3fdd2cd8daea16d34fba88457d09397fff71af15 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440222 Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* subcmds: reduce multiprocessing serialization overheadKuang-che Wu2024-10-2311-172/+228
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Follow the same approach as 39ffd9977e to reduce serialization overhead. Below benchmarks are tested with 2.7k projects on my workstation (warm cache). git tracing is disabled for benchmark. (seconds) | v2.48 | v2.48 | this CL | this CL | | -j32 | | -j32 ----------------------------------------------------------- with clean tree state: branches (none) | 5.6 | 5.9 | 1.0 | 0.9 status (clean) | 21.3 | 9.4 | 19.4 | 4.7 diff (none) | 7.6 | 7.2 | 5.7 | 2.2 prune (none) | 5.7 | 6.1 | 1.3 | 1.2 abandon (none) | 19.4 | 18.6 | 0.9 | 0.8 upload (none) | 19.7 | 18.7 | 0.9 | 0.8 forall -c true | 7.5 | 7.6 | 0.6 | 0.6 forall -c "git log -1" | 11.3 | 11.1 | 0.6 | 0.6 with branches: start BRANCH --all | 21.9 | 20.3 | 13.6 | 2.6 checkout BRANCH | 29.1 | 27.8 | 1.1 | 1.0 branches (2) | 28.0 | 28.6 | 1.5 | 1.3 abandon BRANCH | 29.2 | 27.5 | 9.7 | 2.2 Bug: b/371638995 Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* sync: reduce multiprocessing serialization overheadKuang-che Wu2024-10-232-87/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: - Manifest object is large (for projects like Android) in terms of serialization cost and size (more than 1mb). - Lots of Project objects usually share only a few manifest objects. Before this CL, Project objects were passed to workers via function parameters. Function parameters are pickled separately (in chunk). In other words, manifests are serialized again and again. The major serialization overhead of repo sync was O(manifest_size * projects / chunksize) This CL uses following tricks to reduce serialization overhead. - All projects are pickled in one invocation. Because Project objects share manifests, pickle library remembers which objects are already seen and avoid the serialization cost. - Pass the Project objects to workers at worker intialization time. And pass project index as function parameters instead. The number of workers is much smaller than the number of projects. - Worker init state are shared on Linux (fork based). So it requires zero serialization for Project objects. On Linux (fork based), the serialization overhead is O(projects) --- one int per project On Windows (spawn based), the serialization overhead is O(manifest_size * min(workers, projects)) Moreover, use chunksize=1 to avoid the chance that some workers are idle while other workers still have more than one job in their chunk queue. Using 2.7k projects as the baseline, originally "repo sync" no-op sync takes 31s for fetch and 25s for checkout on my Linux workstation. With this CL, it takes 12s for fetch and 1s for checkout. Bug: b/371638995 Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921 Tested-by: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* Fix incremental syncs for prjs with submodulesKaushik Lingarkar2024-10-181-1/+3
| | | | | | | | | | | | | | | | | | | When performing an incremental sync (re-running repo init with an updated manifest revision) with --fetch-submodules or sync-s=true, there is an attempt to get a list of all projects (including submodules) before projects are actually fetched. However, we can only list submodules of a project if we have already fetched its revision. Instead of throwing an error when we don't have the revision, assume there are no submodules for that project. In the sync cmd, we already update the list of projects to include submodules after fetching superprojects. Change-Id: I48bc68c48b5b10117356b18f5375d17f9a89ec05 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439761 Commit-Queue: Kaushik Lingarkar <kaushik.lingarkar@linaro.org> Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-13 16:53:49 +0000