summaryrefslogtreecommitdiffstats
path: root/tests/test_subcmds_sync.py
Commit message (Collapse)AuthorAgeFilesLines
* sync: Always use WORKER_BATCH_SIZEv2.48Josip Sokcevic2024-10-071-0/+24
| | | | | | | | | | | | | | | | | | | With 551285fa35ccd0836513e9cf64ee8d3372e5e3f4, the comment about number of workers no longer stands - dict is shared among multiprocesses and real time information is available. Using 2.7k projects as the baseline, using chunk size of 4 takes close to 5 minutes. A chunk size of 32 takes this down to 40s - a reduction of rougly 8 times which matches the increase. R=gavinmak@google.com Bug: b/371638995 Change-Id: Ida5fd8f7abc44b3b82c02aa0f7f7ae01dff5eb07 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438523 Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* sync: Fix sorting for nested projectsJosip Sokcevic2024-03-081-10/+35
| | | | | | | | | | | | | | | | | | | The current logic to create checkout layers doesn't work in all cases. For example, let's assume there are three projects: "foo", "foo/bar" and "foo-bar". Sorting lexicographical order is incorrect as foo-bar would be placed between foo and foo/bar, breaking layering logic. Instead, we split filepaths based using path delimiter (always /) and then use lexicographical sort. BUG=b:325119758 TEST=./run_tests, manual sync on chromiumos repository Change-Id: I76924c3cc6ba2bb860d7a3e48406a6bba8f58c10 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412338 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: George Engelbrecht <engeg@google.com>
* sync: Introduce git checkout levelsv2.42Josip Sokcevic2024-02-271-0/+26
| | | | | | | | | | | | | | | | | | | | | If a repo manifest is updated so that project B is placed within a project A, and if project A had content in new B's location in the old checkout, then repo sync could break depending on checkout order, since B can't be checked out before A. This change introduces checkout levels which enforces right sequence of checkouts while still allowing for parallel checkout. In an example above, A will always be checked out first before B. BUG=b:325119758 TEST=./run_tests, manual sync on ChromeOS repository Change-Id: Ib3b5e4d2639ca56620a1e4c6bf76d7b1ab805250 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410421 Tested-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Greg Edelston <gredelston@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Gavin Mak <gavinmak@google.com>
* sync: Fix partial sync false positiveMatt Schulte2023-11-301-0/+38
| | | | | | | | | | | | | | | | | | | | | In the case of a project being removed from the manifest, and in the path in which the project used to exist, and symlink is place to another project repo will start to warn about partial syncs when a partial sync did not occur. Repro steps: 1) Create a manifest with two projects. Project a -> a/ and project b -> b/ 2) Run `repo sync` 3) Remove project b from the manifest. 4) Use `link` in the manifest to link all of Project a to b/ Bug: 314161804 Change-Id: I4a4ac4f70a7038bc7e0c4e0e51ae9fc942411a34 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/395640 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Matt Schulte <matsch@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* tests: test_subcmds_sync.py: fix for py3.6 & 3.7Daniel Kutik2023-09-131-2/+6
| | | | | | | | | | | | | | | | tests/test_subcmds_sync.py::LocalSyncState::test_prune_removed_projects was failing in Python 3.6 and 3.7 due to topdir not being set with the following error message: TypeError: expected str, bytes or os.PathLike object, not MagicMock topdir is accessed from within PruneRemovedProjects(). Test: tox with Python 3.6 to 3.11 Change-Id: I7ba5144df0a0126c01776384e2178136c3510091 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/382816 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com> Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
* isort: format codebasev2.36Mike Frysinger2023-08-221-3/+4
| | | | | | | | Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* upload: Suggest full sync if hooks fail with partially synced treeGavin Mak2023-08-221-2/+2
| | | | | | | | | | | Pre-upload hooks may fail because of partial syncs. Bug: b/271507654 Change-Id: I124cd386c5af2c34e1dcaa3e86916624e235b1e3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383474 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
* sync: Ignore repo project when checking partial syncsGavin Mak2023-08-221-2/+20
| | | | | | | | | | | | The repo project is fetched at most once a day and should be ignored when checking if the tree is partially synced. Bug: b/286126621, b/271507654 Change-Id: I684ed1669c3b3b9605162f8cc9d57185bb3dfe8e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383494 Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* sync: Warn if partial sync state is detectedGavin Mak2023-08-101-0/+61
| | | | | | | | | | | | | | | Partial syncs are not supported and can lead to strange behavior like deleting files. Explicitly warn users on partial sync. Bug: b/286126621, b/271507654 Change-Id: I471f78ac5942eb855bc34c80af47aa561dfa61e8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/382154 Reviewed-by: Jason Chang <jasonnc@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* Refactor errors for sync commandJason Chang2023-08-021-0/+83
| | | | | | | | | | | | | | | | | | Per discussion in go/repo-error-update updated aggregated and exit errors for sync command. Aggregated errors are errors that result in eventual command failure. Exit errors are errors that result in immediate command failure. Also updated main.py to log aggregated and exit errors to git sessions log Bug: b/293344017 Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614 Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Jason Chang <jasonnc@google.com> Commit-Queue: Jason Chang <jasonnc@google.com>
* sync: Track last completed fetch/checkoutGavin Mak2023-07-281-0/+75
| | | | | | | | | | | | Save the latest time any project is fetched and checked out. This will be used to detect partial checkouts. Bug: b/286126621 Change-Id: I53b264dc70ba168d506076dbd693ef79a696b61d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380514 Commit-Queue: Gavin Mak <gavinmak@google.com> Reviewed-by: Joanna Wang <jojwang@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
* Format codebase with black and check formatting in CQGavin Mak2023-03-221-94/+121
| | | | | | | | | | | | Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* sync: fix manifest sync-j handlingMike Frysinger2022-12-081-0/+47
| | | | | | | | | | | | | Since --jobs defaults to 0, not None, we never pull the value out of the manifest. Treat values of 0 and None the same to fix. Bug: http://b/239712300 Bug: http://b/260908907 Change-Id: I9b1026682072366616825fd72f90bd90c10a252f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354254 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Sam Saccone <samccone@google.com>
* sync: clear preciousObjects when set in error.LaMont Jones2022-11-031-7/+48
| | | | | | | | | | | | | | If this is a project that is not using object sharing (there is only one copy of the remote project) then clear preciousObjects. To override this for a project, run: git config --replace-all repo.preservePreciousObjects true Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* sync: refactor use of self.manifestLaMont Jones2022-04-141-1/+1
| | | | | | | | | | | We need to iterate over multiple manifests, and generally use the outer_client.manifest for multi-manifest support. This refactors the use of self.manifest into a chosen manifest. Change-Id: I992f21d610c929675e99555ece9c38df4b635839 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334699 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* sync: respect `sync-c` manifest optionDaniel Andersson2022-04-081-0/+45
The documentation states that a `sync-c` attribute in the manifest file can set a default for whether only the current branch should be fetched or all branches. This seems to have been broken for some time. Commit 7356114 introduced the `--no-current-branch` CLI option and relied on getting `None` via `optparse` if neither `--current-branch` nor `--no-current-branch` was set to distinguish it from a boolean value. If `None` was received, it would read the value from the manifest option `sync-c`. The parsing went through the utility function `_GetCurrentBranchOnly` which returned `True` if `--current-branch` had been given on the command-line, or fell back on the "superproject" setting, which would either return `True` or `None`. This would incorrectly make `repo` fall back to the manifest setting even if the user had given `--no-current-branch` if no superproject was requested -- the manifest became "too powerful": Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No From manifest Yes True --current-branch No True --current-branch Yes True --no-current-branch No From manifest ← wrong --no-current-branch Yes True In commit 0cb6e92 the superproject configuration value reading changed from something that could return `None` to something that always returned a boolean. If it returned `False`, this would then incorrectly make `repo` ignore the manifest option even if neither `--current-branch` nor `--no-current-branch` had been given. The manifest default became useless: Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No False ← wrong Yes True --current-branch No True --current-branch Yes True --no-current-branch No False --no-current-branch Yes True By swapping the order in which the command-line option target and the superproject setting is evaluated, things should work as documented: Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No From manifest Yes True --current-branch No True --current-branch Yes True --no-current-branch No False --no-current-branch Yes True Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175 Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270 Reviewed-by: Mike Frysinger <vapier@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-06-25 10:32:27 +0000