summaryrefslogtreecommitdiffstats
path: root/subcmds
Commit message (Collapse)AuthorAgeFilesLines
* forall: allow interactive commands with -j1v2.13.7Mike Frysinger2021-03-181-1/+16
| | | | | | | | | | | | | Historically forall has been interactive since it ran in serial. Recent rework in here dropped that to enable parallel processing. Restore support for interactive commands when running -j1 or with an explicit --interactive option. Bug: https://crbug.com/gerrit/14256 Change-Id: I502007186f771914cfd7830846a4e1938b5e1f38 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300722 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: fix reporting of failed local checkoutsv2.13.6Mike Frysinger2021-03-151-1/+1
| | | | | | | | | | | The refactor to multiprocessing broke status reporting slightly when checking out projects. Make sure we mark the step as failed if any of the projects failed, not just when --fail-fast is set. Change-Id: I0efb56ce83b068b2c334046df3fef23d797599c9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299882 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: pass groups to ToXml method.Raman Tenneti2021-03-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added the following methods to XmlManifest class. + GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower() + GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func. + Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr. + Used the above methods to get groups in command.py::GetProjects and part of init.py. TODO: clean up these funcs to take structured group data more instead of passing strings around everywhere that need parsing. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL and verified prebuilts/fullsdk-linux directory has all the folders. Tested repo init and repo sync with --use-superproject and without --use-superproject argument. $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j32 Bug: [google internal] b/181804931 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422 Reviewed-by: Jonathan Nieder <jrn@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* init: expose --worktree optionMike Frysinger2021-03-091-3/+1
| | | | | | | | | | | There's a few rough edges here still, but no known corruption ones, so open it up a bit for people to experiment with. Bug: https://crbug.com/gerrit/11486 Change-Id: I81e0122ab6d3e032c546c8239dd4f03740676e80 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299242 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: Display status messages during repo init/sync.v2.13.3Raman Tenneti2021-03-042-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Superproject objects accept the optional argument “quiet”. The following progress messages are displayed if quiet is false. Displayed the following message whenever we find we have to make a new folder (aka new remote), because if you started with repo init android and later do googleplex-android that is when it will be slow. "<location>: Performing initial setup for superproject; this might take several minutes.". After fetch completion, added the following notification: "<location>: Initial setup for superproject completed." Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL. $ repo_dev init -u persistent-https://googleplex-android.git.corp.google.com/platform/manifest -b rvc-dev --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject Bug: [google internal] b/181178282 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia7fb85c6fb934faaa90c48fc0c55e7f41055f48a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299122 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* forall: greatly speed up processing overheadv2.13Mike Frysinger2021-03-011-42/+17
| | | | | | | | | | | | | | | | With the recent commit 0501b29e7ae072e0b10ea9ddd913ec6d5975f690 ("status: Use multiprocessing for `repo status -j<num>` instead of threading"), the limitation with project serialization no longer applies. It turns out that ad-hoc logic is expensive. In the CrOS checkout (~1000 projects w/8 jobs by default), it adds about ~7sec overhead to all invocations. With a fast nop run: time repo forall -j8 -c true This goes from ~11sec to ~4sec -- more than 50% speedup. Change-Id: Ie6bcccd21eef20440692751b7ebd36c890d5bbcc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298724 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: simplify arg passing to worker childrenMike Frysinger2021-03-011-20/+6
| | | | | | | | | | | The ProjectArgs function can be inlined which simplifies it quite a bit. We shouldn't need the custom exception handling here either. This also makes the next commit easier to review. Change-Id: If3be04f58c302c36a0f20b99de0f67e78beac141 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298723 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: move nested func out to the classMike Frysinger2021-03-011-5/+7
| | | | | | | | | | | This is in preparation for simplifying the jobs support. The nested function is referenced in the options object which can't be pickled, so pull it out into a static method instead. Change-Id: I01d3c4eaabcb8b8775ddf22312a6e142c84cb77d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298722 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* download: handle shared projects a bit betterMike Frysinger2021-03-011-2/+19
| | | | | | | | | | | | | | If a manifest checksout a project multiple times, repo download isn't able to accurately pick the right project. We were just picking the first result which could be a bit random for the user. If we hit that situation, check if the cwd is one of the projects, and if it isn't, we emit an error and tell the user it's an ambiguous request. Bug: https://crbug.com/gerrit/13070 Change-Id: Id1059b81330229126b48c7312569b37504808383 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298702 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* checkout: add --jobs supportMike Frysinger2021-02-271-10/+27
| | | | | | | | | | | Use multiprocessing to run in parallel. When operating on multiple projects, this can speed things up. Across 1000 repos, it goes from ~9sec to ~5sec with the default -j8. Change-Id: Ida6dd565db78ff7bac0ecb25d2805e8a1bf78048 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297982 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: switch local checkout to multiprocessingMike Frysinger2021-02-271-124/+52
| | | | | | | | | | | | | | | | This avoids GIL limitations with using threads for parallel processing. In a CrOS checkout with ~1000 repos, the nop case goes from ~6 sec down to ~4 sec with -j8. Not a big deal, but shows that this actually works to speed things up unlike the threading model. This reworks the checkout logic to return results for processing in the main thread instead of leaving every thread to do its own processing. Bug: https://crbug.com/gerrit/12389 Change-Id: I143e5e3f7158e83ea67e2d14e5552153a874248a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298063 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* abandon/start: add --jobs supportMike Frysinger2021-02-272-38/+83
| | | | | | | | | | | Use multiprocessing to run in parallel. When operating on multiple projects, this can greatly speed things up. Across 1000 repos, it goes from ~30sec to ~3sec with the default -j8. Change-Id: I0dc62d704c022dd02cac0bd67fe79224f4e34095 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297484 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* grep: move nested func out to the classMike Frysinger2021-02-251-37/+35
| | | | | | | | | | | This is in preparation for adding jobs support. The nested function is referenced in the options object which can't be pickled, so pull it out into a static method instead. Change-Id: I280ed2bf26390a0203925517a0d17c13053becaa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297983 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: capture all git output by defaultMike Frysinger2021-02-251-0/+7
| | | | | | | | | | | | The default sync output should show a progress bar only for successful commands, and the error output for any commands that fail. Implement that policy here. Bug: https://crbug.com/gerrit/11293 Change-Id: I85716032201b6e2b45df876b07dd79cb2c1447a5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297905 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress/sync: include active number of jobsMike Frysinger2021-02-251-11/+16
| | | | | | | | | | Provide a bit more info to users that things are actively running. Bug: https://crbug.com/gerrit/11293 Change-Id: Ie8eeaa8804d1ca71cf5c78ad850fa2d17d26208c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297904 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress: always enable always_print_percentageMike Frysinger2021-02-251-2/+1
| | | | | | | | | | | | | | | | | | | The idea for skipping some progress updates was to avoid spending too much time on the progress bar itself. Unfortunately, for large projects (100s if not 1000s) of repos, we get into the situation with large/slow checkouts that we skip showing updates when a repo finishes, but not enough repos finished to increase the percent. Since the progress bar should be relatively fast compared to the actual network & local dick operations, have it show an update whenever the caller requests it. A test with ~1000 repos shows that the progress bar in total adds <100ms. Bug: https://crbug.com/gerrit/11293 Change-Id: I708a0c4bd923c59c7691a5b48ae33eb6fca4cd14 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297903 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: support -b HEAD as a shortcut to "the default"Mike Frysinger2021-02-251-4/+8
| | | | | | | | | | | When people switch to non-default branches, they sometimes want to switch back to the default, but don't know the exact name for that branch. Add a -b HEAD shortcut for that. Change-Id: I090230da25f9f5a169608115d483f660f555624f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297843 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: use superproject if manifest's config has superproject enabled.Raman Tenneti2021-02-251-1/+3
| | | | | | | | | | | | | | | | | | | If --use-superproject is passed as argument to "repo init", then --use-superproject need not be specified during "repo sync". Tested the code with the following commands. $ time repo_dev sync -c -j32 ... WARNING: --use-superproject is experimental and not for general use Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Change-Id: Ibb33f3038a2515f74a6c4f7cb785d354b26ee680 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298102 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Ian Kasprzak <iankaz@google.com>
* sync: update event is_set APIMike Frysinger2021-02-231-7/+7
| | | | | | | | | Python 3 renamed this method from isSet to is_set. Change-Id: I8f9bb0b302d55873bed3cb20f2d994fa2d082157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297742 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* diff: add --jobs supportMike Frysinger2021-02-231-4/+38
| | | | | | | | | Use multiprocessing to run diff in parallel. Change-Id: I61e973d9c2cde039d5eebe8d0fe8bb63171ef447 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297483 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* forall: rewrite parallel logicMike Frysinger2021-02-221-66/+40
| | | | | | | | | | | | | | | | | This fixes intermingling of parallel jobs and simplifies the code by switching to subprocess.run. This also provides stable output in the order of projects by returning the output as a string that the main loop outputs. This drops support for interactive commands, but it's unclear if anyone was relying on that, and the default behavior (-j2) made that unreliable. If it turns out someone still wants this, we can look at readding it. Change-Id: I7555b4e7a15aad336667292614f730fb7a90bd26 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297482 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: improve pool logicMike Frysinger2021-02-221-15/+10
| | | | | | | | | | Use a pool contextmanager to take care of the messy details like properly cleaning it up when aborting. Change-Id: I264ebb591c2e67c9a975b6dcc0f14b29cc66a874 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297243 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* status: improve parallel execution stabilityMike Frysinger2021-02-222-13/+13
| | | | | | | | | | | | | | | | | The status command runs a bunch of jobs in parallel, and each one is responsible for writing to stdout directly. When running many noisy jobs in parallel, output can get intermingled. Pass down a StringIO buffer for writing to so we can return the entire output as a string so the main job can handle displaying it. This fixes interleaved output as well as making the output stable: we always display results in the same project order now. By switching from map to imap, this ends up not really adding any overhead. Bug: https://crbug.com/gerrit/12231 Change-Id: Ic18b07c8074c046ff36e306eb8d392fb34fb6eca Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297242 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* command: unify --job option & default valuesMike Frysinger2021-02-224-25/+13
| | | | | | | | | | | | Extend the Command class to support adding the --jobs option to the parser if the command declares it supports running in parallel. Also pull the default value used for the number of local jobs into the command module so local commands can share it. Change-Id: I22b0f8d2cf69875013cec657b8e6c4385549ccac Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297024 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* use simpler super() magicMike Frysinger2021-02-191-2/+2
| | | | | | | | | | Python 3 has a simpler super() style so switch to it to make the code a little simpler and to stop pylint warnings. Change-Id: I1b3ccf57ae968d56a9a0bcfc1258fbd8bfa3afee Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297383 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: make --manifest-url flag optionalMike Frysinger2021-02-181-4/+16
| | | | | | | | | | | Since the --manifest-url flag is always required when creating a new checkout, allow the url to be specified via a positional argument. This brings it a little closer to the `git clone` UI. Change-Id: Iaf18e794ae2fa38b20579243d067205cae5fae2f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297322 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* git_command: rework stdin handlingMike Frysinger2021-02-171-3/+1
| | | | | | | | | | | We only provide input to GitCommand in one place, so inline the logic to be more synchronous and similar to subprocess.run. This makes the code simpler and easier to understand. Change-Id: Ibe498fedf608774bae1f807fc301eb67841c468b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297142 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* trim redundant pass statementsMike Frysinger2021-02-162-2/+0
| | | | | | | | | Clean up a few linter warnings. Change-Id: I531d0263a202435d32d83d87ec24998f4051639c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297062 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: use subprocess.run to verify tagsMike Frysinger2021-02-161-13/+4
| | | | | | | | | The code is a bit simpler & easier to reason about. Change-Id: I149729c7d01434b08b58cc9715dcf0f0d11201c2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297022 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: added --use-superproject option to clone superproject.v2.12.2Raman Tenneti2021-02-112-19/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added --no-use-superproject to repo and init.py to disable use of manifest superprojects. Replaced the term "sha" with "commit id". Added _GetBranch method to Superproject object. Moved shared code between init and sync into SyncSuperproject function. This function either does git clone or git fetch. If git fetch fails it does git clone. Changed Superproject constructor to accept manifest, repodir and branch to avoid passing them to multiple functions as argument. Changed functions that were raising exceptions to return either True or False. Saved the --use-superproject option in config as repo.superproject. Updated internal-fs-layout.md document. Updated the tests to work with the new API changes in Superproject. Performance for the first time sync has improved from 20 minutes to around 15 minutes. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL. $ repo init took around 20 seconds longer because of cloning of superproject. $ time repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject ... real 0m35.919s user 0m21.947s sys 0m8.977s First run $ time repo sync --use-superproject ... real 16m41.982s user 100m6.916s sys 19m18.753s No difference in repo sync time after the first run. Bug: [google internal] b/179090734 Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Change-Id: I12df92112f46e001dfbc6f12cd633c3a15cf924b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296382 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* diffmanifests/sync: simplify repodir lookupMike Frysinger2021-02-112-3/+3
| | | | | | | | | | We have access to repodir on the command object itself, so we don't need to pull it indirectly out of the manifest object. Change-Id: I8688fb1c84979825efa966dc787e78c6f7ba3823 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296542 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* git_config: add SetBoolean helperMike Frysinger2021-02-111-7/+7
| | | | | | | | | A little sugar simplifies the code a bit. Change-Id: Ie2b8a965faa9f9ca05c7be479d03e8e073cd816d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296522 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: pass --bare option when doing git clone of superproject.v2.12.1Raman Tenneti2021-02-081-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | Changed "git pull" to "git fetch" as we are using --bare option. Used the following command to fetch: git fetch origin +refs/heads/*:refs/heads/* --prune Pass --branch argument to Superproject's UpdateProjectsRevisionId function. Returned False/None when directories don't exist instead of raise GitError exception from _Fetch and _LsTree functions. The caller of Fetch does Clone if Fetch fails. Tested the code with the following commands. $ ./run_tests -v Tested the init and sync code by copying all the repo changes into my Android AOSP checkout and running repo sync with --use-superproject option. Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Tested-by: Raman Tenneti <rtenneti@google.com> Change-Id: I3e441ecdfc87c735f46eff0eb98efa63cc2eb22a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296222 Reviewed-by: Mike Frysinger <vapier@google.com>
* sync: superproject performance changes.Raman Tenneti2021-02-071-36/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After updating all project’s revsionIds with the SHAs from superproject, write the updated manifest into superproject_override.xml file. Reload that file for future Reloads. This file is created in exp-superproject directory. Moved most of the code that is superproject specific into git_superproject.py and wrote test code. If git pull fails, did a git clone of the superproject. We saw performance gains for consecutive repo sync's. The time to sync went down from around 120 secs to 40 secs when repo sync is executed consecutively. Tested the code with the following commands. $ ./run_tests -v tests/test_git_superproject.py $ ./run_tests -v Tested the sync code by copying all the repo changes into my Android AOSP checkout and doing a repo sync --use-superproject twice. First run $ time repo sync --use-superproject ... real 21m3.745s user 97m59.380s sys 19m11.286s After two consecutive sync runs $ time repo sync -c -j8 --use-superproject real 0m39.626s user 0m29.937s sys 0m38.155s Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Tested-by: Raman Tenneti <rtenneti@google.com> Change-Id: Id79a0d7c4d20babd65e9bd485196c6f8fbe9de5e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296082 Reviewed-by: Ian Kasprzak <iankaz@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* sync: Disable info about disabling pruning when quietAnders Björklund2021-01-221-2/+3
| | | | | | | | | | If you have a lot of shared projects, it spams. Bug: https://crbug.com/gerrit/13961 Change-Id: If3f5baef65930830af9a2cd01a1b593dd518ab09 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/294049 Tested-by: Anders Björklund <anders.bjorklund.2@volvocars.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* sync: Added --use-superproject option and support for superproject.v2.12Raman Tenneti2021-01-211-1/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Added "--use-superporject" option to sync.py to fetch project SHAs from superproject. If there are any missing projects in superprojects, it prints the missing entries and exits. If there are no missing entries, it will use SHAs from superproject to fetch the projects from git. Tested the code with the following commands. $ ./run_tests tests/test_manifest_xml.py $ ./run_tests -v tests/test_git_superproject.py $ ./run_tests -v Tested the sync code by copying all the repo changes into my Android AOSP checkout and adding <superporject> tag to default.xml. With local modification to the code to print the status, .../WORKING_DIRECTORY$ repo sync --use-superproject repo: executing 'git clone' url: sso://android/platform/superproject repo: executing 'git ls-tree' Success: [] Bug: https://crbug.com/gerrit/13709 Tested-by: Raman Tenneti <rtenneti@google.com> Change-Id: Id18665992428dd684c04b0e0b3a52f46316873a0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293822 Reviewed-by: Mike Frysinger <vapier@google.com>
* version: add remote tracking informationMike Frysinger2021-01-081-0/+2
| | | | | | | | | | This tells us what --repo-rev the user is using. Bug: https://crbug.com/1164415 Change-Id: Idb6c48e6ca5a4783c529717e6be38266bf7038b0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293143 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* drop pyversion & is_python3 checkingMike Frysinger2021-01-064-38/+6
| | | | | | | | | | | We're committed to Python 3 at this point, so purge all the is_python3 related dynamic checks. Bug: https://crbug.com/gerrit/10418 Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-0628-87/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Add parallelism to 'branches' commandChris McDonald2020-12-141-4/+42
| | | | | | | | | | | | | | | Spread the operation of querying which local branches exist across a pool of processes and build the name map of projects -> branches as these tasks finish rather than blocking on the entire query. The search operations are submitted in batches to reduce the overhead of interprocess communication. The `chunksize` argument used to control this batch size was selected by incrementing through powers of two until it stopped being faster. Change-Id: Ie3d7f799ee8e83e5058536caf53e2979175408b7 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/291342 Tested-by: Chris Mcdonald <cjmcdonald@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* init: reset hard to --repo-revMike Frysinger2020-12-061-1/+1
| | | | | | | | | | | | | | | | When updating the tracking ref to whatever the user requested, make sure we reset state completely rather than trying to update the ref to it. This avoids confusing git as to the current state of the tree, and is more inline with user intentions: if they made a local change to the checkout, but ran repo init with a specific rev, we shouldn't stay wedged forever until they manually clean it all up. Bug: https://crbug.com/gerrit/12801 Change-Id: Ieba8d9c15781b4d0649bf01c7460694da63387b2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/290923 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Concentrate the RepoHook knowledge in the RepoHook classRemy Bohmer2020-11-231-53/+11
| | | | | | | | | | | | | | | | The knowledge about running hooks and all its exception handling is scattered over multiple files. This makes the code harder to read, but also it requires duplication of logic in case other RepoHooks are added to different commands. This refactoring also creates uniform behavior of the hooks across multiple commands and it guarantees the re-use of the same arguments on all of them. Signed-off-by: Remy Bohmer <github@bohmer.net> Change-Id: Ia4d90eab429e4af00943306e89faec8db35ba29d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/277562 Tested-by: Remy Bohmer <oss@bohmer.net> Reviewed-by: Mike Frysinger <vapier@google.com>
* manifest_xml: refactor manifest parsing from client managementMike Frysinger2020-11-185-10/+10
| | | | | | | | | | | | | | We conflate the manifest & parsing logic with the management of the repo client checkout in a single class. This makes testing just one part (the manifest parsing) hard as it requires a full checkout too. Start splitting the two apart into separate classes to make it easy to reason about & test. Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: respect --force-sync when fetching manifest project updatesErwan Yvin2020-11-171-0/+1
| | | | | | | | | | | The --force-sync option was being passed down for all updates except for the manifest project, so add that there too. Bug: https://crbug.com/gerrit/11034 Change-Id: I33818b652f828c6b847dbc70f1fedfac5ac17bbe Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/228146 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* manifest: add a --json output optionMike Frysinger2020-11-171-4/+26
| | | | | | | | | | | | Sometimes parsing JSON is easier than parsing XML, especially when the XML format is limited (which ours is). Add a --json option to the manifest command to quickly emit that form. Bug: https://crbug.com/gerrit/11743 Change-Id: Ia2bb254a78ae2b70a851638b4545fcafe8c1a76b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280436 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* upload: improve tip for fixing upload remoteMike Frysinger2020-11-161-4/+4
| | | | | | | | | | | | Instead of assuming the repo client is tracking the "master" branch of the manifest repo, use the existing info we have to display the right info to the user. Bug: https://crbug.com/gerrit/13339 Change-Id: I8b265f4b2e075fdc41909b1f3dff9aee87384353 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/287279 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Move RepoHook class from project.py file to dedicated fileRemy Bohmer2020-11-031-1/+1
| | | | | | | | | | | | The project.py file is huge and contains multiple classes. By moving it to seperate class files the code becomes more readable and maintainable. Signed-off-by: Remy Bohmer <github@bohmer.net> Change-Id: Ida9d99d31751d627ae1ea0373418080696d2e14b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/281293 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Remy Bohmer <linux@bohmer.net>
* init: use the remote default manifest branchMike Frysinger2020-09-091-11/+15
| | | | | | | | | | | | | | | | Instead of hardcoding "master" as our default, use the remote server's default branch instead. For most people, this should be the same as "master" already. For projects moving to "main", it means we'll use the new name automatically rather than forcing people to use -b main. For repositories that never set up a default HEAD, we should still use the historical "master" default. Bug: https://crbug.com/gerrit/13339 Change-Id: I4117c81a760c9495f98dbb1111a3e6c127f45eba Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280799 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* status: Use multiprocessing for `repo status -j<num>` instead of threadingKimiyuki Onaka2020-09-091-33/+15
| | | | | | | | | | | | | This change increases the speed of the command with parallelization with processes. The parallelization with threads doesn't work well, and increasing the number of jobs to many (8 threads ~) didn't increase the speed. Possibly, the global interpreter lock of Python affects. Bug: https://crbug.com/gerrit/12389 Change-Id: Icbe5df8ba037dd91422b96f4e43708068d7be924 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/279936 Tested-by: Kimiyuki Onaka <kimiyuki@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* manifest: drop support for local_manifest.xmlMike Frysinger2020-09-081-1/+1
| | | | | | | | | | We deprecated this 8 years ago. Time to drop it to simplify the code as it'll help with refactoring in this module to not migrate it. Change-Id: I2deae5496d1f66a4491408fcdc95cd527062f8b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280798 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Michael Mortensen <mmortensen@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-31 20:32:57 +0000