summaryrefslogtreecommitdiffstats
path: root/subcmds/init.py
Commit message (Collapse)AuthorAgeFilesLines
* superproject - More friendly user message when superproject failed.v2.16.1Raman Tenneti2021-07-141-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | superproject is going to be default for some users. This change doesn't fail for repo init or repo sync if source couldn't be synced because of errors in superproject and superproject=true in the config file. The commands will fail if --use-superproject is specified on the command line explicitly. The error messages are logged with trace2 event logs and will be monitored. + sync - When there are errors with superproject and git_superproject says it is fatal failure, sync will exit only when --use-superproject option is specified on the command line. + init - command doesn't fail *if there are any superproject errors), but it will print a warning message and logs message via trace2 event logs. For fatal errors, init will exit only when --use-superproject option is specified on the command line. + All git commands log the command that is being executed so trace2 event logs will know the manifest, remote url and the branch name. There is no functional change other than fatal errors are honored with --use-supeproject option with init/sync commands. Tested the code with the following commands. $ ./run_tests -v Test 1 - sync'ing without errors -------------------------------- Added the following lines to '~/.repoconfig/config [repo] superproject = true $ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2 NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2 warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal, and you can continue to run repo sync, please run repo init with the --no-use-superproject option to stop seeing this warning Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in /usr/local/google/home/rtenneti/work/drive2/android/test $ repo_dev sync remote: Total 4 (delta 1), reused 4 (delta 1) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` /usr/local/google/home/rtenneti/work/drive2/android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... Test 2 - init and sync fail when --use-superproject option is passed -------------------------------------------------------------------- $ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2 --use-superproject remote: Total 57 (delta 16), reused 56 (delta 16) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2 warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal, and you can continue to run repo sync, please run repo init with the --no-use-superproject option to stop seeing this warning rtenneti@rtenneti2:~/work/drive2/android/test$ repo_dev sync --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2 warning: Cannot get project commit ids from manifest warning: Update of revisionId from superproject has failed, repo sync will not use superproject to fetch the source. Please resync with the --no-use-superproject option to avoid this repo warning. Test 3 - git fetch command fails and git command is printed ----------------------------------------------------------- With config change $ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2 ... NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/test/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Performing initial setup for superproject; this might take several minutes. repo: error: git fetch call failed,command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2 warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal and you can continue to run repo sync please run repo init with the --no-use-superproject option to avoid the repo warning Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .... Test 4 - no superproject tag ----------------------------- $ repo_dev init -u https://android.googlesource.com/platform/manifest -b pie-dev ... NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/pie_dev/.repo/manifest.xml warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal and you can continue to run repo sync please run repo init with the --no-use-superproject option to avoid the repo warning Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in ... $ repo_dev sync NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: /usr/local/google/home/rtenneti/work/drive2/android/pie_dev/.repo/manifest.xml warning: Cannot get project commit ids from manifest warning: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option to avoid the repo warning. Bug: [google internal] b/192614798 Bug: [google internal] b/Bug: [google internal] b/192614798 Change-Id: I9a97a0e7d9e609fad151bd7dd9cfc523eaa887cd Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/311502 Reviewed-by: Amith Dsouza <amithds@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: Don't exit if superproject tag doesn't exist in manifest.v2.16Raman Tenneti2021-06-161-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Don't exit if there are missing commit ids in superproject. This change implements the following suggestion from delphij@: "we should note the event (so we know that --use-superproject but there were some errors, e.g. manifest didn't specify commit id for some reason, or if there is no superproject but --use-superproject is used), print out a message telling the use that this is not support, but continue as if --no-use-superproject was specified?" Changes: superproject: + Added git_trace2_event_log as an argument to the constructor. + Sync method returns SyncResult a NamedTuple of ++ success - True if sync of superproject is successful, or False. ++ fatal - True if caller should exit, Or False. + UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of ++ manifest_path - path name of the overriding manifest file instead of None ++ fatal - True if caller should exit, Or False + _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of ++ commit_ids - a dictionary with the projects/commit ids on success, otherwise None ++ fatal - True if caller should exit, Or False + Added _SkipUpdatingProjectRevisionId a helper function to see if a project's revision id needs to be updated or not. This function is used to exclude projects from local manifest file. + Added the following error events into git_trace2_event_log ++ If superproject is missing in a manifest ++ If there are missing commit ids for projects. command.py: + Deleted unused import - platform + Added git_trace2_event_log as a member so all subcmds can log error events. main.py: + Initialized git_trace2_event_log as a member of command object. init.py: + Deleted unused import - optparse init.py: + Called sys.exit only if Sync returns exit=True sync.py: + Called sys.exit only if Superproject's UpdateProjectsRevisionId returns exit=True + Reloaded the manifest only if manifest path is returned by UpdateProjectsRevisionId. If not, fall back to the old way of doing repo sync. test_git_superproject: + Added code to verify error events are being logged. + Added a test for no superproject tag + Added test for UpdateProjectsRevisionId not updating the revision id with the commit ids. Tested the code with the following commands. + Positive test case with aosp-master. $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .../android/aosp $ repo_dev sync -j40 --use-superproject remote: Total 12 (delta 4), reused 12 (delta 4) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... repo sync has finished successfully. + Negative test case without superproject tag. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml error: Cannot get project commit ids from manifest error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.589s repo sync has finished successfully. + Test for missing commit_id for a project. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint'] error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.364s repo sync has finished successfully. $ ./run_tests -v ... ...== 164 passed in 2.87s ==... Bug: [google internal] b/189371541 Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* commands: document the "common" class attributeMike Frysinger2021-06-151-1/+1
| | | | | | | | | | Switch it to uppercase to make it clear it's a constant, and add documentation so its usage is clear. Change-Id: I6d281a66a90b5908b3131585c9945e88cfe815ea Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309322 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* command: make --verbose/--quiet available to all subcommandsMike Frysinger2021-04-131-3/+3
| | | | | | | | | | | Add new CommonOptions entry points to move the existing --jobs to, and relocate all --verbose/--quiet options to that. This provides both a consistent interface for users as well as for code. Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: Added --partial-clone-exclude option.Raman Tenneti2021-04-131-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | partial-clone-exclude option excludes projects during partial clone. This is a comma-delimited project names (from manifest.xml). This option is persisted and it is used by the sync command. A project that has been unparital'ed will remain unpartial if that project's name is specified in the --partial-clone-exclude option. The project name should match exactly. Added $ ./run_tests -v Bug: [google internal] b/175712967 "I can't "unpartial" my androidx-main checkout" $ rm -rf androidx-main/ $ mkdir androidx-main/ $ cd androidx-main/ $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml $ repo_dev sync -c -j8 + Verify a project is partial $ cd frameworks/support/ $ git config -l | grep 'partial' + Unpartial a project. $ /google/bin/releases/android/git_repack/git_unpartial + Verify project is unpartial $ git config -l | grep 'partial' $ cd ../.. + Exclude the project from being unparial'ed after init and sync. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml + Verify project is unpartial $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. + Remove the project from exclude list and verify that project is partially cloned. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* init: merge subcmd & wrapper parsersMike Frysinger2021-04-091-102/+1
| | | | | | | | | | These are manually kept in sync which is a pain. Have the init subcmd reuse the wrapper code directly. Change-Id: Ica73211422c64377bacc9bb3b1d1a8d9d5f7f4ca Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/302762 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* repo init: Added --no-partial-clone and made it persist. Bumped version to 2.14.Raman Tenneti2021-04-051-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Saved the repo.partialclone when --no-partial-clone option is passed to init, so repo sync will honor the no-partial-clone option. $ ./run_tests -v Bug: [google internal] b/175712967 $ mkdir androidx-main && cd androidx-main $ repo init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M $ repo sync -c -j32 $ cd frameworks/support/ && /google/bin/releases/android/git_repack/git_unpartial $ git config -l | grep 'partialclonefilter=blob' Observe partialclone is not enabled. $ cd ../.. $ repo init -u https://android.googlesource.com/platform/manifest -b androidx-main $ repo sync -c -j32 $ cd frameworks/support/ && git config -l | grep 'partialclonefilter=blob' Observe partialclone is enabled. $ /google/bin/releases/android/git_repack/git_unpartial Observe partialclone is not enabled. $ cd ../.. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --no-partial-clone $ repo sync -c -j32 $ cd frameworks/support/ && git config -l | grep 'partialclonefilter=blob' Observe partialclone is not enabled. Change-Id: I4400ad7803b106319856bcd0fffe00bafcdf014e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/302122 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: pass groups to ToXml method.Raman Tenneti2021-03-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added the following methods to XmlManifest class. + GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower() + GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func. + Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr. + Used the above methods to get groups in command.py::GetProjects and part of init.py. TODO: clean up these funcs to take structured group data more instead of passing strings around everywhere that need parsing. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL and verified prebuilts/fullsdk-linux directory has all the folders. Tested repo init and repo sync with --use-superproject and without --use-superproject argument. $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j32 Bug: [google internal] b/181804931 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422 Reviewed-by: Jonathan Nieder <jrn@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* init: expose --worktree optionMike Frysinger2021-03-091-3/+1
| | | | | | | | | | | There's a few rough edges here still, but no known corruption ones, so open it up a bit for people to experiment with. Bug: https://crbug.com/gerrit/11486 Change-Id: I81e0122ab6d3e032c546c8239dd4f03740676e80 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299242 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: Display status messages during repo init/sync.v2.13.3Raman Tenneti2021-03-041-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Superproject objects accept the optional argument “quiet”. The following progress messages are displayed if quiet is false. Displayed the following message whenever we find we have to make a new folder (aka new remote), because if you started with repo init android and later do googleplex-android that is when it will be slow. "<location>: Performing initial setup for superproject; this might take several minutes.". After fetch completion, added the following notification: "<location>: Initial setup for superproject completed." Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL. $ repo_dev init -u persistent-https://googleplex-android.git.corp.google.com/platform/manifest -b rvc-dev --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject Bug: [google internal] b/181178282 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia7fb85c6fb934faaa90c48fc0c55e7f41055f48a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299122 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* init: support -b HEAD as a shortcut to "the default"Mike Frysinger2021-02-251-4/+8
| | | | | | | | | | | When people switch to non-default branches, they sometimes want to switch back to the default, but don't know the exact name for that branch. Add a -b HEAD shortcut for that. Change-Id: I090230da25f9f5a169608115d483f660f555624f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297843 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: make --manifest-url flag optionalMike Frysinger2021-02-181-4/+16
| | | | | | | | | | | Since the --manifest-url flag is always required when creating a new checkout, allow the url to be specified via a positional argument. This brings it a little closer to the `git clone` UI. Change-Id: Iaf18e794ae2fa38b20579243d067205cae5fae2f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297322 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* init: added --use-superproject option to clone superproject.v2.12.2Raman Tenneti2021-02-111-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added --no-use-superproject to repo and init.py to disable use of manifest superprojects. Replaced the term "sha" with "commit id". Added _GetBranch method to Superproject object. Moved shared code between init and sync into SyncSuperproject function. This function either does git clone or git fetch. If git fetch fails it does git clone. Changed Superproject constructor to accept manifest, repodir and branch to avoid passing them to multiple functions as argument. Changed functions that were raising exceptions to return either True or False. Saved the --use-superproject option in config as repo.superproject. Updated internal-fs-layout.md document. Updated the tests to work with the new API changes in Superproject. Performance for the first time sync has improved from 20 minutes to around 15 minutes. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL. $ repo init took around 20 seconds longer because of cloning of superproject. $ time repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject ... real 0m35.919s user 0m21.947s sys 0m8.977s First run $ time repo sync --use-superproject ... real 16m41.982s user 100m6.916s sys 19m18.753s No difference in repo sync time after the first run. Bug: [google internal] b/179090734 Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Change-Id: I12df92112f46e001dfbc6f12cd633c3a15cf924b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296382 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* git_config: add SetBoolean helperMike Frysinger2021-02-111-7/+7
| | | | | | | | | A little sugar simplifies the code a bit. Change-Id: Ie2b8a965faa9f9ca05c7be479d03e8e073cd816d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296522 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* drop pyversion & is_python3 checkingMike Frysinger2021-01-061-9/+1
| | | | | | | | | | | We're committed to Python 3 at this point, so purge all the is_python3 related dynamic checks. Bug: https://crbug.com/gerrit/10418 Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-4/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: reset hard to --repo-revMike Frysinger2020-12-061-1/+1
| | | | | | | | | | | | | | | | When updating the tracking ref to whatever the user requested, make sure we reset state completely rather than trying to update the ref to it. This avoids confusing git as to the current state of the tree, and is more inline with user intentions: if they made a local change to the checkout, but ran repo init with a specific rev, we shouldn't stay wedged forever until they manually clean it all up. Bug: https://crbug.com/gerrit/12801 Change-Id: Ieba8d9c15781b4d0649bf01c7460694da63387b2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/290923 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* manifest_xml: refactor manifest parsing from client managementMike Frysinger2020-11-181-2/+2
| | | | | | | | | | | | | | We conflate the manifest & parsing logic with the management of the repo client checkout in a single class. This makes testing just one part (the manifest parsing) hard as it requires a full checkout too. Start splitting the two apart into separate classes to make it easy to reason about & test. Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: use the remote default manifest branchMike Frysinger2020-09-091-11/+15
| | | | | | | | | | | | | | | | Instead of hardcoding "master" as our default, use the remote server's default branch instead. For most people, this should be the same as "master" already. For projects moving to "main", it means we'll use the new name automatically rather than forcing people to use -b main. For repositories that never set up a default HEAD, we should still use the historical "master" default. Bug: https://crbug.com/gerrit/13339 Change-Id: I4117c81a760c9495f98dbb1111a3e6c127f45eba Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280799 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: reject unknown argsMike Frysinger2020-09-021-0/+3
| | | | | | | | | | | If you pass args to `repo init` when first creating a checkout, the repo launcher throws an error. But the init subcommand that runs in an existing checkout silently ignores them. Throw a proper error. Change-Id: I433bfcc73902d25f6b6a2974e77f6a977a75ed16 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/279696 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Make partial clone imply no-clone-bundle by default.Xin Li2020-05-211-2/+9
| | | | | | | | | | | | | | | | | | For large projects, clone bundle is useful because it provided a way to efficiently transfer a large portion of git objects through CDN, without needing to interact with git server. However, with partial clones, the intention is to not download most of the objects, so the use of clone bundles would defeat the space savings normally seen with partial clones, as they are downloaded before the first fetch. A new option, --clone-bundle is added to override this behavior. Add a new repo.clonebundle variable which remembers the choice if explicitly given from command line at repo init. Change-Id: I03638474af303a82af34579e16cd4700690b5f43 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/268452 Tested-by: Xin Li <delphij@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* init: respect --repo-rev changesMike Frysinger2020-03-251-0/+11
| | | | | | | | | | | | | | We respect this option when running the first `repo init`, but then silently ignore it once the initial sync is done. Make sure users are able to change things on the fly. We refactor the wrapper API to allow reuse between the two init's. Bug: https://crbug.com/gerrit/11045 Change-Id: Icb89a8cddca32f39a760a6283152457810b2392d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/260032 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* init: rename --repo-branch to --repo-revMike Frysinger2020-03-181-3/+4
| | | | | | | | | | | | We refer to this as "revision" in help text, and in REPO_REV envvar, so rename to --repo-rev to be consistent. We keep --repo-branch for backwards compatibility, but as a hidden option. Bug: https://crbug.com/gerrit/11045 Change-Id: I1ecc282fba32917ed78a63850360c08469db849a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259352 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* init: respect --repo-url changesMike Frysinger2020-03-171-0/+8
| | | | | | | | | | | | We respect this option when running the first `repo init`, but then silently ignore it once the initial sync is done. Make sure users are able to change things on the fly. Bug: https://crbug.com/gerrit/11045 Change-Id: I129ec5fec43a85067d555bb60c0d1ae02465f139 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/258893 Tested-by: David Pursehouse <dpursehouse@collab.net> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* repo/init: improve basic progress messagesMike Frysinger2020-02-221-1/+2
| | | | | | | | | | | | | We produce some simple "Get" messages that aren't super clear as to what they're doing, especially for people not familiar with repo. Rephrase these to explicitly state the thing we're doing so it's clear why we're downloading a particular source. Bug: https://crbug.com/gerrit/11293 Change-Id: I0749504f17c5385c6c65274a274e0ae25b117413 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256455 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* repo/init: add --verbose flagsMike Frysinger2020-02-221-3/+9
| | | | | | | | | | We don't use these for much yet, but init passes it down to the project sync layers which already has support for verbose mode. Change-Id: I651794f1b300be1688eeccf3941ba92c776812b5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256454 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* init: hide summary output when using --quietMike Frysinger2020-02-221-15/+19
| | | | | | | Change-Id: I5e30a6d6a1c95fb8d75d8b0f4d63b497e9aac526 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256452 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* add experimental git worktree supportMike Frysinger2020-02-191-0/+24
| | | | | | | | | | | | | | | | | This provides initial support for using git worktrees internally instead of our own ad-hoc symlink tree. It's been lightly tested which is why it's not currently exposed via --help. When people opt-in to worktrees in an existing repo client checkout, no projects are migrated. Instead, only new projects will use the worktree method. This allows for limited testing/opting in without having to completely blow things away or get a second checkout. Bug: https://crbug.com/gerrit/11486 Change-Id: Ic3ff891b30940a6ba497b406b2a387e0a8517ed8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254075 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* avoid negative variablesMike Frysinger2020-02-191-5/+5
| | | | | | | | | | | Trying to use booleans with names like "no_xxx" are hard to follow due to the double negatives. Invert all of them so we only have positive meanings to follow. Change-Id: Ifd37d0368f97034d94aa2cf38db52c723ac0c6ed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255493 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* Fix blank line issues reported by flake8David Pursehouse2020-02-121-0/+1
| | | | | | | | | | | | | | | | | | | | - E301 expected 1 blank line - E302 expected 2 blank lines - E303 too many blank lines - E305 expected 2 blank lines after class or function definition - E306 expected 1 blank line before a nested definition Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E301,E302,E303,E305,E306 Manually fix issues in project.py caused by misuse of block comments. Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fix indentation issues reported by flake8David Pursehouse2020-02-121-5/+5
| | | | | | | | | | | | | | | | | | | | | - E121 continuation line under-indented for hanging indent - E122 continuation line missing indentation or outdented - E125 continuation line with same indent as next logical line - E126 continuation line over-indented for hanging indent - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E121,E122,E125,E126,E127,E128,E129,E131 Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fix various whitespace issues reported by pyflakesDavid Pursehouse2020-02-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | - E201 whitespace after '[' - E202 whitespace before '}' - E221 multiple spaces before operator - E222 multiple spaces after operator - E225 missing whitespace around operator - E226 missing whitespace around arithmetic operator - E231 missing whitespace after ',' - E261 at least two spaces before inline comment - E271 multiple spaces after keyword Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E201,E202,E221,E222,E225,E226,E231,E261,E271 Change-Id: I367113eb8c847eb460532c7c2f8643f33040308c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254601 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* repo: lower min version of git a bitMike Frysinger2020-02-121-2/+7
| | | | | | | | | | | | | | | | | | | | | | We were perhaps a bit too hasty to jump to git-2.10. Existing LTS releases of Ubuntu are quite old still: Trusty has 1.9 while Xenial has 2.5. While we plan on dropping support for those eventually as we migrate to Python 3.6, we don't need to be so strict just yet on the git versions. We also want to disconnect the version the repo launcher requires from the version the rest of the source tree requires. The repo launcher doesn't need as many features, and being flexible there allows us more freedom to upgrade & rollback as needed. So we'll allow git-1.7 again, but start warning on any users older than git-1.9. This aligns better with existing LTS releases, and gives users a chance to start upgrading before we cut them off. Change-Id: I140305dd8e42c9719c84e2aee0dc6a5c5b18da25 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254573 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Fix inverted logic around [gitc-]init and -cDan Willemsen2020-02-071-1/+1
| | | | | | | | | | | | | Instead of not using '-c' for '--current-branch' when using gitc, we were only using '-c' when using gitc, so we still had the conflict with the gitc option, and other users still couldn't use '-c'. Test: repo init -u https://android.googlesource.com/platform/manifest; repo init -c Test: repo gitc-init -u ... -b ... -c testing Change-Id: I71e4950a49c281418249f0783c6a2ea34f0d3e2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253795 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Dan Willemsen <dwillemsen@google.com>
* init: handle -c conflicts with gitc-initMike Frysinger2020-02-051-2/+7
| | | | | | | | | | | | We keep getting requests for init to support -c. This conflicts with gitc-init which allocates -c for its own use. Lets make this dynamic so we keep it with "init" but omit it for "gitc-init". Bug: https://crbug.com/gerrit/10200 Change-Id: Ibf69c2bbeff638e28e63cb08926fea0c622258db Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253252 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* init: Add '-c' as an alias to '--current-branch'Diogo Ferreira2020-01-241-1/+1
| | | | | | | | | | | This makes it consistent with the short option for current-branch in repo sync. Change-Id: I2848e87f45a66ef8d829576d0c0c4c0f7a8636a0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241700 Tested-by: Diogo Ferreira <deovferreira@gmail.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net> Reviewed-by: Mike Frysinger <vapier@google.com>
* split out cli validation from executionv1.13.5Mike Frysinger2019-08-281-6/+5
| | | | | | | | | | | | | | | | | | | A common pattern in our subcommands is to verify the arguments & options before executing things. For some subcommands, that check stage is quite long which makes the execution function even bigger. Lets split that logic out of the execute phase so it's easier to manage these. This is most noticeable in the sync subcommand whose Execute func is quite large, and the option checking makes up ~15% of it. The manifest command's Execute can be simplified significantly as the optparse configuration always sets output_file to a string. Change-Id: I7097847ff040e831345e63de6b467ee17609990e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234834 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Add support for partial clone.v1.13.4Xin Li2019-07-161-1/+20
| | | | | | | | | | | | | A new option, --partial-clone is added to 'repo init' which tells repo to utilize git's partial clone functionality, which reduces disk and bandwidth usage when downloading by omitting blob downloads initially. Different from restricting clone-depth, the user will have full access to change history, etc., as the objects are downloaded on demand. Change-Id: I60326744875eac16521a007bd7d5481112a98749 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* use print() instead of sys.stdout.write()Mike Frysinger2019-07-111-3/+9
| | | | | | | | | | | | | | | | We're relying on sys.stdout.write() to flush its buffer which isn't guaranteed, and is not the case in Python 3. Change to use print() everywhere to be standard, and utilize the end= keyword to get the EOL semantics we need. We can't use print's flush= keyword as that's only in Python 3. Leave behind a TODO to clean it up when we can drop Python 2. Bug: https://crbug.com/gerrit/10418 Change-Id: I562128c7f1e6d154f4a6ecdf33a70fa2811dc2af Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/230392 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+1
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* Honor --depth during repo initNasser Grainawi2019-05-211-2/+2
| | | | | | | | | | | If a user is asking for a shallow clone of the repos, they probably expect a shallow clone of the manifest repo too. For very large manifest repos, this can be a huge space and time savings. For one real-world repo, a 'repo init --no-tags --current-branch' used 350MB of disk space and took 7 minutes. Adding --depth 1 and this change reduced it to 10MB and 2.5 minutes. Change-Id: I6fa662e174e623ede8861efc862ce26d65d4958d
* init: Remove -c short option for --current-branchv1.13.1Ereth McKnight-MacNeil2018-12-201-1/+1
| | | | | | | This option conflicts with the gitc-init -c short option. Bug: https://crbug.com/gerrit/10200 Change-Id: I06f37564429ca0bd4c0bbea6066daae4f663c838
* init: --dissociate option to copy objects borrowed with --referenceNikolai Merinov2018-10-191-0/+11
| | | | | | | | | | | | | | | | | | | | | | "repo init --reference" has two purposes: to decrease bandwidth used at clone time, and to decrease disk usage afterward, by using the reference repositories as an alternate store of objects even after the clone. The downside is that it makes the borrowing repositories dependent on the reference repositories, so it is easy to end up with missing objects by mistake after a cleanup operation like "git gc". To prevent that, v2.3.0-rc0~30^2 (clone: --dissociate option to mark that reference is only temporary, 2014-10-14), "git clone" gained a --dissociate option that makes --reference reuse objects from the reference repository at clone time but copy them over instead of using the reference as an alternate. This is more straightforward to use than plain --reference, at the cost of higher disk usage. Introduce a --dissociate to "repo init" that brings the same benefits to repo. The option is simply passed on to "git clone". Change-Id: Ib50a549eb71e0a2b3e234aea57537923962a80d4
* update markdown/help header formatMike Frysinger2018-10-101-2/+1
| | | | | | | Since gitiles recommends using # headers over ---/=== underlines, change the manifest-format.md over and all our help texts. Change-Id: I96391d41fba769e9f26870d497cf7cf01c8d8ab3
* init: Remove string concat in no-op os.path.joinSamuel Holland2018-01-221-1/+2
| | | | | | This also fixes a line length warning. Change-Id: I9c1ab65f83a35581dd657a707c7bc3c69db2b1dc
* Support relative paths in --referenceSamuel Holland2018-01-221-1/+1
| | | | | | | | Put the correctly-expanded relative paths in objects/info/alternates. From gitrepository-layout(5), this path should be "relative to the object database, not to the repository". Change-Id: I7b2027ae23cf7d367b80f5a187603c4cbacdb2de
* Merge changes from topic "windows-support"David Pursehouse2017-08-301-2/+2
|\ | | | | | | | | | | | | | | * changes: Port os.rename calls to work on Windows Workaround shutil.rmtree limitation on Windows Add support for creating symbolic links on Windows Make "git command" and "forall" work on Windows
| * Workaround shutil.rmtree limitation on WindowsRenaud Paquay2017-05-291-2/+2
| | | | | | | | | | | | | | | | | | | | By default, shutil.rmtree raises an exception when deleting readonly files on Windows. Replace all shutil.rmtree with platform_utils.rmtree, which adds an error handler to make files read-write when they can't be deleted. Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
* | init: add missing submodule argMartin Kelly2017-07-101-1/+1
|/ | | | | | | | | The submodule argument to Sync_LocalHalf was missing in MetaBranchSwitch, causing submodules not to get synced when the -b/--manifest-branch argument to init is used. Change-Id: Ie86d271abac2020725770be36ead83be3326e64b Signed-off-by: Martin Kelly <mkelly@xevo.com>
* init: allow relative path on --reference argumentYOUNG HO CHA2017-05-281-1/+1
| | | | | Change-Id: I41d6be6bc035fdddb5a27c072994439986d58d58 Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-19 02:06:37 +0000