summaryrefslogtreecommitdiffstats
path: root/command.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove gitc support from repoJosip Sokcevic2024-12-031-4/+0
| | | | | | | | | | gitc is no longer available. Change-Id: I0cbfdf936832f2cdd4876104ae3cc5a6e26154e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/444841 Reviewed-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Tested-by: Josip Sokcevic <sokcevic@google.com>
* subcmds: reduce multiprocessing serialization overheadKuang-che Wu2024-10-231-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Follow the same approach as 39ffd9977e to reduce serialization overhead. Below benchmarks are tested with 2.7k projects on my workstation (warm cache). git tracing is disabled for benchmark. (seconds) | v2.48 | v2.48 | this CL | this CL | | -j32 | | -j32 ----------------------------------------------------------- with clean tree state: branches (none) | 5.6 | 5.9 | 1.0 | 0.9 status (clean) | 21.3 | 9.4 | 19.4 | 4.7 diff (none) | 7.6 | 7.2 | 5.7 | 2.2 prune (none) | 5.7 | 6.1 | 1.3 | 1.2 abandon (none) | 19.4 | 18.6 | 0.9 | 0.8 upload (none) | 19.7 | 18.7 | 0.9 | 0.8 forall -c true | 7.5 | 7.6 | 0.6 | 0.6 forall -c "git log -1" | 11.3 | 11.1 | 0.6 | 0.6 with branches: start BRANCH --all | 21.9 | 20.3 | 13.6 | 2.6 checkout BRANCH | 29.1 | 27.8 | 1.1 | 1.0 branches (2) | 28.0 | 28.6 | 1.5 | 1.3 abandon BRANCH | 29.2 | 27.5 | 9.7 | 2.2 Bug: b/371638995 Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* sync: reduce multiprocessing serialization overheadKuang-che Wu2024-10-231-4/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: - Manifest object is large (for projects like Android) in terms of serialization cost and size (more than 1mb). - Lots of Project objects usually share only a few manifest objects. Before this CL, Project objects were passed to workers via function parameters. Function parameters are pickled separately (in chunk). In other words, manifests are serialized again and again. The major serialization overhead of repo sync was O(manifest_size * projects / chunksize) This CL uses following tricks to reduce serialization overhead. - All projects are pickled in one invocation. Because Project objects share manifests, pickle library remembers which objects are already seen and avoid the serialization cost. - Pass the Project objects to workers at worker intialization time. And pass project index as function parameters instead. The number of workers is much smaller than the number of projects. - Worker init state are shared on Linux (fork based). So it requires zero serialization for Project objects. On Linux (fork based), the serialization overhead is O(projects) --- one int per project On Windows (spawn based), the serialization overhead is O(manifest_size * min(workers, projects)) Moreover, use chunksize=1 to avoid the chance that some workers are idle while other workers still have more than one job in their chunk queue. Using 2.7k projects as the baseline, originally "repo sync" no-op sync takes 31s for fetch and 25s for checkout on my Linux workstation. With this CL, it takes 12s for fetch and 1s for checkout. Bug: b/371638995 Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921 Tested-by: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* cleanup: use new dict & set generator stylesJason R. Coombs2023-10-211-1/+1
| | | | | | | | Change-Id: Ie34ac33ada7855945c77238da3ce644f8a9f8306 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390374 Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
* cleanup: leverage yield from in more placesJason R. Coombs2023-10-201-2/+1
| | | | | | | | Change-Id: I4f9cb27d89241d3738486764817b51981444a903 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390274 Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* delete Python 2 (object) compatMike Frysinger2023-10-201-3/+3
| | | | | | | | | Bug: 302871152 Change-Id: I39636d73a6e1d69efa8ade74f75c5381651e6dc8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390054 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* isort: format codebasev2.36Mike Frysinger2023-08-221-3/+3
| | | | | | | | Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* gitc: drop supportJason Chang2023-08-151-8/+0
| | | | | | | | | Bug: b/282775958 Change-Id: Ib6383d6fd82a017d0a6670d6558a905d41be321f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375314 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Jason Chang <jasonnc@google.com> Commit-Queue: Jason Chang <jasonnc@google.com>
* Raise repo exit errors in place of sys.exitJason Chang2023-08-071-2/+6
| | | | | | | | | Bug: b/293344017 Change-Id: I92d81c78eba8ff31b5252415f4c9a515a6c76411 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/381774 Tested-by: Jason Chang <jasonnc@google.com> Reviewed-by: Joanna Wang <jojwang@google.com> Commit-Queue: Jason Chang <jasonnc@google.com>
* Format codebase with black and check formatting in CQGavin Mak2023-03-221-382/+449
| | | | | | | | | | | | Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* Fix flake8 warnings for some filesSergiy Belozorov2023-01-051-1/+2
| | | | | | | Change-Id: If67f8660cfb0479f0e710b3566285ef401fcf077 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/355969 Tested-by: Sergiy Belozorov <sergiyb@chromium.org> Reviewed-by: Mike Frysinger <vapier@google.com>
* Only sync superproject if it will be used.LaMont Jones2022-06-081-0/+12
| | | | | | | | | | | | If the user says `--no-use-superproject`, then do not bother syncing the superproject. Also add/update docstrings and comments throughout. Change-Id: I9cdad706130501bab9a22d3099a1dae605e9c194 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/338975 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* sync: add multi-manifest supportv2.26LaMont Jones2022-05-261-4/+7
| | | | | | | | | | | | | With this change, partial syncs (sync with a project list) are again supported. If the updated manifest includes new sub manifests, download them inheriting options from the parent manifestProject. Change-Id: Id952f85df2e26d34e38b251973be26434443ff56 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* Add multi-manifest support with <submanifest> elementv2.22LaMont Jones2022-02-171-14/+70
| | | | | | | | | | | To be addressed in another change: - a partial `repo sync` (with a list of projects/paths to sync) requires `--this-tree-only`. Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* man: make output system independentMike Frysinger2021-07-311-1/+9
| | | | | | | | | | | | | | The current help output might change based on the number of CPU cores available (since it reflects the dynamic --jobs logic). This is good for users running repo locally, but not good for shipping static man pages. Hook the help output to have it generate the same output all the time. Change-Id: I3098ceddc0ad914b0b8e3b25d660b5a264cb41ee Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312882 Reviewed-by: Roger Shimizu <rosh@debian.org> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: Don't exit if superproject tag doesn't exist in manifest.v2.16Raman Tenneti2021-06-161-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Don't exit if there are missing commit ids in superproject. This change implements the following suggestion from delphij@: "we should note the event (so we know that --use-superproject but there were some errors, e.g. manifest didn't specify commit id for some reason, or if there is no superproject but --use-superproject is used), print out a message telling the use that this is not support, but continue as if --no-use-superproject was specified?" Changes: superproject: + Added git_trace2_event_log as an argument to the constructor. + Sync method returns SyncResult a NamedTuple of ++ success - True if sync of superproject is successful, or False. ++ fatal - True if caller should exit, Or False. + UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of ++ manifest_path - path name of the overriding manifest file instead of None ++ fatal - True if caller should exit, Or False + _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of ++ commit_ids - a dictionary with the projects/commit ids on success, otherwise None ++ fatal - True if caller should exit, Or False + Added _SkipUpdatingProjectRevisionId a helper function to see if a project's revision id needs to be updated or not. This function is used to exclude projects from local manifest file. + Added the following error events into git_trace2_event_log ++ If superproject is missing in a manifest ++ If there are missing commit ids for projects. command.py: + Deleted unused import - platform + Added git_trace2_event_log as a member so all subcmds can log error events. main.py: + Initialized git_trace2_event_log as a member of command object. init.py: + Deleted unused import - optparse init.py: + Called sys.exit only if Sync returns exit=True sync.py: + Called sys.exit only if Superproject's UpdateProjectsRevisionId returns exit=True + Reloaded the manifest only if manifest path is returned by UpdateProjectsRevisionId. If not, fall back to the old way of doing repo sync. test_git_superproject: + Added code to verify error events are being logged. + Added a test for no superproject tag + Added test for UpdateProjectsRevisionId not updating the revision id with the commit ids. Tested the code with the following commands. + Positive test case with aosp-master. $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .../android/aosp $ repo_dev sync -j40 --use-superproject remote: Total 12 (delta 4), reused 12 (delta 4) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... repo sync has finished successfully. + Negative test case without superproject tag. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml error: Cannot get project commit ids from manifest error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.589s repo sync has finished successfully. + Test for missing commit_id for a project. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint'] error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.364s repo sync has finished successfully. $ ./run_tests -v ... ...== 164 passed in 2.87s ==... Bug: [google internal] b/189371541 Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* commands: pass settings via __init__Mike Frysinger2021-06-151-3/+9
| | | | | | | | | | Instead of setting properties on the instantiated command, pass them via the constructor like normal objects. Change-Id: I8787499bd2be68565875ffe243c3cf2024b36ae7 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309324 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* commands: document the "event_log" class attributeMike Frysinger2021-06-151-1/+6
| | | | | | | | | | Add some notes explaining why it's instantiated at the Command class level and not individual objects. Change-Id: Ib8081bb8480e85f6d3dfc23953c6bbc6ecc64934 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309323 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* commands: document the "common" class attributeMike Frysinger2021-06-151-1/+5
| | | | | | | | | | Switch it to uppercase to make it clear it's a constant, and add documentation so its usage is clear. Change-Id: I6d281a66a90b5908b3131585c9945e88cfe815ea Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309322 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* command: add a helper for the parallel execution boilerplateMike Frysinger2021-04-151-0/+40
| | | | | | | | | | | Now that we have a bunch of subcommands doing parallel execution, a common pattern arises that we can factor out for most of them. We leave forall alone as it's a bit too complicated atm to cut over. Change-Id: I3617a4f7c66142bcd1ab030cb4cca698a65010ac Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/301942 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* command: make --verbose/--quiet available to all subcommandsMike Frysinger2021-04-131-2/+23
| | | | | | | | | | | Add new CommonOptions entry points to move the existing --jobs to, and relocate all --verbose/--quiet options to that. This provides both a consistent interface for users as well as for code. Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: pass groups to ToXml method.Raman Tenneti2021-03-111-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added the following methods to XmlManifest class. + GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower() + GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func. + Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr. + Used the above methods to get groups in command.py::GetProjects and part of init.py. TODO: clean up these funcs to take structured group data more instead of passing strings around everywhere that need parsing. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL and verified prebuilts/fullsdk-linux directory has all the folders. Tested repo init and repo sync with --use-superproject and without --use-superproject argument. $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j32 Bug: [google internal] b/181804931 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422 Reviewed-by: Jonathan Nieder <jrn@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* status: improve parallel execution stabilityMike Frysinger2021-02-221-0/+9
| | | | | | | | | | | | | | | | | The status command runs a bunch of jobs in parallel, and each one is responsible for writing to stdout directly. When running many noisy jobs in parallel, output can get intermingled. Pass down a StringIO buffer for writing to so we can return the entire output as a string so the main job can handle displaying it. This fixes interleaved output as well as making the output stable: we always display results in the same project order now. By switching from map to imap, this ends up not really adding any overhead. Bug: https://crbug.com/gerrit/12231 Change-Id: Ic18b07c8074c046ff36e306eb8d392fb34fb6eca Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297242 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* command: unify --job option & default valuesMike Frysinger2021-02-221-0/+14
| | | | | | | | | | | | Extend the Command class to support adding the --jobs option to the parser if the command declares it supports running in parallel. Also pull the default value used for the number of local jobs into the command module so local commands can share it. Change-Id: I22b0f8d2cf69875013cec657b8e6c4385549ccac Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297024 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-2/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* command: add a `repo help` tip to --help outputMike Frysinger2020-02-191-1/+2
| | | | | | | | | | | For people used to running `repo xxx --help`, they might not realize that there are detailed man pages behind `repo help xxx`. Add a note to all --help commands to improve discoverability. Change-Id: I84af58aa0514cc7ead185f6c2534a8f88e09a236 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255853 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* Fix blank line issues reported by flake8David Pursehouse2020-02-121-0/+2
| | | | | | | | | | | | | | | | | | | | - E301 expected 1 blank line - E302 expected 2 blank lines - E303 too many blank lines - E305 expected 2 blank lines after class or function definition - E306 expected 1 blank line before a nested definition Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E301,E302,E303,E305,E306 Manually fix issues in project.py caused by misuse of block comments. Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* command: Add parentheses on wrapped conditionDavid Pursehouse2020-02-121-3/+3
| | | | | | | | | | | Surround the condition with parentheses rather than using backslashes. This prevents confusion about indentation when running flake8/autoflake8. Change-Id: I01775b96f817ee616f545b55369a4864fa1d6712 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254603 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* command: filter projects by active manifest groupsv1.13.7Mike Frysinger2019-10-081-2/+5
| | | | | | | | | | | | | | | | | | | `repo forall <proj>` will look up all <proj> in the manifest for all manifest groups regardless of which are active. If <proj> is checked out to different locations depending on the group, this ultimately fails as we're unable to locate all of them. Simple fix is to only include projects that match the manifest groups that we already passed down & initialized to the active set, and that we already use when getting the default project list. Bug: https://crbug.com/gerrit/11677 Bug: https://crbug.com/1011226 Change-Id: I975f10f9a9e5a1cad7d87344123f8003732dab27 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239652 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Raul Rangel <rrangel@chromium.org> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* split out cli validation from executionv1.13.5Mike Frysinger2019-08-281-0/+10
| | | | | | | | | | | | | | | | | | | A common pattern in our subcommands is to verify the arguments & options before executing things. For some subcommands, that check stage is quite long which makes the execution function even bigger. Lets split that logic out of the execute phase so it's easier to manage these. This is most noticeable in the sync subcommand whose Execute func is quite large, and the option checking makes up ~15% of it. The manifest command's Execute can be simplified significantly as the optparse configuration always sets output_file to a string. Change-Id: I7097847ff040e831345e63de6b467ee17609990e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234834 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+1
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* Remove unused pylint suppressionsDavid Pursehouse2018-07-241-7/+0
| | | | | | | | | | | pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that commit has not been done, but given that this project is no longer being actively developed I don't think it's worth spending time doing it. Leaving the pylint suppressions causes confusion because it leads people to think that we are still using pylint. Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
* sync: Add support to dump a JSON event log of all sync events.David Riley2017-05-291-0/+2
| | | | Change-Id: Id4852968ac1b2bf0093007cf2e5ca951ddab8b3b
* repo: Repo does not always handle '.' parameter correctlyMark E. Hamilton2016-04-081-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The repo script allows a manifest to specify a '.' as the path the top-level directory, which co-locates the .git and .repo directories, and places files from the git repository at the top-level: <project name="proj_name" path="." /> <project name="sierra.other.git" path="other" /> Most commands work correctly with this setup. Some commands, however, fail to find the project. For instance, 'repo sync' works, and 'repo sync .' works in a sub-project ('other' in this case) but 'repo sync .' in the top-level directory fails with the error: error: project . not found There are two reasons for this: 1. The self.worktree attribute of the Project object is not normalized, so with a '.' for path its value would be '/my/project/root/.'. This is fine when used as a path, since it's the same path as '/my/project/root', but when used in a string comparison it fails. This commit applies os.path.normpath() to that value before storing it. 2. The _GetProjectByPath method in command.py was not checking the path against manifest.topdir, so even once it was normalized the project was not found. This commit adds a check against manifest.topdir if the loop drops out without finding a project. Change-Id: Ic84d053f1bbb5a357cad566805d5a326ae8246d2
* Add --inverse-regex option to forall subcommandTakeshi Kanemoto2016-04-051-2/+8
| | | | | | | | | | Make it possible to exclude projects using regex/wildcard. The syntax is similar to that of the -r option, e.g.: repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT' Change-Id: Id250de5665152228c044c79337d3ac15b5696484
* command.py: Cleaned up pylint/pep8 violationsMark E. Hamilton2016-03-021-12/+16
| | | | | | | | | | | I noticed when running pylint (as the SUBMITTING_PATCHES file directs) that there were a few violations reported. This makes it difficult to see violations I might have introduced. This commit corrects all pylint violations in the command.py script. This script now has a pylint score of 10.0. Change-Id: Ibb35fa9af0e0b9b40e02ae043682b3af23286748
* Add GitcClientCommand class for GITC-specific commandsDan Willemsen2015-09-291-1/+6
| | | | | | | These won't show up as common commands in the help text unless in a GITC client, and will refuse to execute. Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
* gitc: Improve help visibilityv1.12.28Dan Willemsen2015-09-011-0/+5
| | | | | | | This improves the visiblity of gitc-init if we can get the gitc config, and hides it otherwise. Change-Id: I82830b0b07c311e8c74397ba79eb4c361f8b6fb5
* GITC: Add repo start support.Simran Basi2015-08-281-8/+11
| | | | | | | | | | | | | | | | | | | | | Add repo start support for GITC checkouts. If the user is in the GITC FS view, they can now run repo start to check out the sources and create a new working branch. When "repo start" is called on a GITC project, the revision tag is set to an empty string and saved in a new tag: old-revision. This tells the GITC filesystem to display the local copy of the sources when being viewed. The local copy is created by pulling the project sources and the new branch is created based off the original project revision. Updated main.py to setup each command's gitc_manifest when appropriate. Updated repo sync's logic to sync opened projects and updating the GITC manifest file for the rest. Change-Id: I7e4809d1c4fc43c69b26f2f1bebe45aab0cae628
* Support filtering by group on forall and list subcmdGraham Christensen2015-07-301-2/+3
| | | | | | | | | | | | | | | | Enable operating against groups of repositories. As it stands, it isn't compatible with `-r/--regex`. `repo forall -g groupname -c pwd` will run `pwd` for all projects in groupname. `repo forall -g thisgroup,-butnotthisone -c pwd` will run `pwd` for all projects in `thisgroup` but not `butnotthisone`. `repo list -g groupname -n` will list all the names of repos in `groupname`. Change-Id: Ia75c50ce52541d1c8cea2874b20a4db2e0e54960
* repo: Support multiple branches for the same project.David James2013-10-141-11/+15
| | | | | | | | | | | | | | | | | | | | | | | | | It is often useful to be able to include the same project more than once, but with different branches and placed in different paths in the workspace. Add this feature. This CL adds the concept of an object directory. The object directory stores objects that can be shared amongst several working trees. For newly synced repositories, we set up the git repo now to share its objects with an object repo. Each worktree for a given repo shares objects, but has an independent set of references and branches. This ensures that repo only has to update the objects once; however the references for each worktree are updated separately. Storing the references separately is needed to ensure that commits to a branch on one worktree will not change the HEAD commits of the others. One nice side effect of sharing objects between different worktrees is that you can easily cherry-pick changes between the two worktrees without needing to fetch them. Bug: Issue 141 Change-Id: I5e2f4e1a7abb56f9d3f310fa6fd0c17019330ecd
* Optimise regex pattern compilation in FindProjectsDavid Pursehouse2013-04-301-2/+2
| | | | | | | Make a list of compiled patterns once, and then iterate over that per project, instead of compiling the patterns again on every project. Change-Id: I91ec430d3060ec76d5e6b61facf6b13e343c90a7
* Add regex support for subcommand forallZhiguang Li2013-04-291-0/+11
| | | | | | | | Filter the project list based on regex or wildcard matching of strings, then we can handle subset of all projects. Change-Id: Ib6c23aec79e7d981f7b6a5eb0ae93c44effec467 Signed-off-by: Zhiguang Li <muzili@gmail.com>
* Some fixes for supporting python3Chirayu Desai2013-04-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix imports. * Use python3 syntax. * Wrap map() calls with list(). * Use list() only wherever needed. (Thanks Conley!) * Fix dictionary iteration methods (s/iteritems/items/). * Make use of sorted() in appropriate places * Use iterators directly in the loop. * Don't use .keys() wherever it isn't needed. * Use sys.maxsize instead of sys.maxint TODO: * Make repo work fully with python3. :) Some of this was done by the '2to3' tool [1], by applying the needed fixes in a way that doesn't break compatibility with python2. Links: [1]: http://docs.python.org/2/library/2to3.html Change-Id: Ibdf3bf9a530d716db905733cb9bfef83a48820f7 Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
* Special handling for manifest group "default"David Holmer2013-04-031-1/+1
| | | | | | | | | | | | | | | | | | | | | Change Details: * Make "default" a special manifest group that matches any project that does not have the special project group "notdefault" * Use "default" instead of "all,-notdefault" when user does not specify manifest group * Expand -g option help to include example usage of manifest groups Change Benefits: * Allow a more intuitive and expressive manifest groups specification: * "default" instead of "all,-notdefault" * "default,foo" instead of "all,-notdefault,foo" * "default,-foo" instead of "all,-notdefault,-foo" * "foo,-default" which has no equivalent * Default manifest groups behavior can be restored by the command 'repo init -g default'. This is significantly more intuitive than the current equivalent command 'repo init -g all,-notdefault'. Change-Id: I6d0673791d64a650110a917c248bcebb23b279d3
* Represent git-submodule as nested projects, take 2Che-Liang Chiou2012-11-191-25/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Previous submission of this change broke Android buildbot due to incorrect regular expression for parsing git-config output. During investigation, we also found that Android, which pulls Chromium, has a workaround for Chromium's submodules; its manifest includes Chromium's submodules. This new change, in addition to fixing the regex, also take this type of workarounds into consideration; it adds a new attribute that makes repo not fetch submodules unless submodules have a project element defined in the manifest, or this attribute is overridden by a parent project element or by the default element.) We need a representation of git-submodule in repo; otherwise repo will not sync submodules, and leave workspace in a broken state. Of course this will not be a problem if all projects are owned by the owner of the manifest file, who may simply choose not to use git-submodule in all projects. However, this is not possible in practice because manifest file owner is unlikely to own all upstream projects. As git submodules are simply git repositories, it is natural to treat them as plain repo projects that live inside a repo project. That is, we could use recursively declared projects to denote the is-submodule relation of git repositories. The behavior of repo remains the same to projects that do not have a sub-project within. As for parent projects, repo fetches them and their sub-projects as normal projects, and then checks out subprojects at the commit specified in parent's commit object. The sub-project is fetched at a path relative to parent project's working directory; so the path specified in manifest file should match that of .gitmodules file. If a submodule is not registered in repo manifest, repo will derive its properties from itself and its parent project, which might not always be correct. In such cases, the subproject is called a derived subproject. To a user, a sub-project is merely a git-submodule; so all tips of working with a git-submodule apply here, too. For example, you should not run `repo sync` in a parent repository if its submodule is dirty. Change-Id: I4b8344c1b9ccad2f58ad304573133e5d52e1faef
* Allow command options to be set from environment variablesDavid Pursehouse2012-11-171-0/+40
| | | | | | | | | | | | Extend the Command base class to allow options to be set from values in environment variables, if the user has not given the option on the command line and the environment variable is set. Derived classes of Command can override the implementation of the method _GetEnvironmentOptions to configure which of its options may be set from environment variables. Change-Id: I7c780bcf9644d6567893d9930984c054bce7351e
* Even more coding style cleanupDavid Pursehouse2012-10-301-1/+1
| | | | | | | | | | | Fixing some more pylint warnings: W1401: Anomalous backslash in string W0623: Redefining name 'name' from outer scope W0702: No exception type(s) specified E0102: name: function already defined line n Change-Id: I5afcdb4771ce210390a79981937806e30900a93c
* Revert "Represent git-submodule as nested projects"v1.11.1Shawn O. Pearce2012-10-261-48/+24
| | | | | | | | | | | This reverts commit 69998b0c6ff724bf620480140ccce648fec7d6a9. Broke Android's non-gitmodule use case. Conflicts: project.py subcmds/sync.py Change-Id: I68ceeb63d8ee3b939f85a64736bdc81dfa352aed
* Add pylint configuration and instructionsDavid Pursehouse2012-10-241-2/+2
| | | | | | | | | | pylint configuration file (.pylintrc) is added, and submission instructions are updated to include pylint usage steps. Deprecated pylint suppression (`disable-msg`) is updated in a few modules to make it work properly with the latest version (0.26). Change-Id: I4ec2ef318e23557a374ecdbf40fe12645766830c
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-06-25 19:00:48 +0000