summaryrefslogtreecommitdiffstats
path: root/subcmds/forall.py
Commit message (Collapse)AuthorAgeFilesLines
* forall: Fix returning results earlyJosip Sokcevic2024-10-301-3/+3
| | | | | | | | | | | | | rc should be returned only after all results are processed. R=jojwang@google.com Bug: b/376454189 Change-Id: I8200b9954240dd3e8e9f2ab82494779a3cb38627 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440901 Tested-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Joanna Wang <jojwang@google.com>
* subcmds: reduce multiprocessing serialization overheadKuang-che Wu2024-10-231-42/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Follow the same approach as 39ffd9977e to reduce serialization overhead. Below benchmarks are tested with 2.7k projects on my workstation (warm cache). git tracing is disabled for benchmark. (seconds) | v2.48 | v2.48 | this CL | this CL | | -j32 | | -j32 ----------------------------------------------------------- with clean tree state: branches (none) | 5.6 | 5.9 | 1.0 | 0.9 status (clean) | 21.3 | 9.4 | 19.4 | 4.7 diff (none) | 7.6 | 7.2 | 5.7 | 2.2 prune (none) | 5.7 | 6.1 | 1.3 | 1.2 abandon (none) | 19.4 | 18.6 | 0.9 | 0.8 upload (none) | 19.7 | 18.7 | 0.9 | 0.8 forall -c true | 7.5 | 7.6 | 0.6 | 0.6 forall -c "git log -1" | 11.3 | 11.1 | 0.6 | 0.6 with branches: start BRANCH --all | 21.9 | 20.3 | 13.6 | 2.6 checkout BRANCH | 29.1 | 27.8 | 1.1 | 1.0 branches (2) | 28.0 | 28.6 | 1.5 | 1.3 abandon BRANCH | 29.2 | 27.5 | 9.7 | 2.2 Bug: b/371638995 Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com>
* subcmds: Use repo loggerAravind Vasudevan2023-09-141-4/+6
| | | | | | | | | Bug: b/292704435 Change-Id: Ia3a45d87fc0bf0d4a1ba53050d9c3cd2dba20e55 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386236 Reviewed-by: Jason Chang <jasonnc@google.com> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Aravind Vasudevan <aravindvasudev@google.com>
* isort: format codebasev2.36Mike Frysinger2023-08-221-8/+7
| | | | | | | | Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* Format codebase with black and check formatting in CQGavin Mak2023-03-221-231/+285
| | | | | | | | | | | | Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* forall: fix multi-manifest variables.LaMont Jones2022-04-141-2/+8
| | | | | | | | | | | | | - REPO_PATH is relative to the root of the client. REPO_OUTERPATH is not needed. - REPO_INNERPATH is relative to the sub manifest root. - REPO_OUTERPATH is the path for the sub manifest root relative to the root of the client. Change-Id: I031692891cfef2634d1358584d27a6a4df735c20 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334899 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* Add multi-manifest support with <submanifest> elementv2.22LaMont Jones2022-02-171-5/+7
| | | | | | | | | | | To be addressed in another change: - a partial `repo sync` (with a list of projects/paths to sync) requires `--this-tree-only`. Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* commands: document the "common" class attributeMike Frysinger2021-06-151-1/+1
| | | | | | | | | | Switch it to uppercase to make it clear it's a constant, and add documentation so its usage is clear. Change-Id: I6d281a66a90b5908b3131585c9945e88cfe815ea Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309322 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* subcmds: force consistent help text formatMike Frysinger2021-05-041-8/+8
| | | | | | | | | | | | We're inconsistent with help text as to whether it uses title case and whether it ends in a period. Add a test to enforce a standard, and use the style that Python optparse & argparse use themselves (e.g. with the --help option): always lowercase, and never trailing period. Change-Id: Ic1defae23daeac0ac9116aaf487427f50b34050d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305144 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* command: make --verbose/--quiet available to all subcommandsMike Frysinger2021-04-131-6/+1
| | | | | | | | | | | Add new CommonOptions entry points to move the existing --jobs to, and relocate all --verbose/--quiet options to that. This provides both a consistent interface for users as well as for code. Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: handle missing project refs betterv2.13.8Mike Frysinger2021-03-251-2/+9
| | | | | | | | | | | | | | If the project exists, but the ref the manifest wants doesn't exist, don't throw an error (and abort the process in general). This can come up with a partially synced tree: the manifest is up-to-date, but not all the projects have yet been synced. Bug: https://crbug.com/gerrit/14289 Change-Id: Iba97413c476544223ffe518198c900c2193a00ed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/301262 Reviewed-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall/list: delete spurious "Mike Frysinger2021-03-191-1/+1
| | | | | | | Change-Id: I6995d48be1d8fc5d93f4b9fa617fad70b5b3429f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300902 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: allow interactive commands with -j1v2.13.7Mike Frysinger2021-03-181-1/+16
| | | | | | | | | | | | | Historically forall has been interactive since it ran in serial. Recent rework in here dropped that to enable parallel processing. Restore support for interactive commands when running -j1 or with an explicit --interactive option. Bug: https://crbug.com/gerrit/14256 Change-Id: I502007186f771914cfd7830846a4e1938b5e1f38 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300722 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: greatly speed up processing overheadv2.13Mike Frysinger2021-03-011-42/+17
| | | | | | | | | | | | | | | | With the recent commit 0501b29e7ae072e0b10ea9ddd913ec6d5975f690 ("status: Use multiprocessing for `repo status -j<num>` instead of threading"), the limitation with project serialization no longer applies. It turns out that ad-hoc logic is expensive. In the CrOS checkout (~1000 projects w/8 jobs by default), it adds about ~7sec overhead to all invocations. With a fast nop run: time repo forall -j8 -c true This goes from ~11sec to ~4sec -- more than 50% speedup. Change-Id: Ie6bcccd21eef20440692751b7ebd36c890d5bbcc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298724 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: simplify arg passing to worker childrenMike Frysinger2021-03-011-20/+6
| | | | | | | | | | | The ProjectArgs function can be inlined which simplifies it quite a bit. We shouldn't need the custom exception handling here either. This also makes the next commit easier to review. Change-Id: If3be04f58c302c36a0f20b99de0f67e78beac141 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298723 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: move nested func out to the classMike Frysinger2021-03-011-5/+7
| | | | | | | | | | | This is in preparation for simplifying the jobs support. The nested function is referenced in the options object which can't be pickled, so pull it out into a static method instead. Change-Id: I01d3c4eaabcb8b8775ddf22312a6e142c84cb77d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298722 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: rewrite parallel logicMike Frysinger2021-02-221-66/+40
| | | | | | | | | | | | | | | | | This fixes intermingling of parallel jobs and simplifies the code by switching to subprocess.run. This also provides stable output in the order of projects by returning the output as a string that the main loop outputs. This drops support for interactive commands, but it's unclear if anyone was relying on that, and the default behavior (-j2) made that unreliable. If it turns out someone still wants this, we can look at readding it. Change-Id: I7555b4e7a15aad336667292614f730fb7a90bd26 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297482 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* forall: improve pool logicMike Frysinger2021-02-221-15/+10
| | | | | | | | | | Use a pool contextmanager to take care of the messy details like properly cleaning it up when aborting. Change-Id: I264ebb591c2e67c9a975b6dcc0f14b29cc66a874 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297243 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* command: unify --job option & default valuesMike Frysinger2021-02-221-4/+4
| | | | | | | | | | | | Extend the Command class to support adding the --jobs option to the parser if the command declares it supports running in parallel. Also pull the default value used for the number of local jobs into the command module so local commands can share it. Change-Id: I22b0f8d2cf69875013cec657b8e6c4385549ccac Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297024 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* trim redundant pass statementsMike Frysinger2021-02-161-1/+0
| | | | | | | | | Clean up a few linter warnings. Change-Id: I531d0263a202435d32d83d87ec24998f4051639c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297062 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-3/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Expose upstream and dest-branch attributes through environmentSean McAllister2020-04-161-0/+4
| | | | | | | | | | | | | | | | | | | | | | Recent changes in ChromeOS Infra to ensure we're reading from snapshot manifests properly have exposed several bugs in our assumptions about manifest files. Mainly that the revision field for a project does _not_ have to refer to a ref, it can just be a commit hash. Several places assume that the revision field can be parsed as a ref to get the branch the project is on, which isn't true. To fix this we need to be able to look at the upstream and dest-branch attributes of the repo, so we expose them through the environment variables set in `repo forall`. Test: manual 'repo forall' run Bug: https://crbug.com/1032441 Change-Id: I2c039e0f4b2e0f430602932e91b782edb6f9b1ed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263132 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Sean McAllister <smcallis@google.com>
* forall.py: Close file after removing the streamKarsten Pfeiffer-Raddatz2020-04-141-1/+1
| | | | | | | | | | | | In order to remove the stream fileno() will be called on the filedescriptor. If the file is already closed fileno() will raise an error and forall will fail. Bug: https://crbug.com/gerrit/12563 Change-Id: Ib7b895fe881c844e3eb3672b011fdcdbdae63024 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/262838 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Karsten Pfeiffer-Raddatz <raddatz.karsten@gmail.com>
* assume environment always accepts stringsMike Frysinger2020-02-191-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Different Python & OS versions have different environ behavior wrt accepted types & encoding. Since we're migrating to be Python 3 only, lets change our code to assume strings always work as that's what the newer Python 3 does. This will fail under Python 2 for some env vars, mostly on Windows, but the effort of maintaining shim layers that can handle these edge cases isn't worth it when we're dropping that code. We leave the logic in the `repo` launcher for now as it is simple, and we want it to be able to switch versions a bit longer than the rest of the tree. Here's the support table: | *NIX | Windows | Python 2 | ASCII string | str or bytes, not unicode | Python 3 | str or bytes | str only | Windows uses strings natively in its environment all the time. But it doesn't allow unicode strings under Python 2, so we have to encode. Python 2 on *NIX is funky in that it always lowers to ASCII, so we had to manually encode to avoid errors regardless of unicode or str. Python 3 on Windows & *NIX will accept strings. *NIX will also accept bytes but Windows will not. Bug: https://crbug.com/gerrit/12145 Change-Id: I3cf8f95a06902754ea1f08ad4b28503f7063531b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/248972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Michael Mortensen <mmortensen@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* pyflakes: Fix remaining "E501 line too long" warningsDavid Pursehouse2020-02-131-1/+2
| | | | | | | | | | | We increased the max line length to 100 columns which got rid of the majority of these warnings, but there were still a few lines that exceeded 100 columns. Change-Id: Ib3372868ca2297f83073a14f91c8ae3df9d0d0e6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254699 Tested-by: David Pursehouse <dpursehouse@collab.net> Reviewed-by: Mike Frysinger <vapier@google.com>
* Fix blank line issues reported by flake8David Pursehouse2020-02-121-0/+3
| | | | | | | | | | | | | | | | | | | | - E301 expected 1 blank line - E302 expected 2 blank lines - E303 too many blank lines - E305 expected 2 blank lines after class or function definition - E306 expected 1 blank line before a nested definition Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E301,E302,E303,E305,E306 Manually fix issues in project.py caused by misuse of block comments. Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fix indentation issues reported by flake8David Pursehouse2020-02-121-20/+20
| | | | | | | | | | | | | | | | | | | | | - E121 continuation line under-indented for hanging indent - E122 continuation line missing indentation or outdented - E125 continuation line with same indent as next logical line - E126 continuation line over-indented for hanging indent - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E121,E122,E125,E126,E127,E128,E129,E131 Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fix various whitespace issues reported by pyflakesDavid Pursehouse2020-02-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | - E201 whitespace after '[' - E202 whitespace before '}' - E221 multiple spaces before operator - E222 multiple spaces after operator - E225 missing whitespace around operator - E226 missing whitespace around arithmetic operator - E231 missing whitespace after ',' - E261 at least two spaces before inline comment - E271 multiple spaces after keyword Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E201,E202,E221,E222,E225,E226,E231,E261,E271 Change-Id: I367113eb8c847eb460532c7c2f8643f33040308c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254601 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fixing forall subcommand for Py3v2.0Jiri Tyr2020-02-101-1/+1
| | | | | | | | | | | | | | | | Execution of 'repo forall -p -c' doesn't work with Py3 and ends up with an error: Got an error, terminating the pool: TypeError: can only concatenate str (not "bytes") to str That's fixed by using the decode() method. Change-Id: Ice01aaa1822dde8d957b5bf096021dd5a2b7dd51 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253659 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Jiri Tyr <jiri.tyr@gmail.com>
* forall: add an --ignore-missing optionv1.13.7.1Mike Frysinger2019-10-221-0/+7
| | | | | | | | | | | | | | | In CrOS, our infra has to deal with partial checkouts constantly (for a variety of reasons). To help reset back to a good state, we run git commands via `repo forall`, but don't care about the missing checkouts. Add a flag so we can disambiguate between missing repos and failing git subcommands. Bug: https://crbug.com/1013377 Bug: https://crbug.com/1013623 Change-Id: Ie3498c6d111276c60d2ecedbba21bfa778588d50 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241935 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* forall: exit 1 if we skip any reposMike Frysinger2019-10-011-3/+3
| | | | | | | | | | If a repo doesn't exist (e.g. an incomplete sync), make sure we exit non-zero when they get skipped. Change-Id: Ifff711e374416b1e6b9b8da4fdc6f14b27ced450 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239235 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* split out cli validation from executionv1.13.5Mike Frysinger2019-08-281-1/+2
| | | | | | | | | | | | | | | | | | | A common pattern in our subcommands is to verify the arguments & options before executing things. For some subcommands, that check stage is quite long which makes the execution function even bigger. Lets split that logic out of the execute phase so it's easier to manage these. This is most noticeable in the sync subcommand whose Execute func is quite large, and the option checking makes up ~15% of it. The manifest command's Execute can be simplified significantly as the optparse configuration always sets output_file to a string. Change-Id: I7097847ff040e831345e63de6b467ee17609990e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234834 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Remove double forall from "repo help forall" outputSolomon Kinard2019-07-081-1/+1
| | | | | | | | | | | | %prog represents the full subcommand ("repo" + subcommand name), not a Windows-style environment variable for "repo". The current help output shows repo forall% forall ... Correct the variable usage so it shows "repo forall ..." instead. Change-Id: I1fea55572428cc922ddf24ace1168a3d8f82dad0
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+1
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* update markdown/help header formatMike Frysinger2018-10-101-4/+2
| | | | | | | Since gitiles recommends using # headers over ---/=== underlines, change the manifest-format.md over and all our help texts. Change-Id: I96391d41fba769e9f26870d497cf7cf01c8d8ab3
* Remove unused pylint suppressionsDavid Pursehouse2018-07-241-2/+0
| | | | | | | | | | | pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that commit has not been done, but given that this project is no longer being actively developed I don't think it's worth spending time doing it. Leaving the pylint suppressions causes confusion because it leads people to think that we are still using pylint. Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
* forall: Clarify expansion of REPO_ environment values with -cDavid Pursehouse2017-10-311-0/+7
| | | | | | | | | | | | | | | | | | | | | | | If a user executes: repo forall -c echo $REPO_PROJECT then $REPO_NAME is expanded by the user's shell first, and passed as $1 to the shell that executes echo. This will either result in no output, or output of whatever REPO_NAME is set to in the user's shell. Either way, this is an unexpected result. The correct way to do it is: repo forall -c 'echo $REPO_PROJECT' such that $REPO_NAME is passed in to the shell literally, and then expanded to the value set in the environment that was passed to the shell. Update the documentation to make this clearer. Change-Id: I713caee914172ad8d8f0fafacd27026502436f0d
* Make "git command" and "forall" work on WindowsRenaud Paquay2017-05-291-20/+9
| | | | | | | | | | Python on Windows does not support non blocking file operations. To workaround this issue, we instead use Threads and a Queue to simulate non-blocking calls. This is happens only when running with the native Windows version of Python, meaning Linux and Cygwin are not affected by this change. Change-Id: I4ce23827b096c5138f67a85c721f58a12279bb6f
* Add --inverse-regex option to forall subcommandTakeshi Kanemoto2016-04-051-3/+8
| | | | | | | | | | Make it possible to exclude projects using regex/wildcard. The syntax is similar to that of the -r option, e.g.: repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT' Change-Id: Id250de5665152228c044c79337d3ac15b5696484
* Better error display on forallAlexandre Garnier2016-02-181-2/+4
| | | | | | | It was only displaying 'Project list error: GitError()' without any useful info about the project nor the error Change-Id: Iad66cbaa03cad1053b5ae9ecc90d7772aa42ac13
* Support filtering by group on forall and list subcmdGraham Christensen2015-07-301-1/+4
| | | | | | | | | | | | | | | | Enable operating against groups of repositories. As it stands, it isn't compatible with `-r/--regex`. `repo forall -g groupname -c pwd` will run `pwd` for all projects in groupname. `repo forall -g thisgroup,-butnotthisone -c pwd` will run `pwd` for all projects in `thisgroup` but not `butnotthisone`. `repo list -g groupname -n` will list all the names of repos in `groupname`. Change-Id: Ia75c50ce52541d1c8cea2874b20a4db2e0e54960
* forall: use smart sync override manifest if it existsDavid Pursehouse2015-07-061-0/+7
| | | | | | | | | | | | If a workspace is synced with the -s or -t option, the included projects may be different to those in the original manifest. However, when using the forall command, the list of the projects from the original manifest is used. If the smart sync manifest file exists, use it to override the original manifest. Change-Id: Iaefcbe148d2158ac046f158d98bbd8b5a5378ce7
* forall: Don't try to get lrev of projects in mirror workspaceDavid Pursehouse2015-07-061-1/+5
| | | | | | | | | git rev-parse fails for projects that don't have an explicit revision specified, and don't have a branch of the same name as the default revision. This can be the case in a workspace synced with the smart sync (-s) or smart tag (-t) option. Change-Id: I19bfe9fe7396170379415d85f10f6440dc6ea08f
* forall: setenv, only encode val if encode existsAnthony King2015-06-041-1/+3
| | | | Change-Id: I655e3043d0118c4e929897d3a51e5e013e5758dc
* Catch exceptions in project list generatorColin Cross2015-05-131-4/+19
| | | | | | | | | | | | | | | | If the generator that produces per-project worker arguments raises an exception it triggers python bug http://bugs.python.org/issue8296. Rewrite the generator expression as a generator function, and catch Exceptions and KeyboardInterrupts to end the iteration. Also add a pool worker initializer to disable SIGINT to prevent KeyboardInterrupts inside multiprocessing.Pool in the worker threads causing the same problem. Fixes easy-to-reproduce hangs when hitting ctrl-c during repo forall -c echo Change-Id: Ie4a65b3e1e07a64ed6bb6ff20f3912c4326718ca
* forall: use a generator to map the PoolAnthony King2015-04-081-2/+2
| | | | | | | | Before, a list was generated, which is why there was a massive delay. Using a generator will allow processes to start straight away Change-Id: Ia325b0b340cc328c08c9bcc92a6709bbdaf6a664
* Add --jobs option to forall subcommandTakeshi Kanemoto2014-06-241-113/+177
| | | | | | | | | | | | | Enable '--jobs' ('-j') option in the forall subcommand. For -jn where n > 1, the '-p' option can no longer guarantee the continuity of console output between the project header and the output from the worker process. SIG_INT is sent to all worker processes upon keyboard interrupt (Ctrl+C). Bug: Issue 105 Change-Id: If09afa2ed639d481ede64f28b641dc80d0b89a5c
* Add total count and iteration count to forall environmentMitchel Humpherys2014-03-311-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | For long-running forall commands sometimes it's useful to know which iteration is currently running. Add REPO_I and REPO_COUNT environment variables to reflect the current iteration count as well as the total number of iterations so that the user can build simple status indicators. Example: $ repo forall -c 'echo $REPO_I / $REPO_COUNT; git gc' 1 / 579 Counting objects: 41, done. Delta compression using up to 8 threads. Compressing objects: 100% (19/19), done. Writing objects: 100% (41/41), done. Total 41 (delta 21), reused 41 (delta 21) 2 / 579 Counting objects: 53410, done. Delta compression using up to 8 threads. Compressing objects: 100% (10423/10423), done. Writing objects: 100% (53410/53410), done. Total 53410 (delta 42513), reused 53410 (delta 42513) 3 / 579 ... Change-Id: I9f28b0d8b7debe423eed3b4bc1198b23e40c0c50 Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
* Print project name for -p on mirror clientsJorge Gonzalez2013-06-031-1/+6
| | | | | | | It doesn't make sense to print the relpath, since there's nothing checked out there and the dir shouldn't even exist. Change-Id: Id43631a8e0895929d3a5ad4ca8c2dc9e3d233e70
* Add regex support for subcommand forallZhiguang Li2013-04-291-1/+13
| | | | | | | | Filter the project list based on regex or wildcard matching of strings, then we can handle subset of all projects. Change-Id: Ib6c23aec79e7d981f7b6a5eb0ae93c44effec467 Signed-off-by: Zhiguang Li <muzili@gmail.com>