summaryrefslogtreecommitdiffstats
path: root/progress.py
Commit message (Collapse)AuthorAgeFilesLines
* progress: always show done messageKuang-che Wu2024-10-241-9/+4
| | | | | | | | | | | | The done message was omitted if the task is shorter than 0.5s. This might confuse users. Bug: b/371638995 Change-Id: I3fdd2cd8daea16d34fba88457d09397fff71af15 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440222 Tested-by: Kuang-che Wu <kcwu@google.com> Commit-Queue: Kuang-che Wu <kcwu@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* cleanup: Update codebase to expect Python 3.6Jason R. Coombs2023-10-311-3/+3
| | | | | | | | | | | - Bump minimum version to Python 3.6. - Use f-strings in a lot of places. Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034 Tested-by: Jason R. Coombs <jaraco@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Jason R. Coombs <jaraco@google.com>
* delete Python 2 (object) compatMike Frysinger2023-10-201-1/+1
| | | | | | | | | Bug: 302871152 Change-Id: I39636d73a6e1d69efa8ade74f75c5381651e6dc8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390054 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* isort: format codebasev2.36Mike Frysinger2023-08-221-0/+2
| | | | | | | | Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
* sync: Handle case when output isn't connected to a terminalGavin Mak2023-06-091-5/+6
| | | | | | | | | | | Currently `repo sync | tee` exits with an OSError. Bug: https://crbug.com/gerrit/17023 Change-Id: I91ae05f1c91d374b5d57721d45af74db1b2072a5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/376414 Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* sync: Show number of running fetch jobsv2.34Gavin Mak2023-05-251-4/+5
| | | | | | | | | | | | | Last of the recent `repo sync` UX changes. Show number of fetch jobs eg: "Fetching: 3% (8/251) 0:03 | 8 jobs | 0:01 chromiumos/overlays/chrom.." Bug: https://crbug.com/gerrit/11293 Change-Id: I1b3dcf3e56ae6731c6c6cb73cfce069b2f374b69 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374920 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Joanna Wang <jojwang@google.com>
* sync: Show elapsed time for the longest syncing projectGavin Mak2023-05-181-19/+30
| | | | | | | | | | | | | | "Last synced: X" is printed only after a project finishes syncing. Replace that with a message that shows the longest actively syncing project. Bug: https://crbug.com/gerrit/11293 Change-Id: I84c7873539d84999772cd554f426b44921521e85 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/372674 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Reviewed-by: Joanna Wang <jojwang@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
* sync: Display total elapsed fetch timeGavin Mak2023-05-021-8/+64
| | | | | | | | | | | | | Give users an indication that `repo sync` isn't stuck if taking a long time to fetch. Bug: https://crbug.com/gerrit/11293 Change-Id: Iccdaec918f86c9cc2db5dc12f9e3eef7ad0bcbda Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/371414 Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Joanna Wang <jojwang@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* Format codebase with black and check formatting in CQGavin Mak2023-03-221-101/+122
| | | | | | | | | | | | Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* trace: restore Progress indicator.v2.29.9LaMont Jones2022-11-101-3/+3
| | | | | | | | | | | If we are not tracing to stderr, then we should still have progress indication. Change-Id: Ifc9678e1fccbd92251e972fcf25aad6369d60e15 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/351195 Reviewed-by: Sam Saccone <samccone@google.com> Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Xin Li <delphij@google.com>
* progress: optimize progress bar updates a bitv2.24Mike Frysinger2022-04-191-11/+16
| | | | | | | | | | | | | Rather than erase the entire line first then print out the new content, print out the new content on top of the old and then erase anything we didn't update. This should result in a lot less flashing with faster terminals. Bug: https://crbug.com/gerrit/11293 Change-Id: Ie2920b0bf3d5e6f920b8631a1c406444b23cd12d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335214 Reviewed-by: LaMont Jones <lamontjones@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress: hide progress bar when --quietMike Frysinger2021-04-131-1/+9
| | | | | | | | | | | | We want progress bars in the default output mode, but not when the user specifies --quiet. Add a setting to the Progress bar class so it takes care of not displaying anything itself rather than having to update every subcommand to conditionally setup & call the object. Change-Id: I1134993bffc5437bc22e26be11a512125f10597f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303225 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: switch network fetch to multiprocessingMike Frysinger2021-04-011-2/+2
| | | | | | | | | | | | | | | | | | This avoids GIL limitations with using threads for parallel processing. This reworks the fetch logic to return results for processing in the main thread instead of leaving every thread to do its own processing. We have to tweak the chunking logic a little here because multiprocessing favors batching over returning immediate results when using a larger value for chunksize. When a single job can be quite slow, this tradeoff is not good UX. Bug: https://crbug.com/gerrit/12389 Change-Id: I0f0512d15ad7332d1eb28aff52c29d378acc9e1d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298642 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress: include execution time summaryMike Frysinger2021-02-261-4/+23
| | | | | | | | | | We're already keeping tracking of the start time, so might as well use it to display overall execution time for steps. Change-Id: Ib4cf8b2b0dfcdf7b776a84295d59cc569971bdf5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298482 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress/sync: include active number of jobsMike Frysinger2021-02-251-1/+19
| | | | | | | | | | Provide a bit more info to users that things are actively running. Bug: https://crbug.com/gerrit/11293 Change-Id: Ie8eeaa8804d1ca71cf5c78ad850fa2d17d26208c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297904 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* progress: always enable always_print_percentageMike Frysinger2021-02-251-16/+10
| | | | | | | | | | | | | | | | | | | The idea for skipping some progress updates was to avoid spending too much time on the progress bar itself. Unfortunately, for large projects (100s if not 1000s) of repos, we get into the situation with large/slow checkouts that we skip showing updates when a repo finishes, but not enough repos finished to increase the percent. Since the progress bar should be relatively fast compared to the actual network & local dick operations, have it show an update whenever the caller requests it. A test with ~1000 repos shows that the progress bar in total adds <100ms. Bug: https://crbug.com/gerrit/11293 Change-Id: I708a0c4bd923c59c7691a5b48ae33eb6fca4cd14 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297903 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-2/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Fix blank line issues reported by flake8David Pursehouse2020-02-121-0/+1
| | | | | | | | | | | | | | | | | | | | - E301 expected 1 blank line - E302 expected 2 blank lines - E303 too many blank lines - E305 expected 2 blank lines after class or function definition - E306 expected 1 blank line before a nested definition Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E301,E302,E303,E305,E306 Manually fix issues in project.py caused by misuse of block comments. Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Fix indentation issues reported by flake8David Pursehouse2020-02-121-18/+18
| | | | | | | | | | | | | | | | | | | | | - E121 continuation line under-indented for hanging indent - E122 continuation line missing indentation or outdented - E125 continuation line with same indent as next logical line - E126 continuation line over-indented for hanging indent - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E121,E122,E125,E126,E127,E128,E129,E131 Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* sync: merge project updates with status barMike Frysinger2019-11-121-2/+3
| | | | | | | | | | | | | | The current sync output displays "Fetching project" and "Checking out project" messages and progress bar updates independently leading to a lot of spam. Lets merge these periodic outputs with the status bar to get a little bit tighter output in the normal case. This doesn't solve all our problems, but gets us closer. Bug: https://crbug.com/gerrit/11293 Change-Id: Icd627830af4dd934a9355b7ace754b56dc96cfef Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244934 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* sync: improve output with intermingled progress bars and statusMike Frysinger2019-09-131-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When displaying progress bars, we use \r to reset the cursor to the start of the line before showing the new update. This assumes the new line will fully erase whatever was displayed there previously. The "done" codepath tries to handle this by including a few extra spaces at the end of the message to "white out" what was there. Lets replace that hack with the standard ECMA escape sequence that clears the current line completely. This is the CSI "erase in line" sequence that the terminal will use to delete all content. The \r is still needed to move the cursor to the start of the line. Using this sequence should be OK since we're already assuming the terminal is ECMA compliant with our use of coloring sequences. We also put the \r after the CSI sequence on the off chance the terminal can't process it and displays a few bytes of garbage. The other improvement is to the syncbuffer API. When it dumps its status information, it almost always comes after a progress bar update which leads to confusing comingled output. Something like: Fetching projects: 100% (2/2) error: src/platform2/: branch ... Since the progress bar is "throw away", have the syncbuffer reset the current output to the start of the line before showing whatever messages it has queued. Bug: https://crbug.com/gerrit/11293 Change-Id: I6544d073fe993d98ee7e91fca5e501ba5fecfe4c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236615 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* rename local trace moduleMike Frysinger2019-08-271-1/+1
| | | | | | | | | | There is a standard Python "trace" module, so having a local trace.py prevents us being able to import that. Rename the module to avoid. Change-Id: I23e29ec95a2204bb168a641323d05e76968d9b57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+1
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* Always print percentage when syncing quietlyTim Schumacher2017-07-151-2/+4
| | | | Change-Id: I574396e63520781067ed1e991c41caf7640e5731
* Add a newline after "Fetching projects" progress outputTim Schumacher2017-06-131-3/+5
| | | | | | | | | | | | | | | | | Output before change: Fetching project platform/packages/providers/UserDictionaryProvider Fetching projects: 66% (773/1171) Fetching project platform/external/regex-re2 Fetching project device/generic/mini-emulator-x86_64 Output after change: Fetching project platform/packages/providers/UserDictionaryProvider Fetching projects: 66% (773/1171) Fetching project platform/external/regex-re2 Fetching project device/generic/mini-emulator-x86_64 Change-Id: I4da84da58316c69294e4da2792f83885dc942701
* Support units in progress messagesShawn O. Pearce2011-09-191-7/+8
| | | | | | | | This allows our progress meter to be used for bytes transferred, by setting the units to KB or MB to let the user know the size. Change-Id: Ie8653d4a40d79439026c18bd51915845b2c5bba9 Signed-off-by: Shawn O. Pearce <sop@google.com>
* Do not emit progress if stderr is not a ttyShawn O. Pearce2010-05-271-2/+5
| | | | | | | | Avoids logging progress data into cron logs, etc. Suggested-by: Michael Richardson <mcr@sandelman.ottawa.on.ca> Change-Id: I4eefa2c282f0ca0a95a0185612b52e2146669e4c Signed-off-by: Shawn O. Pearce <sop@google.com>
* Only display a progress meter once we spend 0.5 seconds on a taskShawn O. Pearce2009-04-181-1/+10
| | | | | | | | | | | | | | | The point of the progress meter is to let the user know that the task is progressing, and give them a chance to estimate when it will be complete. If the task completes in under 0.5 seconds then it is sufficiently fast enough that the user doesn't need to be kept up-to-date on its progress; in fact showing the meter may just slow the task down waiting on the tty to redraw. We now delay the progress meter 0.5 seconds (or 1 second if the Python time.time() function isn't accurate enough) to avoid any really fast tasks, like a no-op local sync. Signed-off-by: Shawn O. Pearce <sop@google.com>
* Disable the progress meter when trace is enabledShawn O. Pearce2009-04-181-0/+7
| | | | | | | | | The trace output often interfers with the progress meter, so its easier to just disable the progress meter if trace is active. Its already verbose enough to let the user know we are working, which is all the progress meter is there for anyway. Signed-off-by: Shawn O. Pearce <sop@google.com>
* Don't divide by zero in progress meterShawn O. Pearce2009-04-161-14/+27
| | | | | | | | | If there are no projects to fetch, the progress meter would have divided by zero during `repo sync`, and that throws a ZeroDivisionError. Instead we report the progress with an unknown amount remaining. Signed-off-by: Shawn O. Pearce <sop@google.com>
* Add a project progress meter to 'repo sync'Shawn O. Pearce2009-04-101-0/+45
This way users can see how much is left during fetch. Its especially useful when most syncs are no-ops but there are hundreds of repositories to poll. Signed-off-by: Shawn O. Pearce <sop@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-14 06:52:46 +0000