summaryrefslogtreecommitdiffstats
path: root/project.py
Commit message (Collapse)AuthorAgeFilesLines
...
* use open context managers in more placesMike Frysinger2019-11-121-13/+4
| | | | | | | | | | Use open() as a context manager to simplify the close logic and make the code easier to read & understand. This is also more Pythonic. Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* sync: make .git init more robustMike Frysinger2019-11-121-30/+34
| | | | | | | | | | | | | | | | Hitting Ctrl-C in the middle of this func will leave the .git in a bad state that requires manual recovery. The code tries to catch all exceptions and recover by deleting the incomplete .git dir, but it omits KeyboardInterrupt which Exception misses. We could add that to the recovery path, but we can make this more robust with a different approach: set up everything in .git.tmp/ and only move it to .git/ once we've fully initialized it. Change-Id: I0f5b97f2e19fc39cffc3e5e23993a2da7220f4e3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244733 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* project: allow CurrentBranch to return None on errorsMike Frysinger2019-10-011-3/+10
| | | | | | | | | | | If the repo client checkout is in an incomplete sync state, the work git repo might be in a bad way. Turn errors parsing HEAD into None since callers of CurrentBranch already need to account for it. Change-Id: Ia7682e29ef4182006b1fb5f5e57800f8ab67a9f4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239239 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* diff: handle errors gracefullyMike Frysinger2019-10-011-5/+14
| | | | | | | | | | | | If `git diff` fails in any project checkout (e.g. an incomplete sync), make sure we print that error clearly rather than blowing up, and exit non-zero in the process. Bug: https://crbug.com/gerrit/11613 Change-Id: I12f278427cced20f23f8047e7e3dba8f442ee25e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239236 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Fix tag clobbering when -c is used.v1.13.6Xin Li2019-09-161-3/+3
| | | | | | | | Bug: b/140189154 Change-Id: I8861a6115b20c9a3d88ddec5344c75326ae44823 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/237572 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* sync: improve output with intermingled progress bars and statusMike Frysinger2019-09-131-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When displaying progress bars, we use \r to reset the cursor to the start of the line before showing the new update. This assumes the new line will fully erase whatever was displayed there previously. The "done" codepath tries to handle this by including a few extra spaces at the end of the message to "white out" what was there. Lets replace that hack with the standard ECMA escape sequence that clears the current line completely. This is the CSI "erase in line" sequence that the terminal will use to delete all content. The \r is still needed to move the cursor to the start of the line. Using this sequence should be OK since we're already assuming the terminal is ECMA compliant with our use of coloring sequences. We also put the \r after the CSI sequence on the off chance the terminal can't process it and displays a few bytes of garbage. The other improvement is to the syncbuffer API. When it dumps its status information, it almost always comes after a progress bar update which leads to confusing comingled output. Something like: Fetching projects: 100% (2/2) error: src/platform2/: branch ... Since the progress bar is "throw away", have the syncbuffer reset the current output to the start of the line before showing whatever messages it has queued. Bug: https://crbug.com/gerrit/11293 Change-Id: I6544d073fe993d98ee7e91fca5e501ba5fecfe4c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236615 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Add repo start option to create the branch based off HEADTheodore Dubois2019-09-041-2/+6
| | | | | | | | | | | | This makes it way easier to recover from forgetting to run repo start before committing: just run `repo start -b new-branch`, instead of all that tedious mucking around with reflogs. Change-Id: I56d49dce5d027e28fbba0507ac10cd763ccfc36d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232712 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* rename local trace moduleMike Frysinger2019-08-271-1/+1
| | | | | | | | | | There is a standard Python "trace" module, so having a local trace.py prevents us being able to import that. Rename the module to avoid. Change-Id: I23e29ec95a2204bb168a641323d05e76968d9b57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Explicitly allow clobbering tags when fetching from remote.Xin Li2019-08-221-0/+2
| | | | | | | | Bug: b/139860049 Change-Id: I3c4134eda7e9e75c9d72b233e269bcc0e624d1e8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234632 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* project: fix encoding handling with git commandsMike Frysinger2019-08-041-9/+6
| | | | | | | | | | | | | | | | | The GitCommand Wait helper takes care of decoding bytes to strings for us. That means we don't have to decode stdout ourselves which is what our local rev list, ls-remote, and generic get_attr helpers were doing. If we don't use Wait though to capture the output but instead go directly to the subprocess stdout, we do have to handle decoding ourselves. This is what the diff helpers were doing. Bug: https://crbug.com/gerrit/10418 Change-Id: I057ca245af3ff18d6b4a074e3900887f06a5617d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233076 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* hooks: support external hooks running different Python versionMike Frysinger2019-07-271-21/+126
| | | | | | | | | | | | | | | | | | | | | | | | | As we convert repo to support Python 3, the version of Python that we use might not be the version that repo hooks users have written for. Since repo upgrades are not immediate, and not easily under direct control of end users (relative to the projects maintaining the hook code), allow hook authors to declare the version of Python that they want to use. Now repo will read the shebang from the hook script and compare it against the version of Python repo itself is running under. If they differ, we'll try to execute a separate instance of Python and have it load & execute the hook. If things are compatible, then we still use the inprocess execution logic that we have today. This allows repo hook users to upgrade on their own schedule (they could even upgrade to Python 3 ahead of us) without having to worry about their supported version being exactly in sync with repo's. Bug: https://crbug.com/gerrit/10418 Change-Id: I97c7c96b64fb2ee465c39b90e9bdcc76394a146a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/228432 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Add support for partial clone.v1.13.4Xin Li2019-07-161-5/+13
| | | | | | | | | | | | | A new option, --partial-clone is added to 'repo init' which tells repo to utilize git's partial clone functionality, which reduces disk and bandwidth usage when downloading by omitting blob downloads initially. Different from restricting clone-depth, the user will have full access to change history, etc., as the objects are downloaded on demand. Change-Id: I60326744875eac16521a007bd7d5481112a98749 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* Don't try to decode when checking clone bundlesPierre Tardy2019-07-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | This fix exception with python3 with stack-trace: error: Cannot fetch platform_external_grpc-grpc-java.git (UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 640: invalid start byte) [...] File "[...]project.py", line 2255, in _IsValidBundle if f.read(16) == '# v2 git bundle\n': File "/usr/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) Even if we ask 16 characters, python buffered decoder will try to decode more in the buffer The patch works for python2 and python3, and open the file in byte mode so that decoding is not attemped Signed-off-by: Pierre Tardy <tardyp@gmail.com> Change-Id: I837ae3c5cd724b34670fc2a84e853431f482b20d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/224642 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* project: rev_list: simplify executionMike Frysinger2019-07-051-6/+1
| | | | | | | | | | | | | | Currently we read the binary stream from the subprocess code directly before waiting for it to finish, but there's no need to do so as we aren't streaming the output to the user. This also means we pass up binary data to the caller as we don't go through GitCommand's internal logic which decodes the stream as utf-8. Simplify the code by calling Wait first, then splitting the entire captured output in one line. Bug: https://crbug.com/gerrit/10418 Change-Id: I7a57904be8cb546a229980fb79c829fc3df31e7d
* Support clone bundle through persistent-http[s].Xin Li2019-06-261-5/+9
| | | | | Bug: https://crbug.com/gerrit/11075 Change-Id: I367c6bfe8da47d886c017a2ac614d4ccb3f8a438
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+2
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* project: Set config option to skip lfs process filterv1.13.3Francois Ferrand2019-05-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | During sync, repo runs `git read-tree --reset -u -v HEAD` which causes git-lfs's smudge filter to run, which fails because git-lfs does not work with bare repositories. This was fixed in I091ff37998131e2e6bbc59aa37ee352fe12d7fcd to automatically disable this smudge filter. However, later versions of Git (2.11.0) introduced a new filter protocol [1], to avoid spawning a new command for each filtered file. This was implemented in Git-LFS 1.5.0 [2]. This patch fixes the issue by setting the git lfs process filter, in addition to the smudge filter. For any projects that have LFS objects, `git lfs pull` must still be executed manually afterwards. [1] https://github.com/git/git/commit/edcc85814c87ebd7f3b1b7d3979fac3dfb84d308 [2] https://github.com/git-lfs/git-lfs/pull/1617 Bug: https://crbug.com/gerrit/10911 Change-Id: I277fc68fdefc91514a2412b3887e3be9106cab48
* Merge "Print project name when work tree initialization fails"Jonathan Nieder2019-05-161-1/+1
|\
| * Print project name when work tree initialization failsMikhail Naganov2019-05-091-1/+1
| | | | | | | | | | | | | | When syncing a lot of projects in parallel, it is not otherwise clear which one of them has failed to init work tree. Change-Id: I8edfb4955023389a499e99cfa511bdc0d2850ba2
* | Use %topic=topic instead of deprecated /topic syntax on pushDavid Pursehouse2019-05-151-2/+3
|/ | | | | Bug: https://crbug.com/gerrit/9930 Change-Id: Iefa202d42ef6e6b8b2b1a3f9b8baa5f0d65cbd60
* Ignore submodules when calculating 'git diff-files'Eli Ribble2019-05-021-0/+1
| | | | | | | | | | | This allows projects to include submodules inside of projects that use repo without repo incorrectly believing the area is dirty just because a submodule has updates. This is in line with git porcelain commands which generally require a commandline flag to include submodules (git add, git rebase). Change-Id: Ide8a292162a42ab35145b5c4ca8ca0d020cdfe81
* Don't print "persistent ref" message when syncing quietlyTim Schumacher2019-04-151-2/+3
| | | | | | | | | The newly introduced "Already have persistent ref" message prevents repo from overwriting the last line when syncing quietly. Omit the message when syncing quietly to clean up the output and to restore the previous behaviour. Change-Id: Idf42751c67f95924d6de50092ba54d4c6fe93096
* sync: respect --force-sync when fetching updatesMike Frysinger2019-03-181-2/+6
| | | | | | | | | | | | | | | | | | If a tag is rewritten on the server (which is bad), trying to recover locally with `repo sync --force-sync` doesn't actually work. People have to manually delete things themselves to fix syncing. While tags should never be rewritten in practice, allow users to easily recover from broken servers. We updated some of these code paths already (see commit 6e53844f1edd3 "Allow clobbering of existing tags from remote."), but the incremental update flow was missed. Bug: b/120778183 Bug: chromium:932651 Test: delete local tag & recreate to diff commit, then check `repo sync` & `repo sync --force-sync` behavior Change-Id: I3648f7d2526732c06016b691a9a36c003157618d
* project: Relax the submodule name pattern to accept dotsSebastian Schuberth2019-03-111-2/+2
| | | | | | | | Even if dots are used as separators for Git config keys, they are not forbidden as part of submodule names. This fixes the issue of submodules with a name like e.g. "long.js" to be skipped from checkout. Change-Id: I77da07925ad207fa3d043067dfbbcb4a1ebdac4d
* Allow clobbering of existing tags from remote.v1.13.0Xin Li2018-12-101-1/+1
| | | | | Bug: 120778183 Change-Id: Id44e2b68abc410a3afd4e07a3c943b0936347e38
* Add option for git-repo to support 'silent' uploadsVadim Bendebury2018-11-051-0/+5
| | | | | | | | | | | | | | | | | | | When --ne/--no-emails is added to 'repo upload' command line, gerrit server will not generate notification emails. project.py:Project.UploadForReview method is modified to accept a string recognizable by gerrit to indicate different sets of destination email addressees, but the upload command line allows only one option - disable sending emails completely. Default repo upload behavior is not being changed. TEST=tried in the Chrome OS repo, observed that patches uploaded with --ne or --no-emails indeed do not trigger any emails, while patches uploaded without these command line options still trigger email notifications. Change-Id: I0301edec984907aedac277d883bd0e6d3099aedc
* upload: Unify option passing in ssh and other transportsJonathan Nieder2018-11-051-15/+9
| | | | | | | | | | | | Pass options through the refspec for all transports, including ssh. This means the behavior will be more consistent between the ssh and https cases. A downside is that this prevents passing special characters in reviewer options. That already didn't work over https, so it seems okay. It could be fixed by using push options instead. Change-Id: Ia38d16e350cb8cb0de14463bfb3d9724e13bc4bf
* Merge "init: --dissociate option to copy objects borrowed with --reference"David Pursehouse2018-10-281-0/+10
|\
| * init: --dissociate option to copy objects borrowed with --referenceNikolai Merinov2018-10-191-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "repo init --reference" has two purposes: to decrease bandwidth used at clone time, and to decrease disk usage afterward, by using the reference repositories as an alternate store of objects even after the clone. The downside is that it makes the borrowing repositories dependent on the reference repositories, so it is easy to end up with missing objects by mistake after a cleanup operation like "git gc". To prevent that, v2.3.0-rc0~30^2 (clone: --dissociate option to mark that reference is only temporary, 2014-10-14), "git clone" gained a --dissociate option that makes --reference reuse objects from the reference repository at clone time but copy them over instead of using the reference as an alternate. This is more straightforward to use than plain --reference, at the cost of higher disk usage. Introduce a --dissociate to "repo init" that brings the same benefits to repo. The option is simply passed on to "git clone". Change-Id: Ib50a549eb71e0a2b3e234aea57537923962a80d4
* | Add support for long pathsRenaud Paquay2018-10-221-8/+8
|/ | | | | | | | | | | | | | | | | | | | * Add more file i/o wrappers in platform_utils to allow using long paths (length > MAX_PATH) on Windows. * Paths using the long path syntax ("\\?\" prefix) should never escape the platform_utils API surface area, so that this specific syntax is not visible to the rest of the repo code base. * Forward many calls from os.xxx to platform_utils.xxx in various place to ensure long paths support, specifically when repo decides to delete obsolete directories. * There are more places that need to be converted to support long paths, this commit is an initial effort to unblock a few common use cases. * Also, fix remove function to handle directory symlinks Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
* Remove unused pylint suppressionsDavid Pursehouse2018-07-241-4/+2
| | | | | | | | | | | pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that commit has not been done, but given that this project is no longer being actively developed I don't think it's worth spending time doing it. Leaving the pylint suppressions causes confusion because it leads people to think that we are still using pylint. Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
* Pass refs to ls-remoteAkshay Verma2018-03-241-2/+2
| | | | | | This will fix the issue of parsing large output locally Change-Id: I9a5cf1238147a02c92a3fca53eab9bd57f9d16b4
* Download latest patch when no patch is specifiedAkshay Verma2018-03-171-0/+10
| | | | | | | | | | | When someone does "repo download -c <project> <change>" without specifying a patch number, by default patch 1 is downloaded. An alternative is to look for the latest patch and download the same when no explicit patch is given. This commit does the same by identifying the latest patch using "git ls-remote". Change-Id: Ia5fa7364415f53a3d9436df4643e38f3c90ded58
* Merge changes I9c1ab65f,I7b2027aeDavid Pursehouse2018-03-161-1/+6
|\ | | | | | | | | | | * changes: init: Remove string concat in no-op os.path.join Support relative paths in --reference
| * Support relative paths in --referenceSamuel Holland2018-01-221-1/+6
| | | | | | | | | | | | | | | | Put the correctly-expanded relative paths in objects/info/alternates. From gitrepository-layout(5), this path should be "relative to the object database, not to the repository". Change-Id: I7b2027ae23cf7d367b80f5a187603c4cbacdb2de
* | implement optional 'sync-tags' in the manifest fileYOUNG HO CHA2018-02-141-0/+8
|/ | | | | | | | Allow the 'default' and 'project' element in the manifest file to apply "--no-tags" option equivalent. Change-Id: I7e0f8c17a0e25cca744d45df049076d203c52ff5 Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
* Sync correctly when subproject url is a relative url to its parent urlShouheng Zhang2017-12-211-0/+2
| | | | | | | | | | | Issue: when subproject url is a relative in .gitmodules repo tool cannot handle this and cause: "fatal: '***' does not appear to be a git repository fatal: Could not read from remote repository." issue. Signed-off-by: Shouheng Zhang <shouheng.zhang@intel.com> Change-Id: I2a24c291ea0074ba13a740b32a11c0c25975e72b
* Support --push-option in upload subcommandMasaya Suzuki2017-11-131-3/+10
| | | | Change-Id: I44836f8c66ded5a96cbf5431912e027e681f6529
* Provide more specific error message for symlinks errors on WindowsRenaud Paquay2017-08-311-2/+10
| | | | Change-Id: Ia6099beef37ae6b6143eba243fe7fbe02b74a9bb
* Replace all os.remove callsRenaud Paquay2017-08-311-14/+14
| | | | | | | | os.remove raises an exception when deleting read-only files on Windows. Replace all calls with calls to platform_utils.remove, which deals with deals with that issue. Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
* Implement islink, readlink and realpath using Win32 apiRenaud Paquay2017-08-311-13/+13
| | | | Change-Id: I18452cbb32d24db73601ad10485dbe6bb278731c
* Handle Windows line endings when reading binary filesRenaud Paquay2017-08-311-3/+3
| | | | | | | | | Without this change, '.git\HEAD' files, for examples, are sometime read incorrectly resulting in the current branch to be reset to "master" when running a "repo init -b xxx" on an already initialized repository. Change-Id: I48c7ef85ff81626edf156914329a560e14252f2a
* Merge "Add option '--no-cert-checks' for 'upload' sub command."David Pursehouse2017-08-301-4/+7
|\
| * Add option '--no-cert-checks' for 'upload' sub command.Łukasz Gardoń2017-08-231-4/+7
| | | | | | | | | | | | | | This option allow to bypass verification ssl certification while establishing connection with Gerrit to upload review. Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
* | Merge changes from topic "windows-support"David Pursehouse2017-08-301-11/+14
|\ \ | | | | | | | | | | | | | | | | | | | | | * changes: Port os.rename calls to work on Windows Workaround shutil.rmtree limitation on Windows Add support for creating symbolic links on Windows Make "git command" and "forall" work on Windows
| * | Port os.rename calls to work on WindowsRenaud Paquay2017-05-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | os.rename fails on Windows if the destination exists, so replace os.rename to platform_utils.rename which handles the platform differences. Change-Id: I15a86f10f65eedee5b003b80f88a0c28a3e1aa48
| * | Workaround shutil.rmtree limitation on WindowsRenaud Paquay2017-05-291-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, shutil.rmtree raises an exception when deleting readonly files on Windows. Replace all shutil.rmtree with platform_utils.rmtree, which adds an error handler to make files read-write when they can't be deleted. Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
| * | Add support for creating symbolic links on WindowsRenaud Paquay2017-05-291-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace all calls to os.symlink with platform_utils.symlink. The Windows implementation calls into the CreateSymbolicLinkW Win32 API, as os.symlink is not supported. Separate the Win32 API definitions into a separate module platform_utils_win32 for clarity. Change-Id: I0714c598664c2df93383734e609d948692c17ec5
* | | Print a message when fetching is skipped for an immutable refZac Livingston2017-08-251-4/+6
| |/ |/| | | | | | | | | | | | | | | The output indicates that fetching happens even when it is skipped. To avoid confusion, print a message when fetching is skipped for an immutable ref so that the user knows when and why a fetch is skipped. Change-Id: Id6e4812cebc5e57d379feb76a9d034af0b93043b
* | Revert "Migrate git-repo to create private changes rather than drafts"Jonathan Nieder2017-08-081-1/+9
| | | | | | | | | | | | | | | | This reverts commit d88f53e2b9634b9645f80b2d8e653beac24711fe. I merged it too hastily without paying enough attention to compatibility with released Gerrit versions. Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-13 12:26:40 +0000