summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* hooks: support external hooks running different Python versionMike Frysinger2019-07-274-21/+224
| | | | | | | | | | | | | | | | | | | | | | | | | As we convert repo to support Python 3, the version of Python that we use might not be the version that repo hooks users have written for. Since repo upgrades are not immediate, and not easily under direct control of end users (relative to the projects maintaining the hook code), allow hook authors to declare the version of Python that they want to use. Now repo will read the shebang from the hook script and compare it against the version of Python repo itself is running under. If they differ, we'll try to execute a separate instance of Python and have it load & execute the hook. If things are compatible, then we still use the inprocess execution logic that we have today. This allows repo hook users to upgrade on their own schedule (they could even upgrade to Python 3 ahead of us) without having to worry about their supported version being exactly in sync with repo's. Bug: https://crbug.com/gerrit/10418 Change-Id: I97c7c96b64fb2ee465c39b90e9bdcc76394a146a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/228432 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Add support for partial clone.v1.13.4Xin Li2019-07-166-28/+206
| | | | | | | | | | | | | A new option, --partial-clone is added to 'repo init' which tells repo to utilize git's partial clone functionality, which reduces disk and bandwidth usage when downloading by omitting blob downloads initially. Different from restricting clone-depth, the user will have full access to change history, etc., as the objects are downloaded on demand. Change-Id: I60326744875eac16521a007bd7d5481112a98749 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* repo/main: add module docstringsMike Frysinger2019-07-122-0/+13
| | | | | | | | | This should help people get some bearings in the codebase. Change-Id: I951238fe617a3ecb04a47ead3809ec72c8fbf5a1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231232 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* use print() instead of sys.stdout.write()Mike Frysinger2019-07-112-8/+19
| | | | | | | | | | | | | | | | We're relying on sys.stdout.write() to flush its buffer which isn't guaranteed, and is not the case in Python 3. Change to use print() everywhere to be standard, and utilize the end= keyword to get the EOL semantics we need. We can't use print's flush= keyword as that's only in Python 3. Leave behind a TODO to clean it up when we can drop Python 2. Bug: https://crbug.com/gerrit/10418 Change-Id: I562128c7f1e6d154f4a6ecdf33a70fa2811dc2af Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/230392 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* run_tests: add a helper for invoking unittestsMike Frysinger2019-07-113-6/+76
| | | | | | | | | | | This makes it very easy for people to run all our unittests with just `./run_tests`. There doesn't seem to be any other way currently to quickly invoke any of the tests. Change-Id: I1f9a3745fa397a1e797bd64065c2ba7f338de4a1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/227613 Tested-by: David Pursehouse <dpursehouse@collab.net> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* Don't try to decode when checking clone bundlesPierre Tardy2019-07-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | This fix exception with python3 with stack-trace: error: Cannot fetch platform_external_grpc-grpc-java.git (UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 640: invalid start byte) [...] File "[...]project.py", line 2255, in _IsValidBundle if f.read(16) == '# v2 git bundle\n': File "/usr/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) Even if we ask 16 characters, python buffered decoder will try to decode more in the buffer The patch works for python2 and python3, and open the file in byte mode so that decoding is not attemped Signed-off-by: Pierre Tardy <tardyp@gmail.com> Change-Id: I837ae3c5cd724b34670fc2a84e853431f482b20d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/224642 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* main: user-agent: include full git version infoMike Frysinger2019-07-111-1/+1
| | | | | | | | | | | | | | We've been truncating the git version info in the user agent to the first three components. So given an example `git --version` like "2.22.0.510.g264f2c817a-goog", we were cutting it down to "2.22.0". For user-agent purposes, we usually want that full string, so use the original full value instead. Bug: https://crbug.com/gerrit/11144 Change-Id: I8ffe3186bdaac96164c34ac835a54bb3fc85527e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231056 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* git_command: drop custom version helperMike Frysinger2019-07-113-13/+48
| | | | | | | | | | | | Since ParseGitVersion can call `git --version` automatically, we don't need this duplicate version() helper anymore. The only other user is the `repo version` code, so convert that to version_tuple().full. Bug: https://crbug.com/gerrit/11144 Change-Id: I9d77822fc39f4ba28884d9183359169cabf5f17d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231055 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* repo: all ParseGitVersion to load git version info itselfMike Frysinger2019-07-111-4/+15
| | | | | | | | | | | | | All code that calls ParseGitVersion needs to run `git --version` itself and parse the output before passing it in. To avoid that duplication, allow ParseGitVersion to run `git --version` itself if ver_str=None. Bug: https://crbug.com/gerrit/11144 Change-Id: Ie07793ca57a40c0231af808df04a576118d5eea3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231054 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
* repo: return a namedtuple with full version infoMike Frysinger2019-07-111-2/+10
| | | | | | | | | | | | | | | We were returning an e.g. tuple(1,2,3), but that strips off the full version string which we might want in some places e.g. '1.2.3-rc3'. Change the return value to a namedtuple so we can pass back up the full version string. For code doing a compare with three elements (all code today), things still work fine as the namedtuple will DTRT in this scenario. Bug: https://crbug.com/gerrit/11144 Change-Id: Ib897b5df308116ad1550b0cf18f49afeb662423e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231053 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Remove double forall from "repo help forall" outputSolomon Kinard2019-07-081-1/+1
| | | | | | | | | | | | %prog represents the full subcommand ("repo" + subcommand name), not a Windows-style environment variable for "repo". The current help output shows repo forall% forall ... Correct the variable usage so it shows "repo forall ..." instead. Change-Id: I1fea55572428cc922ddf24ace1168a3d8f82dad0
* fix raise syntaxMike Frysinger2019-07-051-1/+1
| | | | | | | This takes a single argument (the error message), not multiple arguments that get formatted implicitly. Change-Id: Idfbc913ea9f93820edb7e955e9e4f57618c8cd1b
* project: rev_list: simplify executionMike Frysinger2019-07-051-6/+1
| | | | | | | | | | | | | | Currently we read the binary stream from the subprocess code directly before waiting for it to finish, but there's no need to do so as we aren't streaming the output to the user. This also means we pass up binary data to the caller as we don't go through GitCommand's internal logic which decodes the stream as utf-8. Simplify the code by calling Wait first, then splitting the entire captured output in one line. Bug: https://crbug.com/gerrit/10418 Change-Id: I7a57904be8cb546a229980fb79c829fc3df31e7d
* handle binary stream from urllib.request.urlopenMike Frysinger2019-07-042-2/+3
| | | | | | | | Python 3 returns bytes by default with urlopen. Adjust our code to handle that scenario and decode as necessary. Bug: https://crbug.com/gerrit/10418 Change-Id: Icf4cd80e7ef92d71a3eefbc6113f1ba11c32eebc
* Support clone bundle through persistent-http[s].Xin Li2019-06-261-5/+9
| | | | | Bug: https://crbug.com/gerrit/11075 Change-Id: I367c6bfe8da47d886c017a2ac614d4ccb3f8a438
* sync: use integer division with job countsMike Frysinger2019-06-141-2/+2
| | | | | | | | Neither of the fields here expect floats so make sure we use integer division when calculating things. Bug: https://crbug.com/gerrit/10418 Change-Id: Ibda068b16a7bba7ff3efba442c4bbff4415caa6e
* status: import print_functionMike Frysinger2019-06-141-0/+2
| | | | | | | | This module uses print() so make sure we import the print function. It doesn't really impact the current code due to the simple way it is calling print, but we should be sane to avoid future issues. Change-Id: I0b15344678c1dcb71207faa333c239b3fced1d62
* add license header to a few more filesMike Frysinger2019-06-132-0/+29
| | | | Change-Id: I24e6b1df5f15a8e71c0f4a9edac505a8902ec267
* repo: drop Python 3 warningMike Frysinger2019-06-131-4/+0
| | | | | | | Lets get people to start filing bugs :). Bug: https://crbug.com/gerrit/10418 Change-Id: I1d55bf0c60dbdbd6537d30b2cf9ea91d2928e387
* repo: restore use of print_functionMike Frysinger2019-06-131-89/+80
| | | | | | | | | | | We avoided this future import because Python 2.4 & 2.5 did not support it. We've dropped support for Python 2.6 at this point, and those versions are long dead. Since this workaround adds a bit of complexity to the codebase, drop it. Considering we are not running any actual tests against older versions, there's no sense in trying to support them anymore. Change-Id: Icda874861e8a8eb4fa07c624a9e7c5ee2a0da401
* set default file encoding to utf-8Mike Frysinger2019-06-1350-0/+53
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* tweak raise/dict syntax for Python 3 compatMike Frysinger2019-06-133-8/+8
| | | | | | | | | | | | | | | | Use the `raise` statement directly. Switch to using .items() instead of .iteritems(). Python 3 doesn't have .iteritems() as .items() is a generator, and these are small enough that the Python 2 overhead should be negligible. We have to run .keys() through list() in a few places as Python 3 uses a generator and we sometimes want to iterate more than once. That's why we don't change all .keys() or .items() calls -- most are in places where generators are fine. Bug: https://crbug.com/gerrit/10418 Change-Id: I469899d9b77ffd77ccabb831bc4b217407fefe6f
* repo: standardize help behaviorMike Frysinger2019-06-131-2/+2
| | | | | | | | | | Standard utilities exit normally/zero when users explicitly request --help, and they write to stdout. Exiting non-zero & using stderr is meant for incorrect tool usage instead. We're already doing this for `repo help <init|gitc-init>` calls, so lets fix `repo help` and `repo --help|-h` to match. Change-Id: Ia4f352b431c91eefef70dcafc11f00209ee69809
* wrapper: drop shebangMike Frysinger2019-06-131-2/+0
| | | | | | | | This isn't executable (+x), nor does it have a main func or code that would run if it were. It's simply an imported module like most others in here. Drop the shebang to avoid confusion. Change-Id: I5e2881eb1de5e809a3fa9e8f49220ed797034fb1
* Merge "docs: start a release document"Mike Frysinger2019-06-131-0/+167
|\
| * docs: start a release documentMike Frysinger2019-06-121-0/+167
| | | | | | | | Change-Id: I884639665c020338ec9ceeb1add5c3b862583674
* | diffmanifests: honor user-supplied manifest pathsBasil Gello2019-06-062-23/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current implementation ignores the user-specified paths to manifests. if the "repo diffmanifests" is invoked with absolute file paths for one or both manifests, the command fails with message: fatal: duplicate path ... in /tmp/manifest-old.xml Also the current implementation fails to expand the absolute path to manifest files if "repo diffmanifests" is invoked with relative paths, i.e "repo diffmanifests manifest-old.xml manifest-new.xml". fatal: manifest manifest-old.xml not found This commit fixes the first issue by disabling the local manifest discovery for diffmanifests command, and the second issue by expanding paths to manifests within "diffmanifests" sub-command. Test: repo manifest --revision-as-HEAD -o /tmp/manifest-old.xml repo sync repo manifest --revision-as-HEAD -o /tmp/manifest-new.xml repo diffmanifests /tmp/manifest-old.xml /tmp/manifest-new.xml Change-Id: Ia125d769bfbea75adb9aba81abbd8c636f2168d4 Signed-off-by: Vasyl Gello <vasek.gello@gmail.com>
* | Merge "platform_utils_win32: remove an unnecessary workaround"Jonathan Nieder2019-06-041-12/+5
|\ \ | |/ |/|
| * platform_utils_win32: remove an unnecessary workaroundРоман Донченко2019-03-211-12/+5
| | | | | | | | | | | | | | | | | | | | The comment in _create_symlink is incorrect. The return value of CreateSymbolicLink is as documented, it was just declared with the wrong return type. The actual return type is BOOLEAN, not BOOL. Fixing this allows us to simplify the code a bit. Change-Id: I4d2190a50d45ba41dd9814bf7079a5784fc0a366
* | project: Set config option to skip lfs process filterv1.13.3Francois Ferrand2019-05-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During sync, repo runs `git read-tree --reset -u -v HEAD` which causes git-lfs's smudge filter to run, which fails because git-lfs does not work with bare repositories. This was fixed in I091ff37998131e2e6bbc59aa37ee352fe12d7fcd to automatically disable this smudge filter. However, later versions of Git (2.11.0) introduced a new filter protocol [1], to avoid spawning a new command for each filtered file. This was implemented in Git-LFS 1.5.0 [2]. This patch fixes the issue by setting the git lfs process filter, in addition to the smudge filter. For any projects that have LFS objects, `git lfs pull` must still be executed manually afterwards. [1] https://github.com/git/git/commit/edcc85814c87ebd7f3b1b7d3979fac3dfb84d308 [2] https://github.com/git-lfs/git-lfs/pull/1617 Bug: https://crbug.com/gerrit/10911 Change-Id: I277fc68fdefc91514a2412b3887e3be9106cab48
* | Honor --depth during repo initNasser Grainawi2019-05-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | If a user is asking for a shallow clone of the repos, they probably expect a shallow clone of the manifest repo too. For very large manifest repos, this can be a huge space and time savings. For one real-world repo, a 'repo init --no-tags --current-branch' used 350MB of disk space and took 7 minutes. Adding --depth 1 and this change reduced it to 10MB and 2.5 minutes. Change-Id: I6fa662e174e623ede8861efc862ce26d65d4958d
* | Merge "Print project name when work tree initialization fails"Jonathan Nieder2019-05-161-1/+1
|\ \
| * | Print project name when work tree initialization failsMikhail Naganov2019-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | When syncing a lot of projects in parallel, it is not otherwise clear which one of them has failed to init work tree. Change-Id: I8edfb4955023389a499e99cfa511bdc0d2850ba2
* | | Use %topic=topic instead of deprecated /topic syntax on pushDavid Pursehouse2019-05-151-2/+3
|/ / | | | | | | | | Bug: https://crbug.com/gerrit/9930 Change-Id: Iefa202d42ef6e6b8b2b1a3f9b8baa5f0d65cbd60
* | Ignore submodules when calculating 'git diff-files'Eli Ribble2019-05-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | This allows projects to include submodules inside of projects that use repo without repo incorrectly believing the area is dirty just because a submodule has updates. This is in line with git porcelain commands which generally require a commandline flag to include submodules (git add, git rebase). Change-Id: Ide8a292162a42ab35145b5c4ca8ca0d020cdfe81
* | info: Use the non-formatting printer for headtextSebastian Schuberth2019-04-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If "repo init" was run in a path containing "%", "repo info" would fail printing the path with File ".repo/repo/color.py", line 173, in f return fmt % args TypeError: not enough arguments for format string as the "%" in the path name is interpreted as the start of a formatting specifier. Avoid that by using the non-formatting printer for headtext which does not require any formatting so there is no need to try to expand "%" sequences. Change-Id: Ie193b912191fe7cdabdce5c97bb100f0714f6e76 Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
* | Don't print "persistent ref" message when syncing quietlyTim Schumacher2019-04-151-2/+3
| | | | | | | | | | | | | | | | | | The newly introduced "Already have persistent ref" message prevents repo from overwriting the last line when syncing quietly. Omit the message when syncing quietly to clean up the output and to restore the previous behaviour. Change-Id: Idf42751c67f95924d6de50092ba54d4c6fe93096
* | sync: deleted unused repos in reversed order (children before parent)Kuang-che Wu2019-04-061-1/+2
|/ | | | | Bug: chromium:950002 Test: repo sync chromeos using release manifest file Change-Id: I613df6a1973eb36acd806a703e72f5172554bcc7
* sync: respect --force-sync when fetching updatesMike Frysinger2019-03-181-2/+6
| | | | | | | | | | | | | | | | | | If a tag is rewritten on the server (which is bad), trying to recover locally with `repo sync --force-sync` doesn't actually work. People have to manually delete things themselves to fix syncing. While tags should never be rewritten in practice, allow users to easily recover from broken servers. We updated some of these code paths already (see commit 6e53844f1edd3 "Allow clobbering of existing tags from remote."), but the incremental update flow was missed. Bug: b/120778183 Bug: chromium:932651 Test: delete local tag & recreate to diff commit, then check `repo sync` & `repo sync --force-sync` behavior Change-Id: I3648f7d2526732c06016b691a9a36c003157618d
* docs: fixed typo error.Kyunam Jo2019-03-181-1/+1
| | | | Change-Id: Ic3ec1bfb150ec932e05ba5eda43537784f1fdcda
* Merge "project: Relax the submodule name pattern to accept dots"Sebastian Schuberth2019-03-141-2/+2
|\
| * project: Relax the submodule name pattern to accept dotsSebastian Schuberth2019-03-111-2/+2
| | | | | | | | | | | | | | | | Even if dots are used as separators for Git config keys, they are not forbidden as part of submodule names. This fixes the issue of submodules with a name like e.g. "long.js" to be skipped from checkout. Change-Id: I77da07925ad207fa3d043067dfbbcb4a1ebdac4d
* | Merge "sync: Add option '--force-remove-dirty'"Oleksii Okolielov2019-03-111-3/+17
|\ \ | |/ |/|
| * sync: Add option '--force-remove-dirty'Oleksii Okolielov2019-03-111-3/+17
| | | | | | | | | | | | | | | | | | | | Forcefully remove dirty projects if option '--force-remove-dirty' is given. The '--force-remove-dirty' option can be used to remove previously used projects with uncommitted changes. WARNING: This may cause data to be lost since uncommitted changes may be removed with projects that no longer exist in the manifest. Change-Id: I844a6e943ded522fdc7b1b942c0a1269768054bc
* | docs: document a Python 3 migration planMike Frysinger2019-02-011-0/+32
| | | | | | | | | | Bug: https://crbug.com/gerrit/10418 Change-Id: I72d82ce3a2d9af45d942bb10de82340110864ea5
* | event_log: turn id generation from a generator to a func callv1.13.2Mike Frysinger2019-01-141-12/+11
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | Running lots of sync processes in parallel can hit the failure: Fetching projects: 23% (124/523)Exception in thread Thread-201: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/src/repo/subcmds/sync.py", line 278, in _FetchProjectList success = self._FetchHelper(opt, project, *args, **kwargs) File "/usr/local/src/repo/subcmds/sync.py", line 357, in _FetchHelper start, finish, success) File "/usr/local/src/repo/event_log.py", line 104, in AddSync event = self.Add(project.relpath, task_name, start, finish, success) File "/usr/local/src/repo/event_log.py", line 74, in Add 'id': (kind, next(self._next_id)), ValueError: generator already executing It looks like, while we lock the multiprocessing value correctly, the generator that wraps the value isn't parallel safe. Since we don't have a way of doing that (as it's part of the language), turn it into a plain function call instead. Bug: https://crbug.com/gerrit/10293 Change-Id: I0db03601986ca0370a1699bab32adb03e7b2910a
* init: Remove -c short option for --current-branchv1.13.1Ereth McKnight-MacNeil2018-12-202-3/+3
| | | | | | | This option conflicts with the gitc-init -c short option. Bug: https://crbug.com/gerrit/10200 Change-Id: I06f37564429ca0bd4c0bbea6066daae4f663c838
* Merge "README: link in new bug tracker"Mike Frysinger2018-12-201-1/+1
|\
| * README: link in new bug trackerMike Frysinger2018-12-201-1/+1
| | | | | | | | Change-Id: I043afc0b77e709919e49ce548dff47776fddaddf
* | Leverage the next keyword from python 2.7Eli Ribble2018-12-193-9/+10
|/ | | | | | | This is literally what the next keyword is for. https://www.python.org/dev/peps/pep-3114/ Change-Id: I843755910b847737b077ff2361ba3e04409db0f0
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-14 03:20:55 +0000