summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* | | | | | | | | Check if SHA1 presents in repositoryAnatol Pomazau2012-04-231-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously repo had incorrect code that did not really check if sha1 presents in a project. It worked for tags though. Check if a revision (either tag or sha1) is present by using 'git rev_parse' functionality. Change-Id: I1787f3348573948573948753987394839487572b
* | | | | | | | | Add manifest groupsv1.8.2Colin Cross2012-04-138-7/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allows specifying a list of groups with a -g argument to repo init. The groups act on a group= attribute specified on projects in the manifest. All projects are implicitly labelled with "default" unless they are explicitly labelled "-default". Prefixing a group with "-" removes matching projects from the list of projects to sync. If any non-inverted manifest groups are specified, the default label is ignored. Change-Id: I3a0dd7a93a8a1756205de1d03eee8c00906af0e5 Reviewed-on: https://gerrit-review.googlesource.com/34570 Reviewed-by: Shawn Pearce <sop@google.com> Tested-by: Shawn Pearce <sop@google.com>
* | | | | | | | | Option for 'repo diff' to generate output suitable for 'patch' cmdpelya2012-04-132-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The -u option causes 'repo diff' to generate diff output with file paths relative to the repository root, so the output can be applied to the Unix 'patch' command. The name '-u' was selected for convenience, because both 'diff' and 'git diff' accept the option with the same name to generate an 'unified diff' output suitable for 'patch' command. Change-Id: I79c8356db4ed20ecaccc258b3ba139db76666fe0 Reviewed-on: https://gerrit-review.googlesource.com/34380 Reviewed-by: Shawn Pearce <sop@google.com> Tested-by: Shawn Pearce <sop@google.com>
* | | | | | | | | New flag for repo upload: --current_branch (--cbr)Daniel Sandler2012-04-061-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A convenient equivalent to `repo upload --br=<current git branch>`. Note that the head branch will be selected for each project uploaded by repo, so different branches may be uploaded for different projects. Change-Id: I10ad8ceaa63f055105c2d847c6e329fa4226dbaf
* | | | | | | | | Ignore /clone.bundle on HTTP 401, 403 and 404v1.8.1Shawn O. Pearce2012-03-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 401: Unauthorized, authentication may be required. This is usually handled internally by the HTTP client in Python. If it reaches our code in repo, the Python HTTP client didn't find a password in ~/.netrc that it could use. 403: Authentication was supplied, but is incorrect. It might be that the CDN doesn't want to offer this clone.bundle file to the client, but the Git fetch operation would still be successful. This might arise if branch level read controls were used in Gerrit Code Review and the /clone.bundle file contained branches not visible to the client. 404: The server has no /clone.bundle file available. In all of these cases, sliently ignore the /clone.bundle file HTTP error and let the Git operation take over. Change-Id: I1787f3cac86b017351952bbb793fe5874d83c72b
* | | | | | | | | Do not change branch.foo.merge in case of manifest syncAnatol Pomazau2012-03-201-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In case of manifest/smart sync repo changes ".merge" config option from branch to SHA. Doing 'repo upload' fails as repo tries to upload to a remote branch that looks like SHA (e.g. refs/for/23423423423423423423423) Do not update the .merge in case if revision is SHA. Change-Id: I9139708fa17f21eec5a7e23c3255333626bf529e
* | | | | | | | | sync: --no-clone-bundle disables the clone bundle supportv1.8.0Shawn O. Pearce2012-03-142-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ia9ed7da8451b273c1be620c3dd0dcad777b29096
* | | | | | | | | Permit - in URL schemes for special URLsShawn O. Pearce2012-03-142-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clients might be using their own special git-remote-* helper that has a hypen in its name. Permit - in the scheme part of the URL when trying to decide if it is an SSH URL and assume it is *not* SSH if the URL matches "foo-bar://" style. Change-Id: I7ba2d810a614f6e605a441d5972902c4a14e73fd
* | | | | | | | | Add support for Apache Digest authentication for repo init.Xiaodong Xu2012-03-142-6/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | repo tool supports only Basic authentication for now. For those who want to use this tool to manage their own projects, in case the administrator has configured the Apache server with Digest authentication method, users will fail to be authenticated when they run the command 'repo init'. Add the digest authentication password manager to the handler list will fix this issue. Since Git HTTP protocol will require the user be authenticated for fetch operation first before pushing commits to the remote, it is unlikely for the administrator to implement anonymous read (aka pull) access and write access (aka push) for authenticated user. Both read and write have to be authenticated. Be aware that the user may have to add an extra line in his ~/.netrc file: ------------------- account example.com ------------------- where 'example.com' is the realm for Apache Digest authentication. Change-Id: I76eb27b205554426d9ce1965deaaf727b87916cd Signed-off-by: Xiaodong Xu <stid.smth@gmail.com>
* | | | | | | | | repo status to print project name on clean gitsAli Utku Selen2012-03-122-14/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | repo status just prints "# on branch oprofile" if you have branched in clean status. This doesn't really tell which branch is meant. Instead we can use the same syntax with modified gits which will give us detailed information. Change-Id: I55fe5154d278e10a814281dd2ba501ec6e956730
* | | | | | | | | Add 'rebase="false"' attribute to the <project/> XML.Mike Pontillo2012-03-122-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This new attribute can prevent 'repo sync' from automatically rebasing. I hit a situation in where one of the git repositories I was tracking was actually an external repository that I wanted to pull commits into and merge myself. (NOT rebase, since that would lose the merge history.) In this case, I'm not using 'repo upload', I'm manually managing the merges to and from this repository. Everything was going great until I typed 'repo sync' and it rebased my manually-merged tree. Hence the option to skip it. Change-Id: I965e0dd1acb87f4a56752ebedc7e2de1c502dbf8
* | | | | | | | | Avoid missing content-length header in project.pyConley Owens2012-03-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Occassionally, the content-length may be missing when using urlib in python 2.6 and 2.7. This change assumes the value of the header is 0 if it doesn't exist Change-Id: Iaf1c8a796bc667823d4d7c30f9b617644b271d00
* | | | | | | | | Update SUBMITTING_PATCHESShawn O. Pearce2012-02-281-9/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The review server is now at gerrit-review.googlesource.com. Change-Id: I4be67fdb1876eb2e2af4420ac63557596b9e233b
* | | | | | | | | sync: Add manifest_name parameterChris Wolfe2012-01-261-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This parameter changes the manifest used by 'repo sync' for only this execution. It should be useful for developers wishing to get the repo temporarily into a known state, without clobbering their existing manifest. Tested by shifting Chrome OS between minilayout and full, and between several release-builder-generated manifests. Change-Id: I14194b665195b0e78f368d9ec8b8a83227af2627
* | | | | | | | | upload: Support uploading to Gerrit over https://v1.7.8.2Shawn O. Pearce2012-01-112-64/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If SSH is not available, Gerrit returns NOT_AVAILABLE to the /ssh_info query made by repo upload. In this case fallback to the /p/$PROJECT URL that Gerrit also exports and use that for uploads. Change-Id: I1e3e39ab709ecc0a692614a41a42446426f39c08
* | | | | | | | | Revert "Default repo manifest settings in git config"v1.7.8.1Shawn O. Pearce2011-11-303-43/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit ee1c2f5717fcc137ab887a4aae8a08d50a539b9a. This breaks a lot of buildbot systems. Rolling it back for now until we can understand what the breakage was and how to fix it.
* | | | | | | | | Describe the repo launch version in `repo version`Shawn O. Pearce2011-11-292-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | repo version v1.7.8 (from https://android.googlesource.com/tools/repo.git) repo launcher version 1.14 (from /home/sop/bin/repo) git version 1.7.8.rc2.256.gcc761 Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] Change-Id: Ifcbe5b0e226a1a6ca85455eb62e4da5e9a0f0ca0
* | | | | | | | | Improve error handling when reading loose refsv1.7.8Shawn O. Pearce2011-11-291-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When repo is trying to figure out branches the repository has by traversing refs/heads, add exception handling for readline. Change-Id: If3b2a3720c6496f52f629aa9a2539f186d6ec882
* | | | | | | | | Propagate result codes from subcmds to sys.exit().Daniel Sandler2011-11-291-10/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allows scripts driving repo to know when git failures have occurred, not just repo internal errors. Change-Id: Id20fbbb405c35a148e72c87b822da3f3bf93839c
* | | | | | | | | Don't prompt the user for name/email unless necessaryVictor Boivie2011-11-292-1/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the user has already configured a workspace, use these values when re-running 'repo init'. Otherwise, if the user has global name and e-mail set, use these. It's always possible to override this and be prompted by specifying --config-name when running 'repo init'. Change-Id: If45f0e4b14884071439fb02709dc5cb53f070f60
* | | | | | | | | Default repo manifest settings in git configVictor Boivie2011-11-293-6/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A default manifest URL can be specified using: git config --global repo-manifest.<id>.url <url> A default manifest server can be specified using: git config --global repo-manifest.<id>.server <url> A default git mirror reference can be specified using: git config --global repo-manifest.<id>.reference <path> This will allow the user to use 'repo init -u <id>' as a shorter alternative to specifying the full URL. Also, manifest server will not have to be specified in the manifest XML and the reference will not have to be specified on the command line. If they are, they will override these default values however. Change-Id: Ifdbc160bd5909ec7df9efb0c5d7136f1d9351754 Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
* | | | | | | | | Added remote destination branch information when uploading.Christer Fletcher2011-11-291-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several times one have done an upload only to later notice in gerrit that the upload was done to the wrong branch as the git has not yet been branched for the current git. This change will make repo print what the destination branch is when asking the user if she wants to go through with the upload. Change-Id: Ia9c3a92a6a04c022edfebf4f8d651ac062bb1f3b
* | | | | | | | | repo: capitalize default prompt charMike Frysinger2011-11-292-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is common in command line tools to indicate what the default answer will be if the user simply hits enter. In repo, the display is just "y/n" with no indication as to which is the default. So change the n to N in the messages since that is how repo operates. Change-Id: I81819ae630355072eb0365e59168b0921289498f
* | | | | | | | | Fixed UnicodeDecodeError while uploading changes.chenguodong2011-11-291-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When commit with comment that has non-ASCII characters, UnicodeDecodeError will be raised while uploading multiple project/branch changes. Because some strings in script are not str type, but unicode. So all the strings are decoded to unicode, and python use ascii to do this, it can not decode non-ASCII characters, so UnicodeDecodeError raised. Signed-off-by: chenguodong <chenguodong@huawei.com> Change-Id: I46447f489a4b9760a5899c7ba9d764b688594e46
* | | | | | | | | Fix typoAnatol Pomazau2011-11-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Idd68ad0a34fcf4bd4e18b0248f50187a539d610a
* | | | | | | | | Stabilize repo communication with subprocesses.Ulrik Sjölin2011-11-291-19/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make repo use the standard way in python to work with pipes. Communication via pipes to sub processes is done by calling communicate(). This will make repo not hang every now and then. Change-Id: Ibe2c4ecbdbcbe72f0b725ca50d54088e5646fc5d
* | | | | | | | | Add a sync flag that fetches only current branchAnatol Pomazau2011-11-032-21/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is also shortcuts in case if the "current branch" is a persistent revision such as tag or sha1. We check if the persistent revision is present locally and if it does - do no fetch anything from the server. This greately reduces sync time and size of the on-disk repo Change-Id: I23c6d95185474ed6e1a03c836a47f489953b99be
* | | | | | | | | Remove extra '/' in RemoteSpecv1.7.7.6Conley Owens2011-11-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | urljoin appends a '/' if only the domain is in the url path. This change strips that off before creating a RemoteSpec
* | | | | | | | | Correctly name projects when mirroringv1.7.7.5Conley Owens2011-10-201-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A bug introduced by relative urls caused projects such as manifest.git to be placed in the root directory instead of the directory they should by in. This fix creates and refers to a resolvedFetchUrl in the _XmlRemote class in order to get a fetchUrl that is never relative.
* | | | | | | | | Fix sync on Python 2.6.6v1.7.7.4Shawn O. Pearce2011-10-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python 2.6.6 has the same bug as Python 2.7, where HTTP authentication just stops working, but does not have the setter method to clear the retry counter. Work around by setting the field directly if it exists. Change-Id: I6a742e606bb7750dc66c33fc7c5d1310541db2c8 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | help: Fix help syncShawn O. Pearce2011-10-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | help sync crashed as sync required the manifest to be configured to create the option parser, as the default number of jobs is required. Change-Id: Ie75e8d75ac0e38313e4aab451cbb24430e84def5 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | upload: Honor REPO_HOST_PORT_INFO environment variablev1.7.7.3Shawn O. Pearce2011-10-111-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | REPO_HOST_PORT_INFO can be set to 'host:port' and be used instead of the review URL given in the manifest. Change-Id: I440bdecb2c2249fe5285ec5d0c28a937b4053450 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Fix Python 2.4 supportShawn O. Pearce2011-10-113-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I89521ae52fa564f0d849cc51e71fee65b3c47bab Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Work around Python 2.7 urllib2 bugShawn O. Pearce2011-10-112-34/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the remote is using authenticated HTTP, but does not have $GIT_URL/clone.bundle files in each repository, an initial sync would fail around 8 projects in due to the library not resetting the number of failures after getting a 404. Work around this by updating the retry counter ourselves. The urllib2 library is also not thread-safe. Make it somewhat safer by wrapping the critical section with a lock. Change-Id: I886e2750ef4793cbe2150c3b5396eb9f10974f7f Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Fix AttributeError: 'HTTPError' object has no attribute 'reason'v1.7.7.2Shawn O. Pearce2011-10-112-7/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Not every version of urllib2 supplies a reason object on the HTTPError exception that it throws from urlopen(). Work around this by using str(e) instead and hope the string formatting includes sufficient information. Change-Id: I0f4586dba0aa7152691b2371627c951f91fdfc8d Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Work around Python 2.7 failure to initialize base classShawn O. Pearce2011-10-111-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | urllib2 returns a malformed HTTPError object in certain situations. For example, urllib2 has a couple of places where it creates an HTTPError object with no fp: if self.retried > 5: # retry sending the username:password 5 times before failing. raise HTTPError(req.get_full_url(), 401, "basic auth failed", headers, None) When it does that, HTTPError's ctor doesn't call through to addinfourl's ctor: # The addinfourl classes depend on fp being a valid file # object. In some cases, the HTTPError may not have a valid # file object. If this happens, the simplest workaround is to # not initialize the base classes. if fp is not None: self.__super_init(fp, hdrs, url, code) Which means the 'headers' slot in addinfourl is not initialized and info() fails. It is completely insane that urllib2 decides not to initialize its own base class sometimes. Change-Id: I32a0d738f71bdd7d38d86078b71d9001e26f1ec3 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | sync: Fetch after applying bundle and retry after errorsv1.7.7.1Shawn O. Pearce2011-10-031-62/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After a $GIT_URL/clone.bundle has been applied to the new local repository, perform an incremental fetch using `git fetch` to ensure the local repository is up-to-date. This allows the hosting server to offer stale /clone.bundle files to bootstrap a new client. If a single git fetch fails, it may succeed again after a short delay. Transient failures are typical in environments where the remote Git server happens to have limits on how many requests it can serve at once (the anonymous git daemon, or an HTTP server). Wait a randomized delay between 30 and 45 seconds and retry the failed project once. This delay gives the site time to recover from a transient traffic spike, and the randomization makes it less likely that a spike occurs again from all of the same clients. Change-Id: I97fb0fcb33630fb78ac1a21d1a4a3e2268ab60c0 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | sync: Support downloading bundle to initialize repositoryv1.7.7Shawn O. Pearce2011-09-286-24/+250
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An HTTP (or HTTPS) based remote server may now offer a 'clone.bundle' file in each repository's Git directory. Over an http:// or https:// remote repo will first ask for '$URL/clone.bundle', and if present download this to bootstrap the local client, rather than relying on the native Git transport to initialize the new repository. Bundles may be hosted elsewhere. The client automatically follows a HTTP 302 redirect to acquire the bundle file. This allows servers to direct clients to cached copies residing on content delivery networks, where the bundle may be closer to the end-user. Bundle downloads are resumeable from where they last left off, allowing clients to initialize large repositories even when the connection gets interrupted. If a bundle does not exist for a repository (a HTTP 404 response code is returned for '$URL/clone.bundle'), the native Git transport is used instead. If the client is performing a shallow sync, the bundle transport is not used, as there is no way to embed shallow data into the bundle. Change-Id: I05dad17792fd6fd20635a0f71589566e557cc743 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Allow remote url to be relative to manifst urlConley Owens2011-09-281-6/+19
| | | | | | | | |
* | | | | | | | | sync: Update default -j flag from manifestShawn O. Pearce2011-09-261-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the manifest is updated and the default sync-j attribute was modified, honor it during this sync session if the user has not supplied a -j flag on the command line. Change-Id: I127ee5c779e2bbbb40b30bddc10ec1fa704b3bf3 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Ignore missing ~/.netrcv1.7.6.1Shawn O. Pearce2011-09-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ifa6065d57a6cb11ad57ddd44bc88d9690fe234ab Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | sync: Allow -j to have a default in manifestv1.7.6Shawn O. Pearce2011-09-223-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This permits manifest authors to suggest a number of parallel fetch operations against a remote server. For example, Gerrit Code Review servers support queuing of requests and processes them in first-in, first-out order. Running concurrent fetches can utilize multiple CPUs on the Gerrit server, but will also decrease overall operation latency by having the request put into the queue ready to execute as soon as a CPU is free. Change-Id: I3d3904acb6f63516bae4b071c510ad57a2afab18 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | sync: Limit -j to file descriptorsShawn O. Pearce2011-09-221-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each worker thread requires at least 3 file descriptors to run the forked 'git fetch' child to operate against the local repository. Mac OS X has the RLIMIT_NOFILE set to 256 by default, which means a sync -j128 often fails when the workers run out of pipes within the Python parent process. Change-Id: I2cdb14621b899424b079daf7969bc8c16b85b903 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Add global option --time to track executionShawn O. Pearce2011-09-221-1/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This prints a simple line after a command ends, providing information about how long it executed for using real wall clock time. Its mostly useful for looking at sync times. Change-Id: Ie0997df0a0f90150270835d94b58a01a10bc3956 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Support units in progress messagesShawn O. Pearce2011-09-191-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows our progress meter to be used for bytes transferred, by setting the units to KB or MB to let the user know the size. Change-Id: Ie8653d4a40d79439026c18bd51915845b2c5bba9 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Add support for url.*.insteadofShawn O. Pearce2011-09-191-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Teach repo how to resolve URLs using the url.insteadof feature that C Git natively uses during clone, fetch or push. This will later allow repo to resolve a URL before accessing it directly. We do not want to pre-resolve things and store the resolved URL into individual projects, as this makes it impossible for the user to undo the insteadof mapping at a later date. Change-Id: I0f62e811197c53fbc8a8be424e3cabf4ed07b4cb Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Support ~/.netrc for HTTP Basic authenticationShawn O. Pearce2011-09-191-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If repo tries to access a URL over HTTP and the user needs to authenticate, offer a match from ~/.netrc. This matches behavior with the Git command line client. Change-Id: I803f3c5d562177ea0330941350cff3cc1e1bef08 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Enhance HTTP supportShawn O. Pearce2011-09-192-18/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Setting REPO_CURL_VERBOSE=1 in the environment will register a debug level HTTPHandler on the urllib2 library, showing HTTP requests and responses on the stderr channel of repo. During any HTTP or HTTPS request created inside of the repo process, a custom User-Agent header is now defined: User-Agent: git-repo/1.7.5 (Linux) git/1.7.7 Python/2.6.5 Change-Id: Ia5026fb1e1500659bd2af27416d85e205048bf26 Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Honor http_proxy variable globallyShawn O. Pearce2011-09-112-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the http_proxy environment variable was set, honor it during the entire repo session for any Python created HTTP connections. Change-Id: Ib4ae833cb2cdd47ab0126949f6b399d2c142887d Signed-off-by: Shawn O. Pearce <sop@google.com>
* | | | | | | | | Change default REPO_URL to code.google.comShawn O. Pearce2011-09-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: If7700daf96fb8f3ee449e5774017272ef31b4b44