summaryrefslogtreecommitdiffstats
path: root/project.py
Commit message (Collapse)AuthorAgeFilesLines
...
* project: stop symlinking svn under .git/Mike Frysinger2022-01-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | This path only matters to users of `git svn` who manually run it in local projects after they get a full repo client checkout. With svn usage falling in general, and with the fact that the source checkout now symlinks its .git/ state to the internal projects/ path, we don't need to manage this anymore. It means the path won't be shared among multiple local projects that have the same remote, but so it goes. It was an optimization only, not functionality required for correctness. We want to simplify the internals to stop messing with git state, and this particular path doesn't seem worth the effort to maintain. We don't undo any existing svn symlinks if they exist since repo does not care about them, and their existence doesn't hurt anything. Bug: https://crbug.com/gerrit/15460 Change-Id: Ie8496b275bcc589771aa9f4ee874ed2ee6d5241d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327517 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: clean up now unused codeMike Frysinger2022-01-061-34/+4
| | | | | | | | | | | | | Now that we symlink worktree .git/ paths to .repo/projects/, we never set share_refs=True anywhere, which means all of this logic is dead code. Throw it all away. Do it as a separate commit to make the parent commit easier to review. Bug: https://crbug.com/gerrit/15273 Change-Id: If496d39029d3d3bd523ba24c603ce47a63ad9b51 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/326817 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jack Neus <jackneus@google.com>
* project: migrate worktree .git/ dirs to symlinksv2.19Mike Frysinger2021-12-011-36/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Historically we created a .git/ subdir in each source checkout and symlinked individual files to the .repo/projects/ paths. This layer of indirection isn't actually needed: the .repo/projects/ paths are guaranteed to only ever have a 1-to-1 mapping with the actual git checkout. So we don't need to worry about having files in .git/ be isolated. To that end, change how we manage the actual project checkouts from a dir full of symlinks (and a few files) to a symlink to the internal .repo/projects/ dir. This makes the code simpler & faster. The directory structure we have today is: .repo/ project-objects/chromiumos/third_party/kernel.git/ <paths omitted as not relevant to this change> projects/src/third_party/kernel/ v3.8.git/ config description -> …/project-objects/…/config FETCH_HEAD HEAD hooks/ -> …/project-objects/…/hooks/ info/ -> …/project-objects/…/info/ logs/ objects/ -> …/project-objects/…/objects/ packed-refs refs/ rr-cache/ -> …/project-objects/…/rr-cache/ src/third_party/kernel/ v3.8/ .git/ config -> …/projects/…/v3.8.git/config description -> …/project-objects/…/v3.8.git/description HEAD hooks/ -> …/project-objects/…/v3.8.git/hooks/ index info/ -> …/project-objects/…/v3.8.git/info/ logs/ -> …/projects/…/v3.8.git/logs/ objects/ -> …/project-objects/…/v3.8.git/objects/ packed-refs -> …/projects/…/v3.8.git/packed-refs refs/ -> …/projects/…/v3.8.git/refs/ rr-cache/ -> …/project-objects/…/v3.8.git/rr-cache/ The directory structure we have after this commit: .repo/ <nothing changes> src/third_party/kernel/ v3.8/ .git -> …/projects/…/v3.8.git Bug: https://crbug.com/gerrit/15273 Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695 Reviewed-by: Mike Nichols <mikenichols@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: Handle tag ref in "upstream" fieldRobin Schneider2021-11-161-1/+4
| | | | | | | | | | | | | | | | | | | | | | repo sync only handles a git tag properly when it is in the "revision" field. However, "revision locked manifests" (`repo manifest --revision-as-HEAD`) specifies the tag in the "upstream" field. The issue is that this tag is not fetched. Only the commit that the tag points to is fetched. This cases issues as self._CheckForImmutableRevision() runs and comes to the conclusion that the tag was changed while in fact, it was just not fetched. This causes a full sync. File docs/manifest-format.md, section Element-project: > Attribute upstream: Name of the Git ref in which a sha1 can be found. Used when syncing a revision locked manifest in -c mode to avoid having to sync the entire ref space. Project elements not setting their own upstream will inherit this value. Change-Id: I0507d3a5f30aee8920a9f820bafedb48dd5db554 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323620 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Robin Schneider <ypid@riseup.net>
* project: init hooks in objdir onlyMike Frysinger2021-11-151-7/+4
| | | | | | | | | | | | | | | objdir is the .repo/project-objects/ dir based on the remote path. gitdir is the .repo/projects/ dir based on the local source checkout path. When we setup the gitdir, we symlink "hooks" to the one in the objdir. But when we go to initialize the hooks, we do it via gitdir. There is a 1-to-many mapping from project-objects to projects, so initializing via gitdir can be repetitive. Collapse the hook init logic to the objdir init path. Change-Id: I828fca60ce6e125d6706c709cdb2797faa40aa50 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323815 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: fix format string in error messageLaMont Jones2021-11-051-1/+1
| | | | | | | | | BUG=None Change-Id: I0b195fd919c6db8cb3547e8d6f4c733f2bd4a535 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322735 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Xin Li <delphij@google.com>
* Update revisionId if required when using extend-projectMichael Kelly2021-09-281-8/+9
| | | | | | | | | | | | | When a hard revision ID is provided in a regular project tag then the revisionId is updated as well if it is a commit hash. The difference is that if the revisionExpr is a commit, git-repo needs to update refs/remotes/m/master with update-ref not symbolic-ref, as the latter must refer to another ref, not to a specific commit. Change-Id: I215a62dabb30225e480ad2c731416d775fc0c750 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310963 Tested-by: Michael Kelly <mkelly@arista.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Add 'dest-path' to extend-project to support changing pathMichael Kelly2021-09-281-17/+22
| | | | | | | | | | This allows us to move the repository to a new location in the source tree without having to remove-project + add a new project tag. Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Michael Kelly <mkelly@arista.com>
* make file removal a bit more robustMike Frysinger2021-09-281-14/+6
| | | | | | | | | | | | Some of the file removal calls are subject to race conditions (if something else deletes the file), so extend our remove API to have an option to ignore ENOENT errors. Then update a bunch of random call sites to use this new functionality. Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195 Reviewed-by: Sean McAllister <smcallis@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* repo: properly handle remote annotations in manifest_xmlJack Neus2021-07-231-2/+18
| | | | | | | | | | BUG=b:192664812 TEST=tests/ Change-Id: I1aa50260f4a00d3cebbd531141e1626825e70127 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312643 Tested-by: Jack Neus <jackneus@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Force a fetch when superproject has a newer SHA1 for remote branch.v2.16.3Xin Li2021-07-201-0/+1
| | | | | | | | | | | | | | | | | | For older git-repo versions, we might have only fetched the SHA1 revision that was provided by the project, but have remote branch left intact as long as they exist. When the remote branch become stale, some repo operations like rebase would fail, and repo sync would not correct this situation. Fix this by tightening the requirement to also require the superproject provided SHA1 be an ancestor or equal to the tip-of-tree of the remote branch. Bug: [google internal] b/193798453 Change-Id: Ie34c5d860dabb1cbd9f822da929088ec69c79cf6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312642 Tested-by: Xin Li <delphij@google.com> Reviewed-by: Raman Tenneti <rtenneti@google.com>
* Fix an issue when syncing with --use-superproject and clone bundles.Xin Li2021-06-301-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | It is possible that a clone bundle contained the object referenced by the branch in the manifest and in the superproject, but not the branch itself (for example, the branch may be newly created from an existing branch, or is not vislble to the user downloading the clone bundle). When --use-superproject is enabled, because we are overriding revisionExpr with the SHA1 revision provided by the superproject, the verification step would succeed, but because the expected branch do not exist, it would confuse git-repo at a later time, as it is expecting the remote branch to exist in the local clone. In project.py, fix this by making SetRevisionId() to always remember the actual branch name and verify it in _CheckForImmutableRevision() so that we only skip the fetch step when both objects exists locally. Bug: [google internal] b/191974277 Change-Id: I49d3ca0667f524c8c45f416492faf95b1dd822fb Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310802 Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* project: fix error display when output_redir is disabledv2.15.2Mike Frysinger2021-05-201-4/+5
| | | | | | | | | | | | | We always pass in output_redir when syncing, but that's the common case: there are a few situations (like `repo init`) where we don't pass in a buffer, and if any errors show up in that case, we'd crash. Rely on the print function to handle this logic for us. Bug: https://crbug.com/gerrit/14568 Change-Id: I8cd47e82329797ffc42534418a3dfbd8429205be Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/307222 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* ssh: rewrite proxy management for multiprocessing usagev2.15Mike Frysinger2021-05-101-2/+2
| | | | | | | | | | | | | | | | We changed sync to use multiprocessing for parallel work. This broke the ssh proxy code as it's all based on threads. Rewrite the logic to be multiprocessing safe. Now instead of the module acting as a stateful object, callers have to instantiate a new ProxyManager class that holds all the state, an pass that down to any users. Bug: https://crbug.com/gerrit/12389 Change-Id: I4b1af116f7306b91e825d3c56fb4274c9b033562 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305486 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* ssh: move proxy usage to the sync subcommandMike Frysinger2021-05-101-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | The only time we really need ssh proxies is when we want to run many connections and reuse them. That only happens when running sync. Every other command makes at most two connections, and even then it's only one or none. So the effort of setting up & tearing down ssh proxies isn't worth it most of the time. The big reason we want to move this logic to sync is that it's now using multiprocessing for parallel work. The current ssh proxy code is all based on threads, which means none of the logic is working correctly. The current ssh design makes it hard to fix when all of the state lives in the global/module scope. So the first step to fixing this is top move the setup & teardown to the one place that really needs it: sync. No other commands will use proxies anymore, just direct connections. Bug: https://crbug.com/gerrit/12389 Change-Id: Ibd351acdec39a87562b3013637c5df4ea34e03c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305485 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: fix print error when handling server errorMike Frysinger2021-05-041-1/+1
| | | | | | | | | | | When converting this logic from print() to the output buffer, this error codepath should have dropped the use of the file= redirect. Bug: https://crbug.com/gerrit/14482 Change-Id: Ib484924a2031ba3295c1c1a5b9a2d816b9912279 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305142 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init/sync: add --no-tags to match --tagsMike Frysinger2021-05-041-3/+3
| | | | | | | | | | | | | While this provides a way to undo earlier command line options (e.g. `repo sync --tags --no-tags`) which can be helpful for scripting & automation, this more importantly allows the user to override the manifest settings for syncing tags from a project. Bug: https://crbug.com/gerrit/12401 Change-Id: Id4c36cd82e6ca7cb073b5d63a09f6c7ccdebba83 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304904 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* add --no-current-branch option to invert --current-branchMike Frysinger2021-05-041-2/+2
| | | | | | | | | | | | | | | | | For most commands, this is more about providing a way to undo earlier command line options (e.g. `repo info -c --no-current-branch`) which can be helpful for scripting & automation. But for the sync command, this is helpful to undo the setting that exists in the manifest itself. With this in place, tweak the sync current_branch_only logic to only apply the manifest settings when the user hasn't specified a command line option. Bug: https://crbug.com/gerrit/12401 Change-Id: I21e2384624680cc740d1b5d1e49c50589d2fe6a0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304903 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: fix m/ generation when switching manifest branchesMike Frysinger2021-05-021-8/+8
| | | | | | | | | | | | | We were updating the per-checkout m/ pseudo ref when syncing, but we only created the common m/ redirect when initializing a project for the first time. This is fine unless the user switches the manifest branch in an existing project, then we never create that redirect. Bug: https://crbug.com/gerrit/14468 Change-Id: I5325e7e602dcb4ce150bef258901ba5e9fdea461 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304822 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: Fix a corner case when both superproject and depth used.Xin Li2021-04-291-0/+5
| | | | | | | | | | | | | | | | When depth is used, we would fetch only SHA1 when superproject is used, as the result, only the manifest branch is being recorded, and commands like repo start would fail. Fix this by saving the upstream branch value in the overlay manifest and add the upstream branch to fetch list. Bug: [google internal] b/185951360 Change-Id: Ib36f56067723f2572ed817785b31cc928ddfec0a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304562 Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Xin Li <delphij@google.com>
* sync: Fix exception in an exsiting clone (without partial-clone).v2.14.2Raman Tenneti2021-04-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Default the partial_clone_exclude argument to an empty set. Fixes the following report by Emil Medve. With this change (up to v2.14.1), on an existing "normal" clone (without partial-clone options) I'm seeing this traceback during `repo selfupdate`: Traceback (most recent call last): File ".../.repo/repo/main.py", line 630, in <module> _Main(sys.argv[1:]) File ".../.repo/repo/main.py", line 604, in _Main result = run() File ".../.repo/repo/main.py", line 597, in <lambda> run = lambda: repo._Run(name, gopts, argv) or 0 File ".../.repo/repo/main.py", line 261, in _Run result = cmd.Execute(copts, cargs) File ".../.repo/repo/subcmds/selfupdate.py", line 54, in Execute if not rp.Sync_NetworkHalf(): File ".../.repo/repo/project.py", line 1091, in Sync_NetworkHalf if self.name in partial_clone_exclude: TypeError: argument of type 'NoneType' is not iterable $ ./run_tests -v Change-Id: I71e744e4ef2a37b13aa9ba42eba3935e78c4e40a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304082 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* sync: cleanup sleep+retry logic a bitv2.14Mike Frysinger2021-04-191-14/+18
| | | | | | | | | | | | | | | | | | | Make sure we print a message whenever we retry so it's clear to the user why repo is pausing for a long time, and why repo might have passed even though it displayed some errors earlier. Also unify the sleep logic so we don't have two independent methods. This makes it easier to reason about. Also don't sleep if we're in the last iteration of the for loop. It doesn't make sense to and needlessly slows things down when there are real errors. Bug: https://crbug.com/gerrit/12494 Change-Id: Ifceace5b2dde75c2dac39ea5388527dd37376336 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303402 Reviewed-by: Sam Saccone 🐐 <samccone@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* init: Added --partial-clone-exclude option.Raman Tenneti2021-04-131-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | partial-clone-exclude option excludes projects during partial clone. This is a comma-delimited project names (from manifest.xml). This option is persisted and it is used by the sync command. A project that has been unparital'ed will remain unpartial if that project's name is specified in the --partial-clone-exclude option. The project name should match exactly. Added $ ./run_tests -v Bug: [google internal] b/175712967 "I can't "unpartial" my androidx-main checkout" $ rm -rf androidx-main/ $ mkdir androidx-main/ $ cd androidx-main/ $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml $ repo_dev sync -c -j8 + Verify a project is partial $ cd frameworks/support/ $ git config -l | grep 'partial' + Unpartial a project. $ /google/bin/releases/android/git_repack/git_unpartial + Verify project is unpartial $ git config -l | grep 'partial' $ cd ../.. + Exclude the project from being unparial'ed after init and sync. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml + Verify project is unpartial $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. + Remove the project from exclude list and verify that project is partially cloned. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* manifest: allow toplevel project checkoutsv2.13.5Mike Frysinger2021-03-121-0/+12
| | | | | | | | | | | | | | Re-allow checking out projects to the top of the repo client checkout. We add checks to prevent checking out files under .repo/ as that path is only managed by us, and projects cannot inject content or settings into it. Bug: https://crbug.com/gerrit/14156 Bug: https://crbug.com/gerrit/14200 Change-Id: Id6bf9e882f5be748442b2c35bbeaee3549410b25 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299623 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* prune: minor optimization & robustificationMike Frysinger2021-03-121-0/+5
| | | | | | | | | | | | | If the current project doesn't have any local branches, then there's nothing to prune, so return right away. This avoids running a few git commands when we aren't actually going to use the results, and it avoids checking repository validity. Since we aren't going to do anything in here, no need to check it. Change-Id: Ie9d5c75a954e42807477299f3e5a63a92fac138b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299742 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: fix diff printing with embedded %v2.13.4Mike Frysinger2021-03-091-1/+1
| | | | | | | | | | | | The recent commit 84230009ee4282b947482f0d4fc4fe9e9ebc9e01 ("project: make diff tools synchronous") broke repo diff if it includes % formats. Add an explicit format string to fix. Bug: https://crbug.com/gerrit/14208 Change-Id: Ie255a43c5b767488616b2b3dd15abc18f93bfab2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299402 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: fix variable typov2.13.2Mike Frysinger2021-03-031-1/+1
| | | | | | | | | Bug: https://crbug.com/gerrit/11293 Reported-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: I37bac58aa1dc9ecc10e29253d14ff9e6fb42427c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298942 Reviewed-by: Ian Kasprzak <iankaz@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: skip clone bundles when we've already initialized the object dirMike Frysinger2021-03-011-0/+6
| | | | | | | | | | | | | | The clone bundle logic assumes there is a one-to-one mapping between the projects/ and project-objects/ trees. When using shared projects (where we checkout different branches from the same project), this would lead us to fetching the same clone bundle multiple times. Automatically skip the clone bundle logic if the project-objects/ dir already exists. Bug: https://crbug.com/gerrit/10993 Change-Id: I82c6fa1faf8605fd56c104fcea2a43dd4eecbce4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298682 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: capture all git output by defaultMike Frysinger2021-02-251-6/+15
| | | | | | | | | | | | The default sync output should show a progress bar only for successful commands, and the error output for any commands that fail. Implement that policy here. Bug: https://crbug.com/gerrit/11293 Change-Id: I85716032201b6e2b45df876b07dd79cb2c1447a5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297905 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: fix http error retry logicMike Frysinger2021-02-251-4/+6
| | | | | | | | | | | | When sync moved to consume clone output, it merged stdout & stderr, but the retry logic in this function is based on stderr only. Move it over to checking stdout. Change-Id: I71bdc18ed25c978055952721e3a768289d7a3bd2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297902 Reviewed-by: Michael Mortensen <mmortensen@google.com> Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* diff: add --jobs supportMike Frysinger2021-02-231-1/+3
| | | | | | | | | Use multiprocessing to run diff in parallel. Change-Id: I61e973d9c2cde039d5eebe8d0fe8bb63171ef447 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297483 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
* use simpler super() magicMike Frysinger2021-02-191-3/+3
| | | | | | | | | | Python 3 has a simpler super() style so switch to it to make the code a little simpler and to stop pylint warnings. Change-Id: I1b3ccf57ae968d56a9a0bcfc1258fbd8bfa3afee Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297383 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: make diff tools synchronousMike Frysinger2021-02-181-52/+44
| | | | | | | | | | | | | | | | | | | These are the only users in the tree that process the output as it's produced. All others capture all the output first and then process the results. However, these functions still don't fully return until it's finished processing, and these funcs are in turn used in other synchronous code paths. So it's unclear whether anyone will notice that it's slightly slower or less interactive. Let's try it out and see if users report issues. This will allow us to simplify our custom GitCommand code and move it over to Python's subprocess.run, and will help fix interleaved output when running multiple commands in parallel (e.g. `repo diff -j8`). Change-Id: Ida16fafc47119d30a629a8783babeba890515de0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297144 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* git_config: add SetBoolean helperMike Frysinger2021-02-111-4/+1
| | | | | | | | | A little sugar simplifies the code a bit. Change-Id: Ie2b8a965faa9f9ca05c7be479d03e8e073cd816d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296522 Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Update _CheckForImmutableRevision to use git rev-listIan Kasprzak2021-02-051-1/+3
| | | | | | | | | | | | | | | | | | | | | _CheckForImmutableRevision is used to see if repo can skip fetching a project, but 'git rev-parse' with partial clone does a data fetch to accomplish this. Changed to use: 'git rev-list -1 --missing=allow-any <SHA>^0' which checks the local ref without fetching from the server first. Bug: [google internal] b/179477822 Testing: - Unit tests - Verified init/sync working on aosp-master - Verified wwith a pinned manifest that local ref check works (no fetch) Change-Id: If327b893c6658421f41df1f58c337f53b4c60ce6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296142 Reviewed-by: Dan Willemsen <dwillemsen@google.com> Tested-by: Ian Kasprzak <iankaz@google.com>
* sync: Added --use-superproject option and support for superproject.v2.12Raman Tenneti2021-01-211-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Added "--use-superporject" option to sync.py to fetch project SHAs from superproject. If there are any missing projects in superprojects, it prints the missing entries and exits. If there are no missing entries, it will use SHAs from superproject to fetch the projects from git. Tested the code with the following commands. $ ./run_tests tests/test_manifest_xml.py $ ./run_tests -v tests/test_git_superproject.py $ ./run_tests -v Tested the sync code by copying all the repo changes into my Android AOSP checkout and adding <superporject> tag to default.xml. With local modification to the code to print the status, .../WORKING_DIRECTORY$ repo sync --use-superproject repo: executing 'git clone' url: sso://android/platform/superproject repo: executing 'git ls-tree' Success: [] Bug: https://crbug.com/gerrit/13709 Tested-by: Raman Tenneti <rtenneti@google.com> Change-Id: Id18665992428dd684c04b0e0b3a52f46316873a0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293822 Reviewed-by: Mike Frysinger <vapier@google.com>
* drop pyversion & is_python3 checkingMike Frysinger2021-01-061-10/+1
| | | | | | | | | | | We're committed to Python 3 at this point, so purge all the is_python3 related dynamic checks. Bug: https://crbug.com/gerrit/10418 Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-3/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: detach HEAD in internal worktree checkout.Remy Böhmer2020-12-261-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When checkout is done with Git worktrees then the HEAD in the bare-git repositories point to the initialized default (e.g. 'refs/heads/master'). This default branch does not exist locally and is not automatically created. When a user now creates a branch in any git repository named 'master' then it is no longer possible to get rid of this branch, neither is it possible to switch to another branch and switch back to this master branch. Git concludes the 'master' branch is already checked out (in the bare Git) and that results in a lockdown of this master branch. To repoduce this issue, run these commands in a repo tree checked out with --worktree: - git checkout master # assuming the remote repo has a master branch, # a local tracking branch master is created here - git checkout -b temp - git checkout master # This one now fails - git branch -d master # fails too The failure is caused by Git assuming the master branch is checked out by the bare git repository since HEAD is pointing towards it. To workaround this, we always detach HEAD in the bare-git when syncing. We don't need it to point to a ref in general, but we would like it to be valid so git tools "just work" if they're run in here. Signed-off-by: Remy Bohmer <oss@bohmer.net> Change-Id: I15c96604363c41f0d01c42f533174393097daeb5 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/290985 Reviewed-by: Mike Frysinger <vapier@google.com>
* Fix --reference option under WindowsRemy Bohmer2020-11-231-1/+2
| | | | | | | | | | | | | | | | | | | | When intializing a new repo with the --reference option on Windows 10 the objects/info/alternates in each git repository is created with Windows line endings (\r\n), leading to the following error: error: object directory C:/<PATH_TO_MIRROR>/<REPO_NAME>.git/objects? does not exist; check .git/objects/info/alternates This can be fixed by simply using unix line endings on both Windows and unix platforms. Reported-by: Francisco Javier Alvarez Garcia <javier.alvarez.garcia.17@gmail.com> Follow-up-from: I268fe029ede68802c21037b0f2ae8a95afb85e48 Bug: https://crbug.com/gerrit/13208 Change-Id: I6da60c4ca957778b3c42ab6b9ad85c40483f0042 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289431 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Remy Bohmer <oss@bohmer.net>
* Always use Unix EOL for worktree .git and gitdir filesRemy Bohmer2020-11-201-4/+6
| | | | | | | | | | | | | | Worktree .git and gitdir reference files are written by Git with Unix line ending, even on Windows & macOS. The conversion to relative paths makes these files end with DOS line endings in Windows. The Git integration in Visual Studio 2019 cannot deal with these DOS line endings and considers these worktrees invalid. Signed-off-by: Remy Bohmer <github@bohmer.net> Change-Id: I088cfd994f3cc31db4e0ca7791fa0a4ee3ac222f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289310 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Remy Bohmer <linux@bohmer.net>
* project: do not update local published/ refs in dryrun modeMike Frysinger2020-11-201-4/+5
| | | | | | | | Bug: https://crbug.com/gerrit/13087 Change-Id: I197e6d6d07c7d325ac294b597d42e895f77c737f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289182 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* manifest_xml: refactor manifest parsing from client managementMike Frysinger2020-11-181-3/+3
| | | | | | | | | | | | | | We conflate the manifest & parsing logic with the management of the repo client checkout in a single class. This makes testing just one part (the manifest parsing) hard as it requires a full checkout too. Start splitting the two apart into separate classes to make it easy to reason about & test. Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Move RepoHook class from project.py file to dedicated fileRemy Bohmer2020-11-031-403/+1
| | | | | | | | | | | | The project.py file is huge and contains multiple classes. By moving it to seperate class files the code becomes more readable and maintainable. Signed-off-by: Remy Bohmer <github@bohmer.net> Change-Id: Ida9d99d31751d627ae1ea0373418080696d2e14b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/281293 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Remy Bohmer <linux@bohmer.net>
* init: use the remote default manifest branchMike Frysinger2020-09-091-0/+21
| | | | | | | | | | | | | | | | Instead of hardcoding "master" as our default, use the remote server's default branch instead. For most people, this should be the same as "master" already. For projects moving to "main", it means we'll use the new name automatically rather than forcing people to use -b main. For repositories that never set up a default HEAD, we should still use the historical "master" default. Bug: https://crbug.com/gerrit/13339 Change-Id: I4117c81a760c9495f98dbb1111a3e6c127f45eba Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280799 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* status: Use multiprocessing for `repo status -j<num>` instead of threadingKimiyuki Onaka2020-09-091-0/+7
| | | | | | | | | | | | | This change increases the speed of the command with parallelization with processes. The parallelization with threads doesn't work well, and increasing the number of jobs to many (8 threads ~) didn't increase the speed. Possibly, the global interpreter lock of Python affects. Bug: https://crbug.com/gerrit/12389 Change-Id: Icbe5df8ba037dd91422b96f4e43708068d7be924 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/279936 Tested-by: Kimiyuki Onaka <kimiyuki@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Fix Git base version for worktreeconfig extensionAdrien Bioteau2020-07-301-1/+1
| | | | | | | | | worktreeconfig extension only appears with version Git 2.20.0 Change-Id: I3ea8b7d9f8a1f7953e536edd77b09cbc4f8f3158 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/276700 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Adrien Bioteau <adrien.bioteau@gmail.com>
* project.py: Fix check for wild cardsv2.7Angel Petkov2020-05-051-3/+3
| | | | | | | | | | | | | | The intention of the check is to verify whether the target file name contains a wild card. The code, however, assumes that if the file is non-existent - it contains a wild card. This has the side effect that a target file that does not exist at the moment of the check is considered to contain a wild card, this leads itself to softlink not being created. Change-Id: I4e4cd7b5e1b8ce2e4b2edc9abf5a1147cd86242f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/265736 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Angel Petkov <apetkov86@gmail.com>
* project.py: Remove extraneous ','v2.6George Engelbrecht2020-04-151-1/+1
| | | | | | | | | | Bug: https://crbug.com/1061473 Change-Id: I0f02f122d6313679c1ae5ad6fb4e05f68b764186 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263112 Tested-by: George Engelbrecht <engeg@google.com> Reviewed-by: George Engelbrecht <engeg@google.com> Reviewed-by: SPA SARC <spanc.sarc@gmail.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* project: have clone.bundle failures print better diagnosticsGeorge Engelbrecht2020-04-151-1/+3
| | | | | | | | | Bug: https://crbug.com/1061473 Change-Id: If066dc56ca575720bfb25c1a9892dbd6f4af15c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/261852 Tested-by: George Engelbrecht <engeg@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-13 18:53:50 +0000