summaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* sync: fix manifest sync-j handlingMike Frysinger2022-12-081-0/+47
| | | | | | | | | | | | | Since --jobs defaults to 0, not None, we never pull the value out of the manifest. Treat values of 0 and None the same to fix. Bug: http://b/239712300 Bug: http://b/260908907 Change-Id: I9b1026682072366616825fd72f90bd90c10a252f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354254 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com> Reviewed-by: Sam Saccone <samccone@google.com>
* test_manifest_config_properties: use assertEqualDaniel Kutik2022-12-051-4/+4
| | | | | | | | | | | The method assertEquals is an deprecated alias for assertEqual. See: https://docs.python.org/3/library/unittest.html#deprecated-aliases Change-Id: Id94ba6d6055bdc18b87c53e8729902bb278855aa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354035 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
* Fix ManifestProject.partial_clone_exclude property.Joanna Wang2022-12-021-0/+79
| | | | | | | | | | Bug: b/256358360 Change-Id: Ic6e3a049aa38827123d0324c8b14157562c5986e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353574 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Joanna Wang <jojwang@google.com> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
* Fixed wrapper related warnings in testsDaniel Kutik2022-11-282-8/+9
| | | | | | | | | | | Multiple "Could not find reference" warnings in test_wrapper.py and test_git_command.py resolved. Test: tox Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: Ic254c378bbdae6bc3f8f29682ababb37db76adfe Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353235 Reviewed-by: Mike Frysinger <vapier@google.com>
* test_capture: allow both Unix and Windows line sepDaniel Kutik2022-11-281-1/+3
| | | | | | | | | | | On Linux/macOS we allow \n in the end of the line. On Windows we allow both \r\n and \n. Here we also allow Unix line seperators as tests might be excuted in for example git-shell. Change-Id: I3975b563cf95407da92e5479980e670eb748b30e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353181 Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* test_bad_path_name_checks: allow Windows path sepDaniel Kutik2022-11-281-1/+1
| | | | | | | | | | With this change if a path ends with '/' on Linux/macOS and ends with either '/' or '\' on Windows, the test will pass. Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: Id7d1b134f9c0bdf7ceaf149af304bbf90cbd7b21 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353180 Reviewed-by: Mike Frysinger <vapier@google.com>
* test: Fix char encoding issues on windowsDaniel Kutik2022-11-271-2/+2
| | | | | | | | | | | Some tests were failing due to Windows not using utf-8 by default when executing the tests. Enforcing usage of utf-8 resolves these issues. Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: If42f6be2a2b688a6105ecf4fcdb541aade24519a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353179 Reviewed-by: Mike Frysinger <vapier@google.com>
* test: fix path seperator errors on windowsDaniel Kutik2022-11-273-20/+20
| | | | | | | | | | Fixing multiple errors when running tests on Windows related to path seperator being different ('\' instead of '/'). Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: I26b44d092b925edecab46a4d88e77dd9dcb8df28 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353178 Reviewed-by: Mike Frysinger <vapier@google.com>
* trace: make test timeout after 2minDaniel Kutik2022-11-261-1/+1
| | | | | | | | | | Before this commit, the test was hanging forever when run on a Windows host. This should resolve that issue. Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com> Change-Id: Id9ea6d54926b797db3d2978a2ae2930088201eec Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353125 Reviewed-by: Mike Frysinger <vapier@google.com>
* Extract env building into a testable helper.v2.30Sam Saccone2022-11-161-0/+33
| | | | | | | | | | | | | Previously env dict building was untested and mixed with other mutative actions. Extract the dict building into a dedicated function and author tests to ensure the functionality is working as expected. BUG: b/255376186 BUG: https://crbug.com/gerrit/16247 Change-Id: I0c88e53eb285c5c3fb27f8e6b3a903aedb8e02a8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/351874 Reviewed-by: LaMont Jones <lamontjones@google.com> Tested-by: Sam Saccone <samccone@google.com>
* tests: Fix update-manpages test.LaMont Jones2022-11-161-1/+1
| | | | | | | Change-Id: I58d85e06edeb9208a782957acc982e996c026ed2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/351854 Reviewed-by: Sam Saccone <samccone@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* update-manpages: explicitly strip color codesLaMont Jones2022-11-142-0/+28
| | | | | | | | | | | | On some systems, help2man produces color codes in the output. Remove them to avoid manpage churn. Also begin adding unit tests. Change-Id: I3f0204b19d9cae524d3cb5fcfb61ee309b0931fc Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349655 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Xin Li <delphij@google.com>
* sync: clear preciousObjects when set in error.LaMont Jones2022-11-031-7/+48
| | | | | | | | | | | | | | If this is a project that is not using object sharing (there is only one copy of the remote project) then clear preciousObjects. To override this for a project, run: git config --replace-all repo.preservePreciousObjects true Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* Set tracing to always on and save to .repo/TRACE_FILE.Joanna Wang2022-11-034-0/+26
| | | | | | | | | | | | - add `--trace_to_stderr` option so stderr will include trace outputs and any other errors that get sent to stderr - while TRACE_FILE will only include trace outputs piggy-backing on: https://gerrit-review.googlesource.com/c/git-repo/+/349154 Change-Id: I3895a84de4b2784f17fac4325521cd5e72e645e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350114 Reviewed-by: LaMont Jones <lamontjones@google.com> Tested-by: Joanna Wang <jojwang@google.com>
* tests: Make the tests pass for Python < 3.8Peter Kjellerstedt2022-10-281-7/+7
| | | | | | | | | | | | | | | | Before Python 3.8, xml.dom.minidom sorted the attributes of an element when writing it to a file, while later versions output the attributes in the order they were created. Avoid these differences by sorting the attributes for each element before comparing the generated manifests with the expected ones. This corresponds to commit 5d58c18, but for new tests introduced since it was integrated. Change-Id: I5c360656a0968e6e8d57eb068c8e87da7dfa61c1 Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349917 Reviewed-by: LaMont Jones <lamontjones@google.com>
* manifest: allow extend-project to override dest-branch and upstreamErik Elmeke2022-09-201-0/+24
| | | | | | | | Bug: https://crbug.com/gerrit/16238 Change-Id: Id6eff34791525b3df690e160c911c0286331984b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/345144 Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Omit local_manifest groups from superproject override.v2.28LaMont Jones2022-07-152-3/+31
| | | | | | | | | | | | | | When we create superproject_override.xml, do not include projects that are present from local_manifests/*. Such projects are fully under the control of the local_manifests/ file. Bug: b/238934278 Test: manual, ./run_tests Change-Id: I40382ceb82d9cf7b8dc7b5f2abed3f6d4d80017e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340877 Tested-by: Xin Li <delphij@google.com> Reviewed-by: Xin Li <delphij@google.com> Reviewed-by: Sam Saccone 🐐 <samccone@google.com>
* tests: switch to tempfile.TemporaryDirectoryMike Frysinger2022-05-204-40/+20
| | | | | | | | | | Now that we don't need to support Python 2, we can switch to this API for better contextmanager logic. Change-Id: I2d03e391121886547e7808a3b5c3b470c411533f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337515 Reviewed-by: LaMont Jones <lamontjones@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* sync: refactor use of self.manifestLaMont Jones2022-04-141-1/+1
| | | | | | | | | | | We need to iterate over multiple manifests, and generally use the outer_client.manifest for multi-manifest support. This refactors the use of self.manifest into a chosen manifest. Change-Id: I992f21d610c929675e99555ece9c38df4b635839 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334699 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* manifest_xml: use Superproject to hold XML contentv2.23LaMont Jones2022-04-122-58/+55
| | | | | | | | | | | | | | | Always create Superproject when there is a <superproject> tag, and have it hold the XML content, similar to how other manifest elements are handled. This also adds SetQuiet and SetPrintMessages to Superproject consistent with manifest.SetUseLocalManifests. Change-Id: I522bf3da542006575799f0640c67f7052704f266 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Raman Tenneti <rtenneti@google.com> Tested-by: LaMont Jones <lamontjones@google.com>
* sync: respect `sync-c` manifest optionDaniel Andersson2022-04-081-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The documentation states that a `sync-c` attribute in the manifest file can set a default for whether only the current branch should be fetched or all branches. This seems to have been broken for some time. Commit 7356114 introduced the `--no-current-branch` CLI option and relied on getting `None` via `optparse` if neither `--current-branch` nor `--no-current-branch` was set to distinguish it from a boolean value. If `None` was received, it would read the value from the manifest option `sync-c`. The parsing went through the utility function `_GetCurrentBranchOnly` which returned `True` if `--current-branch` had been given on the command-line, or fell back on the "superproject" setting, which would either return `True` or `None`. This would incorrectly make `repo` fall back to the manifest setting even if the user had given `--no-current-branch` if no superproject was requested -- the manifest became "too powerful": Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No From manifest Yes True --current-branch No True --current-branch Yes True --no-current-branch No From manifest ← wrong --no-current-branch Yes True In commit 0cb6e92 the superproject configuration value reading changed from something that could return `None` to something that always returned a boolean. If it returned `False`, this would then incorrectly make `repo` ignore the manifest option even if neither `--current-branch` nor `--no-current-branch` had been given. The manifest default became useless: Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No False ← wrong Yes True --current-branch No True --current-branch Yes True --no-current-branch No False --no-current-branch Yes True By swapping the order in which the command-line option target and the superproject setting is evaluated, things should work as documented: Command-line Using superproject → `current_branch_only` ------------ ------------------ ----------------------- No From manifest Yes True --current-branch No True --current-branch Yes True --no-current-branch No False --no-current-branch Yes True Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175 Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270 Reviewed-by: Mike Frysinger <vapier@google.com>
* trace: allow writing traces to a socketJosh Steadmon2022-03-161-0/+62
| | | | | | | | | | | | | | | | | | | Git can write trace2 events to a Unix domain socket [1]. This can be specified via Git's `trace2.eventTarget` config option, which we read to determine where to log our own trace2 events. Currently, if the Git config specifies a socket as the trace2 target, we fail to log any traces. Fix this by adding support for writing to a Unix domain socket, following the same specification that Git supports. [1]: https://git-scm.com/docs/api-trace2#_enabling_a_target Change-Id: I928bc22ba04fba603a9132eb055141845fa48ab2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/332339 Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Josh Steadmon <steadmon@google.com>
* test_project: use os.readlink instead of Path.readlinkSebastian Wagner2022-01-111-1/+1
| | | | | | | | | | | Path.readlink is only available on Python 3.9, breaking compatibility with all python versions below. os.readlink is already used in other places of this file, so use it here as well. Change-Id: I5acf8f5334a3e7c8de9cea1939d7e2b9af5f30ae Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327844 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Sebastian Wagner <sebix@sebix.at>
* project: abort a bit earlier before migrating .git/Mike Frysinger2022-01-071-2/+26
| | | | | | | | | | | | | Verify all the .git/ paths will be handled by the migration logic before starting the migration. This way we still abort & log an error, but the user gets to see it before we put the tree into a state that they have to manually recover. Also add a few more known-safe-to-clobber paths. Bug: https://crbug.com/gerrit/15273 Change-Id: If49d69b341bc960ddcafa30da333fb5ec7145b51 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327557 Reviewed-by: Colin Cross <ccross@android.com> Tested-by: Mike Frysinger <vapier@google.com>
* project: migrate worktree .git/ dirs to symlinksv2.19Mike Frysinger2021-12-011-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Historically we created a .git/ subdir in each source checkout and symlinked individual files to the .repo/projects/ paths. This layer of indirection isn't actually needed: the .repo/projects/ paths are guaranteed to only ever have a 1-to-1 mapping with the actual git checkout. So we don't need to worry about having files in .git/ be isolated. To that end, change how we manage the actual project checkouts from a dir full of symlinks (and a few files) to a symlink to the internal .repo/projects/ dir. This makes the code simpler & faster. The directory structure we have today is: .repo/ project-objects/chromiumos/third_party/kernel.git/ <paths omitted as not relevant to this change> projects/src/third_party/kernel/ v3.8.git/ config description -> …/project-objects/…/config FETCH_HEAD HEAD hooks/ -> …/project-objects/…/hooks/ info/ -> …/project-objects/…/info/ logs/ objects/ -> …/project-objects/…/objects/ packed-refs refs/ rr-cache/ -> …/project-objects/…/rr-cache/ src/third_party/kernel/ v3.8/ .git/ config -> …/projects/…/v3.8.git/config description -> …/project-objects/…/v3.8.git/description HEAD hooks/ -> …/project-objects/…/v3.8.git/hooks/ index info/ -> …/project-objects/…/v3.8.git/info/ logs/ -> …/projects/…/v3.8.git/logs/ objects/ -> …/project-objects/…/v3.8.git/objects/ packed-refs -> …/projects/…/v3.8.git/packed-refs refs/ -> …/projects/…/v3.8.git/refs/ rr-cache/ -> …/project-objects/…/v3.8.git/rr-cache/ The directory structure we have after this commit: .repo/ <nothing changes> src/third_party/kernel/ v3.8/ .git -> …/projects/…/v3.8.git Bug: https://crbug.com/gerrit/15273 Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695 Reviewed-by: Mike Nichols <mikenichols@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* git_config: Fixed test.gitconfig getting updated when running tests.v2.17Raman Tenneti2021-09-302-29/+19
| | | | | | | | | | | | | | | | | | Moved test_GetSyncAnalysisStateData to GitConfigReadWriteTests class. Deleted [repo "syncstate*..] data from tests/fixtures/test.gitconfig. Tested: ./run_tests ... tests/test_git_config.py::GitConfigReadWriteTests::test_GetSyncAnalysisStateData PASSED [ 84%] ... Bug: https://crbug.com/gerrit/15103 Change-Id: I8cb89ce10b025994a045106c9c66dd243ae8ba50 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319557 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* trace2_event: Add remove_prefix to fix failing tests on Linux & macOS.Raman Tenneti2021-09-291-1/+9
| | | | | | | | | | | | | | | | | | removeprefix is available i python 3.9. Mac and Linux are running in a version below 3.9. Thus tests are failing with the following error: "AttributeError: 'str' object has no attribute 'removeprefix' " Replaced the removeprefix with custom function which we will delete once Linux and macOS versions are updated. Tested: $ ./run_tests Bug: [google internal] b/201453085 Change-Id: I9b4d564ff1176e1b4471805ef05472c1914cd9f9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319375 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Add 'dest-path' to extend-project to support changing pathMichael Kelly2021-09-281-0/+46
| | | | | | | | | | This allows us to move the repository to a new location in the source tree without having to remove-project + add a new project tag. Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Michael Kelly <mkelly@arista.com>
* superproject: Log syncstate's parameter as data-json it it is an array.Raman Tenneti2021-09-281-1/+1
| | | | | | | | | | | | | | | | | | | All the values of syncstate are strings, check the first byte and last byte to see if it is an array. For syncstate data, there were no false positives. Tested: $ repo_dev sync Verified event logged for argv is "data-json". $./run_tests Bug: [google internal] b/201102002 Change-Id: Id56adb532b80267f08d09147ac663cdd5987ce87 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319075 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* make file removal a bit more robustMike Frysinger2021-09-281-0/+50
| | | | | | | | | | | | Some of the file removal calls are subject to race conditions (if something else deletes the file), so extend our remove API to have an option to ignore ENOENT errors. Then update a bunch of random call sites to use this new functionality. Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195 Reviewed-by: Sean McAllister <smcallis@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* superproject: Log argv parameter of syncstate as 'data-json'.Raman Tenneti2021-09-271-4/+7
| | | | | | | | | | | | | | | | | | | | Fixed: "we need to make a special case for logging the argv; it should probably be a "data-json" event so that we log this directly as an array rather than an encoded string. Tested: $ repo_dev sync Verified event logged for argv is "data-json". $./run_tests Bug: [google internal] b/201102002 Change-Id: I18ccec79c73c8dc931cb8afc472b2361db8aea4c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319055 Reviewed-by: Josh Steadmon <steadmon@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: support a new revision attribute.Xin Li2021-09-272-1/+70
| | | | | | | | | | | | | | | | | | | | | | | Tested: $ ./run_tests Verified that a manifest that specified superproject revision would use the specified revision, and superproject will use the default revision. Note that this is a slight behavior change from earlier repo versions, which would always use the branch name of the manifest itself. However, the new behavior would be more consisitent with regular "project" element and would allow superproject be used even if it is not enabled for the particular manifest branch, so we have decided to make the change as it would provide more flexibility and better matches what other elements would do. Bug: [google internal] b/187868160 Change-Id: I35255ee347aff6e65179f7879d52931f168b477e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643 Tested-by: Xin Li <delphij@google.com> Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* manifest: make repo-hooks more robust wrt element orderingJack Neus2021-09-231-0/+13
| | | | | | | | | | | | | Currently, repo will fail to sync to a manifest if the definition of the repo-hooks project comes after the repo-hooks element. BUG=none TEST=new test, run_tests Change-Id: I0bf85625173492af6c6404d4b67543e96e670562 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318520 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Jack Neus <jackneus@google.com>
* sync: Log repo sync state events as 'data' events.Raman Tenneti2021-09-142-1/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git_trace2_event_log.py: + Added LogDataConfigEvents method to log 'data' events. Sync's current_sync_state and previous_sync_state are logged as 'data' events in the current log. It logs are key/value in the |config| argument. Each key is prefixed with |prefix| argument. The following are sample events that are logged during repo sync. {"event":"data", "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0", "thread":"MainThread", "time":"2021-09-14T18:16:19.935846Z", "key":"previous_sync_state/repo.syncstate.main.synctime", "value":"2021-09-14T17:27:11.573717Z"} {"event":"data", "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0", "thread":"MainThread", "time":"2021-09-14T18:16:19.955546Z", "key":"current_sync_state/repo.syncstate.main.synctime", "value":"2021-09-14T18:16:19.935979Z"} tests/test_git_trace2_event_log.py: + Added unit tests sync.py: + Changed logging calls to LogDataConfigEvents. Tested: $ ./run_tests Tested it by running the following command multiple times. $ repo_dev sync -j 20 repo sync has finished successfully Verified config data is looged in trace2 event logs. Bug: [google internal] b/199758376 Change-Id: I75fd830e90c1811ec28510538c99a2632b104e85 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317823 Reviewed-by: Josh Steadmon <steadmon@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: print messages if the manifest has superproject tag.v2.16.5Raman Tenneti2021-08-132-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) If the manifest has superproject tag (git_master, etc), then display error/warning messages (as it is doing today) 2) If the manifest doesn't have superproject tag (nest, chromeos manifests), then don't display any error/warning messages about superrproject (behave as though user has specified --no-use-superproject). 3) Print error/warning messages if --use-superproject passed as argument to repo sync. 4) No change in behavior for the repo init command. git_superproject.py: + Fixed typo in _WriteManifestFile method name + Superproject accepts print_message as an argument and it defaults to True. All messages that are printed to stderr are controlled by this flag. If it is True, then messages get printed. + Added PrintMessages function which return true if either --use-superproject is specified on the command line or if the manifest has a superproject tag. sync.py: + Displays the warning message if PrintMessgages are enabled and passes that as argument to superproject object. + Added 'hassuperprojecttag' trace2 log entry for analysis. We can find users/branches that are using superproject, but the manifest is missing the superproject tag. Tested: $ ./run_tests + Verified printing of messages with and without superproject tag, with with --use-superproject option. + aosp-master $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest $ repo_dev sync + A manifest without superproject tag. $ repo_dev init -m $(pwd)/manifest_7482982.xml $ repo_dev sync -n -c -j32 -m $(pwd)/manifest_7482982.xml Bug: [google internal] b/196411099 Change-Id: I92166dcad15a4129fab82edcf869e7c8db3efd4b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/314982 Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: prepend messages with - "repo superproject"Raman Tenneti2021-08-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Changed _LogError method to _LogWarning. Replaced 'repo error:' with "repo superproject warning:"(except IOError message, which is still an "repo superproject error:" message) Tested: $ ./run_tests Tested the errors and warnings by forcing the error/warning. $ repo_dev sync -j 20 --use-superproject platform/packages/apps/Music ... repo superproject warning: please file a bug using go/repo-bug to report missing commit_ids for: [] ... repo superproject error: cannot write manifest to : /sdc/android/src/aosp/.repo/exp-superproject/superproject_override.xml ... Bug: [google internal] b/193711236 Change-Id: Ia0b6c830e04cf18dfc1a2ce325181a5b1160e054 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/314642 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Ian Kasprzak <iankaz@google.com> Reviewed-by: Xin Li <delphij@google.com>
* sync: Remove '_' from the repo.syncstate.* keys when saved to config.v2.16.4Raman Tenneti2021-07-292-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | GitConfig doesn't save keys if the keys contain "_" characters. Some of the options like mp_update, use_superproject have underscores. This fixes issue with previous_sync_state missing some of the options. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the sync command multiple times and verifing previous_sync_state and current_sync_state have the same keys. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I16b52a164f9dd1633d7dad1d8cf6b151c629fcb1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313242 Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* sync: Added logging of repo sync state and config options for analysis.Raman Tenneti2021-07-292-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git_config.py: + Added SyncAnalysisState class, which saves the following data into the config object. ++ sys.argv, options, superproject's logging data. ++ repo.*, branch.* and remote.* parameters from config object. ++ current time as synctime. ++ Version number of the object. + All the keys for the above data are prepended with 'repo.syncstate.' + Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods to GitConfig object to save/get the above data. git_trace2_event_log.py: + Added LogConfigEvents method with code from DefParamRepoEvents to log events. sync.py: + superproject_logging_data is a dictionary that collects all the superproject data that is to be logged as trace2 event. + Sync at the end logs the previously saved syncstate.* parameters as previous_sync_state. Then it calls config's UpdateSyncAnalysisState to save and log all the current options, superproject logged data. docs/internal-fs-layout.md: + Added doc string explaining [repo.syncstate ...] sections of .repo/manifests.git/config file. test_git_config.py: + Added unit test for the new methods of GitConfig object. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the following command multiple times. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Xin Li <delphij@google.com>
* repo: properly handle remote annotations in manifest_xmlJack Neus2021-07-231-1/+28
| | | | | | | | | | BUG=b:192664812 TEST=tests/ Change-Id: I1aa50260f4a00d3cebbd531141e1626825e70127 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312643 Tested-by: Jack Neus <jackneus@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Gracefully ignore bad remove-project lineMichael Kelly2021-07-081-0/+50
| | | | | | | | | | | | | | | | Sometimes, we don't care if the remove project is referring to a non-existing project and we can just ignore it. This change allows us to ignore remove-project entries if the project that they refer to doesn't exist, making them effectively a no-op. Because this change breaks existing configuration, we allow this to be configuration controlled using the `optional` attribute in the remove-project tag. Change-Id: I6313a02983e81344eadcb4e47d7d6b037ee7420e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310964 Tested-by: Michael Kelly <mkelly@arista.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* Fix an issue when syncing with --use-superproject and clone bundles.Xin Li2021-06-302-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | It is possible that a clone bundle contained the object referenced by the branch in the manifest and in the superproject, but not the branch itself (for example, the branch may be newly created from an existing branch, or is not vislble to the user downloading the clone bundle). When --use-superproject is enabled, because we are overriding revisionExpr with the SHA1 revision provided by the superproject, the verification step would succeed, but because the expected branch do not exist, it would confuse git-repo at a later time, as it is expecting the remote branch to exist in the local clone. In project.py, fix this by making SetRevisionId() to always remember the actual branch name and verify it in _CheckForImmutableRevision() so that we only skip the fetch step when both objects exists locally. Bug: [google internal] b/191974277 Change-Id: I49d3ca0667f524c8c45f416492faf95b1dd822fb Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310802 Reviewed-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Xin Li <delphij@google.com>
* superproject - don't update the commit ids of projects that have revisionId.Raman Tenneti2021-06-291-2/+49
| | | | | | | | | | | | | | | | | | | | Pinned manifests and release manifest have revisionId set for all projects. For such projects don't update the commit ids. Tested the code with the following commands. $ ./run_tests -v $ repo_dev sync --use-superproject -j8 $ repo_dev sync -n -c -j32 -m $(pwd)/manifest_7482982.xml Bug: [google internal] b/191995372 Change-Id: I4681135b1d15f4a63527b6f0356d76ec842485d6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310582 Reviewed-by: Xin Li <delphij@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: Don't exit if superproject tag doesn't exist in manifest.v2.16Raman Tenneti2021-06-161-18/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Don't exit if there are missing commit ids in superproject. This change implements the following suggestion from delphij@: "we should note the event (so we know that --use-superproject but there were some errors, e.g. manifest didn't specify commit id for some reason, or if there is no superproject but --use-superproject is used), print out a message telling the use that this is not support, but continue as if --no-use-superproject was specified?" Changes: superproject: + Added git_trace2_event_log as an argument to the constructor. + Sync method returns SyncResult a NamedTuple of ++ success - True if sync of superproject is successful, or False. ++ fatal - True if caller should exit, Or False. + UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of ++ manifest_path - path name of the overriding manifest file instead of None ++ fatal - True if caller should exit, Or False + _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of ++ commit_ids - a dictionary with the projects/commit ids on success, otherwise None ++ fatal - True if caller should exit, Or False + Added _SkipUpdatingProjectRevisionId a helper function to see if a project's revision id needs to be updated or not. This function is used to exclude projects from local manifest file. + Added the following error events into git_trace2_event_log ++ If superproject is missing in a manifest ++ If there are missing commit ids for projects. command.py: + Deleted unused import - platform + Added git_trace2_event_log as a member so all subcmds can log error events. main.py: + Initialized git_trace2_event_log as a member of command object. init.py: + Deleted unused import - optparse init.py: + Called sys.exit only if Sync returns exit=True sync.py: + Called sys.exit only if Superproject's UpdateProjectsRevisionId returns exit=True + Reloaded the manifest only if manifest path is returned by UpdateProjectsRevisionId. If not, fall back to the old way of doing repo sync. test_git_superproject: + Added code to verify error events are being logged. + Added a test for no superproject tag + Added test for UpdateProjectsRevisionId not updating the revision id with the commit ids. Tested the code with the following commands. + Positive test case with aosp-master. $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. Your identity is: Raman Tenneti <rtenneti@google.com> If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .../android/aosp $ repo_dev sync -j40 --use-superproject remote: Total 12 (delta 4), reused 12 (delta 4) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... repo sync has finished successfully. + Negative test case without superproject tag. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml error: Cannot get project commit ids from manifest error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.589s repo sync has finished successfully. + Test for missing commit_id for a project. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint'] error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.364s repo sync has finished successfully. $ ./run_tests -v ... ...== 164 passed in 2.87s ==... Bug: [google internal] b/189371541 Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* repo: properly handle NoneType in Default/Remote equality checksJack Neus2021-06-111-0/+31
| | | | | | | | | | BUG=none TEST=none Change-Id: I4ccdbbc7ba4b6f6e20c6959db1b46fdb44ea2819 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308982 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Jack Neus <jackneus@google.com>
* superproject: add projects from local manifest to local::<filename> group.Raman Tenneti2021-06-101-4/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With repo sync --use-superproject, don't update the commit ids of every project that comes from local manifest. Tested the code with the following commands. $ ./run_tests -v + Test with local.xml 1. repo init --use-superproject -u persistent-https://googleplex-android.git.corp.google.com/a/platform/manifest 2. cd .repo cp -r /google/src/head/depot/google3/wireless/android/build_tools/translations/pipeline/local_manifests local_manifests cd .. local$ time repo_dev sync --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../local/.repo/exp-superproject/feb2c2847da5e274f3d530d5ab438af8-superproject.git: Initial setup for superproject completed. ... Bug: [google internal] b/189360443 Bug: [google internal] b/189139268 Bug: https://crbug.com/gerrit/14499 Change-Id: Ideaf268c294e9b500b2b9726ffbd733dd8d63004 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308822 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com>
* trace2_event_log: Added logging of error events.Raman Tenneti2021-06-091-0/+24
| | | | | | | | | | | | | | | Added error event in preperation for superproject to log errors. Testing: + Unit tests ./run_tests -v Bug: [google internal] b/189371541 Change-Id: Ife1dd28d52d9e9925b7b34ae913f8eb5fa19037c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308863 Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Xin Li <delphij@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
* superproject: revert not updating commit ids if remote is different.Raman Tenneti2021-06-081-45/+0
| | | | | | | | | | | | | | superproject supports multiple remotes. Get all commit ids from superproject for all projects that are in the manifest. $ ./run_tests -v Bug: [google internal] b/186395810 Change-Id: I6edce3918853a7a3a65aec5528e6a43a544eff53 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308862 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Jonathan Nieder <jrn@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* tests: Make the tests pass for Python < 3.8Peter Kjellerstedt2021-05-262-28/+55
| | | | | | | | | | | | | | Before Python 3.8, xml.dom.minidom sorted the attributes of an element when writing it to a file, while later versions output the attributes in the order they were created. Avoid these differences by sorting the attributes for each element before comparing the generated manifests with the expected ones. Bug: https://crbug.com/gerrit/14382 Change-Id: Ie2597727afcc48f9063a7261ad970e8a549f0587 Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303326 Reviewed-by: Mike Frysinger <vapier@google.com>
* superproject: Use bugurl from contactinfo in the missing commits error message.v2.15.1Raman Tenneti2021-05-181-1/+1
| | | | | | | | | | | | | | | | | | + In XmlManifest._Unload set 'bugurl' to Wrapper().BUG_URL. + contactinfo returns a namedtuple. + bug_url can be accessed as self._manifest.contactinfo.bugurl. Tested the code with the following commands. $ ./run_tests -v Added contactinfo tag to default.xml and verified that bugurl is used. Bug: [google internal] b/186220520. Change-Id: Iaafd6465e072b2e47a0a0b548bf6cb608a0b0a04 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/306342 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* ssh: rewrite proxy management for multiprocessing usagev2.15Mike Frysinger2021-05-101-4/+26
| | | | | | | | | | | | | | | | We changed sync to use multiprocessing for parallel work. This broke the ssh proxy code as it's all based on threads. Rewrite the logic to be multiprocessing safe. Now instead of the module acting as a stateful object, callers have to instantiate a new ProxyManager class that holds all the state, an pass that down to any users. Bug: https://crbug.com/gerrit/12389 Change-Id: I4b1af116f7306b91e825d3c56fb4274c9b033562 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305486 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-15 09:44:32 +0000