summaryrefslogtreecommitdiffstats
path: root/docs/smart-sync.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/smart-sync.md')
-rw-r--r--docs/smart-sync.md129
1 files changed, 129 insertions, 0 deletions
diff --git a/docs/smart-sync.md b/docs/smart-sync.md
new file mode 100644
index 00000000..1769572f
--- /dev/null
+++ b/docs/smart-sync.md
@@ -0,0 +1,129 @@
1# repo Smart Syncing
2
3Repo normally fetches & syncs manifests from the same URL specified during
4`repo init`, and that often fetches the latest revisions of all projects in
5the manifest. This flow works well for tracking and developing with the
6latest code, but often it's desirable to sync to other points. For example,
7to get a local build matching a specific release or build to reproduce bugs
8reported by other people.
9
10Repo's sync subcommand has support for fetching manifests from a server over
11an XML-RPC connection. The local configuration and network API are defined by
12repo, but individual projects have to host their own server for the client to
13communicate with.
14
15This process is called "smart syncing" -- instead of blindly fetching the latest
16revision of all projects and getting an unknown state to develop against, the
17client passes a request to the server and is given a matching manifest that
18typically specifies specific commits for every project to fetch a known source
19state.
20
21[TOC]
22
23## Manifest Configuration
24
25The manifest specifies the server to communicate with via the
26the [`<manifest-server>` element](manifest-format.md#Element-manifest_server)
27element. This is how the client knows what service to talk to.
28
29```xml
30 <manifest-server url="https://example.com/your/manifest/server/url" />
31```
32
33If the URL starts with `persistent-`, then the
34[`git-remote-persistent-https` helper](https://github.com/git/git/blob/HEAD/contrib/persistent-https/README)
35is used to communicate with the server.
36
37## Credentials
38
39Credentials may be specified directly in typical `username:password`
40[URI syntax](https://en.wikipedia.org/wiki/URI#Syntax) in the
41`<manifest-server>` element directly in the manifest.
42
43If they are not specified, `repo sync` has `--manifest-server-username=USERNAME`
44and `--manifest-server-password=PASSWORD` options.
45
46If those are not used, then repo will look up the host in your
47[`~/.netrc`](https://docs.python.org/3/library/netrc.html) database.
48
49When making the connection, cookies matching the host are automatically loaded
50from the cookiejar specified in
51[Git's `http.cookiefile` setting](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpcookieFile).
52
53## Manifest Server
54
55Unfortunately, there are no public reference implementations. Google has an
56internal one for Android, but it is written using Google's internal systems,
57so wouldn't be that helpful as a reference.
58
59That said, the XML-RPC API is pretty simple, so any standard XML-RPC server
60example would do. Google's internal server uses Python's
61[xmlrpc.server.SimpleXMLRPCDispatcher](https://docs.python.org/3/library/xmlrpc.server.html).
62
63## Network API
64
65The manifest server should implement the following RPC methods.
66
67### GetApprovedManifest
68
69> `GetApprovedManifest(branch: str, target: Optional[str]) -> str`
70
71The meaning of `branch` and `target` is not strictly defined. The server may
72interpret them however it wants. The recommended interpretation is that the
73`branch` matches the manifest branch, and `target` is an identifier for your
74project that matches something users would build.
75
76See the client section below for how repo typically generates these values.
77
78The server will return a manifest or an error. If it's an error, repo will
79show the output directly to the user to provide a limited feedback channel.
80
81If the user's request is ambiguous and could match multiple manifests, the
82server has to decide whether to pick one automatically (and silently such that
83the user won't know there were multiple matches), or return an error and force
84the user to be more specific.
85
86### GetManifest
87
88> `GetManifest(tag: str) -> str`
89
90The meaning of `tag` is not strictly defined. Projects are encouraged to use
91a system where the tag matches a unique source state.
92
93See the client section below for how repo typically generates these values.
94
95The server will return a manifest or an error. If it's an error, repo will
96show the output directly to the user to provide a limited feedback channel.
97
98If the user's request is ambiguous and could match multiple manifests, the
99server has to decide whether to pick one automatically (and silently such that
100the user won't know there were multiple matches), or return an error and force
101the user to be more specific.
102
103## Client Options
104
105Once repo has successfully downloaded the manifest from the server, it saves a
106copy into `.repo/manifests/smart_sync_override.xml` so users can examine it.
107The next time `repo sync` is run, this file is automatically replaced or removed
108based on the current set of options.
109
110### --smart-sync
111
112Repo will call `GetApprovedManifest(branch[, target])`.
113
114The `branch` is determined by the current manifest branch as specified by
115`--manifest-branch=BRANCH` when running `repo init`.
116
117The `target` is defined by environment variables in the order below. If none
118of them match, then `target` is omitted. These variables were decided as they
119match the settings Android build environments automatically setup.
120
1211. `${SYNC_TARGET}`: If defined, the value is used directly.
1222. `${TARGET_PRODUCT}-${TARGET_RELEASE}-${TARGET_BUILD_VARIANT}`: If these
123 variables are all defined, then they are merged with `-` and used.
1243. `${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}`: If these variables are all
125 defined, then they are merged with `-` and used.
126
127### --smart-tag=TAG
128
129Repo will call `GetManifest(TAG)`.