summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual/dev-manual-newbie.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual/dev-manual-newbie.xml')
-rw-r--r--documentation/dev-manual/dev-manual-newbie.xml1590
1 files changed, 1590 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml
new file mode 100644
index 0000000..694bb7c
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-newbie.xml
@@ -0,0 +1,1590 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='dev-manual-newbie'>
6
7<title>The Yocto Project Open Source Development Environment</title>
8
9<para>
10 This chapter helps you understand the Yocto Project as an open source development project.
11 In general, working in an open source environment is very different from working in a
12 closed, proprietary environment.
13 Additionally, the Yocto Project uses specific tools and constructs as part of its development
14 environment.
15 This chapter specifically addresses open source philosophy, using the
16 Yocto Project in a team environment, source repositories, Yocto Project
17 terms, licensing, the open source distributed version control system Git,
18 workflows, bug tracking, and how to submit changes.
19</para>
20
21<section id='open-source-philosophy'>
22 <title>Open Source Philosophy</title>
23
24 <para>
25 Open source philosophy is characterized by software development directed by peer production
26 and collaboration through an active community of developers.
27 Contrast this to the more standard centralized development models used by commercial software
28 companies where a finite set of developers produces a product for sale using a defined set
29 of procedures that ultimately result in an end product whose architecture and source material
30 are closed to the public.
31 </para>
32
33 <para>
34 Open source projects conceptually have differing concurrent agendas, approaches, and production.
35 These facets of the development process can come from anyone in the public (community) that has a
36 stake in the software project.
37 The open source environment contains new copyright, licensing, domain, and consumer issues
38 that differ from the more traditional development environment.
39 In an open source environment, the end product, source material, and documentation are
40 all available to the public at no cost.
41 </para>
42
43 <para>
44 A benchmark example of an open source project is the Linux Kernel, which was initially conceived
45 and created by Finnish computer science student Linus Torvalds in 1991.
46 Conversely, a good example of a non-open source project is the
47 <trademark class='registered'>Windows</trademark> family of operating
48 systems developed by <trademark class='registered'>Microsoft</trademark> Corporation.
49 </para>
50
51 <para>
52 Wikipedia has a good historical description of the Open Source Philosophy
53 <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>.
54 You can also find helpful information on how to participate in the Linux Community
55 <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>.
56 </para>
57</section>
58
59<section id="usingpoky-changes-collaborate">
60 <title>Using the Yocto Project in a Team Environment</title>
61
62 <para>
63 It might not be immediately clear how you can use the Yocto
64 Project in a team environment, or scale it for a large team of
65 developers.
66 One of the strengths of the Yocto Project is that it is extremely
67 flexible.
68 Thus, you can adapt it to many different use cases and scenarios.
69 However, these characteristics can cause a struggle if you are trying
70 to create a working setup that scales across a large team.
71 </para>
72
73 <para>
74 To help with these types of situations, this section presents
75 some of the project's most successful experiences,
76 practices, solutions, and available technologies that work well.
77 Keep in mind, the information here is a starting point.
78 You can build off it and customize it to fit any
79 particular working environment and set of practices.
80 </para>
81
82 <section id='best-practices-system-configurations'>
83 <title>System Configurations</title>
84
85 <para>
86 Systems across a large team should meet the needs of
87 two types of developers: those working on the contents of the
88 operating system image itself and those developing applications.
89 Regardless of the type of developer, their workstations must
90 be both reasonably powerful and run Linux.
91 </para>
92
93 <section id='best-practices-application-development'>
94 <title>Application Development</title>
95
96 <para>
97 For developers who mainly do application level work
98 on top of an existing software stack,
99 here are some practices that work best:
100 <itemizedlist>
101 <listitem><para>Use a pre-built toolchain that
102 contains the software stack itself.
103 Then, develop the application code on top of the
104 stack.
105 This method works well for small numbers of relatively
106 isolated applications.</para></listitem>
107 <listitem><para>When possible, use the Yocto Project
108 plug-in for the <trademark class='trade'>Eclipse</trademark> IDE
109 and other pieces of Application Development
110 Technology (ADT).
111 For more information, see the
112 "<link linkend='application-development-workflow'>Application
113 Development Workflow</link>" section as well as the
114 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
115 </para></listitem>
116 <listitem><para>Keep your cross-development toolchains
117 updated.
118 You can do this through provisioning either as new
119 toolchain downloads or as updates through a package
120 update mechanism using <filename>opkg</filename>
121 to provide updates to an existing toolchain.
122 The exact mechanics of how and when to do this are a
123 question for local policy.</para></listitem>
124 <listitem><para>Use multiple toolchains installed locally
125 into different locations to allow development across
126 versions.</para></listitem>
127 </itemizedlist>
128 </para>
129 </section>
130
131 <section id='best-practices-core-system-development'>
132 <title>Core System Development</title>
133
134 <para>
135 For core system development, it is often best to have the
136 build system itself available on the developer workstations
137 so developers can run their own builds and directly
138 rebuild the software stack.
139 You should keep the core system unchanged as much as
140 possible and do your work in layers on top of the core system.
141 Doing so gives you a greater level of portability when
142 upgrading to new versions of the core system or Board
143 Support Packages (BSPs).
144 You can share layers amongst the developers of a particular
145 project and contain the policy configuration that defines
146 the project.
147 </para>
148
149 <para>
150 Aside from the previous best practices, there exists a number
151 of tips and tricks that can help speed up core development
152 projects:
153 <itemizedlist>
154 <listitem><para>Use a
155 <ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>
156 (sstate) among groups of developers who are on a
157 fast network.
158 The best way to share sstate is through a
159 Network File System (NFS) share.
160 The first user to build a given component for the
161 first time contributes that object to the sstate,
162 while subsequent builds from other developers then
163 reuse the object rather than rebuild it themselves.
164 </para>
165 <para>Although it is possible to use other protocols for the
166 sstate such as HTTP and FTP, you should avoid these.
167 Using HTTP limits the sstate to read-only and
168 FTP provides poor performance.
169 </para></listitem>
170 <listitem><para>Have autobuilders contribute to the sstate
171 pool similarly to how the developer workstations
172 contribute.
173 For information, see the
174 <link linkend='best-practices-autobuilders'>Autobuilders</link>
175 section.</para></listitem>
176 <listitem><para>Build stand-alone tarballs that contain
177 "missing" system requirements if for some reason
178 developer workstations do not meet minimum system
179 requirements such as latest Python versions,
180 <filename>chrpath</filename>, or other tools.
181 You can install and relocate the tarball exactly as you
182 would the usual cross-development toolchain so that
183 all developers can meet minimum version requirements
184 on most distributions.</para></listitem>
185 <listitem><para>Use a small number of shared,
186 high performance systems for testing purposes
187 (e.g. dual six core Xeons with 24GB RAM and plenty of
188 disk space).
189 Developers can use these systems for wider, more
190 extensive testing while they continue to develop
191 locally using their primary development system.
192 </para></listitem>
193 <listitem><para>Enable the PR Service when package feeds
194 need to be incremental with continually increasing
195 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'>PR</ulink>
196 values.
197 Typically, this situation occurs when you use or
198 publish package feeds and use a shared state.
199 You should enable the PR Service for all users who
200 use the shared state pool.
201 For more information on the PR Service, see the
202 "<link linkend='working-with-a-pr-service'>Working With a PR Service</link>".
203 </para></listitem>
204 </itemizedlist>
205 </para>
206 </section>
207 </section>
208
209 <section id='best-practices-source-control-management'>
210 <title>Source Control Management (SCM)</title>
211
212 <para>
213 Keeping your
214 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
215 and any software you are developing under the
216 control of an SCM system that is compatible
217 with the OpenEmbedded build system is advisable.
218 Of the SCMs BitBake supports, the
219 Yocto Project team strongly recommends using
220 <link linkend='git'>Git</link>.
221 Git is a distributed system that is easy to backup,
222 allows you to work remotely, and then connects back to the
223 infrastructure.
224 <note>
225 For information about BitBake and SCMs, see the
226 BitBake manual located in the
227 <filename>bitbake/doc/manual</filename> directory of the
228 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
229 </note>
230 </para>
231
232 <para>
233 It is relatively easy to set up Git services and create
234 infrastructure like
235 <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>,
236 which is based on server software called
237 <filename>gitolite</filename> with <filename>cgit</filename>
238 being used to generate the web interface that lets you view the
239 repositories.
240 The <filename>gitolite</filename> software identifies users
241 using <filename>ssh</filename> keys and allows branch-based
242 access controls to repositories that you can control as little
243 or as much as necessary.
244 </para>
245
246 <note>
247 The setup of these services is beyond the scope of this manual.
248 However, sites such as these exist that describe how to perform
249 setup:
250 <itemizedlist>
251 <listitem><para><ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>:
252 Describes how to install <filename>gitolite</filename>
253 on the server.</para></listitem>
254 <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
255 All topics for <filename>gitolite</filename>.
256 </para></listitem>
257 <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
258 Documentation on how to create interfaces and frontends
259 for Git.</para></listitem>
260 </itemizedlist>
261 </note>
262 </section>
263
264 <section id='best-practices-autobuilders'>
265 <title>Autobuilders</title>
266
267 <para>
268 Autobuilders are often the core of a development project.
269 It is here that changes from individual developers are brought
270 together and centrally tested and subsequent decisions about
271 releases can be made.
272 Autobuilders also allow for "continuous integration" style
273 testing of software components and regression identification
274 and tracking.
275 </para>
276
277 <para>
278 See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
279 for more information and links to buildbot.
280 The Yocto Project team has found this implementation
281 works well in this role.
282 A public example of this is the Yocto Project
283 Autobuilders, which we use to test the overall health of the
284 project.
285 </para>
286
287 <para>
288 The features of this system are:
289 <itemizedlist>
290 <listitem><para>Highlights when commits break the build.
291 </para></listitem>
292 <listitem><para>Populates an sstate cache from which
293 developers can pull rather than requiring local
294 builds.</para></listitem>
295 <listitem><para>Allows commit hook triggers,
296 which trigger builds when commits are made.
297 </para></listitem>
298 <listitem><para>Allows triggering of automated image booting
299 and testing under the QuickEMUlator (QEMU).
300 </para></listitem>
301 <listitem><para>Supports incremental build testing and from
302 scratch builds.</para></listitem>
303 <listitem><para>Shares output that allows developer
304 testing and historical regression investigation.
305 </para></listitem>
306 <listitem><para>Creates output that can be used for releases.
307 </para></listitem>
308 <listitem><para>Allows scheduling of builds so that resources
309 can be used efficiently.</para></listitem>
310 </itemizedlist>
311 </para>
312 </section>
313
314 <section id='best-practices-policies-and-change-flow'>
315 <title>Policies and Change Flow</title>
316
317 <para>
318 The Yocto Project itself uses a hierarchical structure and a
319 pull model.
320 Scripts exist to create and send pull requests
321 (i.e. <filename>create-pull-request</filename> and
322 <filename>send-pull-request</filename>).
323 This model is in line with other open source projects where
324 maintainers are responsible for specific areas of the project
325 and a single maintainer handles the final "top-of-tree" merges.
326 </para>
327
328 <note>
329 You can also use a more collective push model.
330 The <filename>gitolite</filename> software supports both the
331 push and pull models quite easily.
332 </note>
333
334 <para>
335 As with any development environment, it is important
336 to document the policy used as well as any main project
337 guidelines so they are understood by everyone.
338 It is also a good idea to have well structured
339 commit messages, which are usually a part of a project's
340 guidelines.
341 Good commit messages are essential when looking back in time and
342 trying to understand why changes were made.
343 </para>
344
345 <para>
346 If you discover that changes are needed to the core layer of the
347 project, it is worth sharing those with the community as soon
348 as possible.
349 Chances are if you have discovered the need for changes, someone
350 else in the community needs them also.
351 </para>
352 </section>
353
354 <section id='best-practices-summary'>
355 <title>Summary</title>
356
357 <para>
358 This section summarizes the key recommendations described in the
359 previous sections:
360 <itemizedlist>
361 <listitem><para>Use <link linkend='git'>Git</link>
362 as the source control system.</para></listitem>
363 <listitem><para>Maintain your Metadata in layers that make sense
364 for your situation.
365 See the "<link linkend='understanding-and-creating-layers'>Understanding
366 and Creating Layers</link>" section for more information on
367 layers.</para></listitem>
368 <listitem><para>Separate the project's Metadata and code by using
369 separate Git repositories.
370 See the "<link linkend='yocto-project-repositories'>Yocto Project
371 Source Repositories</link>" section for information on these
372 repositories.
373 See the "<link linkend='getting-setup'>Getting Set Up</link>" section
374 for information on how to set up various Yocto Project related
375 Git repositories.</para></listitem>
376 <listitem><para>Set up the directory for the shared state cache
377 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
378 where it makes sense.
379 For example, set up the sstate cache on a system used
380 by developers in the same organization and share the
381 same source directories on their machines.
382 </para></listitem>
383 <listitem><para>Set up an Autobuilder and have it populate the
384 sstate cache and source directories.</para></listitem>
385 <listitem><para>The Yocto Project community encourages you
386 to send patches to the project to fix bugs or add features.
387 If you do submit patches, follow the project commit
388 guidelines for writing good commit messages.
389 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
390 section.</para></listitem>
391 <listitem><para>Send changes to the core sooner than later
392 as others likely run into the same issues.
393 For some guidance on mailing lists to use, see the list in the
394 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
395 section.
396 For a description of the available mailing lists, see the
397 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
398 section in the Yocto Project Reference Manual.
399 </para></listitem>
400 </itemizedlist>
401 </para>
402 </section>
403</section>
404
405<section id='yocto-project-repositories'>
406 <title>Yocto Project Source Repositories</title>
407
408 <para>
409 The Yocto Project team maintains complete source repositories for all Yocto Project files
410 at <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>.
411 This web-based source code browser is organized into categories by function such as
412 IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth.
413 From the interface, you can click on any particular item in the "Name" column and
414 see the URL at the bottom of the page that you need to clone a Git repository for
415 that particular item.
416 Having a local Git repository of the Source Directory (poky) allows you to
417 make changes, contribute to the history, and ultimately enhance the Yocto Project's
418 tools, Board Support Packages, and so forth.
419 </para>
420
421 <para>
422 Conversely, if you are a developer that is not interested in contributing back to the
423 Yocto Project, you have the ability to simply download and extract release tarballs
424 and use them within the Yocto Project environment.
425 All that is required is a particular release of the Yocto Project and
426 your application source code.
427 </para>
428
429 <para>
430 For any supported release of Yocto Project, you can go to the
431 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
432 select the "Downloads" tab and get a released tarball of the
433 <filename>poky</filename> repository or any supported BSP tarballs.
434 Unpacking these tarballs gives you a snapshot of the released
435 files.
436 <note>
437 The recommended method for setting up the Yocto Project
438 <link linkend='source-directory'>Source Directory</link> and the
439 files for supported BSPs (eg., <filename>meta-intel</filename>) is to
440 use <link linkend='git'>Git</link> to create a local copy of the
441 upstream repositories.
442 </note>
443 </para>
444
445 <para>
446 In summary, here is where you can get the project files needed for development:
447 <itemizedlist>
448 <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis>
449 This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto
450 Metadata Layers.
451 You can create local copies of Git repositories for each of these areas.</para>
452 <para>
453 <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
454 </para></listitem>
455 <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis>
456 This area contains index releases such as
457 the <trademark class='trade'>Eclipse</trademark>
458 Yocto Plug-in, miscellaneous support, poky, pseudo, installers for cross-development toolchains,
459 and all released versions of Yocto Project in the form of images or tarballs.
460 Downloading and extracting these files does not produce a local copy of the
461 Git repository but rather a snapshot of a particular release or image.</para>
462 <para>
463 <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" />
464 </para></listitem>
465 <listitem><para><emphasis>"Downloads" page for the
466 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:</emphasis>
467 Access this page by going to the website and then selecting
468 the "Downloads" tab.
469 This page allows you to download any Yocto Project
470 release or Board Support Package (BSP) in tarball form.
471 The tarballs are similar to those found in the
472 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para>
473 <para>
474 <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
475 </para></listitem>
476 </itemizedlist>
477 </para>
478</section>
479
480<section id='yocto-project-terms'>
481 <title>Yocto Project Terms</title>
482
483 <para>
484 Following is a list of terms and definitions users new to the Yocto Project development
485 environment might find helpful.
486 While some of these terms are universal, the list includes them just in case:
487 <itemizedlist>
488 <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to
489 a recipe file.
490 Append files are known as BitBake append files and <filename>.bbappend</filename> files.
491 The OpenEmbedded build system expects every append file to have a corresponding
492 recipe (<filename>.bb</filename>) file.
493 Furthermore, the append file and corresponding recipe file
494 must use the same root filename.
495 The filenames can differ only in the file type suffix used (e.g.
496 <filename>formfactor_0.0.bb</filename> and <filename>formfactor_0.0.bbappend</filename>).
497 </para>
498 <para>Information in append files overrides the information in the similarly-named recipe file.
499 For an example of an append file in use, see the
500 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section.
501 </para></listitem>
502 <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis>
503 The task executor and scheduler used by
504 the OpenEmbedded build system to build images.
505 For more information on BitBake, see the BitBake documentation
506 in the <filename>bitbake/doc/manual</filename> directory of the
507 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
508 <listitem>
509 <para id='build-directory'><emphasis>Build Directory:</emphasis>
510 This term refers to the area used by the OpenEmbedded build system for builds.
511 The area is created when you <filename>source</filename> the setup
512 environment script that is found in the Source Directory
513 (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
514 or
515 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
516 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
517 variable points to the Build Directory.</para>
518
519 <para>
520 You have a lot of flexibility when creating the Build
521 Directory.
522 Following are some examples that show how to create the
523 directory.
524 The examples assume your
525 <link linkend='source-directory'>Source Directory</link> is
526 named <filename>poky</filename>:
527 <itemizedlist>
528 <listitem><para>Create the Build Directory inside your
529 Source Directory and let the name of the Build
530 Directory default to <filename>build</filename>:
531 <literallayout class='monospaced'>
532 $ cd $HOME/poky
533 $ source &OE_INIT_FILE;
534 </literallayout></para></listitem>
535 <listitem><para>Create the Build Directory inside your
536 home directory and specifically name it
537 <filename>test-builds</filename>:
538 <literallayout class='monospaced'>
539 $ cd $HOME
540 $ source poky/&OE_INIT_FILE; test-builds
541 </literallayout></para></listitem>
542 <listitem><para>Provide a directory path and
543 specifically name the build directory.
544 Any intermediate folders in the pathname must
545 exist.
546 This next example creates a Build Directory named
547 <filename>YP-&POKYVERSION;</filename>
548 in your home directory within the existing
549 directory <filename>mybuilds</filename>:
550 <literallayout class='monospaced'>
551 $cd $HOME
552 $ source $HOME/poky/&OE_INIT_FILE; $HOME/mybuilds/YP-&POKYVERSION;
553 </literallayout></para></listitem>
554 </itemizedlist>
555 </para></listitem>
556 <listitem><para><emphasis>Build System:</emphasis> In the context of the Yocto Project,
557 this term refers to the OpenEmbedded build system used by the project.
558 This build system is based on the project known as "Poky."
559 For some historical information about Poky, see the
560 <link linkend='poky'>Poky</link> term.
561 </para></listitem>
562 <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation
563 and inheritance so that commonly used patterns can be defined once and then easily used
564 in multiple recipes.
565 Class files end with the <filename>.bbclass</filename> filename extension.
566 </para></listitem>
567 <listitem><para><emphasis>Configuration File:</emphasis> Configuration information in various
568 <filename>.conf</filename> files provides global definitions of variables.
569 The <filename>conf/local.conf</filename> configuration file in the
570 <link linkend='build-directory'>Build Directory</link>
571 contains user-defined variables that affect each build.
572 The <filename>meta-yocto/conf/distro/poky.conf</filename> configuration file
573 defines Yocto "distro" configuration
574 variables used only when building with this policy.
575 Machine configuration files, which
576 are located throughout the
577 <link linkend='source-directory'>Source Directory</link>, define
578 variables for specific hardware and are only used when building for that target
579 (e.g. the <filename>machine/beagleboard.conf</filename> configuration file defines
580 variables for the Texas Instruments ARM Cortex-A8 development board).
581 Configuration files end with a <filename>.conf</filename> filename extension.
582 </para></listitem>
583 <listitem><para id='cross-development-toolchain'>
584 <emphasis>Cross-Development Toolchain:</emphasis>
585 In general, a cross-development toolchain is a collection of
586 software development tools and utilities that run on one
587 architecture and allow you to develop software for a
588 different, or targeted, architecture.
589 These toolchains contain cross-compilers, linkers, and
590 debuggers that are specific to the target architecture.
591 </para>
592
593 <para>The Yocto Project supports two different cross-development
594 toolchains:
595 <itemizedlist>
596 <listitem><para>A toolchain only used by and within
597 BitBake when building an image for a target
598 architecture.</para></listitem>
599 <listitem><para>A relocatable toolchain used outside of
600 BitBake by developers when developing applications
601 that will run on a targeted device.
602 Sometimes this relocatable cross-development
603 toolchain is referred to as the meta-toolchain.
604 </para></listitem>
605 </itemizedlist>
606 </para>
607
608 <para>
609 Creation of these toolchains is simple and automated.
610 For information on toolchain concepts as they apply to the
611 Yocto Project, see the
612 "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>"
613 section in the Yocto Project Reference Manual.
614 You can also find more information on using the
615 relocatable toolchain in the
616 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project
617 Application Developer's Guide</ulink>.
618 </para></listitem>
619 <listitem><para><emphasis>Image:</emphasis> An image is the result produced when
620 BitBake processes a given collection of recipes and related Metadata.
621 Images are the binary output that run on specific hardware or QEMU
622 and for specific use cases.
623 For a list of the supported image types that the Yocto Project provides, see the
624 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
625 chapter in the Yocto Project Reference Manual.</para></listitem>
626 <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core,
627 a BSP, or an application stack.
628 For a discussion on BSP Layers, see the
629 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
630 section in the Yocto Project Board Support Packages (BSP)
631 Developer's Guide.</para></listitem>
632 <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
633 A term sometimes used for
634 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
635 </para></listitem>
636 <listitem><para id='metadata'><emphasis>Metadata:</emphasis>
637 The files that BitBake parses when building an image.
638 In general, Metadata includes recipes, classes, and
639 configuration files.
640 In the context of the kernel ("kernel Metadata"),
641 it refers to Metadata in the <filename>meta</filename>
642 branches of the kernel source Git repositories.
643 </para></listitem>
644 <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of Metadata originating
645 with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
646 This Metadata is found in the <filename>meta</filename> directory of the
647 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
648 <listitem><para><emphasis>Package:</emphasis> In the context of the Yocto Project,
649 this term refers to the packaged output from a baked recipe.
650 A package is generally the compiled binaries produced from the recipe's sources.
651 You "bake" something by running it through BitBake.</para>
652 <para>It is worth noting that the term "package" can, in general, have subtle
653 meanings. For example, the packages referred to in the
654 "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Packages</ulink>" section are
655 compiled binaries that when installed add functionality to your Linux
656 distribution.</para>
657 <para>Another point worth noting is that historically within the Yocto Project,
658 recipes were referred to as packages - thus, the existence of several BitBake
659 variables that are seemingly mis-named,
660 (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>,
661 <ulink url='&YOCTO_DOCS_REF_URL;#var-PRINC'><filename>PRINC</filename></ulink>,
662 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
663 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
664 </para></listitem>
665 <listitem><para id='poky'><emphasis>Poky:</emphasis> The term "poky" can mean several things.
666 In its most general sense, it is an open-source project that was initially developed
667 by OpenedHand. With OpenedHand, poky was developed off of the existing OpenEmbedded
668 build system becoming a build system for embedded images.
669 After Intel Corporation acquired OpenedHand, the project poky became the basis for
670 the Yocto Project's build system.
671 Within the Yocto Project source repositories, <filename>poky</filename>
672 exists as a separate Git repository
673 that can be cloned to yield a local copy on the host system.
674 Thus, "poky" can refer to the local copy of the Source Directory used to develop within
675 the Yocto Project.</para></listitem>
676 <listitem><para><emphasis>Recipe:</emphasis> A set of instructions for building packages.
677 A recipe describes where you get source code and which patches to apply.
678 Recipes describe dependencies for libraries or for other recipes, and they
679 also contain configuration and compilation options.
680 Recipes contain the logical unit of execution, the software/images to build, and
681 use the <filename>.bb</filename> file extension.</para></listitem>
682 <listitem>
683 <para id='source-directory'><emphasis>Source Directory:</emphasis>
684 This term refers to the directory structure created as a result of either downloading
685 and unpacking a Yocto Project release tarball or creating a local copy of
686 the <filename>poky</filename> Git repository
687 <filename>git://git.yoctoproject.org/poky</filename>.
688 Sometimes you might hear the term "poky directory" used to refer to this
689 directory structure.
690 <note>
691 The OpenEmbedded build system does not support file or directory names that
692 contain spaces.
693 Be sure that the Source Directory you use does not contain these types
694 of names.
695 </note></para>
696 <para>The Source Directory contains BitBake, Documentation, Metadata and
697 other files that all support the Yocto Project.
698 Consequently, you must have the Source Directory in place on your development
699 system in order to do any development using the Yocto Project.</para>
700
701 <para>For tarball expansion, the name of the top-level directory of the Source Directory
702 is derived from the Yocto Project release tarball.
703 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
704 results in a Source Directory whose top-level folder is named
705 <filename>&YOCTO_POKY;</filename>.
706 If you create a local copy of the Git repository, you can name the repository
707 anything you like.
708 Throughout much of the documentation, <filename>poky</filename> is used as the name of
709 the top-level folder of the local copy of the poky Git repository.
710 So, for example, cloning the <filename>poky</filename> Git repository results in a
711 local Git repository whose top-level folder is also named <filename>poky</filename>.</para>
712
713 <para>It is important to understand the differences between the Source Directory created
714 by unpacking a released tarball as compared to cloning
715 <filename>git://git.yoctoproject.org/poky</filename>.
716 When you unpack a tarball, you have an exact copy of the files based on the time of
717 release - a fixed release point.
718 Any changes you make to your local files in the Source Directory are on top of the release.
719 On the other hand, when you clone the <filename>poky</filename> Git repository, you have an
720 active development repository.
721 In this case, any local changes you make to the Source Directory can be later applied
722 to active development branches of the upstream <filename>poky</filename> Git
723 repository.</para>
724
725 <para>Finally, if you want to track a set of local changes while starting from the same point
726 as a release tarball, you can create a local Git branch that
727 reflects the exact copy of the files at the time of their release.
728 You do this by using Git tags that are part of the repository.</para>
729
730 <para>For more information on concepts related to Git repositories, branches, and tags,
731 see the
732 "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>"
733 section.</para></listitem>
734 <listitem><para><emphasis>Tasks:</emphasis> Arbitrary groups of software Recipes.
735 You use tasks to hold recipes that, when built, usually accomplish a single task.
736 For example, a task could contain the recipes for a company’s proprietary or value-add software.
737 Or, the task could contain the recipes that enable graphics.
738 A task is really just another recipe.
739 Because task files are recipes, they end with the <filename>.bb</filename> filename
740 extension.</para></listitem>
741 <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
742 that are not local to the development system but located in a master area that is controlled
743 by the maintainer of the source code.
744 For example, in order for a developer to work on a particular piece of code, they need to
745 first get a copy of it from an "upstream" source.</para></listitem>
746 </itemizedlist>
747 </para>
748</section>
749
750<section id='licensing'>
751 <title>Licensing</title>
752
753 <para>
754 Because open source projects are open to the public, they have different licensing structures in place.
755 License evolution for both Open Source and Free Software has an interesting history.
756 If you are interested in this history, you can find basic information here:
757 <itemizedlist>
758 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink>
759 </para></listitem>
760 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license
761 history</ulink></para></listitem>
762 </itemizedlist>
763 </para>
764
765 <para>
766 In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology
767 (MIT) License.
768 MIT licensing permits the reuse of software within proprietary software as long as the
769 license is distributed with that software.
770 MIT is also compatible with the GNU General Public License (GPL).
771 Patches to the Yocto Project follow the upstream licensing scheme.
772 You can find information on the MIT license at
773 <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
774 You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>
775 here</ulink>.
776 </para>
777
778 <para>
779 When you build an image using the Yocto Project, the build process uses a
780 known list of licenses to ensure compliance.
781 You can find this list in the
782 <link linkend='source-directory'>Source Directory</link> at
783 <filename>meta/files/common-licenses</filename>.
784 Once the build completes, the list of all licenses found and used during that build are
785 kept in the
786 <link linkend='build-directory'>Build Directory</link> at
787 <filename>tmp/deploy/licenses</filename>.
788 </para>
789
790 <para>
791 If a module requires a license that is not in the base list, the build process
792 generates a warning during the build.
793 These tools make it easier for a developer to be certain of the licenses with which
794 their shipped products must comply.
795 However, even with these tools it is still up to the developer to resolve potential licensing issues.
796 </para>
797
798 <para>
799 The base list of licenses used by the build process is a combination of the Software Package
800 Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects.
801 <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation
802 that maintains a specification
803 for a standard format for communicating the components, licenses, and copyrights
804 associated with a software package.
805 <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source
806 Definition and the effort for reviewing and approving licenses that are OSD-conformant.
807 </para>
808
809 <para>
810 You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses
811 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/files/common-licenses'>here</ulink>.
812 </para>
813
814 <para>
815 For information that can help you to maintain compliance with various open source licensing
816 during the lifecycle of a product created using the Yocto Project, see the
817 "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>" section.
818 </para>
819</section>
820
821<section id='git'>
822 <title>Git</title>
823
824 <para>
825 The Yocto Project makes extensive use of Git,
826 which is a free, open source distributed version control system.
827 Git supports distributed development, non-linear development, and can handle large projects.
828 It is best that you have some fundamental understanding of how Git tracks projects and
829 how to work with Git if you are going to use the Yocto Project for development.
830 This section provides a quick overview of how Git works and provides you with a summary
831 of some essential Git commands.
832 </para>
833
834 <para>
835 For more information on Git, see
836 <ulink url='http://git-scm.com/documentation'></ulink>.
837 If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>.
838 </para>
839
840 <section id='repositories-tags-and-branches'>
841 <title>Repositories, Tags, and Branches</title>
842
843 <para>
844 As mentioned earlier in the section
845 "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>",
846 the Yocto Project maintains source repositories at
847 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.
848 If you look at this web-interface of the repositories, each item is a separate
849 Git repository.
850 </para>
851
852 <para>
853 Git repositories use branching techniques that track content change (not files)
854 within a project (e.g. a new feature or updated documentation).
855 Creating a tree-like structure based on project divergence allows for excellent historical
856 information over the life of a project.
857 This methodology also allows for an environment from which you can do lots of
858 local experimentation on projects as you develop changes or new features.
859 </para>
860
861 <para>
862 A Git repository represents all development efforts for a given project.
863 For example, the Git repository <filename>poky</filename> contains all changes
864 and developments for Poky over the course of its entire life.
865 That means that all changes that make up all releases are captured.
866 The repository maintains a complete history of changes.
867 </para>
868
869 <para>
870 You can create a local copy of any repository by "cloning" it with the Git
871 <filename>clone</filename> command.
872 When you clone a Git repository, you end up with an identical copy of the
873 repository on your development system.
874 Once you have a local copy of a repository, you can take steps to develop locally.
875 For examples on how to clone Git repositories, see the
876 "<link linkend='getting-setup'>Getting Set Up</link>" section.
877 </para>
878
879 <para>
880 It is important to understand that Git tracks content change and not files.
881 Git uses "branches" to organize different development efforts.
882 For example, the <filename>poky</filename> repository has
883 <filename>denzil</filename>, <filename>danny</filename>,
884 <filename>dylan</filename>, <filename>dora</filename>,
885 and <filename>master</filename> branches among others.
886 You can see all the branches by going to
887 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
888 clicking on the
889 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
890 link beneath the "Branch" heading.
891 </para>
892
893 <para>
894 Each of these branches represents a specific area of development.
895 The <filename>master</filename> branch represents the current or most recent
896 development.
897 All other branches represent off-shoots of the <filename>master</filename>
898 branch.
899 </para>
900
901 <para>
902 When you create a local copy of a Git repository, the copy has the same set
903 of branches as the original.
904 This means you can use Git to create a local working area (also called a branch)
905 that tracks a specific development branch from the source Git repository.
906 in other words, you can define your local Git environment to work on any development
907 branch in the repository.
908 To help illustrate, here is a set of commands that creates a local copy of the
909 <filename>poky</filename> Git repository and then creates and checks out a local
910 Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development:
911 <literallayout class='monospaced'>
912 $ cd ~
913 $ git clone git://git.yoctoproject.org/poky
914 $ cd poky
915 $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
916 </literallayout>
917 In this example, the name of the top-level directory of your local
918 <link linkend='source-directory'>Source Directory</link>
919 is <filename>poky</filename>,
920 and the name of that local working area (local branch) you just
921 created and checked out is <filename>&DISTRO_NAME;</filename>.
922 The files in your local repository now reflect the same files that
923 are in the <filename>&DISTRO_NAME;</filename> development
924 branch of the Yocto Project's <filename>poky</filename>
925 upstream repository.
926 It is important to understand that when you create and checkout a
927 local working branch based on a branch name,
928 your local environment matches the "tip" of that development branch
929 at the time you created your local branch, which could be
930 different from the files at the time of a similarly named release.
931 In other words, creating and checking out a local branch based on the
932 <filename>&DISTRO_NAME;</filename> branch name is not the same as
933 cloning and checking out the <filename>master</filename> branch.
934 Keep reading to see how you create a local snapshot of a Yocto Project Release.
935 </para>
936
937 <para>
938 Git uses "tags" to mark specific changes in a repository.
939 Typically, a tag is used to mark a special point such as the final change
940 before a project is released.
941 You can see the tags used with the <filename>poky</filename> Git repository
942 by going to <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
943 clicking on the
944 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
945 link beneath the "Tag" heading.
946 </para>
947
948 <para>
949 Some key tags are <filename>bernard-5.0</filename>, <filename>denzil-7.0</filename>,
950 and <filename>&DISTRO_NAME;-&POKYVERSION;</filename>.
951 These tags represent Yocto Project releases.
952 </para>
953
954 <para>
955 When you create a local copy of the Git repository, you also have access to all the
956 tags.
957 Similar to branches, you can create and checkout a local working Git branch based
958 on a tag name.
959 When you do this, you get a snapshot of the Git repository that reflects
960 the state of the files when the change was made associated with that tag.
961 The most common use is to checkout a working branch that matches a specific
962 Yocto Project release.
963 Here is an example:
964 <literallayout class='monospaced'>
965 $ cd ~
966 $ git clone git://git.yoctoproject.org/poky
967 $ cd poky
968 $ git checkout -b my-&DISTRO_NAME;-&POKYVERSION; &DISTRO_NAME;-&POKYVERSION;
969 </literallayout>
970 In this example, the name of the top-level directory of your local Yocto Project
971 Files Git repository is <filename>poky</filename>.
972 And, the name of the local branch you have created and checked out is
973 <filename>my-&DISTRO_NAME;-&POKYVERSION;</filename>.
974 The files in your repository now exactly match the Yocto Project &DISTRO;
975 Release tag (<filename>&DISTRO_NAME;-&POKYVERSION;</filename>).
976 It is important to understand that when you create and checkout a local
977 working branch based on a tag, your environment matches a specific point
978 in time and not the entire development branch.
979 </para>
980 </section>
981
982 <section id='basic-commands'>
983 <title>Basic Commands</title>
984
985 <para>
986 Git has an extensive set of commands that lets you manage changes and perform
987 collaboration over the life of a project.
988 Conveniently though, you can manage with a small set of basic operations and workflows
989 once you understand the basic philosophy behind Git.
990 You do not have to be an expert in Git to be functional.
991 A good place to look for instruction on a minimal set of Git commands is
992 <ulink url='http://git-scm.com/documentation'>here</ulink>.
993 If you need to download Git, you can do so
994 <ulink url='http://git-scm.com/download'>here</ulink>.
995 </para>
996
997 <para>
998 If you don’t know much about Git, you should educate
999 yourself by visiting the links previously mentioned.
1000 </para>
1001
1002 <para>
1003 The following list briefly describes some basic Git operations as a way to get started.
1004 As with any set of commands, this list (in most cases) simply shows the base command and
1005 omits the many arguments they support.
1006 See the Git documentation for complete descriptions and strategies on how to use these commands:
1007 <itemizedlist>
1008 <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository.
1009 You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem>
1010 <listitem><para><emphasis><filename>git clone</filename>:</emphasis> Creates a clone of a repository.
1011 During collaboration, this command allows you to create a local repository that is on
1012 equal footing with a fellow developer’s repository.</para></listitem>
1013 <listitem><para><emphasis><filename>git add</filename>:</emphasis> Stages updated file contents
1014 to the index that
1015 Git uses to track changes.
1016 You must stage all files that have changed before you can commit them.</para></listitem>
1017 <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a "commit" that documents
1018 the changes you made.
1019 Commits are used for historical purposes, for determining if a maintainer of a project
1020 will allow the change, and for ultimately pushing the change from your local Git repository
1021 into the project’s upstream (or master) repository.</para></listitem>
1022 <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that
1023 possibly need to be staged and committed.</para></listitem>
1024 <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes
1025 your working branch.
1026 This command is analogous to "cd".</para></listitem>
1027 <listitem><para><emphasis><filename>git checkout –b &lt;working-branch&gt;</filename>:</emphasis> Creates
1028 a working branch on your local machine where you can isolate work.
1029 It is a good idea to use local branches when adding specific features or changes.
1030 This way if you do not like what you have done you can easily get rid of the work.</para></listitem>
1031 <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports
1032 existing local branches and
1033 tells you the branch in which you are currently working.</para></listitem>
1034 <listitem><para><emphasis><filename>git branch -D &lt;branch-name&gt;</filename>:</emphasis>
1035 Deletes an existing local branch.
1036 You need to be in a local branch other than the one you are deleting
1037 in order to delete <filename>&lt;branch-name&gt;</filename>.</para></listitem>
1038 <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information
1039 from an upstream Git
1040 repository and places it in your local Git repository.
1041 You use this command to make sure you are synchronized with the repository
1042 from which you are basing changes (.e.g. the master branch).</para></listitem>
1043 <listitem><para><emphasis><filename>git push</filename>:</emphasis>
1044 Sends all your committed local changes to an upstream Git
1045 repository (e.g. a contribution repository).
1046 The maintainer of the project draws from these repositories
1047 when adding changes to the project’s master repository or
1048 other development branch.
1049 </para></listitem>
1050 <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one
1051 local branch of your repository with another branch.
1052 When you create a local Git repository, the default branch is named "master".
1053 A typical workflow is to create a temporary branch for isolated work, make and commit your
1054 changes, switch to your local master branch, merge the changes from the temporary branch into the
1055 local master branch, and then delete the temporary branch.</para></listitem>
1056 <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific
1057 commits from one branch into another branch.
1058 There are times when you might not be able to merge all the changes in one branch with
1059 another but need to pick out certain ones.</para></listitem>
1060 <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches
1061 and changes in your local Git repository.
1062 This command is a good way to graphically see where things have diverged in your
1063 local repository.</para></listitem>
1064 <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the
1065 repository.</para></listitem>
1066 <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences
1067 between your local working files and the same files in the upstream Git repository that your
1068 branch currently tracks.</para></listitem>
1069 </itemizedlist>
1070 </para>
1071 </section>
1072</section>
1073
1074<section id='workflows'>
1075 <title>Workflows</title>
1076
1077 <para>
1078 This section provides some overview on workflows using Git.
1079 In particular, the information covers basic practices that describe roles and actions in a
1080 collaborative development environment.
1081 Again, if you are familiar with this type of development environment, you might want to just
1082 skip this section.
1083 </para>
1084
1085 <para>
1086 The Yocto Project files are maintained using Git in a "master" branch whose Git history
1087 tracks every change and whose structure provides branches for all diverging functionality.
1088 Although there is no need to use Git, many open source projects do so.
1089 For the Yocto Project, a key individual called the "maintainer" is responsible for the "master"
1090 branch of a given Git repository.
1091 The "master" branch is the “upstream” repository where the final builds of the project occur.
1092 The maintainer is responsible for allowing changes in from other developers and for
1093 organizing the underlying branch structure to reflect release strategies and so forth.
1094 <note>For information on finding out who is responsible (maintains)
1095 for a particular area of code, see the
1096 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1097 section.
1098 </note>
1099 </para>
1100
1101 <para>
1102 The project also has contribution repositories known as "contrib" areas.
1103 These areas temporarily hold changes to the project that have been submitted or committed
1104 by the Yocto Project development team and by community members that contribute to the project.
1105 The maintainer determines if the changes are qualified to be moved from the "contrib" areas
1106 into the "master" branch of the Git repository.
1107 </para>
1108
1109 <para>
1110 Developers (including contributing community members) create and maintain cloned repositories
1111 of the upstream "master" branch.
1112 These repositories are local to their development platforms and are used to develop changes.
1113 When a developer is satisfied with a particular feature or change, they "push" the changes
1114 to the appropriate "contrib" repository.
1115 </para>
1116
1117 <para>
1118 Developers are responsible for keeping their local repository up-to-date with "master".
1119 They are also responsible for straightening out any conflicts that might arise within files
1120 that are being worked on simultaneously by more than one person.
1121 All this work is done locally on the developer’s machines before anything is pushed to a
1122 "contrib" area and examined at the maintainer’s level.
1123 </para>
1124
1125 <para>
1126 A somewhat formal method exists by which developers commit changes and push them into the
1127 "contrib" area and subsequently request that the maintainer include them into "master"
1128 This process is called “submitting a patch” or "submitting a change."
1129 For information on submitting patches and changes, see the
1130 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section.
1131 </para>
1132
1133 <para>
1134 To summarize the environment: we have a single point of entry for changes into the project’s
1135 "master" branch of the Git repository, which is controlled by the project’s maintainer.
1136 And, we have a set of developers who independently develop, test, and submit changes
1137 to "contrib" areas for the maintainer to examine.
1138 The maintainer then chooses which changes are going to become a permanent part of the project.
1139 </para>
1140
1141 <para>
1142 <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
1143 </para>
1144
1145 <para>
1146 While each development environment is unique, there are some best practices or methods
1147 that help development run smoothly.
1148 The following list describes some of these practices.
1149 For more information about Git workflows, see the workflow topics in the
1150 <ulink url='http://book.git-scm.com'>Git Community Book</ulink>.
1151 <itemizedlist>
1152 <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit
1153 small as compared to bundling many disparate changes into a single commit.
1154 This practice not only keeps things manageable but also allows the maintainer
1155 to more easily include or refuse changes.</para>
1156 <para>It is also good practice to leave the repository in a state that allows you to
1157 still successfully build your project. In other words, do not commit half of a feature,
1158 then add the other half as a separate, later commit.
1159 Each commit should take you from one buildable project state to another
1160 buildable state.</para></listitem>
1161 <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and
1162 delete local branches in your working Git repository.
1163 You can name these branches anything you like.
1164 It is helpful to give them names associated with the particular feature or change
1165 on which you are working.
1166 Once you are done with a feature or change and have merged it
1167 into your local master branch, simply discard the temporary
1168 branch.</para></listitem>
1169 <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename>
1170 command allows you to take the
1171 changes from one branch and fold them into another branch.
1172 This process is especially helpful when more than a single developer might be working
1173 on different parts of the same feature.
1174 Merging changes also automatically identifies any collisions or "conflicts"
1175 that might happen as a result of the same lines of code being altered by two different
1176 developers.</para></listitem>
1177 <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should
1178 use a system where branches indicate varying levels of code readiness.
1179 For example, you can have a "work" branch to develop in, a "test" branch where the code or
1180 change is tested, a "stage" branch where changes are ready to be committed, and so forth.
1181 As your project develops, you can merge code across the branches to reflect ever-increasing
1182 stable states of the development.</para></listitem>
1183 <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the
1184 concept of developers "pushing" local commits to a remote repository, which is
1185 usually a contribution repository.
1186 This workflow is also based on developers "pulling" known states of the project down into their
1187 local development repositories.
1188 The workflow easily allows you to pull changes submitted by other developers from the
1189 upstream repository into your work area ensuring that you have the most recent software
1190 on which to develop.
1191 The Yocto Project has two scripts named <filename>create-pull-request</filename> and
1192 <filename>send-pull-request</filename> that ship with the release to facilitate this
1193 workflow.
1194 You can find these scripts in the <filename>scripts</filename>
1195 folder of the
1196 <link linkend='source-directory'>Source Directory</link>.
1197 For information on how to use these scripts, see the
1198 "<link linkend='pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</link>" section.
1199 </para></listitem>
1200 <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the
1201 maintainer through an email that you have a change (or patch) you would like considered
1202 for the "master" branch of the Git repository.
1203 To send this type of change, you format the patch and then send the email using the Git commands
1204 <filename>git format-patch</filename> and <filename>git send-email</filename>.
1205 For information on how to use these scripts, see the
1206 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1207 section.
1208 </para></listitem>
1209 </itemizedlist>
1210 </para>
1211</section>
1212
1213<section id='tracking-bugs'>
1214 <title>Tracking Bugs</title>
1215
1216 <para>
1217 The Yocto Project uses its own implementation of
1218 <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs.
1219 Implementations of Bugzilla work well for group development because they track bugs and code
1220 changes, can be used to communicate changes and problems with developers, can be used to
1221 submit and review patches, and can be used to manage quality assurance.
1222 The home page for the Yocto Project implementation of Bugzilla is
1223 <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>.
1224 </para>
1225
1226 <para>
1227 Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself
1228 such as when discovering an issue with some component of the build system that acts contrary
1229 to the documentation or your expectations.
1230 Following is the general procedure for submitting a new bug using the Yocto Project
1231 Bugzilla.
1232 You can find more information on defect management, bug tracking, and feature request
1233 processes all accomplished through the Yocto Project Bugzilla on the wiki page
1234 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>here</ulink>.
1235 <orderedlist>
1236 <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit
1237 a bug.</para></listitem>
1238 <listitem><para>When submitting a new bug, be sure to choose the appropriate
1239 Classification, Product, and Component for which the issue was found.
1240 Defects for the Yocto Project fall into one of six classifications: Yocto Project
1241 Components, Infrastructure, Build System &amp; Metadata, Documentation,
1242 QA/Testing, and Runtime.
1243 Each of these Classifications break down into multiple Products and, in some
1244 cases, multiple Components.</para></listitem>
1245 <listitem><para>Use the bug form to choose the correct Hardware and Architecture
1246 for which the bug applies.</para></listitem>
1247 <listitem><para>Indicate the Yocto Project version you were using when the issue
1248 occurred.</para></listitem>
1249 <listitem><para>Be sure to indicate the Severity of the bug.
1250 Severity communicates how the bug impacted your work.</para></listitem>
1251 <listitem><para>Select the appropriate "Documentation change" item
1252 for the bug.
1253 Fixing a bug may or may not affect the Yocto Project
1254 documentation.</para></listitem>
1255 <listitem><para>Provide a brief summary of the issue.
1256 Try to limit your summary to just a line or two and be sure to capture the
1257 essence of the issue.</para></listitem>
1258 <listitem><para>Provide a detailed description of the issue.
1259 You should provide as much detail as you can about the context, behavior, output,
1260 and so forth that surrounds the issue.
1261 You can even attach supporting files for output from logs by
1262 using the "Add an attachment" button.</para></listitem>
1263 <listitem><para>Be sure to copy the appropriate people in the
1264 "CC List" for the bug.
1265 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1266 section for information about finding out who is responsible
1267 for code.</para></listitem>
1268 <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem>
1269 </orderedlist>
1270 </para>
1271</section>
1272
1273<section id='how-to-submit-a-change'>
1274 <title>How to Submit a Change</title>
1275
1276 <para>
1277 Contributions to the Yocto Project and OpenEmbedded are very welcome.
1278 Because the system is extremely configurable and flexible, we recognize that developers
1279 will want to extend, configure or optimize it for their specific uses.
1280 You should send patches to the appropriate mailing list so that they
1281 can be reviewed and merged by the appropriate maintainer.
1282 </para>
1283
1284 <para>
1285 Before submitting any change, be sure to find out who you should be
1286 notifying.
1287 Several methods exist through which you find out who you should be copying
1288 or notifying:
1289 <itemizedlist>
1290 <listitem><para><emphasis>Maintenance File:</emphasis>
1291 Examine the <filename>maintainers.inc</filename> file, which is
1292 located in the
1293 <link linkend='source-directory'>Source Directory</link>
1294 at <filename>meta-yocto/conf/distro/include</filename>, to
1295 see who is responsible for code.
1296 </para></listitem>
1297 <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis>
1298 For BSP maintainers of supported BSPs, you can examine
1299 individual BSP <filename>README</filename> files.
1300 In addition, some layers (such as the <filename>meta-intel</filename> layer),
1301 include a <filename>MAINTAINERS</filename> file which contains
1302 a list of all supported BSP maintainers for that layer.
1303 </para></listitem>
1304 <listitem><para><emphasis>Search by File:</emphasis>
1305 Using <link linkend='git'>Git</link>, you can enter the
1306 following command to bring up a short list of all commits
1307 against a specific file:
1308 <literallayout class='monospaced'>
1309 git shortlog -- &lt;filename&gt;
1310 </literallayout>
1311 Just provide the name of the file for which you are interested.
1312 The information returned is not ordered by history but does
1313 include a list of all committers grouped by name.
1314 From the list, you can see who is responsible for the bulk of
1315 the changes against the file.
1316 </para></listitem>
1317 </itemizedlist>
1318 </para>
1319
1320 <para>
1321 For a list of the Yocto Project and related mailing lists, see the
1322 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in
1323 the Yocto Project Reference Manual.
1324 </para>
1325
1326 <para>
1327 Here is some guidance on which mailing list to use for what type of change:
1328 <itemizedlist>
1329 <listitem><para>For changes to the core
1330 <link linkend='metadata'>Metadata</link>, send your patch to the
1331 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list.
1332 For example, a change to anything under the <filename>meta</filename> or
1333 <filename>scripts</filename> directories
1334 should be sent to this mailing list.</para></listitem>
1335 <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename>
1336 directory), send your patch to the
1337 <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem>
1338 <listitem><para>For changes to <filename>meta-yocto</filename>, send your patch to the
1339 <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem>
1340 <listitem><para>For changes to other layers hosted on
1341 <filename>yoctoproject.org</filename> (unless the
1342 layer's documentation specifies otherwise), tools, and Yocto Project
1343 documentation, use the
1344 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem>
1345 <listitem><para>For additional recipes that do not fit into the core Metadata,
1346 you should determine which layer the recipe should go into and submit the
1347 change in the manner recommended by the documentation (e.g. README) supplied
1348 with the layer. If in doubt, please ask on the
1349 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or
1350 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink>
1351 mailing lists.</para></listitem>
1352 </itemizedlist>
1353 </para>
1354
1355 <para>
1356 When you send a patch, be sure to include a "Signed-off-by:"
1357 line in the same style as required by the Linux kernel.
1358 Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1
1359 as follows:
1360 <literallayout class='monospaced'>
1361 Developer's Certificate of Origin 1.1
1362
1363 By making a contribution to this project, I certify that:
1364
1365 (a) The contribution was created in whole or in part by me and I
1366 have the right to submit it under the open source license
1367 indicated in the file; or
1368
1369 (b) The contribution is based upon previous work that, to the best
1370 of my knowledge, is covered under an appropriate open source
1371 license and I have the right under that license to submit that
1372 work with modifications, whether created in whole or in part
1373 by me, under the same open source license (unless I am
1374 permitted to submit under a different license), as indicated
1375 in the file; or
1376
1377 (c) The contribution was provided directly to me by some other
1378 person who certified (a), (b) or (c) and I have not modified
1379 it.
1380
1381 (d) I understand and agree that this project and the contribution
1382 are public and that a record of the contribution (including all
1383 personal information I submit with it, including my sign-off) is
1384 maintained indefinitely and may be redistributed consistent with
1385 this project or the open source license(s) involved.
1386 </literallayout>
1387 </para>
1388
1389 <para>
1390 In a collaborative environment, it is necessary to have some sort of standard
1391 or method through which you submit changes.
1392 Otherwise, things could get quite chaotic.
1393 One general practice to follow is to make small, controlled changes.
1394 Keeping changes small and isolated aids review, makes merging/rebasing easier
1395 and keeps the change history clean when anyone needs to refer to it in future.
1396 </para>
1397
1398 <para>
1399 When you make a commit, you must follow certain standards established by the
1400 OpenEmbedded and Yocto Project development teams.
1401 For each commit, you must provide a single-line summary of the change and you
1402 should almost always provide a more detailed description of what you did (i.e.
1403 the body of the commit message).
1404 The only exceptions for not providing a detailed description would be if your
1405 change is a simple, self-explanatory change that needs no further description
1406 beyond the summary.
1407 Here are the guidelines for composing a commit message:
1408 <itemizedlist>
1409 <listitem><para>Provide a single-line, short summary of the change.
1410 This summary is typically viewable in the "shortlist" of changes.
1411 Thus, providing something short and descriptive that gives the reader
1412 a summary of the change is useful when viewing a list of many commits.
1413 This short description should be prefixed by the recipe name (if changing a recipe), or
1414 else the short form path to the file being changed.
1415 </para></listitem>
1416 <listitem><para>For the body of the commit message, provide detailed information
1417 that describes what you changed, why you made the change, and the approach
1418 you used. It may also be helpful if you mention how you tested the change.
1419 Provide as much detail as you can in the body of the commit message.
1420 </para></listitem>
1421 <listitem><para>If the change addresses a specific bug or issue that is
1422 associated with a bug-tracking ID, include a reference to that ID in
1423 your detailed description.
1424 For example, the Yocto Project uses a specific convention for bug
1425 references - any commit that addresses a specific bug should include the
1426 bug ID in the description (typically at the beginning) as follows:
1427 <literallayout class='monospaced'>
1428 [YOCTO #&lt;bug-id&gt;]
1429
1430 &lt;detailed description of change&gt;
1431 </literallayout></para></listitem>
1432 Where &lt;bug-id&gt; is replaced with the specific bug ID from the
1433 Yocto Project Bugzilla instance.
1434 </itemizedlist>
1435 </para>
1436
1437 <para>
1438 You can find more guidance on creating well-formed commit messages at this OpenEmbedded
1439 wiki page:
1440 <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>.
1441 </para>
1442
1443 <para>
1444 The next two sections describe general instructions for both pushing
1445 changes upstream and for submitting changes as patches.
1446 </para>
1447
1448 <section id='pushing-a-change-upstream'>
1449 <title>Using Scripts to Push a Change Upstream and Request a Pull</title>
1450
1451 <para>
1452 The basic flow for pushing a change to an upstream "contrib" Git repository is as follows:
1453 <itemizedlist>
1454 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1455 <listitem><para>Stage your changes by using the <filename>git add</filename>
1456 command on each file you changed.</para></listitem>
1457 <listitem><para>Commit the change by using the <filename>git commit</filename>
1458 command and push it to the "contrib" repository.
1459 Be sure to provide a commit message that follows the project’s commit message standards
1460 as described earlier.</para></listitem>
1461 <listitem><para>Notify the maintainer that you have pushed a change by making a pull
1462 request.
1463 The Yocto Project provides two scripts that conveniently let you generate and send
1464 pull requests to the Yocto Project.
1465 These scripts are <filename>create-pull-request</filename> and
1466 <filename>send-pull-request</filename>.
1467 You can find these scripts in the <filename>scripts</filename> directory
1468 within the <link linkend='source-directory'>Source Directory</link>.</para>
1469 <para>Using these scripts correctly formats the requests without introducing any
1470 whitespace or HTML formatting.
1471 The maintainer that receives your patches needs to be able to save and apply them
1472 directly from your emails.
1473 Using these scripts is the preferred method for sending patches.</para>
1474 <para>For help on using these scripts, simply provide the
1475 <filename>-h</filename> argument as follows:
1476 <literallayout class='monospaced'>
1477 $ poky/scripts/create-pull-request -h
1478 $ poky/scripts/send-pull-request -h
1479 </literallayout></para></listitem>
1480 </itemizedlist>
1481 </para>
1482
1483 <para>
1484 You can find general Git information on how to push a change upstream in the
1485 <ulink url='http://book.git-scm.com/3_distributed_workflows.html'>Git Community Book</ulink>.
1486 </para>
1487 </section>
1488
1489 <section id='submitting-a-patch'>
1490 <title>Using Email to Submit a Patch</title>
1491
1492 <para>
1493 You can submit patches without using the <filename>create-pull-request</filename> and
1494 <filename>send-pull-request</filename> scripts described in the previous section.
1495 However, keep in mind, the preferred method is to use the scripts.
1496 </para>
1497
1498 <para>
1499 Depending on the components changed, you need to submit the email to a specific
1500 mailing list.
1501 For some guidance on which mailing list to use, see the list in the
1502 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1503 section.
1504 For a description of the available mailing lists, see the
1505 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
1506 section in the Yocto Project Reference Manual.
1507 </para>
1508
1509 <para>
1510 Here is the general procedure on how to submit a patch through email without using the
1511 scripts:
1512 <itemizedlist>
1513 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1514 <listitem><para>Stage your changes by using the <filename>git add</filename>
1515 command on each file you changed.</para></listitem>
1516 <listitem><para>Commit the change by using the
1517 <filename>git commit --signoff</filename> command.
1518 Using the <filename>--signoff</filename> option identifies you as the person
1519 making the change and also satisfies the Developer's Certificate of
1520 Origin (DCO) shown earlier.</para>
1521 <para>When you form a commit, you must follow certain standards established by the
1522 Yocto Project development team.
1523 See the earlier section
1524 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1525 for Yocto Project commit message standards.</para></listitem>
1526 <listitem><para>Format the commit into an email message.
1527 To format commits, use the <filename>git format-patch</filename> command.
1528 When you provide the command, you must include a revision list or a number of patches
1529 as part of the command.
1530 For example, either of these two commands takes your most
1531 recent single commit and formats it as an email message in
1532 the current directory:
1533 <literallayout class='monospaced'>
1534 $ git format-patch -1
1535 </literallayout>
1536 or
1537 <literallayout class='monospaced'>
1538 $ git format-patch HEAD~
1539 </literallayout></para>
1540 <para>After the command is run, the current directory contains a
1541 numbered <filename>.patch</filename> file for the commit.</para>
1542 <para>If you provide several commits as part of the command,
1543 the <filename>git format-patch</filename> command produces a
1544 series of numbered files in the current directory – one for each commit.
1545 If you have more than one patch, you should also use the
1546 <filename>--cover</filename> option with the command, which generates a
1547 cover letter as the first "patch" in the series.
1548 You can then edit the cover letter to provide a description for
1549 the series of patches.
1550 For information on the <filename>git format-patch</filename> command,
1551 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the
1552 <filename>man git-format-patch</filename> command.</para>
1553 <note>If you are or will be a frequent contributor to the Yocto Project
1554 or to OpenEmbedded, you might consider requesting a contrib area and the
1555 necessary associated rights.</note></listitem>
1556 <listitem><para>Import the files into your mail client by using the
1557 <filename>git send-email</filename> command.
1558 <note>In order to use <filename>git send-email</filename>, you must have the
1559 the proper Git packages installed.
1560 For Ubuntu, Debian, and Fedora the package is <filename>git-email</filename>.</note></para>
1561 <para>The <filename>git send-email</filename> command sends email by using a local
1562 or remote Mail Transport Agent (MTA) such as
1563 <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct
1564 <filename>smtp</filename> configuration in your Git <filename>config</filename>
1565 file.
1566 If you are submitting patches through email only, it is very important
1567 that you submit them without any whitespace or HTML formatting that
1568 either you or your mailer introduces.
1569 The maintainer that receives your patches needs to be able to save and
1570 apply them directly from your emails.
1571 A good way to verify that what you are sending will be applicable by the
1572 maintainer is to do a dry run and send them to yourself and then
1573 save and apply them as the maintainer would.</para>
1574 <para>The <filename>git send-email</filename> command is the preferred method
1575 for sending your patches since there is no risk of compromising whitespace
1576 in the body of the message, which can occur when you use your own mail client.
1577 The command also has several options that let you
1578 specify recipients and perform further editing of the email message.
1579 For information on how to use the <filename>git send-email</filename> command,
1580 see <filename>GIT-SEND-EMAIL(1)</filename> displayed using
1581 the <filename>man git-send-email</filename> command.
1582 </para></listitem>
1583 </itemizedlist>
1584 </para>
1585 </section>
1586</section>
1587</chapter>
1588<!--
1589vim: expandtab tw=80 ts=4
1590-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-13 12:02:56 +0000