summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/technical-details.xml')
-rw-r--r--documentation/ref-manual/technical-details.xml1358
1 files changed, 1358 insertions, 0 deletions
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
new file mode 100644
index 0000000..6d4c6b7
--- /dev/null
+++ b/documentation/ref-manual/technical-details.xml
@@ -0,0 +1,1358 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='technical-details'>
6<title>Technical Details</title>
7
8 <para>
9 This chapter provides technical details for various parts of the Yocto Project.
10 Currently, topics include Yocto Project components,
11 shared state (sstate) cache, x32, and Licenses.
12 </para>
13
14<section id='usingpoky-components'>
15 <title>Yocto Project Components</title>
16
17 <para>
18 The BitBake task executor together with various types of configuration files form the
19 OpenEmbedded Core.
20 This section overviews these by describing what they are used for
21 and how they interact.
22 </para>
23
24 <para>
25 BitBake handles the parsing and execution of the data files.
26 The data itself is of various types:
27 <itemizedlist>
28 <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular
29 pieces of software.</para></listitem>
30 <listitem><para><emphasis>Class Data:</emphasis> Abstracts common build
31 information (e.g. how to build a Linux kernel).</para></listitem>
32 <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings,
33 policy decisions, and so forth.
34 Configuration data acts as the glue to bind everything together.</para></listitem>
35 </itemizedlist>
36 For more information on data, see the
37 "<ulink url='&YOCTO_DOCS_DEV_URL;#yocto-project-terms'>Yocto Project Terms</ulink>"
38 section in the Yocto Project Development Manual.
39 </para>
40
41 <para>
42 BitBake knows how to combine multiple data sources together and refers to each data source
43 as a layer.
44 For information on layers, see the
45 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
46 Creating Layers</ulink>" section of the Yocto Project Development Manual.
47 </para>
48
49 <para>
50 Following are some brief details on these core components.
51 For more detailed information on these components, see the
52 "<link linkend='ref-structure'>Source Directory Structure</link>" chapter.
53 </para>
54
55 <section id='usingpoky-components-bitbake'>
56 <title>BitBake</title>
57
58 <para>
59 BitBake is the tool at the heart of the OpenEmbedded build system
60 and is responsible for parsing the
61 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
62 generating a list of tasks from it, and then executing those tasks.
63 To see a list of the options BitBake supports, use either of
64 the following commands:
65 <literallayout class='monospaced'>
66 $ bitbake -h
67 $ bitbake --help
68 </literallayout>
69 </para>
70
71 <para>
72 The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
73 <filename>packagename</filename> is the name of the package you want to build
74 (referred to as the "target" in this manual).
75 The target often equates to the first part of a <filename>.bb</filename> filename.
76 So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
77 might type the following:
78 <literallayout class='monospaced'>
79 $ bitbake matchbox-desktop
80 </literallayout>
81 Several different versions of <filename>matchbox-desktop</filename> might exist.
82 BitBake chooses the one selected by the distribution configuration.
83 You can get more details about how BitBake chooses between different
84 target versions and providers in the
85 "<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section.
86 </para>
87
88 <para>
89 BitBake also tries to execute any dependent tasks first.
90 So for example, before building <filename>matchbox-desktop</filename>, BitBake
91 would build a cross compiler and <filename>eglibc</filename> if they had not already
92 been built.
93 <note>This release of the Yocto Project does not support the <filename>glibc</filename>
94 GNU version of the Unix standard C library. By default, the OpenEmbedded build system
95 builds with <filename>eglibc</filename>.</note>
96 </para>
97
98 <para>
99 A useful BitBake option to consider is the <filename>-k</filename> or
100 <filename>--continue</filename> option.
101 This option instructs BitBake to try and continue processing the job as much
102 as possible even after encountering an error.
103 When an error occurs, the target that
104 failed and those that depend on it cannot be remade.
105 However, when you use this option other dependencies can still be processed.
106 </para>
107 </section>
108
109 <section id='usingpoky-components-metadata'>
110 <title>Metadata (Recipes)</title>
111
112 <para>
113 The <filename>.bb</filename> files are usually referred to as "recipes."
114 In general, a recipe contains information about a single piece of software.
115 This information includes the location from which to download the
116 unaltered source, any source patches to be applied to that source
117 (if needed), which special configuration options to apply,
118 how to compile the source files, and how to package the compiled output.
119 </para>
120
121 <para>
122 The term "package" is sometimes used to refer to recipes. However,
123 since the word "package" is used for the packaged output from the OpenEmbedded
124 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
125 this document avoids using the term "package" when referring to recipes.
126 </para>
127 </section>
128
129 <section id='usingpoky-components-classes'>
130 <title>Classes</title>
131
132 <para>
133 Class files (<filename>.bbclass</filename>) contain information that
134 is useful to share between
135 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
136 An example is the Autotools class, which contains
137 common settings for any application that Autotools uses.
138 The "<link linkend='ref-classes'>Classes</link>" chapter provides details
139 about common classes and how to use them.
140 </para>
141 </section>
142
143 <section id='usingpoky-components-configuration'>
144 <title>Configuration</title>
145
146 <para>
147 The configuration files (<filename>.conf</filename>) define various configuration variables
148 that govern the OpenEmbedded build process.
149 These files fall into several areas that define machine configuration options,
150 distribution configuration options, compiler tuning options, general common configuration
151 options, and user configuration options in <filename>local.conf</filename>, which is found
152 in the
153 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
154 </para>
155 </section>
156</section>
157
158<section id="cross-development-toolchain-generation">
159 <title>Cross-Development Toolchain Generation</title>
160
161 <para>
162 The Yocto Project does most of the work for you when it comes to
163 creating
164 <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
165 This section provides some technical background information on how
166 cross-development toolchains are created and used.
167 For more information on toolchains, you can also see the
168 <ulink url='&YOCTO_DOCS_ADT_URL;'>the Yocto Project Application Developer's Guide</ulink>.
169 </para>
170
171 <para>
172 In the Yocto Project development environment, cross-development
173 toolchains are used to build the image and applications that run on the
174 target hardware.
175 With just a few commands, the OpenEmbedded build system creates
176 these necessary toolchains for you.
177 </para>
178
179 <para>
180 The following figure shows a high-level build environment regarding
181 toolchain construction and use.
182 </para>
183
184 <para>
185 <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
186 </para>
187
188 <para>
189 Most of the work occurs on the Build Host.
190 This is the machine used to build images and generally work within the
191 the Yocto Project environment.
192 When you run BitBake to create an image, the OpenEmbedded build system
193 uses the host <filename>gcc</filename> compiler to bootstrap a
194 cross-compiler named <filename>gcc-cross</filename>.
195 The <filename>gcc-cross</filename> compiler is what BitBake uses to
196 compile source files when creating the target image.
197 You can think of <filename>gcc-cross</filename> simply as an
198 automatically generated cross-compiler that is used internally within
199 BitBake only.
200 </para>
201
202 <para>
203 The chain of events that occurs when <filename>gcc-cross</filename> is
204 bootstrapped is as follows:
205 <literallayout class='monospaced'>
206 gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> eglibc-initial -> eglibc -> gcc-cross -> gcc-runtime
207 </literallayout>
208 <itemizedlist>
209 <listitem><para><filename>gcc</filename>:
210 The build host's GNU Compiler Collection (GCC).
211 </para></listitem>
212 <listitem><para><filename>binutils-cross</filename>:
213 The bare minimum binary utilities needed in order to run
214 the <filename>gcc-cross-initial</filename> phase of the
215 bootstrap operation.
216 </para></listitem>
217 <listitem><para><filename>gcc-cross-initial</filename>:
218 An early stage of the bootstrap process for creating
219 the cross-compiler.
220 This stage builds enough of the <filename>gcc-cross</filename>,
221 the C library, and other pieces needed to finish building the
222 final cross-compiler in later stages.
223 This tool is a "native" package (i.e. it is designed to run on
224 the build host).
225 </para></listitem>
226 <listitem><para><filename>linux-libc-headers</filename>:
227 Headers needed for the cross-compiler.
228 </para></listitem>
229 <listitem><para><filename>eglibc-initial</filename>:
230 An initial version of the Embedded GLIBC needed to bootstrap
231 <filename>eglibc</filename>.
232 </para></listitem>
233 <listitem><para><filename>gcc-cross</filename>:
234 The final stage of the bootstrap process for the
235 cross-compiler.
236 This stage results in the actual cross-compiler that
237 BitBake uses when it builds an image for a targeted
238 device.
239 <note>
240 If you are replacing this cross compiler toolchain
241 with a custom version, you must replace
242 <filename>gcc-cross</filename>.
243 </note>
244 This tool is also a "native" package (i.e. it is
245 designed to run on the build host).
246 </para></listitem>
247 <listitem><para><filename>gcc-runtime</filename>:
248 Runtime libraries resulting from the toolchain bootstrapping
249 process.
250 This tool produces a binary that consists of the
251 runtime libraries need for the targeted device.
252 </para></listitem>
253 </itemizedlist>
254 </para>
255
256 <para>
257 You can use the OpenEmbedded build system to build an installer for
258 the relocatable SDK used to develop applications.
259 When you run the installer, it installs the toolchain, which contains
260 the development tools (e.g., the
261 <filename>gcc-cross-canadian</filename>),
262 <filename>binutils-cross-canadian</filename>, and other
263 <filename>nativesdk-*</filename> tools you need to cross-compile and
264 test your software.
265 The figure shows the commands you use to easily build out this
266 toolchain.
267 This cross-development toolchain is built to execute on the
268 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
269 which might or might not be the same
270 machine as the Build Host.
271 <note>
272 If your target architecture is supported by the Yocto Project,
273 you can take advantage of pre-built images that ship with the
274 Yocto Project and already contain cross-development toolchain
275 installers.
276 </note>
277 </para>
278
279 <para>
280 Here is the bootstrap process for the relocatable toolchain:
281 <literallayout class='monospaced'>
282 gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers -> eglibc-initial -> nativesdk-eglibc -> gcc-crosssdk -> gcc-cross-canadian
283 </literallayout>
284 <itemizedlist>
285 <listitem><para><filename>gcc</filename>:
286 The build host's GNU Compiler Collection (GCC).
287 </para></listitem>
288 <listitem><para><filename>binutils-crosssdk</filename>:
289 The bare minimum binary utilities needed in order to run
290 the <filename>gcc-crosssdk-initial</filename> phase of the
291 bootstrap operation.
292 </para></listitem>
293 <listitem><para><filename>gcc-crosssdk-initial</filename>:
294 An early stage of the bootstrap process for creating
295 the cross-compiler.
296 This stage builds enough of the
297 <filename>gcc-crosssdk</filename> and supporting pieces so that
298 the final stage of the bootstrap process can produce the
299 finished cross-compiler.
300 This tool is a "native" binary that runs on the build host.
301 </para></listitem>
302 <listitem><para><filename>linux-libc-headers</filename>:
303 Headers needed for the cross-compiler.
304 </para></listitem>
305 <listitem><para><filename>eglibc-initial</filename>:
306 An initial version of the Embedded GLIBC needed to bootstrap
307 <filename>nativesdk-eglibc</filename>.
308 </para></listitem>
309 <listitem><para><filename>nativesdk-eglibc</filename>:
310 The Embedded GLIBC needed to bootstrap the
311 <filename>gcc-crosssdk</filename>.
312 </para></listitem>
313 <listitem><para><filename>gcc-crosssdk</filename>:
314 The final stage of the bootstrap process for the
315 relocatable cross-compiler.
316 The <filename>gcc-crosssdk</filename> is a transitory compiler
317 and never leaves the build host.
318 Its purpose is to help in the bootstrap process to create the
319 eventual relocatable <filename>gcc-cross-canadian</filename>
320 compiler, which is relocatable.
321 This tool is also a "native" package (i.e. it is
322 designed to run on the build host).
323 </para></listitem>
324 <listitem><para><filename>gcc-cross-canadian</filename>:
325 The final relocatable cross-compiler.
326 When run on the
327 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
328 this tool
329 produces executable code that runs on the target device.
330 </para></listitem>
331 </itemizedlist>
332 </para>
333</section>
334
335<section id="shared-state-cache">
336 <title>Shared State Cache</title>
337
338 <para>
339 By design, the OpenEmbedded build system builds everything from scratch unless
340 BitBake can determine that parts do not need to be rebuilt.
341 Fundamentally, building from scratch is attractive as it means all parts are
342 built fresh and there is no possibility of stale data causing problems.
343 When developers hit problems, they typically default back to building from scratch
344 so they know the state of things from the start.
345 </para>
346
347 <para>
348 Building an image from scratch is both an advantage and a disadvantage to the process.
349 As mentioned in the previous paragraph, building from scratch ensures that
350 everything is current and starts from a known state.
351 However, building from scratch also takes much longer as it generally means
352 rebuilding things that do not necessarily need to be rebuilt.
353 </para>
354
355 <para>
356 The Yocto Project implements shared state code that supports incremental builds.
357 The implementation of the shared state code answers the following questions that
358 were fundamental roadblocks within the OpenEmbedded incremental build support system:
359 <itemizedlist>
360 <listitem><para>What pieces of the system have changed and what pieces have
361 not changed?</para></listitem>
362 <listitem><para>How are changed pieces of software removed and replaced?</para></listitem>
363 <listitem><para>How are pre-built components that do not need to be rebuilt from scratch
364 used when they are available?</para></listitem>
365 </itemizedlist>
366 </para>
367
368 <para>
369 For the first question, the build system detects changes in the "inputs" to a given task by
370 creating a checksum (or signature) of the task's inputs.
371 If the checksum changes, the system assumes the inputs have changed and the task needs to be
372 rerun.
373 For the second question, the shared state (sstate) code tracks which tasks add which output
374 to the build process.
375 This means the output from a given task can be removed, upgraded or otherwise manipulated.
376 The third question is partly addressed by the solution for the second question
377 assuming the build system can fetch the sstate objects from remote locations and
378 install them if they are deemed to be valid.
379 </para>
380
381 <note>
382 The OpenEmbedded build system does not maintain
383 <link linkend='var-PR'><filename>PR</filename></link> information
384 as part of the shared state packages.
385 Consequently, considerations exist that affect maintaining shared
386 state feeds.
387 For information on how the OpenEmbedded works with packages and can
388 track incrementing <filename>PR</filename> information, see the
389 "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
390 section.
391 </note>
392
393 <para>
394 The rest of this section goes into detail about the overall incremental build
395 architecture, the checksums (signatures), shared state, and some tips and tricks.
396 </para>
397
398 <section id='overall-architecture'>
399 <title>Overall Architecture</title>
400
401 <para>
402 When determining what parts of the system need to be built, BitBake
403 works on a per-task basis rather than a per-recipe basis.
404 You might wonder why using a per-task basis is preferred over a per-recipe basis.
405 To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
406 In this case, <filename>do_install</filename> and <filename>do_package</filename>
407 outputs are still valid.
408 However, with a per-recipe approach, the build would not include the
409 <filename>.deb</filename> files.
410 Consequently, you would have to invalidate the whole build and rerun it.
411 Rerunning everything is not the best solution.
412 Also, in this case, the core must be "taught" much about specific tasks.
413 This methodology does not scale well and does not allow users to easily add new tasks
414 in layers or as external recipes without touching the packaged-staging core.
415 </para>
416 </section>
417
418 <section id='checksums'>
419 <title>Checksums (Signatures)</title>
420
421 <para>
422 The shared state code uses a checksum, which is a unique signature of a task's
423 inputs, to determine if a task needs to be run again.
424 Because it is a change in a task's inputs that triggers a rerun, the process
425 needs to detect all the inputs to a given task.
426 For shell tasks, this turns out to be fairly easy because
427 the build process generates a "run" shell script for each task and
428 it is possible to create a checksum that gives you a good idea of when
429 the task's data changes.
430 </para>
431
432 <para>
433 To complicate the problem, there are things that should not be included in
434 the checksum.
435 First, there is the actual specific build path of a given task -
436 the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
437 It does not matter if the working directory changes because it should not
438 affect the output for target packages.
439 Also, the build process has the objective of making native or cross packages relocatable.
440 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
441 The simplistic approach for excluding the working directory is to set
442 <filename>WORKDIR</filename> to some fixed value and create the checksum
443 for the "run" script.
444 </para>
445
446 <para>
447 Another problem results from the "run" scripts containing functions that
448 might or might not get called.
449 The incremental build solution contains code that figures out dependencies
450 between shell functions.
451 This code is used to prune the "run" scripts down to the minimum set,
452 thereby alleviating this problem and making the "run" scripts much more
453 readable as a bonus.
454 </para>
455
456 <para>
457 So far we have solutions for shell scripts.
458 What about Python tasks?
459 The same approach applies even though these tasks are more difficult.
460 The process needs to figure out what variables a Python function accesses
461 and what functions it calls.
462 Again, the incremental build solution contains code that first figures out
463 the variable and function dependencies, and then creates a checksum for the data
464 used as the input to the task.
465 </para>
466
467 <para>
468 Like the <filename>WORKDIR</filename> case, situations exist where dependencies
469 should be ignored.
470 For these cases, you can instruct the build process to ignore a dependency
471 by using a line like the following:
472 <literallayout class='monospaced'>
473 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
474 </literallayout>
475 This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not
476 depend on the value of <filename>MACHINE</filename>, even if it does reference it.
477 </para>
478
479 <para>
480 Equally, there are cases where we need to add dependencies BitBake is not able to find.
481 You can accomplish this by using a line like the following:
482 <literallayout class='monospaced'>
483 PACKAGE_ARCHS[vardeps] = "MACHINE"
484 </literallayout>
485 This example explicitly adds the <filename>MACHINE</filename> variable as a
486 dependency for <filename>PACKAGE_ARCHS</filename>.
487 </para>
488
489 <para>
490 Consider a case with in-line Python, for example, where BitBake is not
491 able to figure out dependencies.
492 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
493 produces output when it discovers something for which it cannot figure out
494 dependencies.
495 The Yocto Project team has currently not managed to cover those dependencies
496 in detail and is aware of the need to fix this situation.
497 </para>
498
499 <para>
500 Thus far, this section has limited discussion to the direct inputs into a task.
501 Information based on direct inputs is referred to as the "basehash" in the
502 code.
503 However, there is still the question of a task's indirect inputs - the
504 things that were already built and present in the
505 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
506 The checksum (or signature) for a particular task needs to add the hashes
507 of all the tasks on which the particular task depends.
508 Choosing which dependencies to add is a policy decision.
509 However, the effect is to generate a master checksum that combines the basehash
510 and the hashes of the task's dependencies.
511 </para>
512
513 <para>
514 At the code level, there are a variety of ways both the basehash and the
515 dependent task hashes can be influenced.
516 Within the BitBake configuration file, we can give BitBake some extra information
517 to help it construct the basehash.
518 The following statement effectively results in a list of global variable
519 dependency excludes - variables never included in any checksum:
520 <literallayout class='monospaced'>
521 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
522 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
523 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
524 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
525 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
526 </literallayout>
527 The previous example excludes
528 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
529 since that variable is actually constructed as a path within
530 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
531 the whitelist.
532 </para>
533
534 <para>
535 The rules for deciding which hashes of dependent tasks to include through
536 dependency chains are more complex and are generally accomplished with a
537 Python function.
538 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
539 of this and also illustrates how you can insert your own policy into the system
540 if so desired.
541 This file defines the two basic signature generators <filename>OE-Core</filename>
542 uses: "OEBasic" and "OEBasicHash".
543 By default, there is a dummy "noop" signature handler enabled in BitBake.
544 This means that behavior is unchanged from previous versions.
545 <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
546 through this setting in the <filename>bitbake.conf</filename> file:
547 <literallayout class='monospaced'>
548 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
549 </literallayout>
550 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
551 "OEBasic" version but adds the task hash to the stamp files.
552 This results in any
553 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
554 change that changes the task hash, automatically
555 causing the task to be run again.
556 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
557 values, and changes to Metadata automatically ripple across the build.
558 </para>
559
560 <para>
561 It is also worth noting that the end result of these signature generators is to
562 make some dependency and hash information available to the build.
563 This information includes:
564 <literallayout class='monospaced'>
565 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe
566 BB_BASEHASH_&lt;filename:taskname&gt; - the base hashes for each dependent task
567 BBHASHDEPS_&lt;filename:taskname&gt; - The task dependencies for each task
568 BB_TASKHASH - the hash of the currently running task
569 </literallayout>
570 </para>
571 </section>
572
573 <section id='shared-state'>
574 <title>Shared State</title>
575
576 <para>
577 Checksums and dependencies, as discussed in the previous section, solve half the
578 problem of supporting a shared state.
579 The other part of the problem is being able to use checksum information during the build
580 and being able to reuse or rebuild specific components.
581 </para>
582
583 <para>
584 The shared state class (<filename>sstate.bbclass</filename>)
585 is a relatively generic implementation of how to "capture" a snapshot of a given task.
586 The idea is that the build process does not care about the source of a task's output.
587 Output could be freshly built or it could be downloaded and unpacked from
588 somewhere - the build process does not need to worry about its origin.
589 </para>
590
591 <para>
592 There are two types of output, one is just about creating a directory
593 in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
594 A good example is the output of either <filename>do_install</filename> or
595 <filename>do_package</filename>.
596 The other type of output occurs when a set of data is merged into a shared directory
597 tree such as the sysroot.
598 </para>
599
600 <para>
601 The Yocto Project team has tried to keep the details of the implementation hidden in
602 <filename>sstate.bbclass</filename>.
603 From a user's perspective, adding shared state wrapping to a task
604 is as simple as this <filename>do_deploy</filename> example taken from
605 <filename>deploy.bbclass</filename>:
606 <literallayout class='monospaced'>
607 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
608 SSTATETASKS += "do_deploy"
609 do_deploy[sstate-name] = "deploy"
610 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
611 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
612
613 python do_deploy_setscene () {
614 sstate_setscene(d)
615 }
616 addtask do_deploy_setscene
617 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
618 </literallayout>
619 In this example, we add some extra flags to the task, a name field ("deploy"), an
620 input directory where the task sends data, and the output
621 directory where the data from the task should eventually be copied.
622 We also add a <filename>_setscene</filename> variant of the task and add the task
623 name to the <filename>SSTATETASKS</filename> list.
624 </para>
625
626 <para>
627 If you have a directory whose contents you need to preserve, you can do this with
628 a line like the following:
629 <literallayout class='monospaced'>
630 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
631 </literallayout>
632 This method, as well as the following example, also works for multiple directories.
633 <literallayout class='monospaced'>
634 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
635 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
636 do_package[sstate-lockfile] = "${PACKAGELOCK}"
637 </literallayout>
638 These methods also include the ability to take a lockfile when manipulating
639 shared state directory structures since some cases are sensitive to file
640 additions or removals.
641 </para>
642
643 <para>
644 Behind the scenes, the shared state code works by looking in
645 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
646 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
647 for shared state files.
648 Here is an example:
649 <literallayout class='monospaced'>
650 SSTATE_MIRRORS ?= "\
651 file://.* http://someserver.tld/share/sstate/PATH \n \
652 file://.* file:///some/local/dir/sstate/PATH"
653 </literallayout>
654 <note>
655 The shared state directory (<filename>SSTATE_DIR</filename>) is
656 organized into two-character subdirectories, where the subdirectory
657 names are based on the first two characters of the hash.
658 If the shared state directory structure for a mirror has the
659 same structure as <filename>SSTATE_DIR</filename>, you must
660 specify "PATH" as part of the URI to enable the build system
661 to map to the appropriate subdirectory.
662 </note>
663 </para>
664
665 <para>
666 The shared state package validity can be detected just by looking at the
667 filename since the filename contains the task checksum (or signature) as
668 described earlier in this section.
669 If a valid shared state package is found, the build process downloads it
670 and uses it to accelerate the task.
671 </para>
672
673 <para>
674 The build processes use the <filename>*_setscene</filename> tasks
675 for the task acceleration phase.
676 BitBake goes through this phase before the main execution code and tries
677 to accelerate any tasks for which it can find shared state packages.
678 If a shared state package for a task is available, the shared state
679 package is used.
680 This means the task and any tasks on which it is dependent are not
681 executed.
682 </para>
683
684 <para>
685 As a real world example, the aim is when building an IPK-based image,
686 only the <filename>do_package_write_ipk</filename> tasks would have their
687 shared state packages fetched and extracted.
688 Since the sysroot is not used, it would never get extracted.
689 This is another reason why a task-based approach is preferred over a
690 recipe-based approach, which would have to install the output from every task.
691 </para>
692 </section>
693
694 <section id='tips-and-tricks'>
695 <title>Tips and Tricks</title>
696
697 <para>
698 The code in the build system that supports incremental builds is not
699 simple code.
700 This section presents some tips and tricks that help you work around
701 issues related to shared state code.
702 </para>
703
704 <section id='debugging'>
705 <title>Debugging</title>
706
707 <para>
708 When things go wrong, debugging needs to be straightforward.
709 Because of this, the Yocto Project team included strong debugging
710 tools:
711 <itemizedlist>
712 <listitem><para>Whenever a shared state package is written out, so is a
713 corresponding <filename>.siginfo</filename> file.
714 This practice results in a pickled Python database of all
715 the metadata that went into creating the hash for a given shared state
716 package.</para></listitem>
717 <listitem><para>If you run BitBake with the <filename>--dump-signatures</filename>
718 (or <filename>-S</filename>) option, BitBake dumps out
719 <filename>.siginfo</filename> files in
720 the stamp directory for every task it would have executed instead of
721 building the specified target package.</para></listitem>
722 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
723 can process <filename>.siginfo</filename> files.
724 If you specify one of these files, BitBake dumps out the dependency
725 information in the file.
726 If you specify two files, BitBake compares the two files and dumps out
727 the differences between the two.
728 This more easily helps answer the question of "What
729 changed between X and Y?"</para></listitem>
730 </itemizedlist>
731 </para>
732 </section>
733
734 <section id='invalidating-shared-state'>
735 <title>Invalidating Shared State</title>
736
737 <para>
738 The OpenEmbedded build system uses checksums and shared state
739 cache to avoid unnecessarily rebuilding tasks.
740 Collectively, this scheme is known as "shared state code."
741 </para>
742
743 <para>
744 As with all schemes, this one has some drawbacks.
745 It is possible that you could make implicit changes to your
746 code that the checksum calculations do not take into
747 account (i.e. implicit changes).
748 These implicit changes affect a task's output but do not trigger
749 the shared state code into rebuilding a recipe.
750 Consider an example during which a tool changes its output.
751 Assume that the output of <filename>rpmdeps</filename> changes.
752 The result of the change should be that all the
753 <filename>package</filename> and
754 <filename>package_write_rpm</filename> shared state cache
755 items become invalid.
756 However, because the change to the output is
757 external to the code and therefore implicit,
758 the associated shared state cache items do not become
759 invalidated.
760 In this case, the build process uses the cached items rather
761 than running the task again.
762 Obviously, these types of implicit changes can cause problems.
763 </para>
764
765 <para>
766 To avoid these problems during the build, you need to
767 understand the effects of any changes you make.
768 Realize that changes you make directly to a function
769 are automatically factored into the checksum calculation.
770 Thus, these explicit changes invalidate the associated area of
771 sstate cache.
772 However, you need to be aware of any implicit changes that
773 are not obvious changes to the code and could affect the output
774 of a given task.
775 </para>
776
777 <para>
778 When you identify an implicit change, you can easily take steps
779 to invalidate the cache and force the tasks to run.
780 The steps you can take are as simple as changing a function's
781 comments in the source code.
782 For example, to invalidate package shared state files, change
783 the comment statements of <filename>do_package</filename> or
784 the comments of one of the functions it calls.
785 Even though the change is purely cosmetic, it causes the
786 checksum to be recalculated and forces the OpenEmbedded build
787 system to run the task again.
788 </para>
789
790 <note>
791 For an example of a commit that makes a cosmetic change to
792 invalidate shared state, see this
793 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
794 </note>
795 </section>
796 </section>
797</section>
798
799<section id='x32'>
800 <title>x32</title>
801
802 <para>
803 x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
804 An ABI defines the calling conventions between functions in a processing environment.
805 The interface determines what registers are used and what the sizes are for various C data types.
806 </para>
807
808 <para>
809 Some processing environments prefer using 32-bit applications even when running
810 on Intel 64-bit platforms.
811 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
812 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
813 leaving the system underutilized.
814 Now consider the x86_64 psABI.
815 This ABI is newer and uses 64-bits for data sizes and program pointers.
816 The extra bits increase the footprint size of the programs, libraries,
817 and also increases the memory and file system size requirements.
818 Executing under the x32 psABI enables user programs to utilize CPU and system resources
819 more efficiently while keeping the memory footprint of the applications low.
820 Extra bits are used for registers but not for addressing mechanisms.
821 </para>
822
823 <section id='support'>
824 <title>Support</title>
825
826 <para>
827 While the x32 psABI specifications are not fully finalized, this Yocto Project
828 release supports current development specifications of x32 psABI.
829 As of this release of the Yocto Project, x32 psABI support exists as follows:
830 <itemizedlist>
831 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
832 </para></listitem>
833 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
834 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
835 <filename>core-image-sato</filename> images.</para></listitem>
836 </itemizedlist>
837 </para>
838 </section>
839
840 <section id='stabilizing-and-completing-x32'>
841 <title>Stabilizing and Completing x32</title>
842
843 <para>
844 As of this Yocto Project release, the x32 psABI kernel and library
845 interfaces specifications are not finalized.
846 </para>
847
848 <para>
849 Future Plans for the x32 psABI in the Yocto Project include the following:
850 <itemizedlist>
851 <listitem><para>Enhance and fix the few remaining recipes so they
852 work with and support x32 toolchains.</para></listitem>
853 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
854 <listitem><para>Support larger images.</para></listitem>
855 </itemizedlist>
856 </para>
857 </section>
858
859 <section id='using-x32-right-now'>
860 <title>Using x32 Right Now</title>
861
862 <para>
863 Follow these steps to use the x32 spABI:
864 <itemizedlist>
865 <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
866 machines by editing the <filename>conf/local.conf</filename> like this:
867 <literallayout class='monospaced'>
868 MACHINE = "qemux86-64"
869 DEFAULTTUNE = "x86-64-x32"
870 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
871 or 'INVALID'), True) or 'lib'}"
872 #MACHINE = "genericx86"
873 #DEFAULTTUNE = "core2-64-x32"
874 </literallayout></para></listitem>
875 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
876 Here is an example:
877 <literallayout class='monospaced'>
878 $ bitbake core-image-sato
879 </literallayout></para></listitem>
880 <listitem><para>As usual, run your image using QEMU:
881 <literallayout class='monospaced'>
882 $ runqemu qemux86-64 core-image-sato
883 </literallayout></para></listitem>
884 </itemizedlist>
885 </para>
886 </section>
887</section>
888
889<section id="wayland">
890 <title>Wayland</title>
891
892 <para>
893 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Wayland</ulink>
894 is a computer display server protocol that
895 provides a method for compositing window managers to communicate
896 directly with applications and video hardware and expects them to
897 communicate with input hardware using other libraries.
898 Using Wayland with supporting targets can result in better control
899 over graphics frame rendering than an application might otherwise
900 achieve.
901 </para>
902
903 <para>
904 The Yocto Project provides the Wayland protocol libraries and the
905 reference Weston compositor as part of its release.
906 This section describes what you need to do to implement Wayland and
907 use the compositor when building an image for a supporting target.
908 </para>
909
910 <section id="wayland-support">
911 <title>Support</title>
912
913 <para>
914 The Wayland protocol libraries and the reference Weston compositor
915 ship as integrated packages in the <filename>meta</filename> layer
916 of the
917 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
918 Specifically, you can find the recipes that build both Wayland
919 and Weston at <filename>meta/recipes-graphics/wayland</filename>.
920 </para>
921
922 <para>
923 You can build both the Wayland and Weston packages for use only
924 with targets that accept the
925 <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
926 which is also known as Mesa DRI.
927 This implies that you cannot build and use the packages if your
928 target uses, for example, the
929 <trademark class='registered'>Intel</trademark> Embedded Media and
930 Graphics Driver (<trademark class='registered'>Intel</trademark>
931 EMGD) that overrides Mesa DRI.
932 </para>
933
934 <note>
935 Due to lack of EGL support, Weston 1.0.3 will not run directly on
936 the emulated QEMU hardware.
937 However, this version of Weston will run under X emulation without
938 issues.
939 </note>
940 </section>
941
942 <section id="enabling-wayland-in-an-image">
943 <title>Enabling Wayland in an Image</title>
944
945 <para>
946 To enable Wayland, you need to enable it to be built and enable
947 it to be included in the image.
948 </para>
949
950 <section id="enable-building">
951 <title>Building</title>
952
953 <para>
954 To cause Mesa to build the <filename>wayland-egl</filename>
955 platform and Weston to build Wayland with Kernel Mode
956 Setting
957 (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
958 support, include the "wayland" flag in the
959 <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
960 statement in your <filename>local.conf</filename> file:
961 <literallayout class='monospaced'>
962 DISTRO_FEATURES_append = " wayland"
963 </literallayout>
964 </para>
965
966 <note>
967 If X11 has been enabled elsewhere, Weston will build Wayland
968 with X11 support
969 </note>
970 </section>
971
972 <section id="enable-installation-in-an-image">
973 <title>Installing</title>
974
975 <para>
976 To install the Wayland feature into an image, you must
977 include the following
978 <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
979 statement in your <filename>local.conf</filename> file:
980 <literallayout class='monospaced'>
981 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
982 </literallayout>
983 </para>
984 </section>
985 </section>
986
987 <section id="running-weston">
988 <title>Running Weston</title>
989
990 <para>
991 To run Weston inside X11, enabling it as described earlier and
992 building a Sato image is sufficient.
993 If you are running your image under Sato, a Weston Launcher appears
994 in the "Utility" category.
995 </para>
996
997 <para>
998 Alternatively, you can run Weston through the command-line
999 interpretor (CLI), which is better suited for development work.
1000 To run Weston under the CLI, you need to do the following after
1001 your image is built:
1002 <orderedlist>
1003 <listitem><para>Run these commands to export
1004 <filename>XDG_RUNTIME_DIR</filename>:
1005 <literallayout class='monospaced'>
1006 mkdir -p /tmp/$USER-weston
1007 chmod 0700 /tmp/$USER-weston
1008 export XDG_RUNTIME_DIR=/tmp/$USER=weston
1009 </literallayout></para></listitem>
1010 <listitem><para>Launch Weston in the shell:
1011 <literallayout class='monospaced'>
1012 weston
1013 </literallayout></para></listitem>
1014 </orderedlist>
1015 </para>
1016 </section>
1017</section>
1018
1019<section id="licenses">
1020 <title>Licenses</title>
1021
1022 <para>
1023 This section describes the mechanism by which the OpenEmbedded build system
1024 tracks changes to licensing text.
1025 The section also describes how to enable commercially licensed recipes,
1026 which by default are disabled.
1027 </para>
1028
1029 <para>
1030 For information that can help you maintain compliance with various open
1031 source licensing during the lifecycle of the product, see the
1032 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
1033 in the Yocto Project Development Manual.
1034 </para>
1035
1036 <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
1037 <title>Tracking License Changes</title>
1038
1039 <para>
1040 The license of an upstream project might change in the future.
1041 In order to prevent these changes going unnoticed, the
1042 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
1043 variable tracks changes to the license text. The checksums are validated at the end of the
1044 configure step, and if the checksums do not match, the build will fail.
1045 </para>
1046
1047 <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
1048 <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
1049
1050 <para>
1051 The <filename>LIC_FILES_CHKSUM</filename>
1052 variable contains checksums of the license text in the source code for the recipe.
1053 Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
1054 <literallayout class='monospaced'>
1055 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
1056 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
1057 file://licfile2.txt;endline=50;md5=zzzz \
1058 ..."
1059 </literallayout>
1060 </para>
1061
1062 <para>
1063 The build system uses the
1064 <filename><link linkend='var-S'>S</link></filename> variable as
1065 the default directory used when searching files listed in
1066 <filename>LIC_FILES_CHKSUM</filename>.
1067 The previous example employs the default directory.
1068 </para>
1069
1070 <para>
1071 Consider this next example:
1072 <literallayout class='monospaced'>
1073 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
1074 md5=bb14ed3c4cda583abc85401304b5cd4e"
1075 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
1076 </literallayout>
1077 </para>
1078
1079 <para>
1080 The first line locates a file in
1081 <filename>${S}/src/ls.c</filename>.
1082 The second line refers to a file in
1083 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
1084 </para>
1085 <para>
1086 Note that <filename>LIC_FILES_CHKSUM</filename> variable is
1087 mandatory for all recipes, unless the
1088 <filename>LICENSE</filename> variable is set to "CLOSED".
1089 </para>
1090 </section>
1091
1092 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
1093 <title>Explanation of Syntax</title>
1094 <para>
1095 As mentioned in the previous section, the
1096 <filename>LIC_FILES_CHKSUM</filename> variable lists all the
1097 important files that contain the license text for the source code.
1098 It is possible to specify a checksum for an entire file, or a specific section of a
1099 file (specified by beginning and ending line numbers with the "beginline" and "endline"
1100 parameters, respectively).
1101 The latter is useful for source files with a license notice header,
1102 README documents, and so forth.
1103 If you do not use the "beginline" parameter, then it is assumed that the text begins on the
1104 first line of the file.
1105 Similarly, if you do not use the "endline" parameter, it is assumed that the license text
1106 ends with the last line of the file.
1107 </para>
1108
1109 <para>
1110 The "md5" parameter stores the md5 checksum of the license text.
1111 If the license text changes in any way as compared to this parameter
1112 then a mismatch occurs.
1113 This mismatch triggers a build failure and notifies the developer.
1114 Notification allows the developer to review and address the license text changes.
1115 Also note that if a mismatch occurs during the build, the correct md5
1116 checksum is placed in the build log and can be easily copied to the recipe.
1117 </para>
1118
1119 <para>
1120 There is no limit to how many files you can specify using the
1121 <filename>LIC_FILES_CHKSUM</filename> variable.
1122 Generally, however, every project requires a few specifications for license tracking.
1123 Many projects have a "COPYING" file that stores the license information for all the source
1124 code files.
1125 This practice allows you to just track the "COPYING" file as long as it is kept up to date.
1126 </para>
1127
1128 <tip>
1129 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
1130 error and displays the correct "md5" parameter value during the build.
1131 The correct parameter is also captured in the build log.
1132 </tip>
1133
1134 <tip>
1135 If the whole file contains only license text, you do not need to use the "beginline" and
1136 "endline" parameters.
1137 </tip>
1138 </section>
1139 </section>
1140
1141 <section id="enabling-commercially-licensed-recipes">
1142 <title>Enabling Commercially Licensed Recipes</title>
1143
1144 <para>
1145 By default, the OpenEmbedded build system disables
1146 components that have commercial or other special licensing
1147 requirements.
1148 Such requirements are defined on a
1149 recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable
1150 definition in the affected recipe.
1151 For instance, the
1152 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1153 recipe contains the following statement:
1154 <literallayout class='monospaced'>
1155 LICENSE_FLAGS = "commercial"
1156 </literallayout>
1157 Here is a slightly more complicated example that contains both an
1158 explicit recipe name and version (after variable expansion):
1159 <literallayout class='monospaced'>
1160 LICENSE_FLAGS = "license_${PN}_${PV}"
1161 </literallayout>
1162 In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
1163 definition to be enabled and included in an image, it
1164 needs to have a matching entry in the global
1165 <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable
1166 typically defined in your <filename>local.conf</filename> file.
1167 For example, to enable
1168 the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1169 package, you could add either the string
1170 "commercial_gst-plugins-ugly" or the more general string
1171 "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
1172 See the
1173 "<link linkend='license-flag-matching'>License Flag Matching</link>" section
1174 for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
1175 Here is the example:
1176 <literallayout class='monospaced'>
1177 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
1178 </literallayout>
1179 Likewise, to additionally enable the package built from the recipe containing
1180 <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
1181 that the actual recipe name was <filename>emgd_1.10.bb</filename>,
1182 the following string would enable that package as well as
1183 the original <filename>gst-plugins-ugly</filename> package:
1184 <literallayout class='monospaced'>
1185 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
1186 </literallayout>
1187 As a convenience, you do not need to specify the complete license string
1188 in the whitelist for every package.
1189 You can use an abbreviated form, which consists
1190 of just the first portion or portions of the license string before
1191 the initial underscore character or characters.
1192 A partial string will match
1193 any license that contains the given string as the first
1194 portion of its license.
1195 For example, the following
1196 whitelist string will also match both of the packages
1197 previously mentioned as well as any other packages that have
1198 licenses starting with "commercial" or "license".
1199 <literallayout class='monospaced'>
1200 LICENSE_FLAGS_WHITELIST = "commercial license"
1201 </literallayout>
1202 </para>
1203
1204 <section id="license-flag-matching">
1205 <title>License Flag Matching</title>
1206
1207 <para>
1208 License flag matching allows you to control what recipes the
1209 OpenEmbedded build system includes in the build.
1210 Fundamentally, the build system attempts to match
1211 <filename>LICENSE_FLAGS</filename> strings found in
1212 recipes against <filename>LICENSE_FLAGS_WHITELIST</filename>
1213 strings found in the whitelist.
1214 A match causes the build system to include a recipe in the
1215 build, while failure to find a match causes the build system to
1216 exclude a recipe.
1217 </para>
1218
1219 <para>
1220 In general, license flag matching is simple.
1221 However, understanding some concepts will help you
1222 correctly and effectively use matching.
1223 </para>
1224
1225 <para>
1226 Before a flag
1227 defined by a particular recipe is tested against the
1228 contents of the whitelist, the expanded string
1229 <filename>_${PN}</filename> is appended to the flag.
1230 This expansion makes each <filename>LICENSE_FLAGS</filename>
1231 value recipe-specific.
1232 After expansion, the string is then matched against the
1233 whitelist.
1234 Thus, specifying
1235 <filename>LICENSE_FLAGS = "commercial"</filename>
1236 in recipe "foo", for example, results in the string
1237 <filename>"commercial_foo"</filename>.
1238 And, to create a match, that string must appear in the
1239 whitelist.
1240 </para>
1241
1242 <para>
1243 Judicious use of the <filename>LICENSE_FLAGS</filename>
1244 strings and the contents of the
1245 <filename>LICENSE_FLAGS_WHITELIST</filename> variable
1246 allows you a lot of flexibility for including or excluding
1247 recipes based on licensing.
1248 For example, you can broaden the matching capabilities by
1249 using license flags string subsets in the whitelist.
1250 <note>When using a string subset, be sure to use the part of
1251 the expanded string that precedes the appended underscore
1252 character (e.g. <filename>usethispart_1.3</filename>,
1253 <filename>usethispart_1.4</filename>, and so forth).
1254 </note>
1255 For example, simply specifying the string "commercial" in
1256 the whitelist matches any expanded
1257 <filename>LICENSE_FLAGS</filename> definition that starts with
1258 the string "commercial" such as "commercial_foo" and
1259 "commercial_bar", which are the strings the build system
1260 automatically generates for hypothetical recipes named
1261 "foo" and "bar" assuming those recipes simply specify the
1262 following:
1263 <literallayout class='monospaced'>
1264 LICENSE_FLAGS = "commercial"
1265 </literallayout>
1266 Thus, you can choose to exhaustively
1267 enumerate each license flag in the whitelist and
1268 allow only specific recipes into the image, or
1269 you can use a string subset that causes a broader range of
1270 matches to allow a range of recipes into the image.
1271 </para>
1272
1273 <para>
1274 This scheme works even if the
1275 <filename>LICENSE_FLAGS</filename> string already
1276 has <filename>_${PN}</filename> appended.
1277 For example, the build system turns the license flag
1278 "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
1279 match both the general "commercial" and the specific
1280 "commercial_1.2_foo" strings found in the whitelist, as
1281 expected.
1282 </para>
1283
1284 <para>
1285 Here are some other scenarios:
1286 <itemizedlist>
1287 <listitem><para>You can specify a versioned string in the
1288 recipe such as "commercial_foo_1.2" in a "foo" recipe.
1289 The build system expands this string to
1290 "commercial_foo_1.2_foo".
1291 Combine this license flag with a whitelist that has
1292 the string "commercial" and you match the flag along
1293 with any other flag that starts with the string
1294 "commercial".</para></listitem>
1295 <listitem><para>Under the same circumstances, you can
1296 use "commercial_foo" in the whitelist and the
1297 build system not only matches "commercial_foo_1.2" but
1298 also matches any license flag with the string
1299 "commercial_foo", regardless of the version.
1300 </para></listitem>
1301 <listitem><para>You can be very specific and use both the
1302 package and version parts in the whitelist (e.g.
1303 "commercial_foo_1.2") to specifically match a
1304 versioned recipe.</para></listitem>
1305 </itemizedlist>
1306 </para>
1307 </section>
1308
1309 <section id="other-variables-related-to-commercial-licenses">
1310 <title>Other Variables Related to Commercial Licenses</title>
1311
1312 <para>
1313 Other helpful variables related to commercial
1314 license handling exist and are defined in the
1315 <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
1316 <literallayout class='monospaced'>
1317 COMMERCIAL_AUDIO_PLUGINS ?= ""
1318 COMMERCIAL_VIDEO_PLUGINS ?= ""
1319 COMMERCIAL_QT = ""
1320 </literallayout>
1321 If you want to enable these components, you can do so by making sure you have
1322 statements similar to the following
1323 in your <filename>local.conf</filename> configuration file:
1324 <literallayout class='monospaced'>
1325 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
1326 gst-plugins-ugly-mpegaudioparse"
1327 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
1328 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
1329 COMMERCIAL_QT ?= "qmmp"
1330 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
1331 </literallayout>
1332 Of course, you could also create a matching whitelist
1333 for those components using the more general "commercial"
1334 in the whitelist, but that would also enable all the
1335 other packages with <filename>LICENSE_FLAGS</filename> containing
1336 "commercial", which you may or may not want:
1337 <literallayout class='monospaced'>
1338 LICENSE_FLAGS_WHITELIST = "commercial"
1339 </literallayout>
1340 </para>
1341
1342 <para>
1343 Specifying audio and video plug-ins as part of the
1344 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
1345 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
1346 or commercial Qt components as part of
1347 the <filename>COMMERCIAL_QT</filename> statement (along
1348 with the enabling <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
1349 plug-ins or components into built images, thus adding
1350 support for media formats or components.
1351 </para>
1352 </section>
1353 </section>
1354</section>
1355</chapter>
1356<!--
1357vim: expandtab tw=80 ts=4
1358-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-13 12:14:48 +0000