blob: 25111924da939626011eaf7736f830eabeaced34 [file] [log] [blame]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
See the License for the specific language governing permissions and
limitations under the License.
.. _handling_files_composition:
In this chapter we will explore how to create *compositions* of multiple
input filesystem trees, using the :mod:`compose <elements.compose>` element.
.. note::
This example is distributed with BuildStream
in the `doc/examples/composition
Composing a directory tree based on a set of build dependencies is often
one of the important steps you might perform in order to create a single artifact
which can be checked out and deployed.
In order to use the :mod:`compose <elements.compose>` element, it is important
to first understand the concept of :ref:`split rules <public_split_rules>`, which
we will cover in this chapter.
Introducing split rules
The :ref:`split rules <public_split_rules>` of an element declaration denote
which sets of files in the given element's resulting artifact belong to which
*domain name*.
The *domains* can then be used in various ways, using plugins which understand
*split rule domains*.
BuildStream's :ref:`default project configuration <project_builtin_defaults>`
contains a sensible set of default *split rule domains* for the purpose of
artifact splitting, they can be overridden in :ref:`your project.conf <project_split_rules>`,
and finally on a per element basis in the :ref:`public data <public_builtin>`
of your element declarations.
.. note::
Projects are free to add additional *split rule domains* on top of the
default domains provided by the default project configuration.
There is nothing wrong with defining split rule domains which *overlap*,
possibly capturing some of the same files also captured by another
*domain*, however you should be aware of this when later using your
split rules with a plugin which processes them, like the
:mod:`compose <elements.compose>` element described in this chapter.
Example of split rule declaration
In an element, you might need to define or extend the ``split-rules``
in order to capture files in custom locations in a logical *domain*.
Here is an example of how you might use the
:ref:`list append directive <format_directives_list_append>`
to append an additional rule to your ``split-rules`` list in order to
capture additional data files which your application or library might
want to include in the *runtime domain*:
.. code:: yaml
# Add our .dat files to the runtime domain
- |
Split rules are absolute paths which denote files within an artifact's root
directory. The globbing patterns supported in split rules are defined in the
:func:`reference documentation here <buildstream.utils.glob>`.
.. important::
Note that because of variable expansion, split rules can often be
*resolved differently* for elements which have overridden path
related variables, like ``%{prefix}``.
This usually means that you do not need to explicitly extend or override
split rules on a specific element unless your element installs files to
special case locations.
Project structure
In this example we expand on the chapter about
:ref:`integration commands <tutorial_integration_commands>`, so we will
only discuss the files which are added or changed from that example.
.. literalinclude:: ../../examples/composition/elements/base/alpine.bst
:language: yaml
Here we have modified the base runtime, so as to specify that for this
element, we want to also include the runtime linker into the *runtime domain*.
.. literalinclude:: ../../examples/composition/elements/runtime-only.bst
:language: yaml
As we can see, this :mod:`compose <elements.compose>` element has
been configured to only include files from the *runtime domain*.
Using the project
Now that we've presented how :ref:`split rules <public_split_rules>`
work and shown how to use them in the context of this example, lets
use the :mod:`compose <elements.compose>` element we've created and
observe the results.
Building the project
.. raw:: html
:file: ../sessions/composition-build.html
As you can see in the output, this composition has only a few hundred
files, but the complete ``alpine.bst`` runtime has several thousand
List the content
At the risk of this being a long list, let's :ref:`list the
contents of this artifact <invoking_artifact_list_contents>`
.. raw:: html
:file: ../sessions/composition-list-contents.html
Some things to observe here:
* The list does include the ``/usr/bin/hello`` program and
also the ``/usr/lib/`` shared library.
These paths are both captured by the default split rules
for the *runtime domain*.
* The list does not include the ``/usr/include/libhello.h``
header file which was used to compile ``/usr/bin/hello``.
The header file is not captured by the *runtime domain*
by default. It is however captured by the *devel domain*.
* The runtime linker ``/lib/``, as this was
explicitly added to the *runtime domain* for the ``base/alpine.bst``
element which provides this file.
.. tip::
The reader at this time might want to list the content of
other elements built from this project, such as the
``hello.bst`` element by itself, or the ``base/alpine.bst``
Run the program
Finally, lets just run the program we built.
.. raw:: html
:file: ../sessions/composition-shell.html
Here we can see that we at least have the required files to run
our hello world program, however we would not have if we were
missing the runtime linker which we added in ``base/alpine.bst``.
In this chapter we've gotten familiar with :ref:`split rules <public_split_rules>`
annotations, and we've learned enough about the :mod:`compose <elements.compose>`
element such that we can start creating our own compositions using
*split domains*.
We've also used the :ref:`list append directive <format_directives_list_append>`
and we are now observing the contents of artifacts using
:ref:`bst artifact list-contents <invoking_artifact_list_contents>`.