NEWS - buildstream - Git at Google

 =================
 buildstream 1.3.1
 =================

   o BREAKING CHANGE: YAML New World Order. The parsing and loading of .bst files
     and node handling has been completely rewritten and is now faster. This now
     requires that plugin authors must use the Plugin API to access or modify node
     members. Regular dictionary accesses are no longer valid.

     This change has also removed the need of a YAML cache as it proved to no longer
     be of benefit.

   o Added `bst artifact delete` subcommand. This command removes artifacts from
     the local cache. Mulitple element names and artifact refs can be specified
     as arguments.

   o BREAKING CHANGE: The top level commands `checkout`, `push` and `pull` have
     been moved to the `bst artifact` subcommand group and are now obsolete.
     For example, you must now use `bst artifact pull hello.bst`.

     The behaviour of `checkout` has changed. The previously mandatory LOCATION
     argument should now be specified with the `--directory` option. In addition
     to this, `--tar` is no longer a flag, it is a mutually incompatible option
     to `--directory`. For example, `bst artifact checkout foo.bst --tar foo.tar.gz`.


   o Added `bst artifact log` subcommand for viewing build logs.

   o BREAKING CHANGE: The bst source-bundle command has been removed. The
     functionality it provided has been replaced by the `--include-build-scripts`
     option of the `bst source-checkout` command. To produce a tarball containing
     an element's sources and generated build scripts you can do the command
     `bst source-checkout --include-build-scripts --tar foo.bst some-file.tar`

   o BREAKING CHANGE: `bst track` and `bst fetch` commands are now obsolete.
     Their functionality is provided by `bst source track` and
     `bst source fetch` respectively.

   o Added new `bst source checkout` command to checkout sources of an element.

   o BREAKING CHANGE: Default strip-commands have been removed as they are too
     specific. Recommendation if you are building in Linux is to use the
     ones being used in freedesktop-sdk project, for example

   o Running commands without elements specified will now attempt to use
     the default targets defined in the project configuration.
     If no default target is defined, all elements in the project will be used.

   o All elements must now be suffixed with `.bst`
     Attempting to use an element that does not have the `.bst` extension,
     will result in a warning.

   o BREAKING CHANGE: The 'manual' element lost its default 'MAKEFLAGS' and 'V'
     environment variables. There is already a 'make' element with the same
     variables. Note that this is a breaking change, it will require users to
     make changes to their .bst files if they are expecting these environment
     variables to be set.

   o BREAKING CHANGE: The 'auto-init' functionality has been removed. This would
     offer to create a project in the event that bst was run against a directory
     without a project, to be friendly to new users. It has been replaced with
     an error message and a hint instead, to avoid bothering folks that just
     made a mistake.

   o BREAKING CHANGE: The unconditional 'Are you sure?' prompts have been
     removed. These would always ask you if you were sure when running
     'bst workspace close --remove-dir' or 'bst workspace reset'. They got in
     the way too often.

   o Failed builds are included in the cache as well.
     `bst checkout` will provide anything in `%{install-root}`.
     A build including cached fails will cause any dependant elements
     to not be scheduled and fail during artifact assembly,
     and display the retry prompt during an interactive session.

   o Due to enabling the use of relative workspaces, "Legacy" workspaces
     may need to be closed and remade before the changes will affect them.
     Downgrading after using this feature may result in workspaces
     not functioning correctly

   o Elements may now specify 'build-depends' and 'runtime-depends' fields
     to avoid having to specify the dependency type for every entry in
     'depends'.

   o Elements may now specify cross-junction dependencies as simple strings
     using the format '{junction-name}:{element-name}'.

   o Junction elements may now specify another junction as their target, using
     the `target` configuration option.

   o Source plugins may now request access access to previous during track and
     fetch by setting `BST_REQUIRES_PREVIOUS_SOURCES_TRACK` and/or
     `BST_REQUIRES_PREVIOUS_SOURCES_FETCH` attributes.

   o Add new `pip` source plugin for downloading python packages using pip,
     based on requirements files from previous sources.

   o Generate Docker images from built artifacts using
     `contrib/bst-docker-import` script.

   o Added Documentation on how to create out of source builds. This includes the
     new the `conf-root` variable to make the process easier. And there has been
     a bug fix to workspaces so they can be build in workspaces too.

   o Creating a build shell through the interactive mode or `bst shell --build`
     will now use the cached build tree if available locally. It is now easier to
     debug local build failures.

   o `bst shell --sysroot` now takes any directory that contains a sysroot,
     instead of just a specially-formatted build-root with a `root` and `scratch`
     subdirectory.

   o Due to the element `build tree` being cached in the respective artifact their
     size in some cases has significantly increased. In *most* cases the build trees
     are not utilised when building targets, as such by default bst 'pull' & 'build'
     will not fetch build trees from remotes. This behaviour can be overridden with
     the cli main option '--pull-buildtrees', or the user configuration cache group
     option 'pull-buildtrees = True'. The override will also add the build tree to
     already cached artifacts. When attempting to populate an artifactcache server
     with cached artifacts, only 'complete' elements can be pushed. If the element
     is expected to have a populated build tree then it must be cached before pushing.

   o `bst workspace open` now supports the creation of multiple elements and
     allows the user to set a default location for their creation. This has meant
     that the new CLI is no longer backwards compatible with buildstream 1.2.

   o Add sandbox API for command batching and use it for build, script, and
     compose elements.

   o BREAKING CHANGE: The `git` plugin does not create a local `.git`
     repository by default.  If `git describe` is required to work, the
     plugin has now a tag tracking feature instead. This can be enabled
     by setting 'track-tags'.

   o Opening a workspace now creates a .bstproject.yaml file that allows buildstream
     commands to be run from a workspace that is not inside a project.

   o Specifying an element is now optional for some commands when buildstream is run
     from inside a workspace - the 'build', 'checkout', 'fetch', 'pull', 'push',
     'shell', 'show', 'source-checkout', 'track', 'workspace close' and 'workspace reset'
     commands are affected.

   o bst 'build' now has '--remote, -r' option, inline with bst 'push' & 'pull'.
     Providing a remote will limit build's pull/push remote actions to the given
     remote specifically, ignoring those defined via user or project configuration.

   o Artifacts can now be cached explicitly with an empty `build tree` when built.
     Element types without a build-root were already cached with an empty build tree
     directory, this can now be extended to all or successful artifacts to save on cache
     overheads. The cli main option '--cache-buildtrees' or the user configuration cache
     group option 'cache-buildtrees' can be set as 'always', 'auto' or 'never', with
     the default being 'auto'. Note, as the cache-key for the artifact is independent of
     the cached build tree input it will remain unaltered, however the availbility of the
     build tree content may differ.

   o BREAKING CHANGE: Symlinks are no longer resolved during staging and absolute
     symlinks are now preserved instead of being converted to relative symlinks.

   o BREAKING CHANGE: Overlap whitelists now require absolute paths. This allows
     use of variables such as %{prefix} and matches the documentation.


 =================
 buildstream 1.1.5
 =================

   o Add a `--tar` option to `bst checkout` which allows a tarball to be
     created from the artifact contents.

   o Fetching and tracking will consult mirrors defined in project config,
     and the preferred mirror to fetch from can be defined in the command
     line or user config.

   o Added new `remote` source plugin for downloading file blobs

   o Add support for the new include '(@)' directive in project.conf and .bst files


 =================
 buildstream 1.1.4
 =================

   o `bst workspace` commands and `bst track` will substitute their
     source elements when performing those operations, e.g. performing
     `bst track` on a filter element will track the sources on the
     element that it depends on (if it has sources).

   o Added new simple `make` element

   o Switch to Remote Execution CAS-based artifact cache on all platforms.

     Artifact servers need to be migrated.

   o BuildStream now requires python version >= 3.5

   o BuildStream will now automatically clean up old artifacts when it
     runs out of space. The exact behavior is configurable in the user's
     buildstream.conf.


 =================
 buildstream 1.1.3
 =================

   o Added new `bst init` command to initialize a new project.

   o Cross junction tracking is now disabled by default for projects
     which can support this by using project.refs ref-storage

     New options have been added to explicitly enable cross-junction
     tracking.

   o Failed jobs are now summarised at the end of a build.

     Use `--verbose` and `--no-verbose` to adjust the amount of detail given.

   o BuildElements' `configure-commands` are only run once for
     workspaces now, which allows for incremental builds.

     Appropriate API for plugins is also exposed through
     `Element.prepare`.

   o The `cmake` plugin now supports building with ninja with the
     newly added `generator` configuration option.

   o `bst workspace close` and `bst workspace reset` now support multiple
     elements. All elements can be specified using `--all`.

   o The elements whose cache keys had to be determined during the build
     are summarised at the end of the build.

   o Fixed versioning introspection to be dynamic, many users use
     a developer install mode so they can update with git, now the
     version information is always up to date in logs.

     This causes a minor API break: The --version output now only
     outputs the version.


 =================
 buildstream 1.1.2
 =================

   o New ref-storage option allows one to store source refs, such
     as git shas, in one central project.refs file instead of inline
     with the source declarations.

   o Deprecated `--track-save` optionality in `bst build`, this
     does not make sense to support now that we have project.refs.

   o Added the `sandbox` configuration option which can be used in
     `project.conf` and elements, to control the user ID and group ID
     used in build sandboxes.

   o Added new `deb` source implementation, for staging of downloaded
     deb package files.


 =================
 buildstream 1.1.1
 =================

   o New project configuration controlling how the sandbox behaves
     when `bst shell` is used; allowing projects to provide a more
     functional shell environment.

   o The `bst shell` command now has a `--mount` option allowing
     users to mount files and directories into the sandbox for
     testing purposes.

   o Log lines are now configurable with the new "message-format"
     user configuration, allowing one to express optional fields
     such as microsecond precision and wallclock time.

   o Newly added filter element

   o Git source plugin now allows disabling of submodule checkouts

   o In the same way we allow overriding element configurations
     by their 'kind' in project.conf, we now support the same
     for source plugin configurations.

   o Tar and zip sources now automatically recall an `etag`
     from the http headers, optimizing tracking of tarballs
     significantly (issue #62)


 =================
 buildstream 1.1.0
 =================

   o Multiple artifact caches are now supported in project and
     user configuration with a priority order (issue #85)
   o Add junction support for subprojects
   o Changes towards incremental builds in workspaces
   o `bst shell --build` now creates true build sandbox
   o Many bug fixes


 =================
 buildstream 1.0.0
 =================
 First stable release of BuildStream

 BuildStream 1.0.0 is all about API stability - for the past months we
 have been reviewing our various API surfaces, implementing strategies
 for revisioning of our interfaces and cleaning up. Long term stability
 is very important for build reproducibility over time, and this release
 is the first promise we are making on any API surfaces.

 Stable API surfaces include:
   o The command line interface
   o The YAML user configuration file format
   o The YAML project `.bst` file format
   o The core Python module imported by external plugins
	=================
	buildstream 1.3.1
	=================

	o BREAKING CHANGE: YAML New World Order. The parsing and loading of .bst files
	and node handling has been completely rewritten and is now faster. This now
	requires that plugin authors must use the Plugin API to access or modify node
	members. Regular dictionary accesses are no longer valid.

	This change has also removed the need of a YAML cache as it proved to no longer
	be of benefit.

	o Added `bst artifact delete` subcommand. This command removes artifacts from
	the local cache. Mulitple element names and artifact refs can be specified
	as arguments.

	o BREAKING CHANGE: The top level commands `checkout`, `push` and `pull` have
	been moved to the `bst artifact` subcommand group and are now obsolete.
	For example, you must now use `bst artifact pull hello.bst`.

	The behaviour of `checkout` has changed. The previously mandatory LOCATION
	argument should now be specified with the `--directory` option. In addition
	to this, `--tar` is no longer a flag, it is a mutually incompatible option
	to `--directory`. For example, `bst artifact checkout foo.bst --tar foo.tar.gz`.


	o Added `bst artifact log` subcommand for viewing build logs.

	o BREAKING CHANGE: The bst source-bundle command has been removed. The
	functionality it provided has been replaced by the `--include-build-scripts`
	option of the `bst source-checkout` command. To produce a tarball containing
	an element's sources and generated build scripts you can do the command
	`bst source-checkout --include-build-scripts --tar foo.bst some-file.tar`

	o BREAKING CHANGE: `bst track` and `bst fetch` commands are now obsolete.
	Their functionality is provided by `bst source track` and
	`bst source fetch` respectively.

	o Added new `bst source checkout` command to checkout sources of an element.

	o BREAKING CHANGE: Default strip-commands have been removed as they are too
	specific. Recommendation if you are building in Linux is to use the
	ones being used in freedesktop-sdk project, for example

	o Running commands without elements specified will now attempt to use
	the default targets defined in the project configuration.
	If no default target is defined, all elements in the project will be used.

	o All elements must now be suffixed with `.bst`
	Attempting to use an element that does not have the `.bst` extension,
	will result in a warning.

	o BREAKING CHANGE: The 'manual' element lost its default 'MAKEFLAGS' and 'V'
	environment variables. There is already a 'make' element with the same
	variables. Note that this is a breaking change, it will require users to
	make changes to their .bst files if they are expecting these environment
	variables to be set.

	o BREAKING CHANGE: The 'auto-init' functionality has been removed. This would
	offer to create a project in the event that bst was run against a directory
	without a project, to be friendly to new users. It has been replaced with
	an error message and a hint instead, to avoid bothering folks that just
	made a mistake.

	o BREAKING CHANGE: The unconditional 'Are you sure?' prompts have been
	removed. These would always ask you if you were sure when running
	'bst workspace close --remove-dir' or 'bst workspace reset'. They got in
	the way too often.

	o Failed builds are included in the cache as well.
	`bst checkout` will provide anything in `%{install-root}`.
	A build including cached fails will cause any dependant elements
	to not be scheduled and fail during artifact assembly,
	and display the retry prompt during an interactive session.

	o Due to enabling the use of relative workspaces, "Legacy" workspaces
	may need to be closed and remade before the changes will affect them.
	Downgrading after using this feature may result in workspaces
	not functioning correctly

	o Elements may now specify 'build-depends' and 'runtime-depends' fields
	to avoid having to specify the dependency type for every entry in
	'depends'.

	o Elements may now specify cross-junction dependencies as simple strings
	using the format '{junction-name}:{element-name}'.

	o Junction elements may now specify another junction as their target, using
	the `target` configuration option.

	o Source plugins may now request access access to previous during track and
	fetch by setting `BST_REQUIRES_PREVIOUS_SOURCES_TRACK` and/or
	`BST_REQUIRES_PREVIOUS_SOURCES_FETCH` attributes.

	o Add new `pip` source plugin for downloading python packages using pip,
	based on requirements files from previous sources.

	o Generate Docker images from built artifacts using
	`contrib/bst-docker-import` script.

	o Added Documentation on how to create out of source builds. This includes the
	new the `conf-root` variable to make the process easier. And there has been
	a bug fix to workspaces so they can be build in workspaces too.

	o Creating a build shell through the interactive mode or `bst shell --build`
	will now use the cached build tree if available locally. It is now easier to
	debug local build failures.

	o `bst shell --sysroot` now takes any directory that contains a sysroot,
	instead of just a specially-formatted build-root with a `root` and `scratch`
	subdirectory.

	o Due to the element `build tree` being cached in the respective artifact their
	size in some cases has significantly increased. In most cases the build trees
	are not utilised when building targets, as such by default bst 'pull' & 'build'
	will not fetch build trees from remotes. This behaviour can be overridden with
	the cli main option '--pull-buildtrees', or the user configuration cache group
	option 'pull-buildtrees = True'. The override will also add the build tree to
	already cached artifacts. When attempting to populate an artifactcache server
	with cached artifacts, only 'complete' elements can be pushed. If the element
	is expected to have a populated build tree then it must be cached before pushing.

	o `bst workspace open` now supports the creation of multiple elements and
	allows the user to set a default location for their creation. This has meant
	that the new CLI is no longer backwards compatible with buildstream 1.2.

	o Add sandbox API for command batching and use it for build, script, and
	compose elements.

	o BREAKING CHANGE: The `git` plugin does not create a local `.git`
	repository by default. If `git describe` is required to work, the
	plugin has now a tag tracking feature instead. This can be enabled
	by setting 'track-tags'.

	o Opening a workspace now creates a .bstproject.yaml file that allows buildstream
	commands to be run from a workspace that is not inside a project.

	o Specifying an element is now optional for some commands when buildstream is run
	from inside a workspace - the 'build', 'checkout', 'fetch', 'pull', 'push',
	'shell', 'show', 'source-checkout', 'track', 'workspace close' and 'workspace reset'
	commands are affected.

	o bst 'build' now has '--remote, -r' option, inline with bst 'push' & 'pull'.
	Providing a remote will limit build's pull/push remote actions to the given
	remote specifically, ignoring those defined via user or project configuration.

	o Artifacts can now be cached explicitly with an empty `build tree` when built.
	Element types without a build-root were already cached with an empty build tree
	directory, this can now be extended to all or successful artifacts to save on cache
	overheads. The cli main option '--cache-buildtrees' or the user configuration cache
	group option 'cache-buildtrees' can be set as 'always', 'auto' or 'never', with
	the default being 'auto'. Note, as the cache-key for the artifact is independent of
	the cached build tree input it will remain unaltered, however the availbility of the
	build tree content may differ.

	o BREAKING CHANGE: Symlinks are no longer resolved during staging and absolute
	symlinks are now preserved instead of being converted to relative symlinks.

	o BREAKING CHANGE: Overlap whitelists now require absolute paths. This allows
	use of variables such as %{prefix} and matches the documentation.


	=================
	buildstream 1.1.5
	=================

	o Add a `--tar` option to `bst checkout` which allows a tarball to be
	created from the artifact contents.

	o Fetching and tracking will consult mirrors defined in project config,
	and the preferred mirror to fetch from can be defined in the command
	line or user config.

	o Added new `remote` source plugin for downloading file blobs

	o Add support for the new include '(@)' directive in project.conf and .bst files


	=================
	buildstream 1.1.4
	=================

	o `bst workspace` commands and `bst track` will substitute their
	source elements when performing those operations, e.g. performing
	`bst track` on a filter element will track the sources on the
	element that it depends on (if it has sources).

	o Added new simple `make` element

	o Switch to Remote Execution CAS-based artifact cache on all platforms.

	Artifact servers need to be migrated.

	o BuildStream now requires python version >= 3.5

	o BuildStream will now automatically clean up old artifacts when it
	runs out of space. The exact behavior is configurable in the user's
	buildstream.conf.


	=================
	buildstream 1.1.3
	=================

	o Added new `bst init` command to initialize a new project.

	o Cross junction tracking is now disabled by default for projects
	which can support this by using project.refs ref-storage

	New options have been added to explicitly enable cross-junction
	tracking.

	o Failed jobs are now summarised at the end of a build.

	Use `--verbose` and `--no-verbose` to adjust the amount of detail given.

	o BuildElements' `configure-commands` are only run once for
	workspaces now, which allows for incremental builds.

	Appropriate API for plugins is also exposed through
	`Element.prepare`.

	o The `cmake` plugin now supports building with ninja with the
	newly added `generator` configuration option.

	o `bst workspace close` and `bst workspace reset` now support multiple
	elements. All elements can be specified using `--all`.

	o The elements whose cache keys had to be determined during the build
	are summarised at the end of the build.

	o Fixed versioning introspection to be dynamic, many users use
	a developer install mode so they can update with git, now the
	version information is always up to date in logs.

	This causes a minor API break: The --version output now only
	outputs the version.


	=================
	buildstream 1.1.2
	=================

	o New ref-storage option allows one to store source refs, such
	as git shas, in one central project.refs file instead of inline
	with the source declarations.

	o Deprecated `--track-save` optionality in `bst build`, this
	does not make sense to support now that we have project.refs.

	o Added the `sandbox` configuration option which can be used in
	`project.conf` and elements, to control the user ID and group ID
	used in build sandboxes.

	o Added new `deb` source implementation, for staging of downloaded
	deb package files.


	=================
	buildstream 1.1.1
	=================

	o New project configuration controlling how the sandbox behaves
	when `bst shell` is used; allowing projects to provide a more
	functional shell environment.

	o The `bst shell` command now has a `--mount` option allowing
	users to mount files and directories into the sandbox for
	testing purposes.

	o Log lines are now configurable with the new "message-format"
	user configuration, allowing one to express optional fields
	such as microsecond precision and wallclock time.

	o Newly added filter element

	o Git source plugin now allows disabling of submodule checkouts

	o In the same way we allow overriding element configurations
	by their 'kind' in project.conf, we now support the same
	for source plugin configurations.

	o Tar and zip sources now automatically recall an `etag`
	from the http headers, optimizing tracking of tarballs
	significantly (issue #62)


	=================
	buildstream 1.1.0
	=================

	o Multiple artifact caches are now supported in project and
	user configuration with a priority order (issue #85)
	o Add junction support for subprojects
	o Changes towards incremental builds in workspaces
	o `bst shell --build` now creates true build sandbox
	o Many bug fixes


	=================
	buildstream 1.0.0
	=================
	First stable release of BuildStream

	BuildStream 1.0.0 is all about API stability - for the past months we
	have been reviewing our various API surfaces, implementing strategies
	for revisioning of our interfaces and cleaning up. Long term stability
	is very important for build reproducibility over time, and this release
	is the first promise we are making on any API surfaces.

	Stable API surfaces include:
	o The command line interface
	o The YAML user configuration file format
	o The YAML project `.bst` file format
	o The core Python module imported by external plugins