| commit | 660188266aa171ac536d1182486fabf411dc18be | [log] [tgz] |
|---|---|---|
| author | Steve Lawrence <slawrence@apache.org> | Thu Jul 21 15:12:25 2022 -0400 |
| committer | Steve Lawrence <slawrence@apache.org> | Wed Jul 27 07:18:55 2022 -0400 |
| tree | 5ec64a141bad24fb5691e9f2c13823e439d57940 | |
| parent | c521ae323638a7e7a9099eae2c46f87d77d5d9d7 [diff] |
Improve release candidate container build cachability and reproducability The release candidate container setup script makes reusing cached image layers difficult because any change to the setup script (e.g. sbt/npm version) invalidates the previous cached layers. This means anytime a change is made to this script the entire container image must be rebuilt which takes a lot of time and wasted space. To avoid this, the container setup script is removed and all functionality is moved to the Dockerfile. This results in more layers, but makes it less likely to need to rebuild layers if there are changes. Commands that are more likely to change due to versions or contents (e.g. sbt/yarn install, release-candidate script) are put closer to the end so if they are changed, less needs to be rebuilt. More importantly, the installation of yarn and sbt do not provide a specific version, so the version in the container depends on the latest version available in the npm/sbt repositories when the container is built. To improve reproducability, they are changed to install a pinned version (the latest versions as of this change). Additional minor updates to the container: - Switch from fedora-minimal to fedora. The size difference is minimal compared to the final size of the container, and it replaces microdnf with dnf, which has more capabilities to make the Dockerfile cleaner, (e.g. quieter build, repo/repofrompath options). - Add the "--group-add keep-groups" option to the podman run command needed for newer versions of podman. - Some files only need to be verified when building Daffodil and not VS Code, only mention the correct files to avoid confusion. - Update sbt-pgp plugin to latest version - Update wix_wine.sh to remove debug message that SBT makes look like errors. DAFFODIL-2713
Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset. This infoset is commonly converted into XML or JSON to enable the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of serializing or “unparsing” data back to the original data format. The DFDL infoset can also be converted directly to/from the data structures carried by data processing frameworks so as to bypass any XML/JSON overheads.
For more information about Daffodil, see https://daffodil.apache.org/.
See BUILD.md for more details.
SBT is the officially supported tool to build Daffodil. Below are some of the more commonly used commands for Daffodil development.
Compile source code:
sbt compile
Run unit tests:
sbt test
Run command line interface tests:
sbt IntegrationTest/test
Build the command line interface (Linux and Windows shell scripts in daffodil-cli/target/universal/stage/bin/; see the Command Line Interface documentation for details on their usage):
sbt daffodil-cli/stage
Run Apache RAT (license audit report in target/rat.txt and error if any unapproved licenses are found):
sbt ratCheck
Run sbt-scoverage (report in target/scala-ver/scoverage-report/):
sbt clean coverage test IntegrationTest/test sbt coverageAggregate
You can ask questions on the dev@daffodil.apache.org or users@daffodil.apache.org mailing lists. You can report bugs via the Daffodil JIRA.
Apache Daffodil is licensed under the Apache License, v2.0.