Fix choice branch selection per errata 5.60

- Update unparser choice branch selection to match errata 5.60: prefer the first branch fully defaultable/OVC/optional instead of the first empty one.
- Make required arrays return `true` for `isRequiredStreamingUnparserEvent` for consistency with defaulting behavior.
- Add subset error when arrays with required elements have default values.
- Rename `canUnparseIfHidden` for clarity—it isn’t limited to hidden elements.
- Mark required but empty arrays as `PossiblyZeroArrayOccurrencesDetector` to suppress separators.
- Simplify `optDefaultBranch` logic: `canUnparseIfNoEvents` subsumes `isOpen`/`isEmpty`.
- Remove `MultipleChoiceBranches` warning—multiple valid defaults are allowed.
- In `pneResolver`, handle arrays using `possibleSelfPlusNextLexicalSiblingStreamingUnparserElements`.
- For complex elements, check both element and member requiredness for unparse eligibility.
- Update CLIDebugger test schema to cover suspension cases.
- Disallow hidden IVCs except in unselected choice branches; require IVCs in the infoset.
- Remove support for hidden IVCs in sequences and add tests confirming hidden IVCs aren’t selected by `optDefaultBranch`.

Discussion
Hidden IVCs are elements computed during parse but never represented in the
  infoset or needed during unparse. They are not meaningful for unparsing and
  should not be supported. So we remove !e.isRepresented from the ElementBase
  case. Unparsing should instead require IVC elements (minOccurs = maxOccurs = 1)
  to appear in the infoset, even though their values are ignored at unparse
  time. This behavior is more intuitive and is in line with IVC being MustExist in
  unparserInfosetElementDefaultingBehavior and preserves expected branch selection semantics for
  choice/dispatch constructs. Hidden IVCs remain permissible only within choice branches that
  are not selected during unparse. Note that we will never select a default branch with a Hidden
  IVC because it will always be canUnparseIfNoEvents=false

Deprecation/Compatibility
Per DFDL errata 5.60, the first branch that can be unparsed entirely without any infoset events (i.e. made up of elements that are dfdl:outputValueCalc, option branches, zero length arrays, or defaultable elements) in a choice is selected, rather than prioritizing the first empty branch. Also, required arrays are no longer treated as optional and hidden IVCs are only permissible in choices.

DAFFODIL-2324
18 files changed
tree: 5fa775d19169cd39715d2013ed8f5f8aab9cb3ab
  1. .github/
  2. containers/
  3. daffodil-cli/
  4. daffodil-codegen-c/
  5. daffodil-core/
  6. daffodil-macro-lib/
  7. daffodil-propgen/
  8. daffodil-schematron/
  9. daffodil-slf4j-logger/
  10. daffodil-tdml-junit/
  11. daffodil-tdml-lib/
  12. daffodil-tdml-processor/
  13. daffodil-test/
  14. daffodil-test-ibm1/
  15. daffodil-test-integration/
  16. project/
  17. scripts/
  18. test-stdLayout/
  19. tutorials/
  20. .asf.yaml
  21. .codecov.yml
  22. .gitattributes
  23. .gitignore
  24. .sbtopts
  25. .scalafmt.conf
  26. .sonar-project.properties
  27. BUILD.md
  28. build.sbt
  29. DEVELOP.md
  30. KEYS
  31. LICENSE
  32. NOTICE
  33. README.md
  34. VERSION
README.md

Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset. This infoset is commonly converted into XML or JSON to enable the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of serializing or “unparsing” data back to the original data format. The DFDL infoset can also be converted directly to/from the data structures carried by data processing frameworks so as to bypass any XML/JSON overheads.

For more information about Daffodil, see https://daffodil.apache.org/.

Build Requirements

  • Java 8 or higher
  • sbt 0.13.8 or higher
  • C compiler C99 or higher
  • Mini-XML Version 3.0 or higher

See BUILD.md for more details and DEVELOP.md for a developer guide.

Getting Started

sbt is the officially supported tool to build Daffodil. Below are some of the more commonly used commands for Daffodil development.

Compile

Compile source code:

sbt compile

Test

Check all unit tests pass:

sbt test

Check all integration tests pass:

sbt daffodil-test-integration/test

Format

Check format of source and sbt files:

sbt scalafmtCheckAll scalafmtSbtCheck

Reformat source and sbt files if necessary:

sbt scalafmtAll scalafmtSbt

Build

Build the Daffodil command line interface (Linux and Windows shell scripts in daffodil-cli/target/universal/stage/bin/; see the Command Line Interface documentation for details on their usage):

sbt daffodil-cli/stage

Publish the Daffodil jars to a Maven repository (for Java projects) or Ivy repository (for Scala or schema projects).

Maven (for Java or mvn):

sbt publishM2

Ivy (for Scala or sbt):

sbt publishLocal

Check Licenses

Run Apache RAT (license audit report in target/rat.txt and error if any unapproved licenses are found):

sbt ratCheck

Check Coverage

Run sbt-scoverage (report in target/scala-ver/scoverage-report/):

sbt clean coverage test daffodil-test-integration/test
sbt coverageAggregate

Getting Help

You can ask questions on the dev@daffodil.apache.org or users@daffodil.apache.org mailing lists. You can report bugs via the Daffodil JIRA.

License

Apache Daffodil is licensed under the Apache License, v2.0.