Remove warnings in serialized parser

Currently, when a DataProcessor is serialized using the save() function,
warnings generated during compilation are also serialized. But this
behavior is not needed, and can sometimes lead to excessive diagnostics
when reloading.

To avoid this, this moves the diagnostics member from the
SchemaSetRuntimeData to the DataProcessor, and clears that member before
serialization.

This also discovered an issue where the TDML runner was not correctly
checking for errors vs warnings and would accept messages of the wrong
type. This fixes that, which requires changes to some expected
errors/warnings.

That change also uncovered a bug where we only output a warning instead
of an error if the main schema is not a DFDL schema. This was caused by
an incorrect comparison in the "allSchemaFiles" logic, which also needed
to be a LV to handle the SDE correctly.

Also, the DaffodilTDMLDFDLProcessor has a getDiagnostics() function to
return compile time diagnostics. However, with this change, if a
processor is saved and reloaded by the TDML runner (which happens for
most tests) those diagnostics are lost. Because getDiagnostics may or
may not actually have diagnostics, this is an unreliable API and so is
removed. Instead, diagnostics should only be taken from the
CompileResult. For consistency, the isError function is also removed and
should only be determined based on the CompileResult.

Some asserts were added to make sure the CompileResult makes sense, and
dead code removed where it was impossible to have a bad processor.

DAFFODIL-2803
19 files changed
tree: a5084b55583f4ed6882f1387f0626273f219d5c1
  1. .github/
  2. containers/
  3. daffodil-cli/
  4. daffodil-codegen-c/
  5. daffodil-core/
  6. daffodil-io/
  7. daffodil-japi/
  8. daffodil-lib/
  9. daffodil-macro-lib/
  10. daffodil-propgen/
  11. daffodil-runtime1/
  12. daffodil-runtime1-layers/
  13. daffodil-runtime1-unparser/
  14. daffodil-sapi/
  15. daffodil-schematron/
  16. daffodil-slf4j-logger/
  17. daffodil-tdml-lib/
  18. daffodil-tdml-processor/
  19. daffodil-test/
  20. daffodil-test-ibm1/
  21. daffodil-test-integration/
  22. daffodil-udf/
  23. project/
  24. scripts/
  25. test-stdLayout/
  26. tutorials/
  27. .asf.yaml
  28. .codecov.yml
  29. .gitattributes
  30. .gitignore
  31. .sbtopts
  32. .scalafmt.conf
  33. .sonar-project.properties
  34. BUILD.md
  35. build.sbt
  36. DEVELOP.md
  37. KEYS
  38. LICENSE
  39. NOTICE
  40. README.md
README.md

Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset. This infoset is commonly converted into XML or JSON to enable the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of serializing or “unparsing” data back to the original data format. The DFDL infoset can also be converted directly to/from the data structures carried by data processing frameworks so as to bypass any XML/JSON overheads.

For more information about Daffodil, see https://daffodil.apache.org/.

Build Requirements

  • Java 8 or higher
  • sbt 0.13.8 or higher
  • C compiler C99 or higher
  • Mini-XML Version 3.0 or higher

See BUILD.md for more details and DEVELOP.md for a developer guide.

Getting Started

sbt is the officially supported tool to build Daffodil. Below are some of the more commonly used commands for Daffodil development.

Compile

Compile source code:

sbt compile

Tests

Run unit tests:

sbt test

Run slower integration tests:

sbt daffodil-test-integration/test

Command Line Interface

Build the command line interface (Linux and Windows shell scripts in daffodil-cli/target/universal/stage/bin/; see the Command Line Interface documentation for details on their usage):

sbt daffodil-cli/stage

License Check

Run Apache RAT (license audit report in target/rat.txt and error if any unapproved licenses are found):

sbt ratCheck

Test Coverage Report

Run sbt-scoverage (report in target/scala-ver/scoverage-report/):

sbt clean coverage test daffodil-test-integration/test
sbt coverageAggregate

Getting Help

You can ask questions on the dev@daffodil.apache.org or users@daffodil.apache.org mailing lists. You can report bugs via the Daffodil JIRA.

License

Apache Daffodil is licensed under the Apache License, v2.0.