Skip to content

Metadata and Infoset data walking refactored #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mbeckerle wants to merge 8 commits into from
Closed

Metadata and Infoset data walking refactored #55

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
355592f
Refactoring to expose clean metadata and infoset walkers
mbeckerle Oct 11, 2023
2f8a41f
Verion 3.7.0-SNAPSHOT
mbeckerle Oct 29, 2023
59c544f
Removed use of DIComplex, DISimple from SAPI tests
mbeckerle Oct 29, 2023
189e63c
Silence a warning
mbeckerle Oct 29, 2023
8d1a5dc
Try to turn off doc generation
mbeckerle Oct 29, 2023
79bddbb
ARRRRRRG
mbeckerle Oct 29, 2023
3aa93c6
OptionTypes don't work in Java
mbeckerle Oct 29, 2023
5b56900
OptionTypes don't work in Java
mbeckerle Oct 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 36 additions & 28 deletions build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ lazy val genVersion = taskKey[Seq[File]]("Generate VERSION file")

lazy val daffodil = project
.in(file("."))
.enablePlugins(JavaUnidocPlugin, ScalaUnidocPlugin)
// .enablePlugins(JavaUnidocPlugin, ScalaUnidocPlugin)
.aggregate(
cli,
codeGenC,
Expand All @@ -52,7 +52,13 @@ lazy val daffodil = project
tutorials,
udf,
)
.settings(commonSettings, nopublish, ratSettings, unidocSettings, genCExamplesSettings)
.settings(
commonSettings,
nopublish,
ratSettings,
// unidocSettings,
genCExamplesSettings,
)

lazy val macroLib = Project("daffodil-macro-lib", file("daffodil-macro-lib"))
.settings(commonSettings, nopublish)
Expand Down Expand Up @@ -219,7 +225,7 @@ val minSupportedJavaVersion: String =

lazy val commonSettings = Seq(
organization := "org.apache.daffodil",
version := "3.6.0",
version := "3.7.0-SNAPSHOT",
scalaVersion := "2.12.18",
crossScalaVersions := Seq("2.12.18"),
scalacOptions ++= buildScalacOptions(scalaVersion.value),
Expand All @@ -241,7 +247,7 @@ lazy val commonSettings = Seq(
),
licenses := Seq(License.Apache2),
homepage := Some(url("https://daffodil.apache.org")),
releaseNotesURL := Some(url(s"https://daffodil.apache.org/releases/${version.value}/")),
// releaseNotesURL := Some(url(s"https://daffodil.apache.org/releases/${version.value}/")),
unmanagedBase := baseDirectory.value / "lib" / "jars",
sourceManaged := baseDirectory.value / "src_managed",
resourceManaged := baseDirectory.value / "resource_managed",
Expand Down Expand Up @@ -412,30 +418,30 @@ lazy val ratSettings = Seq(
ratFailBinaries := true,
)

lazy val unidocSettings = Seq(
ScalaUnidoc / unidoc / unidocProjectFilter := inProjects(sapi, udf),
ScalaUnidoc / unidoc / scalacOptions := Seq(
"-doc-title",
"Apache Daffodil " + version.value + " Scala API",
"-doc-root-content",
(sapi / baseDirectory).value + "/root-doc.txt",
),
JavaUnidoc / unidoc / unidocProjectFilter := inProjects(japi, udf),
JavaUnidoc / unidoc / javacOptions := Seq(
"-windowtitle",
"Apache Daffodil " + version.value + " Java API",
"-doctitle",
"<h1>Apache Daffodil " + version.value + " Java API</h1>",
"-notimestamp",
"-quiet",
),
JavaUnidoc / unidoc / unidocAllSources := (JavaUnidoc / unidoc / unidocAllSources).value.map {
sources =>
sources.filterNot { source =>
source.toString.contains("$") || source.toString.contains("packageprivate")
}
},
)
//lazy val unidocSettings = Seq(
// ScalaUnidoc / unidoc / unidocProjectFilter := inProjects(sapi, udf),
// ScalaUnidoc / unidoc / scalacOptions := Seq(
// "-doc-title",
// "Apache Daffodil " + version.value + " Scala API",
// "-doc-root-content",
// (sapi / baseDirectory).value + "/root-doc.txt",
// ),
// JavaUnidoc / unidoc / unidocProjectFilter := inProjects(japi, udf),
// JavaUnidoc / unidoc / javacOptions := Seq(
// "-windowtitle",
// "Apache Daffodil " + version.value + " Java API",
// "-doctitle",
// "<h1>Apache Daffodil " + version.value + " Java API</h1>",
// "-notimestamp",
// "-quiet",
// ),
// JavaUnidoc / unidoc / unidocAllSources := (JavaUnidoc / unidoc / unidocAllSources).value.map {
// sources =>
// sources.filterNot { source =>
// source.toString.contains("$") || source.toString.contains("packageprivate")
// }
// },
//)

lazy val genCExamplesSettings = Seq(
Compile / genCExamples := {
Expand Down Expand Up @@ -474,3 +480,5 @@ lazy val genCExamplesSettings = Seq(
res
},
)

Compile / packageDoc / publishArtifact := false
2 changes: 1 addition & 1 deletion daffodil-cli/src/main/scala/org/apache/daffodil/cli/InfosetTypes.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ import org.apache.daffodil.runtime1.api.DFDL.DataProcessor
import org.apache.daffodil.runtime1.api.DFDL.ParseResult
import org.apache.daffodil.runtime1.api.DFDL.UnparseResult
import org.apache.daffodil.runtime1.infoset.InfosetInputter
import org.apache.daffodil.runtime1.infoset.InfosetOutputter
import org.apache.daffodil.runtime1.infoset.JDOMInfosetInputter
import org.apache.daffodil.runtime1.infoset.JDOMInfosetOutputter
import org.apache.daffodil.runtime1.infoset.JsonInfosetInputter
Expand All @@ -60,6 +59,7 @@ import com.siemens.ct.exi.grammars.GrammarFactory
import com.siemens.ct.exi.main.api.sax.EXIResult
import com.siemens.ct.exi.main.api.sax.EXISource
import org.apache.commons.io.IOUtils
import org.apache.daffodil.runtime1.api.InfosetOutputter
import org.xml.sax.Attributes
import org.xml.sax.ContentHandler
import org.xml.sax.InputSource
Expand Down
2 changes: 1 addition & 1 deletion ...dil-core/src/test/scala/org/apache/daffodil/core/dpath/TestDFDLExpressionEvaluation.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import org.junit.Test; object INoWarn2 { ImplicitsSuppressUnusedImportWarning()
import org.apache.daffodil.core.infoset.TestInfoset
import org.apache.daffodil.core.util.TestUtils
import org.apache.daffodil.io.InputSourceDataInputStream
import org.apache.daffodil.runtime1.infoset.InfosetDocument
import org.apache.daffodil.runtime1.api.InfosetDocument
import org.apache.daffodil.runtime1.infoset.NullInfosetOutputter
import org.apache.daffodil.runtime1.processors.DataProcessor
import org.apache.daffodil.runtime1.processors.parsers.PState
Expand Down
5 changes: 3 additions & 2 deletions daffodil-core/src/test/scala/org/apache/daffodil/core/general/TestRuntimeProperties.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ import org.apache.daffodil.core.util.TestUtils
import org.apache.daffodil.io.InputSourceDataInputStream
import org.apache.daffodil.lib.Implicits.intercept
import org.apache.daffodil.lib.util.SchemaUtils
import org.apache.daffodil.runtime1.api.InfosetSimpleElement
import org.apache.daffodil.runtime1.dpath.NodeInfo
import org.apache.daffodil.runtime1.infoset.DISimple
import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetInputter
import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetOutputter

import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
Expand All @@ -37,7 +37,8 @@ import org.junit.Test
*/
class RedactingScalaXMLInfosetOutputter extends ScalaXMLInfosetOutputter {

override def startSimple(diSimple: DISimple): Unit = {
override def startSimple(se: InfosetSimpleElement): Unit = {
val diSimple = se.asInstanceOf[DISimple]
super.startSimple(diSimple)

val runtimeProperties = diSimple.erd.runtimeProperties
Expand Down
50 changes: 27 additions & 23 deletions daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfoset.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,9 @@ class TestInfoset1 {
val list_erd = infoset.erd
assertEquals(list_erd, infoset.runtimeData)
val Seq(w_erd) = list_erd.childERDs
val wItem = infoset.getChild(w_erd, tunable).asInstanceOf[InfosetSimpleElement]
val wItem = infoset.getChild(w_erd, tunable).asSimple
assertEquals(infoset, wItem.parent)
assertEquals(4, wItem.dataValue.getAnyRef)
assertEquals(4, wItem.getAnyRef)

}

Expand All @@ -160,10 +160,10 @@ class TestInfoset1 {
val list_erd = infoset.erd
val Seq(w_erd, _, _, c_erd) = list_erd.childERDs
assertEquals(list_erd, infoset.runtimeData)
val wItem = infoset.asComplex.getChild(w_erd, tunable).asInstanceOf[InfosetSimpleElement]
assertEquals(4, wItem.dataValue.getAnyRef)
val cItem = infoset.asComplex.getChild(c_erd, tunable).asInstanceOf[InfosetSimpleElement]
assertEquals(7, cItem.dataValue.getAnyRef)
val wItem = infoset.asComplex.getChild(w_erd, tunable).asSimple
assertEquals(4, wItem.getAnyRef)
val cItem = infoset.asComplex.getChild(c_erd, tunable).asSimple
assertEquals(7, cItem.getAnyRef)
assertEquals(infoset, cItem.parent)
}

Expand All @@ -185,13 +185,13 @@ class TestInfoset1 {
infoset.getChildArray(w_erd, tunable) match {
case arr: DIArray => {
assertEquals(2, arr.length)
var a = arr(1).asInstanceOf[InfosetSimpleElement]
var a = arr.getOccurrence(1).asSimple
Copy link
Owner Author

@mbeckerle mbeckerle Oct 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revisit. Why doesn't apply(1) work here?

assertEquals(w_erd, a.runtimeData)

assertEquals(4, a.dataValue.getAnyRef)
assertEquals(4, a.getAnyRef)
assertEquals(infoset, a.parent)
a = arr(2).asInstanceOf[InfosetSimpleElement] // 1-based
assertEquals(5, a.dataValue.getAnyRef)
a = arr.getOccurrence(2).asSimple // 1-based
assertEquals(5, a.getAnyRef)
assertEquals(infoset, a.parent)
}
}
Expand Down Expand Up @@ -221,18 +221,19 @@ class TestInfoset1 {
val Seq(w_erd, _, _, c_erd) = list_erd.childERDs
infoset.getChildArray(w_erd, tunable) match {
case arr: DIArray => {
var a = arr(1).asInstanceOf[InfosetSimpleElement]
var a = arr.getOccurrence(1).asSimple
assertEquals(2, arr.length)
assertEquals(w_erd, a.runtimeData)
assertEquals(4, a.dataValue.getAnyRef)
assertEquals(4, a.getAnyRef)
assertEquals(infoset, a.parent)
a = arr(2).asInstanceOf[InfosetSimpleElement] // 1-based
assertEquals(5, a.dataValue.getAnyRef)
a = arr.getOccurrence(2).asSimple
assertEquals(5, a.getAnyRef)
assertEquals(infoset, a.parent)
}
}
infoset.getChild(c_erd, tunable) match {
case s: DISimple => assertEquals(7, s.dataValue.getAnyRef)
case s: DISimple => assertEquals(7, s.getAnyRef)
case _ => fail("children should be DISimple")
}
}

Expand Down Expand Up @@ -261,7 +262,7 @@ class TestInfoset1 {
xchild match {
case arr: DIArray => {
assertEquals(1, arr.length)
val xa = arr(1)
val xa = arr.getOccurrence(1)
assertEquals(x_erd, xa.runtimeData)
assertTrue(xa.isNilled)
}
Expand Down Expand Up @@ -338,13 +339,14 @@ class TestInfoset1 {
infoset.getChildArray(x_erd, tunable) match {
case arr: DIArray => {
assertEquals(2, arr.length)
var xa = arr(1).asInstanceOf[InfosetComplexElement]
var xa = arr.getOccurrence(1).asComplex
assertEquals(x_erd, xa.runtimeData)
assertTrue(xa.isNilled)
xa = arr(2).asInstanceOf[InfosetComplexElement] // 1-based
xa = arr.getOccurrence(2).asComplex // 1-based
val c = xa.getChild(c_erd, tunable)
c match {
case c: DISimple => assertEquals(7, c.dataValue.getAnyRef)
case c: DISimple => assertEquals(7, c.getAnyRef)
case _ => fail("should be DISimple")
}
}
}
Expand Down Expand Up @@ -390,16 +392,18 @@ class TestInfoset1 {
infoset.getChildArray(x_erd, tunable) match {
case arr: DIArray => {
assertEquals(2, arr.length)
var xa = arr(1).asInstanceOf[InfosetComplexElement]
var xa = arr.getOccurrence(1).asComplex
assertEquals(x_erd, xa.runtimeData)
val c = xa.getChild(c_erd, tunable)
c match {
case c: DISimple => assertEquals(7, c.dataValue.getAnyRef)
case c: DISimple => assertEquals(7, c.getAnyRef)
case _ => fail("should be DISimple")
}
xa = arr(2).asInstanceOf[InfosetComplexElement]
xa = arr.getOccurrence(2).asComplex
val b = xa.getChild(b_erd, tunable)
b match {
case c: DISimple => assertEquals(8, c.dataValue.getAnyRef)
case c: DISimple => assertEquals(8, c.getAnyRef)
case _ => fail("should be DISimple")
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion daffodil-core/src/test/scala/org/apache/daffodil/core/util/TestUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,13 @@ import org.apache.daffodil.lib.xml._
import org.apache.daffodil.runtime1.api.DFDL
import org.apache.daffodil.runtime1.debugger._
import org.apache.daffodil.runtime1.infoset.InfosetInputter
import org.apache.daffodil.runtime1.infoset.InfosetOutputter
import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetInputter
import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetOutputter
import org.apache.daffodil.runtime1.processors.DataProcessor
import org.apache.daffodil.runtime1.processors.VariableMap

import org.apache.commons.io.output.NullOutputStream
import org.apache.daffodil.runtime1.api.InfosetOutputter
import org.junit.Assert.assertEquals

object INoWarnU2 { ImplicitsSuppressUnusedImportWarning() }
Expand Down
12 changes: 6 additions & 6 deletions daffodil-japi/build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@
* limitations under the License.
*/

enablePlugins(GenJavadocPlugin)
enablePlugins(PublishJavadocPlugin)
// enablePlugins(GenJavadocPlugin)
Copy link
Owner Author

@mbeckerle mbeckerle Oct 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was me trying desperately to turn off sbt doc processing on publishLocal. Failed.
It cannot be shut off.

// enablePlugins(PublishJavadocPlugin)

// Scala Steward may try to update this version to include the Scala version,
// for example 0.18_2.12.15. This is incorrect because the unidoc plugin uses
// crossVersion to figure out the Scala version. This should be set to just the
// version of the genjavadoc plugin, without the Scala version.
unidocGenjavadocVersion := "0.18"
// unidocGenjavadocVersion := "0.18"

Genjavadoc / sources := (Genjavadoc / sources).value.filterNot { source =>
source.toString.contains("$") || source.toString.contains("packageprivate")
}
//Genjavadoc / sources := (Genjavadoc / sources).value.filterNot { source =>
// source.toString.contains("$") || source.toString.contains("packageprivate")
// }
2 changes: 1 addition & 1 deletion ...odil/japi/infoset/XMLTextEscapeStyle.java → ...dil/japi/infoset/XMLTextEscapeStyle1.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
/**
* XMLTextEscapeStyles for determining whether to wrap info in CDATA tags
*/
public enum XMLTextEscapeStyle {
public enum XMLTextEscapeStyle1 {
Copy link
Owner Author

@mbeckerle mbeckerle Oct 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temporarily eliminated much of the SAPI/JAPI abstractions that were making refactoring too hard.

/**
* Special characters (quotation mark, ampersand, less-than, greater-than) in the
* text of xs:string elements are escaped, while non-special characters are written
Expand Down
Loading