Approval for XQJ 1.0
Labels: XQJ
Labels: XQJ
Labels: XQJ
...During the bindSequence() call, the complete xqs1 sequence is consumed. Subsequently we can safely close the xqe1 expression, freeing up any runtime resources it held. On the other hand, consuming the complete sequence during bindSequence() implies that the XQJ implementation has to buffer the data one way or the other for subsequent query evaluations. All this works perfectly fine as long as we're handling relative small XML instances. But as the data is buffered, it breaks all opportunities for the underlying XQuery processor to take advantage of its streaming capabilities.If you know that the data bound to the external variable will be used for only a single XQuery execution, is there then a way to inform the XQJ/XQuery implementation of possible optimization opportunities, and use its streaming capabilities?The default binding mode in XQJ is 'immediate', which means the value bound to an external variable is consumed during the bindXXX() method. In addition, an application has the ability to set the binding mode to 'deferred'. With deferred binding mode, the application gives a hint to the XQJ-implementation and underlying XQuery processor, to take advantage of its streaming capabilities. In deferred binding mode, bindings are only active for a single execution cycle. The application is required to explicitly re-bind values to every external variable before each execution.You can change the binding mode through the XQStaticContext interface, as shown in the next example. Refer to Part VI in this series for more information on how to manipulate the static context.
XQExpression xqe1;
XQSequence xqs1;
xqe1 = xqc.createExpression();
xqs1 = xqe1.executeQuery("doc('orders.xml')//order");
XQExpression xqe2;
xqe2 = xqc.createExpression();
xqe2.bindSequence(xqs1);
xqe1.close();
XQSequence xqs2;
xqs2 = xqe2.executeQuery(
"declare variable $orders as element(*,xs:untyped) external; " +
"for $order in $orders " +
"where $order/@status = 'closed' " +
"return " +
" <closed_order id = '{$order/@id}'>{ " +
" $order/* " +
" </closed_order>";
xqs2.writeSequence(System.out, null);
xqe2.close();
...
...In deferred mode the application cannot assume that the bound value will be consumed during the invocation of the bindXXX() method. The XQJ-implementation is free to read the bound value either at bind time or during the subsequent evaluation and processing of the query results. This has some consequences on when resources can be cleaned up. If we consider the first example again, it will not work properly in deferred binding mode. Note that xqe1 was closed right after calling bindSequence(). The example needs to be modified as follows,
XQStaticContext xqsc = xqc.getStaticContext();
// change the binding mode
xqsc.setBindingMode(XQConstants.BINDING_MODE_DEFERRED);
// make the changes effective
xqc.setStaticContext(xqsc);
...
...This example shows how to build a pipeline of xqueries. But deferred binding mode applies also to the other bindXXX() methods. In the next example we show how to bind a StreamSource to the context item. As binding mode is deferred, the implementation can handle the query in streaming mode and as such process huge XML documents that don't fit in available memory.
XQExpression xqe1;
XQSequence xqs1;
xqe1 = xqc.createExpression();
xqs1 = xqe1.executeQuery("doc('orders.xml')//orders");
XQExpression xqe2 = xqc.createExpression();
xqe2.bindSequence(xqs1);
XQSequence xqs2 = xqe2.executeQuery(
"declare variable $orders as element(*,xs:untyped) external; " +
"for $order in $orders " +
"where $order/@status = 'closed' " +
"return " +
" <closed_order id = '{$order/@id}'>{ " +
" $order/* " +
" </closed_order>";
xqs2.writeSequence(System.out, null);
xqe2.close();
xqe1.close();
...
...To conclude, using deferred binding mode requires a little more care than immediate. But the potential improvements when querying large XML documents is enormous. Of course, the API needs to provide the necessary functionality, but the heavy lifting is performed in the underlying XQuery processor. Especially with DataDirect XQuery, where deferred binding mode allows you to both take advantage of XML document projection and its XML streaming capabilities. This allows to query XML documents in the hundreds of megabytes, even in the gigabytes!
XQStaticContext xqsc = xqc.getStaticContext();
// change the binding mode
xqsc.setBindingMode(XQConstants.BINDING_MODE_DEFERRED);
// make the changes effective
xqc.setStaticContext(xqsc);
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqe.bindDocument(
XQConstants.CONTEXT_ITEM,
new StreamSource("large_orders_document.xml"));
xqs = xqe.executeExpression("/orders/order")
...
Labels: XQJ
Labels: XQJ
...In this example, both queries in the pipeline are executed in the context of a single XQConnection object. But this is not required; it is perfectly possible to have two different connection objects, possibly from different XQJ implementations.The application writer shouldn’t be concerned how the query result from the first is passed to the second xquery. Whether for example a DOM, SAX, StAX, serialized XML or some other (proprietary) mechanism is used, is an implementation detail. The application can assume that the most appropriate mechanism is used.Let’s now look how an XQuery can be integrated in a pipeline with other XML processors. We'll work out two scenarios, the result of an xquery is passed to an XSLT transformation, and next vice versa, an xquery processing the results of an XSLT transformation. On the Java platform, XSLT transformations are processed through the JAXP api. JAXP makes use of the Source and Result interfaces to query XML documents and handle the results of transformations. As XQJ has built-in support for these interfaces, it is possible to integrate both JAXP and XQJ in a pipeline.The next example invokes an XSLT transformation, and the results are passed as input to an XQuery. Under the covers, SAX events will be. It is favorable to use SAX from a performance and scalability perspective, rather than when a serialized XML stream would be passed, not to talk about the use of a DOM tree.First an XMLFilter for the XLST transformation is created. And a SAXSource, the input for the XSLT Transformation.
XQExpression xqe1 = xqc.createExpression();
XQSequence xqs1;
xqs1 = xqe1.executeQuery("doc('orders.xml')//orders");
XQExpression xqe2 = xqc.createExpression
xqe2.bindSequence(xqs1);
XQSequence xqs2 = xqe2.executeQuery(
"declare variable $orders as " +
" element(*,xs:untyped) external; " +
"for $order in $orders " +
"where $order/@status = 'closed' " +
"return " +
" <closed_order id = '{$order/@id}'>{ " +
" $order/* " +
" </closed_order>";
xqs2.writeSequence(System.out, null);
xqe2.close();
xqe1.close();
...
...As you notice in this example, the pipeline is activated starting from the end. Under the covers, the XQJ implementation will invoke the parse() method on the SAXSource bound to the external variable. Which will start invoking SAX callbacks to the XSLT transformation and this will on its turn perform callbacks to the XQJ implementation. These callbacks represent the result of the XSLT transformation, allowing the xquery to yield results back to the application. Let's now have a closer look at a pipeline where the results of an xquery are passed on to an XSLT transformation.First we present a utility class, XQJFilter, an XMLFiler based on an XQJ XQPreparedExpression object.
// Build an XMLFilter for the XSLT transformation
SAXTransformerFactory stf;
stf = (SAXTransformerFactory) SAXTransformerFactory.newInstance();
XMLFilter stage1;
stage1 = stf.newXMLFilter(new StreamSource("stage1.xsl"));
// Create a SAX source, the input for the XSLT transformation
SAXSource saxSource;
saxSource = new SAXSource(stage1,new InputSource("input.xml"));
// Create the XQuery expression
XQPreparedExpression stage2;
stage2 = xqc.prepareExpression(new FileInputStream("stage2.xquery"));
// Bind the input document as a Source object to stage2
stage2.bindDocument(new QName("var"),saxSource);
// Execute the query (stage2)
XQSequence result = stage2.executeQuery();
// Process query results
result.writeSequenceToResult(
new StreamResult(System.out));
...
public class XQJFilter extends XMLFilterImpl {
XQPreparedExpression _expression;
public XQJFilter(XQPreparedExpression expression) {
_expression = expression;
}
public void parse(InputSource source) throws SAXException {
try {
XQSequence xqs = _expression.executeQuery();
Result result = new SAXResult(this.getContentHandler());
xqs.writeSequenceToResult(result);
xqs.close();
} catch (XQException e) {
throw new SAXException(e);
}
}
}Next we’ll use this XQJFilter implementation to build the pipeline,...As with the previous example, also here the pipeline is activated through the end, by calling the parse method on the 2nd stage.In our next post we discuss deferred binding mode, working with external variables, this is essential functionality to handle huge XML documents.
// Create an XQuery expression
XQPreparedExpression xqp;
xqp = xqc.prepareExpression(new FileInputStream ("query.xq"));
// Create the XQJFilter, the first stage in our pipeline
XQJFilter stage1 = new XQJFilter(xqp);
// Create an XMLFilter for the XSLT transformation, the 2nd stage
SAXTransformerFactory stf;
stf = (SAXTransformerFactory) SAXTransformerFactory.newInstance();
XMLFilter stage2 = stf.newXMLFilter(new StreamSource("stage2.xsl"));
stage2.setParent(stage1);
// Make sure to capture the SAX events as result of the pipeline
stage2.setContentHandler(...);
// Activate the pipeline
stage2.parse("");
...
Labels: XQJ
...Remember that every XQConnection is an XQDataFactory, in the example we've used our XQConnection xqc, to create these XQItemType instances. However, the XQItemType objects are completely independent of the connection.Where the above example shows how to create XQItemType objects representing one of the built-in atomic XML Schema types, there is a second flavor of createAtomicType() for user-defined atomic types. Assume a hatsize user-defined atomic type derived from xs:integer in the http://www/hatsize.com schema,
XQItemType xsinteger = xqc.createAtomicType(
XQItemType.XQBASETYPE_INTEGER);
XQItemType xsstring = xqc.createAtomicType(
XQItemType.XQBASETYPE_STRING);
XQItemType xsdecimal = xqc.createAtomicType(
XQItemType.XQBASETYPE_DECIMAL);
...
...Beside atomic types, also element types are frequently used. In the next example we create an XQItemType representing element(person),
XQItemType hatsize;
hatsize = xqc.createAtomicType(
XQItemType.XQBASETYPE_INTEGER,
new QName("http://www.hatsizes.com", "hatsize"),
new URI("http://www.hatsizes.com"));
...
...The first argument to createElementType() is a QName. Where in the example a person element in no namespace is created, the next example creates an element type person in the namespace http://www.foo.com. The second argument can be any of the predefined types, beside xs:anyType also xs:untyped is frequently used,
XQItemType type;
type = xqc.createElementType(
new QName("person"),
XQItemType.XQBASETYPE_ANYTYPE);
...
...The first argument can also be null, which is assumed to be the wild card, the following code snippet shows the creation of element(*, xs:untyped),
XQItemType type;
type = xqc.createElementType(
new QName("person","http://www.example.com"),
XQItemType.XQBASETYPE_UNTYPED);
...
...What about document-node() types? In the next example we create two XQItemType instances, a first representing any document and a second representing a well-formed untyped document,
XQItemType type;
type = xqc.createElementType(
null,
XQItemType.XQBASETYPE_UNTYPED);
...
...In addition to XQItemTypes, also XQSequenceType objects can be created.
XQItemType type1;
XQItemType type2;
type1 = xqc.createDocumentType();
type2 = xqc.createDocumentElementType(
xqc.createElementType(
null,
XQItemType.XQBASETYPE_UNTYPED));
...
...Uisng typesSo far so good, but why would one need to create all these types?Assume an XQSequence, iterating over the items, if the item is a node retrieve is through the DOM, and get atomic values as Strings. This can be accomplished using the instanceOf() method, passing in an XQItemType object
XQItemType itemType;
XQSequenceType sequenceType;
itemType = xqc.createAtomicType(
XQItemType.XQBASETYPE_STRING);
sequenceType = xqc.createSequenceType(
itemType,
XQSequenceType.OCC_ZERO_OR_MORE);
...
...Some XQuery implementations have support for the Static Typing Feature as defined in XQuery. This requires implementations to detect and report type errors during the static analyses phase.
XQItemType nodeType = xqc.createNodeType();
XQSequence xqs = ...
...
while (xqs.next()) {
if (xqs.instanceOf(nodeType)) {
org.w3c.dom.Node node = xqs.getNode();
...
} else {
String s = xqs.getAtomicValue();
...
}
}
...
...As last use case of XQItemType, remember some of the examples of the previous post in this series, Binding external variables.
XQItemType documentType;
documentType = xqc.createDocumentElementType(
xqc.createElementType(
null,
XQItemType.XQBASETYPE_UNTYPED));
XQStaticContext xqsc = xqc.getStaticContext();
xqsc.setContextItemStaticType(documentType);
...
XQPreparedExpression xqp;
xqp = xqc.prepareExpression("//address",
xqsc);
...
...Creating XDM instancesHaving discussed the ability to create XQItemType and XQSequenceType instances, XQDataFactory offers also the ability the create XQItem and XQSequence instances.There is basically nothing new under the sun. If you understand the way binding to an XQDynamicContext works, as discussed in our previous post, you almost know how XQItem instances are created. For every bindXXX() method defined on XQDynamicContext, there is corresponding createItemFromXXX() method.Let's show a simple example, binding a java.math.BigDecimal to an external variable $d,
XQItemType xsshort;
xsshort = xqc.createAtomicType(XQItemType.XQBASETYPE_SHORT);
XQPreparedExpression xqp;
xqp = xqc.prepareExpression(
"declare variable $v as xs:short external; " +
"$v + 1");
xqp.bindInt(new QName("v"), 22, xsshort);
...
...
XQExpression xqe = ...
xqe.bindObject(new QName("d"),new BigDecimal("174"), null);
XQItem xqi = xqc.createItemFromObject(new BigDecimal("174"), null);Note that the XQItem objects created through XQDataFactory are independent of any connection.Suppose you execute a query returning a single item, and subsequently close the connection but still require access to the XQItem. Closing the XQConnection will invalidate the XQItem object resulting from the query execution. As such XQDataFactory has an XQItem copy method. createItem() accepts a single XQItem argument, and returns a (deep) copy of the specified item.XQConnection xqc = ...Suppose you have an XML document which needs to be queried multiple times, but don’t want to go through the XML parsing overhead, each time it is queried. In the following example, two queries are executed and as such, books.xml will be parsed twice,
XQExpression xqe = xqc.createExpression();
XQSequence xqs;
xqs = xqe.executeQuery("(doc('book.xml')//paragraph)[1]");
xqs.next();
XQItem xqi = xqc.createItem(xqs.getItem());
xqc.close();
// although the connection is closed, xqi is still valid.
...Or suppose you receive a transient XML stream, for example in a servlet environment, and need to query the stream multiple times. Then one way or the other the data will need to be buffered in order to query it more than once.How can we make a) an XML document being parsed only once, b) in case the XML stream is transient, make it 'queryable' multiple times?Suppose two XQPreparedExpression objects, xqp1 and xqp2. The next example will create first an XQItem representing the XML document, as such it will be parsed only once. Second, it will be bound to 2 different XQPreparedExpression object,
XQExpression xqe = xqc.createExpression();
xqe.executeQuery("fn:doc('book.xml')//paragraph[contains(.,'XQuery')]");
xqe.executeQuery("fn:doc('book.xml')//paragraph[contains(.,'SQL')]");
...
...One of the disadvantages of such apporach, especially with large document, are the scalability aspects and memory consumption. For example, in case of DataDirect XQuery, the streaming capabilities will not be of much use as the complete XML document is instantiated in-memory. We'll come back to the topic of processing large input documents in a future post of the XQJ series.Finally, XQDynamicContext also allows to create XQSequence objects.
InputStream input = ...
XQItem doc = xqc.createItemFromDocument(input, null);
...
xqp1.bindItem(new QName("doc"), doc);
...
xqp2.bindItem(new QName("doc"), doc);
...
...Pipelines is the next topic we will discuss. How can one create a pipeline of xqueries, or pipelining an xquery with an XSLT transformations? Watch out for the next post.
// assume an ArrayList of BigDecimal objects
ArrayList list = ...
XQSequence s = xqc.createSequence(list.iterator());
...
Labels: XQJ
...The bindObject() method is defined in the XQDynamicContext interface. It provides a number of methods to bind values to external variables. As XQDynamicContext is both the base for XQExpression and XQPreparedExpression, as such both expression implementations support binding values to external variables.
XQPreparedExpression xqp;
XQSequence xqs;
xqp = xqc.prepareExpression(
"declare variable $id as xs:integer external; " +
"doc('orders.xml')//order[id=$id]");
xqp.bindObject(new QName("id"),new Integer(174), null);
xqs = xqp.executeQuery();
...
Java type | XQuery type | ||
|---|---|---|---|
java.lang.Integer | xs:int | ||
java.lang.BigInteger | xs:integer | ||
java.lang.BigDecimal | xs:decimal | ||
java.lang.String | xs:untypedAtomic | ||
org.w3c.dom.Document | untyped document node | ||
org.w3c.dom.Element | untyped element node | ||
... | ... | ||
...This example yields a sequence of 4 xs:boolean instances,
XQItemType xsinteger;
xsinteger = xqc.createAtomicType(XQItemType.XQBASETYPE_INTEGER);
XQPreparedExpression xqp;
XQSequence xqs;
xqp = xqc.prepareExpression(
"declare variable $v1 external; " +
"declare variable $v2 external; " +
"$v1 instance of xs:integer, "+
"$v1 instance of xs:int, "+
"$v2 instance of xs:integer, "+
"$v2 instance of xs:int");
xqp.bindObject(new QName("v1"),new Integer(174), null);
xqp.bindObject(new QName("v2"),new Integer(174), xsinteger);
...
true, true, true, falseA Java Integer is by default mapped to xs:int. xs:int extends by restriction xs:integer, as such the first two 'instance of' expressions evaluate to true. The second external variable is bound with an xs:integer instance as the application explicitly specifies to create such XDM instance. As such the last 'instance of' evaluates to false, as xs:integer is not extending xs:int.Note that various error conditions can occur during the binding process,
...In contrast, the following two bindAtomicValue() invocations will fail. The first because "abc" is not in the value spaces of xs:integer. The second one because no type has been specified as third parameter, unlike with bindObject(), bindAtomicValue() has no default mapping and a XQItemType must be specified as third argument.
xqp.bindAtomicValue(new QName("v1"), "123",
xqc.createAtomicType(XQItemType.XQBASETYPE_STRING));
xqp.bindAtomicValue(new QName("v2"), "123",
xqc.createAtomicType(XQItemType.XQBASETYPE_INTEGER));
xqp.bindAtomicValue(new QName("v3"), "123",
xqc.createAtomicType(XQItemType.XQBASETYPE_DOUBLE));
...
...Further XQDynamicContext also provides bindXXX() methods for each of the Java primitive types,
xqp.bindAtomicValue(new QName("e"), "abc",
xqc.createAtomicType(XQItemType.XQBASETYPE_INTEGER));
xqp.bindAtomicValue(new QName("e"), "123", null);
...
xqp.bindInt(new QName("v"), 123,
xqc.createAtomicType(XQItemType.XQBASETYPE_INTEGER));Further binding a DOM node is also possible, basically the is equivalent to bindObject, with the restriction that the argument must be a DOM node and as such the XDM instance is always a node, never an atomic value. Of course in addition to DOM, also the SAX and StAX APIs are supported through XQDynamicContext.Let’s read an XML document foo.xml through DOM, SAX and StAX and each time bind it to an external variable $v.The DOM version,...The StAX version,
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);
DocumentBuilder parser = factory.newDocumentBuilder();
Document domDocument = parser.parse("foo.xml");
xqp.bindNode(new QName("e"), domDocument,null);
...
...And the SAX version, for which we need to implement an XML Filter.
XMLInputFactory factory = XMLInputFactory.newInstance();
FileInputStream doc = new FileInputStream("foo.xml");
XMLStreamReader reader = factory.createXMLStreamReader(doc);
xqp.bindDocument(new QName("e"), reader, null);
...
...But of course, this is something you only want to use in specific scenarios. The simple use case of binding an xml file, can easily be accomplished in a single line. The XQJ implementation will make sure the XML file is parsed and queried.
XMLFilter xmlReader = new XMLFilterImpl() {
public void parse(String systemId) throws IOException, SAXException {
super.parse("foo.xml");
}
};
// the parent XML Reader is a SAX parser, this one will do the actual
// work of parsing the XML document
xmlReader.setParent(org.xml.sax.helpers.XMLReaderFactory.createXMLReader());
xqp.bindDocument (new QName("e"), xmlReader);
...
...An XQItem or a complete XQSequence can also be bound to an external variable. We’ll discuss this soon in this XQJ series, in a post on pipelining. Talking about pipelining, XQJ also supports the JAXP Source and Result interfaces, these too will be discussed.[1] In this series we have not yet introduced the createAtomicType() method defined on XQDataFactory. This will be handled in the next post. Anyway, for now it’s sufficient to know that it returns an XQItemType object representing the specified atomic type.
xqp.bindDocument(new QName("e"), new FileInputStream("foo.xml"));
...
Labels: XQJ
...XQJ defines constants for each of the item kinds representable in XQuery SequenceType syntax,
XQSequenceType xqtype = ...
XQItemType xqitype = xqtype.getItemType();
int itemKind = xqitype.getItemKind();
int schemaType = xqitype.getBaseType();
...
Sequence Type | XQJ definition | ||
|---|---|---|---|
QName | XQITEMKIND_ATOMIC | ||
element(...) | XQITEMKIND_ELEMENT | ||
attribute(...) | XQITEMKIND_ATTRIBUTE | ||
comment() | XQITEMKIND_COMMENT | ||
document-node() | XQITEMKIND_DOCUMENT | ||
document-node(element(...)) | XQITEMKIND_DOCUMENT_ELEMENT | ||
processing-instruction(...) | XQITEMKIND_PI | ||
text() | XQITEMKIND_TEXT | ||
item() | XQITEMKIND_ITEM | ||
node() | XQITEMKIND_NODE | ||
XML Schema type | XQJ definition | ||
|---|---|---|---|
xs:string | XQBASETYPE_STRING | ||
xs:integer | XQBASETYPE_INTEGER | ||
xs:untypedAtomic | XQBASETYPE_UNTYPEDATOMIC | ||
... | ... | ||
XQSequence xqs = ...OK, this can make your code rather complex and long. Sometimes it is needed, but most of the time a number of shortcuts can be taken. As explained in XQJ Part IV - Processing query results, you can use some of the more the general purpose methods. Suppose you need a DOM node in case the query returns a node, and the string value for all atomic values. The next simple example shows how to do this,
while (xqs.next()) {
XQItemType xqtype = xqs.getItemType();
if (xqtype.getItemKind() == XQItemType.XQITEMKIND_ATOMIC) {
// We have an atomic type
switch (xqtype.getBaseType()) {
case XQItemType.XQBASETYPE_STRING:
case XQItemType.XQBASETYPE_UNTYPEDATOMIC: {
String s = (String)xqs.getObject();
...
break;
}
case XQItemType.XQBASETYPE_INTEGER: {
long l = xqs.getLong();
...
break;
}
...
}
} else {
// We have a node, retrieve it as a DOM node
org.w3c.dom.Node node = xqs.getNode();
...
}
}
XQSequence xqs = ...That's it for the dynamic type of items. The next example shows how to retrieve the static type of a query (for the JDBC, ODBC and SQL users, this is somehow similar to "describe information")
while (xqs.next()) {
XQItemType xqtype = xqs.getItemType();
if (xqtype.getItemKind() == XQItemType.XQITEMKIND_ATOMIC) {
// We have an atomic type
String s = xqs.getAtomicValue();
...
} else {
// We have a node, retrieve it as a DOM node
org.w3c.dom.Node node = xqs.getNode();
...
}
}
...With DataDirect XQuery this examples outputs xs:integer to stdout.
XQPreparedExpression xqe = xqc.prepareExpression("1+2");
XQSequenceType xqtype = xqe.getStaticResultType();
System.out.println(xqtype.toString());
...
...Why would one care about all this? Let's have a quick look at a use case.
XQPreparedExpression xqe = xqc.prepareExpression(
"declare variable $i as xs:integer external; $i+1");
QName variables[] = xqe.getAllExternalVariables();
for (int i=0; i<variables.length; i++) {
XQSequenceType xqtype = xqe.getStaticVariableType(variables[i]);
System.out.println(variables[i] + ": " + xqtype.toString());
}
...
...All the information required to generate such XML schema definition, is available in the sequence type of each declared variable. And through XQJ this information becomes immediately accessible. We can write a piece of code translating the relevant item kinds and base types to an XML Schema definition as shown above, only a matter of a number of Java switch statements. But is there an easier way?XQJ defines toString() on XQItemType as implementation dependent. Well, more precisely, it is a requirement to return a human-readable string. In any case, with DataDirect XQuery the string representation is based on the XQuery sequence type syntax, where the QName prefixes are as follows,
<xs:element name="XXX">
<xs:complexType>
<xs:sequence>
<xs:element name="employeeName" type="xs:string"/>
<xs:element name="hiringDate" type="xs:date"/>
</xs:sequence>
</xs:complexType>
</xs:element>
...
Labels: XQJ
XQuery defines the Static Context as follows,
The static context of an expression is the information that is available during static analysis of the expression, prior to its evaluation.Refer to the XQuery spec for the complete list, but the static context includes for example information like,Most of the components in the static context can be initialized or augmented in the query prolog. In the next example, the boundary-space policy is explicitly specified,
declare boundary-space preserve;If a static context component is not initialized in the query prolog, an implementation default is used. Indeed, although that XQuery defines default values for each of the components in the static context, as outlined in Appendix C of the XQuery specification, implementations are free to override and/or extend these defaults.
<e> </e>
...First retrieve the implementation's default values for the static context components through an XQStaticContext object. XQStaticContext defines setter and getter methods for the various static context components.
// get a static context object with the implementation's defaults
XQStaticContext xqsc = xqc.getStaticContext();
// make sure boundary-space policy is preserve
xqsc.setBoundarySpacePolicy(XQConstants.BOUNDARY_SPACE_PRESERVE);
// make the changes effective
xqc.setStaticContext(xqsc);
...
...In the previous examples, the static context is updated at the connection level, and as such all subsequent created XQExpression and XQPreparedExpression object are affected. This is great if you want all your XQuery expressions to be based on the same defaults in the static context. But what if the default values need to be different for some XQExpression and XQPreparedExpression objects? The application has also the ability to specify an XQStaticContext during the creation of XQ(Prepared)Expression objects.
// the boundary-space for this first query is implementation defined,
// i.e. depends on the implementation's defaults
XQPreparedExpression xqp1 = xqc.prepareExpression("<e> </e>");
// set the boundary-space policy to preserve
XQStaticContext xqsc = xqc.getStaticContext();
xqsc.setBoundarySpacePolicy(XQConstants.BOUNDARY_SPACE_PRESERVE);
xqc.setStaticContext(xqsc);
// the boundary-space policy for this second query *is* preserve
XQPreparedExpression xqp2 = xqc.prepareExpression("<e> </e>");
...
...Again, such approach is useful if some static context components need to be changed for a specific expression, but want to keep the default values for most other expression being executed.Almost all static context components are accessible through XQStaticContext. Here is the list,
// change the boundary-space policy in the static context object
// but don't apply those change at the connection level
XQStaticContext xqsc = xqc.getStaticContext();
xqsc.setBoundarySpacePolicy(XQConstants.BOUNDARY_SPACE_PRESERVE);
// create a prepared expression using the modified static context
// other expressions subsequently created are not affected
XQPreparedExpression xqp1 = xqc.prepareExpression("<e> </e>", xqsc);
...
In addition, XQStaticContext includes a number of XQJ specific properties,
Labels: XQJ
Serialization defines a number of parameters which influence this process. The specification includes a detailed description for each of these parameters. We'll explain some through examples later on,
Note that XQuery Serialization is an Optional Feature in XQuery. However, XQJ is more strict and requires every implementation to support serialization. XQJ does does not require every parameter defined in the XQuery Serialization spec to be supported in its full extend, but at least a default value for each of the parameters needs to be documented and behave conformant to the spec.
For DataDirect XQuery all parameters are documented here.
Suppose you want to serialize your query results in a file, fairly simple as shown in the next example,
...
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"doc('orders.xml')/*/ORDERS[O_ORDERKEY = '39']");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
new Properties());
...
Note the second argument of writeSequence() is an empty Properties object. You can also specify null. Both an empty Properties object and null are implying that the XQJ driver uses the default values for each of the serialization parameters.
You might get something as follows (assume this to be one line),
<ORDERS><O_ORDERKEY>39</O_ORDERKEY><O_CUSTKEY>
8177</O_CUSTKEY><O_ORDERSTATUS>O</O_ORDERSTATUS>
<O_TOTALPRICE>307811.89</O_TOTALPRICE><O_ORDERDATE>
1996-09-20T00:00:00</O_ORDERDATE><O_ORDERPRIORITY>3-MEDIUM
</O_ORDERPRIORITY><O_CLERK>Clerk#000000659</O_CLERK>
<O_SHIPPRIORITY>0</O_SHIPPRIORITY><O_COMMENT>furiously
unusual pinto beans above the furiously ironic asymptot
</O_COMMENT> </ORDERS>
Not really readable, some indentation would help. It's also good practice to add the XML declaration including an encoding. Suppose we want to encode the XML file as UTF-16,
...
Properties serializationProps = new java.util.Properties();
// make sure we output xml
serializationProps.setProperty("method", "xml");
// pretty printing
serializationProps.setProperty("indent", "yes");
// serialize as UTF-16
serializationProps.setProperty("encoding", "UTF-16");
// want an XML declaration
serializationProps.setProperty("omit-xml-declaration", "no");
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"doc('orders.xml')/*/ORDERS[O_ORDERKEY = '39']");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
serializationProps);
...
Much better what we get now,
<?xml version="1.0" encoding="UTF-16"?>
<ORDERS>
<O_ORDERKEY>39</O_ORDERKEY>
<O_CUSTKEY>8177</O_CUSTKEY>
<O_ORDERSTATUS>O</O_ORDERSTATUS>
<O_TOTALPRICE>307811.89</O_TOTALPRICE>
<O_ORDERDATE>1996-09-20T00:00:00</O_ORDERDATE>
<O_ORDERPRIORITY>3-MEDIUM</O_ORDERPRIORITY>
<O_CLERK>Clerk#000000659</O_CLERK>
<O_SHIPPRIORITY>0</O_SHIPPRIORITY>
<O_COMMENT>furiously unusual pinto
beans above the furiously ironic
asymptot</O_COMMENT>
</ORDERS>
Note that during serialization characters are escaped as needed for the specified encoding. Suppose a query returning a document with a registered trademark character, and the specified encoding is US-ASCII,
...
Properties serializationProps = new java.util.Properties();
serializationProps.setProperty("method", "xml");
serializationProps.setProperty("encoding", "ASCII");
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"<product>DataDirect XQuery®</product>");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
serializationProps);
...
And you'll get the following, note that the ® character is serialized as a character reference because it is not defined in the ASCII character set,
<product>DataDirect XQuery®</product>
In some use cases, the cdata-section-elements parameter is useful. Suppose you're serializing some XML elements including ampersand characters. By default the & characters will be escaped, using CDATA sections might be preferable to make the XML file more human readable.
...
Properties serializationProps = new java.util.Properties();
serializationProps.setProperty("method", "xml");
serializationProps.setProperty("cdata-section-elements", "product");
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"<product>DataDirect XQuery & XML Converters</product>");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
null);
...
Is serialized as follows,
<product><![CDATA[DataDirect XQuery & XML Converters]]></product>
Note that multiple elements can be specified through the cdata-section-elements parameter, separated by a semi-colon character. And in case the element is in a namespace, add the namespace uri using the James Clark notation, "{"+namespace uri+"}"localname
...
Properties serializationProps = new java.util.Properties();
serializationProps.setProperty("method", "xml");
serializationProps.setProperty("encoding", "UTF-8");
serializationProps.setProperty("omit-xml-declaration", "no");
serializationProps.setProperty("cdata-section-elements",
"product;{uri}product");
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"<e xmlns:p='uri'> " +
" <product>DataDirect XQuery & XML Converters</product>" +
" <p:product>DataDirect XQuery & XML Converters</p:product>" +
"</e>");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
null);
...
Yields the following result,
<?xml version="1.0" encoding="UTF-8"?>
<e xmlns:p="uri">
<product><![CDATA[DataDirect XQuery & XML Converters]]></product>
<p:product><![CDATA[DataDirect XQuery & XML Converters]]></p:product>
</e>
In addition to the XML output method, the XQuery serialization defines other output methods like HTML and XHTML. Note that these serialization methods will not "automagically" produce (X)HTML. It is still the query's responsibility to produce results conform to (X)HTML. But the serializer will consider the (X)HTML rules outputting the results. For example <br> elements will be serialized without a closing </br>.
Note for example the difference between the following result.xml and result.html
...
Properties serializationProps = new java.util.Properties();
XQPreparedExpression xqpe = xqc.createPreparedExpression(
"<html>line1<br/>line2</html>");
XQSequence xqs = xqpe.executeQuery();
serializationProps.setProperty("method", "xml");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.xml"),
serializationProps);
XQSequence xqs = xqpe.executeQuery();
serializationProps.setProperty("method", "html");
xqs.writeSequence(
new FileOutputStream("/home/jimmy/result.html"),
serializationProps);
...
result.xml is as follows,
<html>line1<br/>line2</html>
where results.html will look as follows,
<html>line1<br>line2</html>
If your interested in all the details about (X)HTML serialization, look here for HTML and here for XHTML.
In all previous examples, we've serialized the query results in a FileOutputStream. In addition an XQSequence can also be serialized into a java.io.Writer using the writeSequence() method. And getSequenceAsString() serializes to a java.lang.String.
Similar to serializing the complete XQSequence, there are methods to serialize the current item in the XQSequence. In the following example, the items in the query result are saved into individual files, result1.xml, result2.xml, and so on.
...
Properties serializationProps = new java.util.Properties();
serializationProps.setProperty("method", "xml");
serializationProps.setProperty("indent", "yes");
serializationProps.setProperty("encoding", "UTF-8");
serializationProps.setProperty("omit-xml-declaration", "no");
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery("doc('orders.xml')/*/ORDERS");
int i = 1;
while (xqs.next()) {
FileOutputStream file;
file = new FileOutputStream("/home/jimmy/result" +
i + ".xml");
xqs.writeItem(file, serializationProps);
file.close();
}
...
To conclude this post, note that XML serialization doesn’t always result in a well-formed XML document. More precisely it is either a well-formed XML document or a well-formed XML external general parsed entity. This is further explained in the serialization specification.
In the next upcoming post, we'll talk about manipulating the XQuery Static Context through the XQJ API.
Labels: XQJ
The application can browse through an XQSequence using the next() method. Initially the current position of the XQSequence is before the first item. next() moves the current position forward and returns true if there is another item to be consumed. Once all items in the sequence have been read, next() returns false.
Let's iterate through a sequence,
...
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery("doc('orders.xml')//order[id='174']");
while (xqs.next()) {
...
}
...
Positioned on an item, the application can retrieve the data using one of the getXXX() methods. To give a taste, we'll go through some of these methods by example.
An application can use getObject() to retrieve the current item of an XQSequence as a Java object. XQJ defines a mapping for each of the XQuery item types to a Java object value.
One of the most common scenario is probably a query returning a sequence of elements. Using getObject(), XQJ defines a mapping to Java DOM elements,
...
org.w3c.dom.Element employee;
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery("doc(employees.xml)//employee");
while (xqs.next()) {
employee = (org.w3c.dom.Element)xqs.getObject();
...
}
...
But actually, XQJ defines a mapping for every XQuery type to Java objects, including all the atomic types. Assume for example a query retuning xs:decimal values, using getObject() your Java application retrieves the items as java.math.BigDecimal objects,
...
java.math.BigDecimal price;
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(
"doc('orders.xml')/orders/order/xs:decimal(total_price)");
while (xqs.next()) {
price = (java.math.BigDecimal)xqs.getObject();
...
}
...
Suppose you have a query returning atomic values, and want to retrieve a textual representation of these. For example to output to System.out. getAtomicValue() returns a string representation of an atomic value according to the XQuery xs:string casting rules, and throws an exception if the item is a node.
In the next example the query returns a sequence of atomic values, note that the items are not all of the same type.
...
XQExpression xqe = xqc.createExpression();
XQSequence xqs = xqe.executeQuery(
"'Hello world!', 123, 1E1, xs:QName('abc')");
while (xqs.next()) {
System.out.println(xqs.getAtomicValue());
}
...
Beside the DOM, XQJ also provides native support for 2 other popular XML APIs, SAX and StAX. In the next example each of the items is returned to the application through SAX,
...
ContentHandler ch = ...
XQExpression xqe = xqc.createExpression();
XQSequence xqs = xqe.executeQuery(
"doc(employees.xml)//employee");
while (xqs.next()) {
xqs.writeItemToSAX(ch);
}
...
Up to now we have seen a number of examples where the application iterates over all the items in the sequence, and retrieves them one-by-one. The XQSequence interface also offers functionality to retrieve the complete sequence within a single call. In the next example, we execute a query and serialize the complete result into a SAX event stream.
...
ContentHandler ch = ...
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery("doc('employees.xml')//employee");
xqs.writeSequenceToSAX(ch);
...
Or in a similar way, read the complete sequence as a StAX event stream.
...
XQExpression xqe = xqc.createExpression();
XQSequence xqs = xqe.executeQuery("doc('employees.xml')");
XMLStreamReader xmlReader = xqs.getSequenceAsStream();
while (xmlReader.next() != XMLStreamConstants.END_DOCUMENT) {
...
}
...
Beside exposing the sequence through a SAX or StAX event stream, XQSequence also provides the ability to serialize into a binary or character stream. Here we're entering the arena of XSLT 2.0 and XQuery 1.0 Serialization, that's what the next post will be about.
Last, the above examples all iterate forward through the XQSequence objects. XQJ has also the notion of scrollable sequences, allowing to move both forward and backwards, set the cursor to an absolute position and allowing to iterate through the XQSequence more than once. We'll come back to it later.
Labels: XQJ
In XQJ an XQExpression objects allows you to execute an XQuery expression. Such XQExpression object is created in the context of an XQConnection. The next example creates an XQExpression and subsequently uses it to execute an XQuery expression,
...
// assume an XQConnection xqc
XQExpression xqe = xqc.createExpression();
xqe.executeQuery("doc('orders.xml')//order[id='174']");
...
The result of a query evaluation is a sequence, which is modeled as an XQSequence object in XQJ. Hence, the result of the executeQuery() method is an XQSequence. In typical scenarios the code example of above will look actually as follows,
...
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery("doc('orders.xml')//order[id='174']");
// process the query results
...
In the next post, XQJ Part IV, we'll discuss the XQSequence functionality in detail.
An XQExpression object can be reused, each time a different XQuery expression can be executed. The next example retrieves all orders with id 174 and next the orders with id 267,
...
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
// execute a first query
xqs = xqe.executeQuery("doc('orders.xml')//order[id='174']");
// process the query results
...
// execute a second query
xqs = xqe.executeQuery("doc('orders.xml')//order[id='267']");
// process the query results
...
In this last example we execute twice almost the same query, only the value compared against changes.
XQJ supports the concept of prepared queries. The idea here is to "prepare" the query only once, and subsequently "execute" it several times. During the prepare phase, the query is parsed, statically validated and an optimized execution plan is generated. This can be a relative expensive operation, hence using XQPreparedExpression objects can improve performance if the same query is executed multiple times.
Using prepared queries often implies the use of external variables in your query. The application can bind with each execution different values to each of the external variables.
Coming back to the previous example, here is the XQPreparedExpression variant. Note that the XQuery expression is specified when the XQPreparedExpression object is created, not at execute time,
...
XQPreparedExpression xqp;
XQSequence xqs;
xqp = xqc.prepareExpression(
"declare variable $id as xs:string external; " +
"doc('orders.xml')//order[id=$id]");
// execute a first query and process the query results
xqp.bindString(new QName("id"),"174", null);
xqs = xqp.executeQuery();
...
// execute a second query and process the query results
xqp.bindString(new QName("id"), "267", null);
xqs = xqp.executeQuery();
...
The previous example demonstrated how to bind values to XQPreparedExpression objects, it is also possible to bind values to external variables using XQExpression objects. Suppose you have a DOM tree and want to query it.
...
Document domDocument = ...;
XQExpression xqe = xqc.createExpression();
xqe.bindNode(new QName("doc"), domDocument, null);
XQSequence xqs = xqe.executeQuery(
"declare variable $doc as document-node(element(*,xs:untyped));" +
"$doc//order[id='174']");
// process the query results
...
XQuery has the concept of a context item, represented by a "dot" in your queries. In the previous examples we demonstrated values can be bound to XQExpression and XQPreparedExpression objects by name. In XQuery, the context item has no name, as such XQJ defines a name to bind the context item, XQConstants.CONTEXT_ITEM. The next example is similar to the last one, but it binds the DOM document to the initial context item rather than to an external variable.
...
Document domDocument = ...;
XQExpression xqe = xqc.createExpression();
xqe.bindNode(XQConstants.CONTEXT_ITEM, domDocument, null);
XQSequence xqs = xqe.executeQuery(".//order[id='174']");
// process the query results
...
In a later post, we will come back on binding values to external variables or the context item.
Note that in all the above examples, the XQuery expressions are specified as Java Strings. XQJ also allows to specify an InputStream, as shown in the next example, where the query in the getorders.xquery file is executed,
...
InputStream query;
query = new FileInputStream("home/joe/getorders.xquery")
XQExpression xqe;
XQSequence xqs;
xqe = xqc.createExpression();
xqs = xqe.executeQuery(query);
// process the query results
...
Interesting to note here is that an xquery can optionally specify the encoding in the query prolog. Good XQJ/XQuery implementations will use that information and properly parse the InputStream. If no encoding is specified, the assumed encoding depends on the implementation, for DataDirect XQuery this is UTF-8.
We know how to execute queries and have introduced the concept of prepared expressions, next in this series we will focus on processing query results, expect XQJ Part IV soon.
Labels: XQJ
An XQJ application always starts with accessing an XQDataSource object. Such object encapsulates all parameters and settings needed to create a session with a specific implementation and eventually execute XQuery expressions and process results.
Every XQJ driver has its own XQDataSource implementation. This XQDataSource object supports a number of implementation-specific properties. For each of these properties, a "getter" and "setter" method is provided.
Assume an application wants to query both an Oracle database, and some files located in "/usr/joe/data" with DataDirect XQuery. Two properties need to be specified, the base uri and the jdbc url to connect to the Oracle database (feel free to replace Oracle with your favourite database, be it SQL server, MySQL or yet another one),
DDXQDataSource xqds = new DDXQDataSource();
xqds.setBaseUri("/usr/joe/data");
xqds.setJdbcUrl("jdbc:xquery:oracle://sales:1521;SID=ORA10");
Or envision another Oracle specific implementation where the server parameters are specified in individual properties rather than through a jdbc url. It could be as follows,
OracleDataSource xqds = new OracleDataSource()
xqds.setServerName("sales");
xqds.setPortNumber(1521);
xqds.setSID("ORA10");
Having access to an XQDataSource object, what's next? An XQDataSource is a factory for XQConnection objects. The XQConnection object represents a session in which XQuery expression are executed.
Establishing such a session is straightforward,
XQConnection xqc = xqds.getConnection();
In case user credentials are needed, these can be specified as arguments to the getConnection() method,
XQConnection xqc = xqds.getConnection("joe", "topsecret");So far so good. Using the approach outlined above to create XQDataSource objects, makes the application dependent on a specific XQJ implementation. The proprietary classes DDXQDataSource and OracleDataSource are referenced. This is not necessarily wrong, there are scenarios where hard-coding the underlying XQJ implementation makes sense.
But often this is not desirable, XQJ is all about making your application independent from the underlying XQuery implementation. How can we make our application independent of the XQJ implementation? We'll show two approaches.
Assume all the XQDataSource properties are stored in a Java properties file, and in addition a property ClassName to identify the XQDataSource implementation to use.
For the DataDirect XQuery example above, the properties file would look as follows,
ClassName = com.ddtek.xquery3.xqj.DDXQDataSource
BaseUri = /usr/joe/data
JdbcUrl = jdbc:xquery:oracle://sales:1521;SID=ORA10
For the Oracle implementation,
ClassName = org.example.xqj.OracleDataSource
ServerName = sales
PortNumber = 1521
SID = ORA10
Using such properties file, an application can easily abstract out any hard-coded dependencies on the underlying XQJ implementation. The XQDataSource class is loaded through reflection and next it is simply a matter of passing in the properties.
// load the properties file
Properties p = new Properties();
p.load(new FileInputStream("/tmp/xqjds.prop"));
// create an XQDataSource instance using reflection
String xqdsClassName = properties.getProperty("ClassName");
Class xqdsClass = Class.forName(xqdsClassName);
XQDataSource xqds = (XQDataSource)xqdsClass.newInstance();
// remove the ClassName property
// the XQJ implementation will not recognize
// it and raise an error
p.remove("ClassName");
// set the remaining properties
xqds.setProperties(tmpProperties);
// create an XQConnection
XQConnection xqc = xqds.getConnection();
Of course, this is just an example, it might well be that for some applications another means to load the datasource properties is better suited.
Similar to JDBC, running in a J2EE environment, the XQDataSource object can be stored in a JNDI-enabled naming service. This allows your application to access the XQDataSource by simply specifying a logical name.
// get the initial JNDI context
Context ctx = new InitialContext();
// load the XQDataSource instance
XQDataSource xqds = (XQDataSource)ctx.lookup("xqj/sales");
// create an XQConnection
XQConnection xqc = xqds.getConnection();
For the readers with a JDBC background; JDBC has two mechanisms to establish a connection
Don't look in XQJ for DriverManager-like functionality, XQJ doesn't offer this legacy functionality.
We have now learned how to create an XQConnection. In our next post we will do some real work, and show how to execute queries.
Labels: XQJ
I hope this series will give a good overview of the major interfaces and functionality offered by XQJ, mostly through example code. The majority of the code should be implementation independent, but I have to admit, it will be tested using DataDirect XQuery 3.0, which supports the XQJ Public Draft.
We have today the following essays in the XQJ series (the list is updated as more posts become available),
XQDataSource xqjd = new DDXQDataSource();I assume most of it speaks for itself. But don't worry if you have some questions, things will become clear in the upcoming posts.
XQConnection xqjc = xqjd.getConnection();
XQExpression xqje = xqjc.createExpression();
XQSequence xqjs = xqje.executeQuery("'Hello World!'");
xqjs.writeSequence(System.out, null);
xqjc.close();
Some of you might recognize some of the JDBC concepts in the code example above.
Indeed, the XQJ expert group decided to make XQJ stylistic consistent with JDBC. Similar to JDBC is has concepts like,
Labels: XQJ