Introduction to XQuery

by K. Yue

1. Introduction

Resources:

• XQuery:
  • W3C XQuery page: http://www.w3.org/standards/xml/query
  • XQuery 1.0 recommendation: http://www.w3.org/TR/xquery/
  • XQuery 3.0 (formerly 1.1) recommendation: http://www.w3.org/TR/2014/REC-xqueryx-30-20140408/
  • XQuery standards: http://www.w3.org/standards/techs/xquery#w3c_all
• XPath:
  • XPath 1.0: http://www.w3.org/TR/xpath
    • Still many examples in the Web.
    • Example: http://www.whitebeam.org/library/guide/TechNotes/xpathtestbed.rhtm
  • XPath 2.0: http://www.w3.org/TR/xpath20/
    • Not quite the same as XPath 1.0.
    • Probably currently the most popular version.
  • XPath 3.0: http://www.w3.org/TR/xpath-30/:
    • generally a superset of XPath 2.0
• XQuery and XPath functions:
  • 1.0: http://www.w3.org/TR/xquery-operators/.
  • 3.0: http://www.w3.org/TR/2014/REC-xpath-functions-30-20140408/.
• XML Schema: http://www.w3.org/XML/Schema
• EditiX (free edition): http://free.editix.com/
  • Free and good enough for our course.

Basics:

• XQuery is designed for effectively query and retrieve information from diversified XML sources.
• The XML sources can be one or more XML documents.
• XQuery is derived from Quilt, and has borrowed features from XPath, XQL, SQL, etc.
• It is a functional language where a query is an expression.
• There are three faces of the XQuery languages:
  1. A "surface" syntax that programmers may probably use.
  2. An XML-based syntax that machine may probably use (XQueryX).
  3. A formal semantic that XQuery engine implementors use.
• XQuery 1.0 extends XPath 2.0.
• The type system of XQuery is based on XML Schema.
• A limitation of XQuery: No update or insert. The basic building block of XQuery is expressions. (In this sense, like SQL, XQuery is not a full programming language.)
• XQuery is a superset of XPath.

Comparisons to SQL:

XQuery Data Types

• XQuery is strongly typed.
• XQuery types are based on
  • XML Schema: using the namespace prefix xs and url: http://www.w3.org/2001/XMLSchema.
  • XPath functions and operators: using the namespace prefix xdt and url: http://www.w3.org/2004/07/xpath-datatypes

• xdt:untyped is used to denote element nodes not yet validated.
• xdt:untypedAtomic is used to denote atomic types that has not been assigned a more specific type.

Query Expressions

• A query in XQuery is an expression for reading XML documents or fragments and returning a sequence of well-formed XML fragments
• Everything in XQuery is an expression that is evaluated to a value.
• Some common XQuery expressions are (these appear in most tutorials):
  1. path expressions
  2. element constructors
  3. FLWR or FLOWR (pronounced as "flower") expressions
  4. list expressions
  5. conditional expressions
  6. quantified expressions
  7. datatype expressions
• Examples of other expressions include:
  1. primary expressions
  2. sequence expressions
  3. arithmetic expressions
  4. logical expressions
  5. comparison expressions
  6. sorting expressions
  7. validate expressions
• XQuery comments are embedded within (: and :).

Functions

• Supports a collection of about 200 built-in operators and functions to be used within expressions.
• Input functions in XQuery include doc() and collection(). They are used to identify the sources of the XML documents.

Prolog

• XQuery may have prologs for declarations.
• Examples:
  1. Variable declarations
  2. Function declarations
  3. Base-URI declarations
  4. Version declarations
  5. Module import

Example: Variable declaration

Format: declare variable $name = expression;

declare variable $a := doc("census.xml")//person ;

Path Expressions

• XQuery 1.0 is a superset of XPath 2.0.
• An XPath expression is also an XQuery expression.

Example: In Editix,

Use "View > Windows > XQuery Builder" For XQ files

Use "XSLT/XQuery > Transform using an XQuery Request…"

Specify source xq file, xml file and output file. Use .xml extension. If you use .txt extension, only text node contents are output.

declare base-uri "whatever-path";
doc("bib.xml")/*

Return basically bib.xml.

doc("bib.xml")//*

• Return many nodes (in a sequence).
• Results are not well-formed.

Element Constructors

• Element constructors can be used to construct XML elements to return.
• If the name, attributes, and content of the element are all constants, the element constructor is based on standard XML notation and is called a direct element constructor (W3C).

Example: The XQuery

<authors>
    <author>Bun Yue</author>
</authors>

returns

<authors>
    <author>Bun Yue</author>
</authors>

• XQuery expressions can be embedded in the direct element constructors within a pair of curly braces, {}.
• For the characters '{' and '}', use '{{' and '}}' respectively.
• XQuery expressions may be separated by commas.

Example:

Using film.xml:

<films>
   <titles>
      { //film/title }
   </titles>
</films>

returns:

<?xml version="1.0" encoding="UTF-8"?>
<films>
   <titles>
      <title>ACADEMY DINOSAUR</title>
      <title>ACE GOLDFINGER</title>
      <title>ADAPTATION HOLES</title>
      ...
      <title>ZOOLANDER FICTION</title>
      <title>ZORRO ARK</title>
   </titles>
</films> 

• Computed constructors can also be used to declare nodes:
  • Use the keywords element, attribute, document, text, processing-instruction, comment, or namespace to declare the type of the nodes.
  • Specify the node names for those node types with names (element, attribute, processing instruction, and namespace nodes)
  • Use a pair of braces to define the content expressions.
  • Note the use of commas to separate expressions in the context.

Example:

<films>
   { element titles
      { //film/title }
   }
</films> 

return the same result as the previous example.

element films {
   attribute numFilms {
      count(//film)
   }
}

returns

<?xml version="1.0" encoding="UTF-8"?>
<films numFilms="1000"/>

<categories>
{   for $c in fn:distinct-values(//category/text())
   return
   <category>
      <name>{ $c }</name>
   </category>
}
</categories>

returns

<?xml version="1.0" encoding="UTF-8"?>
<categories>
   <category>
      <name>Documentary</name>
   </category>
   <category>
      <name>Horror</name>
   </category>
   <category>
      <name>Family</name>
   </category>
   <category>
      <name>Foreign</name>
   </category>
   <category>
      <name>Comedy</name>
   </category>
   <category>
      <name>Sports</name>
   </category>
   <category>
      <name>Music</name>
   </category>
   <category>
      <name>Classics</name>
   </category>
   <category>
      <name>Animation</name>
   </category>
   <category>
      <name>Action</name>
   </category>
   <category>
      <name>New</name>
   </category>
   <category>
      <name>Sci-Fi</name>
   </category>
   <category>
      <name>Drama</name>
   </category>
   <category>
      <name>Travel</name>
   </category>
   <category>
      <name>Games</name>
   </category>
   <category>
      <name>Children</name>
   </category>
</categories>

Example:

<categories>
{   for $c in fn:distinct-values(//category/text())
   return
   <category>
      <name>{ $c }</name>
      {   element numFiles
         { count(//film/category/text()=$c) }
      }
   </category>
}
</categories>

returns:

<?xml version="1.0" encoding="UTF-8"?>
<categories>
   <category>
      <name>Documentary</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Horror</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Family</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Foreign</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Comedy</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Sports</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Music</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Classics</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Animation</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Action</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>New</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Sci-Fi</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Drama</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Travel</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Games</name>
      <numFiles>1</numFiles>
   </category>
   <category>
      <name>Children</name>
      <numFiles>1</numFiles>
   </category>
</categories>

The XQuery:

<categories>
{   for $c in fn:distinct-values(//category/text())
   return
   element { $c }
   {   attribute numFiles
         { count(//film[category/text()=$c]) }
   }
}
</categories>

returns:

<?xml version="1.0" encoding="UTF-8"?>
<categories>
   <Documentary numFiles="68"/>
   <Horror numFiles="56"/>
   <Family numFiles="69"/>
   <Foreign numFiles="73"/>
   <Comedy numFiles="58"/>
   <Sports numFiles="74"/>
   <Music numFiles="51"/>
   <Classics numFiles="57"/>
   <Animation numFiles="66"/>
   <Action numFiles="64"/>
   <New numFiles="63"/>
   <Sci-Fi numFiles="61"/>
   <Drama numFiles="62"/>
   <Travel numFiles="57"/>
   <Games numFiles="61"/>
   <Children numFiles="60"/>
</categories>
 

FLWOR expressions

• FLWOR expressions are one of the most important constructs in XQuery.
• You may compare with the SELECT statement of SQL.

From W3C:

[42]    FLWORExpr    ::=    (ForClause | LetClause)+ WhereClause? OrderByClause? "return" ExprSingle
[43]    ForClause    ::=    "for" "$" VarName TypeDeclaration? PositionalVar? "in" ExprSingle ("," "$" VarName TypeDeclaration? PositionalVar? "in" ExprSingle)*
[45]    LetClause    ::=    "let" "$" VarName TypeDeclaration? ":=" ExprSingle ("," "$" VarName TypeDeclaration? ":=" ExprSingle)*
[123]    TypeDeclaration    ::=    "as" SequenceType
[44]    PositionalVar    ::=    "at" "$" VarName
[46]    WhereClause    ::=    "where" Expr
[47]    OrderByClause    ::=    ("order" "by" | "stable" "order" "by") OrderSpecList
[48]    OrderSpecList    ::=    OrderSpec ("," OrderSpec)*
[49]    OrderSpec    ::=    ExprSingle OrderModifier
[50]    OrderModifier    ::=    ("ascending" | "descending")? (("empty" "greatest") | ("empty" "least"))? ("collation" StringLiteral)?
 

• FLWOR expressions allow:
  1. For: Iteration through items in XPath 2.0 sequences. Create a tuple stream where each tuple contains a distinct binding for each variable to a distinct value.
  2. Let: Variables binding
  3. Where: Predicate application for inclusion in the iteration.
  4. Order by: Ordering data set for the iteration.
  5. Return: Constructing new result for returning.

Example:

List the film categories with 60 or more films in ascending orders of category names.

<categories>
{   for $cat in fn:distinct-values(//category/text())
   let $count := count(//film[category/text()=$cat])
   where $count >= 60
   order by $cat
   return
   element { $cat }
   {   attribute numFiles
         { $count }
   }
}
</categories>

Output:

<?xml version="1.0" encoding="UTF-8"?>
<categories>
   <Action numFiles="64"/>
   <Animation numFiles="66"/>
   <Children numFiles="60"/>
   <Documentary numFiles="68"/>
   <Drama numFiles="62"/>
   <Family numFiles="69"/>
   <Foreign numFiles="73"/>
   <Games numFiles="61"/>
   <New numFiles="63"/>
   <Sci-Fi numFiles="61"/>
   <Sports numFiles="74"/>
</categories>

List ids of actors 'KILMER'.

XQuery:

element actors
{   for $aid in fn:distinct-values
      (//actor[text()[contains(.,'KILMER')]]/@id)
   return
   element actor
   {    attribute id { $aid }
   }
}

Output:

<?xml version="1.0" encoding="UTF-8"?>
<actors>
   <actor id="162"/>
   <actor id="23"/>
   <actor id="55"/>
   <actor id="45"/>
   <actor id="153"/>
</actors>
 

List names and ids of actors 'KILMER'.

XQuery:

element actors
{   for $aid in fn:distinct-values
      (//actor[text()[contains(.,'KILMER')]]/@id)
   let $aname := (//actor[@id=$aid][1]/text())[1]
   return
   element actor
   {   attribute id { $aid },
      element name { $aname }
   }
}

Output:

<?xml version="1.0" encoding="UTF-8"?>
<actors>
   <actor id="162">
      <name>OPRAH KILMER</name>
   </actor>
   <actor id="23">
      <name>SANDRA KILMER</name>
   </actor>
   <actor id="55">
      <name>FAY KILMER</name>
   </actor>
   <actor id="45">
      <name>REESE KILMER</name>
   </actor>
   <actor id="153">
      <name>MINNIE KILMER</name>
   </actor>
</actors>

List names, ids and number of films of actors 'KILMER'.

XQuery:

element actors
{   for $aid in fn:distinct-values
      (//actor[text()[contains(.,'KILMER')]]/@id)
   let $name := (//actor[@id=$aid]/text())[1],
       $n_films := count(//film[actor/@id = $aid])
   order by number($n_films) descending
   return
   element actor
   {    attribute id { $aid }, 
       element name { $name },
       element nFilms { $n_films }
   }
}

Output:

<?xml version="1.0" encoding="UTF-8"?>
<actors>
   <actor id="23">
      <name>SANDRA KILMER</name>
      <nFilms>37</nFilms>
   </actor>
   <actor id="45">
      <name>REESE KILMER</name>
      <nFilms>32</nFilms>
   </actor>
   <actor id="162">
      <name>OPRAH KILMER</name>
      <nFilms>25</nFilms>
   </actor>
   <actor id="55">
      <name>FAY KILMER</name>
      <nFilms>20</nFilms>
   </actor>
   <actor id="153">
      <name>MINNIE KILMER</name>
      <nFilms>20</nFilms>
   </actor>
</actors>
 

• There are many other XQuery expressions.
• Since this is not a course on XML, we will not cover them.