XML/JSON Data Source
Makes XML or JSON content from local or web resources
available to CW layers.
For processing of the
XML or JSON content, XQuery is used. XQuery
is a standardized
language for finding and extracting elements and
attributes from XML documents. The power of the XQuery language makes
it possible to perform a wide-range of filtering, sorting and
preprocessing
operations on the input data.
Clicking on the Execute
XML Query Now button will retrieve XML (or JSON) input data
from the XML
Path/URL,
execute the XQuery
on the input data and update the data
source table
with the results.
XML
Path/URL should point to a file or web resource with
XML (or JSON) data. When query execution
is triggered (manually or automatically), the input data will be
retrieved
from this location and it will be processed by the XQuery.
Results of
query execution are returned by the XQuery engine
in XML format. CW parses these query results to fill
the data source table. For this to work, it is
expected that the XQuery is written so that the output will conform to
the following general structure:
<row>
<column1>value1</column1>
<column2 column3="value3">value2</column2>
...
</row>
The tags and attributes row, column1, column3,
etc. given above are
only examples and
they can be arbitrary,
as long as the XML follows this two-level
structure. Inner tag names (e.g. column1, column2column3
above) will be used as the column (field)
names in the generated table. The Extract setting determines whether
data will be extracted from tag (element) content (e.g. value1,
value2
above), attributes (e.g. value3
above), or both.
Certain data sources provide their data in HTML-encoded
format, where
special characters are replaced with their escaped versions (e.g. >
replaced with <). To decode these properly, the
Processing
setting can be set to "HTML Decode". It must be noted that this setting
also performs space normalization.
Data source tables can be
edited manually (or even created from scratch).
Rows and
columns can be inserted or removed. Cell contents can be edited. Field
(column) names can also be changed, as long as they remain unique to
the table.
A data source must be updated
once
editing or query execution is complete (unless Safe
Editing is disabled). The update can be performed
manually or automatically. See Updating
Data Sources for more information on how the update process
works.
Instead
of text or numeric data, data source cells may contain media
(images, video or audio). Upon
query execution, or manual editing of table cells, if the
value of a table cell appears to be a media filename, or the path/URL
to a
media file, CW will attempt to retrieve the media file and put it in
the table cell. If the cell contains a relative path, or just a
filename, then CW will look for the media file in the list specified in
Footage
Search Paths/URLs. Semicolons are used to separate multiple
path/URL entries in the
list. If the cell contains an absolute path, the
footage search paths setting is ignored.
A single
cell
cannot include both text and media at the same time. Furthermore, a
column may not include both types of cells. During manual editing, CW
automatically determines the type
of cells (image or text) on a column basis, by checking if the
most-recently
edited cell in the column contains a media filename or URL.
Table-style data sources
provide a selection
mechanism which makes it possible to quickly
determine which rows will be included in the data set. There are three
selection
modes:
- In the single
mode, each row of a
table-style data source
contains a selection radio button on the leftmost side. The data source
data set contains a single row; the one with the active radio
button.
- The dual
mode is similar to the single mode, except that there are two columns
of radio buttons. The data source data set contains two rows;
the first row is the row with the active radio button on the
left column, and the second row is the row with the
active radio button on the right column,
- In the multi
mode, each row of a
table-style data source
contains a selection checkbox on the leftmost side. A row is included
in the data source only if the selection checkbox is checked. The
topmost checkbox (in the table header area) can be used to select or
unselect all rows quickly.
For example, if a data source contains information
on players in a soccer team, where each row corresponds to one player,
one or more players may be selected (and the rest of the players
deselected) using the selection mechanism, so that a
relevant motion will show
data for these players when it is played back. The choice of the
selection mode depends on the graphics: For a player bio, one might use
the single mode. For player comparison
graphics, dual mode may be more suitable. For more general graphics,
such as lineup tables, multi mode is often the only appropriate choice.
ALT+left-click on a data source row provides a quick way to iterates
though the selection states for that data row.
Each
time a data source query is executed, the selection is reset.
To keep the selection unchanged, enable Maintain
Selection mode. By default, maintain selection mode
retains checkbox states based on row number. It
is also possible to maintain selection based on the content of a key column. A data
source column can be set as selection
key
using the right-click menu. If a selection key column is set, then for
each row in the incoming query results, CW will check to see
if a row
with the same key already existed in the data source, and if so, the
selection state from the existing row is used for the incoming
row. If not, the selection will be cleared for that row.
When File
Tracking is enabled, the XQuery
is automatically executed whenever the file XML
Path
points
to is modified. The query execution will take place in the background
even when the data source is not displayed in the data editor.
This feature only works with local files, not with web URLs.
When Execute
XML Query Periodically
is enabled, the XQuery
is executed periodically, at specified regular
intervals. The query execution will continuously take place in the
background even when the data source is not selected in the data
editor. This setting is useful when retrieving data from web URLs,
since file
tracking
is not available for these.
The Query
Processing section contains three sub-tabs that
show inputs and outputs corresponding to
each stage of query execution. The contents of each of these tabs
are updated each time the XQuery for the selected data source is
executed.
- Reply displays
the data that was received from XML
Path/URL
as a result of the initial data retrieval
stage of query execution.
- XQuery
Input displays the XML data
that is fed as input to XQuery processing. In the
case of XML input data, the contents of this tab are the same as those
of the Reply tab. For JSON input data, this tab contains the automatic
conversion of the JSON data to XML format. Any problems with
the
JSON-to-XML conversion process can be identified through this tab.
- XQuery
Output displays the results
of the XQuery processing. Any errors related
to
the XQuery code itself, or any errors that occur during the execution
of the XQuery are also shown here. The output shown in this
tab should adhere to the two-level structure indicated above,
otherwise, the results
table will come up empty, or incorrect.
The Input
Format setting specifies the expected format for the input
data. If set to Auto, CW tries to determine the format from the
filename extension or the Content-Type response header returned by the
server. If set to JSON (or if CW detects JSON input in Auto mode) JSON
to XML conversion is performed before XQuery execution.
When retrieving input
data from the web, any special headers for the
HTTP request (e.g. Accept
for content negotiation,
Authorization
for HTTP authentication)
can be specified in the Request
Headers section.
The
default XQuery contained in a newly-created XML/JSON Data
Source will receive and parse an RSS feed, which should serve as a
basic example to illustrate the concepts outlined above. In addition,
the CW installation comes with several sample projects
that demonstrate interfacing with XML/JSON sources: Crawls_JSON,
RSS_Ticker_Channel_News,
Weather_Samples.
Sample Project: Weather_Europe, Weather_Crawl
Video: City
Weather Forecast with CW
See also: Data Source,
Updating
Data Sources, XQuery
Troubleshooting.
