wiki:SilkRouteIssues

Version 1 (modified by trac, 12 years ago) (diff)

--

Issues with SilkRoute in production use


The ability to query a non-trivial XML view of a relational database is a powerful idea. It allows users to keep existing data sources and integrate them with varieties of data efficiently using a single query language. However, for all of its promise, the current implementation has some deficiencies that make it difficult to use in a production environment. There are 2 broad categories of problems: uninformative errors and an incomplete implementation of XQuery.

Query Syntax Errors

Take the example of a query with a simple syntax error, such as a misspelled "where" keyword:

$ /usr/bin/silkroute -db pg_bmap_repo -u xbrain_user -p Yo_YoMa userquery51255.uq  
SilkRoute error was: Fatal error: exception Error.Query(_)

The above error tells me almost nothing except that something is wrong with the query. It gives me no indication that I misspelled “where” in the query. Even if I were to use the number(object) function to cast a value to a number, I get the same error. A possible solution to this is to validate the syntax before translating it to SQL. Saxon provides some syntax validation:

$ java -cp $CP net.sf.saxon.Query myquery.xq
Error on line 11 column 2 of file:myquery.xq:
  XPST0003: XQuery syntax error in #...in $a/results/patient whare $#:
    expected "return", found name "whare"
Failed to compile query

XQuery Problems

A worse problem is SilkRoute's incomplete implementation of XQuery. One major failing is that SilkRoute does not implement all of the built in functions. This makes one of our queries nearly impossible to write as it requires a cast (find all labels with value less than 10, where label is a varchar). To make matters worse, this query initially returned incorrect results because SilkRoute silently did a string comparison.

Another failing is an incorrect implementation of XQuery comparisons, equals on 2 sequences, appears to be broken. We should be able to select all trials that with error code 2 or 9 by using trial/trialcode = (’2’, ‘9’). Instead this returns no results in SilkRoute. With little active development, it also appears unlikely that SilkRoute will keep up with XQuery or Galax developments. This will only put SilkRoute further behind.

Possible Workarounds

Fixing SilkRoute

The obvious solution is to apply our programming expertise to fix SilkRoute and/or add the required functionality to Galax, the XQuery engine Unfortunately, SilkRoute and Galax are written in OCaml and none of us have OCaml programming experience, not to mention we are not familiar with the codebase.

Database functions

Another possible solution would be to implement the XQuery/XPath built in functions in the database using stored routines and provide no helpful casting in the translation. This would require a lot of work and debugging.

In-Memory Alternative

The solution that seems most promising is to continue to use SilkRoute for its strength (an XQuery view over a RDB), but process queries through a different XQuery engine. This could be done either by retaining SilkRoute for rough initial results and then post-processing, or by fully materializing the XML view of the database and using an in-memory XQuery engine.

We are already performing some post-processing of results; the main shortcoming of this approach is that we never know exactly what will cause SilkRoute to fail until we try it or notice some inconsistencies in results.

Eider explored using the materialized database (currently a 22MB XML file) with an in-memory query processor. To ensure that it has adequate performance he benchmarked Saxon with our database pre-loaded into memory:

<rowstyle="background-color: #C0C0C0;">Query Saxon results SilkRoute speed
P50 All mapped sites0.3 sfast
All trials with error code ‘2’1.1 sfast
All trials with error code ‘2’ or ‘9’1.2 sslow or incorrect
All sites with label < 100.9 sincorrect

SilkRoute was not accurately benchmarked, but “fast” should be interpreted as under 1 s and “slow” as greater than 15 s. In a Web Service context Saxon is slightly faster than above and comparable to SilkRoute’s fast results. The disadvantage of this approach is that is requires materializing the view and takes a lot of memory (currently 200-300Mb for the 22MB file).

Conclusion

Both expert and novice users are likely to gain much of their experience in XQuery from tutorials, and trial and error. Uninformative error messages makes the trial and error process burdensome. An incomplete implementation makes it difficult to learn from tutorials, since users are constantly annoyed by finding described features missing. To make the problem worse, no information is given that a query failed because a feature was missing (as opposed to a syntax error). These issues are problematic, but are possible to work around. The cases where incorrect results are returned are a much more vexing obstacle. While a fully conforming SilkRoute remains the ideal solution, the current implementation is not adequate for our needs. Unless SilkRoute sees further improvements, a Saxon based Web Service is a more viable alternative for us.