The Wayback Machine - https://web.archive.org/web/20120226143801/http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/
DBpedia D2R Server R2R Silk LDIF NG4J Marbles WIQA Pubby RAP
Open Source projects by the Web-based Systems Group:  
Chris Bizer
Richard Cyganiak

D2R Server is a tool for publishing relational databases on the Semantic Web.
It enables RDF and HTML browsers to navigate the content of the database,
and allows applications to query the database using the SPARQL query language.

Download D2R Server

v0.7, released 2009-08-10

Live Demo

Web-based Systems Group database

News

Contents

  1. About D2R Server
  2. Quick start
  3. Customizing the database mapping
  4. Serving vocabulary classes and properties
  5. Server configuration
  6. Publishing metadata
  7. Optimizing performance
  8. Running D2R Server as a service on Windows
  9. Running D2R Server in an existing Servlet Container
  10. Creating RDF dumps
  11. Pre-built mapping files
  12. Support and Feedback
  13. Source code and development
  14. Public servers
  15. Related projects

About D2R Server

D2R Server is a tool for publishing the content of relational databases on the Semantic Web, a global information space consisting of linked data.

Data on the Semantic Web is modelled and represented in RDF. D2R Server uses a customizable D2RQ mapping to map database content into this format, and allows the RDF data to be browsed and searched – the two main access paradigms to the Semantic Web.

D2R Server's Linked Data interface makes RDF descriptions of individual resources available over the HTTP protocol. An RDF description can be retrieved simply by accessing the resource's URI over the Web. Using a Semantic Web browser like Tabulator (slides) or Disco, you can follow links from one resource to the next, surfing the Web of Data.

The SPARQL interface enables applications to search and query the database using the SPARQL query language over the SPARQL protocol.

A traditional HTML interface offers access to the familiar Web browsers.

D2R Server architecture diagram

Requests from the Web are rewritten into SQL queries via the mapping. This on-the-fly translation allows publishing of RDF from large live databases and eliminates the need for replicating the data into a dedicated RDF triple store.

Read more about the interfaces offered by D2R Server, including example HTTP requests and responses, in the Technical Note Publishing Databases on the Semantic Web.

Quick start

Also see Vadim Eisenberg's tutorial for getting D2R Server 0.7 up and running with a simple example database and mapping.

You need:

What to do:

  1. Download and extract the archive into a suitable location.

  2. Download a JDBC driver from your database vendor. Place the driver's JAR file into D2R Server's /lib directory. A list of JDBC drivers from different vendors is maintained by Sun. Also take note of the driver class name (e.g. org.postgresql.Driver for PostgreSQL or oracle.jdbc.driver.OracleDriver for Oracle) and JDBC URL pattern (e.g. jdbc:mysql://servername/database for MySQL) from the driver's documentation. Drivers for MySQL and PostgreSQL are already included with D2R Server.

  3. Generate a mapping file for your database schema. Change into the D2R Server directory and run:

    generate-mapping -o mapping.n3 -d driver.class.name
    -u db-user -p db-password jdbc:url:...

    mapping.n3 is the name for the new mapping file. -d can be skipped for MySQL.

  4. Start the server:

    d2r-server mapping.n3

  5. Test the Server: Open http://localhost:2020/ in a web browser.

    Screenshot of D2R Server's web interface

    You can browse the database content or use the SPARQL Explorer to execute queries and display results in a number of formats.

    To test the data in an RDF browser, open any resource URI in Tabulator. You may have to tweak your Firefox settings first – see the Tabulator help.

Customizing the database mapping

D2R Server uses the D2RQ Mapping Language to map the content of a relational database to RDF. A D2RQ mapping specifies how resources are identified and which properties are used to describe the resources.

The generate-mapping script automatically generates a D2RQ mapping from the table structure of a database. The tool generates a new RDF vocabulary for each database, using table names as class names and column names as property names. Semantic Web client applications will understand more of your data if you customize the mapping and replace the auto-generated terms with terms from well-known and publicly accessible RDF vocabularies.

The mapping file can be edited with any text editor. Its syntax is described in the D2RQ Manual. D2R Server will automatically detect changes to the mapping file and reload appropriately when you hit the browser's refresh button.

Note: The HTML and RDF browser interfaces only work for URI patterns that are relative and do not contain the hash (#) character. For example, a URI pattern such as entries/@@mytable.id@@ is browsable, but http://example.com/entries#@@mytable.id@@ is not. The mapping generator only creates browsable patterns. Non-browsable patterns still work in the SPARQL interface and in RDF dumps.

Serving vocabulary classes and properties

A D2R deployment often requires the introduction of custom classes and properties. In the spirit of Linked Data, vocabulary data should be dereferencable by clients. D2RQ infers types of classes and properties, and allows the user to provide labels, comments and additional properties.

D2R Server automatically serves data for vocabularies placed under http://baseURI/vocab/resource/, with RDF and HTML representations located at http://baseURI/vocab/data/ and http://baseURI/vocab/page/, respectively. The following mapping illustrates the intended usage: 

@prefix vocabClass: <http://localhost:2020/vocab/resource/class/> .
@prefix vocabProperty: <http://localhost:2020/vocab/resource/property/> .

map:offer a d2rq:ClassMap;
	d2rq:classDefinitionLabel "Offer"@en;
	d2rq:classDefinitionComment "This is an offer"@en;
	d2rq:class vocabClass:Offer;
	.

When dereferenced, http://localhost:2020/vocab/resource/class/Offer will return the specified label and comment as well as the automatically inferred type rdfs:Class. Note that the prefixes are bound to absolute URIs because relative URIs would be based under http://localhost:2020/resource/.

D2RQ offers several constructs to provide data for classes and properties. Please refer to the D2RQ Language Specification for more details.

Server configuration

The server can be configured by adding a configuration block to the mapping file (all parts are optional):

@prefix d2r: <http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/config.rdf#> .

<> a d2r:Server;
    rdfs:label "D2R Server";
    d2r:baseURI <http://localhost:2020/>;
    d2r:port 2020;
    d2r:vocabularyIncludeInstances true;
    d2r:metadataTemplate "metadata.n3";
    d2r:documentMetadata [
        rdfs:comment "This comment is custom document metadata.";
    ];
    .

The rdfs:label is the server name displayed throughout the HTML interface.

The d2r:baseURI and d2r:port must be changed to make the D2R Server accessible to remote machines.

The d2r:vocabularyIncludeInstances setting controls whether the RDF and HTML representations of vocabulary classes and properties will also list related instances (defaults to true). As an alternative to disabling the serving of vocabulary instances, d2rq:resultSizeLimit may be used to limit the amount of returned data. Vocabulary serving is a feature of D2RQ and may be controlled using the d2rq:serveVocabulary property.

The d2r:autoReloadMapping setting specifies whether changes to the mapping file should be detected automatically (defaults to true). This feature has performance implications, so this value should be set to false for high-traffic production systems or when running benchmarks.

With large databases, some queries will produce too many results. Adding a d2rq:resultSizeLimit to the d2rq:Database section of the mapping file will add a LIMIT clause to all generated statements.

d2rq:resultSizeLimit 500;

Not that this effectively “cripples” the server and can cause somewhat unpredictable results. We still recommend it until we can offer a better solution.

d2r:metadataTemplate is used to specify a metadata template file within the directory WEB-INF/templates (more details).

All statements inside the d2r:documentMetadata block will be added as document metadata to all RDF documents.

Publishing metadata

D2R Server provides a mechanism for adding metadata such as licensing and provenance information to the RDF graphs served by its Linked Data interface. The supplied metadata will be shown in a "Metadata" table below the usual property-value table, which provides a tree-like visualization of the metadata. The (more) links in the visualization allow to expand the view to show the properties of the corresponding entities.

The metadata added to the provided RDF graphs is generated from a template RDF graph by replacing placeholders in the template. These placeholders refer to run-time data, typical configuration options, as well as to additional configuration variables. The metadata extension includes the default template metadata.n3, which adds various pieces of provenance information to the published data.

In order to include metadata along with the graphs served by D2R Server, the path to the metadata template is specified relative to WEB-INF/templates subdirectory using the predicate d2r:metadataTemplate in a d2r:Server configuration block:

<> a d2r:Server;
    d2r:metadataTemplate "metadata.n3";

Template RDF Graph

The metadata template is an RDF graph that may contain placeholder URIs. These placeholders allow a customization of the metadata that is served for each resource. They use the about: URI scheme and may be specified in the subject and object position of the triples in the template graph.

Template placeholders are organized into three groups that represent the available data sources: runtime, config, and metadata. All placeholder URIs follow the scheme about:metadata:groupname:identifier.

The following placeholder groups are currently available:

runtime

Placeholders in this group are replaced with D2R Server runtime data. Currently, D2R Server supports the following runtime placeholders:

about:metadata:runtime:time Replaced by a xsd:dateTime literal which represents the current date and time.
about:metadata:runtime:query Replaced by a string literal with the SPARQL DESCRIBE query which has been used to get the data about the requested resource.
about:metadata:runtime:resource Replaced by the URI of the requested resource.
about:metadata:runtime:graph Replaced by the URI of the current RDF graph that is served by Pubby.

config

This group of placeholders corresponds to the configuration options for the d2r:Server instance. For example, the placeholder about:metadata:config:baseURI is replaced with the value of the d2r:baseURI property defined on the d2r:Server instance.

metadata

Placeholders in this group allow the inclusion of additional metadata attributes, whose replacement attributes are defined as properties of the d2r:Server instance from the http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata# namespace (abbreviated with the prefix meta: in the following). Each placeholder about:metadata:metadata:identifier is replaced by the value of the property specified using a corresponding meta:identifier predicate.

In the following example, the property meta:d2rUser is provided, whose value will replace mentions of about:metadata:metadata:d2rUser in the metadata template:

@prefix meta: <http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#> .

<> a d2r:Server;
    d2r:metadataTemplate "metadata.n3";
    meta:d2rUser <http://example.org/URI_of_publisher> ;

Sample Template

@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix xhv:  <http://www.w3.org/1999/xhtml/vocab#> .

<about:metadata:runtime:graph> foaf:primaryTopic <about:metadata:runtime:resource> .
<about:metadata:runtime:graph> xhv:license <about:metadata:metadata:license> .

This template adds two triples to the RDF graphs served by the corresponding D2R Server instance. The first triple asserts that the primary topic of the served RDF graph is the resource that was requested. The second triple identifies the license of the served RDF graph and requires the metadata attribute http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#license to be specified in the mapping file.

Default Template

The default metadata template that comes with D2R Server automatically provides a description of the provenance of the served RDF graphs. This description uses the Provenance Vocabulary and includes information about the creation of served RDF graphs, the D2R Server instance, the DESCRIBE query used for creation, etc. The following additional metadata attributes may be configured in the mapping file to provide more detailed information about the D2R Server instance:

meta:d2rUser This property refers to the URI of the data publisher who uses this D2R Server instance to publish her/his data. The URI should be an HTTP URI that links to data about the publisher.
meta:d2rUserName, meta:d2rUserHomepage These properties provide the name and the homepage address of the data publisher who uses this D2R Server instance. They could be used as an alternative to meta:d2rUser if the publisher has no HTTP-dereferencable URI yet.
meta:d2rOperator This property refers to the URI of the service provider who operates this D2R Server instance. The service provider and the data publisher may be the same entity; nonetheless, it is recommended to specify both properties (using the same value) so that an application can infer the fact that publisher and operator are the same.
meta:d2rOperatorName, meta:d2rOperatorHomepage These properties provide the name and the homepage address of the service provider who operates this D2R Server instance. These properties could be used as an alternative to meta:d2rOperator.
meta:dataset This property refers to the URI of the whole linked dataset that is published by this D2R Server instance. This URI usually links to the voiD description of the dataset.

Optimizing performance

Here are some simple hints to improve D2R's performance:

Running D2R Server as a service on Windows

To run D2R Server as a service on Windows, use the install-service script:

install-service servicename mapping.n3

The service can be started from the Services panel in the management console. The service will log all output including startup errors to the files stdout.log and stderr.log.

To uninstall the service, use uninstall-service. You have to uninstall and re-install the service when upgrading to a later version of D2R Server.

Running D2R Server in an existing Servlet Container

By default, D2R Server is a stand-alone server application that includes its own web server. But D2R Server can also be run as a J2EE web application inside an existing servlet container, such as Tomcat:

  1. Make sure that your mapping file includes a configuration block, as described in the server configuration section. Set the port number to that of the servlet container, and set the base URI to something like http://servername/webappname/.
  2. Change the configFile param in /webapp/WEB-INF/web.xml to the name of your configuration file. For deployment, we recommend placing the mapping file into the /webapp/WEB-INF/ directory.
  3. In D2R Server's main directory, Run ant war. This creates the d2r-server.war file. You need Apache Ant for this step.
  4. Optionally, if you want a different name for your web application, rename the file to webappname.war
  5. Deploy the war file into your servlet container, e.g. by copying it into Tomcat's webapps directory.

Creating RDF dumps

Sometimes it is useful to create an RDF dump of the entire database. The dump-rdf script can be used for this purpose. See the D2RQ manual for details.

Pre-built mapping files

We collect mapping files for popular database-driven applications.

Have another one? Please share it.

Support and feedback

You can contact us on the D2RQ mailing list at d2rq-map-devel@lists.sourceforge.net.

Source code and development

D2R Server combines the D2RQ API, the Joseki SPARQL Server and the Jetty webserver.

D2R Server is hosted by SourceForge.net as part of the D2RQ project. The latest source code is available from the project's CVS repository and can be browsed online.

SourceForge Logo

Public servers

Related projects

DOAP project description (View with Disco)