Faceted Views over Large-Scale Linked Data

• Web 1.0: Publishing for All (Citizen Publisher) via Web Sites
• Web 2.0: Commentary for All (Citizen Journalist) via Blogging and Social Networks, with User Generated Content across Data Silos
• Web 3.0: Analysis for All (Citizen Analyst), via Linked Data enabling Data Mobility and Meshing, Your Data Is Your Statement, Applications Float on a Cloud of Data across a federation of HTTP accessible Data Spaces

• Meanwhile, in the DBMS world, ad hoc data access and manipulation has consistently won over hard-coded alternatives e.g.: SQL over CODASYL, Today we see Linked Data as delivering the "ad hoc" factor in "best of both worlds" fashion, relative to alternatives (including RDBMS), across the Web and/or within Intranets & Extranets.

• Scale of Instance Data - 10^9 - 10^11 Triples
• Scale of Ontology 100,000's of Classes
• Faceted Browsing, Text and Structure
• Deployment and Provisioning

• Up until now, you design the warehouse for the application, load the data, make a data island
• With Linked Data, the warehouse is self-filling, based on published data using terms from commonly shared vocabularies
• Virtuoso facilitates the above by integrated RDF-ization middleware; you populate the warehouse as you query, and system simply gets smarter in line with your natural work patterns.

• Having secrets does not mean using a secret language
• Private environments still benefit from common vocabularies and terms
• People and organizations publish anyway: Now it is about publishing for use in applications and integration, internet, extranet, intranet
• Linked data and Virtuoso deliver on the Data Spaces concept: Express any statement for which there is a vocabulary and the data exposed by the statement can be found, joined and processed (e.g. Meshups). Basically, The network is the database.

• Virtuoso 6, Single Server and Cluster Editions
• SPARQL and SQL With The Right Extensions for serious BI style analytics
• Integrated Web Services Platform, Suite of RDF-izers (Extractor & LOD Cloud Lookup variants)
• Server Hosted Facet Browsing Service (via REST API), Entity Ranking, Other Building Blocks for Web 2.0 Style Development

• 4.2 GTriples on 2 Commodity Servers
• Full Text and Structured Querying
• SPARQL End Point
• Faceted Browsing Interface for Quick Discovery and Simple Report Composition
• Usage Statistics across Source & Reference Graph IRIs, plus IFP and owl:sameAs usage stats
• VoiD Graph Providing Rich Description of hosted Data Sets

If OpenLink does not host it with enough capacity or the right data, you can procure your own infrastructure and get the software from us. From now on, anybody who chooses can be a search and analytics player.

• SPARQL Augmented With Run Time Inferencing
• Entity Ranks for Better Search
• Anytime Query Answering for Quick Approximate Results
• A User Interface Combining Discovery and Query Building
• Easy Web Services API's and SPARQL for Developing Applications

• No Materialization, Select Taxonomy At Query Time
• Query Optimization Knows About Class and Property Hierarchies

• Optionally Follow owl:sameAs links
• Optionally consider any two sharing an IFP to be the same
• No materialization, Control sameAs and IFP following at query time, at the triple pattern level
• For Ad Hoc, Do Identity at Run Time
• For Deep Analytics and Batch Processing, Normalize Identities at Load Time

• References and the Rank of the Referrer Contribute to Rank, as In Web Search
• Can Customize Weight By Graph, Predicate
• Can Run Ranks on Selected Subsets
• Ranks Are Calculated in a Batch Run

• Autocompletion of URI's
• Autocompletion of Label-Like Properties
• Ranked List of Synonyms
• Statistics on Where a URI is Defined and Where it is Referenced

• Partial Results in Fixed Time
• Useful for Interactive Browsing, Query Development over large data sets (e.g. LOD Cloud)
• On public SPARQL end points, Protects Against DOS, still giving samples of the answers
• Metering of query resource utilization

• Access via Web Services, SPARQL
• Developed in Virtuoso using SQL, SPARQL, and Stored Procedures
• Part of Virtuoso 6.x Open Source Edition (Single Server Edition only)

• If Data In Memory, Interactive Time and Linear Scale
• RDF Aware Query Optimizer is Key
• Parallel Execution Engine 1 Thread/Query/Partition
• For Generic Linked Data, RDF Representation With 4 Indices, plus Full Text Indexing on Literal Objects
• For Specialized Tasks, SQL + Stored Procedures With Parallel Programming Model (of course output will be Linked Data)
• Unlimited Cross Partition Joining, Near Full Platform Utilization, and not a problem with the right message flow

• Current Live Instance Setup: 2 Linux boxes with 2x4 core Xeons each with 32G RAM for a data set in excess of 4.2 Billion Triples
• 3.2 Million Single Triple Lookups Per Second
• Load Rates over 100K Triples/sec
• Entity Ranks for 4.2 GTriples in 30m/Iteration

Rent of Buy?

To Handle 10 GT 100% in RAM or 50 GT With Decent Working Set:

• For Intermittent Use, 1TB of RAM, 256 Virtual Cores at EC2 is $1228 Per Day
• For Purchase, Cluster of 1TB RAM, 120 Nehalem Cores Lists Around $75K

* April 2009 US retail prices

• Applications exploiting open data access across heterogenous data sources at Web Scale now within anyone's reach
• Usable for public Web Sites or for in-house Business Analytics
• Web enabled Open Data Access & Analysis for All!