OpenLink Virtuoso™ - Next Generation Virtual Database Engine
Virtuoso is a revolutionary, next generation, high-performance virtual database engine for the Distributed Computing Age. It is an essential universal data access middleware technology set to accelerate our advancement towards the emerging Information Age.
Virtuoso provides transparent access to your existing data sources, which are logical references to databases from different database vendors, exposed by data access drivers also provided by different vendors.
Through a single connection, Virtuoso will simultaneously connect your ODBC, JDBC, UDBC, OLE-DB client applications and services to data within Oracle, Microsoft SQL Server, DB/2, Informix, Progress, CA-Ingres, Sybase, PostgresSQL, Solid, Velocis and other ODBC compliant database engines.
Virtuoso provides the end-user or applications developer with a one application or development environment to many database engines relationship abstracting either party from the complexities of heterogeneous data access.
Virtuoso allows end-users and application developers to retain a single data source or database focus, at the same time exposing either party to the benefits of heterogeneous data access, without introducing proportional increases in complexity.
Virtuoso is a type 10 VDB Engine, exposing its services via standards based data access interfaces such as ODBC, JDBC, UDBC, and OLE-DB and performing external Data I/O occurs via ODBC, UDBC, or Native data access interfaces.
Virtuoso has been developed with the following goals in mind:
- Operating System Independence – support for all main stream operating systems
- Small Memory Footprint – no more than 2MB of memory for basic operations
- Small Binary Distribution – maximum of 10MB of disk space for base product installation
- High-Performance – deliver performance levels required by enterprise-wide solutions
- Standards Based Data I/O – use standards such as ODBC from Microsoft and UDBC from OpenLink Software for low-level data access
- Standards Based Interfaces to Services – expose Virtual Database functionality via ODBC, UDBC, JDBC, and OLE-DB data access drivers
- Web Based Configuration & Management – a Web Browser as the standard administration and configuration interface.
Virtuoso’s VDB type 10 architecture is depicted below.
Figure 13 - OpenLink Virtuoso™ VDB Architecture
Note: support for JDBC and OLE-DB based external data I/O is still being developed.
ODBC, JDBC, UDBC, and OLE-DB Drivers for Virtuoso are an integral part of the OpenLink Virtuoso™ product set. Client applications and services consume the virtual database services provided by Virtuoso using one or more of these drivers.
The features of these drivers are listed below:
- Blistering Performance
- ODBC Drivers are ODBC v2.5 compliant
- ODBC Drivers implement driver based Scrollable Cursors
- Drivers for JDBC are JDBC v1.02, v1.1, and v2.0 compliant
- Drivers for JDBC implement driver based Scrollable Cursors
- OLE-DB Data Providers are OLE-DB 2.0 compliant.
Virtuoso’s security features are listed below:
- User Login and Password Verification
- Role Definition and Privilege Control
- Table and Column Level Privilege Control
- Logical "Rules Based" Security - enabling security to be devised around custom rules that derive security conditions from a combination of user, client applications, client operating system, and network address parameters, if required
- Data Encryption – data transmission between Virtuoso
and its clients can be secured via encryption
Virtuoso’s query manager features are as follows:
- Supports ANSI SQL 92
- Parallel Query processing – enabling parallel execution plan assembly, query execution, and query fulfillment.
- Asynchronous Operation – all activities occur asynchronously except for transaction commits and rollbacks which occur synchronously.
- Distributed Join Optimizer - enabling high-performance heterogeneous joins across disparate database engines.
- Scrollable Cursor Engine - facilitates scrollable cursor support across heterogeneous database engines. This mechanism allows the manipulation of ResultSet subsets call RowSets that span multiple data sources and database engines. This sophisticated featured reduces record fetch overhead by migrating chunks of data in parallel from external data sources into Virtuoso and then out again to client applications and services. Virtuoso supports Static, Keyset, Mixed, and Dynamic Cursors.
- Heterogeneous VIEWS support - enabling the development of logical data snapshots that transcend disparate database engines. These VIEWS aren’t updateable
- Generic Stored Procedure Language – enables the development of stored procedures that allow application logic to be stored and implemented at the VDB level, these stored procedures can reference tables hosted in disparate database engines. Virtuoso Stored Procedures are sensitive to metadata changes that occur within externally linked tables.
The Virtuoso Metadata Manger acts as the repository of all Virtuoso’s database objects, this includes Metadata from external tables linked into Virtuoso, Native Tables, Stored Procedures, Views, Object Privileges, Indexes, Referential Integrity constraints, and Database Statistics.
The Metadata Manager does not affect any of the external tables linked into Virtuoso. Thus, an external object dropped by the Metadata Manager simply equates to a reference drop within Virtuoso as opposed to an actual object drop within an external database linked into Virtuoso.
The Virtuoso transaction manager possesses the following capabilities:
- Commits & Rollbacks – Transactions are either fully completed or not at all, when dealing with externally linked tables this behavior is preserved, but not to the point of a full 2-phase commit, this functionality will be available in a subsequent product release
- Supports read-only transactions, which provide no lock overhead and add flexibility to the VDB transaction model.
- Before & After Image Transaction Logging – Virtuoso maintains database state logs before and after transactions. Thus, in time of failure, it is possible to Roll the database back to a prior state preceding VDB server initialization. Virtuoso also allows you to roll forward your database by committing all uncommitted transactions held in After Image transaction logs when recovering from system failure.
- Supports Dirty Read, Read Committed, Repeatable Read, and Serializable transaction isolation levels
The concurrency manager ensures that Virtuoso is capable of supporting multiple concurrent database sessions that execute data INSERTS, UPDATES and DELETIONS access without compromising data integrity or implicitly increasing application or service latency.
The Virtuoso Concurrency Manager supports Optimistic and Pessimistic (Locking) concurrency.
Database Rows, Pages, or Tables aren’t locked when Optimistic concurrency is in use, rather UPDATES or DELETIONS are simply rejected on the bases of other user process modifying the same record(s) that you are about to UPDATE or DELETE. Optimistic concurrency presumes low record UPDATE and DELETE frequency.
Pessimistic concurrency on the other hand presumes a high level frequency of record UPDATE and DELETION activity. The Virtuoso concurrency manager implements Page Level locking, and these locks have configurable timeout settings, which reduce the probability of record deadlock which inadvertently introduce application latency.
Row level locking is currently in development.
Virtuoso posses full native database engine capabilities, the local I/O Manager is responsible for writing data to local storage when Virtuoso is being used as an Embedded Database Engine. This feature of Virtuoso enables it to serve Application Server solutions as an embedded database engine outside the corporate firewall, providing these solutions with local storage at the same time reducing workload on the corporate databases behind the company firewall.
Virtuoso reads and writes data to external database engines using ODBC or UDBC as its external data I/O interface (supported for JDBC and OLE-DB based I/O is also planned).
ODBC support enables Virtuoso to link and manipulate data held in ODBC compliant database engines, implying any database with an ODBC driver is accessible from Virtuoso.
UDBC support enables Virtuoso to be hosted on operating system where ODBC is not’t supported, it also enables Virtuoso to communicate with external databases using their native interfaces.
The Virtuoso Replication Manager enables the automatic synchronization of data across Virtuoso Servers within your computing infrastructure. The replication process is bi-directional and is capable of replicating data across externally linked tables.
Synchronization of replicated databases is automatic should one of the database within a replica group encounter system failure.
This is a HTML based Administrative console that provides a graphical user interface for attaching and detaching external tables associated with your Virtuoso database engine. It also includes configuration wizards and a query editor that enables intuitive configuration and testing of the ODBC and UDBC data sources names that you plan to use with Virtuoso.
Implementation Issues Checklist
Key
| Implemented – I | Partially Implemented - PI | Planned - P |
VDB Implementation Issue |
Status |
| ODBC, UDBC, JDBC, OLE-DB based interface to VDB | I |
| ODBC & UDBC based external data I/O | I |
| JDBC based external data I/O | P |
| OLE-DB based external data I/O | P |
| SQL Support | I |
| OQL Support | P |
| Query Processor | I |
| Heterogeneous Scrollable Cursor Support | I |
| Standard Data Types Support | I |
| VIEW Support | I |
| Stored Procedure Support | I |
| Concurrency Control | I |
| Transaction Isolation | I |
| Distributed Transaction Support | PI |
| User Definable Type Support | I |
| Federated Database Support | I |
| Distributed Database Support | I |
| Security | I |
| Replication Manager | PI |
Feature
& Benefits Analysis (Virtual Database Functionality)
| Feature | Benefit |
| Transparent concurrent access to heterogeneous data sources and backend database engines. | Cost effective mechanism for harnessing information from your existing databases and data sources in real-time. |
| ODBC, JDBC, UDBC, and OLE-DB Driver availability. | Provides exposure of virtual database services to the plethora of commercial ODBC, JDBC, UDBC, and OLE-DB based solutions currently in existence. |
| ODBC, UDBC, or Native external Data I/O. | Freedom to Mix & Match the best combination of database engines and data access types for your organization |
| Cross platform support (95/98/NT, Linux, Solaris, HP-UX, AIX, Digital UNIX, IRIX, SCO Unix, SCO Unixware, DG_UX, Dynix/PTX, BSDI, with VMS, MacOS, OS2, OS400, OS390 planned). | Freedom to Mix & Match the best combination of Application and Database server operating systems for your organization. |
| Multiple database and ODBC data
source name support: Virtuoso Native DBMS Server, Oracle(6/7/8), MS SQL Server(4/6.x/7.x), Informix(4/5/6/7), Progress (6/7/8/9), CA-Ingres (6.4), CA-OpenIngres (1.1/2.x), DB2 (2.12 & 5.x), Sybase SQL Server (4.x/10.x/11.x), PostgresSQL, Solid, Velocis, and third party ODBC Drivers. |
Sustains or bolsters database
independence within your organization. This also ensures that new database driven technologies are usable with your existing database engines and data sources, thereby preserving database investments made in prior years. |
| Multi Threaded Virtual Database Server. | Parallel execution of data access instructions with minimal system overhead resulting in high-performance data throughput even when concurrent user counts are high. |
| Asynchronous Data I/O. | All data I/O except transaction commits and rollbacks are Asynchronous. Thus, reducing the probability of data I/O blocks that increase VDB server latency. |
| Scrollable Cursors Support. | Enables the use of High-Level data access interfaces such as RDO, ADO, OLE-DB, JDBC 2.0 against your existing databases and data sources. |
| Sophisticated Concurrency Control. | Provides scalability to your VDB based solutions. |
| Stored Procedure Support. | You can develop database independent stored procedures that can traverse all the databases in your organization using common stored procedure language. |
| Sophisticated rules based security. | You can control user privileges, define roles, and generally tighten security by adopting a "Rule Book" approach. This allows your security infrastructure be both logical and physical in nature, and driven by rules that you define. |
| Centralized or Decentralized infrastructure management via Web Browser | You can manage one or more
geographically dispersed Virtuoso Servers from a single location or multiple locations
through your Web Browser. Zero Admin solution. |
| Bi-Directional Heterogeneous Replication | Allows numerous Virtuoso and
non-Virtuoso database tables to be kept in sync throughout your distributed computing
infrastructure without user intervention. Reduces the cost typically associated with data transformation and migration. It also enables the development of sophisticated data caching for store-and-forward solutions. |
| Full Relational and Object-Relational Database functionality | Flexibility to build database
solutions using relational or object-relational models. Future proofs your applications development at the data access level. |
| Small footprint | Can operate within 2MB of RAM
and consumes no more than 10MB of disk space for entire product installation. This makes Virtuoso embeddable with solutions that are hosted on devices that have minimal memory and persistent storage capacity. |
Virtuoso is available in 3 distinct product formats:
- OpenLink Virtuoso™ Lite Edition
- OpenLink Virtuoso™ Workgroup Edition
- OpenLink Virtuoso™ Enterprise Edition
OpenLink Virtuoso™ Lite
Edition
This is an entry-level product comprising the following components:
- HTML based Interactive ODBC compliant Query Console
- Virtual Database Server
- Virtuoso Conductor – HTML based remote database management console
- OpenLink Virtuoso™ Single Tier ODBC & UDBC Drivers
- OpenLink Virtuoso™ Type 4 Drivers for JDBC.
- Lightweight Thread Manager.
OpenLink Virtuoso™ Lite presumes the existence of ODBC or UDBC drivers within your existing IS infrastructure. It only includes ODBC and JDBC drivers for accessing the Virtual Database Server. It is important to note that prior to using Virtuoso, you must install and configure database specific network communications software and ODBC drivers provided by each of the database vendors whose products you will be exposing to Virtuoso.
Examples of such database vendor provided products include: Oracle SQL*Net, Oracle Net8, Informix I-Connect, Ingres NET, Microsoft NETLIB, Sybase Open Client, Progress Client Networking, DB2 client enablers etc. Each of these products typically includes a free ODBC Driver.
This product set supports the following operating systems:
Windows 95/98/NT, Linux, Solaris (Sparc & x86), AIX, HP-UX, IRIX, Dynix/PTX, DG-UX, Digital UNIX. MacOS, OS/2, and VMS ports are planned.
OpenLink Virtuoso™
Workgroup Edition
This is a departmental or small Internet/Intranet/Extranet server solution comprising the following components:
- HTML based Interactive ODBC compliant Query Console
- Virtual Database Server
- Virtuoso Conductor – HTML based remote database management console
- OpenLink Virtuoso™ Single Tier ODBC & UDBC Drivers
- OpenLink Virtuoso™ Type 1, 2, and 4 Drivers for JDBC.
- OpenLink Data Access Driver Suite (Lite Edition) supporting: Oracle, Microsoft SQL Server, Informix, Sybase, CA-Ingres, Progress, DB2, PostgresSQL, Velocis, and Solid
- Lightweight Thread Manager.
OpenLink Virtuoso™ Workgroup Edition includes OpenLink Software’s high-performance Single Tier ODBC & JDBC Drivers (a.k.a. OpenLink Lite). These drivers enable you to expose your existing database engines to Virtuoso right out of the box. It is important to note that prior to using Virtuoso, you must install and configure database specific network communications software and ODBC drivers provided by each of the database vendors whose products you will be exposing to Virtuoso.
Examples of such database vendor provided products include: Oracle SQL*Net, Oracle Net8, Informix I-Connect, Ingres NET, Microsoft NETLIB, Sybase Open Client, Progress Client Networking, DB2 client enablers etc. Each of these products typically includes a free ODBC Driver.
This product set supports the following operating systems:
Windows NT, Linux, Solaris (Sparc & x86), AIX, HP-UX, IRIX, Dynix/PTX, DG-UX, Digital UNIX. MacOS, OS/2, and VMS ports are planned.
OpenLink Virtuoso™ Enterprise Edition
This is an enterprise wide or large Internet/Intranet/Extranet server solution comprising
the following components:
- HTML based Interactive ODBC compliant Query Console
- Virtual Database Server – Implemented as an OpenLink Request Broker service
- Virtuoso Conductor – HTML based remote database management console
- OpenLink Virtuoso™ Multi-Tier ODBC & UDBC Drivers
- OpenLink Virtuoso™ Type 1,2,3 & 4 Drivers for JDBC
- OpenLink Data Access Drivers Suite (Multi-Tier Edition) supporting: Oracle, Microsoft SQL Server, Informix, Sybase, CA-Ingres, Progress, DB2, PostgresSQL, Velocis, Solid, and 3rd Party ODBC Drivers
- Heavyweight POSIX or Native Thread Manager.
OpenLink Virtuoso™ Enterprise Edition includes OpenLink Software’s high-performance Multi-Tier ODBC & JDBC Drivers. These drivers enable you to expose your existing database engines to Virtuoso right out of the box, without inherent dependencies on database vendor provided network software for remote database connections.
Examples of such database vendor provided products include: Oracle SQL*Net, Oracle Net8, Informix I-Connect, Ingres NET, Microsoft NETLIB, Sybase Open Client, Progress Client Networking, DB2 client enablers etc.
This product set supports the following operating systems:
Windows NT, Linux, Solaris (Sparc & x86), AIX, HP-UX, IRIX, Dynix/PTX, DG-UX, Digital UNIX. MacOS, OS/2, and VMS ports are planned.
Galvanizing
Your Enterprise with OpenLink Virtuoso™
Virtuoso enables you to maximize the benefits of new distributed computing technologies and paradigms with minimum disruption, if any, to your existing computing infrastructure. The situation examples in the sections that follow demonstrate how Virtuoso fulfills this value proposition.
Your organization seeks to empower its knowledge workers by providing transparent access to corporate information so that they can make timely and accurate decisions.
- Your organizations core business solutions are all driven by database engines provided by different database vendors
- The disparate application databases are hosted on a range of different database server machines running different operating systems
- To maximize the uptake of this initiative, the knowledge workers must be abstracted from the intricacies of heterogeneous data access.
- Information must be available in real-time
- Information must be accurate
- Knowledge workers must be able to manipulate information and make decisions using their existing "off the shelf" desktop productivity tools.
- The data residing in disparate data sources needs to be pulled together in real time to produce information
- High throughput transmission of information culled from disparate application databases to desktop productivity tools being used by the knowledge workers.
Implement a 3-tier distributed computing architecture utilizing ODBC, JDBC, or OLE-DB for data access and OpenLink Virtuoso™ as your "Data Junction Box".
The infrastructure required in order to fulfill the objectives set would be assembled as follows:
- Creation of "Data Junction Box" using a OpenLink Virtuoso™, that holds references to tables hosted by your application database engines
- Hosting one or more of your OpenLink Virtuoso™ servers on one or more designated server machines. These machines could be dedicated application servers or one of the many database servers currently in use
- Creation of ODBC or UDBC data source names on the same machine as the OpenLink Virtuoso™ server. These data sources names will connect OpenLink Virtuoso™ to the different application databases within your computing infrastructure
- Creation of ODBC, JDBC, or OLE-DB data source names that reside on the desktop machines used by your knowledge workers. The data source names created will be used to connect ODBC, JDBC, and OLE-DB based desktop productivity tools to your OpenLink Virtuoso™ servers.
Once the infrastructure has been assembled in the manner described in the previous section, the stage is set for your knowledge workers to commence exploiting the benefits of transparent access to information culled from heterogeneous data sources. The entire process occurs via the use of their chosen "off the shelf" desktop productivity tools.
The entire solution, its infrastructure, and constituent components are depicted below.
Figure 14 - Virtuoso Empowering Knowledge Workers
Note: Databases depicted could be replaced with any ODBC compatible database engine.
Improving Customer & Prospect
Interaction via Your Corporate Web Site
Your organization seeks to increase customer and prospect intimacy by implementing a database driven web site. The site offers customers and prospects online support, quotations, order processing, and product information.
- Your organizations core business solutions are all driven by database engines provided by different database vendors
- The disparate application databases are hosted on a range of different database server machines running different operating systems
- Security implications of corporate database exposure to the Internet
- Availability of "Web to Database" middleware for your Web Server.
- Data transmission throughput between internal databases and web pages
- Scalability of Web Servers, underlying "Web to Database" middleware, and participating database engines
- Information accuracy
- Data updates by site visitors must be routed to the appropriate internal database(s).
- The data residing in disparate data sources needs to be pulled together in real time to produce information
- Delivery of high-performance information access to numerous concurrent site visitors
- Availability of operating system independent "Web to Database" middleware, providing you with the freedom to chose the most suitable operating system for the job at hand.
Implement a 3-tier distributed computing architecture utilizing ODBC, JDBC, or OLE-DB for data access and OpenLink Virtuoso™ as your "Data Junction Box".
The infrastructure required in order to fulfill the objectives set would be assembled as follows:
- Creation of "Data Junction Box" using a OpenLink Virtuoso™, that holds references to tables hosted by your application database engines
- Hosting one or more of your OpenLink Virtuoso™ servers on one or more designated server machines. These machines could be dedicated application servers, the same machine hosting your Web Server, or one of the many database servers residing behind your corporate firewall
- Creation of ODBC or UDBC data source names on the same machine as the OpenLink Virtuoso™ server. These data sources names should point to the different application databases within your computing infrastructure.
- "Web to Database" middleware and Application Server Development kit selection.
- Creation of ODBC, JDBC, or OLE-DB data source names on your Web Server for use by your "Web to Database" middleware for connectivity to OpenLink Virtuoso™
- Use the OpenLink Virtuoso™ Stored Procedure language to implement the data access and manipulation aspects of your application logic.
Once the infrastructure has been assembled in the manner described above, the stage is set for you to develop and implement your database driven web. The application development process retains a single as opposed to multiple database focus due to the sophisticated VDB Engine functionality provided by OpenLink Virtuoso™ without compromises in performance, security, or scalability.
The entire solution, its infrastructure, and constituent components are depicted in the illustration that follows.
Figure 15 - Internet Application Server Solution
NEXT | PREVIOUS | PDF VERSION