Virtuoso Open-Source Wiki
Virtuoso Open-Source, OpenLink Data Spaces, and OpenLink Ajax Toolkit
Advanced Search
Help?
Location: / Dashboard / Main / UpgradingToVOS610

Virtuoso (Open Source and Commercial) Upgrade Notes

Introduction

  • Important: To upgrade from Release 4.x or earlier to Release 6.x or later, you must first perform an incremental upgrade to Release 5.x; then, upgrade from Release 5.x to current.
  • Always make sure the database has been properly shutdown and the transaction log (virtuoso.trx) is empty before performing any of the following updates/upgrades.
  • Before upgrading any database, it is always a wise precaution to make a proper backup.

Upgrading from Release 6.1.x to a newer Release 6.1.x

The database format did not change between various versions of Virtuoso 6.1.x, so from a database standpoint no particular steps need to be performed before upgrading to a newer version of Virtuoso 6.1.x.

Note: See special note below, for transition from Virtuoso 6.1.3 or earlier to Virtuoso 6.1.4 or later.

The only requirement is that you must properly shut down the database prior to installing the latest binaries, as the transaction logs may otherwise have a different version tag. In such case, the Virtuoso server will print a message like the following, and refuse to launch:

The transaction log file has been produced by server version
'06.01.xxxx'. The version of this server is '06.01.yyyy'. If the
transaction log is empty or you do not want to replay it then delete
it and start the server again. Otherwise replay the log using the
server of version '06.01.xxxx' and make checkpoint and shutdown
to ensure that the log is empty, then delete it and start using
new version.

Upgrading from Release 6.1.3 or earlier, to Release 6.1.4 or later

In Virtuoso versions before 6.1.4, some RDF data in the quad store could be stored in a way that could break the sequence of an index, causing wrong results to be passed back.

To fix this, we have added an automated check into 6.1.4 to detect whether this condition is present, and to fix the index when needed.

When a DBA starts the database with this newer Virtuoso binary, the following text will appear in the virtuoso session log:

  21:05:36 PL LOG: This database may contain RDF data that could cause indexing problems on previous versions of the server.
  21:05:36 PL LOG: The content of the DB.DBA.RDF_QUAD table will be checked and an update may automatically be performed if
  21:05:36 PL LOG: such data is found.
  21:05:36 PL LOG: This check will take some time but is made only once.

As noted, this check may take some time, depending on the number of stored quads.

If the check succeeds, the following message is entered in the log, and Virtuoso will flag the check as "done," so it will not affect subsequent restarts of the database server.

21:05:36 PL LOG: No need to update DB.DBA.RDF_QUAD

The database will continue to perform the startup routines and go into an online state.

If the condition is detected, however, the following message will appear in the log, and the Virtuoso server will refuse to start --

  21:05:36 PL LOG: An update is required.
  21:05:36 PL LOG:
  21:05:36 PL LOG: NOTICE: Before Virtuoso can continue fixing the DB.DBA.RDF_QUAD table and its indexes
  21:05:36 PL LOG:         the DB Administrator should check make sure that:
  21:05:36 PL LOG:
  21:05:36 PL LOG:          * there is a recent backup of the database
  21:05:36 PL LOG:          * there is enough free disk space available to complete this conversion
  21:05:36 PL LOG:          * the database can be offline for the duration of this conversion
  21:05:36 PL LOG:
  21:05:36 PL LOG:         Since the update can take a considerable amount of time on large databases
  21:05:36 PL LOG:         it is advisable to schedule this at an appropriate time.
  21:05:36 PL LOG:
  21:05:36 PL LOG: To continue the DBA must change the virtuoso.ini file and add the following flag:
  21:05:36 PL LOG:
  21:05:36 PL LOG:     [Parameters]
  21:05:36 PL LOG:     AnalyzeFixQuadStore = 1
  21:05:36 PL LOG:
  21:05:36 PL LOG: For additional information please contact OpenLink Support <support@openlinksw.com>
  21:05:36 PL LOG: This process will now exit.

Since the update will take a substantial amount of time and additional disk space depending on the size of the quad store, we have decided not to automatically start the update process. Instead, we hand control back to the DBA and let him decide when to perform this update. If the DBA wants to delay the update until a more appropriate time, he should restart with the previous binary as this latest binary will never start the database.

Once the DBA has checked the backups and disk space, and found an appropriate time-slot to run this update, he should edit the virtuoso.ini file and add the following flag:

[Parameters]
AnalyzeFixQuadStore = 1

Upon starting the Virtuoso server, the following messages will appear in the virtuoso.log file:

  21:05:57 PL LOG: This database may contain RDF data that could cause indexing problems on previous versions of the server.
  21:05:57 PL LOG: The content of the DB.DBA.RDF_QUAD table will be checked and an update may automatically be performed if
  21:05:57 PL LOG: such data is found.
  21:05:57 PL LOG: This check will take some time but is made only once.
  21:05:57 PL LOG:
  21:05:57 PL LOG: An update is required.
  21:05:57 PL LOG: Please be patient.
  21:05:57 PL LOG: The table DB.DBA.RDF_QUAD and two of its additional indexes will now be patched.
  21:05:57 PL LOG: In case of an error during the operation, delete the transaction log before restarting the server.
  21:05:57 Checkpoint started
  21:05:57 Checkpoint finished, log off
  21:05:57 PL LOG: Phase 1 of 9: Gathering statistics ...
  21:05:58 PL LOG:  * Index sizes before the processing: 002565531 RDF_QUAD, 002565531 POGS, 001171100 OP
  21:05:58 PL LOG: Phase 2 of 9: Copying all quads to a temporary table ...
  21:07:26 PL LOG: * Index sizes of temporary table: 001171100 OP
  21:07:26 PL LOG: Phase 3 of 9: Cleaning the quad storage ...
  21:07:51 PL LOG: Phase 4 of 9: Refilling the quad storage from the temporary table...
  21:09:17 PL LOG: Phase 5 of 9: Cleaning the temporary table ...
  21:09:41 PL LOG: Phase 6 of 9: Gathering statistics again ...
  21:09:41 PL LOG: * Index sizes after the processing: 002565531 RDF_QUAD, 002565531 POGS, 001171100 OP
  21:09:41 PL LOG: Phase 7 of 9: integrity check (completeness of index RDF_QUAD_POGS of DB.DBA.RDF_QUAD) ...
  21:10:00 PL LOG: Phase 8 of 9: integrity check (completeness of primary key of DB.DBA.RDF_QUAD) ...
  21:10:17 PL LOG: Phase 9 of 9: final checkpoint...
  21:10:20 Checkpoint started
  21:10:22 Checkpoint finished, log off
  21:10:22 PL LOG: Update complete.

If the update process detects any problem, it will put some debug output into the virtuoso.log and exit. At this point the DBA is advised to remove the virtuoso.trx file and contact OpenLink Support.

After the update process has completed, the database is left in an online state.

Upgrading from Release 6.0.x to Release 6.1.x

The database file format has not changed, but the introduction of a newer RDF index requires you run a script to upgrade the RDF_QUAD table. Since this can be a lengthy task and take extra disk space (up to twice the space used by the original RDF_QUAD table may be required during conversion) this is not done automatically on startup.

After upgrading the binary, you cannot perform any SPARQL queries until the new RDF_QUAD table is installed. The steps for this are:

  1. Shut down the database and verify the .trx file is empty.
  2. Check to make sure you have enough disk space to upgrade the RDF_QUAD table.
  3. Check to make sure you have a proper backup of the database.
  4. Edit virtuoso.ini and disable VADInstallDir and optionally the [HTTPServer] section for duration of the upgrade.
  5. Start up the database.
  6. Connect with isql to your database and run the libsrc/Wi/clrdf23.sql script. Depending on the number of quad records this can take several hours. Once the conversion is complete, the database will shut itself down.
  7. Edit virtuoso.ini and re-enable the VADInstallDir and, if disabled above, the [HTTPServer] section.
  8. Start the database as normal.

Upgrading from Release 5.x to Release 6.1.x

The database file format has substantially changed between Virtuoso 5.x and Virtuoso 6.x. To upgrade your database, you must dump all data from the Virtuoso 5.x database and re-load it into Virtuoso 6.x.

***NOTE*** Previous to Virtuoso 6.1.0, Virtuoso 6.x binaries would try to work against Virtuoso 5.x database files. This can cause irreparable corruption. Please proceed with caution.

Regular Virtuoso 5.x RDBMS tables can be dumped with the dbdump tool, producing scripts that can be replayed using the isql tool.

For the Virtuoso 5.x RDF_QUAD table, we have a set of stored procedures that will dump graphs into a set of backup files, which can then be reloaded into the Virtuoso 6.x database. Contact us at <vos.admin@openlinksw.com> for more info.

If you attempt to use a Virtuoso 5.x database file with a Virtuoso 6.1.x or later server, the server will print a message like that below, and refuse to launch:

The database you are opening was last closed with a server of
version xxxx. The present server is of version yyyy. This server
does not read this pre-6.0 format.

Upgrading from Release 5.x to a newer Release 5.x

The database format did not change between various versions of Virtuoso 5.x, so from a database standpoint no particular steps need to be performed before upgrading to a newer version of Virtuoso 5.

The only requirement is that you must properly shut down the database prior to installing the latest binaries, as the transaction logs will otherwise have a different version tag. In such case, the Virtuoso server will print a message like the following, and refuse to launch:

The transaction log file has been produced by server version
'05.00.xxxx'. The version of this server is '05.00.yyyy'. If the
transaction log is empty or you do not want to replay it then delete
it and start the server again. Otherwise replay the log using the
server of version '05.00.xxxx' and make checkpoint and shutdown
to ensure that the log is empty, then delete it and start using
new version.

Upgrading from Release 4.x or earlier, to Release 5.x

If you have Virtuoso Release 4.x or earlier, with or without ODS, preserving your data while upgrading your installation and databases to Release 5.x or later requires the steps described below.

Important: To upgrade from Release 4.x or earlier to Release 6.x or later, you must first perform an incremental upgrade to Release 5.x; then, upgrade from Release 5.x to current.

If you do not have any old data, you can skip this process, and simply use the new demo database, which has all the current ODS applications pre-installed.

Virtuoso binary components

  1. Shut down the previous server.
  2. Start the Release 5.x server executable you have built or downloaded into the same directory.
  3. There is a change in the DAV metadata represented in the RDF store. To update everything, log in as dba with isql and execute:

    SQL> dav_replicate_all_to_rdf_quad (1);

    This is a one-time manual update which changes the DAV graph name, removing its trailing slash.

ODS applications and data

To upgrade the ODS applications and preserve any data you may have in them, do the following:

  1. Shut down the server.
  2. Take a backup of the database file(s), just in case.
  3. Start the server.
  4. Re-install/upgrade the VAD package corresponding to each installed application. For the Open Source Edition, these are found in the /share/virtuoso/vad/ directory in the tree where make install copies the results of the build; for Commercial Edition, they are found in the vad directory. Updated VAD packages for both Commercial and Open Source Editions can be downloaded from our website.
  5. Upgrade the installation through the Conductor (recommended), or by executing the following command in isql:

    SQL> vad_install('ods_framework_dav.vad', 0);

  6. Repeat for each installed ODS or other VAD package.
Powered By Virtuoso