"How Do I Install Virtuoso Docker Container Image?"

<b>1</b>. Choose Docker image
We've put together three Docker Images to choose from <a href="https://hub.docker.com/u/openlink/">https://hub.docker.com/u/openlink/</a>. The Enterprise Edition images require a version-matched license file (about which more details are below), just as if you were running on a local machine or in the cloud; the Open Source (VOS) image does not.
     
	    • <a href="https://hub.docker.com/r/openlink/virtuoso-closedsource-8/">Enterprise Edition 8.3 - openlink/virtuoso-closedsource-8 </a>
      
	    • <a href="https://hub.docker.com/r/openlink/virtuoso-closedsource-7/">Enterprise Edition 7.2 - openlink/virtuoso-closedsource-7</a>
     
	    • <a href="https://hub.docker.com/r/openlink/virtuoso-opensource-7/">Open Source (VOS) 7.2 - openlink/virtuoso-opensource-7</a>

The remainder of this article is focused on the first of these (Enterprise Edition 8.3); most of the content applies to all three, with minor, hopefully obvious, adjustments.

"How Do I Install Virtuoso Docker Container Image?"

<b>2</b>. Download Docker image

To pull the latest Virtuoso 8.2 docker image to your local system, you can use the following command:

<code>$ docker pull openlink/virtuoso-closedsource-8</code>

To check the version of the Virtuoso binary, you can use the following command:

<code>$ docker run openlink/virtuoso-closedsource-8 version</code>

This Docker image is using the following version of Virtuoso:

<pre>OpenLink Virtuoso Universal Server (Enterprise Edition)
Version 08.03.3319-pthreads as of Sep  9 2020 (b5a3cfa1e0)
Compiled for Linux (x86_64-generic-linux-glibc25)
Hosted Runtime Environments:  VDB
Copyright (C) 1998-2020 OpenLink Software</pre>

"How Do I Install Virtuoso Docker Container Image?"

<b>3</b>. Creating a sample Virtuoso Docker instance

Here is a quick example of how to create a new virtuoso instance on your system:
<pre>
$ mkdir my_virtdb
$ cd my_virtdb
$ docker run \
    --name my_virtdb \
    --interactive \
    --tty \
    --env DBA_PASSWORD=mysecret \
    --publish 1111:1111 \
    --publish  8890:8890 \
    --volume `pwd`:/database \
    openlink/virtuoso-closedsource-8:latest
</pre>
This will create a new Virtuoso database in the my_virtdb subdirectory and starts a Virtuoso instance with the HTTP server listening on port 8890 and the ODBC 2 / JDBC / ADO.Net / OLE-DB / ISQL data server listening on port 1111.

The docker image in running in foreground (with -i or --interactive) mode, so you can see what it is doing.

You should now be able to contact the Virtuoso HTTP server using the following URL:

<code>http://localhost:8890/</code>

You can shut down Virtuoso by pressing the CTRL and C buttons in that terminal session.

"How Do I Install Virtuoso Docker Container Image?"

<b>4</b>. Licensing

If the Virtuoso Enterprise Edition binary cannot find a license, it will start with a restrictive courtesy license 1 license which allows a small number of concurrent ODBC 2 / JDBC / ADO.Net / OLE-DB / ISQL connections, with a low limit on the maximum number of rows that can be retrieved in a query result-set, and terminates the instance after 10 minutes of use.

To unlock the full potential of Virtuoso, you can obtain a FREE Evaluation License via our <a href="https://shop.openlinksw.com/license_generator/virtuoso/">License Generator Web Service</a>.

Naturally, following successful evaluation, you can choose from a variety of Enterprise Edition licenses from our online shop 5 or contact us (or your account manager) directly for more specialized Enterprise and/or VAR/ISV/OEM licenses.

Place the resulting virtuoso.lic file in the newly created database/ directory alongside the virtuoso.ini on your local filesystem, so the docker image can pick it up on the next startup:

<pre>
$ docker cp virtuoso.lic my_virtdb:/database
$ docker stop my_virtdb
$ docker start my_virtdb
</pre>

"How Do I Install Virtuoso Docker Container Image?"

<b>5</b>. Passwords

When a new database is created, the docker image will use the Environment settings DBA_PASSWORD and DAV_PASSWORD to set passwords for the dba and dav user accounts.

If the DBA_PASSWORD environment variable is not set, a random password will be assigned to the dba user account, and stored on the internal docker filesystem as /settings/dba_password.

If the DAV_PASSWORD Environment variable is not set, it will be set to the DBA_PASSWORD and stored as /settings/dav_password.

These files will only be readable by the user that started the image. Commands like the following may be used to reveal the randomised passwords:

<code>$ docker exec -i -t my_virtdb cat /settings/dba_password</code>

Without this password, you will not be able to log in to the dba account using either the isql tool or the Virtuoso Conductor.

NOTE: Users are advised to immediately change the password and then remove this file from the filesystem.

"How Do I Install Virtuoso Docker Container Image?"

<b>6</b>. Persistent storage

In order to retain changes to the Virtuoso database, the database documents should be stored on the host file system.

The docker image exposes a /database volume that can be easily mapped to a local directory on the filesystem. If this directory is empty, the docker image will put an initial virtuoso.ini into the mapped directory and then proceeds to create a new database.

"How Do I Install Virtuoso Docker Container Image?"

<b>7</b>. Stopping the image

When the docker image is running in foreground mode (with -i or --interactive), you can shut down Virtuoso by pressing the CTRL and C buttons in that terminal session. You can also use the following command on a different terminal:

<code>$ docker stop my_virtdb</code>

"How Do I Install Virtuoso Docker Container Image?"

<b>8</b>. Restarting the image

Once the docker image has been registered with the docker run or docker create command on your local system, you can start it in the background using:

<code>$ docker start my_virtdb</code>

If you prefer to run the instance in foreground mode, you can use:

<code>$ docker start -i -a my_virtdb</code>

"How Do I Install Virtuoso Docker Container Image?"

<b>9</b>. Checking the startup log

If the docker image is started in background (without -i or --interactive) mode, you can look at the recent output of the virtuoso process by running:

<code>$ docker logs my_virtdb</code>

"How Do I Install Virtuoso Docker Container Image?"

<b>10</b>. Using isql to connect

To connect to your running Virtuoso instance, you can use the following command:

<code>$ docker exec -i my_virtdb isql 1111</code>

You will be prompted for the dba account password.

NOTE: If you provide an incorrect password multiple times, Virtuoso will lock the dba account for a couple of minutes.

"How Do I Install Virtuoso on Windows?"

<b>1</b>. Select the Virtuoso release number and Windows operating system required from the download site 8 and click on the Find Downloads button. Note: Virtuoso 7.x and above is 64-bit only, and requires Vista or Windows Server 2003, or later.

"How Do I Install Virtuoso on Windows?"

<b>2</b>. Download the required Virtuoso Client and/or Client & Server installer archives from the list presented.

"How Do I Install Virtuoso on Windows?"

<b>3</b>. If you don't not already have a license file, one can be obtained by clicking on the Generate Evaluation License button in step 1.

"How Do I Install Virtuoso on Windows?"

<b>4</b>. In addition, a copy of the license file will also be stored in your OpenLink Data Spaces Briefcase 1 folder.

"How Do I Install Virtuoso on Windows?"

<b>5</b>. Note - the file must be named  virtuoso.lic for use. Some browsers may rename the file during download; you can simply rename the file on your machine .

"How Do I Install Virtuoso on Windows?"

<b>6</b>. Double-click the installer archive,  wavpz2zz , to start the process with the Installer Program Introduction.

"How Do I Install Virtuoso on Windows?"

<b>7</b>. Click  Next , and the License Agreement dialog is presented:

"How Do I Install Virtuoso on Windows?"

<b>8</b>. Tick the box for  I accept , and click  Next . Use the  Browse button to select the location for the installation or if the default location is acceptable click the  Next button.

"How Do I Install Virtuoso on Windows?"

<b>9</b>. Use the  Browse button to locate the license file you obtained earlier then, click  Next . The dialog to Select the Components to be installed is presented:

"How Do I Install Virtuoso on Windows?"

<b>10</b>. Select the of the component types to install:
Full -“ installs both Client and Server components

Client - Installs the Client components only

Server - Installs the Server components only

"How Do I Install Virtuoso on macOS?"

<b>1</b>. Download the Virtuoso 8.x DMG archive (mwvpz2zz.dmg) from the download site (<a href="https://download3.openlinksw.com/uda/virtuoso/8.3/universal-apple-macosx10.9-64/mwvpz2zz.dmg">Intel</a>, <a href="https://download3.openlinksw.com/uda/virtuoso/8.3/universal-apple-macosx10.9-64/mwvpz2zz.dmg">M1</a>).  Note:Virtuoso 8.x requires OS X Lion (10.9) or later.

"How Do I Install Virtuoso on macOS?"

<b>2</b>. Open the  mwvpz2zz.dmg file, and drag the  Virtuoso 8.3.app into your  Applications folder or other preferred location.

"How Do I Install Virtuoso on macOS?"

<b>3</b>. Once the copy is complete, you can  Eject the  Virtuoso 8.3 disk, and discard or retain the mwvpz2zz.dmg file as you prefer.

"How Do I Install Virtuoso on macOS?"

<b>4</b>. If you haven't already obtained a suitable Virtuoso 8.x license file, return to the <a href="https://shop.openlinksw.com/license_generator/">download site</a>. Click on the Generate Evaluation License button from step 1 to obtain a Free 30 Day Evaluation License will be generated and emailed to your email account.

"How Do I Install Virtuoso on macOS?"

<b>5</b>. In addition, a copy of the license file will also be stored in your OpenLink Data Spaces Briefcase 3 folder.

"How Do I Install Virtuoso on macOS?"

<b>6</b>. Double-click the  Virtuoso 8.3.app icon in the Applications folder to trigger installation and startup of helper tools - the  Virtuoso Monitor and  OpenLink License Manager .

"How Do I Install Virtuoso on macOS?"

<b>7</b>. Locate the  Virtuoso Monitor in your menu bar -“ it's the  V icon toward the right.

"How Do I Install Virtuoso on macOS?"

<b>8</b>. You may also choose to automatically  Start Virtuoso Monitor at Login , or you may prefer to manually start it when needed (by double-clicking  Virtuoso 8.3.app , as you just did).

"How Do I Install Virtuoso on macOS?"

<b>9</b>. In the Finder's  Go menu, select  Go to Folder... , and input '/Library/Application Support/OpenLink/Licenses`

"How Do I Install Virtuoso on macOS?"

<b>10</b>. Drag the license file ( virtuoso.lic ) you downloaded above into this folder.

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>1</b>. Download the Installer Archive (e.g., Linux Installer Archive 5) to a designated installation directory.

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>2</b>. Un-tar the file to obtain the install script and archive file using a command like this, where XX specifies the OS identifier (e.g., l9 = Linux glibc25 x86_64, sv = Solaris 2.10 x86_64, etc.): 
	
	<code>tar xvf XXvpz2zz.tar</code>

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>3</b>. If you don't not already have a license file, one can be obtained by clicking on the Generate Evaluation License button in step 1.

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>4</b>. In addition, a copy of the license file will also be stored in your OpenLink Data Spaces Briefcase 1folder.

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>5</b>. If a license file exists, place this in the same directory as the installation files, and it will automatically be applied during installation. If upgrading an existing Virtuoso instance, be sure to take a backup of your database file and shut down the existing instance before proceeding.

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>6</b>. Run the install script using the following command:

<code>$  sh install.sh</code>

<blockquote>
- Extracting Virtuoso Universal Server v8.3

- Checking where license file should be stored

Please make sure all licenses are stored in: "/etc/oplmgr"

- Checking for initial Virtuoso license

- Starting OpenLink License Manager
- Using License directory "/etc/oplmgr"

- Creating default environment settings

- Creating default database settings

- Configuring: database
  - Creating directory $VIRTUOSO_HOME/database/backup
  - Creating directory $VIRTUOSO_HOME/database/logs
  - Installing new virtuoso.ini in $VIRTUOSO_HOME/database
  - Installing new php.ini in $VIRTUOSO_HOME/database
  - Creating symlink to $VIRTUOSO_HOME/bin/virtuoso-iodbc-t

- Registering ODBC drivers

- Registering .NET provider for Mono

- Finalizing installation


This concludes the first part of the installation.

To start the Virtuoso database, please use the following command:

    $ bin/virtuoso-start.sh

After the Virtuoso database successfully started, you can continue
setting up your database by opening the following URL in your
browser:

    http://localhost:8890/conductor/

Installation completed
</blockquote>

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>7</b>. Run the <code>bin/virtuoso-start.sh</code> script to start the Virtuoso database. If this is an upgrade then the existing database will be started as is or if this is a new installation then a new database will be created in the <code>database</code> directory and an initial-password set for it:

<code>$ bin/virtuoso-start.sh</code>

<blockquote>
Checking the OpenLink License Manager
- Using License directory "/etc/oplmgr"

Starting Virtuoso instance in [database]
  - Generating a random password
  - Initializing the new database

NOTE: The new database has been initialized with a randomly generated
      password for both dba and dav accounts. Make sure you change it
      at your earliest opportunity.
      The password has been saved to disk in:
            database/.initial-password

  - Starting the database
</blockquote>

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>8</b>. Check the <code>database/.initial-password<code> file to what the random password has been set to for the database:

<code>$ cat database/.initial-password </code>

<blockquote>sH7YX2kp</blockquote>

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>9</b>. The Virtuoso Conductor Database Administration Web interface <code>http://hostname:8890/conductor</code> can now be logged into using the password above:

"How Do I Install Virtuoso on Linux and other Unix Operating Systems?"

<b>10</b>. If Conductor login is successful the installation is now ready for use:

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>1</b>. AWS RedHat Enterprise Linux 7

OpenLink provides Amazon BYOL & PAGO Virtuoso EC2 AMI MarketPlace Offers.

The latest Amazon BYOL & PAGO Virtuoso EC2 AMI MarketPlace Offers, since version 08.03.3323 and above are already hooked into the OpenLink Nexus Repository (https://nexus.openlinksw.com/) , and thus can simply be updated with the command:

<code>yum update</code>

Which will automatically download and install the latest operating system and Virtuoso updates.

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>2</b>. AWS & Azure Ubuntu 18.04 LTS

The Virtuoso Ubuntu 18.04 LTS AWS PAGO 1, AWS BYOL, Azure PAGO and Azure BYOL offers are built from the OpenLink Nexus Repository 6 and are thus automatically bound to the repository and can be updated to the latest Virtuoso packages with the command:

<code>sudo apt upgrade</code>

Which will automatically download and install the latest operating system and Virtuoso updates.

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>3</b>. CentOS 7 and 8

The following sequence can be use to install the Virtuoso 8.3 RPM packages on CentOS 7 and 8 systems, whether in the Cloud or on on-premise standalone systems.

1. Install the EPEL metadata:

   <code>sudo yum install -y epel-release</code>

2. Install the OpenLink repository metadata:

   <code>sudo yum install -y https://nexus.openlinksw.com/repository/openlink-files/openlink-repo.rpm</code>

3. Install Virtuoso 8.3 on a local machine:

   <code>sudo yum install -y virtuoso-8-commercial-full</code>

The Virtuoso installation is now hooked into the OpenLink Nexus Repository 6 and updated to the latest Virtuoso components, and can be started and stopped with the Linux service commands:

   <code>sudo service virtuoso { start | status | stop }</code>

The latest Virtuoso packages can be obtained from the OpenLink Nexus Repository 6 with the command:

   <code>sudo yum update</code>

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>4</b>. RedHat Enterprise Linux 7

The following sequence can be use to install the Virtuoso 8.3 RPM packages on RedHat Enterprise Linux (RHEL) 7 systems, whether in the Cloud or on on-premise standalone systems.

1. Install the EPEL metadata:

   <code>sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm</code>

2. Install the OpenLink repository metadata:

   <code>sudo yum install -y https://nexus.openlinksw.com/repository/openlink-files/openlink-repo.rpm</code>

3. Install Virtuoso 8.3 on a local machine:

   <code>sudo yum install -y virtuoso-8-commercial-full</code>

The Virtuoso installation is now hooked into the OpenLink Nexus Repository 6 and updated to the latest Virtuoso components, and can be started and stopped with the Linux service commands:

   <code>sudo service virtuoso { start | status | stop }</code>

The latest Virtuoso packages can be obtained from the OpenLink Nexus Repository 6 with the command:

   <code>sudo yum update</code>

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>5</b>. RedHat Enterprise Linux 8.x / 9.x

The following command sequence can be use to install the Virtuoso 8.3 RPM packages on RedHat Enterprise Linux (RHEL) 8.x / 9.x systems, whether in the Cloud or on on-premise standalone systems.

1. Install the EPEL metadata:

   <code>sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest.noarch.rpm</code>

2. Install the OpenLink repository metadata:

   <code>sudo yum install -y https://nexus.openlinksw.com/repository/openlink-files/openlink-repo.rpm</code>

3. Install Virtuoso 8.3 on a local machine:

   <code>sudo yum install -y virtuoso-8-commercial-full</code>

The Virtuoso installation is now hooked into the OpenLink Nexus Repository 6 and updated to the latest Virtuoso components, and can be started and stopped with the Linux service commands:

   <code>sudo service virtuoso { start | status | stop }</code>

The latest Virtuoso packages can be obtained from the OpenLink Nexus Repository 6 with the command:

   <code>sudo yum update</code>

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>6</b>. Ubuntu 18.04 / 20.04 / 22.04 LTS and Debian 10 / 11

The following command sequence can be use to install the Virtuoso 8.3 RPM packages on Ubuntu 18.04 / 20.04 / 22.04 LTS and Debian 10 / 11 systems, whether in the Cloud or on-premise standalone systems.

1. Run the following commands to install Virtuoso from the OpenLink Nexus Repository 1:
<pre>
apt update
wget https://nexus.openlinksw.com/repository/openlink-files/openlink-repo.deb
apt install ./openlink-repo.deb
apt update
apt install virtuoso-8-commercial-full
</pre>
2. If no errors are reported during the installation, run the following command to enable the Virtuoso instance to auto-start on reboot:

   <code>sudo systemctl enable virtuoso</code>

The Virtuoso installation is now hooked into the OpenLink Nexus Repository 6 and updated to the latest Virtuoso components, and can be started and stopped with the Linux service commands:

    <code>sudo service virtuoso { start | status | stop }</code>

The latest Virtuoso packages can be obtained from the OpenLink Nexus Repository with the command:

    <code>sudo apt-get upgrade</code>

"How Do I Install or Update Virtuoso Releases across Debian, Ubuntu, RHEL, and CentOS, via the OpenLink Nexus Repo"

<b>7</b>. Apply Virtuoso License

Once the Virtuoso packages for the target Linux distribution have been installed from the repository, if a valid Virtuoso license (virtuoso.lic) is not already in place, one needs to be added to the directory /etc/oplmgr, and the OpenLink License Manager (oplmgr) then stopped and started with the commands:

    <code>sudo service oplmgr { start | status | stop }</code>

If the Virtuoso server is running, it also needs to stopped and started for the license file to be picked up, with the commands:

    <code>sudo service virtuoso { start | status | stop }</code>

Finally, check the /opt/database/virtuoso.log file to ensure the license file is valid and has been accepted for use.

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>1</b>. <code>select count(*) from bad_iri_id where  id_to_iri (id) like 'iri_id_%';</code>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>2</b>. <pre>
create table bad_iri_id (id iri_id_8 primary key);
log_enable (2);
insert into bad_iri_id select ri_id from rdf_iri where subseq (ri_name, 0, 4) = '\0\0\0\x6'; --<<< this takes a long time to complete ~30m
</pre>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>3</b>. <code>delete from   bad_iri_id where  id_to_iri (id) not like  'iri_id_%_with_no_name_entry';</code>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>4</b>. <code>select count(*) from bad_iri_id;</code>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>5</b>. <pre>
	-- delete from RDF_IRI
delete from RDF_IRI where RI_ID in (select id from bad_iri_id) ;
</pre>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>6</b>. <code>checkpoint;</code>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>7</b>. <code>shutdown ;</code>

"How do I clean out an RDF_IRI table following an ACID-unaware ETL process that's created broken IRIs?"

<b>8</b>. <code>restart database server</code>

"Migrating Virtuoso from a 32-Bit Prefixed to 64-Bit Prefixed Database"

<b>1</b>. When starting an existing 8.x database with the new 08.03.3326+ binary, the following message will appear in the virtuoso.log file:
<blockquote>
    NOTE: Your database is using 32-bit prefix IDs in RDF_IRI

    This Virtuoso engine has been upgraded to use 64-bit prefix IDs
    in RDF_IRI to allow for even larger databases.

    To take advantage of this new feature, your database needs to
    be upgraded.

    The performance of your existing database should not be affected,
    except when performing certain bulkload operations.

    Please contact OpenLink Support <support@openlinksw.com> for
    more information.
</blockquote>
As stated in the message, the engine will use a backward compatibility function to handle existing databases without causing a performance degradation when running SPARQL queries, inserts and deletes.

"Migrating Virtuoso from a 32-Bit Prefixed to 64-Bit Prefixed Database"

<b>2</b>. Caveat
Bulkloading operations on an existing database using 32-bit prefix IDs will be restricted to use non-vectored functions. This will cause a drop in bulkload performance, so users who rely on this functionality should upgrade their database as soon as possible.

Calling vectored functions like <code>TTLP_V()</code< and <code>RDF_LOAD_RDFXML_V()</code> will automatically call their non-vectored equivalents like <code>TTLP()</code> and <code>RDF_LOAD_RDFXML()</code>.

Bulkloading using the <code>rdf_loader_run()</code> functions also automatically will downgrade to use non-vectored functions.

Some functions may fail with the following error:

   <blockquote>[42000] Can not use dpipe IRI operations before upgrading the RDF_IRI table to 64-bit prefixes IDs</blockquote>

"Migrating Virtuoso from a 32-Bit Prefixed to 64-Bit Prefixed Database"

<b>3</b>. Upgrade method 1

The preferred way of upgrading to the new 08.03.3326+ format is to perform an NQUAD dump of all your triples using the <a href="https://community.openlinksw.com/t/producing-rdf-dumps-of-virtuoso-quad-store-hosted-rdf-model-data/3323">RDF_DUMP_NQUADS()</a> function and <a href"https://vos.openlinksw.com/owiki/wiki/VOS/VirtBulkRDFLoader">bulk loading</a> them into a new database.

"Migrating Virtuoso from a 32-Bit Prefixed to 64-Bit Prefixed Database"

<b>4</b>. Upgrade method 2

To upgrade an existing database in-place, make sure you have a proper backup of your existing database before performing the following commands:
<pre>
set echo on; 
scheduler_interval(0);
backup '/dev/null'; -- make sure db is consistent
log_enable (2,0);

-- copy
create table DB.DBA.RDF_IRI_64 (RI_NAME varchar not null primary key, RI_ID IRI_ID_8 not null);
insert into DB.DBA.RDF_IRI_64 (RI_ID, RI_NAME) select RI_ID, __iri_name_id_64(RI_NAME) from DB.DBA.RDF_IRI;
checkpoint;

-- rename
drop table DB.DBA.RDF_IRI;
alter table DB.DBA.RDF_IRI_64 rename DB.DBA.RDF_IRI;
create unique index DB_DBA_RDF_IRI_UNQC_RI_ID on DB.DBA.RDF_IRI (RI_ID);

-- set db is upgraded
__dbf_set('rdf_rpid64_mode',1);
shutdown;
</pre>
Note however that depending on the number of records in the <code>DB.DBA.RDF_IRI</code> table, this can take a long time and will increase the size of your database.

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>1</b>. Data Integrity Check reports Broken index RDF_QUAD_XXXX

If when running a database integrity check to check the state of the database with the command:

   <code>backup '/dev/null';</code>

Failure of this command to complete is an indication of the database having some form of corruption, requiring a +backup-dump of the database with the command:

   <code>virtuoso-iodbc-t +configfile virtuoso.ini +backup-dump</code>

If the Virtuoso log output of this command indicates a Broken Index as below:

<blockquote>
08:20:27 Database dump started 
08:20:27 Checkpoint finished, log reused 
08:20:27 Dumping the schema tables 
08:20:27 Dumping the registry 
08:20:27 Dumping the schema done 
09:03:02 Broken index RDF_QUAD_POGS 
09:03:02 GPF: colsearch.c:1753 less ces in seg than indicated in leaf col ref 
GPF: colsearch.c:1753 less ces in seg than indicated in leaf col ref 
Segmentation fault (core dumped)
</blockquote>

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>2</b>. Index Rebuild Steps

Then that index and any others (comma delimited) need to be removed from the +backup-dump by adding them to the BackupIgnoreKeys parameter in the [Parameters] section of the Virtuoso config file (virtuoso.ini):
<pre>
[Parameters] 
BackupIgnoreKeys = RDF_QUAD_POGS
</pre>
Repeat the <code>+backup-dump</code> which should then complete as which time the database can be restored with the +restore-crash-dump command:
<code>
virtuoso-iodbc-t +configfile virtuoso.ini +restore-crash-dump 
</code>
The missing index will then have to be recreated as detailed in the RDF Performance Tuning guide, thus is the case of the RDF_QUAD_POGS index the command would be:
<pre>
DROP INDEX RDF_QUAD_POGS ;

CREATE COLUMN INDEX RDF_QUAD_POGS 
	ON DB.DBA.RDF_QUAD (P, O, G, S) 
	PARTITION (O varchar (-1, 0hexffff));
</pre>

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>3</b>. Database reports Broken index RDF_QUAD_XXXX on startup

If on startup the Virtuoso database fails to start reporting Broken index RDF_QUAD_XXXX:
<blockquote>
11:14:55 built-in procedure "WS.WS.SYS_DAV_RES_RES_CONTENT_HOOK" overruled by the RDBMS
11:14:55 built-in procedure "WS.WS.SYS_DAV_RES_RES_CONTENT_INDEX_HOOK" overruled by the RDBMS
11:14:59 Roll forward started
11:15:03     353 transactions, 3698455 bytes replayed (100 %)
11:15:03 Roll forward complete
11:15:05 Broken index RDF_QUAD_POGS
11:15:05 GPF: colsearch.c:1757 less ces in seg than indicated in leaf col ref
GPF: colsearch.c:1757 less ces in seg than indicated in leaf col ref
</blockquote>

This can be recovered by as follows:

Ensure a full backup or copy of the database is made

Remove or rename the virtuoso.trx transaction log file

Start the database again which should not start successfully and run the command to drop the broken RDF index:

<code>_INTERNAL_DROP_INDEX ('DB.DBA.RDF_QUAD', 'RDF_QUAD_POGS');</code>

Perform a database crash dump of the database with the +crash-dump command:

   <code>virtuoso-iodbc-t +configfile virtuoso.ini +crash-dump </code>

On completion the database can be restored with the +restore-crash-dump command:

   <code>virtuoso-iodbc-t +configfile virtuoso.ini +restore-crash-dump </code>

The missing index can then be recreated as detailed in the RDF Performance Tuning guide, thus is the case of the RDF_QUAD_POGS index the command would be:
<pre>
DROP INDEX RDF_QUAD_POGS ;

CREATE COLUMN INDEX RDF_QUAD_POGS 
	ON DB.DBA.RDF_QUAD (P, O, G, S) 
	PARTITION (O varchar (-1, 0hexffff));
</pre>

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>4</b>. Checking RDF index have been repaired

The following operations can be performed to verify the RDF indexes on the RDF_QUAD table repaired by either of the above methods have been fixed.

Database Integrity Check

Run a database integrity with the command:

<code>backup '/dev/null';</code>

Check RDF index counts

First check with the following count queries on all the indexes on the RDF_QUAD table by running the following queries for Virtuoso:
<pre>
   select count (*) from rdf_quad a table option (index rdf_quad) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_pogs, no exists join) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); 
   select count (*) from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index primary key, no exists join) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); 
   select count (*) from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_op, no exists join) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); 
   select count (*) from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_sp, no exists join) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); 
   select count (*) from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_gs, no exists join) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s);
</pre> 

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>5</b>. If the counts of the queries are not all "0" this indicated the indexes with none "0" counts are corrupt and needs to be repaired, which can be done with the following queries/commands:

<pre>
create table rq_recov (g iri_id_8, s iri_id_8, p iri_id_8, o any, primary key (g,s,p,o));

log_enable (3,1);
insert soft rq_recov (g,s,p,o) select g,s,p,o from RDF_QUAD a table option (index RDF_QUAD) 
    where not exists (select 1 from RDF_QUAD b table option (loop, index RDF_QUAD_POGS, no exists join) 
        where a.G = b.G and a.P = b.P and a.O = b.O and a.S = b.S);

insert soft rq_recov (g,s,p,o) select g,s,p,o from RDF_QUAD a table option (index RDF_QUAD_POGS) 
    where not exists (select 1 from RDF_QUAD b table option (loop, index primary key, no exists join) 
        where a.G = b.G and a.P = b.P and a.O = b.O and a.S = b.S);
insert soft rdf_quad index rdf_quad (g,s,p,o)  select g,s,p,o from rq_recov;
insert soft rdf_quad index rdf_quad_pogs (g,s,p,o)  select g,s,p,o from rq_recov;
insert soft rdf_quad index rdf_quad_sp (g,s,p,o)  select g,s,p,o from rq_recov;
insert soft rdf_quad index rdf_quad_op (g,s,p,o)  select g,s,p,o from rq_recov;
insert soft rdf_quad index rdf_quad_gs (g,s,p,o)  select g,s,p,o from rq_recov;
log_enable (1,1);
drop table rq_recov;
checkpoint;
</pre>
Then run the select count ... queries again for Virtuoso 7 or 8 , as above, to check indexes, the counts ALL of which should all be zero indicating they are repaired.

"Virtuoso reports problems with indexes prior to performing a backup run"

<b>6</b>. Check the primary key on the RDF_QUAD table is not broken by running the command:

   <code>select count (s), count (p ), count (o ), count (g ) from rdf_quad   table option (index rdf_quad, check);</code>

The counts of the `s, p, o, g` columns should all be the same, and if not the primary key index can be repaired by running the following queries:

   <pre>insert into rdf_quad index rdf_quad_pogs (s,p,o,g) select s,p,o,g from rdf_quad a table option (index rdf_quad) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_pogs) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); </pre>

Then the reverse insert query needs to be run to insert any missing rows from the `rdf_quad` index into the `rdf_quad_pogs` full index ie

   <pre>insert into rdf_quad index rdf_quad (s,p,o,g) select s,p,o,g from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad) where a.g = b.g and a.p = b.p and a.o = b.o and a.s = b.s); </pre>

And the following 3 insert queries need to be run to insert any missing rows in the 3 partial indexes ie rdf_quad_gs, rdf_quad_sp & rdf_quad_op as follows:

   <pre>
   insert into rdf_quad index rdf_quad_sp (s,p) select s,p from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_sp) where a.p = b.p and a.s = b.s);

   insert into rdf_quad index rdf_quad_op (o,p) select o,p from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_op) where a.o = b.o and a.p = b.p);
   
   insert into rdf_quad index rdf_quad_gs (g,s) select g,s from rdf_quad a table option (index rdf_quad_pogs) where not exists (select 1 from rdf_quad b table option (loop, index rdf_quad_gs) where a.g = b.g and a.s = b.s);
   </pre> 

Then run the `select count ...` queries again as above, to check indexes, the counts `ALL` of which should all be zero indicating they are repaired.

If the above steps are successful the RDF index has been repaired and the database is ready for use again.