Upgrade Paths for the Airlock Server

Modified on Thu, 14 Sep 2023 at 10:10 AM

Applies to

Airlock Server - (On-premises)

Preface

When upgrading an on-premises Airlock instance, it is important to be aware of certain server versions that may need to be upgraded to, prior to moving onto the latest version.

These paths are required due to architectural changes or updates that occur on certain server versions that make them a touch point when the server is going from an older version of Airlock to a more recent version. When upgrading to these specific versions, the existing Airlock database will be re-indexed in accordance to the changes, once this process is completed the server can be upgraded to the target version.

The two versions that are currently available of Airlock that have this requirement are v4.8.1 and v5.1.0. Server versions of this nature will be referred to as 'touch point server versions' in the steps below.

This means for versions of the Airlock server that are pre-v4.8.1, they will need to be upgraded to 4.8.1 before being upgraded to a more recent version. The same concept also applies for versions pre-v5.1.0, versions below this will need to be upgraded to 5.1.0 before being upgraded to more recent versions.

Process

Prior to any upgrade steps, a snapshot of the server will need to be performed as a safety measure in case any issues occur during or after the upgrade of the Airlock server.

The next step is to check the execution history count for the database, as this can be used to estimate the amount of time the re-indexing will take. While the upgrade itself will only take 20-30 mins, the re-indexing can take anywhere from 30mins to a few hours depending on this count.

To get this count, the following command will need to be ran on the Airlock server. There are two commands to cover both podman and docker. Docker is used on all CentOS/RHEL versions v8.0 and below, podman is used on CentOS/RHEL v8.1 and above:

podman exec -it airlock_server mongo airlock --eval 'db.exechistories.count()'
or
docker exec -it airlock_server mongo airlock --eval 'db.exechistories.count()'

This will print the number of executions within the database. For an approximate scale, ~10mil entries will take ~1hr to re-index.

If there are upwards of 50mil entries, it is recommended that the Database History Retention feature is used to trim the database, this can also be considered a best practice method regardless of the execution history count as well. This setting can be found on the Web Console in Settings --> Database --> Database Execution History Retention. This will have a drop-down menu that will allow the selection of a timescale for the Airlock Server to retain Execution History.

It is important to note that this will not remove any of the repository data for files or metadata that is stored within the database, this will only remove execution history that has occurred prior to the timescale set. Once this has been set, there will need to be a 24hr period of waiting for the Airlock server to completely trim the execution history to within the timescale selected.

After the 24hr period has completed, the exechistories count should drop depending on the timescale that was set. This will in-turn reduce the amount of time the re-indexing will take.

The touch point server installer can now be run. After running the installer, a database log can be tailed and allows the progress of the re-indexing process to be monitored.

tail -f /opt/airlock_data/log/mongodb.log

The logs displayed by the above command should look similar to the screenshot below:

Please note, there may be de-duplication occurring as well, in which case it may take a while (up to 30m, and sometimes longer) for the re-index logs to appear.

Once the re-indexing has completed the log will have significantly reduced reporting. Re-indexing completion can also be confirmed when the log activity ceases. This can be done by running the following commands in order:

#Login to the Container
podman exec -it airlock_server /bin/bash --login
or
docker exec -it airlock_server /bin/bash --login

#Open Mongo
mongo airlock

#Output number of indexes
var sum=0;
db.getCollectionNames().forEach(function(collection) {
        if (!(collection.includes('temp') || collection.includes('tmp'))){
        print(collection + ": " + db[collection].getIndexes().length);
        sum += db[collection].getIndexes().length
        }   
    });
print("Total index Count: " + sum);

This will provide a count of all the indexes within the database, with a total located at the bottom of the results. This total should be above 130, if it is below 130, then the re-indexing may not have completed successfully.

If this is the case and the index count is below 130, the re-indexing process will need to be run again. It may need to run 2-5 time before it is reporting a healthy index count.

If it is run up to 5 times and the index count remains below 130, the Airlock Support Team can be contacted for further analysis. This can be done by either sending and email to support@airlockdigital.com or opening a ticket directly on the Airlock FreshDesk Portal.

If the index count is above 130 then the server is ready to be upgraded to either the latest version of the Airlock Server, or to the next mandatory upgrade path version.

Additional

If the upgrade is planned to be run overnight, the following process will need to be run in order to prevent the server from restarting at midnight, as this will stop the re-indexing process and force it to fail.

[Run:]

crontab -e

[In this VIM comment out the below two lines:]

*       *       *       *       *       /usr/sbin/ald_servertask
0       0       *       *       *       [[ ! -f "/opt/airlock_data/sem/reindexdb_flag" ]] && /usr/sbin/logrotate -f /etc/logrotate.d/airlock-server

[To do add a '#' in front of them, so they will now look like this:]

#*       *       *       *       *       /usr/sbin/ald_servertask
#0       0       *       *       *       [[ ! -f "/opt/airlock_data/sem/reindexdb_flag" ]] && /usr/sbin/logrotate -f /etc/logrotate.d/airlock-server

[This change can be confirmed after the file was saved by running the following:]

crontab -l

After the upgrade and re-indexing is complete, the '#' symbols can be removed in order to reinstate these lines.