Arkindex 1.8.2

We are happy to announce that a new Arkindex release is available. You can explore Arkindex and try out the newest features on our demo instance, demo.arkindex.org.

Secret Management

The Arkindex platform has supported secrets for a while, but their management was only possible for system administrators until this release.

Edit a secret value and share it with other Arkindex users

Using the new Secret menu, available in the top-right corner of the Arkindex interface, you can:

  • view all secrets you have access to

  • edit your secrets

  • share them with other users or group of users

  • delete secrets

All these actions depend on the usual access rights. You may not see immediately secrets on your Arkindex instance if the system administrator did not share them with you !

In upcoming releases, secrets will be easily usable with Arkindex workers.

Removal of worker versions from worker results

As mentioned in the previous Arkindex release 1.8.1, we were able to remove the worker versions set on worker results (transcriptions, meta-datas, classifications and entities).

This is the last step in a migration started more than a year ago, to only rely on worker runs.

This change has no impact on the end users, as most of the impactful work was done over the past releases.

Asynchronous IIIF image validation

The API endpoint CreateIIIFUrl now supports optional width and height parameters. When these parameters are set, the IIIF validation will be done asynchronously (provided your API user can validate images and the targeted image server allows external validation.

This change is useful for large external image providers as it provides an important speed gain. The IIIF image synchronous validation remains the default mode.

More details are available on the technical notes.

Upgrade notes

To upgrade a development instance, follow this documentation.

To upgrade a production instance, you need to:

  • Deploy this release’s Docker image: registry.gitlab.teklia.com/arkindex/backend:1.8.2

  • Run the database migrations: docker exec ark-backend arkindex migrate

  • Update the system workers: docker exec ark-backend arkindex update_system_workers

The main changes impacting developers and system administrators are detailed below.

Removal of worker versions from worker results

The removal of worker versions, started in the previous release, is now complete. The database migrations in this release will remove the old data from the database. The arkindex generate_missing_worker_runs command is no longer available.

On large or old Arkindex instances with many worker results that had a worker version but no worker run, we recommend migrating to Arkindex 1.8.1 first. Skipping from 1.8.0 or earlier to 1.8.2 directly may require a significant downtime or prevent workers from creating any worker results until the database migration is complete, while version 1.8.1 will provide the arkindex generate_missing_worker_runs command which enables a smooth transition.

Multiple database indexes have been updated. If you see any drops in performance after the migrations, run an ANALYZE command on the PostgreSQL database to let it learn about those indexes and use them properly.

New IIIF job queue

A new iiif queue has been added in our RQ worker setup. You may need to update your deployments to add this queue to the arkindex rqworker or arkindex rqworker-pool commands, or to add new workers.

This new queue allows for a new asynchronous validation of IIIF images, which can improve performance on large imports. The CreateIIIFURL API endpoint will use an asynchronous validation, instead of the regular synchronous one, under the following conditions:

This mode is only meant for advanced cases where imports are using images from trusted IIIF servers and closely monitored. This is not used by any import within Arkindex itself.

This change does not affect developers, as our development setup has been updated to include this queue in the local RQ worker.