Arkindex 1.6.4

We are happy to announce that a new Arkindex release is available. You can explore Arkindex and try out the newest features on our demo instance, demo.arkindex.org.

New release cycle

As you may have noticed, we have switched to a shorter release schedule. We will now publish a new version of Arkindex every 2 weeks !

This means that future releases will often have less changes than before, but will be updated a lot more frequently.

Each cycle will span over 2 months including 4 releases:

  • 1 major release (should be 1.X.0)

  • 3 minor releases (should be 1.X.1, 1.X.2, 1.X.3)

We are currently transitioning towards that cycle, so we’ll produce the next releases:

  • 1.6.4 today !

  • 1.6.5 end of october

  • 1.6.6 mid-november

  • 1.7.0 end of november: this will be the next major release.

Then we’ll be officially operating under the new faster release schedule.

flowchart LR 1.6.3{1.6.3} --> 1.6.4 --> 1.6.5 --> 1.6.6 --> 1.7.0{1.7.0} --> 1.7.1
Figure 1. Next releases, with major versions highlighted

File import worker

We have completed our transition towards internal workers for the file import tasks.

The new worker source code is freely available and under the AGPL-V3 licence as most of our projects.

This has no visible changes for end-users (file import still support images, zip archives, transkribus exports, IIIF manifests), but simplifies Arkindex instances deployment as detailed in the technical release notes.

This change allows us to archive the tasks project which hosted all the internal asynchronous operation, and simplify a lot of the backend code related to file imports.

Misc changes

We have also made a few bugfixes or iterative improvements on the Arkindex platform:

  • The deprecated ML Metrics in Arkindex have been removed. We highly recommend using MLFlow or Weight & Biases to track Training metrics.

  • We fixed a bug introduced in Arkindex 1.6.3 which added an extra initialisation task when retrying worker processes under specific conditions.

  • Unused worker types are removed in the arkindex cleanup command.

  • Worker type is now displayed in the worker details.

Upgrade notes

To upgrade a development instance, follow this documentation.

To upgrade a production instance, you need to:

  • Deploy this release’s Docker image: registry.gitlab.teklia.com/arkindex/backend:1.6.4

  • Run the database migrations: docker exec ark-backend arkindex migrate

  • Update the system workers: docker exec ark-backend arkindex update_system_workers

The main changes impacting developers and system administrators are detailed below.

Simpler management of system workers

We have continued the work from release 1.6.3 to simplify internal worker management, by porting the file import code into a dedicated worker (named File Import).

The following keys can be removed from your configuration files as it’s no longer needed:

  • imports_worker_version

  • docker: the whole Docker images section can be now fully removed, as we only use the Arkindex features on worker versions now.

The file_import Arkindex feature is now configurable towards that worker so your users can continue importing files into your instance.

Please note that running the arkindex update_system_workers command will automatically configure that worker for you.