Arkindex 1.8.1
We are happy to announce that a new Arkindex release is available. You can explore Arkindex and try out the newest features on our demo instance, demo.arkindex.org.
Models
The user interface for models has been redesigned to look more consistent with workers and worker versions. New tabs allow viewing and managing the description, versions, compatible workers and memberships for a model.
Workers
When adding a new worker version from the frontend, Arkindex will attempt to detect a version number from the Docker image reference. If the reference has a tag with a valid version number, it will be assigned as a tag for the new worker version.
This allows administrators to create new worker versions that may be assigned as the recommended worker version, without having to update the version manually in the administration interface or having to publish versions with the CLI.
Worker results
Some optimizations have been made on the worker results deletion when deleting from an element. Deleting from its child elements recursively now gives better optimization opportunities to the database, and deleting from an element with no child elements at all is much faster.
While worker versions have been removed from all worker results in the Arkindex API, frontend and exports for a while, they are still stored in the database, and very old results that had worker versions and no worker runs were shown as not having any source at all. This release includes a migration tool for system administrators to create new worker runs for those old results, making the sources of those results visible again. We hope to remove the worker versions from worker results entirely in the next release.
Misc
Following the upgrades performed in the previous release, several issues have been identified and fixed in our frontend:
-
In the process list, the My processes filter now properly applies again.
-
In the process list, clicking a link to a started process now properly opens the process status page without having to refresh the page or type in a URL.
-
In the process status page, the progress bar showing task states for the latest run of the process is now visible again for processes with only one run.
-
In the process elements filtering page, the Load children checkbox is now more visible and better aligned with the other filters.
-
When creating a worker configuration on a worker version with user-configurable parameters, required fields with default values are displayed again.
-
Creating a worker configuration on a worker version with user-configurable parameters that include list or dictionary fields no longer causes browser freezes or crashes.
-
In the worker and model list pages, archiving a worker or model then switching to another no longer unexpectedly reopens the archival confirmation modal.
Upgrade notes
To upgrade a development instance, follow this documentation.
To upgrade a production instance, you need to:
-
Deploy this release’s Docker image:
registry.gitlab.teklia.com/arkindex/backend:1.8.1 -
Run the database migrations:
docker exec ark-backend arkindex migrate -
Update the system workers:
docker exec ark-backend arkindex update_system_workers
The main changes impacting developers and system administrators are detailed below.
Migration of worker results with worker versions
The link between worker versions and all worker results had been removed from all APIs in Arkindex 1.6.2, but it is still present in the Arkindex database.
In this release, in preparation for removing this link and relying exclusively on worker runs, we are providing automated migrations for old worker results that had worker versions and no worker runs.
One migration will create processes and worker runs to maintain the old links to worker versions, and subsequent migrations add new database constraints on each type of worker result to ensure they can only be created with a worker run and a worker version, or neither of those.
On large or old Arkindex instances with millions of worker results to migrate, this process can be slow and costly. Arkindex 1.8.1 can operate without these migrations being completed, so they can be performed gradually without downtime.
A new administrator command is provided to offer more flexibility in migrating existing worker results: arkindex generate_missing_worker_runs. This is equivalent to the documents.0016 Django migration. Once this command reports that there are no more worker results to migrate, the database migrations can be completed with the normal arkindex migrate command.
New Prometheus metrics
Alongside our existing /metrics/ URL for Arkindex-specific metrics, new Python and Django Prometheus metrics have been added at /metrics/views/ through the django-prometheus project. These new metrics can help in monitoring the performance of Arkindex endpoint by endpoint.