Continuous processing

Teklia Spider has been created to manage a large amount of data. Most of the time images ingestion will raise a few errors, that could be handled manually or programmatically.

Inspecting errors

Once the ingestion task is completed, you can filter the generated image links by their validation state.

Image state filter

If the links has been marked as erroneous, you can see the error details by hovering the Error tag.

Erroneous images

Generic error messages

HTTP 404

The image could not be found on the server. You should ensure image’s path is correctly built and the image exists on the storage bucket. If the path is badly formatted (e.g. images folder does not exist), you can download the CSV file containing erroneous lines and edit it before creating a new campaign on the project for the missing images. If the path is correct, the image will probably be missing. In this case you may contact Spider administrators or the corresponding collectivity directly.

HTTP 403

Represent an access right error to the image file. This error can usually be fixed by an administrator.

HTTP 50X

A server error, usually occurring in specific circumstances (e.g. corrupted image file). Please contact Spider administrators in this case.

[Socface] Missing town

Is Socface projects, some images may have no town because of an erroneous entry in the CSV file. In case of a typo, you can edit the CSV file directly. In other cases (multiple town manes match, empty values), it would be more appropriate to discuss the problem with Spider administrators and/or the corresponding municipality.

[Socface] Missing year

As for town, year is a required attribute for validating images on a Socface project. The year must be composed of simple digits, corresponding to a date between 1700 and 2100. In case of a typo, you can edit the CSV file directly, otherwise (non-digits, ranges, empty values) please contact the corresponding municipality.

[Heritus] Missing top folder

In Heritus projects, some images may be missing a top folder, for example if the Top folder column is empty on some rows. You can fill empty rows with a unique default name to fix those images.

Running successive campaigns

If there is at least one image in an error state on the campaign, you can export the errors in a CSV format by hovering the Export errors button.

Export erroneous images

  • CSV lines Supply a copy of the CSV file used to run this campaign, excluding lines that generated valid images.

  • Erroneous images Supply a CSV file containing one line for each image in an error state, with an error message and additional information from the line that generated this image on the CSV file used to run this campaign.

Once you have retrieved the CSV file that suits your case, you can use it to create a new campaign on the same project and ingest missing images. Sometimes you will need to edit the file manually to fix a potential error or configure columns differently.

When running a new campaign on the project, it will first "steal" all validated images on project’s previous campaigns. If some links are generated again, those images will be stolen too then validated a second time. This behavior helps to eliminate errors due to defaults in the CSV file, that are corrected during the successive campaigns. Only images on the same IIIF server will be stolen (running a campaign from IIIF manifests), although multiple IIIF servers are not fully supported.

You can follow the progress over successive campaigns from the project’s page:

Ingested campaigns list

You may want to remove the leftover campaigns, as their errors have been corrected and does not represent a useful information. Only campaigns with no image or errors can be deleted.

When clicking on the trash icon, you will be redirected to a confirmation page. Keep in mind that errors cannot be exported anymore once a campaign is deleted.