Deployment

Grappa is available as a Docker image and can run as a Docker container, Docker swarn service or Kubernetes pod.

Its docker images are root-less and embeds all the libraries needed for both web services and clustering.

Architecture

To run Grappa, you’ll need a few companion services:

  • Postgresql database (version 18 and above)

  • Redis server (for asynchronous communication between backend & workers)

  • a Load balancer (if not readily available on your deployment platform, we recommend to use Traefik

The Grappa application itself needs to run in two containers:

  1. the web application using registry.gitlab.teklia.com/arkindex/grappa/web image

  2. the asynchronous worker using registry.gitlab.teklia.com/arkindex/grappa/worker image

graph TD lb[Load balancer] --> backend[Backend] backend --> postgres[PostgreSQL] backend --> redis[Redis] worker[Worker] --> postgres worker --> redis

Docker compose

You’ll find below a docker-compose file showcasing a Docker container deployment on a single server.

---
services:

  # Grappa web application
  grappa:
    container_name: grappa-app
    image: registry.gitlab.teklia.com/arkindex/grappa/web:X.Y.Z
    env_file:
      - docker.env
    restart: unless-stopped
    depends_on:
      - postgres
    ports:
      # For web access
      - 8000:8000
      # For prometheus metrics only
      - 3000:3000
    networks:
      - grappa

    volumes:
      - storage:/storage

  # Postgresql database
  postgres:
    container_name: grappa-database
    image: postgres:18-alpine
    env_file:
      - docker.env
    volumes:
      - database:/var/lib/postgresql
    networks:
      - grappa

  # Redis
  redis:
    container_name: grappa-redis
    image: redis:8.8.0-alpine
    networks:
      - grappa
    ports:
      - 6379:6379

  # Grappa asynchronous worker
  celery:
    container_name: grappa-celery
    image: registry.gitlab.teklia.com/arkindex/grappa/worker:X.Y.Z
    command: "celery -A grappa.base worker -l INFO"
    env_file:
      - docker.env
    depends_on:
      - redis
    networks:
      - grappa
    volumes:
      - storage:/storage

volumes:
  database:
    driver: local
  storage:
    driver: local

networks:
  grappa:
    driver: bridge

Configuration

The Grappa web application & worker are configured through environment variables.

Here is a sample docker.env file that would work with the docker-compose.yml file above:

# DB settings for postgres container
POSTGRES_USER=grappa
POSTGRES_PASSWORD=teklia12345
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=grappa

# DB settings for grappa
DATABASE_URL=postgres://grappa:teklia12345@postgres:5432/grappa

# Run in prod mode
DEBUG=false

# Allow public host
ALLOWED_HOSTS=localhost
CSRF_TRUSTED_ORIGINS=http://localhost:8000

# Redis
REDIS_URL=redis://redis:6379/0
CACHE_URL=redis://redis:6379/1

# Shared storage
STORAGE_DIR=/storage

Using these docker compose & docker env files, you should be able to reach the website through http://localhost:8000 after a docker compose up.

First run

On the very first run, you’ll need to initialize the database with the following command:

docker exec grappa-app django-admin migrate

Then you should be able to create your own administrative account:

docker exec -it grappa-app django-admin createsuperuser

Once the admin account is created, you can access the admin panel through http://localhost:8000/admin

There you should be able to create more users & projects.

Large projects

If you have projects with 100k+ elements with embeddings, you’ll need to run the asynchronous tasks for embedding reduction on a server with at least 8Gb of free RAM.

As this may not always be possible on VPS or other resource constrained environments, it’s possible to run the RAM-heavy tasks on a dedicated asynchronous queue.

This means you’ll need to run 2 separate workers (using the same Docker image registry.gitlab.teklia.com/arkindex/grappa/worker:X.Y.Z):

  1. for default tasks that can run on the web stack, without any specific needs, using the command celery -A grappa.base worker -l INFO --queues default

  2. for RAM-heavy tasks that should run on dedicated hardware, with some guaranteed free RAM, using the command celery -A grappa.base worker -l INFO --queues large_ram