Files
celtic-life-science/PRODUCTION.md
T
Shubhankar45 0d30c58bf0 Server backup 2026-11-07 of uncommitted work in projects/pharma-web-celtic
Snapshot of the working tree on srv1071960 (branch main), taken during
server cleanup. .env files, node_modules, builds, logs and media were excluded.
2026-11-07 05:43:12 +00:00

20 KiB
Raw Blame History

Production guide

How to deploy, secure, operate and recover the Celtic Life Science field-force system on a single server. This is the operator's document; README.md is the developer's.

Every command below was run against this codebase's Docker stack unless it is marked untested.

Contents

  1. What you are deploying
  2. Requirements
  3. First deployment
  4. Creating the real users
  5. Loading reference data
  6. Configuration
  7. Domain, HTTPS and reverse proxies
  8. Security checklist
  9. Backups and restore
  10. Updating and rolling back
  11. Monitoring and logs
  12. Runbook
  13. Go-live checklist
  14. Known limitations

1. What you are deploying

 browser ──HTTPS──▶ caddy ──/api /docs /health──▶ api (FastAPI, uvicorn)  ──▶ db (PostgreSQL 16)
                      │                             │
                      └──everything else──▶ web     └─▶ pdfstore volume (approved DCR PDFs)
                                        (static React build)
Service Image Role
caddy caddy:2-alpine The only thing that publishes ports. TLS (Let's Encrypt), gzip, security headers, routing.
api built from Dockerfile FastAPI app and PDF rendering (WeasyPrint). Non-root, read-only filesystem, all capabilities dropped. Runs alembic upgrade head on start.
web built from web/Dockerfile The React app, built at image-build time and served as static files.
db postgres:16-alpine Internal network only. Data in the pgdata volume.

Persistent state lives in exactly two volumes: pgdata (the database) and pdfstore (approved DCR PDFs). caddydata holds certificates. Everything else is rebuilt from the code.

Roles: MR (medical representative), ABM (Area Business Manager), AGM (Assistant General Manager), ADMIN. The MR files daily call reports and company forms, the ABM approves, the AGM has read access across territories, the ADMIN maintains people, organisation and master data.

2. Requirements

  • A Linux server with Docker Engine and the Compose plugin (docker compose version). 1 vCPU / 2 GB RAM / 20 GB disk is enough for one company and a few dozen users.
  • git, make, python3 (used by scripts/setup.sh to generate secrets; openssl is the fallback).
  • For HTTPS with a real domain: a DNS A record pointing at the server, and ports 80 and 443 free and reachable from the internet.
  • Outbound internet access at build time (base images, apt, pip, npm).

3. First deployment

git clone <your-repo-url> pharma-web-celtic && cd pharma-web-celtic   # or copy this folder
DOMAIN=app.example.com make deploy      # omit DOMAIN if you only have an IP address

make deploy runs scripts/setup.sh, then docker compose up -d --build, then waits for the API to report healthy. It is safe to re-run. setup.sh:

  • creates .env from .env.example when missing (.env is never committed and is not part of this repository);
  • generates a 64-character SECRET_KEY when missing or a placeholder;
  • sets ENVIRONMENT=production and DEBUG=false;
  • generates POSTGRES_PASSWORD for a new database only — an existing database keeps its password;
  • picks the ports: 80/443 when free, otherwise plain HTTP on the first free port from 2000 (HTTPS side 2443) and tells you which.

Verify:

make status
curl -s http://localhost:<HTTP_PORT>/health
# {"status":"ok","environment":"production","pdf_engine":"weasyprint"}

pdf_engine must say weasyprint. In production the API refuses to start if WeasyPrint's system libraries are missing or SECRET_KEY is a placeholder or shorter than 32 characters, so a bad image fails loudly at boot rather than producing broken PDFs later.

Running this next to another copy on the same machine. Compose derives the project name from the folder name, and volumes are named after it (<project>_pgdata). Two folders with the same name share volumes and fight over ports. Set COMPOSE_PROJECT_NAME=<something unique> in the environment (or .env) before the first make deploy of the second copy.

4. Creating the real users

There is no "create first admin" command. (An Administrator can reset any user's password, email and employee code from Users → Edit; users cannot change their own.) The only way to obtain a first administrator is the seed script, which creates demo accounts that all share one well-known password (ChangeMe123!, in scripts/seed.py). Treat that as a short-lived bootstrap and do the following before the server is reachable by anyone else (firewall closed, or on a private network):

  1. Seed once: docker compose exec api python -m scripts.seed (creates the company row, one headquarters BBSR, four demo users adm001 / gm001 / am001 / mr001 @acmepharma.in, 15 sample doctors and 5 chemists. Idempotent.)
  2. Sign in as adm001@acmepharma.in.
  3. Setup → Organisation: create the real headquarters. Setup → People: create the real users, one of them an Administrator, each with a strong password. Passwords cannot be changed later from the app, so choose them as final and hand them over securely.
  4. Sign out, sign in as the new administrator, open Users and deactivate all four demo accounts (deactivation also revokes their sessions). Verify adm001 can no longer sign in.
  5. Assign an Area Business Manager to each headquarters, then assign doctors and chemists to MRs (Setup → Assignments).

If you would rather not keep the demo doctors, chemists and the BBSR headquarters, deactivate them from Doctors & Chemists / leave them unassigned.

5. Loading reference data

Stock / Products list — the Products page reads the stock_inventory table. It is created and filled (26 products from stock.csv) by the migration h8d4f0a6c3e5, so it is there after the first start with nothing to do. Administrators can add, edit and delete products from the Products page.

Sales Projection and Sales & Stock product lists — the Sales Projection form needs its product list (name and PTS) in the database, and the Sales & Stock Statement lists the same products. Load them once with the non-destructive mode of the import script. The CSVs are deliberately not part of the Docker image and the API container is read-only, so mount the two files into a one-off container:

docker compose run --rm --no-deps -T \
  -v "$PWD/sales_projection.csv:/srv/sales_projection.csv:ro" \
  -v "$PWD/Sales_and_Stock_Statement.csv:/srv/Sales_and_Stock_Statement.csv:ro" \
  api python -m scripts.import_csv_data --only-empty

--only-empty loads a table only if it has no rows and prints skipped, already has N rows otherwise, so it can be re-run safely. Never run the script without --only-empty on a database that holds real data — the plain form deletes every form table before loading.

6. Configuration

Only variables listed in docker-compose.yml reach the API container. Set them in .env.

Variable Default Purpose
SECRET_KEY (required) JWT signing key, 64+ random characters. Changing it signs everyone out.
POSTGRES_PASSWORD (required) Database password. See rotation.
POSTGRES_USER, POSTGRES_DB pharma, pharma Database user and name.
ENVIRONMENT production production enables the boot-time safety checks.
DEBUG false Keep false.
DOMAIN localhost Host name Caddy serves; :80 means plain HTTP (behind another proxy / no TLS).
HTTP_PORT, HTTPS_PORT 80, 443 Host ports Caddy publishes.
APP_TIMEZONE Asia/Kolkata Business timezone. "Today" and the visit-time check use it, not the server's UTC clock. (Set under api.environment: in docker-compose.yml to change it.)
BACKDATING_WINDOW_DAYS 7 How many days back an MR may file a report.
SUBMISSION_DEADLINE_DAYS 1 A report for day D must be submitted by end of D + this many days.
WEB_CONCURRENCY 2 uvicorn worker processes.
RUN_MIGRATIONS 1 Set 0 on the api service to skip migrations at start.

Other application settings (app/config/settings.py) — the letterhead printed on PDFs (PDF_COMPANY_NAME, _ADDRESS, _WEBSITE, _EMAIL, _PHONE), ACCESS_TOKEN_EXPIRE_MINUTES (15), REFRESH_TOKEN_EXPIRE_DAYS (7), LOGIN_MAX_FAILED_ATTEMPTS (10) — are not passed through by the shipped compose file. To change one, add it under api.environment: in docker-compose.yml and run docker compose up -d. The defaults already carry Celtic's details.

7. Domain, HTTPS and reverse proxies

Situation Result
Ports 80/443 free and DOMAIN=app.example.com (DNS already pointing here) Caddy obtains and renews a Let's Encrypt certificate itself.
Ports 80/443 free, no domain DOMAIN=localhost: HTTPS with a self-signed certificate (browser warning).
80/443 owned by another service DOMAIN=:80 on HTTP_PORT (e.g. 2000). Put your existing proxy in front.

Existing nginx terminating TLS in front of Caddy:

server {
    server_name app.example.com;
    listen 443 ssl;                          # + your certificate directives
    location / {
        proxy_pass http://127.0.0.1:2000;    # the HTTP_PORT chosen by setup.sh
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $remote_addr;
    }
}

Block the published port from the internet when a proxy on the same host fronts it (sudo ufw deny 2000/tcp; the proxy still reaches 127.0.0.1:2000).

Once HTTPS works on the real domain, enable HSTS: uncomment the Strict-Transport-Security line in Caddyfile and docker compose up -d. Do not enable it earlier — it locks browsers into HTTPS for a year.

8. Security checklist

  • .env is chmod 600, owned by the deploy user, and is not in any repository or backup that leaves the server unencrypted.
  • SECRET_KEY and POSTGRES_PASSWORD are generated values (setup.sh does this). setup.sh warns if the database still uses the default pharma password — rotate it.
  • The four demo accounts are deactivated (section 4).
  • Only ports 80/443 (or your proxy) are open. The database and API are not published; do not use docker-compose.dev.yml on a server — it publishes them.
  • HTTPS on the real domain, then HSTS enabled.
  • The host is patched, SSH is key-only, and docker group membership is limited to operators.
  • Backups run and a restore has been rehearsed (section 9).
  • Decide about the public API docs: /docs, /redoc and /openapi.json are reachable by anyone through Caddy in the shipped configuration (they list every endpoint, though not data). To hide them, delete the corresponding handle blocks in Caddyfile and docker compose up -d.
  • Someone reads the API logs for repeated 401s (accounts lock after 10 failed sign-ins).

What the containers already enforce: API runs as a non-root user on a read-only filesystem with all Linux capabilities dropped and no-new-privileges; the database is unreachable from outside the compose network; Caddy sends X-Content-Type-Options, X-Frame-Options: DENY and a strict Referrer-Policy, and hides the server banner; JSON logs rotate at 10 MB × 5 files per service.

Audit log hardening (optional, not enabled by default). scripts/harden_audit.sql creates a pharma_app role that can insert into and read audit_log but never update or delete it. The shipped compose file connects with a single owner role that also runs migrations, so enabling it means splitting credentials: run migrations with the owner role (set RUN_MIGRATIONS=0 on api and run alembic upgrade head in a one-off container) and point the running API's DATABASE_URL at pharma_app, after changing that role's placeholder password in the script. This flow is untested in this repository — rehearse it on a staging copy first.

9. Backups and restore

What to back up: the database (pg_dump) and the pdfstore volume. Everything else is code or regenerable. Take a backup before every update.

Database backup (verified):

set -a; . ./.env; set +a
docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > backup-$(date +%F).dump

PDF store backup:

docker run --rm -v "$(basename "$PWD")_pdfstore:/data:ro" -v "$PWD:/backup" alpine \
  tar czf /backup/pdfstore-$(date +%F).tgz -C /data .

(Adjust the volume name if you set COMPOSE_PROJECT_NAME; list volumes with docker volume ls.)

Nightly cron (as the deploy user, crontab -e; copy the dumps off the server too):

15 2 * * * cd /opt/pharma-web-celtic && set -a && . ./.env && set +a && docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > /var/backups/pharma/db-$(date +\%F).dump && find /var/backups/pharma -name 'db-*.dump' -mtime +30 -delete

Restore rehearsal — restores into a scratch database so nothing live is touched (verified):

set -a; . ./.env; set +a
docker compose exec -T db createdb -U "$POSTGRES_USER" restore_check
docker compose exec -T db pg_restore -U "$POSTGRES_USER" -d restore_check --no-owner < backup-YYYY-MM-DD.dump
docker compose exec -T db psql -U "$POSTGRES_USER" -d restore_check \
  -c 'select count(*) from "user"' -c 'select count(*) from stock_inventory'
docker compose exec -T db dropdb -U "$POSTGRES_USER" restore_check

Real disaster recovery (empty server, backup in hand): deploy as in section 3, then docker compose stop api, restore into the empty database (pg_restore -U … -d "$POSTGRES_DB" --clean --if-exists --no-owner < backup.dump), restore the PDF archive into the pdfstore volume, then docker compose up -d. Use the same SECRET_KEY if you want existing sessions to survive; otherwise users simply sign in again.

10. Updating and rolling back

# 1. back up (section 9)
# 2. get the new code
git pull            # or replace the folder contents
# 3. rebuild and restart — migrations run automatically on api start
make deploy
make status

The API image is rebuilt, so pip/apt/npm need network access. Compose recreates only what changed; the database and volumes are untouched. Check docker compose logs api | grep alembic to see which migrations ran.

Rollback: restore the previous code, and if the new release added migrations, step the schema back first, using the new image (it knows those revisions):

docker compose exec api alembic downgrade -1     # once per migration to undo; check with: alembic current
# then deploy the previous code: git checkout <previous-tag> && make deploy

If in doubt, restore the pre-update database backup instead — it is always the safe path.

11. Monitoring and logs

  • Health: GET /health → {"status":"ok", ..., "pdf_engine":"weasyprint"}. Docker health checks run on api and db; docker compose ps shows (healthy). Point an external uptime monitor at https://<domain>/health.
  • Logs: docker compose logs -f api (also caddy, db, web). Rotated automatically.
  • Disk: watch the Docker data directory (docker system df); the pgdata and pdfstore volumes only grow.
  • Errors: no error tracker is wired in. Add Sentry or similar if you need alerting on 500s.

12. Runbook

Task Command
Status / URL make status
Restart the API docker compose restart api
Stop everything (data kept) docker compose down
Delete all data docker compose down -v — never on a live system
Open a database shell docker compose exec db psql -U "$POSTGRES_USER" "$POSTGRES_DB"
Unlock a user (10 failed sign-ins) UPDATE "user" SET failed_login_attempts = 0 WHERE email = '…'; in the shell above
Rotate SECRET_KEY edit .env, docker compose up -d — everyone must sign in again
Rotate the database password see below

Rotating the database password — the database keeps the password it was created with, so changing .env alone would lock the API out:

NEW=$(python3 -c "import secrets;print(secrets.token_urlsafe(24))")
docker compose exec db psql -U pharma -d pharma -c "ALTER USER pharma PASSWORD '$NEW';"
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$NEW|" .env
docker compose up -d

Common failures

Symptom Cause / fix
make deploy says set POSTGRES_PASSWORD in .env Run make setup (or ./scripts/setup.sh).
API restarts in a loop, log says SECRET_KEY must be a random value .env has a placeholder key; run make setup.
API log: WeasyPrint system libraries are required The image was built without Pango/fonts; rebuild with docker compose build --no-cache api.
Caddy cannot bind port 80/443 Another service owns them; run make setup (it moves Caddy to a free port) and put a proxy in front.
API cannot connect after editing .env The database still has the old password — see rotation above.
A page shows "This didn't load — Request failed (500)" docker compose logs api shows the cause; after an update check the migration ran (alembic current).
PDFs look plain-text The API logged the WeasyPrint fallback warning; the image is missing native libraries.

13. Go-live checklist

  1. make deploy finishes and /health reports weasyprint.
  2. Real headquarters and users created; all demo accounts deactivated (section 4).
  3. Reference data loaded (section 5): Products page shows 26 products; Sales Projection shows the product list.
  4. One end-to-end pass with real accounts: MR files a DCR → ABM approves → PDF downloads and opens; each company form downloads a PDF that matches the printed original.
  5. HTTPS works on the final domain; HSTS enabled.
  6. Firewall: only 80/443 (or the proxy) open.
  7. First backup taken and a restore rehearsed; cron installed.
  8. Uptime monitor pointed at /health; someone owns reading the logs.
  9. Business rules confirmed with the client: the back-dating window and submission deadline (BACKDATING_WINDOW_DAYS, SUBMISSION_DEADLINE_DAYS) are shipped assumptions.

14. Known limitations

  • Users cannot change their own password. Only an Administrator can, from Users → Edit (also email and employee code). A reset signs that user out everywhere and clears a lockout, but an access token already issued stays valid until it expires (15 minutes). If the last Administrator password is lost, create a new admin with a one-off script (see scripts/) — there is no self-service recovery and no email flow.
  • Company-form approval is per form, not per row. Tour plan, standard tour plan and fare chart each have one approval state, stored on the server (form_approval): the MR sends a form that has rows, the Area Business Manager approves it, and the PDF signature shows that name. Any change to the rows withdraws the approval. The signature is decided by the server, never by the browser.
  • Audit-log immutability is opt-in (section 8) and untested with the shipped compose file.
  • No built-in alerting or error tracking; only health checks and logs.
  • Single server, single database. There is no high availability; recovery time is "restore the latest backup". That is the deliberate trade-off for this size of deployment.
  • The test suite needs a reachable PostgreSQL; some tests under tests/unit do too.