TieMeasureFlow/docs/DEPLOYMENT.md

# TieMeasureFlow Deployment Guide

This guide covers deploying TieMeasureFlow in development, staging, and production environments.

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Environment Setup](#environment-setup)
3. [Database Setup](#database-setup)
4. [Server Deployment](#server-deployment)
5. [Client Deployment](#client-deployment)
6. [Production Deployment](#production-deployment)
7. [Docker Deployment](#docker-deployment)
8. [SSL/HTTPS](#ssltls)
9. [Backup Strategy](#backup-strategy)
10. [Troubleshooting](#troubleshooting)

## Prerequisites

### System Requirements

- **OS**: Linux, macOS, or Windows
- **Python**: 3.11 or higher
- **MySQL**: 8.0 or higher
- **Node.js**: 16+ (optional, for Tailwind CSS compilation)
- **Disk Space**: 500 MB minimum
- **RAM**: 2 GB minimum

### Software Installation

#### Linux/macOS

```bash
# Python 3.11+
python3 --version

# MySQL 8.0
mysql --version

# Node.js (optional)
node --version
npm --version
```

#### Windows

Download and install:
- [Python 3.11+](https://www.python.org/downloads/)
- [MySQL Community Server](https://dev.mysql.com/downloads/mysql/)
- [Node.js](https://nodejs.org/) (optional)

---

## Environment Setup

### 1. Clone Repository

```bash
git clone <repository-url>
cd TieMeasureFlow
```

### 2. Create .env File

Copy the example and customize for your environment:

```bash
cp .env.example .env
```

### 3. Configure .env

Edit `.env` with your settings:

```env
# =====================================================================
# DATABASE
# =====================================================================
DB_HOST=localhost
DB_PORT=3306
DB_NAME=tiemeasureflow
DB_USER=tmflow
DB_PASSWORD=change_me_in_production

# =====================================================================
# SERVER (FastAPI)
# =====================================================================
SERVER_HOST=0.0.0.0
SERVER_PORT=8000
SERVER_SECRET_KEY=change-this-to-a-random-secret-key-with-32-chars

# =====================================================================
# CLIENT (Flask)
# =====================================================================
CLIENT_HOST=0.0.0.0
CLIENT_PORT=5000

# =====================================================================
# CORS
# =====================================================================
SERVER_CORS_ORIGINS=http://localhost:5000,http://127.0.0.1:5000

# =====================================================================
# FILE UPLOAD
# =====================================================================
UPLOAD_DIR=uploads
MAX_UPLOAD_SIZE_MB=50

# =====================================================================
# RATE LIMITING
# =====================================================================
RATE_LIMIT_LOGIN=5
RATE_LIMIT_GENERAL=100

# =====================================================================
# SSL/HTTPS (Production only)
# =====================================================================
SSL_CERTFILE=
SSL_KEYFILE=
```

### Environment Variables Reference

| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `DB_HOST` | string | localhost | MySQL server hostname |
| `DB_PORT` | int | 3306 | MySQL server port |
| `DB_NAME` | string | tiemeasureflow | Database name |
| `DB_USER` | string | tmflow | Database user |
| `DB_PASSWORD` | string | change_me_in_production | Database password **[CHANGE IN PROD]** |
| `SERVER_HOST` | string | 0.0.0.0 | API server bind address |
| `SERVER_PORT` | int | 8000 | API server port |
| `SERVER_SECRET_KEY` | string | change-this-to... | Secret key for sessions **[CHANGE IN PROD]** |
| `CLIENT_HOST` | string | 0.0.0.0 | Flask client bind address |
| `CLIENT_PORT` | int | 5000 | Flask client port |
| `SERVER_CORS_ORIGINS` | string | http://localhost:5000 | Comma-separated CORS origins |
| `UPLOAD_DIR` | string | uploads | Directory for file uploads |
| `MAX_UPLOAD_SIZE_MB` | int | 50 | Maximum upload file size in MB |
| `RATE_LIMIT_LOGIN` | int | 5 | Login requests per minute |
| `RATE_LIMIT_GENERAL` | int | 100 | General requests per minute |
| `SSL_CERTFILE` | string | (empty) | Path to SSL certificate (production) |
| `SSL_KEYFILE` | string | (empty) | Path to SSL private key (production) |

---

## Database Setup

### 1. Create MySQL Database and User

```bash
mysql -u root -p
```

```sql
-- Create database
CREATE DATABASE tiemeasureflow CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Create user
CREATE USER 'tmflow'@'localhost' IDENTIFIED BY 'secure_password_here';

-- Grant permissions
GRANT ALL PRIVILEGES ON tiemeasureflow.* TO 'tmflow'@'localhost';
FLUSH PRIVILEGES;

-- Exit
EXIT;
```

### 2. Run Database Migrations

```bash
cd server
pip install -r requirements.txt
alembic upgrade head
```

This creates all required tables:
- `users` - System users
- `recipes` - Measurement recipes
- `recipe_versions` - Immutable recipe versions
- `recipe_tasks` - Tasks within recipes
- `recipe_subtasks` - Subtasks within tasks (individual measurements)
- `measurements` - Recorded measurements
- `access_logs` - API access audit trail
- `system_settings` - Configuration key-value pairs
- `recipe_version_audit` - Recipe change history

### 3. Create Initial Admin User (Optional)

Use the Flask client to create the first admin user, or run:

```bash
cd server
python -c "
from database import init_db, SessionLocal
from services.auth_service import create_user
import asyncio

async def init():
    await init_db()
    async with SessionLocal() as db:
        await create_user(
            db,
            username='admin',
            password='change_me_first_login',
            display_name='Admin',
            email='admin@example.com',
            roles=['Maker', 'MeasurementTec', 'Metrologist'],
            is_admin=True
        )
        await db.commit()
        print('Admin user created: admin / change_me_first_login')

asyncio.run(init())
"
```

---

## Server Deployment

### Development Server

```bash
cd server
pip install -r requirements.txt
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

Runs at `http://0.0.0.0:8000`

API Documentation:
- Swagger UI: `http://localhost:8000/docs`
- ReDoc: `http://localhost:8000/redoc`

### Production Server (Gunicorn + Uvicorn)

Install Gunicorn:

```bash
pip install gunicorn
```

Start server:

```bash
cd server
gunicorn main:app \
  --workers 4 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8000 \
  --access-logfile /var/log/tiemeasureflow/access.log \
  --error-logfile /var/log/tiemeasureflow/error.log \
  --log-level info
```

### Production Server (Waitress - Windows)

Install Waitress:

```bash
pip install waitress
```

Start server:

```bash
cd server
waitress-serve --host=0.0.0.0 --port=8000 main:app
```

### systemd Service (Linux)

Create `/etc/systemd/system/tiemeasureflow-api.service`:

```ini
[Unit]
Description=TieMeasureFlow API Server
After=network.target mysql.service

[Service]
Type=notify
User=tiemeasureflow
WorkingDirectory=/opt/tiemeasureflow/server
Environment="PATH=/opt/tiemeasureflow/venv/bin"
ExecStart=/opt/tiemeasureflow/venv/bin/gunicorn main:app \
  --workers 4 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8000

Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
```

Enable and start:

```bash
sudo systemctl daemon-reload
sudo systemctl enable tiemeasureflow-api.service
sudo systemctl start tiemeasureflow-api.service
sudo systemctl status tiemeasureflow-api.service
```

View logs:

```bash
sudo journalctl -u tiemeasureflow-api -f
```

---

## Client Deployment

### Development Server

```bash
cd client
pip install -r requirements.txt
flask run --host 0.0.0.0 --port 5000
```

Runs at `http://0.0.0.0:5000`

### Compile Tailwind CSS (Optional)

```bash
cd client
npx tailwindcss -i static/css/input.css -o static/css/tailwind.css --watch
```

### Production Server (Gunicorn)

```bash
pip install gunicorn
cd client
gunicorn --workers 4 --bind 0.0.0.0:5000 app:app
```

### systemd Service (Linux)

Create `/etc/systemd/system/tiemeasureflow-web.service`:

```ini
[Unit]
Description=TieMeasureFlow Web Client
After=network.target tiemeasureflow-api.service

[Service]
Type=notify
User=tiemeasureflow
WorkingDirectory=/opt/tiemeasureflow/client
Environment="PATH=/opt/tiemeasureflow/venv/bin"
Environment="FLASK_ENV=production"
ExecStart=/opt/tiemeasureflow/venv/bin/gunicorn \
  --workers 4 \
  --bind 0.0.0.0:5000 \
  app:app

Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
```

---

## Production Deployment

### Security Checklist

- [ ] Change all default passwords in `.env`
- [ ] Generate random `SERVER_SECRET_KEY` (32+ characters)
- [ ] Set CORS origins to actual client domains
- [ ] Enable SSL/HTTPS (see SSL/TLS section)
- [ ] Configure firewall rules
- [ ] Set up regular database backups
- [ ] Configure log rotation
- [ ] Use strong database credentials
- [ ] Restrict file upload sizes
- [ ] Keep dependencies updated

### Recommended Architecture

```
┌─────────────────┐
│  Nginx Proxy    │ (Port 80/443)
└────────┬────────┘
         │
    ┌────┴────┐
    │          │
┌───▼──┐  ┌───▼──┐
│API   │  │Web   │
│:8000 │  │:5000 │
└──────┘  └──────┘
    │          │
    └────┬─────┘
         │
    ┌────▼────┐
    │ MySQL   │
    │ :3306   │
    └─────────┘
```

### Nginx Configuration

Create `/etc/nginx/sites-available/tiemeasureflow`:

```nginx
upstream tiemeasureflow_api {
    server 127.0.0.1:8000;
}

upstream tiemeasureflow_web {
    server 127.0.0.1:5000;
}

server {
    listen 80;
    server_name yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name yourdomain.com;

    ssl_certificate /etc/ssl/certs/yourdomain.com.crt;
    ssl_certificate_key /etc/ssl/private/yourdomain.com.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    client_max_body_size 50M;

    # API
    location /api/ {
        proxy_pass http://tiemeasureflow_api;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Web Client
    location / {
        proxy_pass http://tiemeasureflow_web;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Static files (optional caching)
    location /static/ {
        proxy_pass http://tiemeasureflow_web;
        expires 1d;
        add_header Cache-Control "public, immutable";
    }
}
```

Enable and test:

```bash
sudo ln -s /etc/nginx/sites-available/tiemeasureflow /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx
```

---

## Docker Deployment

### Docker Compose (Complete Stack)

The project includes a `docker-compose.yml` for easy deployment:

```bash
# Build images
docker-compose build

# Start services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down
```

### Manual Docker Build

#### API Server

```dockerfile
FROM python:3.11-slim

WORKDIR /app/server

COPY server/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY server/ .
COPY .env ..

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```

Build and run:

```bash
docker build -f server/Dockerfile -t tiemeasureflow-api .
docker run -p 8000:8000 --env-file .env tiemeasureflow-api
```

#### Web Client

```dockerfile
FROM python:3.11-slim

WORKDIR /app/client

COPY client/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY client/ .
COPY .env ..

EXPOSE 5000

CMD ["gunicorn", "--workers", "4", "--bind", "0.0.0.0:5000", "app:app"]
```

Build and run:

```bash
docker build -f client/Dockerfile -t tiemeasureflow-web .
docker run -p 5000:5000 --env-file .env tiemeasureflow-web
```

---

## SSL/TLS

### Self-Signed Certificate (Development)

```bash
openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 365 -nodes
```

### Let's Encrypt (Production)

```bash
sudo apt install certbot python3-certbot-nginx
sudo certbot certonly --nginx -d yourdomain.com
```

### Configure in .env

```env
SSL_CERTFILE=/etc/ssl/certs/yourdomain.com.crt
SSL_KEYFILE=/etc/ssl/private/yourdomain.com.key
```

### Start Server with SSL

```bash
cd server
gunicorn main:app \
  --workers 4 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8443 \
  --certfile /etc/ssl/certs/yourdomain.com.crt \
  --keyfile /etc/ssl/private/yourdomain.com.key
```

---

## Backup Strategy

### Daily Database Backups

Create backup script `/opt/tiemeasureflow/backup.sh`:

```bash
#!/bin/bash

BACKUP_DIR="/opt/tiemeasureflow/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_NAME="tiemeasureflow"
DB_USER="tmflow"

mkdir -p $BACKUP_DIR

# MySQL dump
mysqldump -u $DB_USER -p$MYSQL_PASSWORD $DB_NAME \
  | gzip > $BACKUP_DIR/db_$TIMESTAMP.sql.gz

# Keep only last 30 days
find $BACKUP_DIR -name "db_*.sql.gz" -mtime +30 -delete

# Upload to S3 (optional)
# aws s3 cp $BACKUP_DIR/db_$TIMESTAMP.sql.gz s3://your-bucket/backups/

echo "Backup completed: $BACKUP_DIR/db_$TIMESTAMP.sql.gz"
```

Schedule with crontab:

```bash
crontab -e
```

Add line:

```cron
0 2 * * * /opt/tiemeasureflow/backup.sh >> /var/log/tiemeasureflow/backup.log 2>&1
```

### Restore from Backup

```bash
# Decompress
gunzip < backups/db_20250207_020000.sql.gz | \
  mysql -u tmflow -p tiemeasureflow
```

---

## Troubleshooting

### Database Connection Failed

```
sqlalchemy.exc.OperationalError: (asyncmy.errors.ProgrammingError) (2003, "Can't connect to MySQL server...")
```

**Check:**
```bash
# Verify MySQL is running
mysql -u tmflow -p -e "SELECT 1"

# Check .env variables
grep DB_ .env

# Test connection string
cd server && python -c "from config import settings; print(settings.database_url)"
```

### Port Already in Use

```
OSError: [Errno 48] Address already in use
```

**Solution:**
```bash
# Find process on port 8000
lsof -i :8000
kill -9 <PID>

# Or use different port
uvicorn main:app --port 8001
```

### Import Errors

```
ModuleNotFoundError: No module named 'fastapi'
```

**Solution:**
```bash
cd server
pip install -r requirements.txt
```

### File Upload Not Working

```
/app/server/uploads: Permission denied
```

**Solution:**
```bash
# Create uploads directory with correct permissions
mkdir -p server/uploads
chmod 755 server/uploads
```

### Migrations Failed

```
alembic.util.exc.CommandError: Can't locate revision identified by...
```

**Solution:**
```bash
cd server
# Check migration status
alembic current

# Reset migrations (CAUTION - deletes data)
alembic downgrade base
alembic upgrade head
```

### API Key Not Working

```
detail: "Invalid API key"
```

**Solution:**
```bash
# Regenerate API key for user (admin endpoint)
curl -X POST http://localhost:8000/api/users/1/regenerate-key \
  -H "X-API-Key: admin_token"
```

---

## Performance Tuning

### MySQL

Edit `/etc/mysql/mysql.conf.d/mysqld.cnf`:

```ini
[mysqld]
# Connection pool
max_connections = 100

# Buffer sizes
innodb_buffer_pool_size = 256M
innodb_log_file_size = 100M

# Query optimization
query_cache_size = 0
query_cache_type = 0

# Logging
slow_query_log = 1
long_query_time = 2
```

Restart:
```bash
sudo systemctl restart mysql
```

### Application Workers

Increase Gunicorn workers (rule: 2 * CPU_cores + 1):

```bash
gunicorn --workers 9 ...  # For 4-core system
```

### Nginx Caching

Add to nginx config:

```nginx
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;

location /api/statistics/ {
    proxy_cache api_cache;
    proxy_cache_valid 200 10m;
    add_header X-Cache-Status $upstream_cache_status;
}
```

---

## Monitoring & Logging

### Application Logs

```bash
# API server
tail -f /var/log/tiemeasureflow/api_error.log

# Web client
tail -f /var/log/tiemeasureflow/web_error.log

# System
journalctl -u tiemeasureflow-api -f
```

### Health Check Endpoint

```bash
curl http://localhost:8000/api/health
```

Response:
```json
{
  "status": "ok",
  "service": "TieMeasureFlow API",
  "version": "0.1.0"
}
```

### Database Connection Pooling

Monitor with:

```bash
# MySQL connections
mysql -e "SHOW PROCESSLIST;" | grep tiemeasureflow
```