First Deployment

Step-by-step guide for deploying SysManage in production environments with proper security and scalability.

Production Deployment Overview

This guide covers deploying SysManage for production use, including proper security configuration, database setup, SSL/TLS certificates, and scalability considerations. Follow these steps to ensure a robust, secure deployment.

Prerequisites: Complete the Quick Start Guide first to understand basic SysManage concepts and functionality.

1. Deployment Planning

Infrastructure Requirements

Production Server Specifications

CPU: 4+ cores (8+ for large deployments)
Memory: 8GB RAM minimum (16GB+ recommended)
Storage: 100GB minimum (SSD preferred)
Network: Dedicated IP, firewall access

Database Requirements

PostgreSQL: Version 12 or newer
Storage: Fast SSD with adequate IOPS
Backup: Automated backup solution
High Availability: Consider clustering for critical deployments

Network Architecture

Plan your network architecture considering these components:

Web Interface: HTTPS/443 (public or internal access)
Agent Communication: Custom port (default 8444) with mTLS
Database: Internal network only (port 5432)
Load Balancer: For high availability deployments

2. Database Setup

PostgreSQL Installation

Ubuntu/Debian

# Install PostgreSQL
sudo apt update
sudo apt install postgresql postgresql-contrib

# Create database and user
sudo -u postgres createdb sysmanage
sudo -u postgres createuser sysmanage
sudo -u postgres psql -c "ALTER USER sysmanage WITH PASSWORD 'your-secure-password';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE sysmanage TO sysmanage;"

RHEL/CentOS/Fedora

# Install PostgreSQL
sudo dnf install postgresql postgresql-server postgresql-contrib

# Initialize database
sudo postgresql-setup --initdb
sudo systemctl enable postgresql
sudo systemctl start postgresql

# Create database and user
sudo -u postgres createdb sysmanage
sudo -u postgres createuser sysmanage
sudo -u postgres psql -c "ALTER USER sysmanage WITH PASSWORD 'your-secure-password';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE sysmanage TO sysmanage;"

Database Configuration

Configure PostgreSQL for production use by editing /etc/postgresql/*/main/postgresql.conf:

# Memory settings
shared_buffers = 256MB
effective_cache_size = 1GB
work_mem = 4MB

# Connection settings
max_connections = 100
listen_addresses = 'localhost'

# WAL settings for backups
wal_level = replica
max_wal_senders = 3

Database Security

Configure authentication in /etc/postgresql/*/main/pg_hba.conf:

# Local connections
local   sysmanage    sysmanage                     md5
host    sysmanage    sysmanage    127.0.0.1/32     md5
host    sysmanage    sysmanage    ::1/128          md5

3. SSL/TLS Certificate Configuration

Option 1: Let's Encrypt (Recommended)

# Install Certbot
sudo apt install certbot  # Ubuntu/Debian
sudo dnf install certbot  # RHEL/CentOS/Fedora

# Obtain certificate (DNS challenge recommended for servers)
sudo certbot certonly --standalone -d sysmanage.yourdomain.com

# Certificate files will be created at:
# /etc/letsencrypt/live/sysmanage.yourdomain.com/fullchain.pem
# /etc/letsencrypt/live/sysmanage.yourdomain.com/privkey.pem

Option 2: Self-Signed Certificate (Testing Only)

# Generate self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout /etc/ssl/private/sysmanage.key \
    -out /etc/ssl/certs/sysmanage.crt \
    -subj "/C=US/ST=State/L=City/O=Organization/CN=sysmanage.yourdomain.com"

# Set proper permissions
sudo chmod 600 /etc/ssl/private/sysmanage.key
sudo chmod 644 /etc/ssl/certs/sysmanage.crt

Certificate Renewal Automation

For Let's Encrypt certificates, set up automatic renewal:

# Add to crontab
sudo crontab -e

# Add this line for automatic renewal at 2 AM daily
0 2 * * * /usr/bin/certbot renew --quiet --deploy-hook "systemctl restart sysmanage"

4. SysManage Server Deployment

Production Installation

# Create dedicated user
sudo useradd -r -m -s /bin/bash sysmanage
sudo usermod -aG ssl-cert sysmanage  # For certificate access

# Clone repository
sudo -u sysmanage git clone https://github.com/bceverly/sysmanage.git /opt/sysmanage
cd /opt/sysmanage

# Install dependencies
sudo -u sysmanage python3 -m pip install --user -r requirements.txt

Production Configuration

Create production configuration file at /opt/sysmanage/config/production.yaml:

# SysManage Production Configuration
server:
  host: "0.0.0.0"
  port: 8443
  ssl_enabled: true
  ssl_cert_file: "/etc/letsencrypt/live/sysmanage.yourdomain.com/fullchain.pem"
  ssl_key_file: "/etc/letsencrypt/live/sysmanage.yourdomain.com/privkey.pem"

database:
  type: "postgresql"
  host: "localhost"
  port: 5432
  name: "sysmanage"
  username: "sysmanage"
  password: "your-secure-password"
  pool_size: 20
  max_overflow: 30

security:
  secret_key: "your-256-bit-secret-key-here"
  jwt_expiration_hours: 24
  password_policy:
    min_length: 12
    require_uppercase: true
    require_lowercase: true
    require_numbers: true
    require_symbols: true

agent_communication:
  port: 8444
  ssl_enabled: true
  ssl_cert_file: "/etc/letsencrypt/live/sysmanage.yourdomain.com/fullchain.pem"
  ssl_key_file: "/etc/letsencrypt/live/sysmanage.yourdomain.com/privkey.pem"

logging:
  level: "INFO"
  file: "/var/log/sysmanage/sysmanage.log"
  max_size_mb: 100
  backup_count: 10

monitoring:
  metrics_enabled: true
  health_check_interval: 60

Directory Structure

# Create necessary directories
sudo mkdir -p /var/log/sysmanage
sudo mkdir -p /var/lib/sysmanage
sudo mkdir -p /etc/sysmanage

# Set ownership
sudo chown -R sysmanage:sysmanage /var/log/sysmanage /var/lib/sysmanage /opt/sysmanage
sudo chown root:sysmanage /etc/sysmanage
sudo chmod 750 /etc/sysmanage

5. Systemd Service Configuration

Create Service File

Create /etc/systemd/system/sysmanage.service:

[Unit]
Description=SysManage Infrastructure Management Server
After=network.target postgresql.service
Requires=postgresql.service

[Service]
Type=simple
User=sysmanage
Group=sysmanage
WorkingDirectory=/opt/sysmanage
Environment=PATH=/opt/sysmanage/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Environment=SYSMANAGE_CONFIG=/opt/sysmanage/config/production.yaml
ExecStart=/opt/sysmanage/venv/bin/python -m sysmanage.server
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=sysmanage

# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/log/sysmanage /var/lib/sysmanage
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true

[Install]
WantedBy=multi-user.target

Enable and Start Service

# Reload systemd and enable service
sudo systemctl daemon-reload
sudo systemctl enable sysmanage
sudo systemctl start sysmanage

# Check service status
sudo systemctl status sysmanage

# View logs
sudo journalctl -u sysmanage -f

6. Reverse Proxy Setup (Optional)

Nginx Configuration

For additional security and features, configure Nginx as a reverse proxy:

server {
    listen 80;
    server_name sysmanage.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name sysmanage.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/sysmanage.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/sysmanage.yourdomain.com/privkey.pem;

    # SSL configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512;
    ssl_prefer_server_ciphers off;

    # Security headers
    add_header Strict-Transport-Security "max-age=63072000" always;
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    location / {
        proxy_pass https://127.0.0.1:8443;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

7. Firewall Configuration

UFW (Ubuntu/Debian)

# Enable firewall
sudo ufw enable

# Allow SSH (adjust port as needed)
sudo ufw allow 22/tcp

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Allow SysManage agent communication
sudo ufw allow 8444/tcp

# Check status
sudo ufw status verbose

Firewalld (RHEL/CentOS/Fedora)

# Enable firewall
sudo systemctl enable firewalld
sudo systemctl start firewalld

# Allow services
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --add-port=8444/tcp

# Reload configuration
sudo firewall-cmd --reload

# Check status
sudo firewall-cmd --list-all

8. Monitoring and Logging

Log Rotation

Configure log rotation by creating /etc/logrotate.d/sysmanage:

/var/log/sysmanage/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 644 sysmanage sysmanage
    postrotate
        systemctl reload sysmanage
    endscript
}

Health Monitoring

Create a simple health check script at /opt/sysmanage/scripts/health-check.sh:

#!/bin/bash
# SysManage Health Check

URL="https://localhost:8443/api/health"
RESPONSE=$(curl -s -k -o /dev/null -w "%{http_code}" $URL)

if [ "$RESPONSE" = "200" ]; then
    echo "SysManage is healthy"
    exit 0
else
    echo "SysManage health check failed (HTTP $RESPONSE)"
    exit 1
fi

Performance Monitoring

Monitor these key metrics:

System Resources: CPU, memory, disk usage
Database Performance: Connection count, query performance
Application Metrics: Response times, error rates
Agent Connectivity: Connected agents, communication errors

9. Backup Strategy

Database Backup

Create automated database backup script:

#!/bin/bash
# Database backup script

BACKUP_DIR="/var/backups/sysmanage"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="sysmanage_backup_$DATE.sql"

mkdir -p $BACKUP_DIR
pg_dump -h localhost -U sysmanage sysmanage > $BACKUP_DIR/$BACKUP_FILE
gzip $BACKUP_DIR/$BACKUP_FILE

# Keep only last 30 days of backups
find $BACKUP_DIR -name "*.gz" -mtime +30 -delete

Configuration Backup

Backup configuration files and certificates:

# Configuration backup
tar -czf /var/backups/sysmanage/config_$(date +%Y%m%d).tar.gz \
    /opt/sysmanage/config/ \
    /etc/letsencrypt/ \
    /etc/systemd/system/sysmanage.service

10. High Availability (Advanced)

Load Balancer Setup

For high availability, deploy multiple SysManage instances behind a load balancer:

Deploy 2+ SysManage server instances
Use shared PostgreSQL database with clustering
Configure load balancer (HAProxy, Nginx, or cloud LB)
Implement session affinity for WebSocket connections

Database Clustering

Consider PostgreSQL clustering solutions:

Patroni: Automatic failover and leader election
PostgreSQL Streaming Replication: Built-in replication
Cloud Solutions: AWS RDS, Azure Database, Google Cloud SQL

11. Security Hardening

System Security

Keep system packages updated
Disable unnecessary services
Configure fail2ban for SSH protection
Use strong passwords and SSH keys
Regular security audits

Application Security

Regular SysManage updates
Strong JWT secret keys
Database connection encryption
Proper certificate management
Network segmentation

12. Troubleshooting

Common Issues

Service Won't Start

Check systemd logs: journalctl -u sysmanage
Verify configuration file syntax
Check file permissions
Ensure database is accessible

SSL Certificate Issues

Verify certificate file paths
Check certificate expiration
Ensure proper file permissions
Test certificate with openssl

Database Connection Issues

Test database connectivity
Check PostgreSQL configuration
Verify authentication settings
Monitor connection pool usage

Performance Issues

Monitor system resources
Check database query performance
Review application logs
Optimize configuration settings

Next Steps

After completing your production deployment:

Set up agents: Install and approve agents on your hosts
Learn management tasks: Master day-to-day operations
Administration: Configure users, permissions, and advanced features
Security: Implement security best practices