Documentation > Security > Best Practices

Security Best Practices

Comprehensive security recommendations, deployment guidelines, compliance frameworks, and incident response procedures for SysManage environments.

Security Best Practices Overview

This guide provides enterprise-grade security recommendations for deploying and operating SysManage in production environments. Following these practices ensures robust security posture, regulatory compliance, and operational resilience.

Security Framework Pillars

🔐 Identity & Access

Strong authentication, authorization, and identity management practices

🛡️ Defense in Depth

Multiple security layers providing comprehensive protection

📊 Monitoring & Response

Continuous monitoring, threat detection, and incident response

🔄 Compliance & Governance

Regulatory compliance and security governance frameworks

Production Deployment Security

Pre-Deployment Security Checklist

Infrastructure Security

  • ✅ Deploy on hardened operating systems (CIS benchmarks)
  • ✅ Configure host-based firewalls with minimal open ports
  • ✅ Enable SELinux/AppArmor mandatory access controls
  • ✅ Implement fail2ban for intrusion prevention
  • ✅ Configure log forwarding to centralized SIEM
  • ✅ Enable audit logging at OS and application levels
  • ✅ Implement time synchronization (NTP)
  • ✅ Configure automated security updates

Application Security

  • ✅ Change all default passwords and secrets
  • ✅ Generate strong JWT secret keys
  • ✅ Configure TLS certificates from trusted CA
  • ✅ Enable database encryption at rest
  • ✅ Configure secure session management
  • ✅ Enable comprehensive audit logging
  • ✅ Set up health monitoring and alerting
  • ✅ Configure backup and disaster recovery

Network Security

  • ✅ Implement network segmentation
  • ✅ Configure VPN access for administrators
  • ✅ Set up intrusion detection/prevention systems
  • ✅ Enable DDoS protection
  • ✅ Configure load balancer security features
  • ✅ Implement API rate limiting and throttling
  • ✅ Set up WAF (Web Application Firewall)
  • ✅ Configure DNS security (DNSSEC)

Secure Configuration Templates

Production Environment Variables

# Production security configuration
ENVIRONMENT=production
DEBUG=false
LOG_LEVEL=INFO

# Database security
DATABASE_SSL_MODE=require
DATABASE_SSL_CERT=/path/to/client-cert.pem
DATABASE_SSL_KEY=/path/to/client-key.pem
DATABASE_SSL_CA=/path/to/ca-cert.pem
DATABASE_CONNECTION_POOL_SIZE=20
DATABASE_CONNECTION_TIMEOUT=30

# JWT security
JWT_SECRET_KEY=
JWT_ALGORITHM=HS256
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=15
JWT_REFRESH_TOKEN_EXPIRE_DAYS=7
JWT_BLACKLIST_ENABLED=true

# TLS configuration
TLS_VERSION_MIN=1.2
TLS_CERT_PATH=/etc/ssl/certs/sysmanage.crt
TLS_KEY_PATH=/etc/ssl/private/sysmanage.key
TLS_CA_PATH=/etc/ssl/certs/ca-bundle.crt

# Security headers
SECURITY_HEADERS_ENABLED=true
CSRF_PROTECTION_ENABLED=true
CORS_ORIGINS=["https://sysmanage.company.com"]
RATE_LIMITING_ENABLED=true
RATE_LIMIT_PER_MINUTE=100

# Audit and monitoring
AUDIT_LOGGING_ENABLED=true
AUDIT_LOG_LEVEL=INFO
SECURITY_EVENTS_ENABLED=true
PROMETHEUS_METRICS_ENABLED=true
HEALTH_CHECK_ENABLED=true

Nginx Security Configuration

# /etc/nginx/sites-available/sysmanage
server {
    listen 443 ssl http2;
    server_name sysmanage.company.com;

    # TLS configuration
    ssl_certificate /etc/ssl/certs/sysmanage.crt;
    ssl_certificate_key /etc/ssl/private/sysmanage.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512;
    ssl_prefer_server_ciphers off;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';" always;

    # Rate limiting
    limit_req zone=sysmanage burst=20 nodelay;
    limit_conn addr 10;

    # Proxy configuration
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }

    # WebSocket configuration
    location /ws {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

System Hardening

Operating System Hardening

🔒 Access Controls

  • Disable unnecessary user accounts
  • Configure sudo with minimal privileges
  • Implement SSH key-based authentication
  • Disable password authentication for SSH
  • Configure account lockout policies
  • Enable audit logging for privileged operations

🛡️ Network Hardening

  • Disable unnecessary network services
  • Configure iptables/firewalld rules
  • Enable TCP wrappers (/etc/hosts.allow)
  • Configure fail2ban for intrusion prevention
  • Disable IPv6 if not required
  • Enable SYN flood protection

📁 File System Security

  • Set proper file permissions (644/755)
  • Configure separate partitions for logs
  • Enable file integrity monitoring
  • Configure disk encryption (LUKS)
  • Implement regular security scans
  • Set up automated malware scanning

🔍 Monitoring & Logging

  • Configure centralized log collection
  • Enable process accounting (psacct)
  • Set up file access monitoring (auditd)
  • Configure system resource monitoring
  • Implement log rotation and retention
  • Set up security event alerting

CIS Benchmark Implementation

# Install and run CIS benchmark scanner
# For Ubuntu/Debian
wget https://workbench.cisecurity.org/files/cis-cat-lite.zip
unzip cis-cat-lite.zip
cd cis-cat-lite
sudo ./cis-cat-lite.sh -b benchmarks/CIS_Ubuntu_Linux_20.04_LTS_Benchmark_v1.1.0-xccdf.xml

# For RHEL/CentOS
sudo yum install -y scap-security-guide
sudo oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_cis \
    --results scan-results.xml \
    /usr/share/xml/scap/ssg/content/ssg-centos8-ds.xml

# Automated hardening with Ansible
ansible-playbook -i inventory hardening-playbook.yml \
    --extra-vars "compliance_profile=cis"

Compliance Frameworks

Supported Compliance Standards

📋 SOC 2 Type II

Security, Availability, Processing Integrity

  • Access control implementation
  • System monitoring and logging
  • Change management procedures
  • Incident response protocols
  • Data protection measures

🔐 ISO 27001

Information Security Management

  • Risk assessment and treatment
  • Security policy documentation
  • Asset management controls
  • Supplier relationship security
  • Business continuity planning

🏛️ FedRAMP

Federal Risk Authorization

  • NIST 800-53 control implementation
  • Continuous monitoring requirements
  • Security assessment procedures
  • Configuration management
  • Incident response capabilities

💳 PCI DSS

Payment Card Industry

  • Network security requirements
  • Data protection standards
  • Vulnerability management
  • Access control implementation
  • Regular security testing

Compliance Implementation Guide

SOC 2 Implementation

  1. Scope Definition: Identify systems and processes in scope
  2. Control Design: Implement required security controls
  3. Documentation: Create policies and procedures
  4. Testing: Perform control effectiveness testing
  5. Monitoring: Establish continuous monitoring processes
  6. Audit: Engage qualified auditor for assessment

Control Implementation Example

# SOC 2 Control: CC6.1 - Logical and Physical Access Controls
# Implementation in SysManage

# 1. Access control policy configuration
ACCESS_CONTROL_POLICY = {
    "minimum_password_length": 12,
    "require_mfa": True,
    "session_timeout_minutes": 30,
    "max_failed_attempts": 5,
    "account_lockout_duration": 30,
    "privilege_escalation_approval": True
}

# 2. Role-based access control matrix
RBAC_MATRIX = {
    "admin": [
        "system:configure", "users:manage", "security:audit",
        "hosts:manage", "certificates:manage"
    ],
    "operator": [
        "hosts:manage", "packages:manage", "monitoring:view"
    ],
    "viewer": [
        "hosts:view", "packages:view", "monitoring:view"
    ],
    "auditor": [
        "audit:view", "logs:view", "reports:generate"
    ]
}

# 3. Access logging configuration
AUDIT_EVENTS = [
    "user_login", "user_logout", "permission_granted",
    "permission_denied", "role_assigned", "role_removed",
    "system_configuration_change", "security_event"
]

Audit Procedures

Audit Framework

SysManage implements comprehensive audit capabilities for compliance and security monitoring.

Audit Event Categories

🔐 Authentication Events

  • User login/logout activities
  • Failed authentication attempts
  • Account lockouts and unlocks
  • Password changes and resets
  • MFA enrollment and usage

⚙️ Administrative Actions

  • User account management
  • Role and permission changes
  • System configuration modifications
  • Security setting updates
  • Certificate management operations

🖥️ System Operations

  • Host approval and rejection
  • Package installation and updates
  • Task execution and results
  • File transfers and deployments
  • System monitoring access

🚨 Security Events

  • Suspicious activity detection
  • Privilege escalation attempts
  • Unauthorized access attempts
  • Certificate validation failures
  • Data access violations

Audit Log Format

# Standard audit log entry format (JSON)
{
  "timestamp": "2024-01-15T14:30:00.000Z",
  "event_id": "evt_1234567890",
  "event_type": "authentication",
  "event_category": "login_success",
  "severity": "info",
  "source": {
    "service": "sysmanage-server",
    "version": "1.0.0",
    "host": "sysmanage-prod-01"
  },
  "actor": {
    "user_id": "user_123",
    "username": "jdoe",
    "session_id": "sess_abc123",
    "ip_address": "192.168.1.100",
    "user_agent": "Mozilla/5.0..."
  },
  "target": {
    "resource_type": "system",
    "resource_id": "sysmanage-server",
    "action": "login"
  },
  "details": {
    "authentication_method": "username_password",
    "mfa_used": true,
    "login_location": "New York, US",
    "previous_login": "2024-01-15T08:00:00.000Z"
  },
  "outcome": {
    "result": "success",
    "reason": null,
    "risk_score": 0.2
  }
}

Audit Reporting

Automated Reporting

  • Daily Security Summary: Authentication events, failed attempts, security alerts
  • Weekly Access Report: User activities, permission changes, role assignments
  • Monthly Compliance Report: Control effectiveness, policy violations, risk assessments
  • Quarterly Audit Report: Comprehensive security posture review

Custom Report Generation

# Generate audit reports using SysManage API
curl -X POST "https://sysmanage.company.com/api/reports/audit" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "report_type": "security_events",
    "date_range": {
      "start": "2024-01-01T00:00:00Z",
      "end": "2024-01-31T23:59:59Z"
    },
    "filters": {
      "event_types": ["authentication", "authorization", "admin_action"],
      "severity": ["medium", "high", "critical"],
      "users": ["admin", "operator"]
    },
    "format": "pdf",
    "delivery": {
      "method": "email",
      "recipients": ["security@company.com", "compliance@company.com"]
    }
  }'

Incident Response

Incident Response Framework

SysManage implements a structured incident response process following NIST guidelines.

Incident Response Phases

1. 📋 Preparation

  • Incident response plan development
  • Team roles and responsibilities
  • Communication procedures
  • Tool and resource preparation
  • Training and awareness programs

2. 🔍 Detection & Analysis

  • Automated threat detection
  • Security event correlation
  • Incident classification
  • Impact assessment
  • Evidence collection

3. 🚨 Containment & Eradication

  • Immediate containment actions
  • System isolation procedures
  • Threat neutralization
  • Vulnerability remediation
  • System hardening

4. 🔄 Recovery & Lessons Learned

  • System restoration procedures
  • Monitoring for reoccurrence
  • Post-incident analysis
  • Process improvements
  • Documentation updates

Automated Incident Detection

Security Event Rules

# Incident detection rules configuration
INCIDENT_RULES = {
    "brute_force_attack": {
        "condition": "failed_logins >= 10 within 5 minutes",
        "severity": "high",
        "actions": ["block_ip", "notify_admin", "create_incident"]
    },
    "privilege_escalation": {
        "condition": "sudo_usage by non_admin_user",
        "severity": "critical",
        "actions": ["log_event", "notify_security_team", "require_justification"]
    },
    "unusual_access_pattern": {
        "condition": "login from new_location AND new_device",
        "severity": "medium",
        "actions": ["require_mfa", "notify_user", "monitor_session"]
    },
    "certificate_validation_failure": {
        "condition": "mtls_validation_failure >= 5 within 1 minute",
        "severity": "high",
        "actions": ["block_agent", "investigate_source", "notify_admin"]
    }
}

# Automated response actions
def handle_security_incident(rule_name: str, event_data: dict):
    incident = create_incident_record(rule_name, event_data)

    # Execute automated responses
    for action in INCIDENT_RULES[rule_name]["actions"]:
        execute_response_action(action, event_data)

    # Notify response team
    notify_incident_response_team(incident)

    return incident.id

Incident Response Playbooks

Compromised User Account Playbook

  1. Immediate Actions:
    • Disable compromised user account
    • Revoke all active JWT tokens
    • Reset account password
    • Block source IP addresses
  2. Investigation:
    • Review audit logs for account activity
    • Identify accessed resources and data
    • Check for lateral movement attempts
    • Analyze authentication patterns
  3. Recovery:
    • Re-enable account with new credentials
    • Require MFA re-enrollment
    • Monitor account for suspicious activity
    • Update security awareness training

Agent Communication Compromise

  1. Immediate Actions:
    • Revoke suspected agent certificates
    • Block agent IP addresses
    • Isolate affected network segments
    • Enable enhanced monitoring
  2. Investigation:
    • Analyze mTLS certificate validation logs
    • Review agent communication patterns
    • Check for unauthorized commands
    • Verify certificate chain integrity
  3. Recovery:
    • Generate new agent certificates
    • Update certificate authority if needed
    • Strengthen certificate validation
    • Implement additional monitoring

Continuous Security Monitoring

Monitoring Architecture

Implement comprehensive monitoring for real-time threat detection and response.

Monitoring Layers

🖥️ Infrastructure Monitoring

  • System resource utilization
  • Network traffic analysis
  • Service availability monitoring
  • Performance metrics tracking

📱 Application Monitoring

  • API endpoint performance
  • Database query monitoring
  • Error rate and patterns
  • User session tracking

🔒 Security Monitoring

  • Authentication event monitoring
  • Authorization failure tracking
  • Suspicious activity detection
  • Certificate status monitoring

📊 Business Monitoring

  • Host management activities
  • Package deployment metrics
  • User adoption patterns
  • Compliance status tracking

Monitoring Tool Integration

# Prometheus monitoring configuration
# /etc/prometheus/prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:
  - "sysmanage_alerts.yml"

scrape_configs:
  - job_name: 'sysmanage-server'
    static_configs:
      - targets: ['localhost:8000']
    metrics_path: '/metrics'
    scrape_interval: 5s

  - job_name: 'sysmanage-database'
    static_configs:
      - targets: ['localhost:9187']
    scrape_interval: 10s

alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - alertmanager:9093

# Alert rules for security events
groups:
  - name: sysmanage_security
    rules:
      - alert: HighFailedLogins
        expr: rate(sysmanage_failed_logins_total[5m]) > 0.1
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "High rate of failed login attempts"

      - alert: SuspiciousActivity
        expr: sysmanage_security_events_total{severity="high"} > 0
        for: 0m
        labels:
          severity: critical
        annotations:
          summary: "Critical security event detected"

Data Protection & Privacy

Data Classification

🔴 Highly Sensitive

  • User passwords and authentication tokens
  • Private keys and certificates
  • Personal identifiable information (PII)
  • Security configuration details

🟡 Sensitive

  • System configuration data
  • Host inventory information
  • Package installation details
  • User activity logs

🟢 Internal

  • System performance metrics
  • Non-sensitive log entries
  • Public configuration templates
  • Documentation and guides

Data Protection Measures

Encryption Implementation

# Data encryption configuration
ENCRYPTION_CONFIG = {
    "database": {
        "encryption_at_rest": True,
        "algorithm": "AES-256",
        "key_rotation_days": 90
    },
    "backups": {
        "encryption_enabled": True,
        "algorithm": "AES-256-GCM",
        "compression": True
    },
    "logs": {
        "pii_redaction": True,
        "encryption_in_transit": True,
        "retention_days": 365
    },
    "certificates": {
        "storage_encryption": True,
        "access_control": "strict",
        "audit_logging": True
    }
}

# PII redaction for logs
import re

def redact_sensitive_data(log_message: str) -> str:
    # Redact email addresses
    log_message = re.sub(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
                        '[EMAIL_REDACTED]', log_message)

    # Redact IP addresses
    log_message = re.sub(r'\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b',
                        '[IP_REDACTED]', log_message)

    # Redact JWT tokens
    log_message = re.sub(r'eyJ[A-Za-z0-9-_]+\.eyJ[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+',
                        '[TOKEN_REDACTED]', log_message)

    return log_message

Disaster Recovery & Business Continuity

Backup Strategy

3-2-1 Backup Rule Implementation

  • 3 Copies: Production data + 2 backup copies
  • 2 Media Types: Local storage + cloud storage
  • 1 Offsite: Geographically separated location

Backup Components

# Automated backup script
#!/bin/bash
# /usr/local/bin/sysmanage-backup.sh

BACKUP_DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/var/backups/sysmanage"
S3_BUCKET="s3://company-backups/sysmanage"

# Database backup
pg_dump -h localhost -U sysmanage_user sysmanage_db | \
    gzip > "$BACKUP_DIR/db_backup_$BACKUP_DATE.sql.gz"

# Configuration backup
tar -czf "$BACKUP_DIR/config_backup_$BACKUP_DATE.tar.gz" \
    /etc/sysmanage/ \
    /etc/ssl/sysmanage/ \
    /var/lib/sysmanage/

# Certificate backup (encrypted)
gpg --cipher-algo AES256 --compress-algo 1 --s2k-mode 3 \
    --s2k-digest-algo SHA512 --s2k-count 65536 --symmetric \
    --output "$BACKUP_DIR/certs_backup_$BACKUP_DATE.tar.gz.gpg" \
    /var/lib/sysmanage/certs/

# Upload to cloud storage
aws s3 sync "$BACKUP_DIR" "$S3_BUCKET" --delete

# Clean old backups (keep 30 days local, 365 days cloud)
find "$BACKUP_DIR" -type f -mtime +30 -delete
aws s3 ls "$S3_BUCKET" --recursive | \
    awk '$1 < "'$(date -d '365 days ago' '+%Y-%m-%d')'" {print $4}' | \
    xargs -I {} aws s3 rm "$S3_BUCKET/{}"

Recovery Procedures

Recovery Time Objectives (RTO)

  • Critical Systems: 1 hour
  • Production Environment: 4 hours
  • Full Service Restoration: 24 hours

Recovery Point Objectives (RPO)

  • Database: 15 minutes (continuous replication)
  • Configuration: 1 hour (hourly backups)
  • Logs: 1 hour (real-time shipping)

Security Performance Optimization

Balancing Security and Performance

⚡ Authentication Optimization

  • JWT token caching for validation
  • Efficient RBAC permission lookups
  • Connection pooling for external auth
  • Optimized password hashing (Argon2)

🔒 TLS Performance

  • TLS session resumption
  • HTTP/2 with server push
  • Certificate caching
  • Cipher suite optimization

📊 Monitoring Efficiency

  • Selective audit logging
  • Asynchronous security checks
  • Batched metric collection
  • Intelligent alerting rules

Security Training & Awareness

Training Program

👤 User Security Training

  • Password security best practices
  • Multi-factor authentication setup
  • Phishing and social engineering awareness
  • Incident reporting procedures

👨‍💼 Administrator Training

  • System hardening procedures
  • Certificate management
  • Incident response protocols
  • Compliance requirements

🔧 Developer Security Training

  • Secure coding practices
  • API security implementation
  • Vulnerability assessment
  • Security testing methodologies

Next Steps

After implementing security best practices:

  1. Authentication Setup: Configure robust authentication systems
  2. mTLS Implementation: Set up certificate-based agent authentication
  3. Network Security: Implement network-level protections
  4. Security Scanning: Deploy automated security scanning