Documentation > Server > Testing

Testing

Comprehensive testing strategy for SysManage server including unit tests, integration tests, and end-to-end testing with Playwright.

Testing Strategy Overview

SysManage employs a multi-layered testing approach to ensure reliability, security, and functionality across all components:

🧪 Unit Tests

Comprehensive unit testing for individual components and functions

  • Backend: 1,432 Python tests using pytest
  • Frontend: 63 TypeScript tests using Vitest
  • Coverage: 100% test coverage for both backend and frontend

🔗 Integration Tests

Testing interactions between different system components

  • API endpoint testing
  • Database integration testing
  • WebSocket communication testing
  • Authentication and authorization flow testing

🎭 End-to-End (E2E) Tests

Complete user workflow testing using Playwright

  • Full user journey testing
  • Cross-browser compatibility
  • Real-world scenario validation
  • UI interaction and workflow testing

🎯 Mock Service Worker (MSW)

API mocking for isolated and reliable frontend testing

  • Complete test isolation from backend dependencies
  • Faster test execution without network requests
  • Consistent and predictable test behavior
  • Realistic API response simulation

Running Tests

All Tests

# Run complete test suite
make test

# Run tests with coverage reporting
make test-coverage

Backend Tests (Python/pytest)

# Run all backend tests
python -m pytest tests/ -v

# Run specific test file
python -m pytest tests/test_api_hosts.py -v

# Run with coverage
python -m pytest tests/ --cov=backend --cov-report=html

# Run only unit tests
python -m pytest tests/ -m "not integration"

# Run only integration tests
python -m pytest tests/ -m integration

Frontend Tests (TypeScript/Vitest with MSW)

Frontend tests automatically use Mock Service Worker for API mocking:

# Run all frontend tests (with MSW automatically active)
cd frontend && npm test

# Run tests in watch mode
cd frontend && npm run test:watch

# Run with coverage
cd frontend && npm run test:coverage

# Run specific test file
cd frontend && npm test -- src/__tests__/Pages/Reports.test.tsx

# View MSW request logs in tests
cd frontend && npm test -- --reporter=verbose

🎯 MSW Behavior

When running frontend tests:

  • MSW automatically intercepts all API requests
  • Request logs appear in console output
  • Each test runs in complete isolation
  • Realistic API responses are provided automatically

End-to-End Tests (Playwright)

# Install Playwright (if not already installed)
pip install playwright
npx playwright install

# Run E2E tests
npm run test:e2e

# Run E2E tests in specific browser
npx playwright test --project=chromium
npx playwright test --project=firefox
npx playwright test --project=webkit

# Run E2E tests in headed mode (visible browser)
npx playwright test --headed

# Debug E2E tests
npx playwright test --debug

Mock Service Worker (MSW) Frontend Testing

Overview

Mock Service Worker (MSW) is a powerful API mocking library that intercepts HTTP requests at the network level, providing realistic API responses for frontend testing. SysManage uses MSW extensively to ensure frontend components can be tested in isolation without requiring a running backend server.

💡 Why MSW?

  • Test Isolation: Frontend tests run independently of backend availability
  • Performance: No network latency, tests execute faster
  • Reliability: Consistent responses eliminate flaky tests
  • Development: Work on frontend features before backend APIs are ready
  • CI/CD: No need to spin up backend services in test environments

Installation & Configuration

MSW is already installed and configured in the SysManage frontend. The setup includes:

# MSW installation
npm install --save-dev msw@^2.11.3

# MSW configuration in package.json
{
  "msw": {
    "workerDirectory": ["public"]
  }
}

Implementation Architecture

Request Handlers

MSW handlers are defined in /src/mocks/handlers.ts and provide comprehensive API coverage:

// /src/mocks/handlers.ts
import { http, HttpResponse } from 'msw';

export const handlers = [
  // Catch all /api/ requests and handle them dynamically
  http.get('http://localhost:8080/api/*', ({ request }) => {
    const url = new globalThis.URL(request.url);
    const path = url.pathname;

    // Host data endpoint
    if (path === '/api/host/1' || path === '/api/hosts/1') {
      return HttpResponse.json({
        id: 1,
        fqdn: 'test-host.example.com',
        ipv4: '192.168.1.100',
        active: true,
        status: 'up',
        platform: 'Linux',
        // ... complete host data
      });
    }

    // Package search with query parameter handling
    if (path === '/api/packages/search') {
      const query = url.searchParams.get('query');
      if (!query || query.length < 2) {
        return HttpResponse.json([]);
      }

      const results = availablePackages.filter(pkg =>
        pkg.name.toLowerCase().includes(query.toLowerCase())
      );
      return HttpResponse.json(results);
    }

    // Default response for unmatched endpoints
    return HttpResponse.json([]);
  }),

  // POST endpoints for package management
  http.post('http://localhost:8080/api/packages/install/*', async ({ request }) => {
    const body = await request.json();
    return HttpResponse.json({
      success: true,
      message: 'Package installation has been queued',
      installation_ids: body.package_names.map(() => `uuid-${Math.random()}`)
    });
  }),
];

Test Server Setup

The MSW server is configured for Node.js test environments in /src/mocks/node.ts:

// /src/mocks/node.ts
import { setupServer } from 'msw/node';
import { handlers } from './handlers';

// Setup MSW server for Node.js environment (tests)
export const server = setupServer(...handlers);

Test Integration

MSW is automatically configured for all tests in /src/setupTests.ts with proper lifecycle management:

// /src/setupTests.ts
import { beforeAll, afterEach, afterAll } from 'vitest';
import { server } from './mocks/node';

// Start MSW server before all tests
beforeAll(() => {
  server.listen({
    onUnhandledRequest: 'warn', // Warn about unhandled requests
  });

  if (process.env.CI === 'true') {
    console.log('🚀 MSW server started for CI environment');
  }
});

// Reset handlers after each test for isolation
afterEach(() => {
  server.resetHandlers();
});

// Clean up after all tests
afterAll(() => {
  server.close();
});

Key Features

Dynamic Path Handling

MSW uses broad pattern matching with dynamic path resolution:

  • Wildcard patterns catch all /api/* requests
  • URL parsing extracts specific endpoint paths
  • Graceful fallback for unmatched endpoints

Realistic Response Simulation

MSW provides comprehensive mock data that mirrors real API responses:

  • Complete host information with hardware details
  • User authentication and profile data
  • Software package data with search functionality
  • Package installation/uninstallation operations

CI/CD Integration

MSW includes special handling for continuous integration environments:

// Enhanced logging for CI environments
const logPrefix = process.env.CI === 'true' ? '🔄 MSW-CI:' : 'MSW:';
console.log(`${logPrefix} Handling GET ${path}`);

// CI-specific startup and shutdown messages
if (process.env.CI === 'true') {
  console.log('🚀 MSW server started for CI environment');
}

Usage in Tests

Automatic API Mocking

Most tests automatically benefit from MSW without additional configuration:

// Test automatically uses MSW for API calls
test('renders host information', async () => {
  render();

  // MSW automatically returns mock host data
  await waitFor(() => {
    expect(screen.getByText('test-host.example.com')).toBeInTheDocument();
    expect(screen.getByText('192.168.1.100')).toBeInTheDocument();
  });
});

Custom Handler Overrides

Tests can override specific handlers for custom scenarios:

import { server } from '../../mocks/node';
import { http, HttpResponse } from 'msw';

test('handles API error gracefully', async () => {
  // Override handler for this test
  server.use(
    http.get('http://localhost:8080/api/host/1', () => {
      return HttpResponse.json(
        { error: 'Host not found' },
        { status: 404 }
      );
    })
  );

  render();

  await waitFor(() => {
    expect(screen.getByText('Error loading host')).toBeInTheDocument();
  });
});

React 19 Compatibility

SysManage includes special compatibility fixes for React 19 in the test environment:

// React 19 compatibility fixes in setupTests.ts
globalThis.IS_REACT_ACT_ENVIRONMENT = true;

// Scheduler polyfills for React 19
Object.defineProperty(globalThis, 'Scheduler', {
  value: {
    unstable_scheduleCallback: (callback) => setTimeout(callback, 0),
    unstable_cancelCallback: () => {},
    unstable_shouldYield: () => false,
    // ... additional React 19 compatibility
  }
});

Best Practices

  • Test Isolation: MSW automatically resets handlers between tests
  • Realistic Data: Use mock data that closely matches production API responses
  • API Coverage: Ensure all API endpoints used by components are mocked
  • Error Scenarios: Test both success and error responses
  • Request Logging: Use MSW logging to debug unexpected API calls

🔍 Debugging MSW

If tests fail due to API issues, check:

  • Console output for MSW request logs
  • Warnings about unhandled requests
  • Handler patterns match actual API calls
  • MSW server is properly started in test setup

Playwright E2E Testing

Overview

Playwright provides comprehensive end-to-end testing capabilities for SysManage, allowing you to test complete user workflows across different browsers and platforms.

Installation & Setup

# Install Playwright
npm install -D @playwright/test

# Install browser binaries
npx playwright install

# Install system dependencies (Linux)
npx playwright install-deps

💡 Platform Compatibility

Playwright may not be available on all platforms. For example, it's currently not supported on OpenBSD. In such cases, manual testing or alternative tools should be used.

Configuration

Playwright configuration is defined in playwright.config.ts:

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  webServer: {
    command: 'npm run start',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
  },
});

Example E2E Test

// e2e/login-workflow.spec.ts
import { test, expect } from '@playwright/test';

test('user can login and access dashboard', async ({ page }) => {
  // Navigate to login page
  await page.goto('/login');

  // Fill login form
  await page.fill('[data-testid=username]', 'admin@example.com');
  await page.fill('[data-testid=password]', 'password123');

  // Submit form
  await page.click('[data-testid=login-button]');

  // Verify successful login
  await expect(page).toHaveURL('/dashboard');
  await expect(page.locator('h1')).toContainText('Dashboard');
});

test('user can generate reports', async ({ page }) => {
  // Login first
  await page.goto('/login');
  await page.fill('[data-testid=username]', 'admin@example.com');
  await page.fill('[data-testid=password]', 'password123');
  await page.click('[data-testid=login-button]');

  // Navigate to reports
  await page.click('text=Reports');
  await expect(page).toHaveURL('/reports');

  // Generate PDF report
  await page.click('text=Generate PDF');

  // Wait for download
  const downloadPromise = page.waitForEvent('download');
  const download = await downloadPromise;

  // Verify download
  expect(download.suggestedFilename()).toContain('.pdf');
});

Best Practices

  • Data Attributes: Use data-testid attributes for reliable element selection
  • Page Object Model: Organize tests using page objects for maintainability
  • Test Isolation: Ensure tests are independent and can run in any order
  • Wait Strategies: Use appropriate wait strategies for dynamic content
  • Screenshots: Enable screenshots on failure for debugging
  • Cross-browser: Run tests across multiple browsers to ensure compatibility

Test Coverage

Coverage Reports

SysManage maintains comprehensive test coverage across all layers:

Backend Coverage

100%

1,432 Python tests covering API endpoints, business logic, and database operations

Frontend Coverage

100%

63 TypeScript tests covering React components, services, and utilities

Generating Coverage Reports

# Backend coverage (HTML report)
python -m pytest tests/ --cov=backend --cov-report=html
open htmlcov/index.html

# Frontend coverage
cd frontend && npm run test:coverage
open coverage/index.html

# Combined coverage reporting
make test-coverage

Continuous Integration

GitHub Actions

All tests run automatically in the CI/CD pipeline:

name: CI/CD Pipeline
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          cd frontend && npm install

      - name: Run backend tests
        run: python -m pytest tests/ --cov=backend

      - name: Run frontend tests
        run: cd frontend && npm test

      - name: Install Playwright
        run: npx playwright install --with-deps

      - name: Run E2E tests
        run: npm run test:e2e

Testing Different Components

API Endpoints

Test API endpoints using pytest with FastAPI's test client:

def test_get_hosts(authenticated_client):
    response = authenticated_client.get("/api/hosts")
    assert response.status_code == 200
    assert "hosts" in response.json()

def test_create_host(authenticated_client):
    host_data = {
        "fqdn": "test.example.com",
        "active": True
    }
    response = authenticated_client.post("/api/hosts", json=host_data)
    assert response.status_code == 201

React Components

Test React components using Vitest and React Testing Library:

import { render, screen, fireEvent } from '@testing-library/react';
import { vi } from 'vitest';
import Reports from '../Pages/Reports';

test('generates PDF when button clicked', async () => {
  const mockApi = vi.mocked(api);
  mockApi.get.mockResolvedValue({
    data: new Blob(['pdf'], { type: 'application/pdf' })
  });

  render();

  const generateButton = screen.getByText('Generate PDF');
  fireEvent.click(generateButton);

  await waitFor(() => {
    expect(mockApi.get).toHaveBeenCalledWith(
      '/api/reports/generate/hosts',
      expect.objectContaining({ responseType: 'blob' })
    );
  });
});

WebSocket Communication

Test WebSocket connections and message handling:

def test_websocket_connection(client):
    with client.websocket_connect("/ws") as websocket:
        websocket.send_text("ping")
        data = websocket.receive_text()
        assert data == "pong"

Troubleshooting Tests

Common Issues

Playwright Installation Fails

Problem: Browser binaries fail to install

Solution: Install system dependencies: npx playwright install-deps

Tests Fail in CI but Pass Locally

Problem: Environment differences between local and CI

Solution: Check for timing issues, use proper wait strategies, ensure consistent test data

Frontend Tests Can't Find Elements

Problem: React components not rendering properly in tests

Solution: Use await waitFor() for async operations, check component mocking

Database Tests Interfere with Each Other

Problem: Test database state affects other tests

Solution: Use database transactions and rollback, or isolated test databases

Debug Mode

# Debug backend tests
python -m pytest tests/test_specific.py -v -s --pdb

# Debug frontend tests
cd frontend && npm test -- --run=false

# Debug Playwright tests
npx playwright test --debug

Performance Testing

Performance testing is now fully integrated into SysManage's test suite and runs automatically as part of our CI/CD pipeline.

🚀 Integrated Performance Testing

✅ Current Implementation

SysManage includes comprehensive performance testing that runs on every commit:

  • Artillery: Backend API load testing with realistic scenarios
  • Playwright Performance: Frontend Core Web Vitals monitoring
  • Multi-Platform: Tested across Linux, macOS, and Windows
  • CI/CD Integration: Automated performance regression detection

Running Performance Tests

# Run comprehensive performance test suite
make test-performance

# Run individual performance tests
make test-artillery      # Backend API load testing
make test-ui-performance # Frontend performance testing

# Run specific Playwright performance tests
pytest tests/ui/test_performance_playwright.py -v

Performance Test Coverage

  • Backend Load Testing: API endpoints, authentication flows, WebSocket connections
  • Frontend Performance: Page load times, Core Web Vitals, memory usage
  • Network Analysis: Resource loading optimization and request monitoring
  • Cross-Browser Testing: Performance validation across Chrome, Firefox, and Safari

Performance Budgets

📊 Enterprise Performance Standards

  • API Response Time: 95th percentile < 500ms, 99th percentile < 1000ms
  • First Contentful Paint: < 2000ms
  • DOM Content Loaded: < 1500ms
  • Error Rate: < 1%
  • Minimum Throughput: 8 requests per second

📈 Performance Documentation

For detailed information about our performance testing framework, see the Performance Testing Architecture documentation.

Cross-Platform UI Testing Framework

Revolutionary triple testing framework featuring Playwright (primary), Selenium (fallback), and comprehensive CI/CD integration for cross-platform UI testing excellence.

Framework Overview

SysManage implements a revolutionary triple testing framework that automatically adapts to platform capabilities, ensuring comprehensive UI test coverage across all supported environments.

🏗️ Triple Testing Architecture

  • Playwright (Primary): Linux, macOS, Windows - Full cross-browser support with Chrome, Firefox, and WebKit
  • Selenium (Fallback): OpenBSD, FreeBSD - Reliable WebDriver automation where Playwright is unavailable
  • CI/CD Integration: GitHub Actions with fail-fast logic and comprehensive error reporting

Key Benefits

  • Cross-Platform Excellence: Seamless testing across 5 operating systems
  • Cross-Browser Compatibility: Chrome, Firefox, Safari/WebKit support
  • Intelligent Selection: Framework automatically chooses optimal testing tool per platform
  • Production Parity: Real database integration with proper user lifecycle management

Playwright Testing (Primary Framework)

Modern browser automation framework providing cross-browser testing with enterprise-grade reliability.

🎭 Playwright Features

  • Cross-Browser Testing: Chromium, Firefox, WebKit (Safari) support
  • Async Native: Built for modern JavaScript with native async/await support
  • Auto-Wait: Intelligent waiting for elements without explicit sleeps
  • Network Interception: Mock and monitor network requests
  • Automatic Screenshots: Failure debugging with visual evidence

Browser Support Matrix

Chromium: Full support on all platforms (Linux, macOS, Windows)
Firefox: Full support on all platforms
WebKit (Safari): macOS only - automatically excluded on other platforms 
# Run Playwright tests
make test-ui-playwright

# Run cross-browser tests
make test-ui-cross-browser

# Run with specific browser
npx playwright test --project=chromium
npx playwright test --project=firefox
npx playwright test --project=webkit  # macOS only

Selenium Testing (Fallback Framework)

Reliable WebDriver automation framework serving as intelligent fallback for platforms where Playwright is unavailable.

🔄 When Selenium is Used

  • OpenBSD: Primary testing framework (Playwright unavailable)
  • FreeBSD: Primary testing framework (Playwright unavailable)
  • Legacy Systems: Broader compatibility with older browsers
# Run Selenium tests (automatic fallback on OpenBSD/FreeBSD)
make test-ui-selenium

# Force Selenium usage
TEST_FRAMEWORK=selenium make test-ui

Database Integration

Sophisticated database integration ensuring tests run against real PostgreSQL with proper user lifecycle management.

🔐 Test User Injection

  • Argon2 Password Hashing: Production-grade password security with configurable salt
  • YAML Configuration: Salt and database credentials from sysmanage.yaml
  • Unique Test Users: Each test run creates isolated test users
  • Proper Cleanup: Test users automatically removed after test completion
Database Configuration

UI tests use the same PostgreSQL database as the main application, ensuring perfect production parity. Test users are created with proper Argon2 hashing and automatically cleaned up.

Cross-Platform Testing Strategy

Comprehensive testing strategy ensuring consistent behavior across all supported operating systems.

📊 Platform Testing Matrix

Linux: Playwright with full browser support (Chrome, Firefox, WebKit excluded)
macOS: Playwright with complete browser support (Chrome, Firefox, WebKit)
Windows: Playwright with full browser support (Chrome, Firefox, WebKit excluded)
OpenBSD: Selenium with Chrome/Firefox support
FreeBSD: Selenium with Chrome/Firefox support
NetBSD: Selenium with Chrome-only support (Firefox excluded due to WebDriver compatibility issues)

CI/CD Integration

Enterprise-grade CI/CD integration with GitHub Actions providing automated testing on every commit and pull request.

🚀 GitHub Actions Workflow

  • Fail-Fast Logic: Stop execution immediately on first test failure
  • Parallel Execution: Backend, Frontend, and UI tests run concurrently
  • Comprehensive Logging: Detailed output for debugging failed tests
  • Artifact Collection: Screenshots and logs preserved for analysis
# GitHub Actions workflow automatically runs:
1. Backend Tests (Python/pytest)
2. Frontend Tests (TypeScript/Vitest)
3. UI Tests (Playwright/Selenium)
4. Generate test reports and badges

Makefile Integration

Seamless integration with existing build system providing simple commands for complex testing operations.

🔧 Available Commands

# Run all UI tests with intelligent framework selection
make test-ui

# Force Playwright execution (if available)
make test-playwright

# Force Selenium execution
make test-selenium

# Run complete test suite (Backend + Frontend + UI)
make test

# Install required browser dependencies
make install-browsers

Debugging and Troubleshooting

Comprehensive debugging capabilities ensuring rapid issue resolution and test reliability.

📸 Screenshot Capture

  • Automatic Capture: Screenshots taken on test failure
  • Timestamped Files: Screenshots saved with timestamp for analysis
  • Storage Location: Screenshots saved to /tmp/claude/ for examination

Common Issues and Solutions

Browser Not Found: Install missing browsers with make install-browsers
Timeout Errors: Increase wait times for slow-loading elements
Database Connection: Verify PostgreSQL is running and accessible

Best Practices

Industry best practices for maintainable and reliable UI testing.

🎯 Test Design Principles

  • Page Object Pattern: Encapsulate page interactions in reusable objects
  • Data-Driven Tests: Parameterized tests for comprehensive scenario coverage
  • Test Independence: Each test runs independently without dependencies
  • Idempotent Tests: Tests can run multiple times with consistent results