Testing

Comprehensive testing strategy for SysManage server including unit tests, integration tests, and end-to-end testing with Playwright.

Testing Strategy Overview

SysManage employs a multi-layered testing approach to ensure reliability, security, and functionality across all components:

🧪 Unit Tests

Comprehensive unit testing for individual components and functions

Backend: 1,432 Python tests using pytest
Frontend: 63 TypeScript tests using Vitest
Coverage: 100% test coverage for both backend and frontend

🔗 Integration Tests

Testing interactions between different system components

API endpoint testing
Database integration testing
WebSocket communication testing
Authentication and authorization flow testing

🎭 End-to-End (E2E) Tests

Complete user workflow testing using Playwright

Full user journey testing
Cross-browser compatibility
Real-world scenario validation
UI interaction and workflow testing

Running Tests

All Tests

# Run complete test suite
make test

# Run tests with coverage reporting
make test-coverage

Backend Tests (Python/pytest)

# Run all backend tests
python -m pytest tests/ -v

# Run specific test file
python -m pytest tests/test_api_hosts.py -v

# Run with coverage
python -m pytest tests/ --cov=backend --cov-report=html

# Run only unit tests
python -m pytest tests/ -m "not integration"

# Run only integration tests
python -m pytest tests/ -m integration

Frontend Tests (TypeScript/Vitest)

# Run all frontend tests
cd frontend && npm test

# Run tests in watch mode
cd frontend && npm run test:watch

# Run with coverage
cd frontend && npm run test:coverage

# Run specific test file
cd frontend && npm test -- src/__tests__/Pages/Reports.test.tsx

End-to-End Tests (Playwright)

# Install Playwright (if not already installed)
pip install playwright
npx playwright install

# Run E2E tests
npm run test:e2e

# Run E2E tests in specific browser
npx playwright test --project=chromium
npx playwright test --project=firefox
npx playwright test --project=webkit

# Run E2E tests in headed mode (visible browser)
npx playwright test --headed

# Debug E2E tests
npx playwright test --debug

Playwright E2E Testing

Overview

Playwright provides comprehensive end-to-end testing capabilities for SysManage, allowing you to test complete user workflows across different browsers and platforms.

Installation & Setup

# Install Playwright
npm install -D @playwright/test

# Install browser binaries
npx playwright install

# Install system dependencies (Linux)
npx playwright install-deps

💡 Platform Compatibility

Playwright may not be available on all platforms. For example, it's currently not supported on OpenBSD. In such cases, manual testing or alternative tools should be used.

Configuration

Playwright configuration is defined in playwright.config.ts:

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  webServer: {
    command: 'npm run start',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
  },
});

Example E2E Test

// e2e/login-workflow.spec.ts
import { test, expect } from '@playwright/test';

test('user can login and access dashboard', async ({ page }) => {
  // Navigate to login page
  await page.goto('/login');

  // Fill login form
  await page.fill('[data-testid=username]', 'admin@example.com');
  await page.fill('[data-testid=password]', 'password123');

  // Submit form
  await page.click('[data-testid=login-button]');

  // Verify successful login
  await expect(page).toHaveURL('/dashboard');
  await expect(page.locator('h1')).toContainText('Dashboard');
});

test('user can generate reports', async ({ page }) => {
  // Login first
  await page.goto('/login');
  await page.fill('[data-testid=username]', 'admin@example.com');
  await page.fill('[data-testid=password]', 'password123');
  await page.click('[data-testid=login-button]');

  // Navigate to reports
  await page.click('text=Reports');
  await expect(page).toHaveURL('/reports');

  // Generate PDF report
  await page.click('text=Generate PDF');

  // Wait for download
  const downloadPromise = page.waitForEvent('download');
  const download = await downloadPromise;

  // Verify download
  expect(download.suggestedFilename()).toContain('.pdf');
});

Best Practices

Data Attributes: Use data-testid attributes for reliable element selection
Page Object Model: Organize tests using page objects for maintainability
Test Isolation: Ensure tests are independent and can run in any order
Wait Strategies: Use appropriate wait strategies for dynamic content
Screenshots: Enable screenshots on failure for debugging
Cross-browser: Run tests across multiple browsers to ensure compatibility

Test Coverage

Coverage Reports

SysManage maintains comprehensive test coverage across all layers:

Backend Coverage

100%

1,432 Python tests covering API endpoints, business logic, and database operations

Frontend Coverage

100%

63 TypeScript tests covering React components, services, and utilities

Generating Coverage Reports

# Backend coverage (HTML report)
python -m pytest tests/ --cov=backend --cov-report=html
open htmlcov/index.html

# Frontend coverage
cd frontend && npm run test:coverage
open coverage/index.html

# Combined coverage reporting
make test-coverage

Continuous Integration

GitHub Actions

All tests run automatically in the CI/CD pipeline:

name: CI/CD Pipeline
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          cd frontend && npm install

      - name: Run backend tests
        run: python -m pytest tests/ --cov=backend

      - name: Run frontend tests
        run: cd frontend && npm test

      - name: Install Playwright
        run: npx playwright install --with-deps

      - name: Run E2E tests
        run: npm run test:e2e

Testing Different Components

API Endpoints

Test API endpoints using pytest with FastAPI's test client:

def test_get_hosts(authenticated_client):
    response = authenticated_client.get("/api/hosts")
    assert response.status_code == 200
    assert "hosts" in response.json()

def test_create_host(authenticated_client):
    host_data = {
        "fqdn": "test.example.com",
        "active": True
    }
    response = authenticated_client.post("/api/hosts", json=host_data)
    assert response.status_code == 201

React Components

Test React components using Vitest and React Testing Library:

import { render, screen, fireEvent } from '@testing-library/react';
import { vi } from 'vitest';
import Reports from '../Pages/Reports';

test('generates PDF when button clicked', async () => {
  const mockApi = vi.mocked(api);
  mockApi.get.mockResolvedValue({
    data: new Blob(['pdf'], { type: 'application/pdf' })
  });

  render();

  const generateButton = screen.getByText('Generate PDF');
  fireEvent.click(generateButton);

  await waitFor(() => {
    expect(mockApi.get).toHaveBeenCalledWith(
      '/api/reports/generate/hosts',
      expect.objectContaining({ responseType: 'blob' })
    );
  });
});

WebSocket Communication

Test WebSocket connections and message handling:

def test_websocket_connection(client):
    with client.websocket_connect("/ws") as websocket:
        websocket.send_text("ping")
        data = websocket.receive_text()
        assert data == "pong"

Troubleshooting Tests

Common Issues

Playwright Installation Fails

Problem: Browser binaries fail to install

Solution: Install system dependencies: npx playwright install-deps

Tests Fail in CI but Pass Locally

Problem: Environment differences between local and CI

Solution: Check for timing issues, use proper wait strategies, ensure consistent test data

Frontend Tests Can't Find Elements

Problem: React components not rendering properly in tests

Solution: Use await waitFor() for async operations, check component mocking

Database Tests Interfere with Each Other

Problem: Test database state affects other tests

Solution: Use database transactions and rollback, or isolated test databases

Debug Mode

# Debug backend tests
python -m pytest tests/test_specific.py -v -s --pdb

# Debug frontend tests
cd frontend && npm test -- --run=false

# Debug Playwright tests
npx playwright test --debug

Performance Testing

While not part of the standard test suite, consider these tools for performance testing:

Load Testing

Locust: Python-based load testing for API endpoints
Artillery: Modern load testing toolkit
JMeter: Traditional load testing solution

Frontend Performance

Lighthouse: Web performance auditing
WebPageTest: Detailed performance analysis
Playwright Performance: Performance metrics collection during E2E tests