Smoke Testing MCP Servers — post-deploy validation, deployment gates, and CI/CD integration

What smoke testing is and is not

Smoke testing sits at the fast end of a testing pyramid. It is not exhaustive. A passing smoke test tells you the deployment produced a working server; it does not tell you every tool works correctly, that edge cases are handled, or that performance is within spec. Those claims require broader integration tests and load tests.

The failure classes smoke tests catch that other tests miss:

• Wrong binary deployed. Unit tests pass because they test the code you wrote; the smoke test runs against what was actually deployed. A CI pipeline that accidentally packages the wrong build artifact will pass all pre-deploy tests and fail the smoke test.
• Missing environment variables. A database URL or API key that exists in CI but not in the production environment causes the server to fail at startup, not at code level. The smoke test catches this within seconds of deployment.
• Migration not run. If a deploy depends on a schema migration that was not applied before deployment, the server starts but tools that query new columns fail. The smoke test's representative tool call surfaces this.
• Port binding conflict. A previous deployment did not terminate cleanly and the port is still bound. The new process fails to start. The smoke test catches the connection failure.

The three-check MCP smoke test

Check 1: Protocol handshake

Connect to the server and complete the MCP initialize handshake. This check verifies the server is running, the port is bound, TLS terminates correctly, and the process has loaded enough to respond to the protocol. Budget: 3 seconds.

// smoke-test.ts — run this immediately after each deployment
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js';

async function smokeTest(serverUrl: string, config: SmokeTestConfig) {
  const results: SmokeCheckResult[] = [];
  let client: Client | null = null;

  // Check 1: Protocol handshake (budget: 3s)
  try {
    const transport = new SSEClientTransport(new URL(serverUrl));
    client = new Client({ name: 'smoke-test', version: '1.0' }, {});

    const start = Date.now();
    await withTimeout(client.connect(transport), 3000, 'handshake_timeout');
    results.push({ check: 'protocol_handshake', ok: true, ms: Date.now() - start });
  } catch (err) {
    results.push({ check: 'protocol_handshake', ok: false, error: err.message });
    return { ok: false, results };  // No point continuing if handshake failed
  }

  return { results, client };
}

Check 2: Tool manifest verification

Send tools/list and compare the returned tool names against the expected manifest. The manifest should be committed in your repository alongside the server code — a source-controlled list of which tools the production server is supposed to expose. Budget: 2 seconds.

// Expected tool manifest — commit this next to your server code
// smoke-test-manifest.json
{
  "required_tools": [
    "search_documents",
    "get_user_profile",
    "create_ticket",
    "list_recent_activity"
  ]
}

// Check 2: Tool manifest verification (budget: 2s)
const toolsStart = Date.now();
const toolsResponse = await withTimeout(client.listTools(), 2000, 'tools_list_timeout');
const presentTools = new Set(toolsResponse.tools.map(t => t.name));

const missingTools = config.requiredTools.filter(name => !presentTools.has(name));
results.push({
  check: 'tool_manifest',
  ok: missingTools.length === 0,
  ms: Date.now() - toolsStart,
  missing: missingTools,
  present: toolsResponse.tools.length,
});

Check 3: Representative tool call

Call one tool with a known-safe, idempotent input and verify the response is non-error. Choose a read-only tool (never a write tool for smoke testing — you do not want to create records in production every time you deploy). Budget: 10 seconds.

// Check 3: Representative tool call (budget: 10s)
// Use a read-only tool with deterministic behavior
const callStart = Date.now();
const toolResult = await withTimeout(
  client.callTool({
    name: config.smokeToolName,     // e.g., 'search_documents'
    arguments: config.smokeToolArgs // e.g., { query: 'smoke test sentinel' }
  }),
  10000,
  'tool_call_timeout'
);

// Verify response is non-error and has expected structure
const responseText = toolResult.content?.[0]?.text;
let parsed: unknown;
try {
  parsed = JSON.parse(responseText);
} catch {
  results.push({ check: 'tool_call', ok: false, error: 'response_not_json' });
  return { ok: false, results };
}

// Tool-specific validation — define in your smoke test config
const valid = config.validateResponse(parsed);
results.push({
  check: 'tool_call',
  ok: valid.ok,
  ms: Date.now() - callStart,
  error: valid.ok ? undefined : valid.reason,
});

Wiring smoke tests into CI/CD

Smoke tests belong in the deployment pipeline as a gate between "deploy to environment" and "promote to next stage." The deployment workflow for a canary rollout looks like:

# .github/workflows/deploy.yml
jobs:
  deploy-canary:
    steps:
      - name: Deploy to canary (10% traffic)
        run: |
          # Deploy new version to canary slot
          kubectl set image deployment/mcp-server server=$NEW_IMAGE --record
          kubectl rollout status deployment/mcp-server --timeout=120s

      - name: Wait for canary to stabilize
        run: sleep 30  # Allow health checks and process initialization

      - name: Run smoke tests against canary
        run: npx tsx smoke-test.ts --url $CANARY_MCP_URL --manifest smoke-test-manifest.json
        timeout-minutes: 2

      - name: Promote to full rollout
        if: success()
        run: kubectl patch deployment/mcp-server -p '{"spec":{"strategy":{"rollingUpdate":{"maxUnavailable":0,"maxSurge":1}}}}'

      - name: Roll back on smoke test failure
        if: failure()
        run: |
          kubectl rollout undo deployment/mcp-server
          echo "::error::Smoke tests failed — deployment rolled back"

Key decisions in the CI/CD integration:

• Wait for stabilization before testing. The 30-second sleep is not arbitrary — it gives the server process time to complete initialization (load tools, establish database connections, warm up caches) before the smoke test probes it. A smoke test that runs 2 seconds after deployment will fail on legitimate slow-startup servers.
• Use timeout-minutes: 2. If the smoke test itself hangs (server accepts connection but never responds), the pipeline should not wait forever. Cap the total smoke test duration at 2 minutes.
• Auto-rollback on failure. The value of a smoke test gate is the automatic rollback. Without it, the smoke test is just a log entry — a human still has to notice it failed and manually intervene. Wire the rollback into the pipeline so broken deployments undo themselves.

Smoke test configuration patterns

As your MCP server evolves, maintain the smoke test manifest and tool call configuration as first-class artifacts in your repository. A smoke test that checks for tools that no longer exist will always fail, destroying the gate's utility.

// smoke-test-config.ts — versioned alongside server code
export const smokeTestConfig = {
  // Tools that MUST be present after every deployment
  requiredTools: [
    'search_documents',
    'get_user_profile',
    'create_ticket',
  ],

  // The tool to call during check 3 — must be read-only and idempotent
  smokeToolName: 'search_documents',
  smokeToolArgs: { query: '__smoke_test_sentinel__', top_k: 1 },

  // How to validate the tool call response
  validateResponse(parsed: unknown): { ok: boolean; reason?: string } {
    if (!parsed || typeof parsed !== 'object') {
      return { ok: false, reason: 'non_object_response' };
    }
    // Smoke test doesn't require results — just that the call succeeded
    // without an error and returned parseable JSON with expected structure
    const p = parsed as Record;
    if ('error' in p) {
      return { ok: false, reason: `tool_returned_error: ${p.error}` };
    }
    return { ok: true };
  },

  // Time limits for each check (milliseconds)
  timeouts: {
    handshake: 3000,
    toolsList: 2000,
    toolCall: 10000,
  },
};

Update the manifest whenever you add or remove tools as part of a deployment. The manifest diff in the pull request is a useful communication artifact: reviewers can see at a glance that this deployment adds create_ticket and removes legacy_search. It makes tool surface area changes visible in code review rather than invisible in production.

Smoke testing in different deployment scenarios

Blue-green deployment

Run smoke tests against the green (new) environment before switching the load balancer. The green environment has no production traffic, so a failing smoke test has zero user impact. Promote to green only after smoke tests pass.

Rolling deployment

Run smoke tests against the first pod/instance after it passes Kubernetes readiness checks. If smoke tests fail, pause the rollout (kubectl rollout pause) and investigate. The remaining pods continue serving traffic on the old version while you diagnose.

Serverless (Vercel, Cloudflare Workers)

Smoke test the preview deployment URL before promoting to production. Serverless platforms generate unique deployment URLs per release; wire the smoke test against that URL as a pre-promotion gate in your deployment configuration.

Container restart (PM2, Docker)

After a process restart triggered by a deployment, run the smoke test script against the local server before considering the deployment complete. PM2's post_update hook is the right place:

// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'mcp-server',
    script: 'dist/server.js',
    post_update: ['npm run smoke-test'],  // Runs after pm2 reload
    wait_ready: true,
    listen_timeout: 10000,
  }]
};

AliveMCP as continuous smoke testing between deployments

A CI/CD smoke test runs at deploy time. AliveMCP's continuous protocol probe runs every 60 seconds — permanently. The two complement each other: the CI smoke test prevents you from deploying a broken server; AliveMCP catches when a previously healthy server degrades between deployments.

The failure classes each catches:

Neither can replace the other. Use both.

Frequently asked questions

Further reading

• Synthetic Monitoring for MCP Servers — continuous external probing
• MCP Server Regression Testing — catching performance and schema regressions
• MCP Server Canary Deployment — staged rollout patterns
• MCP Server CI/CD — automated build, test, and deploy pipelines
• MCP Server Health Checks — liveness, readiness, and custom probes
• MCP Server Zero-Downtime Deployment — rolling updates and traffic draining