MCP Server Image Processing — Sharp, ImageContent, and Vision Tools

Sharp setup and initialization

Sharp is a native Node.js module that wraps libvips. It has a binary that must be built for your target platform. On macOS and most Linux distros, npm install sharp downloads a prebuilt binary. In Alpine Linux (common in Docker), add --platform linux/amd64 or install libvips manually: apk add vips-dev.

// lazy-load sharp to avoid penalizing cold starts
let _sharp: typeof import('sharp') | null = null;

async function getSharp() {
  if (!_sharp) {
    // Dynamic import — only executed when an image tool is first called
    const mod = await import('sharp');
    _sharp = mod.default;
    // Validate that Sharp and libvips initialized correctly
    const info = _sharp.versions;
    if (!info?.vips) {
      throw new Error('Sharp loaded but libvips not available');
    }
  }
  return _sharp;
}

// Input size guards — prevent memory bombs from huge images
const MAX_INPUT_BYTES = 20 * 1024 * 1024;  // 20 MB
const MAX_DIMENSION = 8000;                  // 8000 px on either side

function validateImageBuffer(buf: Buffer): void {
  if (buf.length > MAX_INPUT_BYTES) {
    throw new McpError(
      ErrorCode.InvalidParams,
      `Image too large: ${(buf.length / 1e6).toFixed(1)} MB (max 20 MB)`
    );
  }
}

Never accept an image URL from the tool caller and fetch it inside the handler without URL validation. An attacker can pass http://169.254.169.254/latest/meta-data/ and use the image tool as an SSRF vector. Either restrict image input to base64-encoded data in the tool parameters, or validate the URL against an allow-list of trusted domains before fetching.

Resize and format conversion tools

The most common image tool pattern is resize + format conversion. Callers pass base64-encoded image data and get back a processed version. Use z.string() for the base64 input rather than z.string().base64() — Zod's built-in base64 validator rejects valid base64 strings with padding variations in some Node.js versions.

import { z } from 'zod';
import { McpError, ErrorCode } from '@modelcontextprotocol/sdk/types.js';
import { getSharp, validateImageBuffer } from './image.js';

server.tool(
  'resize_image',
  {
    image_base64: z.string().min(1).describe('Base64-encoded image data (no data: URI prefix)'),
    input_mime_type: z.enum(['image/jpeg', 'image/png', 'image/webp', 'image/gif', 'image/tiff']),
    width: z.number().int().min(1).max(MAX_DIMENSION).optional(),
    height: z.number().int().min(1).max(MAX_DIMENSION).optional(),
    fit: z.enum(['cover', 'contain', 'fill', 'inside', 'outside']).default('inside'),
    output_format: z.enum(['jpeg', 'png', 'webp']).default('webp'),
    quality: z.number().int().min(10).max(100).default(85)
  },
  async ({ image_base64, width, height, fit, output_format, quality }) => {
    if (!width && !height) {
      throw new McpError(ErrorCode.InvalidParams, 'At least one of width or height must be provided');
    }

    const inputBuffer = Buffer.from(image_base64, 'base64');
    validateImageBuffer(inputBuffer);

    const sharp = await getSharp();
    let pipeline = sharp(inputBuffer).resize({ width, height, fit, withoutEnlargement: true });

    switch (output_format) {
      case 'jpeg': pipeline = pipeline.jpeg({ quality, mozjpeg: true }); break;
      case 'png':  pipeline = pipeline.png({ compressionLevel: 6 }); break;
      case 'webp': pipeline = pipeline.webp({ quality }); break;
    }

    const outputBuffer = await pipeline.toBuffer();
    const { width: outW, height: outH, format } = await sharp(outputBuffer).metadata();

    return {
      content: [{
        type: 'image',
        data: outputBuffer.toString('base64'),
        mimeType: `image/${output_format}` as 'image/jpeg' | 'image/png' | 'image/webp'
      }, {
        type: 'text',
        text: JSON.stringify({
          width: outW,
          height: outH,
          format,
          bytes: outputBuffer.length,
          compression_ratio: (inputBuffer.length / outputBuffer.length).toFixed(2)
        })
      }]
    };
  }
);

Return both the image content block and a text metadata block. LLM clients that support vision will render the image inline; the metadata block gives the agent programmatic access to the output dimensions and file size without parsing the image.

Crop tool

server.tool(
  'crop_image',
  {
    image_base64: z.string().min(1),
    left: z.number().int().min(0),
    top: z.number().int().min(0),
    width: z.number().int().min(1),
    height: z.number().int().min(1),
    output_format: z.enum(['jpeg', 'png', 'webp']).default('png')
  },
  async ({ image_base64, left, top, width, height, output_format }) => {
    const inputBuffer = Buffer.from(image_base64, 'base64');
    validateImageBuffer(inputBuffer);

    const sharp = await getSharp();
    const meta = await sharp(inputBuffer).metadata();

    // Validate crop region is within bounds
    if (left + width > (meta.width ?? 0) || top + height > (meta.height ?? 0)) {
      throw new McpError(
        ErrorCode.InvalidParams,
        `Crop region [${left},${top},${width},${height}] exceeds image dimensions ${meta.width}×${meta.height}`
      );
    }

    const outputBuffer = await sharp(inputBuffer)
      .extract({ left, top, width, height })
      [output_format]()
      .toBuffer();

    return {
      content: [{
        type: 'image',
        data: outputBuffer.toString('base64'),
        mimeType: `image/${output_format}` as 'image/jpeg' | 'image/png' | 'image/webp'
      }]
    };
  }
);

Exposing images as MCP resources

MCP resources let clients browse image libraries without calling a tool. A common pattern is a content-addressed image store (files named by SHA-256 hash) with URIs like image://{hash}. This lets agents reference images by stable URI and read them via resources/read without needing to call a tool.

import crypto from 'node:crypto';
import fs from 'node:fs/promises';
import path from 'node:path';

const IMAGE_STORE_DIR = process.env.IMAGE_STORE_DIR ?? '/tmp/mcp-images';

// Store an image and return its URI
export async function storeImage(buffer: Buffer, ext: string): Promise<string> {
  await fs.mkdir(IMAGE_STORE_DIR, { recursive: true });
  const hash = crypto.createHash('sha256').update(buffer).digest('hex').slice(0, 16);
  const filename = `${hash}.${ext}`;
  await fs.writeFile(path.join(IMAGE_STORE_DIR, filename), buffer);
  return `image://${filename}`;
}

// List all stored images
server.setRequestHandler(ListResourcesRequestSchema, async () => {
  let files: string[] = [];
  try {
    files = await fs.readdir(IMAGE_STORE_DIR);
  } catch {
    files = [];
  }

  return {
    resources: files
      .filter(f => /\.(png|jpg|jpeg|webp|gif)$/.test(f))
      .map(f => ({
        uri: `image://${f}`,
        name: f,
        mimeType: f.endsWith('.png') ? 'image/png'
          : f.endsWith('.webp') ? 'image/webp'
          : 'image/jpeg'
      }))
  };
});

// Read a specific image
server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
  const { uri } = request.params;
  if (!uri.startsWith('image://')) {
    throw new McpError(ErrorCode.InvalidParams, `Unknown resource URI: ${uri}`);
  }

  const filename = uri.slice('image://'.length);
  // Prevent path traversal
  if (filename.includes('/') || filename.includes('..')) {
    throw new McpError(ErrorCode.InvalidParams, 'Invalid resource URI');
  }

  const filePath = path.join(IMAGE_STORE_DIR, filename);
  const buffer = await fs.readFile(filePath).catch(() => {
    throw new McpError(ErrorCode.InvalidParams, `Resource not found: ${uri}`);
  });

  const ext = path.extname(filename).slice(1).toLowerCase();
  const mimeType = ext === 'png' ? 'image/png' : ext === 'webp' ? 'image/webp' : 'image/jpeg';

  return {
    contents: [{
      uri,
      mimeType,
      blob: buffer.toString('base64')
    }]
  };
});

Health endpoint for image processing monitoring

A naive health check that only pings the process will miss Sharp initialization failures — for example, a missing libvips binary after a botched deployment, or a memory-constrained container where Sharp fails to allocate. Wire a health endpoint that actually runs a Sharp operation.

http.get('/health/image', async (req, reply) => {
  const start = Date.now();
  try {
    const sharp = await getSharp();

    // Probe: create a 10×10 solid color PNG — cheap and tests the full pipeline
    const probeBuffer = await sharp({
      create: {
        width: 10,
        height: 10,
        channels: 3,
        background: { r: 0, g: 128, b: 0 }
      }
    }).png().toBuffer();

    const meta = await sharp(probeBuffer).metadata();

    return reply.send({
      status: 'ok',
      latency_ms: Date.now() - start,
      sharp_version: sharp.versions?.sharp,
      vips_version: sharp.versions?.vips,
      probe_width: meta.width,
      probe_height: meta.height
    });
  } catch (err) {
    return reply.code(503).send({
      status: 'error',
      detail: err instanceof Error ? err.message : String(err),
      latency_ms: Date.now() - start
    });
  }
});

The probe creates a 10×10 green PNG — fast enough to run on every health check cycle without meaningful overhead. If libvips is missing or Sharp fails to allocate, this probe returns 503 and triggers an AliveMCP alert before any real image processing tools are called.

Silent failure modes

Frequently asked questions

Further reading

• MCP Server Playwright — screenshot tools and browser automation
• MCP Server PDF Tools — text extraction and page chunking
• MCP Server Binary Content — handling non-text responses in MCP tools
• MCP Server Health Check — designing endpoints for uptime monitors
• MCP Server Error Handling — mapping Sharp errors to McpError codes