MCP, From “What Is This?” to a Button in My Running App

Every month I pick one thing I’ve only been reading about and make it real in my own code. In September it was the Model Context Protocol (MCP). I’d seen it everywhere: Claude Desktop, IDE plugins, agent frameworks. On paper I understood it as “USB-C for AI tools”, but as a Java developer I kept asking the obvious question:

I already know how to expose a capability. It’s called a @RestController. What does MCP give me that REST doesn’t?

So I gave myself a concrete goal. In runs-app, my running tracker (Spring Boot 4 + React 19), you can already ask an AI to analyze your recent runs. I wanted a “Generate Diagram” button that turns that analysis into a dashboard image. The image would be drawn by a separate service that speaks MCP. I also wanted the same capability available to a chat model, through LiteLLM, with a local Ollama model as a fallback.

That took three weeks of evenings (Sep 7–26), plus one more evening to wire up LiteLLM. This post covers what I built, where REST and MCP each fit, and how LiteLLM fits in.

The 30-second version of MCP

MCP is JSON-RPC with a client side and a server side:

• An MCP server exposes tools (functions with a name, a description and a JSON Schema for the inputs). It can also expose resources and prompts.
• An MCP client connects, calls initialize, asks tools/list, and then calls tools/call with arguments.
• The transport is either stdio (a local process) or Streamable HTTP: one HTTP endpoint (/mcp) that you POST JSON-RPC to, with Server-Sent Events when it needs to stream.

Notice the last point. MCP over Streamable HTTP is still HTTP. It isn’t a competitor to HTTP. It’s a specific contract layered on top of it. Keeping that in mind made most of my confusion go away.

2. runs-ai-analyzer: the MCP client

spring-ai-starter-mcp-client plus YAML gives you an injectable McpSyncClient:

spring:
  ai:
    mcp:
      client:
        enabled: ${DIAGRAM_MAKER_MCP_ENABLED:false}
        type: SYNC
        request-timeout: 60s
        streamable-http:
          connections:
            diagram-maker:
              url: ${diagram-maker.base-url}
              endpoint: /mcp

3. runs-app: the button

RagProxyResource got two pass-through routes (generate and download). The AI Insights page (ai-insights.tsx) got a Generate Diagram / Regenerate Diagram button with loading and error states and an inline preview with a download link.

REST vs MCP: a Java developer’s view, hop by hop through runs-app

This is the section I wish I’d read first. Rather than comparing the two in the abstract, here’s every hop in the feature above, which protocol it uses, and why.

The same idea in both styles

Server side: one is a resource URL for a known caller, the other is a described function for a caller that discovers it.

// REST: diagram-maker's DiagramArtifactController
@RestController
@RequestMapping("/api/diagrams")
class DiagramArtifactController {
    @GetMapping("/{fileName}")
    ResponseEntity<Resource> download(@PathVariable String fileName) { ... }
}

// MCP: diagram-maker's DiagramMakerMcpTools
@McpTool(name = "generate_analysis_diagram", description = "Generates a dashboard image from ...")
RunDiagramResult generateAnalysisDiagram(@McpToolParam(...) UUID documentId, ...) { ... }

Client side: one is a URL and a verb, the other is a tool name and an arguments map.

// REST: runs-ai-analyzer's DiagramArtifactClient
restClient.get()
        .uri(uriBuilder -> uriBuilder.path("/api/diagrams/{fileName}").build(fileName))
        .retrieve()
        .toEntity(byte[].class);

// MCP: runs-ai-analyzer's DiagramMakerMcpClient
client.callTool(new McpSchema.CallToolRequest("generate_analysis_diagram", arguments, null));

Both endpoints sit on top of the same @Service classes (DashboardImageService, DiagramStorageService). That’s the most important design rule I took away: keep business logic in a service, and treat @RestController and @McpTool as two thin adapters over it.

Every hop, and why

What actually differs

Three things I had wrong

1. “MCP replaces REST.” No. Four of the six hops in this feature are REST or messaging, and that’s correct.
2. “MCP needs an LLM.” No. Hop 3 is a plain Java service calling a tool with no model involved. MCP is just a typed, discoverable RPC contract, and the LLM is one possible caller.
3. “An LLM can just call my REST API.” Only if someone writes a tool definition that describes it. MCP is that tool definition, standardized and served by the service itself.

My rule of thumb now: if the caller is code that already knows what it wants, use REST. If the caller is a model that has to figure out what it can do, use MCP. If you have both kinds of caller, put the logic in a @Service and add a thin adapter for each.

LiteLLM: now wired into docker-compose

LiteLLM is an OpenAI-compatible proxy. Clients talk to one URL (:4000), and LiteLLM routes the request to whichever model is configured, handling fallbacks and keys in one place. It can also act as an MCP gateway: you register MCP servers in its config, and it re-exposes their tools behind the same key.

When I first drafted this post, LiteLLM was only a config file and a plan. Here’s what’s actually running now.

litellm_config.yaml

model_list:
  - model_name: default
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  - model_name: fallback
    litellm_params:
      model: ollama_chat/qwen2.5:3b
      api_base: os.environ/OLLAMA_API_BASE

router_settings:
  fallbacks: [{ default: [fallback] }]

mcp_servers:
  diagram_maker:
    url: "http://diagram-maker:8091/mcp"
    transport: "http"
    description: "Generates run-analysis dashboard images for runs-app from runs-ai-analyzer data"

docker-compose.yml (next to diagram-maker)

  litellm:
    image: ghcr.io/berriai/litellm:v1.103.1     # pinned, never :latest
    ports: ["4000:4000"]
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    volumes:
      - ./litellm_config.yaml:/app/config.yaml:ro
    environment:
      - LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY:?Set LITELLM_MASTER_KEY in .env}
      - OLLAMA_API_BASE=${OLLAMA_API_BASE:?Set OLLAMA_API_BASE in .env}
      - OPENAI_API_KEY=${OPENAI_API_KEY:-}
    depends_on:
      diagram-maker:
        condition: service_healthy

Secrets come only from a git-ignored .env, and a committed .env.example documents them. The :? syntax makes Compose refuse to start if a required value is missing, instead of quietly falling back to a default.

Proof it works

I ran this locally with no OpenAI key on purpose:

1. GET /v1/models → ["default", "fallback"]. Without the master key → 401.
2. MCP tools/list against http://localhost:4000/mcp/ → both diagram-maker tools, renamed diagram_maker-generate_analysis_diagram and diagram_maker-generate_run_diagram.
3. A chat completion on model default came back from ollama_chat/qwen2.5:3b, with headers x-litellm-model-group: fallback and x-litellm-attempted-fallbacks: 1. The fallback worked.
4. The full chat path: I asked “Generate a SUMMARY PNG dashboard for analysis document 81f8e7ec-…” with tools: [{type: "mcp", server_url: "litellm_proxy/mcp/diagram_maker", require_approval: "never"}]. The 3-billion-parameter local model picked the right tool and passed the right UUID. diagram-maker fetched the real analysis of my last 8 runs from runs-ai-analyzer and saved a 73,850-byte PNG (the dashboard image above). That’s the same diagram the runs-app button produces, this time requested in plain English.

The layering lesson

My first architecture diagram had “LiteLLM routes to diagram-maker” as if diagram-maker were a model. It isn’t. LiteLLM routes prompts to models and proxies tool calls to MCP servers. Those are two separate jobs that happen to share a port. Once I separated them, the button path became simpler. runs-ai-analyzer already knows the documentId, so it calls the tool directly, and no model or LiteLLM is needed in the middle.

The same “primary + local fallback” idea also exists in Java. runs-ai-analyzer’s analysis path uses Spring AI, so I built the pattern natively: AiProviderChain tries Anthropic → ACG-sandbox Ollama → local Ollama, each tier behind its own Resilience4j circuit breaker. A 401/403 opens that provider’s breaker for 30 minutes, and an AiProvidersHealthIndicator reports every tier in /actuator/health. Same idea, two implementations: LiteLLM for anything that speaks the OpenAI API, and in-process Java where Spring AI owns the call.

Gotchas (so they don’t bite you)

1. The Streamable HTTP handshake is strict. Send Accept: application/json, text/event-stream, call initialize, and reuse the Mcp-Session-Id it returns on every call after that.
2. The annotations are @McpTool, not @Tool. Spring AI MCP Server 2.0.1 uses org.springframework.ai.mcp.annotation.McpTool/@McpToolParam, not the generic @Tool that most tutorials show.
3. Spring Boot 4.1 moved things around. Jackson now lives under tools.jackson.*, UriComponentsBuilder.fromHttpUrl is gone (use fromUriString), and there’s no RestTemplateBuilder on the classpath.
4. My tests weren’t running. Surefire had <skipTests>${skipITs}</skipTests>, so ./mvnw test silently skipped unit tests.
5. Return metadata from tools, serve bytes over REST. It keeps MCP responses small, and the PNG gets proper HTTP headers.
6. Tool parameters are untrusted input, possibly written by an LLM. DiagramStorageService rejects paths that escape the output root (../etc/passwd), and the download side checks toRealPath() so a symlink can’t get out either.
7. Fail fast on dead fallbacks. Spring AI’s default retry template backs off for minutes. For the sandbox tier I set retries to 0 and the connect timeout to 3s.
8. Check your fallback model actually exists. My original config pointed at llama3.2:3b, but the VM’s Ollama only has qwen2.5:3b. The fallback would have failed exactly when it was needed. Run GET /api/tags before trusting it.
9. Use ollama_chat/, not ollama/, in LiteLLM. The chat route uses Ollama’s /api/chat, which supports native tool calling.
10. Inside Compose, localhost is the container itself. LiteLLM reaches diagram-maker as http://diagram-maker:8091/mcp, using the service name.
11. Pin the LiteLLM image. Versions 1.82.7/1.82.8 on PyPI were compromised in March 2026 (a credential-stealing .pth payload). The official Docker image wasn’t affected because it pins its dependencies. I pinned v1.103.1 and checked that the image contains no litellm_init.pth. Also, the image has no curl/wget, so the Compose healthcheck uses its own Python.
12. Gateways rename your tools. Through LiteLLM, generate_analysis_diagram becomes diagram_maker-generate_analysis_diagram. Write descriptions that still make sense to a model that only sees the prefixed name.

What’s next

• A chat box in runs-app’s AI Insights page that talks to LiteLLM, so “draw my last week” works next to the button. Same tool, different caller, which is the reason to use MCP in the first place.
• Move the LiteLLM service into the VM’s main stack next to runs-app and runs-ai-analyzer.
• Set @McpTool hints (the tool currently advertises itself as destructive and non-idempotent by default), and put generated diagrams on a Docker volume so they survive container restarts.
• Fix the key-metrics chart: calories (~5,900) dwarf distance (~28 mi) on one axis, so each metric needs its own scale.

Takeaway: for a Java developer, MCP clicked once I stopped treating it as “an AI thing” and started treating it as a second adapter over the same @Service, written for callers that discover instead of callers that already know. REST still does most of the work in runs-app. MCP makes the same capability usable by a model. LiteLLM decides which model is on the other end, and falls back to a 3B model on my home server when the primary is unavailable.