CumulusBlog

10 articlesNewest first

Apr2026

  1. Engineering

    Inside Ion: Model Execution and GPU Kernels

    How Ion connects model-specific execution, custom GPU kernels, and memory management in dedicated inference deployments, including NVIDIA Grace Hopper.

Mar2026

  1. Engineering

    From If-Else to Declared Routing

    Most production AI systems route between providers with hand-written if-else statements that never get tested. Declared routing rules fix the failover problem before it pages you at 3 AM.

  2. Research

    SFT and Online RL for Visual Generation: How We Built CoSprite's Training Pipeline

    How Cumulus built a production pipeline for consistent AI-generated game previews using best-of-N sampling, deterministic rendering, pairwise judging, supervised fine-tuning, and online reinforcement learning with GRPO.

  1. Two dark forms crossing, one edge catching the light
    Engineering

    Day-0 Support for the Qwen3.5 Family

    What we found inside Qwen3.5's hybrid Mamba-Transformer weights and what it took to make the gated delta rule fast on GH200 — from matrix-valued recurrences to mixed batch state corruption.

  1. Engineering

    What is an Inference Platform?

    Inference is not just a model call. It is routing, caching, observability, evaluation, fine-tuning, and the runtime that executes the call. A complete guide.

Feb2026

  1. Engineering

    Drop-In OpenAI: Migrating in One Line

    How to move an existing OpenAI-SDK application to Cumulus without changing your code — and what you get from the routing, caching, and observability layers once you do.

  2. Product

    5 VLMs, 1 GPU: Beating Together AI on Price and Throughput

    Cheap GPU inference for AI models: we ran 5 VLMs on one GPU and matched Together AI's throughput at a fraction of the cost. Serverless GPU vs dedicated GPU economics.

  1. Close-up of a dense compute board, fans and heatsinks lit from above
    Engineering

    Ionattention: Grace Hopper–Native Inference

    How Cumulus built the fastest GPU inference runtime for NVIDIA GH200 — 7,167 tok/s on a single chip. Coherent CUDA graphs, eager KV writeback, and phantom-tile attention scheduling.

  1. Engineering

    Continuous Evaluation: Beyond Judge Consensus

    LLM-as-judge with majority vote is the default — and it is wrong often enough to lose trust. A better approach combines synthetic data, deterministic heuristics, judges, and shadow evaluation on live traffic.

  2. Company

    Why We Built a Cheaper, Faster GPU Cloud for AI Model Hosting

    Cumulus Labs is building the cheapest serverless GPU cloud for AI model hosting. Here's why dedicated GPU instances waste money and how pay-per-second GPU inference changes the economics.