Self-host MiniMax-M3.1-Flash

MiniMax-M3.1-Flash is a native multimodal Mixture-of-Experts model for coding, agentic workflows, computer use, and cybersecurity, with approximately 428B total parameters, approximately 23B activated parameters per token, and a 1M-token model context limit. It keeps the MoE architecture of MiniMax-M3 and changes three things: sparse attention in every layer, quantization-aware training, and a bundled DSpark speculative-decoding draft. This page covers server-class self-hosting with SGLang, not consumer-device local inference.

Deployment status: Experimental. As reviewed on November 9, 2026, the SGLang Cookbook serves MiniMax-M3.1-Flash from the development image lmsysorg/sglang:dev-minimax-m31-flash. SGLang v0.5.23 includes the model support PR, but the Cookbook has not published a recipe for a tagged release image. Pin both the model revision and image digest below, validate them on your workload, and do not treat this recipe as a production SLA.

Status and reference baseline

Item	Reference value
Deployment status	Experimental
Last documentation review	November 9, 2026
Reference hardware	4 × NVIDIA B300, single node, TP 4, SGLang Balanced recipe
Weights	`MiniMaxAI/MiniMax-M3.1-Flash`: MXFP8 dense layers, NVFP4 routed experts, and the DSpark draft in `dspark/`
Model revision	`ce7f4df87d2e05262f5775f373dac4772f134041`; weight, config, and tokenizer files are unchanged since the initial upload `54e4e69dfc45`
Weight repository size	Approximately 252 GB at the pinned revision, including the 2.3 GB DSpark draft
SGLang image	`lmsysorg/sglang@sha256:614b1509bcb729c653528d448bff9850f8977fcad9d12ba56bd6dbe3d7985e87` (`dev-minimax-m31-flash`, SGLang commit `e7399a46f4`, CUDA 13.4.1, `amd64` and `arm64`)
Runtime	Linux and Docker with the NVIDIA Container Toolkit; Python and CUDA are supplied by the pinned image
Validation source	SGLang MiniMax-M3.1 Cookbook

The official sources do not publish a minimum host-memory requirement, peak GPU-memory measurement, minimum NVIDIA driver version, or complete production disk margin for this recipe. Confirm that the host driver supports the CUDA 13.4 runtime in the pinned image, and provision more disk than the weight repository size before deployment.

The complete weights must be loaded or sharded. “23B activated parameters” does not mean that the deployment only needs memory for 23B parameters. The Hugging Face file summary lists about 220B parameters because it counts each byte of packed FP4 expert weights, which holds two values, as one parameter.

Hardware configurations

The following table reports SGLang’s current support matrix. “Validated” means validated by the SGLang Cookbook, not independently benchmarked by MiniMax in this document.

Hardware	Weights	GPUs / parallelism	SGLang recipes	SGLang status	Weight storage	Peak GPU memory	Host memory
NVIDIA B300	NVFP4	4 / TP 4	Balanced (reference baseline), Low-Latency	Validated	Approximately 252 GB	Not published	Not published
NVIDIA B300	NVFP4	8 / TP 8, attention DP 2, EP 8	High-Throughput	Validated	Approximately 252 GB	Not published	Not published
NVIDIA GB300	NVFP4	4 / TP 4; High-Throughput adds EP 4	Balanced, Low-Latency, High-Throughput	Validated	Approximately 252 GB	Not published	Not published
NVIDIA B200	NVFP4	4 / TP 4; High-Throughput uses 8 / TP 8, attention DP 2, EP 8	Balanced, Low-Latency, High-Throughput	Final verification in progress	Approximately 252 GB	Not published	Not published
NVIDIA GB200	NVFP4	4 / TP 4; High-Throughput adds EP 4	Balanced, Low-Latency, High-Throughput	Final verification in progress	Approximately 252 GB	Not published	Not published
NVIDIA H200	NVFP4, experts served through Marlin W4A16	8 / TP 8; High-Throughput adds attention DP 2	Balanced, High-Throughput; no speculative decoding; context capped at 140,000 tokens	Validated	Approximately 252 GB	Not published	Not published
AMD MI355X	NVFP4, experts requantized to MXFP4 at load	4 / TP 4	Low-Latency, High-Throughput	Validated	Approximately 252 GB	Not published	High-Throughput: 180 GB HiCache host pool per rank
AMD MI355X	Quark MXFP4	4 / TP 4	Low-Latency	Validated	Approximately 240 GB	Not published	Not published
AMD MI350X	NVFP4	4 / TP 4	Low-Latency, High-Throughput	Validated	Approximately 252 GB	Not published	High-Throughput: about 1.4 TB per TP 4 server at the default HiCache ratio
AMD MI350X	Quark MXFP4	4 / TP 4	Low-Latency	Same command as MI355X; not marked verified	Approximately 240 GB	Not published	Not published

SGLang lists 288 GB per GPU for B300, 192 GB for B200, and 186 GB for GB200. At --mem-fraction-static 0.8 on 4 × B300, the KV pool holds about 10.4M tokens; the Low-Latency MonoKernel keeps its own copy of the routed experts (54.1 GiB per rank), which reduces that to about 6.5M. On ROCm, the NVFP4 checkpoint runs at TP 4 or lower and the server refuses a larger TP; on an 8-GPU AMD node, start two TP 4 servers on different ports behind a router. Use the SGLang configuration generator for the exact GB-series, B200, H200, or AMD command. Do not substitute those commands into this B300 baseline without revalidation.

Quickstart: 4 × B300

Before starting, verify that the host has enough free disk for the approximately 252 GB checkpoint, the container image, and caches. The commands below use GPUs 0 to 3; change the device list if those GPUs are in use.

1. Pull the pinned runtime

docker pull lmsysorg/sglang@sha256:614b1509bcb729c653528d448bff9850f8977fcad9d12ba56bd6dbe3d7985e87

2. Verify GPU access

The output must list four B300 GPUs:

docker run --rm --gpus '"device=0,1,2,3"' \
  lmsysorg/sglang@sha256:614b1509bcb729c653528d448bff9850f8977fcad9d12ba56bd6dbe3d7985e87 \
  nvidia-smi

3. Start the server

docker run --rm --name minimax-m3-1-flash \
  --gpus '"device=0,1,2,3"' \
  --shm-size 32g \
  --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -e FMHA_SM100_ALLOW_JIT=1 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v ~/.cache/minfer:/root/.cache/minfer \
  lmsysorg/sglang@sha256:614b1509bcb729c653528d448bff9850f8977fcad9d12ba56bd6dbe3d7985e87 \
  sglang serve \
  --trust-remote-code \
  --model-path MiniMaxAI/MiniMax-M3.1-Flash \
  --revision ce7f4df87d2e05262f5775f373dac4772f134041 \
  --reasoning-parser minimax-m3 \
  --tool-call-parser minimax-m3 \
  --tp 4 \
  --moe-runner-backend flashinfer_trtllm_routed \
  --speculative-algorithm DSPARK \
  --speculative-use-block-verification \
  --cuda-graph-backend-prefill breakable \
  --chunked-prefill-size 32768 \
  --max-running-requests 128 \
  --cuda-graph-max-bs-decode 128 \
  --mem-fraction-static 0.8 \
  --tokenizer-worker-num 8 \
  --host 0.0.0.0 \
  --port 30000

The process listens on all interfaces inside the container, but -p 127.0.0.1:30000:30000 publishes it only on the host loopback interface. The first start downloads approximately 252 GB of weights and compiles the MSA sparse-attention and FlashInfer kernels, so it may take a long time. FMHA_SM100_ALLOW_JIT=1 allows the MSA kernels to compile; they are cached under /root/.cache/minfer, which the command maps to the host so that later starts reuse them. --speculative-algorithm DSPARK loads the draft from the checkpoint’s dspark/ subfolder at the same revision, so no draft path is needed. The log reports DSpark draft found in the target checkpoint's dspark/ subfolder.

4. Check readiness

Run this command in another terminal after the server log reports that it is ready:

curl --fail --silent --show-error http://127.0.0.1:30000/health

A successful check exits with status 0. If it fails, keep the server process running and inspect its logs before sending inference requests. Then confirm that the NVFP4 KV-cache decode path uses the fast kernel:

docker logs minimax-m3-1-flash 2>&1 | grep 'nvfp4_to_fp8 backend='

The expected output contains nvfp4_to_fp8 backend=qmul4. Any other backend means that the server found a CUDA toolkit older than 13.4 and fell back to a slower path without failing.

5. Verify text generation

curl http://127.0.0.1:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [
      {"role": "user", "content": "Explain sparse attention in one sentence."}
    ],
    "temperature": 1.0,
    "top_p": 0.95,
    "max_tokens": 4096
  }'

A successful response contains a non-empty choices[0].message.content. When the model emits a reasoning trace, SGLang returns it separately in choices[0].message.reasoning_content.

6. Verify thinking control

curl http://127.0.0.1:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [
      {"role": "user", "content": "How many prime numbers are there below 50?"}
    ],
    "thinking": {"type": "disabled"},
    "temperature": 1.0,
    "top_p": 0.95,
    "max_tokens": 4096
  }'

The response contains a non-empty choices[0].message.content and an empty or null choices[0].message.reasoning_content.

7. Verify image input

curl http://127.0.0.1:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image in one sentence."},
        {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/sgl-project/sglang/2511743bd784e69e5a81ca3d926a000711dae4ab/examples/assets/example_image.png"}}
      ]
    }],
    "temperature": 1.0,
    "top_p": 0.95,
    "max_tokens": 4096
  }'

This example uses an image maintained by the SGLang project. In production, accept only trusted or allowlisted remote URLs, or send a Base64 data URI after validating the file at your gateway.

Capability boundary

Capability	Model-native capability	This B300 SGLang recipe	Notes
Text input and output	Supported	Validated by SGLang	OpenAI-compatible chat completions
Reasoning	Supported	Shown in the SGLang Cookbook with example output	Requires `--reasoning-parser minimax-m3`; streaming returns the trace in `delta.reasoning_content`
Thinking mode and reasoning effort	`thinking.type`: `enabled`, `disabled`, or `adaptive`; `reasoning_effort`: `low`, `medium`, `high`, `xhigh`, or `max`	Shown in the SGLang Cookbook with example output	Send both as top-level request fields. Omitting `thinking` means `adaptive`; `reasoning_effort` `none` or `minimal` disables thinking. The MiniMax API always enables thinking for this model; disabling it applies only to self-hosted serving
Tool calls	Supported	Shown in the SGLang Cookbook with example output	Requires `--tool-call-parser minimax-m3`
Image input	Supported	Shown in the SGLang Cookbook with example output	OpenAI `image_url` content type
Video input	Supported by the model	Not validated in this SGLang recipe	The Cookbook publishes no video example; do not advertise video serving from this recipe without your own validation
DSpark speculative decoding	Draft head shipped in `dspark/`	Enabled in this recipe	Supported by the SGLang recipes on Blackwell and AMD MI350X / MI355X; the H200 recipes run without speculative decoding
Context length	Up to 1M tokens	This recipe does not cap the context length; SGLang publishes Low-Latency decode measurements at 1K, 256K, and 1M tokens on 4 × B300	The H200 recipes cap the context at 140,000 tokens. Validate your own prompt lengths before setting a production limit

The model card recommends temperature=1.0 and top_p=0.95.

Published performance data

SGLang reports the following measurements. These are upstream reference results, not MiniMax performance guarantees.

Hardware	Recipe	Workload	Result
4 × B300	Low-Latency (MSA-V1 MonoKernel), DSpark acceptance length fixed at 5	Batch-size-1 decode at 1K / 256K / 1M context	1,028 / 919 / 723 tokens/s, against 479 / 465 / 408 tokens/s on the standard path
4 × MI355X	High-Throughput with HiCache	64 concurrent long multi-turn sessions	90,168 tokens/s per GPU; P90 TTFT 1.21 s

The MonoKernel serves batch-size-1 steps only; larger batches take the standard path, so its throughput under concurrency is lower than the Balanced recipe’s. SGLang has not published latency or throughput for the Balanced reference recipe, peak GPU memory, host memory, or cold-start time. For agentic long-context deployments, the Cookbook links complete launch commands to their SemiAnalysis InferenceX runs. Benchmark your prompt lengths, output lengths, multimodal ratio, and concurrency before capacity planning.

Weights, cache, and offline deployment

The runtime downloads Hugging Face files to /root/.cache/huggingface in the container, mapped to ~/.cache/huggingface on the host by the Quickstart command. To prepare an offline copy, install the Hugging Face CLI on a connected machine and download the pinned revision:

hf download MiniMaxAI/MiniMax-M3.1-Flash \
  --revision ce7f4df87d2e05262f5775f373dac4772f134041 \
  --local-dir /data/models/MiniMax-M3.1-Flash

Copy that directory to the deployment host, then start without Hub access. SGLang finds the DSpark draft in the local dspark/ subfolder:

docker run --rm --name minimax-m3-1-flash \
  --gpus '"device=0,1,2,3"' \
  --shm-size 32g \
  --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -e HF_HUB_OFFLINE=1 \
  -e FMHA_SM100_ALLOW_JIT=1 \
  -v /data/models/MiniMax-M3.1-Flash:/models/MiniMax-M3.1-Flash:ro \
  -v ~/.cache/minfer:/root/.cache/minfer \
  lmsysorg/sglang@sha256:614b1509bcb729c653528d448bff9850f8977fcad9d12ba56bd6dbe3d7985e87 \
  sglang serve \
  --trust-remote-code \
  --model-path /models/MiniMax-M3.1-Flash \
  --reasoning-parser minimax-m3 \
  --tool-call-parser minimax-m3 \
  --tp 4 \
  --moe-runner-backend flashinfer_trtllm_routed \
  --speculative-algorithm DSPARK \
  --speculative-use-block-verification \
  --cuda-graph-backend-prefill breakable \
  --chunked-prefill-size 32768 \
  --max-running-requests 128 \
  --cuda-graph-max-bs-decode 128 \
  --mem-fraction-static 0.8 \
  --tokenizer-worker-num 8 \
  --host 0.0.0.0 \
  --port 30000

An official ModelScope mirror is available at MiniMax/MiniMax-M3.1-Flash. The ModelScope page does not identify snapshots with the Hugging Face commit used by this baseline, so revalidate the files before substituting a mirror in a revision-pinned deployment. If a download is interrupted, rerun hf download with the same revision and directory; the client resumes from its cache. If loading reports missing or corrupt shards, verify available disk space and redownload the affected revision before changing runtime parameters.

Production and security

Keep the loopback binding for development. Before exposing the service to another host:

Set an unpredictable SGLang --api-key and send it as Authorization: Bearer <key>.
Terminate TLS at a reverse proxy or trusted ingress, restrict source networks with a firewall or security group, and add request-rate and concurrency limits.
Do not log API keys, full prompts, Base64 media, or model outputs unless your data policy explicitly permits it.
Reject arbitrary remote media URLs at the gateway. Apply a domain allowlist, block private and link-local destinations, limit redirects, download time, byte size, image pixels, and video duration, and validate media types before forwarding a request.
Monitor GPU memory, queue depth, request latency, error rates, process health, and disk utilization. A successful /health response alone is not a production readiness test.

SGLang accepts --api-key, but the current MiniMax-M3.1 Cookbook does not provide a complete remote-media security policy. Enforce those controls in your gateway and network layer. For multi-turn agent traffic, the Cookbook documents a hierarchical KV cache (--enable-hierarchical-cache --hicache-size <GB per rank>) that keeps evicted prefixes in host memory. When several replicas serve one endpoint, put the SGLang router in front with --policy cache_aware so that a session’s later turns reach the replica that holds its prefix. Size host memory before enabling HiCache: --hicache-size is per rank.

Troubleshooting

Symptom	Likely cause	Shortest corrective action
Startup fails with `JIT compilation is disabled and AOT kernel ... was not found`	`FMHA_SM100_ALLOW_JIT` is not set and no compiled MSA kernel is cached	Add `-e FMHA_SM100_ALLOW_JIT=1`. Only the variable’s presence is checked, so `=0` does not disable JIT
The log does not show `nvfp4_to_fp8 backend=qmul4`, and decode is slower than expected	The server found a CUDA toolkit older than 13.4	Use the pinned image, or point `CUDA_HOME` at a CUDA 13.4 toolkit
Every start spends a long time compiling kernels	The MSA kernel cache is not persisted between containers	Keep the `~/.cache/minfer:/root/.cache/minfer` mount
`CUDA out of memory` during startup, CUDA graph capture, or prefill	Insufficient activation headroom or an oversized prefill chunk	Reduce `--mem-fraction-static` below `0.8`, then reduce `--chunked-prefill-size`; this reduces KV capacity or prefill speed, so benchmark again. On B200, use the B200 recipe from the generator rather than B300 values
Startup fails with `M3.1 NVFP4 DSPARK requires Blackwell`	DSpark was enabled on a non-Blackwell NVIDIA GPU, such as H200	Use the H200 recipe from the generator, which runs without speculative decoding
Weights fail to load or a shard is missing	Incomplete download, insufficient disk, or a different revision	Confirm the model path, free disk, and revision; rerun `hf download` with the pinned revision
NCCL initialization hangs or times out	Not all GPUs are visible, insufficient shared memory, or a topology/driver problem	Verify four GPUs with `nvidia-smi`, preserve `--ipc=host` and `--shm-size 32g`, then enable `NCCL_DEBUG=INFO` and inspect the first failing rank
Raw `<mm:think>` or MiniMax tool-call tokens appear in `content`	The corresponding parser was not enabled or the runtime differs from the pinned image	Restore `--reasoning-parser minimax-m3` and `--tool-call-parser minimax-m3`, then restart with the pinned digest
A request returns HTTP 400 for `thinking`	`thinking.type` is not `enabled`, `disabled`, or `adaptive`	Send one of those three values, or omit `thinking`
`reasoning_effort` has no effect	The effort was sent inside `chat_template_kwargs`, which SGLang discards for this model, or thinking is disabled	Send `reasoning_effort` as a top-level field, with thinking enabled or adaptive
`/health` fails after the process starts	The model is still downloading/loading or compiling kernels, a worker exited, or port publication is incorrect	Inspect container logs, verify host disk and GPU state, and confirm `127.0.0.1:30000:30000` is published
A remote image request fails	The server cannot reach the URL, or the gateway blocks it	Use an allowlisted URL or a validated Base64 data URI; do not disable network protections globally

Other serving frameworks

The model card also lists the following paths. They are outside the validated scope of this page; follow the linked upstream recipe in full.

Framework	Upstream recipe	Scope stated by the recipe
vLLM	vLLM recipe	vLLM `0.33.0` or later. TP 4 on Blackwell or MI355X, TP 8 on H100 / H200 with FP8 KV cache and the Marlin MoE kernel, and no DSpark on Hopper. `--block-size 128` is required. The reasoning and tool-call parsers are named `minimax_m3`, and reasoning effort is set through `chat_template_kwargs`
TokenSpeed	TokenSpeed recipe	Selects MXFP8 projections with NVFP4 experts automatically, which requires Blackwell. The example runs TP 4 with the DSpark draft path set to the checkpoint’s `dspark/` subfolder
FlagOS	Multi-chip support and downloads	W8A8 versions requantized from the QAT NVFP4 weights for chips from nine vendors: T-Head, NVIDIA, MetaX, Hygon, Huawei Ascend, Moore Threads, Iluvatar, Kunlunxin, and Tsingmicro. Hardware requirements are in each platform’s model card

License and use restrictions

The weights are released under the MiniMax Community License. It grants use for non-commercial purposes. Commercial use requires displaying “Built with MiniMax M3.1-Flash” and either a one-time notice to MiniMax or, for products and services with more than 20 million US dollars in yearly revenue, prior written authorization. The license also lists prohibited uses, including any military purpose. Read the complete license and complete the applicable process before production or commercial use; this page is not legal advice.

Resources

SGLang MiniMax-M3.1 Cookbook

Review the upstream hardware matrix, configuration generator, Playground, PD disaggregation, and agentic long-context launches.

MiniMax-M3.1-Flash model card

Review the official weights, model capabilities, inference settings, and license.

MiniMax Sparse Attention

Review the MSA kernel source and Blackwell requirements.