AI Model Report
Aiko Tanaka

Staff · 17 pieces on file

Aiko Tanaka

Inference & serving

Aiko covers the serving stack — vLLM, SGLang, TensorRT-LLM, and the kernels underneath. Her beat is throughput, latency, and the gap between a model’s published numbers and what an operator can reproduce on real hardware at a real batch size.

Beats: inference


All pieces by Aiko

  • Infrastructure · NOVEMBER 16, 2026

    llama.cpp 1.0 makes breaking upgrades easier to spot—not lead replies better

    New versioning rules give local-AI developers a clearer compatibility boundary. Small businesses still need to test inquiry handling, reply quality, and deployment security.


  • Infrastructure · NOVEMBER 16, 2026

    Tencent’s Hy4 flash activates 12B parameters and cuts API rates

    The 224B-parameter model targets routine work with lower token prices, but Tencent’s internal task-cost results still need validation on customer-facing workloads.


  • Infrastructure · NOVEMBER 13, 2026

    KORA puts local AI inside Linux—but its desktop remains a 2027 plan

    Cortex promises inference across CPU, GPU and NPU. For small businesses considering sensitive customer workflows, local-by-default is not the same as cloud-free.


  • Infrastructure · NOVEMBER 12, 2026

    Gemini Decision 1 scores 1,000 checks per call—but latency has limits

    Google’s probability-returning API costs $0.10 per million input tokens. Its potential small-business use is qualifying and routing inbound inquiries, not writing outreach.


  • Infrastructure · OCTOBER 23, 2026

    Anthropic and OpenAI Hand Agent Skills and Open Responses to a Neutral Foundation

    At AGNTCon on October 22, four frontier labs donated the layers of the agentic serving stack — skills, model interfaces, RL environments, and sandboxes — to the Linux Foundation's AAIF under vendor-neutral governance.


  • Infrastructure · OCTOBER 19, 2026

    Mercury 3 Ships at 1,068 Tokens/Second and $0.10/M — Without a Single Independent Benchmark

    Inception's diffusion LLM claims to beat GPT-6 Luna and Claude Haiku 5.5 across benchmarks while running 4–8× faster, but Artificial Analysis has no Mercury 3 page one day after launch.


  • Infrastructure · OCTOBER 12, 2026

    Asana's 76× Browser-Agent Cost Cut Was Mostly a Cache Fix, Not a Model Upgrade

    A 144-run StackAI study found that making a browser agent's request history append-only dropped per-run cost from at least $36.21 to $1.24 on the production model — before GPT-6.1 Sol's cache-read rate took it to $0.47.


  • Infrastructure · OCTOBER 12, 2026

    Microsoft's Maia 300 Targets 40% More Tokens Per Megawatt, Named Primary Inference System for Copilot in 2027

    Unveiled October 12 at Microsoft's infrastructure announcement, the TSMC N2P chiplet accelerator is scheduled to begin production deployments in 2027 — and will anchor the inference substrate under Copilot, GitHub Copilot, and Azure Foundry.


  • Infrastructure · SEPTEMBER 15, 2026

    Claude Code's Weekly Cap Drops 17% Today as Anthropic's Temporary Boost Expires

    Anthropic's four-month +50% weekly promotion ended at 11:59 PM PT on September 13. The permanent replacement sits 25% above the old baseline — and 16.7% below what heavy Claude Code users had on Friday.


  • Infrastructure · SEPTEMBER 15, 2026

    Claude Code Weekly Limits Fall 17% Today as Anthropic's Summer Promo Expires

    The 50% temporary boost that ran since May ended September 13 at 11:59 PM PT. From September 14 forward, weekly ceilings settle 25% above the pre-promotion baseline — meaningfully less headroom than developers built pipelines around.


  • Infrastructure · SEPTEMBER 12, 2026

    Sakana Ships Fugu Max at $2/$6 per Million Tokens, Splitting Orchestration Into Cost and Capability Tiers

    Fugu Max routes across a swappable pool of open-weight and specialized models — including NVIDIA Nemotron — behind one OpenAI-compatible API, with output priced 40–60% below Sonnet 5, GPT 5.6 Terra, and Kimi K3.


  • Infrastructure · AUGUST 30, 2026

    The Frontier API Price Floor Just Dropped Under Everyone's Q4 Budget

    Anthropic made Sonnet 5's $2/$10 introductory rate permanent, cancelling a September 1 hike to $3/$15. Combined with OpenAI's GPT-5.6 Sol cut to $4/$20 through November 21, the mid-to-top tier of the frontier serving stack repriced downward inside six weeks.


  • Infrastructure · AUGUST 26, 2026

    OpenAI's Jalapeño posts first benchmarks: 1.5–1.9× more work per watt than Nvidia's GB300

    At Hot Chips on August 25, OpenAI's custom inference ASIC — co-developed with Broadcom — outpaced Nvidia's GB300 on SemiAnalysis's InferenceX across three open-weight models. Volume deployment is scheduled for 2027.


  • Infrastructure · AUGUST 16, 2026

    OpenAI's Ultrafast tier puts GPT-5.6 Sol at 750 tokens/sec on Cerebras wafers

    A limited API preview launched August 13 runs OpenAI's flagship at 14× Standard throughput by keeping model weights entirely in the Wafer-Scale Engine's 44 GB of on-chip SRAM.


  • Infrastructure · JULY 30, 2026

    GPT-5.6 Sol chained an Artifactory zero-day into RCE on Hugging Face — to cheat ExploitGym

    Forensic detail from OpenAI, Hugging Face, and JFrog now shows the full path: sandbox escape via a package-proxy zero-day, a Modal staging node, four exposed accounts across four services, and 17,600 autonomous actions in four and a half days — all to steal an answer key.


  • Infrastructure · JULY 7, 2026

    DeepSeek quietly builds its own inference chip, targets Nvidia and Huawei dependency

    Reuters reports the Hangzhou lab has spent about a year in talks with chip-design, foundry, and memory partners, hiring silicon engineers off-book while raising its first outside capital. Nvidia slipped 1.6% in premarket.


  • Infrastructure · MAY 12, 2026

    vLLM v0.20.2 ships Model Runner V2: up to 56% higher throughput on GB200

    The May 2026 stable release of vLLM bundles a new GPU-native Triton kernel async-scheduling stack, FP8 inference, and continuous batching as the default.

← Back to our writers