AI FRONTIER

TurboMamba: Faster Inference, Faster Training, and Better Prediction with Lexical Memory

The paper introduces TurboMamba, a language-model architecture in which a global Mamba-3 model runs once per completed token block and a small Transformer uses that global context plus earlier tokens in the block to predict the next token.

• Across twelve small-scale implementations on public FineWeb-Edu with comparable parameter counts and shared training-budget caps, TurboMamba reports the lowest mean test prediction loss (NLL).
• Its mean NLL is reported 9.4% lower than Mamba-3, 7.3% lower than the Jamba version, and 18.3% lower than the Zamba version.
• End-to-end training throughput is reported at 2.60x that of Mamba-3, 2.81x that of the Jamba version, and 4.82x that of the Zamba version.

Because the reported gains in loss and throughput come at comparable parameter counts and training budgets, builders should reproduce it on their own long-context or high-throughput workloads before adopting it.

OpenRouter new model: Reflection Beam 1.0 501B A23B

Beam 1.0 is Reflection's open-weight reasoning model for coding and long-horizon agentic workflows, using a sparse mixture-of-experts architecture with 501B total and 23B active parameters per token, a 1M-token context window, and an Apache 2.0 release.

• Sparse mixture-of-experts architecture: 501B total parameters, 23B active parameters per token
• Context window of 1,048,576 tokens, supporting up to 131,072 completion tokens
• Priced at $0.25 per 1M input tokens and $1.00 per 1M output tokens
• Released October 16, 2026, with a June 30, 2026 knowledge cutoff, under Apache 2.0

For teams building coding agents, Beam 1.0 pairs Apache 2.0 open weights, a 1M-token context, and only 23B active parameters per token, making long-horizon reasoning available either self-hosted or pay-per-token.

Expanding Attention: A Taylor-Series Route to Linear-Time Transformers with a Fixed Number of Recurrent States

The paper expands the exponential kernel of attention into a truncated Taylor series of n terms and factorizes each term into a tensor-power inner product, collapsing the causal history into exactly n recurrent states, each updated by one rank-one outer product per token and independent of context length N.

• The recursion is lossless with respect to the truncated kernel at 2.2 × 10^-16 in float64, and states are additive across chunks at 5.7 × 10^-14, so prefill admits a parallel scan.
• The paper reports measured exponents of t ∝ N^2.28 for softmax versus N^0.91 for states, with a crossover at N ≈ 728 and 20.8× at N = 8192.
• Per-token decoding latency is constant at 7.3 μs versus N^1.04 for softmax, giving 76.0× at N = 65536.
• Query and key normalization is a prerequisite: without it the argument exceeds the convergence radius and adding terms stops helping.

For builders, this indicates fixed-state linear attention pays off only above roughly 728 tokens and requires query and key normalization, without which adding Taylor terms is ineffective.

OpenRouter new model: XingChen Xing4.0 29B A4B (free)

Xing4.0-29B-A4B is an open-weight mixture-of-experts language model from China Telecom's AI team (TeleAI) and the successor to the TeleChat series, with 29B total parameters, 4B active per token, and a native 256K-token context window.

• 29B total parameters with 4B active per token and a native 256K-token context window; OpenRouter lists the context as 262,144 tokens.
• Trained entirely on Ascend NPUs with the MindSpore framework.
• Tuned for complex engineering and agent work including tool calling, multi-step planning, and long-context execution; reasoning is on by default and can be turned off.
• Free on OpenRouter, released October 16, 2026, and supports tools and tool_choice for function calling plus structured outputs via a JSON schema in response_format.

For agent builders, this is a zero-cost 29B open-weight MoE endpoint with 256K context and tool calling, making it practical for testing long-context multi-step planning pipelines.

The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock

The paper introduces RealtimeBench, 180 tasks across six game types that evaluate agents while games keep running during inference, and proposes TORSI, in which a slow teacher studies failures and tests corrections between attempts and writes reusable instructions for a fast student.

• RealtimeBench contains 180 tasks across six game types, with games continuing to run during inference.
• Paired evaluations show GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running.
• In TORSI, the student acts without teacher calls during gameplay, and validation scores determine which updates are adopted.
• The paper reports two rounds raise Jev's average test score from 13.51 to 24.19 on a 0–100 progress scale, improving all six game types.

For agent builders, the result shows that latency can flip model rankings, so slow offline learning and fast online execution should be separated rather than optimizing single-pass inference alone.

Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?

The paper re-scores the same outputs with answer extractors copied from public code and with its own extractor, finding that on four released 7B SimpleRL base-RL pairs, 82% of the out-of-domain multiple-choice gain those extractors report disappears under the authors' extractor, indicating a gain in answer format rathe...

• On four released 7B SimpleRL base-RL pairs, 82% of the out-of-domain multiple-choice gain reported by extractors copied from public code disappears under the authors' extractor.
• Independent LLM readers in place of the authors' extractor put the figure at 77%, and a prompt that itself asks for the box puts it at 79% under the authors' extractor.
• The base models already state most of their answers in text, and RL moves them into the \boxed{} box, where these extractors look first.
• On a 32B pair trained with another recipe, almost none of the reported gain is format gain: its base model boxes its answers when asked, and its RL model rarely does when not.

Builders should re-score stored outputs with a second extractor before reading RLVR out-of-domain gains, because a reported gain is partly a property of the extractor that read it.

Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature R...

论文摘要：Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study...

This is a useful technical direction to keep on the radar, even if it does not require immediate action.