Engineering ·
Radar: distilling between Transformers and Mamba
Published on September 29, 2026, the summary describes a study on transferring knowledge from pretrained Transformers to Mamba models and notes memory and throughput advantages for SSMs.
Radar
Daily bookmarks: relevant news, facts and discoveries.
Articles 1216
Engineering ·
KITE expands Transformers with invariant KV computation
A September 29, 2026 entry describes KITE, a scaling paradigm combining scaling with upcycling and expanding decoder components that reuse a KV cache generated by a smaller prefiller.
Engineering ·
Agent memory: preserve context without accumulating personal data
Long context can extend exposure. Separate what an agent needs to remember from the identity of people mentioned in a conversation.
Engineering ·
MiMo RL Environment Explorer
Published on September 29, 2026, the MiMo RL Environment Explorer is described as an interactive space to inspect MiMo-V2.6 reinforcement-learning environments and run rollouts.
Engineering ·
A 7B model learns continuously from prediction markets
A publication dated September 29, 2026 describes a 7B model that continuously updated its weights using prediction-market data and ran continuous backtests over 390 days. It says the model outperformed several larger models in the same test environment.
Engineering ·
OPAL trains linear attention with its own outputs
The approach distills linear attention models in long contexts and, according to the report, fully recovers full-attention retrieval while reaching 83–93% of mathematical reasoning performance with 3 billion tokens.
Engineering ·
OPD tested with a single training example
A paper published on September 29, 2026 tests on-policy distillation with one query across mathematics, coding, instruction following, and agentic tool use.
Engineering ·
PTXBench measures AI-written GPU kernels
Published on September 29, 2026, PTXBench evaluates model-generated matrix multiplication and attention kernels in architecture-specific PTX for H100 and B200 GPUs.
Engineering ·
RESA aims to recover context lost in sparse attention
The September 29, 2026 entry describes RESA, a residual-estimation proposal for recovering context missed when sparse attention selects only part of the KV cache. The post claims the technique adds no computational cost.
Engineering ·
TIRx Harness explores GPU kernel optimization
Announced on September 29, 2026, TIRx Harness is an open compiler harness for agents to explore, debug, and optimize GPU kernels, with compiler analysis, a catalog, hardware specifications, and remote evaluation.
Engineering ·
Vx uses inline MLIR for leading-zero operations
Published on September 29, 2026, the bookmark describes Vx's use of the `mlir!` macro to lower MLIR directly into leading-zero count instructions, avoiding manually maintained routines for different hardware.
Engineering ·
Autonomous agents: constrain tools and process incoming data
An isolated runtime controls what an agent can do. Privacy also requires controlling the content it sends to AI.
Engineering ·
Agents optimize kernels for Kimi Delta Attention
Published on September 28, 2026, the post reports that kernel design agents generated and optimized Kimi Delta Attention kernels, reaching up to 2.96× the performance of FlashKDA on B300 with one-tenth the state error.
Engineering ·
Log-linear block selection for sparse attention
Published on September 28, 2026, the article describes a hierarchical block-selection method for sparse attention in long contexts, reducing selection cost from O(N²) to O(N log N).
Engineering ·
Audio and AI: transcripts need a data policy too
Turning speech into text changes the format, not the sensitivity. Evaluate every stage before introducing audio into a workflow.
Engineering ·
FrontierSmith releases 200 open RL problems
Published on September 28, 2026, the collection presents 200 environments for studying reinforcement learning in challenging, open-ended, long-horizon tasks.
Engineering ·
IFM-AI opens tools for LLM training and serving
A September 28, 2026 announcement presents open-source tools for pretraining, attention, checkpoint conversion, and language-model serving.
Engineering ·
Local inference: evaluate performance and exposure in the same test
Running a model close to data can change the architecture. It does not replace input processing, access control or real evaluation.
Engineering ·
Matryoshka Attribution ranks components
Published on September 28, 2026, the post describes a technique that ranks neural representations or fine-tuning weight changes by their influence on behavior preserved under soft interventions.
Engineering ·
Vx models heterogeneous memory for compiler checks
Published on September 28, 2026, the post presents Vx, which describes memory capacity, connections, and visibility to check programs. It reports two capacity errors found on a laptop without a GPU.