Engineering ·
TinyLoRA tests RL fine-tuning with 13 parameters
A publication dated September 17, 2026 describes TinyLoRA and reports gains on three benchmarks using RL training with 13 parameters.
Radar
Daily bookmarks: relevant news, facts and discoveries.
Articles 1216
Engineering ·
Transformers in a loop as programmable computers
Published on September 17, 2026, the article describes a framework using specific weights and recurrent Transformer layers to emulate computational operations, including conditional branches and program counters.
Engineering ·
Block-recurrent Transformers process sequences with linear complexity
An architecture applies a Transformer layer recurrently to blocks of tokens, using self-attention and cross-attention to process sequences with complexity linear in their length.
Engineering ·
Universal Transformers reuse one layer recurrently
A September 17, 2026 post describes an architecture that repeatedly applies the same Transformer layer instead of stacking distinct layers, and highlights sequential versus parallel computation.
Engineering ·
CocktailASR-1 transcribes a selected speaker in mixed audio
Xiaomi's model uses a reference voice to transcribe a selected speaker in overlapping audio; the publication reports support for English and Chinese.
Engineering ·
Crusoe reports inference performance on MI355X
In MLPerf Inference v6.1 results, Crusoe reported 5.75 million tokens per second for gpt-oss-120b using 512 AMD Instinct MI355X GPUs.
Engineering ·
GOAT proposes trainable priors for attention
Published on September 16, 2026, the article presents attention as Entropic Optimal Transport and proposes replacing the implicit uniform prior with a continuous, trainable prior.
Engineering ·
KATA explores recall in linear attention
A September 16, 2026 article frames associative recall in linear attention as a spherical packing problem and presents KATA activations derived from a homogeneous self-dual cone.
Engineering ·
SAS proposes a context selector inside Softmax
In a September 16, 2026 publication, SAS describes a selector trained inside Softmax to filter context in attention designed for hardware with limited computational resources.
Engineering ·
Softmax Linear Attention aims for efficient global competition
Softmax Linear Attention proposes applying softmax at the head level to restore global competition in linear attention while retaining the goal of linear complexity. The approach can be compared with standard softmax normalization.
Engineering ·
Switch Attention varies attention patterns
A paper presents a dynamic, granular approach combining full and sliding-window attention in hybrid transformers, addressing the former's quadratic cost and the latter's narrower receptive field.
Engineering ·
Thinking with Looped Flows uses local denoising
A post published on September 16, 2026, reports that local denoising objectives helped recurrent Looped Flows models with multi-step computation. It says they outperformed earlier looped models on six reasoning benchmarks, including 58.8% accuracy on ARC-AGI-1.
Engineering ·
Tucker Attention unifies attention variants
The article presents Tucker Attention as a tensor-factorization approach that frames MHA, GQA, and MLA within a shared perspective on low-rank approximate attention.
Engineering ·
Attention in Long-Context Transformers
Published on September 15, 2026, the article examines rank collapse as context grows and analyzes rescaling attention scores.
Engineering ·
auto-gpu-kernel 1.0 generates GPU kernels and harnesses
Version 1.0 introduces an autonomous meta-harness that evolves both GPU kernels and the harness layer; the project won the DSA category in an MLSys 2026 FlashInfer contest.
Engineering ·
Harness adds browser and computer options
Version v0.1.6-alpha.1 adds a Web sidebar with persistent terminals, multiple tabs, and MCP resource support. Experimental options include browser tools and computer drivers.
Engineering ·
Octop presents self-hosted, multi-user AI agents
Published on September 15, 2026, the post describes Octop as an open-source, self-hosted AI assistant platform supporting multiple users and agents.
Engineering ·
RoLA proposes low-rank linear attention for Diffusion Transformers
Published on September 15, 2026, the paper presents RoLA, a low-rank linear attention method with rotary position for Diffusion Transformers. It addresses quadratic scaling in dense spatiotemporal self-attention and a compatibility problem between RoPE and the global branch
Engineering ·
RVC WebUI for training voice conversion models
Published on September 15, 2026, the entry describes an open-source project with a web interface for training voice conversion models using 10 minutes or less of voice data.
Engineering ·
ASR tests isolating one speaker in overlapping speech
Published on September 15, 2026, Xiaomi-CocktailASR-1 demonstrates speaker-conditioned speech recognition on overlapping audio.