Engineering ·
Radar: scaling byte and token models
A paper compares distilled byte and token models trained at scales up to 1 trillion bytes and reports a higher performance ceiling for byte models as compute increases.
Radar
Daily bookmarks: relevant news, facts and discoveries.
Articles 1216
Engineering ·
PC-ALM trains deep residual MLPs with local dynamics
Published on September 14, 2026, the summary presents PC-ALM as a local alternative to backpropagation for training residual MLPs up to 1,000 layers, with performance described as nearly equivalent.
Engineering ·
Method trains sparse selection of KV blocks
Published on September 14, 2026, Simple Attention Sparsification describes how language modeling loss can optimize which KV blocks each query attends to.
Engineering ·
Tencent presents WeKnora, an LLM knowledge framework
Published on September 14, 2026, the post introduces WeKnora, an open-source framework for document understanding, semantic retrieval, and autonomous reasoning.
Engineering ·
GAD proposes on-policy distillation for black-box LLMs
An article dated September 13, 2026 presents a method that learns from a proprietary teacher model’s text outputs, without access to logits or parameters.
Engineering ·
Open book surveys AI infrastructure in 12 chapters
Published on September 13, 2026, the book covers model architectures, accelerators, inference optimization and distribution, training systems, and resource scheduling.
Engineering ·
Post reports alleged KV cache reduction
A post claims KV cache use fell from 389,000 bytes to 890 bytes per token in three years. It says a linked article explains the approach with diagrams.
Engineering ·
Token retention may reduce KV cache use
A publication reports that keeping four initial tokens and a 64-token window, without training, matched or outperformed distilled linear-attention models on almost all benchmarks.
Engineering ·
RLT carries recurrent state between tokens
Published on September 13, 2026, the brief describes a Transformer with a recurrent decoder that carries state between tokens during the prompt and generation.
Engineering ·
A framework for recursive AI self-improvement
An article published on September 13, 2026 presents a review and conceptual framework for cycles in which AI systems identify limitations, validate changes, and retain improvements.
Engineering ·
The Deadlock Empire teaches concurrency through a game
Published on September 12, 2026, the listing describes The Deadlock Empire, a game where players act as schedulers, explore programs with bugs, and learn about concurrency.
Engineering ·
Looped flows train recurrent updates with local denoising
Published on September 12, 2026, the article proposes training recurrent hidden-state updates with local denoising objectives, seeking to address a difficulty associated with backpropagation across only a few updates.
Engineering ·
Large model shifts across GPU, RAM and NVMe
A report published on September 12, 2026 describes running a model on an RTX 5090 with 125.7 GiB of RAM and part of a 502 GB GGUF file on NVMe.
Engineering ·
A visual introduction to contraction operators in RL
An article in an intuitive reinforcement learning series presents contraction operators, a fundamental concept, through a concise visual approach.
Engineering ·
Spinlock optimization and its trade-offs
The article discusses spinlocks, which keep threads busy on the CPU instead of yielding to the scheduler, and notes that this can avoid system calls and context switches.
Engineering ·
Document classifier with agents and SetFit
Published on September 10, 2026, the account describes a document-purpose classifier trained on 200 examples labeled by an agent. The model classified 191,724 documents for about US$0.70 in inference compute.
Engineering ·
Colibri streams MoE experts from disk in C
Published on September 10, 2026, Colibri is described as a dependency-free, pure-C inference engine that streams MoE experts from disk to run models on local hardware.
Engineering ·
PRISM and binary translation on Windows on Arm
Published on September 10, 2026, this entry describes an article examining PRISM, the binary translation system used by Windows on Arm, and its effects on code.
Engineering ·
Repositories opened for V4.1 Flash deployment
On September 10, 2026, the company announced that it opened three repositories to facilitate deployment of V4.1 Flash and later open models.
Engineering ·
C*: programming and verification in C
Published on September 8, 2026, the article presents C*, an approach to bringing programming in C closer to formal verification.