Engineering ·
Jamba combines Transformer and Mamba layers
Published on September 20, 2026, the item describes Jamba, a base language model with a hybrid Transformer-Mamba architecture and mixture-of-experts.
Radar
Daily bookmarks: relevant news, facts and discoveries.
Articles 1216
Engineering ·
On-Demand Attention selects when to use full context
Published on September 20, 2026, the post describes a technique that uses a lightweight head to trigger full context when needed, aiming to speed decoding while recovering performance lost with local attention.
Engineering ·
QLoRA reports fine-tuning a 65-billion-parameter model on a 48 GB GPU
Published on September 20, 2026, the article summary reports fine-tuning a 65-billion-parameter model on one 48 GB GPU, using 4-bit quantization and LoRA adapters.
Engineering ·
RWKV combines parallel training with recurrent inference
Published on September 20, 2026, the article summary presents an architecture combining Transformer-style parallel training with RNN-style recurrent inference.
Engineering ·
Speculative decoding may speed up inference
The technique pairs proposals from an efficient model with parallel checks by a larger model to reduce serial decoding steps without changing the output distribution.
Engineering ·
GEPA adapts a model with prompts and labeled examples
A publication dated September 19, 2026 describes a GEPA demonstration adapting a commercially available model to specific decision criteria by changing its prompt and labeling examples.
Engineering ·
CUDA C++: GPU execution, memory, and correctness
A concise guide to core CUDA programming concepts: kernel launches, thread organization, memory, synchronization, runtime initialization, errors, and validating GPU results against a CPU implementation.
Engineering ·
Radar: language models that self-evaluate
Published on September 19, 2026, the paper studies language-model feedback prompted as LLM-as-a-Judge during training, compared with reward models based on human preferences that remain frozen.
Engineering ·
GuppyLM: a language model with 9 million parameters
A post introduces GuppyLM, a language model with about 9 million parameters, and claims it can be trained from scratch in five minutes. The project is presented as a small-scale way for engineers to explore language-model training.
Engineering ·
ORPO combines SFT and preference optimization
Published on September 19, 2026, the paper introduces ORPO, a method combining preference alignment and supervised fine-tuning without a reference model.
Engineering ·
Open-source tax-page classifier reports 100% accuracy
A GitHub project describes classifying pages of tax documents and reports results on 261 IRS forms, at an estimated cost of about US$0.001 per page.
Engineering ·
JEPA-Anything explores orthogonal factors
The domain-agnostic framework uses orthogonal predictive factorization to decompose latent targets into complementary factors; its authors say the code, checkpoints, and report are open.
Engineering ·
Never Give Up steers RL toward harder problems
A note dated September 18, 2026, reports that reinforcement learning (RL) tends to improve LLM performance more on easy problems than on hard ones. The Never Give Up method uses adaptive sampling to direct compute toward harder cases.
Engineering ·
New OCR models and benchmark result
A September 18, 2026 publication presents WeVisDoc 2B and 4B and jina-ocr-v1, with different licenses, and reports WeVisDoc-4B's OmniDocBench v1.6 score.
Engineering ·
Qualcomm releases IDL FastRPC compiler for Hexagon
On September 18, 2026, Qualcomm announced the release of the IDL FastRPC compiler source code for Hexagon. Engineers can consult the code in the QAIC repository on GitHub.
Engineering ·
Flow Models as recurrent models for reasoning
A paper described on September 17, 2026, turns a flow model into a recurrent model for tasks such as Sudoku, refining predictions iteratively without backpropagation through time.
Engineering ·
Agora records shared agent memory in Git
In a September 17, 2026 publication, Agora describes an append-only Git DAG for recording research-agent contributions and making results and verification status searchable.
Engineering ·
GrainSpeech proposes speech synthesis with less context
Published on September 17, 2026, the GrainSpeech project is described as generating more detail in speech synthesis with less context. Its project page points to an official repository and a technical paper.
Engineering ·
RULER turns agent evaluations into GRPO rewards
Published on September 17, 2026, the report describes RULER in OpenPipe ART: an LLM ranks trajectories using plain-English criteria, and the comparisons are converted into rewards for GRPO training.
Engineering ·
Supersonic TTS: CPU and support for 31 languages
Published on September 17, 2026: the author says Supersonic TTS runs on CPU and supports 31 languages. The author plans to use it to generate synthetic data.