Exploring more
efficient inference.

DualGoose uses forward and backward RWKV-7 streams to denoise text without self-attention. We combine the streams using a merge that depends on the diffusion timestep, then test whether this linear-time design can compete with Transformer denoisers at the same parameter count.

DreamingGoose converts pretrained autoregressive Transformers into recurrent diffusion models in three stages. It changes the architecture and training objective separately. With roughly 30M distillation tokens, the converted 1.7B and 8B models reach 0.73–0.78 top-1 agreement with their teacher.

ASKV aims to reduce KV cache memory at long context lengths. It preserves the tokens needed to route attention and compresses less-used cache entries along the hidden dimension. Compression runs on sealed pages, outside the decode path. The design report and update protocol are complete, with benchmarks still to come.

Workers running distributed inference can see model shards and may try to keep them. RBWO proposes binding those shards to a trusted runtime that holds the secret transforms needed to use them. The goal is to make recovering a usable model more costly. It does not provide a cryptographic confidentiality guarantee.