AI signal, minus the noise.

A hands-on tutorial streams 3,000 documents from the FineWeb sample-10BT subset without downloading the full multi-terabyte corpus. It reproduces quality filters (Gopher, C4, custom), finding most already-passed due to pre-filtering. MinHash-based deduplication with 128 permutations and 0.7 threshold identifies few near-duplicate pairs, consistent with per-crawl deduplication. GPT-2 token counts are verified against the stored field, showing near-perfect match (mean absolute difference ~0). Analytics cover token distribution, language scores, characters per token, and top domains, providing practical insights for scaling corpus preprocessing pipelines.

This tutorial article outlines three different levers that can cause a language model to appear better when its version number increases from 4.8 to 4.9, and cautions against confusing them. It does not reference specific models, benchmarks, or techniques.

The author audited 500 code commits and found that AI-generated code can be identified without relying on watermarks. The detection approach uses the commit graph, a diff parser, and a willingness to handle irregular edge cases. The methodology suggests that AI authorship leaves discernible patterns in the structure of code changes and commit history. The article frames this as a practical pipeline for flagging AI-written contributions in version control.

Fei-Fei Li and Yann LeCun have each raised a billion dollars to back world models for physical AI, marking a shift away from language-centric approaches. The article details how world models decide when physical AI systems can effectively interact with the real world. This funding underscores a major bet against large language models as the sole path to general intelligence.