AI

Background Frequent HBM Access Creates an I/O Bottleneck in Standard Attention The authors focus on the I/O bottleneck in attention, pointing to the frequent HBM accesses in standard implementations as a major cause. To understand what is going on, we first need to look at the memory hierarchy found in GPUs and most other accelerators. Improving Actual Wall-Clock Time, Not Just Reducing FLOPs The authors note that previous work has tried to reduce the compute and memory complexity of attention, but many of those studies do not report improvements in actual wall-clock time. Their explanation is that these approaches tend to focus on reducing theoretical operation counts while overlooking the overhead of memory access. ...

This post summarizes LightMem: Lightweight and Efficient Memory-Augmented Generation. The easiest way to give an LLM agent memory of past conversation is to include the whole history in the prompt every time. That becomes harder to sustain as conversations grow. A long context triggers the “Lost in the Middle” problem, where the model ignores information buried in the middle, and memory systems that re-read the accumulated history on every turn pay for it with higher compute and slower responses. LightMem targets both problems at once. It’s a lightweight memory-generation system that cuts token usage to a fraction of what existing systems need, while outperforming them. ...

This post summarizes Google’s 2017 KDD paper Ego-Splitting Framework: from Non-Overlapping to Overlapping Clusters. Why Non-Overlapping Clustering Falls Short Real-world networks tend to have plenty of medium-sized communities (roughly 100 members), and a single node frequently belongs to several of them at once. Non-overlapping clustering algorithms assign each node to exactly one community, so they can’t capture that structure. Algorithms that attempt overlapping clustering already existed, but most were either too complex, too inflexible, or lacking in theoretical guarantees. ...

Community detection is the problem of finding communities: sets of densely connected nodes within a graph. Think of it as a form of clustering. Methods like GraphRAG use exactly this technique to break a knowledge graph into manageable pieces. This post covers where modularity, the most widely used metric in community detection, comes from. It then compares Louvain and Leiden, the two algorithms most commonly used to optimize it. Modularity: Measuring How Well-Formed a Community Is Say you’ve formed a community. What does it mean for that community to be “good”? The most common answer in community detection is a metric called modularity. The intuition is simple: compute how many edges you’d expect inside this community if the graph’s edges were wired up completely at random, then measure how far the observed edge count exceeds that expectation. If the connections are too dense to explain away as coincidence, that’s evidence you’ve found a real community. ...

This post summarizes the Microsoft Research paper From Local to Global: A GraphRAG Approach to Query-Focused Summarization, also drawing on the video [Paper Review] GraphRAG by Seoul National University’s DSBA Lab. RAG works by building a trusted document collection ahead of time, then, when a question comes in, retrieving the relevant documents and handing them to an LLM as grounding for its answer. The basic pieces are indexing (chunking documents into a searchable form), retrieval (finding documents relevant to the question), and generation (producing an answer from the retrieved documents and the question). ...

This post is a review of SimpleMem: Efficient Lifelong Memory for LLM Agents. Some parts of the paper have been updated after this post was written (2026-01-24), so there may be differences from the version discussed here. Background LLMs are stateless. As a result, previous inference outputs do not directly affect later outputs. Without conversation history in its input, a plain LLM cannot recall what was just discussed, making it difficult to maintain continuity across turns. ...

SIMA (Scalable Instructable Multiworld Agent), which DeepMind published in 2024, is a generalist agent built for 3D virtual environments. Give it a screen and a simple natural-language instruction, and it plays a 3D game almost the way a human would. Not One Game, But Many SIMA isn’t a bot tuned for one specific game, and that’s what makes it interesting. Working with eight game studios, DeepMind trained it across nine titles, including No Man’s Sky (exploring alien planets), Satisfactory (building automated factories on an alien world), and Valheim (a Norse-mythology survival crafting game). It can follow human instructions and play games it has never seen before. ...