https://4f0a9a3b.wangpeng.pages.dev/en/tags/consumption-model/Recent content in Consumption Model on Zhuifeng NotesHugoenTue, 25 Aug 2026 00:00:00 +0000https://4f0a9a3b.wangpeng.pages.dev/en/tech/kafka/storage-consumer/Tue, 25 Aug 2026 00:00:00 +0000https://4f0a9a3b.wangpeng.pages.dev/en/tech/kafka/storage-consumer/<p>Kafka&rsquo;s high performance relies on &ldquo;partitioned log + sequential writes + batched pulls&rdquo;. Understanding the storage and consumption model is essential to configure parallelism correctly and avoid duplicate consumption.</p> <h2 id="topics-and-partitions">Topics and Partitions </h2> <ul> <li>A <strong>topic</strong> is a logical subject; a <strong>partition</strong> is the physical unit of parallelism, and messages are only appended to the end of a partition.</li> <li>Each partition is an ordered, immutable log composed of multiple segment files, maintaining an offset (unique and monotonically increasing within the partition).</li> <li>The partition count determines the upper bound of consumption parallelism: at any moment a partition is consumed by exactly one consumer within a group.</li> </ul> <h2 id="offset-and-consumption-position">Offset and Consumption Position </h2> <p>The position a consumer commits after processing messages is the <strong>offset</strong>, which Kafka stores in the internal topic <code>__consumer_offsets</code>:</p>