We only need to store one reference for each buffer, so avoid storing consecutive references for the same buffer. It might not be the most common pattern, but it is easy to implement.