Asynchronous prefetching is a software design pattern that keeps a read-ahead buffer stocked with data pages (collections of contiguous bytes) that a sequential reader thread will eventually need. The motivation for implementing asynchronous prefetching is to eliminate any I/O wait time the reader would normally incur if it loaded its own pages on an as-needed basis from permanent storage. Aggressively caching pages before they are requested may seem like more trouble than it's worth, but databases and many other programs scan through so many pages from storage that even a tiny delay resulting from a cache miss could quickly add up to a significant amount of I/O wait time. Prefetching ensures that whenever the reader is ready to start processing a new page of data, a suitable page is already loaded and waiting in the cache. Asynchronous prefetching involves the following concepts or elements:
- A reader that tends to process pages in a sequential manner (otherwise there's no way to predict which pages to cache in advance)
- A high-speed cache/buffer of limited size that can hold pages until they are needed
- An asynchronous (detached, runs in the background) process that determines which pages will be needed and loads them into the cache before the reader needs them
Non-technical example: Fresh out of college, you're living the dream of every over-educated, under-employed hipster: writing for a living in a walkup apartment in Brooklyn. On the ground floor, a small convenience store carries your favorite cheap beer, which you purchase often - sometimes alone, sometimes with a new friend. The tiny dorm fridge from college that you're still using can only hold a couple of bottles, but you go outside often enough to stop by the bodega to pick up another bottle or two whenever you're feeling thirsty. It's all good in the hood.
It doesn't take long for you get things pretty much under control. You're meeting new people, making friends, and writing more than you drink. When your first big advance check arrives from a publisher, you know instantly how you're going to celebrate. All those friends you've been drinking with individually over the months are coming to a big party at your place, and you want to show them a good time. The problem is that you have no idea how many will show up - or at what time - but it really doesn't matter anyway, since your fridge can only keep a couple beers cold at a time. You're smart, though, and quickly work out a deal with the convenience store owner downstairs on the day of the party. Before too many guests show up at your apartment, you call the shopkeeper and order up an entire case of cold beer, which he's even agreed to have his assistant shlep up the stairs for you. This keeps you from being a poor host, constantly running down and back up the stairs to fetch a few bottles at a time as your thirsty friends await your return. As each case dwindles to just a few beers, all you have to do is make another call, and another cold case is on its way, ready for you to plop behind the two or three bottles left from the previous case. When it's all over, your party never ran out of cold beer, and you were able to buy only what you needed, without going way over. Your friends consider it to be the best trick ever, but perhaps they're a bit too easy to please.
The fact that you requested an entire case of beer at a time means that you were prefetching, since you knew that your guests would eventually drink them, just not at that exact moment. If you'd waited until your guests drank all the beer in the apartment before ordering the next case, not only would that have been bad form, but that would have been classified as synchronous prefetching, since people would have been waiting at least a little bit for the next case to arrive. However, you were on the ball and kept things running smoothly enough to avoid that dreaded problem entirely. By keeping a close eye on your guests and their beer consumption throughout the party, you also avoided ordering too many cases, minimizing the number of wasted beers.