The efficient handling of many tasks; event-driven I/O-bound programming; or why I like vibe.d
Since their beginning, many things have changed about how computers run. One of those is the difference in the time it takes to do different kinds of tasks. In particular, computers now commonly do tasks that differ in efficiency by a factor of a million.
To get an idea of what this is like, imagine your job is to respond to letters containing addition problems. When a letter arrives, and you
Open and read the letter (15 seconds)
Ask the archivist to check the archives to verify that the sender has an account with you. If the archivist happens to have the right volume of the archives on hand, this takes 5 minutes of flipping through pages looking for the right entry. If not, it might take a transatlantic sailing ship to reach the archives and retrieve the right volume, with a round-trip time of about 2 months.
Solve the addition problem (1 second)
Write and post your reply (30 seconds)
This may seem extreme, but it is based on actual timing tests on my computer, scaled up to a human time-scale. The huge difference between the seconds spent on the reading, solving, and writing and the minutes to months spent accessing the archives is a common problem faced when trying to write efficient code today.
The simplest approach to this problem is to do the fours steps as listed for each letter. If the archivist takes a month, I just wait around for that month. We call this approach “blocking” because accessing the archives blocks you from doing anything else while you wait for the answer. It is easy to program using blocking and easy to reason about such programs’ correctness, but it is not a very efficient way to do your work.
A more efficient alternative splits the task into two pieces: one to do when you get a letter and another to do when you get a response from the archivist.
When a letter arrives, you
Open and read the letter (15 seconds)
File the letter in the “to be answered once verified” box (10 seconds to 5 minutes, depending on how full that box is)
Ask the archivist to check the archives to verify that the sender has an account with you.
When the archivist says “User X has an account”, you
Retrieve the filed letter for that user (10 seconds to 5 minutes)
Solve the addition problem (1 second)
Write and post your reply (30 seconds)
If the archivist has the book on hand, this changes the total time from about 5:46 minutes to about 6:06 (generally not 15:46 because if the book is on hand, the file won’t get very big). A little slower than a blocking response time, but not that bad. However, the real power of event-driven approaches shows up when you get several letters at once: instead of waiting two months each back-to-back, answering the twelfth a year later, you can send off for all the books needed from the archive at around the same time and answer each letter as each book is returned, which will be almost as fast as it would have been if that letter were the only one received.
This two-stage approach is called “event driven”: rather than deciding how to solve a problem, we decide how to react to each event in a way that will eventually result in the problem being solved.
The downside to this is that there are a lot more pieces to understand and get right. We took what was a simple, easy-to-reason-about process and made it longer: we added more steps in total, added a need to reason about event sequences, and added a whole new, potentially-complicated filing system to the process. Suddenly we have to start asking questions like “what happens if there are three unanswered letters from the same user and their account says they’ve only paid for two answers?” and “what happens if all the archive-retrieval ships are in use when a request arrives?”
Many of the complexities of correct event-driven programming are the same across all event-driven programs, whether they be answering letters or logging findings or whatever. Since they are the same, a common approach is to craft the solution just once, in a general form, and provide that solution to others to use.
The oldest such shared solution is part of what operating systems do. Operating systems have several important components, but one of them is acting as the Universal Archivist. To be universal, the archivist can’t be as specific as answering “does this user have the right account,” but it can provide a service like “tell me what’s on page 84 of volume 137 of the user records” with a lot of carefully-designed special-case handling so that an answer always arrives, no matter where that volume is or what the ships are doing. Virtually all complex software today is written using an operating system.
The next level of generalization is to train more versatile archivists, ones that know how to recognize common archive formats and answer questions like “what is stored in the entry labeled ‘user X’?”. These versatile archivists are called “databases management systems” and are very common in software today. There are heated debates about the “right” set of capabilities to give these archivists, but very few people argue against using some form of database management system.
Recently, we’ve seen an increase in a third kind of shared solution: event driver frameworks11 “Event driver” because they help make “event-driven” code. Some people will object to my use of “framework” here, as the word “framework” has formal meaning in some circles, but these frameworks provide the general structure and support of your program without providing the details that make your program yours. . These are sort of like procedure templates: they have full instructions for setting up a filing system, sending requests to the archivist, and matching replies with requests; but they have blanks to fill in for what the content of those requests should be and what you want to do when the answers arrive. As long as you follow these templates, you can trust that all the corner cases and other gotchas of event-driven programming will be handled correctly.
Event drivers are still coming into their own, but two basic approaches to writing them have emerged so far.
Currently the most popular approach22 node.js is probably the best-known example of this approach to event drivers is to use callbacks with closures33 I have tried and failed to create a simple explanation of what closures are and why “closure” is the name for them… . These work by putting an intermediary between you and the archivist; you tell the intermediary not only what you want the archivist to do, but also what you want to be reminded to do with archivist’s response.
Thus, when a letter arrives you might
Open and read the letter (15 seconds)
Ask the intermediary to
ask the archivist for that user’s account information
remind you to answer the letter if the archivist says there is an account
and when reminded to respond to a letter you might
Solve the addition problem in the letter (1 second)
Write and post your reply (30 seconds)
This approach takes care of almost all of the bookkeeping (matching events to requests) for you: all you need to know is what the events are and how to react to them.
An alternative approach, one I prefer to program with, is quite tricky to describe, in part because it relies on something that humans can’t do. Basically, the idea runs as follows:
When a letter arrives, and you
Open and read the letter (15 seconds)
Ask the intermediary to check the archives to verify that the sender has an account with you (you perceive that the intermediary replies immediately with the answer)
Solve the addition problem (1 second)
Write and post your reply (30 seconds)
In other words, this looks to you exactly like the simple blocking solution, except it is faster. But what is happening to give you the perception that the reply was immediate? Well, the intermediary
freezes your brain, opens up your skull, puts your brain on a shelf
sends your request on to the archivist
puts a series of different brains in your head; you do useful work with each, none of which you’ll remember when you get your original brain back
gets a reply from the archivist
restores and re-thaws the brain that was waiting for a reply from the archivist—from your brain’s perspective, no time has passed
Thus the intermediary makes a new brain for you in response to each letter arriving and responds to other events by swapping brains. This means each of your brains sees only one letter and can ignore events completely, but all letters get answered in an event-driven way. In many respects, it is the best of both worlds.
Due to a series of names derived from other names derived from marketing or analogies, the freezable brains in this scenario are called “fibers” or “greenlets”44 For the programmers in my audience: we want cooperative multithreading, not the more traditional preemptively-scheduled multithreading, both for performance reasons and in order to avoid needing to reason about interruption and the associated synchronization challenges in the user code. . Despite the fact that many programming languages have all of the individual components needed to implement this approach, the only framework using this approach of which I am aware is the vibe.d framework for the D programming language.
In summary, many kinds of software involves short bursts of activity separated by long waits for the slower components to respond to requests. Easy-to-reason-about code simply waits for those responses, but more efficient code instead splits tasks up into how to react to various “events”. Correctly and efficiently handling events can be challenging: operating systems and databases make it somewhat easier, but event driver frameworks are the real winners in permitting the writing of simple event-driven code. Popular event driver frameworks like node.js require you to know where an event is happening, but otherwise let you write straightforward code. Fiber-based event driver frameworks like vibe.d let you write code as if every access was fast and there were no events, making efficient code even simpler to write.
And that is why I prefer to use vibe.d over any other approach when writing the kind of programs that handle many small requests and long waits for the archivist.
Looking for comments…