Learning Async Rust With Entirely Too Many Web Servers

Unknown — Fri, 11 Aug 2023 00:10:10 -0400

I've found that one of the best ways to understand a new concept is to start from the very beginning. Start from a place where it doesn't exist yet and recreate it yourself, learning in the process not just how it works, but why it was designed the way it was.

This isn't a practical guide to async, but hopefully some of the background knowledge it covers will help you think about asynchronous problems, or at least fulfill your curiosity, without boring you with too many details.

..It's still really long.

Table Of Contents</summary>

A Simple Web Server</a></li>
A Multithreaded Server</a></li>
A Non-Blocking Server</a></li>
A Multiplexed Server</a></li>
Futures</a>

A Reactor</a></li>
Scheduling Tasks</a></li>
An Async Server</a></li> </ul> </li>
A Functional Server</a></li>
A Graceful Server</a></li>
Back To Reality</a>

Pinning</a></li>
async/await</a></li>
A Tokio Server</a></li> </ul> </li>
Afterword</a></li> </ul> </blockquote> </details>
A Simple Web Server</a></h2>
We'll start our journey into async programming with the simplest of web servers, using nothing but the standard networking types in std::net</code>. Our server just needs to accept HTTP requests and reply with a basic response. We'll ignore most of the HTTP specification, or write any useful application code, for the entirety of this post, focusing instead on the basic flow of the server.
HTTP is a text-based protocol built on top of TCP, so to start we have to accept TCP connections. We can do that by creating a TcpListener</code>.
use std::net::TcpListener; fn main() { let listener = TcpListener::bind("localhost:3000").unwrap(); } </code></pre> And listening for incoming connections, handling them one by one.
use std::net::{TcpListener, TcpStream}; use std::io; fn main() { // ... loop { let (connection, _) = listener.accept().unwrap(); if let Err(e) = handle_connection(connection) { println!("failed to handle connection: {e}") } } } fn handle_connection(connection: TcpStream) -> io::Result<()> { // ... } </code></pre> TCP connections are represented by the TcpStream</code> type, a bidirectional stream of data between us and the client. It implements the Read</code> and Write</code> traits, abstracting away the internal details of TCP, and allowing us to read or write plain old bytes. As a server, we need to receive the HTTP request. We'll initialize a small buffer to hold the request.
fn handle_connection(connection: TcpStream) -> io::Result<()> { let mut request = [0u8; 1024]; // ... Ok(()) } </code></pre> And then call read</code> on the connection, reading the request bytes into the buffer. read</code> will fill the buffer with an arbitrary number of bytes, not necessarily the entire request at once, so we have to keep track of the total number of bytes we've read and call it in a loop, reading the rest of the request into the unfilled part of the buffer.
use std::io::Read; fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut read = 0; let mut request = [0u8; 1024]; loop { // Try reading from the stream. let num_bytes = connection.read(&mut request[read..])?; // Keep track of how many bytes we've read. read += num_bytes; } Ok(()) } </code></pre> Finally, we have to check for the byte sequence \r\n\r\n</code>, which indicates the end of the request.
fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut read = 0; let mut request = [0u8; 1024]; loop { // Try reading from the stream. let num_bytes = connection.read(&mut request[read..])?; // Keep track of how many bytes we've read. read += num_bytes; // Have we reached the end of the request? if request.get(read - 4..read) == Some(b"\r\n\r\n") { break; } } Ok(()) } </code></pre> It's also possible for read</code> to return zero bytes, which can happen when the client disconnects. If the client disconnects without sending an entire request, we can simply return and move on to the next connection. Again, don't worry too much about sticking to the HTTP specification, we're just trying to get something working.
fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut read = 0; let mut request = [0u8; 1024]; loop { // Try reading from the stream. let num_bytes = connection.read(&mut request[read..])?; // The client disconnected. if num_bytes == 0 { println!("client disconnected unexpectedly"); return Ok(()); } // Keep track of how many bytes we've read. read += num_bytes; // Have we reached the end of the request? if request.get(read - 4..read) == Some(b"\r\n\r\n") { break; } } Ok(()) } </code></pre> Once we've read in the entire request, we can convert it to a string and log it to the console.
fn handle_connection(stream: TcpStream) -> io::Result<()> { let mut read = 0; let mut request = [0u8; 1024]; loop { // ... } let request = String::from_utf8_lossy(&request[..read]); println!("{request}"); Ok(()) } </code></pre> Now we have to write our response. Just like read</code>, a call to write</code> may not write the entire buffer at once. We need a second loop to ensure the entire response is written to the client, with each call to write</code> continuing from where the previous left off.
use std::io::Write; fn handle_connection(mut connection: TcpStream) -> io::Result<()> { // ... // "Hello World!" in HTTP let response = concat!( "HTTP/1.1 200 OK\r\n", "Content-Length: 12\n", "Connection: close\r\n\r\n", "Hello world!" ); let mut written = 0; loop { // Write the remaining response bytes. let num_bytes = connection.write(response[written..].as_bytes())?; // The client disconnected. if num_bytes == 0 { println!("client disconnected unexpectedly"); return Ok(()); } written += num_bytes; // Have we written the whole response yet? if written == response.len() { break; } } } </code></pre> And finally, we'll call flush</code> to ensure that the response is written to the client ^{1</a>.} fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut written = 0; loop { // ... } // Flush the response. connection.flush() } </code></pre> There you go, a working HTTP server! $ curl localhost:3000 # => Hello world! </code></pre> A Multithreaded Server</a></h2> Alright, so our server works. But there's a problem. Take a look at our accept loop. let listener = TcpListener::bind("localhost:3000").unwrap(); loop { let (connection, _) = listener.accept().unwrap(); if let Err(e) = handle_connection(connection) { // ... } } </code></pre> See the problem? Our server can only serve a single request at a time. Reading and writing from/to a network connection isn't instantaneous, there's a lot of infrastructure between us and the user. What would happen if two users made a request to our server at the same time, or ten, or ten thousand? Obviously this isn't going to scale, so what do we do? We have a couple of options, but by far the simplest one is to spawn some threads. Spawn a thread for each request and our server becomes infinitely faster, right? fn main() { // ... loop { let (connection, _) = listener.accept().unwrap(); // Spawn a thread to handle each connection. std::thread::spawn(|| { if let Err(e) = handle_connection(connection) { // ... } }); } } </code></pre> In fact, it probably does! Maybe not infinitely, but with each request being handled in a separate thread, the potential throughput of our server increases significantly. How exactly does that work? On Linux, as well as most other modern operation systems, every program is run as a separate process. While it seems like every active program is run at the same time, it's only physically possible for a single CPU core to execute a single task at a time, or possibly two with hyperthreading. To allow all programs to make progress, the kernel constantly switches between them, pausing one program to give another a chance to run. These context switches happen on the order of milliseconds, providing the illusion of parallelism. The kernel scheduler can take advantage of multiple cores by distributing its workload across them. Each core manages a subset of processes 2</a>, meaning that some programs do in fact get to run in parallel. cpu1 cpu2 cpu3 cpu4 |----|----|----|----| | p1 | p3 | p5 | p7 | | |____| |____| | | |____| | |____| p4 | | p8 | | | | p6 |____| | p2 |____| | | | | p3 | | p7 | | | | | | </code></pre> This type of scheduling is known as preemptive multitasking. The kernel decides when one process has been running for too long and preempts it, switching to someone else. Processes work well for distinct programs because the kernel ensures that separate processes aren't allowed to access each other's memory. However, this makes context switching more expensive because the kernel has to flush certain parts of memory before performing a context switch to ensure that memory is properly isolated 3</a>. Threads are similar to processes 4</a> except they are allowed to share memory with other threads in the parent process, which is what allows us to share state between threads within the same program. Scheduling works much the same way. The key insight regarding thread-per-request is that our server is I/O bound. Most of the time inside handle_connection</code> is not spent doing compute work, it's spent waiting to send or receive some data across the network. Functions like read</code>, write</code>, and flush</code> perform blocking I/O. We submit an I/O request, yielding control to the kernel, and it returns control to us when the operation completes. In the meantime, the kernel can execute other runnable threads, which is exactly what we want! In general, most of the time it takes to serve a web request is spent waiting for other tasks to complete, like database queries, or other HTTP requests. Multithreading works great because we can utilize that time to handle other requests. A Non-Blocking Server</a></h2> It seems like threads do exactly what we need, and they're easy to use, so why not stop here? You may have heard that threads are too heavyweight and context switching is very expensive. Nowadays, that's not really true. Modern servers can manage tens of thousands of threads without breaking a sweat. The issue is that blocking I/O yields complete control of our program to the kernel until the requested operation completes. We have no say in when we get to run again. This is problematic because it makes it very difficult to model two operations: cancellation, and selection. Imagine we wanted to implement graceful shutdown for our server. When someone hits Ctrl+C, we should stop accepting new connections but still wait for any active requests to complete, instead of killing the program abruptly. Any requests that take more than thirty seconds to finish are killed as the server exits. This poses a problem when dealing with blocking I/O. Our accept loop blocks until the next connection comes in. We can check for the Ctrl+C signal before or after a new connection comes in, but if the signal is triggered during a call to accept</code>, we have no choice but to wait until the next connection is accepted. The kernel has complete control over the execution of our program. loop { // Check before we call `accept`. if got_ctrl_c() { break; } // _But what if ctrl+c happens here?_ let (connection, _) = listener.accept().unwrap(); // This won't be checked until _after_ we accept a new connection. if got_ctrl_c() { break; } std::thread::spawn(|| /* ... */); } </code></pre> What we want is to listen for both the incoming connection and the ctrl+c signal, at the same time. Like a match</code> statement, but for I/O operations. loop { // If only... match { ctrl_c() => { break; }, Ok((connection, _)) = listener.accept() => { std::thread::spawn(|| ...); } } } </code></pre> And what about timing out long running requests after thirty seconds? We could set a flag that tells threads to stop, but how often would they check it? We again run into the problem that we lose control of our program during I/O and have no choice but to wait until it completes. There's really no good way of force cancelling a thread. Problems like these are where threads and blocking I/O fall apart. Expressing event-based logic becomes very difficult when the kernel holds so much control over the execution of our program. There are ways to accomplish this using platform specific interfaces such as Unix signal handlers</a>. While this approach is simple and can work well, signal handlers often become quite cumbersome to work with outside of simple use cases. By the end of the post you'll see another method of expressing complex control flow and decide what is better for your use case. </blockquote> But what if there was a way to perform I/O without yielding to the kernel? It turns out there is a second way to perform I/O, known as non-blocking I/O. As the name suggests, a non-blocking operation will never block the calling thread. Instead it returns immediately, returning an error if the given resource was not available. We can switch to using non-blocking I/O by putting our TCP listener and streams into non-blocking mode. let listener = TcpListener::bind("localhost:3000").unwrap(); listener.set_nonblocking(true).unwrap(); loop { let (connection, _) = listener.accept().unwrap(); connection.set_nonblocking(true).unwrap(); // ... } </code></pre> Non-blocking I/O works a little differently. If the I/O request cannot be fulfilled immediately, instead of blocking, the kernel simply returns the WouldBlock</code> error code. Despite being represented as an error, WouldBlock</code> isn't really an error condition. It just means the operation could not be performed immediately, giving us the chance to decide what to do instead of blocking. use std::io; // ... listener.set_nonblocking(true).unwrap(); loop { let connection = match listener.accept() { Ok((connection, _)) => connection, Err(e) if e.kind() == io::ErrorKind::WouldBlock => { // The operation was not performed. } Err(e) => panic!("{e}"), }; connection.set_nonblocking(true).unwrap(); // ... } </code></pre> Imagine we called accept</code> when there were no incoming connections. With blocking I/O, the accept</code> would have blocked until a new connection came in. Now instead of yielding control to the kernel, WouldBlock</code> puts the control back in our hands. Our I/O doesn't block, great! But what do we actually do when something isn't ready? WouldBlock</code> is a temporary state, meaning at some point in the future the socket should become ready to read or write from. So, technically, we could just spin until the socket becomes ready. loop { let connection = match listener.accept() { Ok((connection, _)) => connection, Err(e) if e.kind() == io::ErrorKind::WouldBlock => { continue; } Err(e) => panic!("{e}"), }; } </code></pre> But spinning is really just worse than blocking directly — when we block for I/O, the OS gives other threads a chance to run. What we really need is to build some sort of a scheduler for all of our tasks, doing what the operating system used to handle for us. Let's walk things through from the beginning again. We create a TCP listener. let listener = TcpListener::bind("localhost:3000").unwrap(); </code></pre> Set it to non-blocking mode. listener.set_nonblocking(true).unwrap(); </code></pre> And then start our main loop. The first thing we'll do is try accepting a new TCP connection. // ... loop { match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); // ... }, Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } } </code></pre> Now, we can't just continue serving that connection directly and forget about everyone else. Instead, we have to keep track of all our active connections. // ... let mut connections = Vec::new(); loop { match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); connections.push(connection); }, Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } } </code></pre> But we can't keep accepting connections forever. We don't have the luxury of OS scheduling anymore, so we need to handle running a little bit of everything in every iteration of the main loop. After trying to accept</code> once, we need to deal with the already active connections. For each connection, we have to perform whatever operation is needed to move the processing of the request forward, whether that means reading the request, or writing the response. // ... loop { // Try accepting a new connection. match listener.accept() { // ... } // Attempt to make progress on active connections. for connection in connections.iter_mut() { // ... } } </code></pre> Hmm... If you remember the handle_connection</code> function from before, fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut request = [0u8; 1024]; let mut read = 0; loop { let num_bytes = connection.read(&mut request[read..])?; // ... } let request = String::from_utf8_lossy(&request[..read]); println!("{request}"); let response = /* ... */; let mut written = 0; loop { let num_bytes = connection.write(&response[written..])?; // ... } connection.flush().unwrap(); } </code></pre> We perform three different I/O operations, read</code>, write</code>, and flush</code>. With blocking I/O we could write our code sequentially, but now we have to deal with the fact that we could face WouldBlock</code> and not be able to make progress at any point when performing I/O. That means we can't simply drop everything and move on to the next active connection, we need to keep track of its current state in order to resume from the correct point when we come back. We can represent the three possible states of handle_connection</code> in an enum. enum ConnectionState { Read, Write, Flush } </code></pre> Remember, we don't need separate states for things like converting the request to a string, we only need states for places where we might encounter WouldBlock</code>. The Read</code> and Write</code> states also need to hold on to some local state for the request/response buffers and the number of bytes that have already been read/written. These used to be local variables in our function, but now we need them to persist across iterations of our main loop. enum ConnectionState { Read { request: [u8; 1024], read: usize }, Write { response: &'static [u8], written: usize, }, Flush, } </code></pre> Connections start in the Read</code> state with an empty buffer and zero bytes read, the same variables we used to initialize at the very start of handle_connection</code>. // ... let mut connections = Vec::new(); loop { match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); let state = ConnectionState::Read { request: [0u8; 1024], read: 0, }; connections.push((connection, state)); }, Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } } </code></pre> Now we can try to drive each connection forward from its current state. // ... loop { match listener.accept() { // ... } for (connection, state) in connections.iter_mut() { if let ConnectionState::Read { request, read } = state { // ... } if let ConnectionState::Write { response, written } = state { // ... } if let ConnectionState::Flush = state { // ... } } } </code></pre> If the connection is still in the read state, we can continue reading the request just like we did before. The only difference is that when we receive WouldBlock</code>, we have to move on to the next connection. // ... 'next: for (connection, state) in connections.iter_mut() { if let ConnectionState::Read { request, read } = state { loop { // Try reading from the stream. match connection.read(&mut request[*read..]) { Ok(n) => { // Keep track of how many bytes we've read. *read += n } Err(e) if e.kind() == io::ErrorKind::WouldBlock => { // Not ready yet, move on to the next connection. continue 'next; } Err(e) => panic!("{e}"), } // Did we reach the end of the request? if request.get(*read - 4..*read) == Some(b"\r\n\r\n") { break; } } // We're done, print the request. let request = String::from_utf8_lossy(&request[..*read]); println!("{request}"); } // ... } </code></pre> We also have to deal with the case where we read zero bytes. Before we could simply return from the connection handler and the state would be cleaned up for us, but now we have to remove the connection ourselves. Because we're currently iterating through the connections list, we'll store a separate list of indices to remove after we finish. let mut completed = Vec::new(); 'next: for (i, (connection, state)) in connections.iter_mut().enumerate() { if let ConnectionState::Read { request, read } = state { loop { // Try reading from the stream. match connection.read(&mut request[*read..]) { Ok(0) => { println!("client disconnected unexpectedly"); completed.push(i); continue 'next; } Ok(n) => *read += n, // Not ready yet, move on to the next connection. Err(e) if e.kind() == io::ErrorKind::WouldBlock => continue 'next, Err(e) => panic!("{e}"), } // ... } // ... } } // Iterate in reverse order to preserve the indices. for i in completed.into_iter().rev() { connections.remove(i); } </code></pre> Once we finish reading the request, we have to transition into the Write</code> state and attempt to write the response. The control flow around writing the response is very similar to reading, transitioning to the Flush</code> state once we finish. if let ConnectionState::Read { request, read } = state { // ... // Move into the write state. let response = concat!( "HTTP/1.1 200 OK\r\n", "Content-Length: 12\n", "Connection: close\r\n\r\n", "Hello world!" ); *state = ConnectionState::Write { response: response.as_bytes(), written: 0, }; } if let ConnectionState::Write { response, written } = state { loop { match connection.write(&response[*written..]) { Ok(0) => { println!("client disconnected unexpectedly"); completed.push(i); continue 'next; } Ok(n) => { *written += n; } Err(e) if e.kind() == io::ErrorKind::WouldBlock => { // Not ready yet, move on to the next connection. continue 'next; } Err(e) => panic!("{e}"), } // Did we write the whole response yet? if *written == response.len() { break; } } // Successfully wrote the response, try flushing next. *state = ConnectionState::Flush; } </code></pre> And after we successfully flush the response, we can mark the connection as completed and have it removed from the list. if let ConnectionState::Flush = state { match connection.flush() { Ok(_) => { completed.push(i); }, Err(e) if e.kind() == io::ErrorKind::WouldBlock => { // Not ready yet, move on to the next connection. continue 'next; } Err(e) => panic!("{e}"), } } </code></pre> That's it! Here's the new high-level flow of the server: // Bind the listener. let listener = TcpListener::bind("localhost:3000").unwrap(); listener.set_nonblocking(true).unwrap(); let mut connections = Vec::new(); loop { // Try accepting a connection. match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); // Keep track of connection state. let state = ConnectionState::Read { request: Vec::new(), read: 0, }; connections.push((connection, state)); }, Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } let mut completed = Vec::new(); // Try to drive connections forward. 'next: for (i, (connection, state)) in connections.iter_mut().enumerate() { if let ConnectionState::Read { request, read } = state { // ... *state = ConnectionState::Write { response, written }; } if let ConnectionState::Write { response, written } = state { // ... *state = ConnectionState::Flush; } if let ConnectionState::Flush = state { // ... } } // Remove any connections that completed, iterating in reverse order // to preserve the indices. for i in completed.into_iter().rev() { connections.remove(i); } } </code></pre> Now that we have to manage scheduling ourselves.. things are a lot more complicated. And now for the moment of truth... $ curl localhost:3000 # => Hello world! </code></pre> It works! A Multiplexed Server</a></h2> Our server can now handle running multiple requests concurrently on a single thread. Nothing ever blocks. If some operation would have blocked, it remembers the current state and moves on to run something else, much like the kernel scheduler was doing for us. However, our new design introduces two new problems. The first problem is that everything runs on the main thread, utilizing only a single CPU core. We're doing the best we can to use that one core efficiently, but we're still only running a single thing at a time. With threads spread across multiple cores, we could be doing much more. There's a bigger problem though. Our main loop isn't actually very efficient. We're making an I/O request to the kernel for every single active connection, every single iteration of the loop, to check if it's ready. A call to read</code> or write</code>, even if it returns WouldBlock</code> and doesn't actually perform any I/O, is still a syscall. Syscalls aren't cheap. We might have 10k active connections, but only 500 of them are ready. Calling read</code> or write</code> 10k times when only 500 of them will actually make progress is a massive waste of CPU cycles. As the number of connections scales, our loop becomes less and less efficient, wasting more time doing useless work. How do we fix this? With blocking I/O the kernel was able to schedule things efficiently because it knows when resources become ready. With non-blocking I/O, we don't know without checking. But checking is expensive. What we need is an efficient way to keep track of all of our active connections, and somehow get notified when they become ready. It turns out, we aren't the first to run into this problem. Every operating system provides a solution for exactly this. On Linux, it's called epoll. epoll(7)</code> - I/O event notification facility The epoll API performs a similar task to poll(2)</code>: monitoring multiple file descriptors to see if I/O is possible on any of them. The epoll API can be used either as an edge-triggered or a level-triggered interface and scales well to large numbers of watched file descriptors. </blockquote> Sounds perfect! Let's try using it. epoll is a family of Linux system calls that let us work with a set of non-blocking sockets. It isn't terribly ergonomic to use directly, so we'll be using the epoll crate, a thin wrapper around the C interface. To start, we'll initialize an epoll instance using the create</code> function. // ```toml // [dependencies] // epoll = "4.3" // ``` fn main() { let epoll = epoll::create(false).unwrap(); } </code></pre> epoll::create</code> returns a file descriptor that represents the newly created epoll instance. You can think of it as a set of file descriptors that we can add or remove from. In Linux/Unix, everything is considered a file. Actual files on the file system, TCP sockets, and external devices are all files that you can read</code>/write</code> to. A file descriptor is an integer that represents an open "file" in the system. We'll be working with file descriptors a lot throughout the rest of the article. </blockquote> The first file descriptor we have to add is the TCP listener. We can modify the epoll set with the epoll::ctl</code> command. To add to it, we'll use the EPOLL_CTL_ADD</code> flag. use epoll::{Event, Events, ControlOptions::*}; use std::os::fd::AsRawFd; fn main() { let listener = TcpListener::bind("localhost:3000").unwrap(); listener.set_nonblocking(true).unwrap(); // Add the listener to epoll. let event = Event::new(Events::EPOLLIN, listener.as_raw_fd() as _); epoll::ctl(epoll, EPOLL_CTL_ADD, listener.as_raw_fd(), event).unwrap(); } </code></pre> We pass in the file descriptor of the resource we are registering, the TCP listener, along with an Event</code>. An event has two parts, the interest flag, and the data field. The interest flag gives us a way to tell epoll which I/O events we are interested in. In the case of the TCP listener, we want to be notified when new connections come in, so we use the EPOLLIN</code> flag. The data field lets us store an ID that will uniquely identify each resource. Remember, a file descriptor is a unique integer for a given file, so we can just use that. You'll see why this is important in the next step. Now for the main loop. This time, no spinning. Instead we can call epoll::wait</code>. epoll_wait(2)</code>- wait for an I/O event on an epoll file descriptor The epoll_wait()</code> system call waits for events on the epoll(7)</code> instance referred to by the file descriptor epfd. The buffer pointed to by events is used to return information from the ready list about file descriptors in the interest list that have some events available. A call to epoll_wait()</code> will block until either: a file descriptor delivers an event;</li> the call is interrupted by a signal handler; or</li> the timeout expires.</li> </ul> </blockquote> epoll::wait</code> is the magical part of epoll. It lets us block until any of the events we registered become ready, and tells us which ones did. Right now that's just until a new connection comes in, but soon we'll use this same call to block for read, write, and flush events that we were previously spinning for. The fact that epoll::wait</code> is "blocking" might put you off, but remember, it only blocks if there is nothing else to do, where previously we would have been spinning and making pointless syscalls. This idea of blocking on multiple operations simultaneously is known as I/O multiplexing. epoll::wait</code> accepts a list of events that it will populate with information about the file descriptors that became ready. It then returns the number of events that were added. // ... loop { let mut events = [Event::new(Events::empty(), 0); 1024]; let timeout = -1; // Block forever, until something happens. let num_events = epoll::wait(epoll, timeout, &mut events).unwrap(); for event in &events[..num_events] { // ... } } </code></pre> Each event contains the data field associated with the resource that became ready. for event in &events[..num_events] { let fd = event.data as i32; // ... } </code></pre> Remember when we used the file descriptor for the data field? We can use it to check whether the event is for the TCP listener, which means there's an incoming connection ready to accept: for event in &events[..num_events] { let fd = event.data as i32; // Is the listener ready? if fd == listener.as_raw_fd() { // Try accepting a connection. match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); // ... } Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } } } </code></pre> If the call still returns WouldBlock</code> for whatever reason, we can just move one and wait for the next event. Now we have to register the new connection in epoll, just like we did the listener. for event in &events[..num_events] { let fd = event.data as i32; // Is the listener ready? if fd == listener.as_raw_fd() { // Try accepting a connection. match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); let fd = connection.as_raw_fd(); // Register the connection with epoll. let event = Event::new(Events::EPOLLIN | Events::EPOLLOUT, fd as _); epoll::ctl(epoll, EPOLL_CTL_ADD, fd, event).unwrap(); } Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } } } </code></pre> This time we set both EPOLLIN</code> and EPOLLOUT</code>, because we are interested in both read and write events, depending on the state of the connection. Now that we register connections, we'll get events for both the TCP listener and individual connections. We need to store connections and their states in a way that we can look up by file descriptor. Instead of a list, we can use a HashMap</code>. let mut connections = HashMap::new(); loop { // ... 'next: for event in &events[..num_events] { let fd = event.data as i32; // Is the listener ready? if fd == listener.as_raw_fd() { match listener.accept() { Ok((connection, _)) => { // ... let state = ConnectionState::Read { request: [0u8; 1024], read: 0, }; connections.insert(fd, (connection, state)); } Err(e) if e.kind() == io::ErrorKind::WouldBlock => {} Err(e) => panic!("{e}"), } continue 'next; } // Otherwise, a connection must be ready. let (connection, state) = connections.get_mut(&fd).unwrap(); } } </code></pre> Once we have the ready connection and it's state, we can try to make progress the exact same way we did last time. Nothing changes about the way we read or write from the stream, the only difference is that we only ever do I/O when epoll tells us to. Before we had to check every single connection to see if something became ready, but now epoll handles that for us, so we avoid any useless syscalls. // ... // epoll told us this connection is ready. let (connection, state) = connections.get_mut(&fd).unwrap(); if let ConnectionState::Read { request, read } = state { // connection.read... *state = ConnectionState::Write { response, written }; } if let ConnectionState::Write { response, written } = state { // connection.write... *state = ConnectionState::Flush; } if let ConnectionState::Flush = state { // connection.flush... } </code></pre> Once we've finished reading, writing, and flushing the response, we remove the connection from our map and drop it, which automatically unregisters it from epoll. for fd in completed { let (connection, _state) = connections.remove(&fd).unwrap(); // Unregister from epoll. drop(connection); } </code></pre> And that's it! Here's the new high-level flow of our server: // Create an epoll instance. let epoll = epoll::create(false).unwrap(); // Bind the listener. let listener = /* ... */. // Add the listener to epoll. let event = Event::new(Events::EPOLLIN, listener.as_raw_fd() as _); epoll::ctl(epoll, EPOLL_CTL_ADD, listener.as_raw_fd(), event).unwrap(); let mut connections = HashMap::new(); loop { let mut events = [Event::new(Events::empty(), 0); 1024]; // Block until epoll wakes us up. let num_events = epoll::wait(epoll, 0, &mut events).unwrap(); let mut completed = Vec::new(); 'next: for event in &events[..num_events] { let fd = event.data as i32; // Is the listener ready? if fd == listener.as_raw_fd() { match listener.accept() { Ok((connection, _)) => { // ... // Add the connection to epoll. let event = Event::new(Events::EPOLLIN | Events::EPOLLOUT, fd as _); epoll::ctl(epoll, EPOLL_CTL_ADD, fd, event).unwrap(); // Keep track of connection state. let state = ConnectionState::Read { request: [0u8; 1024], read: 0, }; connections.insert(fd, (connection, state)); } continue 'next; } // Otherwise, a connection is ready. let (connection, state) = connections.get_mut(&fd).unwrap(); // Try to drive it forward based on its state. if let ConnectionState::Read { request, read } = state { // connection.read... *state = ConnectionState::Write { response: response.as_bytes(), written: 0, }; } if let ConnectionState::Write { response, written } = state { // connection.write... *state = ConnectionState::Flush; } if let ConnectionState::Flush = state { // connection.flush... } } for fd in completed { let (connection, _state) = connections.remove(&fd).unwrap(); // Unregister from epoll. drop(connection); } } </code></pre> And... $ curl localhost:3000 # => Hello world! </code></pre> It works! Futures</a></h2> Alright, our server can now process multiple requests concurrently on a single thread. And thanks to epoll, it's pretty efficient at doing so. But there's still a problem. We got so caught up in gaining control over the execution of our tasks, and then scheduling them efficiently ourselves, that in the process the complexity of our code has increased dramatically. What went from a simple, sequential accept loop has become a massive event loop managing multiple state machines. And it's not pretty. Making our original server multi-threaded was as simple as adding a single line of code in thread::spawn</code>. If you think about it, our server is still a set of concurrent tasks, we just manage them all messily in a giant loop. This doesn't seem very scalable. The more features we add to our program, the more complex the loop becomes, because everything is so tightly coupled together. What if we could write an abstraction like thread::spawn</code> that let us write our tasks as individual units, and handle the scheduling and event handling for all tasks in a single place, regaining some of that sequential control flow? This idea is generally referred to as asynchronous programming. </blockquote> Let's take a look at the signature of thread::spawn</code>. pub fn spawn<F, T>(f: F) -> JoinHandle<T> where F: FnOnce() -> T + Send + 'static, T: Send + 'static; </code></pre> thread::spawn</code> takes a closure. Our version of thread::spawn</code> however, could not take a closure, because we aren't the operating system and can't arbitrarily preempt code at will. We need to somehow represent a non-blocking, resumable task. // fn spawn<T: Task>(task: T); trait Task {} </code></pre> Handling a request is a task. Reading or writing from/to the connection is also a task. A task is really just a piece of code that needs to be run, representing a value that will resolve sometime in the future. Future</code>, that's a nice name isn't it. trait Future { type Output; fn run(self) -> Self::Output; } </code></pre> Hmm.. that signature doesn't really work. Having run</code> return the output directly means it must be blocking, which is what we're trying so hard to avoid. We instead want a way to attempt to drive the future forward without blocking, like we've been doing with all our state machines in the event loop. What we're really doing when we try to run a future is asking it if the value is ready yet, polling it, and giving it a chance to make progress. trait Future { type Output; fn poll(self) -> Option<Self::Output>; } </code></pre> That looks more like it. Except wait, poll</code> can't take self</code> if we want to call it multiple times, it should probably take a reference. A mutable reference, if we want to mutate the internal state of the task as it makes progress, like ConnectionState</code>. trait Future { type Output; fn poll(&mut self) -> Option<Self::Output>; } </code></pre> Alright, imagine a scheduler that runs these new futures. impl Scheduler { fn run(&mut self) { loop { for future in &mut self.tasks { future.poll(); } } } } </code></pre> That doesn't look right. After initiating the future, the scheduler should only try to call poll</code> when the given future is able to make progress, like when epoll returns an event. But how do we know when that happens? If the future represents an I/O operation, we know it's able to make progress when epoll tells us it is. The problem is the scheduler won't know which epoll event corresponds to which future, because the future handles everything internally in poll</code>. What we need is for the scheduler to pass each future an ID, so that the future can register any I/O resources with epoll using the same ID, instead of their file descriptors. That way the scheduler has a way of mapping epoll events to runnable futures. impl Scheduler { fn spawn<T>(&self, mut future: T) { let id = rand(); // Poll the future once to get it started, passing in it's ID. future.poll(event.id); // Store the future. self.tasks.insert(id, future); } fn run(self) { // ... for event in epoll_events { // Poll the future associated with this event. let future = self.tasks.get_mut(&event.id).unwrap(); future.poll(event.id); } } } </code></pre> You know, it would be nice if there was a more generic way to tell the scheduler about progress than tying every future to epoll. We might have different types of futures that make progress in other ways, like a timer running on a background thread, or a channel that needs to notify tasks that a message is available. What if we gave the futures themselves more control? Instead of just an ID, what if we give every future a way to wake itself up, notifying the scheduler that it's ready to make progress? A simple callback should do the trick. #[derive(Clone)] struct Waker(Arc<dyn Fn() + Send + Sync>); impl Waker { fn wake(&self) { (self.0)() } } trait Future { type Output; fn poll(&mut self, waker: Waker) -> Option<Self::Output>; } </code></pre> The scheduler can provide each future a callback that, when called, updates the scheduler's state for that future, marking it as ready. That way our scheduler is completely disconnected from epoll, or any other individual notification system. Waker</code> is thread-safe, allowing us to use background threads to wake futures. Right now all of our tasks are connected to epoll anyways, but this will come in handy later. A Reactor</a></h2> Consider a future that reads from a TCP connection. It receives a Waker</code> that needs to be called when epoll returns the relevant EPOLLIN</code> event, but the future won't be running when that happens, it will be idle in the scheduler's queue. The future can't wake itself up, so who does? All I/O futures need a way to give their wakers to epoll. In fact, they need more than that, they need some sort of background service that drives epoll, so we can register wakers with it. This service is commonly known as a reactor. The reactor is a simple object holding the epoll descriptor and a map of tasks keyed by file descriptor, just like we had before. The difference is that instead of the map holding the TCP connections themselves, it holds the wakers. thread_local! { static REACTOR: Reactor = Reactor::new(); } struct Reactor { epoll: RawFd, tasks: RefCell<HashMap<RawFd, Waker>>, } impl Reactor { pub fn new() -> Reactor { Reactor { epoll: epoll::create(false).unwrap(), tasks: RefCell::new(HashMap::new()), } } } </code></pre> To keep things simple, the reactor is a thread-local object, mutated through a RefCell</code>. This is important because the reactor will be modified and accessed by different tasks throughout the program. The reactor needs to support a couple simple operations. Adding a task, impl Reactor { // Add a file descriptor with read and write interest. // // `waker` will be called when the descriptor becomes ready. pub fn add(&self, fd: RawFd, waker: Waker) { let event = epoll::Event::new(Events::EPOLLIN | Events::EPOLLOUT, fd as u64); epoll::ctl(self.epoll, EPOLL_CTL_ADD, fd, event).unwrap(); self.tasks.borrow_mut().insert(fd, waker); } } </code></pre> Removing a task, impl Reactor { // Remove the given descriptor from epoll. // // It will no longer receive any notifications. pub fn remove(&self, fd: RawFd) { self.tasks.borrow_mut().remove(&fd); } } </code></pre> And driving epoll. We'll be running the reactor in a loop, just like we were running epoll in a loop before. It works exactly the same way, except all the reactor has to do is wake the associated future for every event. Remember, this will trigger the scheduler to run the future later, and continue the event loop. impl Reactor { // Drive tasks forward, blocking forever until an event arrives. pub fn wait(&self) { let mut events = [Event::new(Events::empty(), 0); 1024]; let timeout = -1; // Forever. let num_events = epoll::wait(self.epoll, timeout, &mut events).unwrap(); for event in &events[..num_events] { let fd = event.data as i32; // Wake the task. if let Some(waker) = self.tasks.borrow().get(&fd) { waker.wake(); } } } } </code></pre> Great, now we have a simple reactor interface. But all of this is still a little abstract. What does it really mean to call wake</code>? Scheduling Tasks</a></h2> We have a reactor, now we need a scheduler to run our tasks. One thing to keep in mind is that because wakers are Send</code>, the scheduler must be global and thread-safe, meaning wake</code> may be called concurrently from other threads. static SCHEDULER: Scheduler = Scheduler { /* ... */ }; #[derive(Default)] struct Scheduler { // ... } </code></pre> We want to be able to spawn tasks onto our scheduler, just like we could spawn threads. For now, we'll only handle spawning tasks that don't return anything, to avoid having to implement a version of JoinHandle</code>. To start, we'll probably need some sort of list of tasks to run, guarded by a Mutex</code> to be thread-safe. struct Scheduler { tasks: Mutex<Vec<Box<dyn Future + Send>>> } impl Scheduler { pub fn spawn(&self, task: impl Future<Output = ()> + Send + 'static) { self.tasks.lock().unwrap().push(Box::new(task)); } pub fn run(&self) { for task in tasks.lock().unwrap().borrow_mut().iter_mut() { // ... } } } </code></pre> Remember, futures are only polled when they are able to make progress. They should always be able to make progress at the start, but after the initial poll, we don't touch them until someone calls wake</code>. There are a couple of ways we could go about this. We could just store a HashMap</code> of tasks with a status flag that indicates whether or not the task was woken, but that means we would have to iterate through the entire map to find out which tasks are runnable. While this isn't incredibly expensive, there is a better way. Instead of storing every spawned task in the map, we'll only store runnable tasks in a queue. use std::collections::VecDeque; type SharedTask = Arc<Mutex<dyn Future<Output = ()> + Send>>; #[derive(Default)] struct Scheduler { runnable: Mutex<VecDeque<SharedTask>>, } </code></pre> The types will make sense soon. When a task is spawned, it's pushed onto the back of the queue: impl Scheduler { pub fn spawn(&self, task: impl Future<Output = ()> + Send + 'static) { self.runnable.lock().unwrap().push_back(Arc::new(Mutex::new(task))); } } </code></pre> The scheduler pops tasks off the queue one by one and calls poll</code>: impl Scheduler { fn run(&self) { loop { // Pop a runnable task off the queue. let task = self.runnable.lock().unwrap().pop_front(); if let Some(task) = task { // And poll it. task.try_lock().unwrap().poll(waker); } } } </code></pre> Notice that we don't even really need a Mutex</code> around the task because it's only going to be accessed by the main thread, but removing it would require unsafe</code>. We'll settle with try_lock().unwrap()</code> for now. </blockquote> Now for the important bit, the waker. The beautiful part of our run queue is that when a task is woken, it's simply pushed back onto the queue. impl Scheduler { fn run(&self) { loop { // Pop a runnable task off the queue. let task = self.runnable.lock().unwrap().pop_front(); if let Some(task) = task { let t2 = task.clone(); // Create a waker that pushes the task back on. let wake = Arc::new(move || { SCHEDULER.runnable.lock().unwrap().push_back(t2.clone()); }); // Poll the task. task.try_lock().unwrap().poll(Waker(wake)); } } } } </code></pre> This is why the task needed to be reference counted — it's not owned by the scheduler, it's referenced by the queue, as well as wherever the waker is being stored. In fact the same task might be on the queue multiple times at once, and the waker might be cloned all over the place. Once we've dealt with all runnable tasks, we need to block on the reactor until another task becomes ready ^{5</a>. Once a task becomes ready, the reactor will call wake</code> and push the future back onto our queue for us to run it again.} pub fn run(&self) { loop { loop { // Pop a runnable task off the queue. let Some(task) = self.runnable.lock().unwrap().pop_front() else { break }; let t2 = task.clone(); // Create a waker that pushes the task back on. let wake = Arc::new(move || { SCHEDULER.runnable.lock().unwrap().push_back(t2.clone()); }); // Poll the task. task.lock().unwrap().poll(Waker(wake)); } // If there are no runnable tasks, block on epoll until something becomes ready. REACTOR.with(|reactor| reactor.wait()); } } </code></pre> Perfect. ...ignoring the Arc<Mutex<T>></code> clutter. Alright! Together, the scheduler and reactor form a runtime for our futures. The scheduler keeps tracks of which tasks are runnable and polls them, and the reactor marks tasks as runnable when epoll tells us something they are interested in becomes ready. trait Future { type Output; fn poll(&mut self, waker: Waker) -> Option<Self::Output>; } static SCHEDULER: Scheduler = Scheduler { /* ... */ }; // The scheduler. #[derive(Default)] struct Scheduler { runnable: Mutex<VecDeque<SharedTask>>, } type SharedTask = Arc<Mutex<dyn Future<Output = ()> + Send>>; impl Scheduler { pub fn spawn(&self, task: impl Future<Output = ()> + Send + 'static); pub fn run(&self); } thread_local! { static REACTOR: Reactor = Reactor::new(); } // The reactor. struct Reactor { epoll: RawFd, tasks: RefCell<HashMap<RawFd, Waker>>, } impl Reactor { pub fn new() -> Reactor; pub fn add(&self, fd: RawFd, waker: Waker); pub fn remove(&self, fd: RawFd); pub fn wait(&self); } </code></pre> We've written the runtime, now let's try to use it! An Async Server</a></h2> It's time to actually write the tasks that our scheduler is going to run. Like before, we'll use enums as state machines to manage the different states of our program. The difference is that this time, each task will manage it's own state independent from other tasks, instead of having the entire program revolve around a messy event loop. To start everything off, we need to write the main task. This task will be pushed on and off the scheduler's run queue for the entirety of our program. fn main() { SCHEDULER.spawn(Main::Start); SCHEDULER.run(); } enum Main { Start, } impl Future for Main { type Output = (); fn poll(&mut self, waker: Waker) -> Option<()> { // ... } } </code></pre> Our task starts off just like before, creating the TCP listener and setting it to non-blocking mode. // impl Future for Main { fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { let listener = TcpListener::bind("localhost:3000").unwrap(); listener.set_nonblocking(true).unwrap(); } None } </code></pre> Now we need to register the listener with epoll. We can do that using our new Reactor</code>. // impl Future for Main { fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { // ... REACTOR.with(|reactor| { reactor.add(listener.as_raw_fd(), waker); }); } } </code></pre> Notice how we give the reactor the waker provided to us by the scheduler. When a connection comes, epoll will return an event and the Reactor</code> will wake the task, causing the scheduler to push our task back onto the queue and poll</code> us again. The waker keeps everything connected. We now need a second state for the next time we're run, Accept</code>. The main task will stay in the Accept</code> state for the rest of the program, attempting to accept new connections. enum Main { Start, Accept { listener: TcpListener }, } fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { // ... *self = Main::Accept { listener }; } if let Main::Accept { listener } = self { match listener.accept() { Ok((connection, _)) => { // ... } Err(e) if e.kind() == io::ErrorKind::WouldBlock => { return None; } Err(e) => panic!("{e}"), } } None } </code></pre> If the listener is not ready, we can simply return None</code>. Remember, this tells the scheduler the future is not yet ready, and it will be rescheduled once the reactor wakes us. If we do accept a new connection, we need to again set it to non-blocking mode. fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { // ... } if let Main::Accept { listener } = self { match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); } Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } } None } </code></pre> And now we need to spawn a new task to handle the request. fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { // ... } if let Main::Accept { listener } = self { match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); SCHEDULER.spawn(Handler { connection, state: HandlerState::Start, }); } Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } } } </code></pre> The handler task looks similar to before, but now it manages the connection itself along with its current state, which is identical to ConnectionState</code> from earlier. struct Handler { connection: TcpStream, state: HandlerState, } enum HandlerState { Start, Read { request: [u8; 1024], read: usize, }, Write { response: &'static [u8], written: usize, }, Flush, } </code></pre> The handler task starts by registering its connection with the reactor to be notified when the connection is ready to read/write to. It also gives the reactor its waker, so that the scheduler knows when to run it again. impl Future for Handler { type Output = (); fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let HandlerState::Start = self.state { // Start by registering our connection for notifications. REACTOR.with(|reactor| { reactor.add(self.connection.as_raw_fd(), waker); }); self.state = HandlerState::Read { request: [0u8; 1024], read: 0, }; } // ... } } </code></pre> The Read</code>, Write</code>, and Flush</code> states work exactly the same as before. When we encounter WouldBlock</code>, we can simply return None</code>, knowing that we'll be run again once our future is woken. // impl Future for Handler { fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let HandlerState::Start = self.state { // ... } // Read the request if let HandlerState::Read { request, read } = &mut self.state { loop { match self.connection.read(&mut request[*read..]) { Ok(0) => { println!("client disconnected unexpectedly"); return Some(()); } Ok(n) => *read += n, Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } // Did we reach the end of the request? let read = *read; if read >= 4 && &request[read - 4..read] == b"\r\n\r\n" { break; } } // We're done, print the request. let request = String::from_utf8_lossy(&request[..*read]); println!("{}", request); // And move into the write state. let response = /* ... */; self.state = HandlerState::Write { response: response.as_bytes(), written: 0, }; } // Write the response. if let HandlerState::Write { response, written } = &mut self.state { // self.connection.write... // Successfully wrote the response, try flushing next. self.state = HandlerState::Flush; } // Flush the response. if let HandlerState::Flush = self.state { match self.connection.flush() { Ok(_) => {} Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } } } </code></pre> Notice how much nicer things are when tasks are independent, encapsulated objects? At the end of the task's lifecycle, it removes its connection from the reactor and returns Some</code>. It will never be run again after that point. fn poll(&mut self, waker: Waker) -> Option<Self::Output> { // ... REACTOR.with(|reactor| { reactor.remove(self.connection.as_raw_fd()); }); Some(()) } </code></pre> Perfect! Our new server looks a lot nicer. Individual tasks are completely independent of each other, and we can spawn new tasks just like threads. fn main() { SCHEDULER.spawn(Main::Start); SCHEDULER.run(); } // Main task: Accept loop. enum Main { Start, Accept { listener: TcpListener }, } impl Future for Main { type Output = (); fn poll(&mut self, waker: Waker) -> Option<()> { if let Main::Start = self { // ... REACTOR.with(|reactor| { reactor.add(listener.as_raw_fd(), waker); }); *self = Main::Accept { listener }; } if let Main::Accept { listener } = self { match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(Handler { connection, state: HandlerState::Start, }); } Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } } None } } // Handler task: Handles every connection. struct Handler { connection: TcpStream, state: HandlerState, } enum HandlerState { Start, Read { request: [u8; 1024], read: usize, }, Write { response: &'static [u8], written: usize, }, Flush, } impl Future for Handler { type Output = (); fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let HandlerState::Start = self.state { REACTOR.with(|reactor| { reactor.add(self.connection.as_raw_fd(), waker); }); self.state = HandlerState::Read { /* .. */ }; } if let HandlerState::Read { request, read } = &mut self.state { // ... self.state = HandlerState::Write { /* .. */ }; } if let HandlerState::Write { response, written } = &mut self.state { // ... self.state = HandlerState::Flush; } if let HandlerState::Flush = self.state { // ... } REACTOR.with(|reactor| { reactor.remove(self.connection.as_raw_fd()); }); Some(()) } } </code></pre> And.... $ curl localhost:3000 # => Hello world! </code></pre> It works! A Functional Server</a></h2> With this new future abstraction, our server is much nicer than before. Futures get to manage their state independently, the scheduler gets to run tasks without worrying about epoll, and tasks can be spawned and woken without worrying about any of the lower level details of the scheduler. It really is a much nicer programming model. It is nice that tasks are encapsulated, but we still have to write everything in a state-machine like way. Granted, Rust makes this pretty easy to do with enums, but could we do better? Looking at the two futures we've written, they have a lot in common. Each future has a number of states. At each state, some code is run. If that code completes successfully, we transition into the next state. If it encounters WouldBlock</code>, we return None</code>, indicating that the future is not yet ready. This seems like something we can abstract over. What we need is a way to create a future from some block of code, and a way to combine two futures, chaining them together. Given a block of code, we need to be able to construct a future... sound like a job for a closure? fn future_fn(f: F) -> impl Future where F: Fn(), { // ... } </code></pre> The closure probably also needs to mutate local state. fn future_fn(f: F) -> impl Future where F: FnMut(), { // ... } </code></pre> And it also needs access to the waker. fn future_fn(f: F) -> impl Future where F: FnMut(Waker), { // ... } </code></pre> And.. it needs to return a value. An optional value, in case it's not ready yet. In fact, we can just copy the signature of poll</code>, because that's really what this closure is. fn poll_fn<F, T>(f: F) -> impl Future<Output = T> where F: FnMut(Waker) -> Option<T>, { // ... } </code></pre> Implementing poll_fn</code> doesn't seem too hard, we just need a wrapper struct that implements Future</code> and delegates poll</code> to the closure. fn poll_fn<F, T>(f: F) -> impl Future<Output = T> where F: FnMut(Waker) -> Option<T>, { struct FromFn<F>(F); impl<F, T> Future for FromFn<F> where F: FnMut(Waker) -> Option<T>, { type Output = T; fn poll(&mut self, waker: Waker) -> Option<Self::Output> { (self.0)(waker) } } FromFn(f) } </code></pre> Alright. Let's try reworking the main task to use the new poll_fn</code> helper. We can easily stick the code of the Main::Start</code> state into a closure. fn main() { SCHEDULER.spawn(listen()); SCHEDULER.run(); } fn listen() -> impl Future<Output = ()> { let start = poll_fn(|waker| { let listener = TcpListener::bind("localhost:3000").unwrap(); listener.set_nonblocking(true).unwrap(); REACTOR.with(|reactor| { reactor.add(listener.as_raw_fd(), waker); }); Some(listener) }); // ... } </code></pre> Remember, Main::Start</code> never waits on any I/O, so it's immediately ready with the listener. We can also use poll_fn</code> to write the Main::Accept</code> future. fn listen() -> impl Future<Output = ()> { let start = poll_fn(|waker| { // ... Some(listener) }); let accept = poll_fn(|_| match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); SCHEDULER.spawn(Handler { connection, state: HandlerState::Start, }); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None, Err(e) => panic!("{e}"), }); } </code></pre> On the other hand, accept</code> always returns None</code> because we want it to be called every time a new connection comes in. It runs for the entirety of our program. We have our two task states, now we need a way to chain them together. fn chain<F1, F2>(future1: F1, future2: F2) -> impl Future<Output = F2::Output> where F1: Future, F2: Future { // ... } </code></pre> Hmm, that doesn't really work. The second future will need to access the output of the first, the TCP listener. Instead of chaining the second future directly, we have to chain a closure over the first future's output. That way the closure can use the output of the first future to construct the second. fn chain<T1, F, T2>(future1: T1, chain: F) -> impl Future<Output = T2::Output> where T1: Future, F: FnOnce(T1::Output) -> T2, T2: Future { // ... } </code></pre> That seems better. We might as well be fancy and have chain</code> be a method on the Future</code> trait. That way we can call .chain</code> as a postfix method on any future. trait Future { // ... fn chain<F, T>(self, chain: F) -> Chain<Self, F, T> where F: FnOnce(Self::Output) -> T, T: Future, Self: Sized, { // ... } } enum Chain<T1, F, T2> { // ... } </code></pre> That looks right, let's try implementing it! The Chain</code> future is a generalization of our state machines, and so is itself a simple state machine. It starts off by polling the first future, holding onto the transition closure for when it finishes. enum Chain<T1, F, T2> { First { future1: T1, transition: F }, } impl<T1, F, T2> Future for Chain<T1, F, T2> where T1: Future, F: FnOnce(T1::Output) -> T2, T2: Future, { type Output = T2::Output; fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let Chain::First { future1, transition } = self { // Poll the first future. match future1.poll(waker) { // ... } } } } </code></pre> Once the first future is finished, it constructs the second future using the transition</code> closure, and starts polling it: enum Chain<T1, F, T2> { First { future1: T1, transition: F }, Second { future2: T2 }, } impl<T1, F, T2> Future for Chain<T1, F, T2> where T1: Future, F: FnOnce(T1::Output) -> T2, T2: Future, { type Output = T2::Output; fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let Chain::First { future1, transition } = self { // Poll the first future. match future1.poll(waker.clone()) { Some(value) => { // The first future is done, transition into the second. let future2 = (transition)(value); *self = Chain::Second { future2 }; } // The first future is not ready, return. None => return None, } } if let Chain::Second { future2 } = self { // The first future is already done, poll the second. return future2.poll(waker); } None } } </code></pre> Notice how the same waker</code> is used to poll both futures. This means that notifications for both futures will be propagated to the Chain</code> parent future, depending on its state. Hmm... that doesn't actually seem to work: error[E0507]: cannot move out of `*transition` which is behind a mutable reference --> src/main.rs:182:33 | 182 | let future2 = (transition)(value); | ^^^^^^^^^^^^ move occurs because `*transition` has type `F`, which does not implement the `Copy` trait </code></pre> Oh right, transition</code> is an FnOnce</code> closure, meaning it is consumed the first time it is called. We only ever call it once based on our state machine, but the compiler doesn't know that. We can wrap it in an Option</code> and use take</code> to call it, replacing it with None</code> and allowing us to get an owned value. This is a common pattern when working with state machines. enum Chain<T1, F, T2> { First { future1: T1, transition: Option<F> }, Second { future2: T2 }, } // ... fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let Chain::First { future1, transition } = self { match future1.poll(waker.clone()) { Some(value) => { let future2 = (transition.take().unwrap())(value); *self = Chain::future2 { future2 }; } None => return None, } } // ... } </code></pre> Perfect. Now the chain</code> method can simply construct our Chain</code> future in it's starting state. trait Future { // ... fn chain<F, T>(self, transition: F) -> Chain<Self, F, T> where F: FnOnce(Self::Output) -> T, T: Future, Self: Sized, { Chain::First { future1: self, transition: Some(transition), } } } </code></pre> Alright. Where were we... oh right, the main future! fn listen() -> impl Future<Output = ()> { let start = poll_fn(|waker| { // ... Some(listener) }); let accept = poll_fn(|_| match listener.accept() { // ... }); } </code></pre> We can combine the two futures using our new chain</code> method: fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { // ... Some(listener) }) .chain(|listener| { poll_fn(move |_| match listener.accept() { // ... }) }) } </code></pre> Huh, that seems really nice! fn main() { SCHEDULER.spawn(listen()); SCHEDULER.run(); } fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { let listener = TcpListener::bind("localhost:3000").unwrap(); // ... REACTOR.with(|reactor| { reactor.add(listener.as_raw_fd(), waker); }); Some(listener) }) .chain(|listener| { poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(Handler { connection, state: HandlerState::Start, }); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None, Err(e) => panic!("{e}"), }) }) } </code></pre> Gone is our manual state machine, our listener can now be expressed in terms of simple closures! We can convert the connection handler to a closure-based future just like we did with the main one. To start we'll separate it out into a function that returns a Future</code>. fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { // ... }) .chain(|listener| { poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(handle(connection)); None } // ... }) }) } fn handle(connection: TcpStream) -> impl Future<Output = ()> { // ... } </code></pre> The first state, HandlerState::Start</code>, is a simple poll_fn</code> closure that registers the connection with the reactor and immediately returns. fn handle(connection: TcpStream) -> impl Future<Output = ()> { poll_fn(move |waker| { REACTOR.with(|reactor| { reactor.add(connection.as_raw_fd(), waker); }); Some(()) }) } </code></pre> The second state, HandlerState::Read</code>, can be chained on quite easily. It initializes its local request state on the stack and moves it into the future, allowing the future to own its state. fn handle(mut connection: TcpStream) -> impl Future<Output = ()> { poll_fn(move |waker| { // ... }) .chain(move |_| { let mut read = 0; let mut request = [0u8; 1024]; poll_fn(move |_| { loop { // Try reading from the stream. match connection.read(&mut request[read..]) { Ok(0) => { println!("client disconnected unexpectedly"); return Some(()); } Ok(n) => read += n, Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } // Did we reach the end of the request? let read = read; if read >= 4 && &request[read - 4..read] == b"\r\n\r\n" { break; } } // We're done, print the request. let request = String::from_utf8_lossy(&request[..read]); println!("{request}"); Some(()) }) }) } </code></pre> HandlerState::Write</code> and HandlerState::Flush</code> can be chained on the same way. fn handle(connection: TcpStream) -> impl Future<Output = ()> { poll_fn(move |waker| { // `REACTOR.register`... }) .chain(move |_| { // `connection.read`... }) .chain(move |_| { let response = /* ... */; let mut written = 0; poll_fn(move |_| { loop { match connection.write(response[written..].as_bytes()) { Ok(0) => { println!("client disconnected unexpectedly"); return Some(()); } Ok(n) => written += n, Err(e) if e.kind() == io::ErrorKind::WouldBlock => return None, Err(e) => panic!("{e}"), } // Did we write the whole response yet? if written == response.len() { break; } } Some(()) }) }) .chain(move |_| { poll_fn(move |_| { match connection.flush() { Ok(_) => {} Err(e) if e.kind() == io::ErrorKind::WouldBlock => { return None; } Err(e) => panic!("{e}"), }; REACTOR.with(|reactor| reactor.remove(connection.as_raw_fd()); Some(()) }) }) } </code></pre> Hmm.... error[E0382]: use of moved value: `connection` --> src/main.rs:59:12 | 51 | fn handle(mut connection: TcpStream) -> impl Future<Output = ()> { | -------------- move occurs because `connection` has type `TcpStream`, which does not implement the `Copy` trait 52 | poll_fn(move |waker| { | ------------- value moved into closure here 53 | REACTOR.with(|reactor| { 54 | reactor.add(connection.as_raw_fd(), waker); | ---------- variable moved due to use in closure ... 59 | .chain(move |_| { | ^^^^^^^^ value used here after move ... 66 | match connection.read(&mut request[read..]) { | ---------- use occurs due to use in closure error[E0382]: use of moved value: `connection` // ... </code></pre> All of our futures use move</code> closures, meaning they take ownership of the connection, but there can only be one owner of the connection. Guess they shouldn't be move</code> closures? error[E0373]: closure may outlive the current function, but it borrows `connection`, which is owned by the current function --> src/main.rs:52:13 | 52 | poll_fn(|waker| { | ^^^^^^^^ may outlive borrowed value `connection` 53 | REACTOR.with(|reactor| { 54 | reactor.add(connection.as_raw_fd(), waker); | ---------- `connection` is borrowed here | note: closure is returned here --> src/main.rs:52:5 | 52 | / poll_fn(|waker| { 53 | | REACTOR.with(|reactor| { 54 | | reactor.add(connection.as_raw_fd(), waker); 55 | | }); ... | 128 | | }) 129 | | }) | |______^ help: to force the closure to take ownership of `connection` (and any other referenced variables), use the `move` keyword | 52 | poll_fn(move |waker| { | ++++ </code></pre> That doesn't seem to work either. The connection</code> needs to live somewhere. What if we only move it into the first future, and have the rest of the futures borrow it? error[E0382]: use of moved value: `connection` --> src/main.rs:59:12 | 51 | fn handle(mut connection: TcpStream) -> impl Future<Output = ()> { | -------------- move occurs because `connection` has type `TcpStream`, which does not implement the `Copy` trait 52 | poll_fn(move |waker| { | ------------- value moved into closure here 53 | REACTOR.with(|reactor| { 54 | reactor.add(connection.as_raw_fd(), waker); | ---------- variable moved due to use in closure ... 59 | .chain(|_| { | ^^^ value used here after move ... 66 | match connection.read(&mut request[read..]) { | ---------- use occurs due to use in closure </code></pre> Nope, that doesn't work either. Under the hood, our chained futures look something like this. The first future owns the connection, and the rest borrow from it. enum Handle { Start { connection: TcpStream, } Read { connection: &'??? TcpStream } } </code></pre> Which of course, doesn't make much sense. Once the state transitions into Read</code>, the connection from Start</code> is dropped, and we have nothing to reference. So how did this work when we were writing futures manually? struct Handler { connection: TcpStream, state: HandlerState, } enum HandlerState { /* ... */ } </code></pre> Right, the connection lived in the outer struct. Maybe we can write another one of those future helpers that allows us to reference some data stored in an outer future? Something like this: struct WithData<D, F> { data: D, future: F, } </code></pre> Seems simple enough. We should be able to construct the future such that it can capture a reference to the data. We can use a closure, just like we did with chain</code>: impl<D, F> WithData<D, F> { pub fn new(data: D, construct: impl Fn(&D) -> F) -> WithData<D, F> { let future = construct(&data); WithData { data, future } } } </code></pre> WithData</code> can implement Future</code> by simply delegating to the inner future: impl<D, F> Future for WithData<D, F> where F: Future, { type Output = F::Output; fn poll(&mut self, waker: Waker) -> Option<Self::Output> { self.future.poll(waker) } } </code></pre> Now we should be able to wrap our future in WithData</code>, giving connection</code> a place to live even after it is returned.. and everything should work! fn handle(connection: TcpStream) -> impl Future<Output = ()> { WithData::new(connection, |connection| { from_fn(move |waker| { // ... }) .chain(move |_| { let mut request = [0u8; 1024]; let mut read = 0; from_fn(move |_| { // ... }) }) .chain(move |_| { let response = /* ... */; let mut written = 0; from_fn(move |_| { // ... }) }) .chain(move |_| { from_fn(move |_| { // ... }) }) }) } </code></pre> A little funky, but if it works... error: lifetime may not live long enough --> src/main.rs:53:9 | 52 | WithData::new(connection, |connection| { | ----------- return type of closure `Chain<Chain<Chain<impl Future<Output = ()>, [closure@src/bin/play.rs:60:16: 86:10], impl Future<Output = ()>>, [closure@src/bin/play.rs:87:16: 113:10], impl Future<Output = ()>>, [closure@src/bin/play.rs:114:16: 130:10], impl Future<Output = ()>>` contains a lifetime `'2` | | | has type `&'1 TcpStream` 53 | / from_fn(move |waker| { 54 | | REACTOR.with(|reactor| { 55 | | reactor.add(connection.as_raw_fd(), waker); 56 | | }); ... | 129 | | }) 130 | | }) | |__________^ returning this value requires that `'1` must outlive `'2` </code></pre> It couldn't be that easy could it. What a weird error message too. return type of closure `Chain<Chain<Chain<impl Future<Output = ()>, [closure @src/bin/play.rs:60:16: 86:10], impl Future<Output = ()>>, [closure@src/bin/ play.rs:87:16: 113: 10], impl Future<Output = ()>>, [closure@src/bin/play.rs :114:16: 130:10], impl Future<Output = ()>>` contains a lifetime `'2` </code></pre> </blockquote> Alright, so, the giant chained future we pass to WithData</code> contains a reference to the connection. That's what we want, for the future to borrow the connection, right? `connection` has type `&'1 TcpStream` ... returning this value requires that `'1` must outlive `'2` </code></pre> </blockquote> Nowhere in our WithData</code> struct did we actually specify that the future borrows from the data. It seems the compiler can't figure out the lifetimes, so we should probably add a lifetime to WithData</code>, right? struct WithData<'data, D, F> { data: D, future: F, } </code></pre> error[E0392]: parameter `'data` is never used --> src/bin/play.rs:160:17 | 160 | struct WithData<'data, D, F> { | ^^^^^ unused parameter | = help: consider removing `'data`, referring to it in a field, or using a marker such as `PhantomData` </code></pre> Adding PhantomData</code> seems like an easy fix. struct WithData<'data, D, F> { data: D, future: F, _data: PhantomData<&'data D>, } </code></pre> The future does reference &'data D</code>, so that sort of makes sense. Now in the constructor, we should say that the future borrows from the data: impl<'data, D, F> WithData<'data, D, F> where F: Future + 'data, { pub fn new( data: D, construct: impl Fn(&'data D) -> F, ) -> WithData<'data, D, F> { let future = construct(&data); WithData { data, future, _data: PhantomData, } } } </code></pre> And that should work, right? All the lifetimes are written out and make sense: error[E0597]: `data` does not live long enough --> src/bin/play.rs:172:30 | 167 | impl<'data, D, F> WithData<'data, D, F> | ----- lifetime `'data` defined here ... 172 | let future = construct(&data); | ----------^^^^^- | | | | | borrowed value does not live long enough | argument requires that `data` is borrowed for `'data` ... 178 | } | - `data` dropped here while still borrowed error[E0505]: cannot move out of `data` because it is borrowed --> src/bin/play.rs:174:13 | 167 | impl<'data, D, F> WithData<'data, D, F> | ----- lifetime `'data` defined here ... 172 | let future = construct(&data); | ---------------- | | | | | borrow of `data` occurs here | argument requires that `data` is borrowed for `'data` 173 | WithData { 174 | data, | ^^^^ move out of `data` occurs here </code></pre> Why doesn't this work? `data` dropped here while still borrowed </code></pre> </blockquote> Wait a minute, that's the same error message we got when we removed move</code> from our future closures?! But the data does have a place to live now... doesn't it? Hmm.. actually, the second error is telling us that moving data</code> is wrong too: cannot move out of `data` because it is borrowed </code></pre> </blockquote> That... actually makes sense. The future we construct borrows the data</code> that lives on the stack. Once we move it, it's no longer in the same place on the stack. Its address changes, so the future's reference to the data is actually invalidated. ## Before Moving ┌─────────────┐ │0101001010010│ data: │001... │ ◄──────── &future.data │ │ │ │ └─────────────┘ ## After Moving ??? ◄──────── &future.data ┌─────────────┐ │0101001010010│ data: │001... │ │ │ │ │ └─────────────┘ </code></pre> We gave the data a place to live, but we didn't give it a stable place to live. It turns out, this is a well-known problem in Rust. What we're trying to create is called a self-referential struct, and it's not possible to do safely. Back when our entire future state was in the Handler</code> struct, there was no self-referencing going on. Everything just worked off the Handler</code>. But now that we're trying to split our futures up into subtasks, we need a way for them to access the data independently. So is it not possible? Well... We could allocate the data with Rc</code> and clone the reference-counted pointer into each of the futures. That way the futures get a stable pointer to the data on the heap, and it's only deallocated after all the futures complete. The code is about to get pretty ugly. fn handle(connection: TcpStream) -> impl Future<Output = ()> { let connection = Rc::new(connection); let read_connection_ref = connection.clone(); let write_connection_ref = connection.clone(); let flush_connection_ref = connection.clone(); poll_fn(move |waker| { // ... }) .chain(move |_| { // ... poll_fn(move |_| { let connection = &*read_connection_ref; loop { match (&mut connection).read(&mut request) { // ... } } // ... }) }) .chain(move |_| { // ... poll_fn(move |_| { let connection = &*write_connection_ref; // ... }) }) .chain(move |_| { poll_fn(move |_| { let connection = &*flush_connection_ref; // ... }) }) } </code></pre> Oh no.. error[E0277]: `Rc<TcpStream>` cannot be sent between threads safely --> src/main.rs:90:37 | 90 | SCHEDULER.spawn(handle(connection)); | ----- ^^^^^^^^^^^^^^^^^^ `Rc<TcpStream>` cannot be sent between threads safely | | | required by a bound introduced by this call ... 100 | fn handle(mut connection: TcpStream) -> impl Future<Output = ()> { | ------------------------ within this `impl Future<Output = ()>` </code></pre> Using Rc</code> in our handler makes it !Send</code>. Even though the connection is only used internally within the future, and futures are only ever run by the main thread, we need to use an Arc</code> to satisfy the compiler. fn handle(connection: TcpStream) -> impl Future<Output = ()> { let connection = Arc::new(connection); // ... } </code></pre> A little sad, but at least it compiles. Our server has no more manual state machines and is looking pretty clean. A lot cleaner than when we started with epoll manually, even with the messy Arc</code> business. fn main() { SCHEDULER.spawn(listen()); SCHEDULER.run(); } fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { let listener = TcpListener::bind("localhost:3000").unwrap(); // ... REACTOR.with(|reactor| { reactor.add(listener.as_raw_fd(), waker); }); Some(listener) }) .chain(|listener| { poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(handle(connection)); None } // ... }) }) } fn handle(connection: TcpStream) -> impl Future<Output = ()> { let connection = Arc::new(connection); let read_connection_ref = connection.clone(); let write_connection_ref = connection.clone(); let flush_connection_ref = connection.clone(); poll_fn(move |waker| { REACTOR.with(|reactor| { reactor.add(connection.as_raw_fd(), waker); }); Some(()) }) .chain(move |_| { let mut request = [0u8; 1024]; let mut read = 0; poll_fn(move |_| { // ... }) }) .chain(move |_| { let response = /* ... */; let mut written = 0; poll_fn(move |_| { // ... }) }) .chain(move |_| { poll_fn(move |_| { // ... REACTOR.with(|reactor| { reactor.remove(flush_connection_ref.as_raw_fd()); }); Some(()) }) }) } </code></pre> And of course... $ curl localhost:3000 # => Hello world! </code></pre> It works! A Graceful Server</a></h2> Whew, that was a lot. One last thing before we finish. To put our task model to the test, we can finally implement the graceful shutdown mechanism we discussed earlier. Imagine we wanted to implement graceful shutdown for our server. When someone hits the keys Ctrl+C, instead of killing the program abruptly, we should stop accepting new connections and wait for any active requests to complete. Any requests that take more than 30 seconds to handle are killed as the server exits. </blockquote> There are a couple things we have to do to set this up. Firstly, we have to actually detect the signal. On Linux, Ctrl+C triggers the SIGINT</code> signal, so we can use the signal_hook</code> crate to wait until the signal is received. use signal_hook::consts::signal::SIGINT; use signal_hook::iterator::Signals; fn ctrl_c() { let mut signal = Signals::new(&[SIGINT]).unwrap(); let _ctrl_c = signal.forever().next().unwrap(); } </code></pre> There's a problem though. forever().next()</code> blocks the thread until the signal is received. Now that our server is async, that means calling ctrl_c()</code> on the main thread will block the entire program. Instead, we need to represent the Ctrl+C signal as a future that resolves when it is received. Something like this. fn ctrl_c() -> impl Future<Output = ()> { poll_fn(move |waker| { // ... }) } </code></pre> So how do we listen for the signal asynchronously? We could register a signal handler with epoll, but we could also use this as an opportunity to learn about handling blocking tasks in an async program. There will be times when there is no asynchronous API to access a necessary resource, but you can't simply make a blocking call on the main thread. Instead, you can run the blocking work on a separate thread, and notify the main thread when it completes. fn spawn_blocking(blocking_work: impl FnOnce()) -> impl Future<Output = ()> { // Run the blocking work on a separate thread. std::thread::spawn(move || { blocking_work(); }); poll_fn(|waker| { // ... })) } </code></pre> The question is, how do we know when the work is done? The blocking work is run on a separate thread, outside of the future. It needs access to the waker so it can notify the future when it completes. We also need a flag that tells the future that the work has completed, in case the work completes before the future is even polled. These two pieces of state can be stored inside a Mutex</code>. let state: Arc<Mutex<(bool, Option<Waker>)>> = Arc::default(); </code></pre> Once the thread completes the work, it must set the flag to true and call wake</code>, if a waker has been stored. It's fine if a waker hasn't been stored yet, the future will see the flag when it's first polled and return immediately. fn spawn_blocking(blocking_work: impl FnOnce() + Send + 'static) -> impl Future<Output = ()> { let state: Arc<Mutex<(bool, Option<Waker>)>> = Arc::default(); let state_handle = state.clone(); // Run the blocking work on a separate thread. std::thread::spawn(move || { // Perform the work. blocking_work(); // Mark the task as done. let (done, waker) = &mut *state_handle.lock().unwrap(); *done = true; // Wake the waker. if let Some(waker) = waker.take() { waker.wake(); } }); poll_fn(|waker| { // ... })) } </code></pre> Now the future needs to access the state and check if the work has completed yet. If not, it stores its waker and returns None</code>, to be woken later when the work does complete. fn spawn_blocking(blocking_work: impl FnOnce() + Send + 'static) -> impl Future<Output = ()> { let state: Arc<Mutex<(bool, Option<Waker>)>> = Arc::default(); let state_handle = state.clone(); // Run the blocking work on a separate thread. std::thread::spawn(move || { // ... }); poll_fn(move |waker| match &mut *state.lock().unwrap() { // The work is not completed, store our waker and come back later. (false, state) => { *state = Some(waker); None } // The work was completed. (true, _) => Some(()), }) } </code></pre> The future returned by spawn_blocking</code> serves as an asynchronous version of JoinHandle</code>. We can wait asynchronously on the main thread while the blocking work is run on a separate thread, allowing other tasks on the main thread to keep making progress. fn ctrl_c() -> impl Future<Output = ()> { spawn_blocking(|| { let mut signal = Signals::new(&[SIGINT]).unwrap(); let _ctrl_c = signal.forever().next().unwrap(); }) } </code></pre> spawn_blocking</code> is an extremely convenient abstraction for dealing with blocking APIs in an async program. Alright, we now have a future that waits for the Ctrl+C signal! If you remember from back when our server used blocking I/O, we wondered how to watch for the signal in a way that aborts the connection listener loop immediately after the signal arrives. We realized we needed some way to listen for both incoming connections, and the ctrl+c signal, at the same time. Because accept</code> was blocking, it wasn't that simple. But with futures, it's actually possible! We can implement this as another future wrapper. Given two futures, we should be able to create a wrapper future that selects between either of the future's outputs, depending on which future completed first. fn select<L, R>(left: L, right: R) -> Select<L, R> { Select { left, right } } struct Select<L, R> { left: L, right: R } enum Either<L, R> { Left(L), Right(R) } impl<L, R> Future for Select<L, R> { type Output = Either<L, R>; fn poll(&mut self, waker: Waker) -> Self::Output { // ... } } </code></pre> It turns out that the implementation of the select future is really simple. We just attempt to poll both futures and return when the first one resolves. impl<L, R> Future for Select<L, R> { type Output = Either<L::Output, R::Output>; fn poll(&mut self, waker: Waker) -> Option<Self::Output> { if let Some(output) = self.left.poll(waker.clone()) { return Some(Either::Left(output)); } if let Some(output) = self.right.poll(waker) { return Some(Either::Right(output)); } None } } </code></pre> Because we pass the same waker to both futures, we will be woken by any progress in either future, and we can check if either of them completed. It really is that simple. Now back to our main program. fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { let listener = TcpListener::bind("localhost:3000").unwrap(); // ... }) .chain(|listener| { poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(handle(connection)); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None, Err(e) => panic!("{e}"), }) }) } </code></pre> We can combine the TCP listener task with the Ctrl+C listener, using our new select</code> combinator. This way we can listen for both at the same time: fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { // ... }) .chain(|listener| { let listen = poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // ... SCHEDULER.spawn(handle(connection)); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None, Err(e) => panic!("{e}"), }); select(listen, ctrl_c()) }) } </code></pre> The TCP listener task never resolves — remember, it represents the loop from our synchronous server. ctrl_c()</code> can resolve though, so we need to chain on another task to handle the shutdown signal. fn listen() -> impl Future<Output = ()> { poll_fn(|waker| { // ... }) .chain(|listener| { let listen = /* ... */; select(listen, ctrl_c()) }) .chain(|_ctrl_c| graceful_shutdown()) } fn graceful_shutdown() -> impl Future<Output = ()> { // ... } </code></pre> Now we need to implement the rest of our shutdown logic. Once the shutdown signal is received, we wait at most thirty seconds for any active requests to complete before shutting down. This sounds like another use case for select</code>! Either thirty seconds elapse, or all active requests complete. fn graceful_shutdown() -> impl Future<Output = ()> { let timer = /* ... */; let request_counter = /* ... */; select(timer, request_counter).chain(|_| { poll_fn(|waker| { // Graceful shutdown process complete, now we actually exit. println!("Graceful shutdown complete"); std::process::exit(0) }) }) } </code></pre> All we need to do now is create the two futures for our shutdown conditions. First we need a timer. We can't simply call thread::sleep</code> on the main thread, because it's a blocking function, but we could run it through spawn_blocking</code> and use the handle to represent our timer future. Note that there are ways to build async timers around epoll, but that's out of scope for this article. </blockquote> use std::thread; use std::time::Duration; fn graceful_shutdown() -> impl Future<Output = ()> { let timer = spawn_blocking(|| thread::sleep(Duration::from_secs(30))); let request_counter = /* ... */; select(timer, request_counter).chain(|_| { poll_fn(|waker| { // Graceful shutdown process complete, now we actually exit. println!("Graceful shutdown complete"); std::process::exit(0) }) }) } </code></pre> That was simple enough. Now for the main shutdown condition. We'll need a counter for active requests in order to know when all active requests are completed. We can keep the counter local to our listen</code> future, and update it whenever tasks are spawned, or complete. fn listen() -> impl Future<Output = ()> { let tasks = Arc::new(Mutex::new(0)); poll_fn(|waker| { // ... }) .chain(move |listener| { let listen = poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { // Increment the counter. *tasks.lock().unwrap() += 1; let handle_connection = handle(connection).chain(|_| { poll_fn(|_| { // Decrement the counter. *tasks.lock().unwrap() -= 1; Some(()) }) }); SCHEDULER.spawn(handle_connection); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None, Err(e) => panic!("{e}"), }); select(listen, ctrl_c()) }) .chain(|_ctrl_c| graceful_shutdown()) } </code></pre> Notice how the decrement of the counter is chained onto the connection handler, so the decrement actually happens from within the spawned task, after it completes. A task counter is great, but we need a little more than that. We can't just check for tasks == 0</code> in a loop, we need the shutdown handler to be notified when the last task completes. And for that, the connection handler that completes needs access to the shutdown handler's waker. What we need is similar to the spawn_blocking</code> solution we created earlier, except instead of a boolean flag, we need a counter. We can wrap up all this state into a small struct. #[derive(Default)] struct Counter { state: Mutex<(usize, Option<Waker>)>, } impl Counter { fn increment(&self) { let (count, _) = &mut *self.state.lock().unwrap(); *count += 1; } fn decrement(&self) { let (count, waker) = &mut *self.state.lock().unwrap(); *count -= 1; // We were the last task. if *count == 0 { // Wake the shutdown handler. if let Some(waker) = waker.take() { waker.wake(); } } } fn wait_for_zero(self: Arc<Self>) -> impl Future<Output = ()> { poll_fn(move |waker| { match &mut *self.state.lock().unwrap() { // The work was completed. (0, _) => Some(()), // The work was not completed, store our waker and come back later. (_, state) => { *state = Some(waker); None } } }) } } </code></pre> When wait_for_zero</code> is first called, it stores its waker in the counter state before returning. Now when the last active task calls decrement</code>, it will also call wake</code>, notifying the wait_for_zero</code> task. When the shutdown handler is woken, it will see that the counter is at zero and shut down the program. Now we can replace our manual counter with the Counter</code> object. fn listen() -> impl Future<Output = ()> { let tasks = Arc::new(Counter::default()); let tasks_ref = tasks.clone(); poll_fn(|waker| { // ... }) .chain(move |listener| { let listen = poll_fn(move |_| match listener.accept() { Ok((connection, _)) => { connection.set_nonblocking(true).unwrap(); // Increment the counter. tasks.increment(); let tasks = tasks.clone(); let handle_connection = handle(connection).chain(|_| { poll_fn(move |_| { // Decrement the counter. tasks.decrement(); Some(()) }) }); SCHEDULER.spawn(handle_connection); None } Err(e) if e.kind() == io::ErrorKind::WouldBlock => None::<()>, Err(e) => panic!("{e}"), }); select(listen, ctrl_c()) }) .chain(|_ctrl_c| graceful_shutdown(tasks_ref)) } </code></pre> Our graceful shutdown handler can use wait_for_zero</code> to wait until all the active tasks are complete. Once they are, or the timer elapses, the graceful shutdown condition is met and the program will exit. fn graceful_shutdown(tasks: Arc<Counter>) -> impl Future<Output = ()> { poll_fn(|waker| { let timer = spawn_blocking(|| thread::sleep(Duration::from_secs(30))); let request_counter = tasks.wait_for_zero(); select(timer, request_counter) }).chain(|_| { // Graceful shutdown process complete, now we actually exit. println!("Graceful shutdown complete"); std::process::exit(0) }) } </code></pre> And that's it! Now if you start the server and hit ctrl+c, it will exit immediately, without blocking for another connection. $ cargo run ^C # => Graceful shutdown complete $ | </code></pre> Looking Back</a></h2> Well, that was quite the journey. Our server is looking pretty good now. From threads, to an epoll event loop, to futures and closure combinators, we've come a long way. There is some manual work that we could abstract over even further, but overall our program is relatively clean. Compared to our original multithreaded program, our code is still clearly more complex. However, it's also a lot more powerful. Composing futures is trivial, and we were able to express complex control flow that would have been very difficult to do with threads. We can even still call out to blocking functions without interrupting our async runtime. There must be a price to pay for all this power, right? Back To Reality</a></h2> Now that we've thoroughly explored concurrency and async ourselves, let's see how it works in the real world. The standard library defines a trait like Future</code>, which looks remarkably similar to the trait we designed. pub trait Future { type Output; fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>; } </code></pre> However, there are a few noticeable differences. The first is that instead of a Waker</code> argument, poll</code> takes a &mut Context</code>. It turns out this isn't much of a difference at all, because Context</code> is simply a wrapper around a Waker</code>. impl Context<'_> { pub fn from_waker(waker: &'a Waker) -> Context<'a> { /* ... */ } pub fn waker(&self) -> &'a Waker { /* ... */ } } </code></pre> And Waker</code>, along with a few other utility methods, has the familiar wake</code> method. impl Waker { pub fn wake(self) { /* ... */ } } </code></pre> Constructing a Waker</code> is a little more complicated, but it's essentially a manual trait object just like the Arc<dyn Fn()></code> we used for our version. All of that happens through the RawWaker</code> type</a>, which you can check out yourself. The second difference is that instead of returning an Option</code>, poll</code> returns a new type called Poll</code>.. which is really just a rebranding of Option</code>. pub enum Poll<T> { Ready(T), Pending, } </code></pre> The final difference is a little more complicated. Pinning</a></h2> Instead of poll</code> taking a mutable reference to self</code>, it takes a pinned mutable reference to self — Pin<&mut Self></code>. What is Pin</code>, you ask? #[derive(Copy, Clone)] pub struct Pin { pointer: P, } </code></pre> Huh. That doesn't seem very useful. It turns out, what makes Pin</code> special is how you create one: impl<P: Deref> Pin { pub fn new(pointer: P) -> Pin where P::Target: Unpin { /* ... */ } pub unsafe fn new_unchecked(pointer: P) -> Pin { /* ... */ } } impl<P: Deref> Deref for Pin { type Target = P::Target; } impl<P: Deref> DerefMut for Pin where P::Target: Unpin { type Target = P::Target; } </code></pre> So you can only create a Pin<&mut T></code> safely if T</code> is Unpin</code>... what's Unpin</code>? pub auto trait Unpin {} /// A marker type which does not implement `Unpin`. pub struct PhantomPinned; impl !Unpin for PhantomPinned {} </code></pre> Unpin</code> seems to be automatically implemented for all types except PhantomPinned</code>. So creating a Pin</code> is safe, except for types that contain PhantomPinned</code>? And Pin</code> prevents you from mutably accessing the T</code>, but only if it is !Unpin</code>? All of this seems a little useless. There is a point to it all though, and it goes back to a problem we ran into earlier. Remember when we tried creating a self-referential struct to hold our task state but it wouldn't work, so we ended up having to allocate our task state with an Arc</code>? It was a bit unfortunate, and it turns out that you actually can create self-referential structs with a little bit of unsafe code, and avoid that Arc</code> allocation. The problem is that you can't just go handing out a self-referential struct in general, because as we realized, moving a self-referential struct breaks its internal references and is unsound. struct SelfReferential { counter: u8, // (X) state: FutureState } enum FutureState { First { // Self-references `counter` (X) counter_ptr: *mut u8 } // ... } </code></pre> let mut future1 = SelfReferential::new(); future1.poll(/* ... */); let mut moved = future1; // Move it // Unsound! `counter_ptr` still point to the old stack location of `counter`. moved.poll(/* ... */); </code></pre> This is where Pin</code> comes in. You can only create a Pin<&mut T></code> if you guarantee that the T</code> will stay in a stable location until it is dropped, meaning that any self-references will remain valid. For most types, Pin</code> doesn't mean anything, which is why Unpin</code> exists. Unpin</code> tells Pin</code> that a type is not self-referential, so pinning it is completely safe and always valid. Pin</code> will even hand out mutable references to Unpin</code> types and let you use mem::swap</code> or mem::replace</code> to move them around. Because you can't safely create a self-referential struct, Unpin</code> is the default and implemented by types automatically. If you did want to create a self-referential future though, you can use the PhantomPinned</code> marker struct to make it !Unpin</code>. Pinning a !Unpin</code> type requires unsafe</code>, so because poll</code> requires Pin<&mut Self></code>, it cannot be called safely on a self-referential future. let mut future = SelfReferential::new(); // SAFETY: We never move `future`. let pinned = unsafe { Pin::new_unchecked(&mut future) }; pinned.poll(/* ... /*); </code></pre> Notice that you can move around the future all you want before pinning it because the self-references are only created after you first call poll</code>. Once you do pin it though, you must uphold the Pin</code> safety contract until the value is dropped. The Pin</code> will prevent you from getting mutable access to the inner value, but it is your responsibility to not tamper with the original binding. There are a couple safe ways of creating a pin though, even for !Unpin</code> types. The first way is with Box::pin</code>. let mut future1: Pin<Box<SelfReferential>> = Box::pin(SelfReferential::new()); future1.as_mut().poll(/* ... */); let mut moved = future1; moved.as_mut().poll(/* ... */); </code></pre> At first glance this may seem unsound, but remember, Box</code> is an allocation. Once the future is allocated it has a stable location on the heap, so you can move around the Box</code> pointer all you want, the internal references will remain stable. The second way you can safely create a pin is with the pin!</code> macro. use std::pin::pin; let mut future1: Pin<&mut SelfReferential> = pin!(SelfReferential::new()); future1.as_mut().poll(/* ... */); </code></pre> With pin!</code>, you can safely pin a struct without even allocating it! The trick is that pin!</code> takes ownership of the future, making it impossible to access except through the Pin<&mut T></code>, which remember, will never give you a mutable reference if T</code> isn't Unpin</code>. The T</code> is completely hidden and thus safe from being tampered with. Pin</code> is a common point of confusion around futures, but once you understand why it exists, the solution is pretty ingenious. async/await</a></h2> Alright, that's the standard Future</code> trait. pub trait Future { type Output; fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>; } </code></pre> So how do we use it? The futures</code></a> crate is where all the useful helpers live. Functions like poll_fn</code> that we wrote before, and combinators like map</code> and and_then</code>, which we called chain</code>. use futures::{FutureExt, future::{ready, poll_fn}}; let future = poll_fn(|_| Poll::Ready(1)) .and_then(|x| poll_fn(|_| Poll::Ready(x + 1))) .and_then(|x| poll_fn(|_| println!("{x}"))); </code></pre> But even with these helpers, as we found out, it's a little cumbersome to write async code. It's still a shift from the simple synchronous code we're used to. Maybe not as drastic as a manual epoll event loop, but still a big change. It turns out there's actually another way to write futures in Rust, with the async/await syntax. Instead of using poll_fn</code> to create futures, you can attach the async</code> keyword to functions: async fn foo() -> usize { 1 } </code></pre> An async function is really just a function that returns an async block: fn foo() -> impl Future<Output = usize> { async { 1 } } </code></pre> Which is really just a function that returns a poll_fn</code> future: fn foo() -> impl Future<Output = usize> { poll_fn(|| Poll::Ready(1)) } </code></pre> The magic comes with the await</code> keyword. await</code> waits for the completion of another future, propagating Poll::Pending</code> until the future is resolved. async fn foo() { let one = one().await; let two = two().await; assert_eq!(one + 1, two); } async fn two() -> usize { one().await + 1 } async fn one() -> usize { 1 } </code></pre> Under the hood, the compiler transforms this into manual state machines, similar to the ones we created with those combinators: fn foo() -> impl Future<Output = ()> { one() .and_then(|one| two().and_then(move |two| poll_fn(move |_| Poll::Ready((one, two))))) .and_then(|(one, two)| poll_fn(move |_| Poll::Ready(assert_eq!(one, two + 1)))) } fn two() -> impl Future<Output = usize> { one().and_then(|one| poll_fn(move |_| Poll::Ready(one + 1))) } fn one() -> impl Future<Output = usize> { poll_fn(|_| Poll::Ready(1)) } </code></pre> Which, as we know all too well, translates into a huge manual state machine that looks something like this: enum FooFuture { One(OneFuture), Two(usize, TwoFuture), } impl Future for FooFuture { type Output = (); fn poll(&mut self, cx: &mut Context<'_>) -> Poll<Self::Output> { if let FooFuture::One(f) = self { match f.poll(cx) { Poll::Ready(one) => *self = Self::Two(one, TwoFuture(OneFuture)), Poll::Pending => return Poll::Pending, } } if let FooFuture::Two(one, f) = self { match f.poll(cx) { Poll::Ready(two) => { assert_eq!(*one + 1, two); return Poll::Ready(()); } Poll::Pending => return Poll::Pending, } } None } } struct TwoFuture(OneFuture); impl Future for TwoFuture { type Output = usize; fn poll(&mut self, cx: &mut Context<'_>) -> Poll<Self::Output> { match self.0.poll(waker) { Poll::Ready(one) => Poll::Ready(one + 1), Poll::Pending => Poll::Pending, } } } struct OneFuture; impl Future for OneFuture { type Output = usize; fn poll(&mut self, cx: &mut Context<'_>) -> Poll<Self::Output> { Poll::Ready(1) } } </code></pre> ... but async/await removes all of that headache. Of course, we aren't actually doing any I/O here so the futures are mostly useless, but you can imagine how helpful this would be for our web server. In fact, it's even better than the combinators. With async</code> functions, you can hold onto local state across await</code> points! async fn foo() { let x = vec![1, 2, 3]; bar(&x).await; baz(&x).await; println!("{x:?}"); } </code></pre> After going through implementing futures ourselves, we can really appreciate the convenience of this. Under the hood the compiler has to generate a self-referential future to give bar</code> and baz</code> access to the state. struct FooFuture { x: Vec<i32>, // (X) state: FooFutureState, } enum FooFutureState { Bar(BarFuture), Baz(BazFuture), } struct BarFuture { x: *mut Vec<i32> /* pointer to (X) */ } struct BazFuture { x: *mut Vec<i32> /* pointer to (X) */ } </code></pre> The compiler takes care of all the unsafe code involved in this, allowing us to work with local state just like we would in a regular function. For this reason, the futures generated by async</code> blocks or functions are !Unpin</code>. async/await removes any complexity that remained with writing futures compared to synchronous code. After implementing futures manually, it almost feels like magic! A Tokio Server</a></h2> So far we've only been looking at how Future</code> works, we haven't discussed how to actually run one, or do any I/O. The thing is, the standard library doesn't provide any of that, it only provides the bare essential types and traits to get started. If you want to actually write an async application, you have to use an external runtime. The most popular general purpose runtime is tokio</code>. tokio</code> provides a task scheduler, a reactor, and a pool to run blocking tasks, just like we wrote earlier, but it also provides timers, async channels, and various other useful types and utilities for async code. On top of that, tokio</code> is multi-threaded, distributing async tasks to take advantage of all your CPU cores. The core ideas behind tokio</code> are very similar to the async runtime we wrote ourselves, but you can read more about it's design in this excellent blog post</a>. It's time to write our final web server, this time using the standard Future</code> trait and tokio. Tokio applications begin with the #[tokio::main]</code> macro. Under the hood, this macro spins up the runtime and runs the async code in main</code>. #[tokio::main] async fn main() { // ... } </code></pre> Tokio does it's best to mirror the standard library for most of it's types. For example, tokio::net::TcpListener</code> works exactly like std::net::TcpListener</code>, except with async</code> methods. Any interactions with epoll and the reactor are hidden under the hood. use tokio::net::{TcpListener, TcpStream}; #[tokio::main] async fn main() { let listener = TcpListener::bind("localhost:3000").await.unwrap(); loop { let (connection, _) = listener.accept().await.unwrap(); if let Err(e) = handle_connection(connection).await { println!("failed to handle connection: {e}") } } } async fn handle_connection(mut connection: TcpStream) -> io::Result<()> { // ... } </code></pre> That isn't exactly right though, we need to spawn the connection handler. We can do so with the tokio::spawn</code> function, which takes a future. #[tokio::main] async fn main() { let listener = TcpListener::bind("localhost:3000").await.unwrap(); loop { let (connection, _) = listener.accept().await.unwrap(); tokio::spawn(async move { if let Err(e) = handle_connection(connection).await { println!("failed to handle connection: {e}") } }); } } </code></pre> Now for the connection handler. With the AsyncReadExt</code> trait and the await</code> keyword, we can read from the TCP stream almost exactly like we did before. use tokio::io::AsyncRead; async fn handle_connection(mut connection: TcpStream) -> io::Result<()> { let mut read = 0; let mut request = [0u8; 1024]; loop { // Try reading from the stream. let num_bytes = connection.read(&mut request[read..]).await?; // The client disconnected. if num_bytes == 0 { println!("client disconnected unexpectedly"); return Ok(()); } // Keep track of how many bytes we've read. read += num_bytes; // Have we reached the end of the request? if request.get(read - 4..read) == Some(b"\r\n\r\n") { break; } } let request = String::from_utf8_lossy(&request[..read]); println!("{request}"); // ... } </code></pre> Writing the response works the same as well. async fn handle_connection(mut connection: TcpStream) -> io::Result<()> { // ... // "Hello World!" in HTTP let response = /* ... */; let mut written = 0; loop { // Write the remaining response bytes. let num_bytes = connection.write(response[written..].as_bytes()).await?; // The client disconnected. if num_bytes == 0 { println!("client disconnected unexpectedly"); return Ok(()); } written += num_bytes; // Have we written the whole response yet? if written == response.len() { break; } } connection.flush().await } </code></pre> Well that was easy. If you notice, our program is exactly the same as our original server, with the exception of a couple uses of the async</code> and await</code> keywords. With async/await, we really can have our cake and eat it too. Now to implement graceful shutdown. The first step is to identify the ctrl+c signal. With tokio, this is as simple as using the tokio::signal::ctrl_c</code> function, an async function that returns once the ctrl+c signal is received. We can also use tokio's select!</code> macro, a more powerful version of the select</code> combinator we implemented earlier. pub async fn main() { let listener = TcpListener::bind("localhost:3000").await.unwrap(); let state = Arc::new((AtomicUsize::new(0), Notify::new())); loop { select! { // New incoming connection. result = listener.accept() => { let (connection, _) = result.unwrap(); tokio::spawn(async move { // .. }); } // Ctrl+C signal. shutdown = ctrl_c() => { let timer = /* ... */; let request_counter = /* .. */; select! { _ = timer => {} _ = request_counter => {} } println!("Gracefully shutting down."); return; } } } } </code></pre> select!</code> runs the branch for whichever future completes first and cancels the others, allowing us to select between incoming connections and the Ctrl+C signal, and run the appropriate code for each. Now we need to create the graceful shutdown condition. For the timer, we can use tokio's asynchronous sleep</code> function. Under the hood this hooks into a custom timer system, a much more efficient version of our spawn_blocking</code> timers. You can read more about how that works in this other excellent post</a>. select! { // New incoming connection. result = listener.accept() => { // ... } // Ctrl+C signal. shutdown = ctrl_c() => { let timer = tokio::time::sleep(Duration::from_secs(30)); let request_counter = /* .. */; select! { _ = timer => {} _ = request_counter => {} } println!("Gracefully shutting down."); return; } } </code></pre> Now for the active request counter. Instead of managing wakers manually, we can use a simple counter and take advantage of tokio's Notify</code> type, which allows tasks to notify each other, or wait to be notified. use tokio::sync::Notify; let state = Arc::new((AtomicUsize::new(0), Notify::new())); </code></pre> When a request comes in, we increment the counter, and when it completes, we decrement it. If the counter reaches zero, the last active task calls notify_one</code>, which will wake up the main thread, letting it know that all active tasks have completed. select! { result = listener.accept() => { let (connection, _) = result.unwrap(); let state = state.clone(); // Increment the counter. state.0.fetch_add(1, Ordering::Relaxed); tokio::spawn(async move { if let Err(e) = handle_connection(connection).await { // ... } // Decrement the counter. let count = state.0.fetch_sub(1, Ordering::Relaxed); if count == 1 { // We were the last active task. state.1.notify_one(); } }); } shutdown = ctrl_c() => { // ... } } </code></pre> The shutdown handler can now simply select between the timer and Notify::notified</code>, which will resolve when someone calls notify_one</code>, indicating that the last active request has completed. select! { result = listener.accept() => { // ... } shutdown = ctrl_c() => { // A 30 second timer. let timer = tokio::time::sleep(Duration::from_secs(30)); // Notified by the last active task. let notification = state.1.notified(); // If the count isn't zero, we have to wait. if state.0.load(Ordering::Relaxed) != 0 { // Wait for either the timer or notification to resolve. select! { _ = timer => {} _ = notification => {} } } println!("Gracefully shutting down."); return; } } </code></pre> Beautiful, isn't it? With tokio</code> and async/await we don't even have to think about wakers, reactors, or anything else that goes on under the hood. All the building blocks are provided for us, we just have to put them together. Afterword</a></h2> Whew, that was quite the journey! We started from the simplest web server, tried multithreading, and then worked our way up to a custom asynchronous runtime built on epoll. All to implement graceful shutdown. Then we circled back and implemented graceful shutdown with tokio</code> in just a few extra lines of code. Hopefully this article has helped you appreciate the power of async Rust, as well as taught you a little more about how it works under the hood. All the code for this repository is available on GitHub</a>. ^{1 1. There's really no way to force flush a network socket so flush</code> is actually a no-op on TcpStream</code>, but we'll call it anyways to be true to io::Write</code>. </div>}^22. The real scheduler is of course more complicated than this. You can read about it here</a>. </div> ^33. For example, flushing the TLB</a>. </div> ^{4 4. In fact on Linux, threads and processes are just "tasks" with different configuration settings. </div> ^{5 5. Technically, our scheduler is unfair. A fair scheduler would check epoll every once in a while even if there are tasks that are runnable, to avoid starving certain tasks. </div>}}

Go in Twenty Minutes

Unknown — Wed, 16 Sep 2020 17:17:28 +0000

A lot of thanks goes to fasterthanlime</a> for inspiring this article with A half-hour to learn Rust</a>. I thought I'd try something similar, but instead of Rust, this article will try provide an overview of Go in twenty minutes.

Ready? Go!

Packages</a></h2>

Every Go program is made up of packages. Programs start by running the main</code> function in the main</code> package.

package </span>main
</span>
</span>func </span>main</span>() {
</span>    </span>// ...
</span>}
</span></code></pre>
From now on, the main</code> function and package declaration will be omitted from examples for brevity.</p>
You can use external packages by importing them:</p>
import </span>"fmt"
</span></code></pre>
You can use exported names in an imported package by using the package name as a qualifier</em>:</p>
import </span>(
</span>  </span>"fmt"
</span>  </span>"math/rand"
</span>)
</span>
</span>
</span>fmt.Println(</span>"My favorite number is"</span>, rand.Intn(</span>10</span>))
</span></code></pre>
You can create a local alias for any import:</p>
import </span>(
</span>  formatter </span>"fmt"
</span>  random </span>"math/rand"
</span>)
</span>
</span>
</span>formatter.Println(</span>"My favorite number is"</span>, random.Intn(</span>10</span>))
</span></code></pre>
Or use a dot to access it without any qualifier:</p>
import </span>(
</span>  . </span>"fmt"
</span>  . </span>"math/rand"
</span>)
</span>
</span>Println(</span>"My favorite number is"</span>, Intn(</span>10</span>))
</span></code></pre>
A name is exported if it begins with a capital letter. This is similar to public</code> and private</code> in other languages:</p>
var </span>x </span>string </span>= </span>"hello" </span>// x is unexported, or private
</span>var </span>A </span>string </span>= </span>"hello" </span>// A will be exported, or public
</span></code></pre>
Packages can have an init</code> function. This will be executed as soon as the package is imported:</p>
func </span>init</span>() {
</span>  performSideEffects()
</span>}
</span></code></pre>
Scope</a></h2>
A pair of brackets declares a block, which has its own scope:</p>
// This prints "in", then "out"
</span>func </span>main</span>() {
</span>  x </span>:= </span>"out"
</span>  {
</span>    </span>// this is a different `x`
</span>    x </span>:= </span>"in"
</span>    </span>println</span>(x) </span>// => in
</span>  }
</span>  </span>println</span>(x) </span>// => out
</span>}
</span></code></pre>
A variable defined outside of any function is in the "global" scope.</p>
var </span>Global </span>string </span>= </span>"I am a global variable!"
</span></code></pre>
Variables</a></h2>
var</code> declares a variable of a given type:</p>
var </span>x </span>int </span>// Declare variable "x" of type int
</span>x </span>= </span>42    </span>// Assign 42 to "x"
</span></code></pre>
This can also be written as a single line:</p>
var </span>x </span>int </span>= </span>42
</span></code></pre>
The type of a variable can also be inferred:</p>
var </span>x </span>= </span>42
</span></code></pre>
Inside of a function body, this can also be written in short form:</p>
x </span>:= </span>42
</span></code></pre>
You can declare many variables at the same time:</p>
var </span>x, y </span>int
</span>x, y </span>= </span>10</span>, </span>20
</span></code></pre>
If you declare a variable without initializing it, it will implicitly be assigned to the zero value of its type:</p>
var </span>a </span>SomeStruct
</span>var </span>b </span>bool
</span>var </span>x </span>int
</span>var </span>y </span>string
</span>var </span>z []</span>int
</span>
</span>fmt.Println(a) </span>// => nil
</span>fmt.Println(b) </span>// => false
</span>fmt.Println(x) </span>// => 0
</span>fmt.Println(y) </span>// => ""
</span>fmt.Println(z) </span>// => []
</span></code></pre>
Type Conversions</a></h2>
You can convert values between different types:</p>
var </span>f </span>float64 </span>= </span>1 </span>// +1.000000
</span>i </span>:= </span>int</span>(f)       </span>// 1
</span>z </span>:= </span>uint</span>(f)      </span>// 1
</span></code></pre>
Note that unlike in other languages, conversions between types must be done explicitly.</p>
Constants</a></h2>
Unchanging values can be declared with the const</code> keyword. Constants can be characters, strings, booleans, or numeric values:</p>
const </span>pi </span>= </span>3.14
</span></code></pre>
You can group the declaration of multiple constants or variables:</p>
const </span>(
</span>  </span>x </span>= </span>2
</span>  </span>y </span>= </span>4
</span>)
</span></code></pre>
Constants are immutable</em> - they cannot be reassigned to:</p>
const </span>x </span>= </span>1
</span>x </span>= </span>2 </span>// error: cannot assign to x
</span></code></pre>
Iota</a></h2>
The iota</code> keyword represents successive integer constants:</p>
const </span>(
</span>  </span>zero </span>= </span>iota
</span>  </span>one  </span>= </span>iota
</span>  </span>two  </span>= </span>iota
</span>)
</span>
</span>fmt.Println(zero, one, two) </span>// "0 1 2"
</span></code></pre>
This can be written in short form:</p>
const </span>(
</span>  </span>zero </span>= </span>iota
</span>  </span>one
</span>  </span>two
</span>)
</span>
</span>fmt.Println(zero, one, two) </span>// "0 1 2"
</span></code></pre>
You can perform operations on an iota:</p>
const </span>(
</span>  </span>four </span>= </span>iota </span>+ </span>4
</span>  </span>five
</span>  </span>six
</span>)
</span>
</span>fmt.Println(four, five, six) </span>// "4 5 6"
</span></code></pre>
And use the blank identifier to skip a value:</p>
const </span>(
</span>  </span>zero </span>= </span>iota
</span>  _    </span>// skip 1
</span>  </span>two
</span>  </span>three
</span>)
</span>fmt.Println(zero, two, three) </span>// "0 2 3"
</span></code></pre>
Iota's are commonly used to represent enums, which Go does not support natively.</p>
The Blank Identifier</a></h2>
The blank identifier _</code> is an anonymous placeholder. It basically means to throw away something:</p>
// this does nothing
</span>_ </span>= </span>42
</span>
</span>// this calls `getThing` but throws away the 
</span>// two return values
</span>_, _ </span>= </span>getThing();
</span></code></pre>
You can use it to import a package solely for its init</code> function:</p>
import </span>_ </span>"log"
</span></code></pre>
Or to avoid compiler errors during development:</p>
var </span>_ </span>= </span>devFunction() </span>// TODO: remove in production
</span></code></pre>
Looping</a></h2>
Go has only one loop, the for</code> loop. It has three components:</p>

init: executed before the first iteration</li>
condition: evaluated before every iteration</li>
post: executed at the end of every iteration</li>
</ul>
Here's an example:</p>
sum </span>:= </span>0
</span>for </span>i </span>:= </span>0</span>; i </span>< </span>10</span>; i</span>++ </span>{
</span>  sum </span>++
</span>}
</span>fmt.Println(sum) </span>// 10
</span></code></pre>
The init and post statements are optional:</p>
// this loop will run forever
</span>yup </span>:= </span>true
</span>for </span>yup {
</span>  fmt.Println(</span>"yup!"</span>)
</span>}
</span></code></pre>
An infinite for loop can also be written like this:</p>
// this loop will also run forever
</span>for </span>{
</span>    fmt.Println(</span>"yup!"</span>)
</span>}
</span></code></pre>
You can use break to break</code> out of a loop:</p>
for </span>{
</span>  </span>if </span>shouldBreak() {
</span>    </span>break
</span>  }
</span>}
</span></code></pre>
And continue</code> to skip to the next iteration</p>
for </span>i </span>:= </span>0</span>; i </span>< </span>5</span>; i</span>++ </span>{
</span>  </span>if </span>i </span>== </span>2 </span>{
</span>    </span>continue
</span>  }
</span>  fmt.Print(i)
</span>}
</span>
</span>// => 0 1 3 4
</span></code></pre>
Control Flow</a></h2>
Go's if</code> statements have a similar syntax to its loops:</p>
if </span>1 </span>== </span>1 </span>{
</span>  fmt.Println(</span>"true"</span>)
</span>}
</span></code></pre>
if</code> conditions can also have an init statement:</p>
// this prints "foo"
</span>if </span>x </span>:= </span>"foo"</span>; x </span>!= </span>"bar" </span>{
</span>  fmt.Println(x)
</span>}
</span>// "x" is now out of scope
</span></code></pre>
Go also has else</code> and else if</code> statements:</p>
if </span>something {
</span>  doSomething()
</span>} </span>else if </span>somethingElse {
</span>  doSomethingElse()
</span>} </span>else </span>{
</span>  </span>return
</span>}
</span></code></pre>
If your if - else</code> statement is getting long, switch to a switch</code> statement :)</p>
switch </span>{
</span>case </span>something:
</span>  doSomething()
</span>case </span>somethingElse </span>> </span>10</span>:
</span>  doSomethingElse()
</span>default</span>:
</span>  </span>return
</span>}
</span></code></pre>
You can also switch on a condition:</p>
switch </span>x </span>:= </span>2</span>; x {
</span>// if x equals 1
</span>case </span>1</span>:
</span>  doSomething()
</span>// if x equals 2
</span>case </span>2</span>:
</span>  doSomethingElse()
</span>// otherwise
</span>default</span>:
</span>  </span>return
</span>}
</span></code></pre>
You can fall through</em> to the case below the current case:</p>
x </span>:= </span>1
</span>
</span>switch </span>x {
</span>case </span>1</span>:
</span>    fmt.Print(</span>"1"</span>)
</span>    </span>fallthrough
</span>case </span>2</span>:
</span>    fmt.Print(</span>"2"</span>)
</span>}
</span>
</span>// => 1 2
</span></code></pre>
In a switch statement, only the first matched case is executed unless you fall through</em>.</p>
Go also has labels</em> to help manage control flow:</p>
MyLabel:
</span></code></pre>
You can jump to a specific label with the goto</code> keyword:</p>
print</span>(</span>1</span>)
</span>
</span>// jump to the "Three" label
</span>goto </span>Three
</span>fmt.Print(</span>2</span>)
</span>
</span>Three:
</span>fmt.Print(</span>3</span>)
</span>
</span>// => 13
</span></code></pre>
Labels have a very specific use case. They can often make code less readable and are avoided by many Go programmers.</p>
Arrays</a></h2>
Arrays have a fixed length:</p>
var </span>a [</span>2</span>]</span>string </span>// an array of 2 strings
</span>a[</span>0</span>] </span>= </span>"Hello"
</span>a[</span>1</span>] </span>= </span>"World"
</span>fmt.Println(a) </span>// => [Hello World]
</span></code></pre>
You create arrays using array literals</em>:</p>
helloWorld </span>:= </span>[</span>2</span>]</span>string</span>{</span>"Hello"</span>, </span>"World"</span>}
</span></code></pre>
You cannot increase an array beyond its capacity. Doing so will cause an error:</p>
helloWorld </span>:= </span>[</span>2</span>]</span>string</span>{</span>"Hello"</span>, </span>"World"</span>}
</span>helloWorld[</span>10</span>] </span>= </span>"Space"
</span>
</span>// panic: invalid array index 10 (out of bounds for 2-element array)
</span></code></pre>
To get around this issue, Go provides slices.</p>
Slices</a></h2>
Slices are more commonly used than arrays. They are flexible in that they do not have a fixed length:</p>
nums </span>:= </span>[]</span>int</span>{</span>1</span>, </span>2</span>, </span>3</span>, </span>4</span>, </span>5</span>}
</span></code></pre>
Slices can also be created with the  make</code> function:</p>
make</span>([]</span>string</span>, initialLength)
</span>
</span>// initialCapacity can be set to avoid allocations
</span>make</span>([]</span>string</span>, initialLength, initialCapacity)
</span></code></pre>
You can also create a slice by slicing</em> an existing array:</p>
nums </span>:= </span>[</span>6</span>]</span>int</span>{</span>1</span>, </span>2</span>, </span>3</span>, </span>4</span>, </span>5</span>, </span>6</span>}
</span>
</span>// take elements 1 - 4 from `nums`
</span>s </span>:= </span>nums[</span>1</span>:</span>4</span>]
</span>
</span>fmt.Println(s)
</span>// => [2 3 4]
</span></code></pre>
You can omit the start or end index when slicing an array, so for this array:</p>
var </span>a [</span>10</span>]</span>int
</span></code></pre>
These expressions are equivalent:</p>
a[:]
</span>a[</span>0</span>:]
</span>a[:</span>10</span>]
</span>a[</span>0</span>:</span>10</span>]
</span></code></pre>
Modifying the elements of a slice will also modify its underlying array:</p>
nums </span>:= </span>[</span>6</span>]</span>int</span>{</span>1</span>, </span>2</span>, </span>3</span>, </span>4</span>, </span>5</span>, </span>6</span>}
</span>
</span>s </span>:= </span>nums[</span>0</span>:</span>4</span>]
</span>
</span>// modify s
</span>s[</span>0</span>] </span>= </span>999
</span>
</span>// which also modifies nums
</span>fmt.Println(nums)
</span>// => [999 2 3 4 5 6]
</span></code></pre>
Slices are just a fancy way to manage an underlying array. You cannot increase a slice beyond its capacity. Doing so will cause a panic:</p>
nums </span>:= </span>[]</span>int</span>{</span>1</span>, </span>2</span>, </span>3</span>, </span>4</span>, </span>5</span>, </span>6</span>}
</span>nums[</span>20</span>] </span>= </span>9
</span>
</span>// panic: runtime error: index out of range [20] 
</span>// with length 6
</span></code></pre>
However, Go has built-in functions to make slices feel like arrays. For examples, append</code> can be used to add items to a slice:</p>
nums </span>:= </span>[]</span>int</span>{</span>1</span>, </span>2</span>, </span>3</span>, </span>4</span>, </span>5</span>}
</span>nums </span>= </span>append</span>(nums, </span>6</span>)
</span>
</span>fmt.Println(nums)
</span>// => [1 2 3 4 5 6]
</span></code></pre>
You can iterate over slices and arrays with range</code>:</p>
names </span>:= </span>[]</span>string</span>{</span>"john"</span>, </span>"joe"</span>, </span>"jessica"</span>}
</span>for </span>index, name </span>:= range </span>names {
</span>  fmt.Println(index, name)
</span>}
</span>
</span>// 0 john
</span>// 1 joe
</span>// 2 jessica
</span></code></pre>
You can use the blank identifier to omit the index, or the value from range</code>:</p>
for </span>_, name </span>:= range </span>names { </span>... </span>}
</span>for </span>index, _ </span>:= range </span>names { </span>... </span>}
</span></code></pre>
If you only want the index, you can omit the second variable entirely:</p>
names </span>:= </span>[]</span>string</span>{</span>"john"</span>, </span>"joe"</span>, </span>"jessica"</span>}
</span>
</span>for </span>index </span>:= range </span>names {
</span>    fmt.Println(index)
</span>}
</span>
</span>// => 0 1 2
</span></code></pre>
To understand the inner workings of slices and arrays in detail, check out: Go Slices: Usage and Internals</a></p>
Maps</a></h2>
Maps are like hashes in ruby or dictionaries in python. You create them with a map literal</em>:</p>
mymap </span>:= map</span>[</span>string</span>]</span>string</span>{</span>"name"</span>: </span>"ibraheem"</span>, </span>"city"</span>: </span>"toronto"</span>}
</span></code></pre>
Or the built-in make</code> function:</p>
mymap </span>:= </span>make</span>(</span>map</span>[</span>string</span>]</span>string</span>)
</span></code></pre>
You can set the value under a particular key:</p>
mymap[</span>"key"</span>] </span>= </span>"value"
</span></code></pre>
And access it:</p>
val </span>:= </span>mymap[</span>"key"</span>]
</span>fmt.Println(val) </span>// "value"
</span></code></pre>
Or delete it:</p>
delete</span>(mymap, </span>"key"</span>)
</span></code></pre>
You can check whether a key is present:</p>
x, ok </span>:= </span>mymap[</span>"key"</span>] </span>// => "value", true
</span>x, ok </span>:= </span>mymap[</span>"doesnt-exist"</span>] </span>// => "", false
</span></code></pre>
And iterate over a map's keys and values:</p>
m </span>:= map</span>[</span>int</span>]</span>string</span>{</span>1</span>: </span>"one"</span>, </span>2</span>: </span>"two"</span>}
</span>for </span>key, value </span>:= range </span>m {
</span>  fmt.Println(key, value)
</span>}
</span>
</span>// 1 "one"
</span>// 2 "two"
</span></code></pre>
Functions</a></h2>
func</code> declares a function.</p>
Here's a void function:</p>
func </span>greet</span>() {
</span>  fmt.Println(</span>"Hi there!"</span>)
</span>}
</span></code></pre>
And here's a function that returns an integer:</p>
func </span>fairDiceRoll</span>() </span>int </span>{
</span>  </span>return </span>4
</span>}
</span></code></pre>
Functions can return multiple values:</p>
func </span>oneTwoThree</span>() (</span>int</span>, </span>int</span>, </span>int</span>) {
</span>  </span>return </span>1</span>, </span>2</span>, </span>3
</span>}
</span></code></pre>
They can also take specified arguments:</p>
func </span>sayHello</span>(name </span>string</span>) {
</span>  fmt.Printf(</span>"hello </span>%s</span>"</span>, name)
</span>}
</span></code></pre>
Or an arbitrary number of arguments:</p>
func </span>variadic</span>(nums </span>...</span>int</span>) {
</span>  fmt.Println(nums)
</span>}
</span></code></pre>
These are called variadic functions</em>. They can be called just like regular functions, except you can pass in as many arguments as you want:</p>
variadic(</span>1</span>, </span>2</span>, </span>3</span>) </span>// => [1 2 3]
</span></code></pre>
If a function takes two or more arguments of the same type, you can group them together:</p>
func </span>multiply</span>(x, y </span>int</span>) </span>int </span>{
</span>  </span>return </span>x </span>* </span>y
</span>}
</span></code></pre>
The defer</code> statement defers the execution of a function until the surrounding function returns:</p>
defer </span>fmt.Println(</span>"world"</span>)
</span>fmt.Println(</span>"hello"</span>)
</span>
</span>// "hello"
</span>// "world"
</span></code></pre>
Defer is often used with an anonymous function:</p>
defer func</span>() {
</span>  </span>println</span>(</span>"I'm done!"</span>)
</span>}()
</span></code></pre>
Anonymous Functions</a></h2>
Go supports anonymous functions. Anonymous functions are useful when you want to define a function inline without having to name it:</p>
one </span>:= func</span>() </span>int </span>{ 
</span>  </span>return </span>1 
</span>}
</span>
</span>fmt.Println(one()) </span>// 1
</span></code></pre>
Functions can take functions as arguments</p>
func </span>printNumber</span>(getNum </span>func</span>() </span>int</span>) {
</span>  num </span>:= </span>getNum()
</span>  fmt.Println(num)
</span>}
</span>
</span>printNumber(</span>func</span>() </span>int </span>{ 
</span>  </span>return </span>1 
</span>})
</span>
</span>// => 1
</span></code></pre>
A function can also return an anonymous function:</p>
func </span>getOne</span>() </span>func</span>() </span>int </span>{
</span>  </span>return func</span>() </span>int </span>{
</span>    </span>return </span>1
</span>  }
</span>}
</span>
</span>one </span>:= </span>getOne()
</span>fmt.Println(one()) 
</span>
</span>// => 1
</span></code></pre>
Anonymous functions allow us to dynamically change what a function does at runtime:</p>
doStuff </span>:= func</span>() {
</span>  fmt.Println(</span>"Doing stuff!"</span>)
</span>}
</span>doStuff()
</span>
</span>doStuff </span>= func</span>() {
</span>  fmt.Println(</span>"Doing other stuff."</span>)
</span>}
</span>doStuff()
</span>
</span>// => Doing stuff!
</span>// => Doing other stuff.
</span></code></pre>
Anonymous functions can form closures</em>. A closure is an anonymous function that references a variable from outside itself:</p>
n </span>:= </span>0
</span>counter </span>:= func</span>() </span>int </span>{
</span>  n </span>+= </span>1
</span>  </span>return </span>n
</span>}
</span>fmt.Println(counter())
</span>fmt.Println(counter())
</span>fmt.Println(counter())
</span>
</span>// => 1
</span>// => 2
</span>// => 3
</span></code></pre>
Note how counter</code> has access to n</code>, even though it was never passed as a parameter.</p>
Pointers</a></h2>
A pointer holds the memory address of a value.</p>
// "p" is a pointer to an integer
</span>var </span>p </span>*</span>int
</span></code></pre>
The &</code> operator generates a pointer to its operand:</p>
i </span>:= </span>42
</span>var </span>p </span>*</span>int </span>= &</span>i </span>// point to i
</span></code></pre>
The *</code> operator lets you access the pointer's underlying value:</p>
i </span>:= </span>42
</span>p </span>:= &</span>i     </span>// point to i
</span>
</span>fmt.Println(p) </span>// 0xc000018050
</span>fmt.Println(</span>*</span>p) </span>// => 42
</span></code></pre>
You can use *</code> to modify the original value:</p>
i </span>:= </span>42
</span>
</span>p </span>:= &</span>i  </span>// point to i 
</span>*</span>p </span>= </span>21  </span>// modify i through the pointer
</span>
</span>fmt.Println(i) </span>// 21
</span></code></pre>
Custom Types and Methods</a></h2>
You can create a custom type with the type</code> keyword:</p>
type </span>UserName </span>string
</span></code></pre>
This is called a type definition</em>. It defines a new, distinct type that is based on the structure of an underlying type.</p>
You can define methods</em> on your custom types. Methods are different than functions in that they take a receiver</em> and are called on an instance of a specific type. The receiver is specified before the method name, and the arguments are specified after:</p>
type </span>MyString </span>string
</span>
</span>// A method called `Print` with a receiver of type `MyString`
</span>func </span>(m </span>MyString</span>) </span>Print</span>() {
</span>  fmt.Println(m)
</span>}
</span>
</span>var </span>x </span>MyString = </span>"hello"
</span>x.Print() </span>// => "hello"
</span></code></pre>
Methods receivers are copied by default, meaning their fields will not be mutated:</p>
func </span>(m </span>MyString</span>) </span>BecomeHello</span>() {
</span>  </span>// m is a copy of the original MyString type
</span>  </span>// this modifies the copy
</span>  m </span>= </span>"hello"
</span>}
</span>
</span>var </span>x </span>MyString = </span>"not hello"
</span>x.BecomeHello()
</span>
</span>fmt.Println(x)
</span>// => "not hello"
</span></code></pre>
To mutate the receiver, use a pointer:</p>
func </span>(m </span>*MyString</span>) </span>BecomeHello</span>() {
</span>  </span>*</span>m </span>= </span>"hello"
</span>}
</span>
</span>var </span>x </span>MyString = </span>"not hello"
</span>x.BecomeHello()
</span>
</span>println</span>(x)
</span>// => "hello"
</span></code></pre>
Pointer receivers are used when you want to modify the value the receiver points to. They can also be used to avoid copying the value for each method call, which can be more efficient when dealing with large amounts of data.</p>
Structs</a></h2>
Structs are a built-in Go type. You can declare one with the struct</code> keyword. They can have any number of fields a specified type:</p>
type </span>MyStruct </span>struct </span>{
</span>  x </span>int
</span>  y </span>int
</span>}
</span></code></pre>
You can create a new struct with a struct literals</em>:</p>
s1 </span>:= </span>MyStruct{ x: </span>1</span>, y: </span>2 </span>}
</span>
</span>// a pointer to a struct
</span>var </span>s2 </span>*MyStruct = &</span>MyStruct{ x: </span>1</span>, y: </span>2 </span>} 
</span></code></pre>
You can omit the names of the fields and instead rely on the order:</p>
s1 </span>:= </span>MyStruct{ </span>1</span>, </span>2 </span>}
</span></code></pre>
Struct fields are accessed using a dot:</p>
s </span>:= </span>MyStruct{ x: </span>1</span>, y: </span>2 </span>}
</span>s.x </span>= </span>1 
</span></code></pre>
If a field is not initialized, it defaults to the default value of its type:</p>
s1 </span>:= </span>MyStruct{}
</span>fmt.Println(s1.x) </span>// => 0
</span></code></pre>
Struct fields can also be accessed and modified through a struct pointer just like a regular variable:</p>
s </span>:= </span>MyStruct{}
</span>
</span>p </span>:= &</span>s
</span>(</span>*</span>p).x </span>= </span>19 </span>// modify `s` through `p`
</span>
</span>fmt.Println(s.x) </span>// => 19
</span></code></pre>
Because this is so common, Go allows you to modify the underlying struct of a pointer without a *</code>:</p>
p </span>:= &</span>s
</span>
</span>// these are the same
</span>(</span>*</span>p).x </span>= </span>19
</span>p.x </span>= </span>19
</span></code></pre>
You can declare methods on your own struct types just like with any other custom type:</p>
type </span>Number </span>struct </span>{
</span>  value </span>int
</span>}
</span>
</span>func </span>(n </span>Number</span>) </span>isStrictlyPositive</span>() </span>bool </span>{
</span>  </span>return </span>n.value </span>>= </span>0
</span>}
</span></code></pre>
And use them as usual:</p>
minusTwo </span>:= </span>Number{ 
</span>  value: </span>-</span>2</span>,
</span>}
</span>
</span>minusTwo.isStrictlyPositive()
</span>// => false
</span></code></pre>
Interfaces</a></h2>
Interfaces are something multiple types have in common. They can contain any number of methods in their method set:</p>
type </span>Signed </span>interface </span>{
</span>  </span>isStrictlyNegative</span>() </span>bool
</span>}
</span></code></pre>
An interface is a type, so it can be used as a function argument, a struct field, or in the place of any other type:</p>
type </span>MyStruct </span>struct </span>{
</span>  signed </span>Signed
</span>}
</span></code></pre>
An interface value holds a value of the underlying concrete type. Calling a method on an interface value will call the method on its underlying type:</p>
func </span>SignedIsStrictlyNegative</span>(s </span>Signed</span>) </span>bool </span>{
</span>  </span>// call a method on the underlying type of `s`
</span>  </span>return </span>s.isStrictlyNegative()
</span>}
</span></code></pre>
A struct implements</em> an interface if it has all of its methods:</p>
type </span>Number </span>struct </span>{
</span>  value </span>int
</span>}
</span>
</span>func </span>(n </span>*Number</span>) </span>isStrictlyNegative</span>() </span>bool </span>{
</span>  </span>return </span>n.value </span>< </span>0
</span>}
</span></code></pre>
Now *Number</code> implements the Signed</code> interface.</p>
So this works:</p>
SignedIsStrictlyNegative(</span>&</span>Number{})
</span></code></pre>
However, Number</code> doesn't implement Signed</code> because isStrictlyNegative</code> is defined only on *Number</code>. This is because Number</code> and *Number</code> are different</em> types:</p>
// this will not compile
</span>SignedIsStrictlyNegative(Number{})
</span>
</span>// but this will
</span>SignedIsStrictlyNegative(</span>&</span>Number{})
</span></code></pre>
Of course, this won't compile either:</p>
// strings do not implement the Signed interface
</span>SignedIsStrictlyNegative(</span>"a string"</span>)
</span></code></pre>
interface{}</code></a></h2>
The interface type that specifies zero methods is known as the empty interface:</p>
interface</span>{} </span>// literally anything
</span></code></pre>
An empty interface may hold values of any type:</p>
var </span>x </span>interface</span>{} </span>= </span>Number{}
</span>var </span>y </span>interface</span>{} </span>= </span>"hello"
</span>var </span>z </span>interface</span>{} </span>= </span>1
</span></code></pre>
Values stored as an interface are type erased</em>.</p>
var </span>x </span>interface</span>{} </span>= </span>Number{}
</span>
</span>x.isStrictlyNegative() </span>// error: x.isStrictlyNegative undefined (type interface {} is interface with no methods)
</span></code></pre>
Because we know what x</code> really is, we can use a type assertion:</p>
var </span>x </span>interface</span>{} </span>= </span>Number{}
</span>
</span>n </span>:= </span>x.(</span>Number</span>)
</span>
</span>// now the compiler knows that n is a Number
</span>// so this is fine
</span>n.isStrictlyNegative()
</span></code></pre>
If a type assertion fails (x</code> doesn't really implement Number</code>), it will trigger a panic:</p>
var </span>x </span>interface</span>{} </span>= </span>"not a number"
</span>
</span>n </span>:= </span>x.(</span>Number</span>)
</span>// panic: interface conversion: interface {} is string, not Number
</span></code></pre>
We can use the second return value of a type assertion to check if it was successful:</p>
var </span>x </span>interface</span>{} </span>= </span>"not a number"
</span>
</span>// this code will not panic
</span>n, ok </span>:= </span>x.(</span>Number</span>)
</span>
</span>// here, ok is false because the type assertion failed
</span>if !</span>ok { </span>... </span>}
</span></code></pre>
To perform multiple type assertions, we can use a type switch</em>:</p>
switch </span>v </span>:= </span>i.(</span>type</span>) {
</span>case </span>int:
</span>  fmt.Println(</span>"i is an int"</span>)
</span>case </span>string:
</span>  fmt.Println(</span>"i is an string"</span>)
</span>default</span>:
</span>  fmt.Printf(</span>"I don't know about this type </span>%T</span>!"</span>, v)
</span>}
</span></code></pre>
Type Aliases</a></h2>
You can create an alias to another type:</p>
type </span>H </span>= map</span>[</span>string</span>]</span>interface</span>{}
</span>
</span>// `map[string]interface{}` is a common pattern
</span>// now we can use `H` as a short form
</span>h </span>:= </span>H{</span>"one"</span>: </span>1</span>, </span>"two"</span>: </span>1</span>}
</span></code></pre>
This is different than a type definition:</p>
// type definition
</span>type </span>H </span>map</span>[</span>string</span>]</span>interface</span>{}
</span>
</span>// type alias
</span>type </span>H </span>= map</span>[</span>string</span>]</span>interface</span>{}
</span></code></pre>
An alias declaration doesn't create a new type. It just introduces an alias name - an alternate spelling.</p>
Struct Composition</a></h2>
Structs can be composed of one another through anonymous fields.</p>
For example, here we have a struct called Animal</code> that has a Talk</code> method:</p>
type </span>Animal </span>struct </span>{
</span>  feet </span>int
</span>}
</span>
</span>func </span>(a </span>Animal</span>) </span>Talk</span>() {
</span>  fmt.Println(</span>"generic animal sound"</span>)
</span>}
</span></code></pre>
And a struct called Cat</code> that embeds an Animal</code>. All of Animal</code>'s fields are promoted to Cat</code>:</p>
type </span>Cat </span>struct </span>{
</span>  </span>Animal
</span>}
</span></code></pre>
You can instantiate a new Cat</code>, setting the embedded Animal's</code> fields:</p>
cat </span>:= &</span>Cat{ Animal{ feet: </span>4 </span>} }
</span></code></pre>
And use any Animal</code> methods or fields:</p>
cat.Talk() </span>// => "generic animal sound" 
</span>cat.feet   </span>// => 4
</span></code></pre>
You can also override an embedded struct's methods:</p>
func </span>(c </span>Cat</span>) </span>Talk</span>() {
</span>  </span>println</span>(</span>"meow"</span>)
</span>}
</span>
</span>cat.Talk() </span>// => "meow"
</span></code></pre>
To imitate a call to super</code> as found in other languages, you can call the method through the embedded type directly:</p>
cat.Animal.Talk() </span>// "generic animal sound"
</span></code></pre>
This behavior is not limited to struct fields. A struct can embed a primitive:</p>
type </span>MyStruct </span>struct </span>{
</span>  </span>string
</span>  </span>int
</span>  </span>bool
</span>}
</span></code></pre>
Or any other type, such as a pointer:</p>
type </span>Cat </span>struct </span>{
</span>  </span>*</span>Animal
</span>}
</span></code></pre>
A collection (array, map, or slice):</p>
// remember, this is a type *alias*
</span>type </span>Assignments </span>= </span>[]</span>string
</span>
</span>type </span>Homework </span>struct </span>{
</span>  </span>Assignments
</span>}
</span></code></pre>
Or an interface:</p>
type </span>MyInterface </span>interface</span>{}
</span>
</span>type </span>MyStruct </span>struct </span>{
</span>  </span>MyInterface
</span>}
</span></code></pre>
Interface Composition</a></h2>
Interfaces can also be composed of each other:</p>
type </span>Animal </span>interface </span>{
</span>  </span>Talk</span>()
</span>  </span>Eat</span>()
</span>}
</span>
</span>type </span>Cat </span>interface </span>{
</span>  </span>Animal
</span>  </span>SomethingElse</span>()
</span>}
</span></code></pre>
Here, Animal's method set is promoted to Cat. This is the equivalent to saying:</p>
type </span>Cat </span>interface </span>{
</span>  </span>Talk</span>()
</span>  </span>Eat</span>()
</span>  </span>SomethingElse</span>()
</span>}
</span></code></pre>
Error Handling</a></h2>
Functions that can fail typically return an error</code>, along with their regular return value:</p>
file, err </span>:= </span>os.Open(</span>"foo.txt"</span>)
</span></code></pre>
Errors are values, so you can perform nil</code> checks on them. You are going to be seeing a lot</em> of this:</p>
file, err </span>:= </span>os.Open(</span>"foo.txt"</span>)
</span>if </span>err </span>!= </span>nil </span>{ 
</span>  </span>return </span>err
</span>}
</span>DoMoreStuff(file)
</span></code></pre>
You can create errors using the errors</code> package:</p>
import </span>"errors"
</span>
</span>var </span>err </span>error </span>= </span>errors.New(</span>"I am an error"</span>)
</span>println</span>(err.Error()) </span>// => "I am an error"
</span></code></pre>
Packages often export common error values. You can perform equality checks between errors:</p>
import </span>"errors"
</span>
</span>err </span>:= </span>makeDatabaseCall()
</span>if </span>errors.Is(err, database.ErrNotFound) {
</span>  </span>return </span>404
</span>}
</span></code></pre>
If code cannot continue because of a certain error, you can stop execution with panic</code>:</p>
if </span>err </span>!= </span>nil </span>{ 
</span>    </span>panic</span>(</span>"OMG!!!"</span>)
</span>}
</span>
</span>// panic: OMG!!!
</span></code></pre>
You can call recover</code> to regain control of a panic. recover</code> will return the value passed to panic:</p>
// if a panic occurs, this function will be called
</span>defer func</span>() {
</span>  </span>if </span>r </span>:= </span>recover</span>(); r </span>!= </span>nil </span>{
</span>    fmt.Println(</span>"Recovered from panic: "</span>, r)
</span>  }
</span>}()
</span>
</span>fmt.Println(</span>"Panicking"</span>)
</span>panic</span>(</span>"AAAHHH!!!"</span>)
</span>
</span>// => Panicking
</span>// => Recovered from panic:  AAAHHH!!!
</span></code></pre>
Goroutines</a></h2>
Go is capable of concurrency through goroutines</em>. A goroutine is a lightweight thread. To start a goroutine, you simple prefix a function call with the keyword go</code>:</p>
go </span>DoSomething()
</span></code></pre>
Goroutines are executed concurrently, not sequentially. For example, this code will take two seconds to complete:</p>
time.Sleep(time.Second </span>* </span>1</span>)
</span>time.Sleep(time.Second </span>* </span>1</span>)
</span>
</span>// exec time: 2 seconds
</span></code></pre>
But this code only takes one second, because both sleep calls are taking place at the same time!</p>
go </span>time.Sleep(time.Second </span>* </span>1</span>)
</span>time.Sleep(time.Second </span>* </span>1</span>)
</span>
</span>// exec time: 1 second
</span></code></pre>
Goroutines are often used with anonymous functions:</p>
go func</span>() {
</span>  fmt.Println(</span>"hello"</span>)
</span>}()
</span></code></pre>
Channels</a></h2>
Goroutines communicate through channels</em>:</p>
ch </span>:= </span>make</span>(</span>chan </span>int</span>)
</span></code></pre>
You can send values to a channel:</p>
go func</span>() { ch </span><- </span>1 </span>}()
</span></code></pre>
And receive values from a channel:</p>
x </span>:= <-</span>ch
</span></code></pre>
Note that data flows in the direction of the arrow.</p>
Here is an example of a simple multiplication function that communicates through channels:</p>
func </span>multiplyByTwo</span>(num </span>int</span>, result </span>chan <- </span>int</span>) {
</span>  </span>// calculate result
</span>  sum </span>:= </span>num </span>* </span>2
</span>  </span>// and send it through the channel
</span>  result </span><- </span>sum
</span>}
</span></code></pre>
We can create a channel and pass it to the function:</p>
result </span>:= </span>make</span>(</span>chan </span>int</span>)
</span>go </span>multiplyByTwo(</span>3</span>, result)
</span></code></pre>
The result channel now contains the number calculated by multiplyByTwo</code>:</p>
fmt.Println(</span><-</span>result)
</span>// => 6
</span></code></pre>
Channels can be created with a limit. If too many values are sent to the channel, the channel will block:</p>
limit </span>:= </span>1
</span>ch </span>:= </span>make</span>(</span>chan </span>int</span>, limit)
</span>
</span>// this is fine
</span>ch </span><- </span>1
</span>
</span>// but now the channel is full
</span>ch </span><- </span>2
</span>
</span>// fatal error: all goroutines are asleep - deadlock!
</span></code></pre>
Channels with a limit are called buffered channels</em>.</p>
You can also close a channel with the close</code> method:</p>
close</span>(channel)
</span></code></pre>
You can test whether a channel has been closed:</p>
v, ok </span>:= <-</span>ch
</span>// if `ok` is true, the channel is still open
</span></code></pre>
To receive all the values from a channel until it is closed, you can use range</code>:</p>
for </span>value </span>:= range </span>channel {
</span>  fmt.Println(value)
</span>}
</span></code></pre>
Handling Multiple Channels</a></h2>
To handle communicating with multiple channels, you can use select</code>:</p>
func </span>doubler</span>(c, quit </span>chan </span>int</span>) {
</span>  x </span>:= </span>1
</span>  </span>for </span>{
</span>    </span>select </span>{
</span>      </span>// send x to channel and double it
</span>      </span>case </span>c </span><- </span>x:
</span>        x </span>+= </span>x
</span>      </span>// ... until a value is sent to "quit"
</span>      </span>case <- </span>quit:
</span>        fmt.Println(</span>"quit"</span>)
</span>        </span>return
</span>    }
</span>  }
</span>}
</span></code></pre>
Let's test our program out by creating a channel, passing it to doubler</code>, and receiving the results:</p>
func </span>main</span>() {
</span>  </span>// create the necessary channels
</span>  c </span>:= </span>make</span>(</span>chan </span>int</span>)
</span>  quit </span>:= </span>make</span>(</span>chan </span>int</span>)
</span>
</span>  </span>go func</span>() {
</span>    </span>// receive five values from the channel
</span>    </span>for </span>i </span>:= </span>0</span>; i </span>< </span>5</span>; i</span>++ </span>{
</span>      </span>println</span>(</span><-</span>c)
</span>    }
</span>    </span>// and then quit
</span>    quit </span><- </span>0
</span>  }()
</span>
</span>  </span>// run doubler
</span>  doubler(c, quit)
</span>}
</span></code></pre>
It works!</p>
1
</span>2
</span>4
</span>8
</span>16
</span>quitting
</span></code></pre>
And with that, we have hit 20 minutes estimated reading time. After reading this, you should be able to read most of the Go code you find online.</p>
Thanks for reading!</p>

Designing A Fast Concurrent Hash Table

Learning Async Rust With Entirely Too Many Web Servers

A Lock-Free Vector

128-bit Atomics Are Practical Now

Astra: A Blocking HTTP Server Built on Top of Hyper

Extending Rust's Async Function Syntax

The Random Number Generator Hidden In Rust's Standard Library

An Alternative Syntax for Async Functions

Go in Twenty Minutes

From Active Record to Mongoid

Designing A Fast Concurrent Hash Table

Learning Async Rust With Entirely Too Many Web Servers

A Lock-Free Vector

128-bit Atomics Are Practical Now

Astra: A Blocking HTTP Server Built on Top of Hyper

Extending Rust's Async Function Syntax

The Random Number Generator Hidden In Rust's Standard Library

An Alternative Syntax for Async Functions

Go in Twenty Minutes

Looping</a></h2> Go has only one loop, the for</code> loop. It has three components:</p> init: executed before the first iteration</li> condition: evaluated before every iteration</li> post: executed at the end of every iteration</li> </ul> Here's an example:</p>

From Active Record to Mongoid