Pranav V Bhat

August 2026 Rust Blr meetup

Pranav V Bhat — Sat, 22 Aug 2026 00:00:00 +0000

Building a WebAssembly Interpreter from Scratch in Rust - Bhavya Bhatt

Existing WASM runtimes are fast but not really observable
Goal was to build an interpreter where stack traces can be mapped back to source lang, not WASM
WASM basically solves what Docker is doing

Languages with a Garbage Collector have to compile their whole runtime along with the application code into WASM to work. Rust has no GC => Better WASM support
Languages targeting WASM are truly cross platform
WASM code structured as:

type section (
    Generic function type definitions that show how many params
    and how many results different functions of that type can have
)

imports section

instructions/function body

exports section

TraceWASM is a stack based VM
A stack holding data references and instructions + memory storing arbitrary data achieves Turing Completeness
WASM memory is just a Vec<u8>.
To be used, a value must be first pushed onto the top of the stack but values in the locals space of the stack may be randomly accessed by the programmer to be pushed to the top

WASM instructions do 3 things: push 0 or more instructions onto the stack, pop 0 or more instructions onto the stack, read data from memory
A WASM interpreter is basically just

loop {
    fetch(instructions)
    pc = exec_instructions(&mut stack, &mut memory) // Mutable references to the stack and memory

    if pc == end { break }
}

Basic syntax for an instruction: <Required stack>.<Instruction> <Value>
Required stack include stuff like local (Function params, local vars etc.), global (Global vars), i32 (Perform work on values of type i32) and other data types
Labels are used to define blocks. Essentially functions and include branches, loops etc. Has to be terminated with end
Can reference other blocks by using relative depth indexes where 0 is the index of the parent block of the branch, 1 is the grandparent and so on

(block ;; Scope 1 (Index 1)
  (block ;; Scope 0 (Index 0)
    br 1 ;; Jumps to the end of Scope 1
  )
)

Parameters of block branched to are left at the top of the stack after branching
TraceWASM supports generic memory to be used. Can program linear, cache hirearchial or distributed memory and observe stack traces for each
Is event based
Register based VMs (like Google's V8) allow local values to be directly accesses instead of first pushing onto the stack. Can perform tasks in lesser instructions but requires more memory accesses and hence it is less cache friendly

Building PgPulse as a Native PostgreSQL Extension - Harshit Ruwali

blog.sp00fexpl01t.rocks

Lost in Tokio

Pranav V Bhat — Fri, 21 Aug 2026 00:00:00 +0000

Here, I try to get an abstracted overview of Tokio's architecture to build the foundation needed to understand its internals in depth later. To understand Tokio and why it exists, we must first look at the problems it was built to solve. We'll start with the simplest model of execution and gradually introduce the abstractions that lead us to an async runtime.

Level 0: Synchronous programming

Most code that you write is executed sequentially

fn synchronous() {
    println!("1");
    println!("2");
    println!("3");
}

Completely innocent synchronous function

This synchronous way is perfectly fine for most tasks, but some operations (like network requests or I/O waits) in the chain can be painfully slow. They 'block' the program from progressing until they are done, resulting in your application just sitting there doing nothing.

fn evil_synchronous() {
    println!("Requesting user data...");
    
    // Execution cannot continue until the database responds.
    let response = get_from_db("Geronimo").unwrap(); 
    
    println!("Got data: {response}");
}

Evil and intimidating blocking code

Blocking delays like this are common when applications wait for I/O operations to finish. But what if your program could do other work while it waits?

Level 1: Concurrency and Parallelism through OS Threads

A very naive way to do this would be to create a new process for each task. But this would be very expensive as a new process would require its own isolated memory context and common data would have to be passed between these processes.

Instead, we use multiple threads inside a single process.

Thread: The smallest sequence of programmed instructions that can be managed independently by a scheduler.

While not as isolated as separate processes, each thread has its own program counter, register set and stack space but share memory space and resources with other threads within the same process. Threads can be either handled in userspace by the process spawning them or directly by the OS kernel. Kernel level threads can be independently scheduled by the kernel, allowing for parallel execution when multiple cores are available but they also have higher scheduling and context switching overhead.

In Rust, threads are exposed for use via the native std::thread interface. A thread spawned by std::thread maps 1:1 onto a kernel level thread scheduled and managed not by Rust but by the OS.

use std::thread;
use std::time::Duration;

fn main() {
    let handle = thread::spawn(|| {
        for i in 1..10 {
            println!("Spawned thread {i}");
            thread::sleep(Duration::from_millis(1));
        }
    });

    for i in 1..5 {
        println!("{i} from the main thread");
        thread::sleep(Duration::from_millis(1));
    }

    handle.join().unwrap(); // main thread should not exit until all spawned threads are done
}

Concurrent execution with OS threads (From doc.rust-lang.org/book/ch16-01-threads.html)

[!NOTE] The basic difference between Concurrency and Parallelism

Concurrency is about structuring multiple independent tasks to execute and progress in overlapping time periods. On a single core, the OS achieves this by rapidly switching between tasks.

Parallelism is when tasks are literally run at the same time across multiple CPU cores.

Although OS threads provide concurrency (and parallelism on multi-core hardware) and are cheaper than creating an entirely new process, they still are relatively expensive.

Historically, the simplest way to handle network traffic was to spawn one OS thread per connection. However, OS threads are heavy. On Linux, each thread reserves a default 8MB of virtual memory for its stack. If an application tried to serve 10,000 concurrent connections this way, it would demand 80GB of virtual address space.

But virtual address space is cheap and abundant on modern systems. The real problem is that the OS kernel has to constantly pause and resume these threads (context switching). This is significantly more expensive while also potentially invalidating cache locality. The CPU would spend all its time just juggling threads rather than doing actual work.

Level 2: Cooperative Multitasking with async/.await

[!NOTE] Preemptive VS Cooperative multitasking

Preemptive multitasking: The OS allocates each thread a time slice to execute in and forcibly pauses the thread when its time is up no matter what it is doing and runs the next scheduled thread.

Cooperative multitasking: Each task voluntarily yields control back when it is idle or has hit a blocking point, giving us a lower context switching overhead.

Luckily for us, Rust provides a the Future trait as an abstraction for asynchronous work. Futures are analogous to a Promise from JavaScript, with the main difference being that a Promise is eagerly executed by the JavaScript runtime while a Future is lazy until it is polled.

Polling is basically giving the future the opportunity to progress by asking, "Hey, make some progress on your work now" The Future can then respond with either "No, I can't progress now" (Poll::Pending) or "Yes, I'm done. Here is the result" (Poll::Ready(val)).

Rust gives us the async/.await syntax, allowing us to write asynchronous code in a way that looks similar to synchronous code. This syntax will be familiar if you're coming from JavaScript or Python.

For example:

fn synchronous_io() {
    let resp = fetch_data();
    println!("{resp}");
}

Standard, blocking I/O

Can be written as:

async fn asynchronous_io() {
    let resp = fetch_data_async().await;
    println!("{resp}");
}

Asynchronous I/O using async/.await

As you can see, the main differences are the async keyword in the function definition and the .await postfix operator after an async function call.
But what exactly are they doing?

async transforms your function into a state machine that implements the Future trait. Each .await marks a suspension point and the boundary between different states, allowing the state machine to pause and resume at these points. This state machine also stores context such as local variables, and child Futures that are being awaited.

When execution reaches an .await, the future being awaited is polled. If it is ready, execution continues normally. Otherwise, the state machine saves its current state, returns Poll::Pending to the caller and yields control so that other work can be done while waiting. Before doing so, the awaited future typically stores a Waker that can later be used to arrange for the task to be polled again when progress becomes possible.

Later, when the async function is polled again, the state machine resumes execution from the previously saved state.

Very neat! Now let us run this function.

fn main() {
    // Remember we need to await a Future to progress it since they are lazy
    asynchronous_io().await;
}

Oh no! The Rust compiler requires any function that calls an async function to also be declared with async

error[E0728]: `await` is only allowed inside `async` functions and blocks
 --> src/main.rs:6:23
  |
5 | fn main() {
  | --------- this is not `async`
6 |     asynchronous_io().await;
  |                       ^^^^^ only allowed inside `async` functions and blocks

For more information about this error, try `rustc --explain E0728`.

This is because .await is a potential suspension point. If the awaited Future isn't ready yet, the caller must save its current state, yield control, and later resume execution from where it left off. Ordinary functions are not capable of doing this, only functions marked with async are.

So no worries, we will just declare main also with the async keyword. Unfortunately, this too does not work.

error[E0752]: `main` function is not allowed to be `async`
 --> src/main.rs:5:1
  |
5 | async fn main() {
  | ^^^^^^^^^^^^^^^ `main` function is not allowed to be `async`

For more information about this error, try `rustc --explain E0752`.

The reason main cannot be async is that someone has to drive the Future returned by main also to completion.

An async fn doesn't execute by itself. Calling it just constructs a value containing all the state required to perform the work, but not when. Futures are lazy, so unless something repeatedly polls them, they never make progress. So who does the polling?

Level 3: Async runtimes

This is where an async runtime comes into play. Most languages that support async have an async runtime built into the core language runtime and thus support async functions out of the box. Rust on the other hand provides only the foundation such as the Future and the async/.await syntax but no async runtime.

This is mainly due to Rust being used in many different areas from web development to systems and bare metal/embedded. There no consensus on a "One True Async Runtime" capable of supporting all of them perfectly. Instead, it is up to the developer to choose from many different community provided crates tailored to their use case.

"Rust caters to a vast array of use cases. We simply cannot bundle everything into the core standard library, but the ecosystem provides a crate for almost every need. Just use one of those"

— Paraphrased quote from Niko Matsakis, Core developer on the Rust programming language

Tokio is the mostly widely used async runtime at the time of writing and thus will be our focus.

[!WARNING] Disclaimer: Tokio is actively being developed, so some information here may quickly become out-of-date. We will be focusing on parts of tokio-rs/tokio v1.53.1 for the rest of this blog

The role of any async runtime is to schedule futures for polling, react to them waking up and coordinate the resources required by the futures to make progress.

The executor is the component of an async runtime responsible for repeatedly polling futures.

loop {
    match future.poll(&mut context) {
        Poll::Ready(_) => break,
        Poll::Pending => {
            // wait for a wakeup
        }
    }
}

A very simple executor

This works well for one task but how does it scale?

Similar to what we have seen with threads above, real applications almost never have only 1 future. We may have thousands of tasks, many of which are waiting on I/O while only a small number are actually ready to run. But like with threads, how does this solve the problem of either consuming too much memory or slowing down from a lot of context switches?

Tokio tasks, unlike threads are very lightweight and are managed by the Tokio runtime, not the OS scheduler. Because tasks are scheduled in userspace by Tokio, switching between tasks does not require OS thread context switches and has a low overhead. They are also cooperatively scheduled rather than preemptively scheduled.

A task is a light weight, non-blocking unit of execution.

Generally, this pattern is known as green threads and is similar to Golang's goroutines. This way, a lot of tasks can be run on a handful of OS threads. A new problem arises now, how to keep track of which tasks are ready to run and which thread it can run on? This leads us to the obvious solution: A scheduler.

The Tokio Scheduler

This scheduler is responsible for deciding which runnable task should be executed next. In a simple runtime, this could be as easy as maintaining a queue of ready tasks and repeatedly choosing one to poll. In Tokio, which is a multi-threaded runtime, the scheduler has to coordinate M tasks (theoretically unlimited) across N threads (limited amount). Having only a global queue means every worker threads has to contend for access, increasing synchronization overhead.

[!NOTE] Tokio, by default, is multi-threaded but can be configured to be a single-threaded event loop AKA current_thread which can actually be easier to work with in most cases as argued here

As you can see, Tokio solves this by having a global queue of tasks (implemented as a FIFO linked list) shared between all threads along with local queues for each thread/worker.

/// Growable, MPMC queue used to inject new tasks into the scheduler and as an
/// overflow queue when the local, fixed-size, array queue overflows.
pub(crate) struct Inject<T: 'static> {
    shared: Shared<T>,
    synced: Mutex<Synced>,
}

Global 'Injection' queue definition (From tokio/src/runtime/scheduler/inject.rs#L21-L26)

The local queue is a ring buffer which can hold upto 256 tasks at a time. When this local queue overflows, roughly half of the tasks from the local queue are moved to the global queue and held. This serves as spillway to catch overflowing tasks. Any task that wakes up from a thread which is not a worker thread is also placed into the global queue thus acting as a shared entry point too.

/// Producer handle. May only be used from a single thread.
pub(crate) struct Local<T: 'static> {
    inner: Arc<Inner<T>>,
}

/// Consumer handle. May be used from many threads.
pub(crate) struct Steal<T: 'static>(Arc<Inner<T>>);

pub(crate) struct Inner<T: 'static> {
    /// Concurrently updated by many threads.
    ///
    /// The `UnsignedShort` indices are intentionally wider than strictly
    /// required for buffer indexing in order to provide ABA mitigation and make
    /// it possible to distinguish between full and empty buffers.
    ///
    /// When both `UnsignedShort` values are the same, there is no active
    /// stealer.
    head: AtomicUnsignedLong,

    /// Only updated by producer thread but read by many threads.
    tail: AtomicUnsignedShort,

    /// Elements
    buffer: Box<[UnsafeCell<MaybeUninit<task::Notified<T>>>; LOCAL_QUEUE_CAPACITY]>,
}

Local queue definition (From tokio/src/runtime/scheduler/multi_thread/queue.rs#L28-L57)

A worker first checks its local queue for any runnable tasks and only checks the global queue if it runs out of tasks or after a configurable number of local tasks have been scheduled.

This can be compared to how cache locality works. First try to retrieve from the closest source and if not found, move to more distant sources. And similar to how you reach for data from memory when it is not in cache, worker threads reach to steal tasks from other workers.

A worker may run out of tasks in both its local and the global queue even while another worker still has a large number of runnable tasks. To keep the workload balanced, an idle worker can steal tasks from another worker's local queue. Tokio moves roughly half of the tasks during stealing rather than just taking a single task.

The stealing operation immediately returns the last task in the stolen batch to the thief for execution and then continues normally.

A neat optimization is that each worker has a single element task slot. Any task placed in this slot can bypass both the local and global queue and gets executed first in the next iteration. This effectively results in the last scheduled task to be run next (LIFO). This optimization improves cache locality which benefits message passing patterns and helps to reduce latency.

So in the above diagram, Worker 2 would execute Task 3 first and then Task 1 and Task 2 ([3]->[1]->[2]). This lets the thief begin executing immediately rather than enqueueing the entire stolen batch and then performing another queue operation to obtain its first task.

Work-stealing involves concurrent, unsynchronized access to head and tail across threads. Because stolen tasks cross thread boundaries, any task spawned on a multi-threaded runtime is forced to satisfy Send + 'static bounds. Single-threaded Tokio on the other hand requires only 'static to be satisfied since there is no work stealing.

Actually running Tasks

We've established before that Tokio tasks are lightweight units of work. These tasks are distributed among workers when they are runnable. But tasks do not remain runnable forever. Tokio is a runtime designed to handle asynchronous I/O bound tasks which spend most of their lifetime waiting for something to happen.

When a task reaches a state where it cannot make any progress without waiting, it returns Poll::Pending. The I/O operation registers interest in the underlying OS resource, while the task provides a Waker that can be used to schedule it again when that resource becomes ready.

A Waker is essentially a handle to something that knows how to make a suspended task runnable again. Internally, Rust represents this through a RawWaker containing a data pointer and a RawWakerVTable. The vtable tells the runtime what to do when the waker is cloned, woken, referenced without consuming it, or dropped.

use std::task::{RawWaker, RawWakerVTable, Waker};

fn wake_task(ptr: *const ()) {
    // schedule the task
}

const VTABLE: RawWakerVTable = RawWakerVTable::new(
    clone,
    wake_task,
    wake_by_ref,
    drop,
);

The vtable and raw waker are defined in the standard library. Tokio's custom vtable from tokio/src/runtime/task/waker.rs#L118-L119

Tokio represents a task as a single heap allocation containing the task header, the future, output/state, and scheduler information. A Waker wraps a raw pointer (NonNull<Header>) with a custom RawWakerVTable. Calling .wake() executes an atomic state transition directly on the task’s Header flags. If the transition succeeds and the task is woken, the task memory pointer is re-enqueued into a scheduler queue.

The I/O driver waits for events from the OS for the registered resources and wake the tasks when they become available. Tokio does not actually handle each specific I/O driver by itself but instead relies on the mio crate to abstract system specific drivers and provide a common API for all of them.

Note that the I/O driver mechanism is not controlled by a dedicated thread but is integrated into each and every worker.

I/O is only one source of task wakeups. Timers and asynchronous synchronization primitives(tokio::sync::*) can also cause a pending task to become runnable again. A timer can wake a task when its deadline expires, while primitives such as channels, notifications, and semaphores can wake tasks when the state they are waiting for changes.

When we are truly jobless

A worker which has finished all its tasks, has no tasks left in its local queue, no tasks to get from the global queue and nothing to steal from other workers is freeloading on precious CPU power. We need to be able to hibernate the worker until it is needed to handle more tasks. Constantly checking for new tasks is just wasteful so Tokio has a park/unpark mechanism for workers to transition into a sleeping state and block instead.

This is handled by a dedicated runtime::park module using a shared runtime driver and a conditional variable (Condvar) as a fallback since the driver is a shared resource and can be used by only one worker at a time either for I/O related or timing related wakeups. park()/unpark() calls are coordinated by an atomic state machine

const EMPTY: usize = 0;
const PARKED_CONDVAR: usize = 1;
const PARKED_DRIVER: usize = 2;
const NOTIFIED: usize = 3;

Atomic state machine states

EMPTY: Nothing is parked and no pending notification
PARKED_CONDVAR: Worker is about to/has parked on the condition variable
PARKED_DRIVER: Worker is parked through the runtime driver
NOTIFIED: A wake-up has been issued

They are connected as below

This state machine is required to prevent a race condition where a wake up notification is issued just before the worker actually sleeps, causing it to sleep forever.

Fin.

Putting it all together, let us follow a single asynchronous operation through Tokio's event loop

clicks

Pranav V Bhat — Mon, 27 Jul 2026 00:00:00 +0000

May 2026 Rust Blr meetup

Pranav V Bhat — Sat, 16 May 2026 00:00:00 +0000

Async runtimes revisited - Sanchayan

Why async?

Regular:

Not scalable
1 OS thread ~= 8MB ⇒ Can’t use many threads for concurrency otherwise too much RAM consumption
Kernel level threads used ⇒ Context switch overhead

Async:

Handful of OS threads can handle millions of concurrent tasks
Yield control to Executor on I/O wait instead of blocking thread in Async model

Async Rust

Lazy evaluation until future is polled (using .await)
- Unlike other languages like JS in which an async fn is evaluated as soon as it is called.
- Avoids overhead
- Async runtimes provide Executors to handle polling of futures until it finishes execution
No callback hell thanks to async/await syntax
- Zero cost state machines
epoll/kqueue (Readiness model):

[!NOTE] Slightly related: nice video on file descriptors

io_uring (Completion model):
- Don’t have to explicitly call read on ready fd
- Just read final result directly from Completion queue when OS async finishes execution and notifies

Async strats

tokio
- M:N tasks:threads
- Multithreaded by default but can use single (current) thread
- epoll mechanism (Readiness model)
- Technically also thread-per-core
compio/glommio/monoio
- What is generally known as “thread-per-core”
- io_uring mechanism mainly (Completion model)
- compio also supports IOCP (Windows’ completion model I/O interface)

Readiness model - Data not returned until read called

Process - Is x ready?
epoll - Yes
Return x to process and process calls read to get final product

[!NOTE]

2 syscalls per I/O task lifecycle (epoll_wait and read) ⇒ Context switch overhead

epoll_ctl is not counted because it is an amortized, one-time setup cost

No batching

Completion model

Process - I want to do x
SQueue - Ok will do
CQueue - Return final product to process

[!NOTE]

Batch submit all tasks in 1 syscall to SQueue (io_uring_enter. Not required if SQPOLL setup)

But no syscall per I/O task

No need for syscall to read from CQueue

Work stealing in the Readiness model

Research paper - Scheduling multi-threaded computations by work stealing
Blog - Making the Tokio scheduler 10x faster
Basic idea: Idle threads steal work from busy threads’ queue
- Steal oldest waiting tasks (queue head) to minimize contention
- Current thread executes next task remaining after stealing
  - Better for cache locality
  - Emulates call stack

Good load balancing
Synchronization overhead between threads + Cold cache penalty in new thread
Future has to implement Send + 'static trait bounds + also Sync if concurrent sharing via references
- Send bound can be relaxed by running only on current thread using tokio::task::LocalSet but not truly parallel

Readiness model good to use generally with mixed workloads and unknown load patterns

All async runtimes are thread-per-core (TPC)
Difference between tokio and the other so-called “thread-per-core” runtimes is that they are also share-nothing ⇒ No work stealing
One thread pinned to each CPU core
No synchronization overhead, better cache locality, no contention and predictable tail latency but can cause load imbalance
Need for app level partitioning of connections
- But not all workloads can be partitioned
- Shared state managed through message passing
Nice reddit thread on implications of completion vs readiness
compio v nice TPC+SN runtime because actively maintained + supports IOCP
Completion good for
- Data plane workloads
- I/O bound workloads
- Highly cache sensitive workloads
- When you can partition cleanly
- Does your tail latency matter?
Cross-core communication is expensive since message passing through channels instead of shared state
Tokio handles a lot of details in bg but has to be manually handled in compio
- Basically async hard mode but offers way better performance for I/O heavy workloads in return
Buffer reference shared (borrowed) in Readiness model whereas buffer has to be owned by the kernel in the Completion model until I/O is complete
- Problem for future cancellation in Completion, Has to be explicitly handled
  - When cancelled in Readiness, reference goes out of scope and ends
  - In Completion, kernel still has pointer to that memory ⇒ Use-After-Free if it has been deallocated when cancelled
- Incurs “Ownership tax” + can’t do concurrent reads with the same buffer
  - Ownership tax = Can’t easily share buffer, Have to handle returned buffer to reuse, Buffer pooling has to be done to avoid constant allocations
Usually tokio is good enough. Consider Completion model runtimes if I/O is bottleneck or cache issue

All links: https://adihegde.com/til/rust-blr-16-may-2026

Cracking games and how to make them crack proof - Ishan

For local first, offline apps

Bake a password into the binary
- Easily cracked by listing strings in the binary
- Can also patch the binary to bypass password checks
- Only problem is takes a bit of time to search for the exact string or the exact location to patch
Signed license. Pubkey baked into bin. Verify with privkey on app launch
- Cracked by patching code to bypass check or replace pubkey with your own pubkey
Hardware fingerprinting. App runs only on one specific system. Verify using system specific details like MAC addr
- Bypass by spoofing fingerprints
Harden during LTO. Distorts binary a bit making it difficult to understand decompiler output
- Fat LTO
- Abort on panic instead of unwind (No stack traces to read and get info)
- Strip debug symbols

Checks usually done by dynamic libraries in games so patch in your own DLLs in windows instead of the original ones or use LD_PRELOAD to load your own libraries and bypass checks.

Building a simple Load Balancer in Rust

Pranav V Bhat — Sat, 10 Aug 2024 00:00:00 +0000

Why do I need a Load Balancer?

Let's say you have a few servers and are hosting a website. Great! Soon your website becomes popular and gets a lot of visitors daily. All well and good until your servers start to become overwhelmed with requests and die. How to fix this? By putting a Load Balancer in between the clients and your servers.

A Load Balancer distributes incoming network traffic and distributes them across multiple servers to ensure no single server is overwhelmed thus optimizing reliability and resource utilization.

A Load Balancer can be physical or a software. It can be further classified base on which layer of the OSI model they operate at.

As part of the Tilde 3.0 Summer mentorship program, the Bal.rs (Pronounced: /ˈbɔːləz/) team have built a simple L7 Load Balancer in Rust. Rust was chosen due to it's performance and safety while provding low level control over the system.

Getting started with Bal.rs

Prerequisites

Building the Application Locally

Clone the repository and build the application using cargo.

git clone https://github.com/homebrew-ec-foss/bal.rs
cd bal.rs
cargo build

For a production-ready build, you can use:

cargo build --release

Using the Application

After building, the main executable will be located in /target/debug or /target/release based on the build command used. Navigate to the directory and type

Balrs help start

in the terminal to get a list of available commands.

Alternatively, from the root directory of Bal.rs, you can use:

cargo run help start

for the same result.

While you can configure the Load Balancer using the command line interface, more configuration options are available through the config.yaml file and multiple different config files can be created.
This feature enables the use of various configuration profiles without altering the original configuration. The desired profile can be specified through the CLI.

Technical Details

This section covers only the lb.rs file which contains the actual Load Balancing logic.

There are 3 key components of our Load Balancer:

Listener: Listens for incoming HTTP requests.
Routing: Does the actual load balancing by forwarding the client request to the servers.
Fault Tolerence: Makes sure the Load Balancer handles any faults gracefully.

Listener

We have used Rust's tokio crate to handle asynchronous processing and the hyper crate for networking. Tokio's TcpListener is used to listen for incoming connections

In this code snippet, we create a TcpListener instance to listen for incoming traffic and set it to listen on the address of the Load Balancer.
If the listener is bound to the Load Balancer successfully, we return the listener object for passing incoming requests to the handle_request function or else the error encountered is displayed.

Routing the Connections

There are 3 functions dealing with client requests.

Handling incoming requests: handle_request function

We lock the LoadBalancer instance to access the server list and filter out any dead servers.
The function then tries to pass the request to the get_request function. If this fails, a message is logged and the loop restarts. If there are no available servers, a HTTP 500 response is returned.

This is a dynamic fault tolerence system that reroutes an incoming request to a different server if one server is not available.
Forwarding requests to server: get_request function

Gets indexes of the servers and selects the server to be used according to the specified algorithm.

Here, the server URL is constructed. Requests are then forwarded to the server using the send_request function. Along with that, a timer is started to measure server response time. The server response is stored in the data variable.

Here, we handle variants of the server response. If we get a successful response, we return the response data or else mark the corresponding server as dead and return None.
Retrieve server response: send_request function

Parse the URL from the request and extract host and port from it. The port defaults to 80 if not specified. Then format the address to a string for a TCP connection.

Establish a TCP connection to the formatted address and wrap it in a tokio IO adapter so that it can be used with hyper.
A hyper client is then initialised using a HTTP/1 handshake.

The HTTP request is prepared with the given URL and HOST header and sent using the hyper client.

The server response body is then collected in chunks and appended to full_body. The complete response is then converted to Bytes and returned.

Fault Tolerence

This is a slightly large piece of code that ensures smooth functioning of the Load Balancer. So, let's break it down.

Here, we create variables to store the required configuration values and clone a LoadBalancer instance for further use.

Spawn an asynchronous tokio task for the health checker and create a vector to hold other tokio tasks.
Each task corresponds to the health check for each server. This is done to ensure all health checks happen simultaneously. We create the required number of tasks using a for loop where len is the number of servers listed in the Load Balancer's configuration.

Inside the task for each server, we retrieve and update relevant server data.

Using Instant::now() and calling .elapsed(), we record the server response time. We check if the server is responding by sending a GET request to each server with a set timeout and then update the server response time.
The subtraction from lb.servers[index].connections is done as to not count the connection opened by the health checker.

This match block is used to handle the result of the HTTP request sent by the health checker.
If the HTTP request is completed sucessfully, we check if the response is an error code(like 404) or if the maximum connections limit is exceeded. This leads to marking of a server as dead and will not be used by the Load Balancer until it is checked again and marked as alive by the health checker.

This marks the end of the Health Checker. After this, all the server tasks are awaited on to be periodically executed.
Here is a simple flowchart of how the health checking process works:

Health checker reports as displayed in the terminal:

Benchmarks

We conducted several tests at different request rates per second (RPS).

Throughput VS Time at 20,000 RPS

Throughput VS Time at 25,000 RPS

Throughput VS Time at 27,000 RPS

The tests had to be stopped here due to our hardware limitations but the trends we observed show us that the Bal.rs Load Balancer can handle much higher loads.

Next Steps and Resources

Pranav V Bhat

Pranav V Bhat — Thu, 01 Jan 1970 00:00:00 +0000

Pranav V Bhat

August 2026 Rust Blr meetup

Building a WebAssembly Interpreter from Scratch in Rust - Bhavya Bhatt

Building PgPulse as a Native PostgreSQL Extension - Harshit Ruwali

Lost in Tokio

Level 0: Synchronous programming

Level 1: Concurrency and Parallelism through OS Threads

Level 2: Cooperative Multitasking with async/.await

Level 3: Async runtimes

The Tokio Scheduler

Actually running Tasks

When we are truly jobless

Fin.

clicks

May 2026 Rust Blr meetup

Async runtimes revisited - Sanchayan

Why async?

Async Rust

Async strats

Readiness model - Data not returned until read called

Completion model

Work stealing in the Readiness model

Thread-Per-Core and Share-nothing

Cracking games and how to make them crack proof - Ishan

Building a simple Load Balancer in Rust

Why do I need a Load Balancer?

Getting started with Bal.rs

Prerequisites

Building the Application Locally

Using the Application

Technical Details

Listener

Routing the Connections

Fault Tolerence

Benchmarks

Next Steps and Resources

Pranav V Bhat