In the previous post, we discussed how to quantize a neural network to half-precision floating point numbers and even 8 bits. This time, we will push the boundaries further by quantizing the network to integers, enabling us to further minimize the space it occupies. Let’s get started with some theory and then we will exercise it on a few examples.

Neural networks can take up a lot of space and parameters are usually stored in Floating Point 32 format. For instance, there are Llama 3.1 7B, 40B and 405B models, where B stands for billions. Each FP32 parameter occupies 32 bits, which is equal to 4 bytes. Thus, the three mentioned versions of LLMs would need 28 GB, 160 GB and 1’620 GB RAM, respectively.

I recently came across a fascinating example in The Book of Why by Judea Pearl and Dana Mackenzie that really made me think about how counterintuitive probability can be. The setup is simple: you’re on a trip to Zanzibar with a tight connection in Frankfurt, and the question is whether your luggage will actually make it to your final destination. There’s a 50% chance your bag makes the transfer and, if it does, a certain chance you’ll see it on the carousel within the first few minutes. Seems straightforward, right? But as I started running through the probabilities in my head, I realized just how easily our intuitions can lead us astray. In this post, I’ll take you through the Bayesian approach to this scenario and show how it helps clarify a problem that feels almost paradoxical at first glance. Let’s dive in and see why waiting for luggage is a perfect puzzle for understanding the quirks of probability.

Hidden Markov Models (HMMs) are a powerful tool for analyzing data where hidden states influence observed outcomes. Imagine tracking how much fuel you use based on which gas station you visit; each station (state) has its own effect on fuel efficiency. By setting up an HMM with two states, we can train the model to identify these underlying states in the data. In this guide, I’ll show you how to set up and use an HMM in Python with the hmmlearn package, and how it can reveal patterns in data that might otherwise stay hidden.

Calculating the Mean Squared Displacement (MSD) is essential for understanding the dynamics of particles, molecules, and bodies across scientific fields like physics, chemistry, and biology. MSD analysis helps in quantifying how far particles move over time, distinguishing between different types of motion, and identifying diffusion characteristics. However, for large datasets or long time trajectories, the traditional MSD calculation approach can be computationally intense, making the analysis slow and inefficient. This post explores faster approaches to MSD computation, specifically using methods like the Fast Fourier Transform (FFT) and Fast Correlation Algorithm (FCA), which reduce the time complexity from N squared to N logN. By optimizing MSD calculations, we can handle larger datasets more effectively, enabling more detailed and efficient studies in particle dynamics.