ForBlog – Salman Naqvi

Visualizing Compressed Latents

HTML

Diffusion

Analyzing Models

This notebook is the source code for a VAE visualization app I created. It allows you to see how the compressed latents look like, and tweak certain parameters that produce the latents. You can explore the APP here. Works best with images whose dimensions are multiples of 8.

Working with Bytes in Python

Programming

Python

How to create a temporary buffer in memory and how to work with data stored there.

Tunneling from SolveIt to your Machine

LLMs

Agents

SolveIt

Shell

This post details how one can provide SolveIt access to your local machine, thereby giving the LLM in SolveIt the context of your computer. Whilst the post focuses on macOS, the logic can be applied to other systems.

Ways to Pull Repos into LLM Context

LLMs

Agents

SolveIt

Good context is key when working with LLMs. No matter if you’re coding, writing, planning, problem solving, and whatnot. The LLM needs to know your context to help you effectively.

Building SolveIt in SolveIt

Agents

LLMs

SolveIt

Programming

HTML

“This blog post is a rendered documentation of a very, very basic re-implementation of SolveIt, using FastHTML as the web framework. The main goal of this exercise was to exercise my web development and HTMX skillz. Play with the app at this link, or watch the brief overview below.””

Translating SolveIt Dialogues

SolveIt

Mandarin

Here, I demonstrate how to update cells in SolveIt dialogs, at the sametime also demonstrating how such messages can be translated with a LLM.

LLM Agents are Python While Loops

LLMs

Agents

SolveIt

Here, I demonstrate how simple LLM agents are, how to implement one as a Python while loop using the Lisette and toolslm libraries.

Decomposing Problems and Working with Data will get us Working Robots

Analyzing Models

Creating Models

Data

Papers

Robotics

Reflection

Most robotic solutions I’ve encountered so far leave me feeling like I’m watching yet another Jenga tower of hacks. Blocks taped together for the purpose of solving the previous blocks’ issues, all without fixing the actual fundamental issue, which is the VLM.

Current Ideas in Spatial Understanding

Computer Vision

LLMs

Papers

Robotics

I went through a bunch of papers relating to enhancing VLM spatial understanding. Here, I’ve extracted the main ideas that have been explored. The worthiness/effectiveness of each idea has not been evaluated.

The State of Pose Estimation

Annotated Presentation

Robotics

Reflection

Computer Vision

This post is an annotated presentation I gave at my university’s multimodal intelligence lab. I’m a newcomber to general purpose robotics, so I had a different perspective going in.

中文里的一些基本数学术语

Mandarin

Mathematics

Some basic mathematical expressions in Chinese.

How Deepseek R1 was trained.

LLMs

Papers

Reinforcement Learning

In a nutshell, R1-Zero is a model created purely through reinforcement learning (RL), with Group Relative Policy Optmization (GRPO) as the policy. R1 is a model created with a combination of multiple RL and supervised finetuning (SFT) stages.

What I Learned during my Second and Third Internships

Reflection

LLMs

Tokenizers

Optimization

They were quite hands on, and involved tasks such as finetuning LLMs to generate Cantonese lyrics to the topic and tones specified.

Implementing a Neural Network from Scratch

Creating Models

Programming

Python

PyTorch

fastai

In this notebook, I implement a neural network from scratch and iteratively reimplement with PyTorch. That is, I will implement each element of the training and inference process from scratch, before using the corresponding PyTorch element.

A Brief Token on Tokenizers

LLMs

Tokenizers

Tokenization is the process whereby text is given a numerical representation. Sentences are split into components known as tokens. These tokens represent numerical values that language models can work with.

Backpropagation Explained using English Words*

*Most words are in English.

Mathematics

Creating Models

Backpropagation sounds and looks daunting. It doesn’t need to be. In fact, it’s just a fancy term for the chain rule. Implementing backpropagation is implementing one big fat chain rule equation.

Implementing and Optimizing Meanshift Clustering

Creating Models

fastai

Optimization

A guide to grouping data together — fast.

Einstein Summation Notation, Intuitively

Mathematics

Programming

An alternative way to write matrix operations.

(Un)successfully Implementing DiffEdit

Diffusion

Creating Models

Papers

fastai

Here I attempt to implement the DiffEdit paper: a diffusion algorithm that allows you to edit an image with text.

Iterators and Generators

Python

Programming

Iterators and generators shown by example.

Implementing Stable Diffusion From Its Components

Diffusion

Creating Models

fastai

In this notebook, I’ll implement stable diffusion from its various components with Hugging Face’s diffusers library.

Stable Diffusion, Summarized

Diffusion

Creating Models

A concise, high level overview on the mechanisms of stable diffusion.

My Musings Through Stable Diffusion

Diffusion

Analyzing Models

Exploring the various knobs and dials of stable diffusion.

Converting Audio to Images

Computer Vision

Data

PyTorch

Here I describe how one can create images from audio by visualizing them as spectrograms, with PyTorch’s torchaudio module.

A No Nonsense Guide on how to use an M-Series Mac GPU with PyTorch

M-Series Macs is better than saying M1/M2 Macs

PyTorch

Squeezing out that extra performance.

Adding Subscriptions to a Quarto Site

Subscribable Subscriptions

Quarto

Collaboration

Three ways to add subscriptions to your Quarto website.

Isaac Flath, Salman Naqvi

AI in a Nutshell

Creating Models

fastai

This nutshell contains very little math! Artificial intelligence. Machine learning. Neural networks. Deep learning. Fancy words. Deceptively simple.

Detecting Floods for Disaster Relief

Creating Models

fastai

Here I create a model that can inform disaster relief personel whether an area shown in an image is flooded or not.

Data Matters More than the Model

Data

Analyzing Models

I created a car classifeir that organized cars to their respective brands. Despite my training set having close to 5000 images, my model having 100 layers, and my traning loop having 20 epochs, I still had a 17.4% error rate.

Reading a Confusion Matrix

Analyzing Models

A to the point guide about reading a confusion matrix.

My first AI model

fastai

My first ever AI model that can tell apart grizzly bears, black bears, or teddy bears.

How to Look at Predictions

Data

Reflecting

A different way to look at data, models, and predictions