Thinking Different, Thinking Slowly: LLMs on a PowerPC Mac

There is something incredibly satisfying about breathing new life into old hardware. Vintage computing is one of my favorite hobbies. The challenge of coaxing modern software onto systems designed decades ago is a puzzle I cannot resist. I have been diving into the world of large language models (LLMs), and a question began to gnaw at me: could I bring the cutting-edge of AI to the nostalgic glow of my trusty 2005 PowerBook G4? Armed with a 1.5GHz processor, a full gigabyte of RAM, and a limiting 32-bit address space, I embarked on an experiment that actually yielded results. I have successfully managed to achieve LLM inference on this classic piece of Apple history, proving that even yesteryear's hardware can have a taste of tomorrow's AI.

PowerBook G4 running TinyStories 110M Llama2 LLM inference

I started by reviewing the llama2.c project from Andrej Karpathy. This brilliant project implements Llama2 LLM inference with just a single file of vanilla C. No accelerators here. Performance is traded for simplicity which makes it easy to understand how inference is carried out.

I forked the core implementation to a project that I have titled ullm. The core algorithm remains the same, but I spent time improving a few aspects of the code so that it would stand up to abuse a little better.

12V: Ryobi 40V Lithium Battery Charger

I have been impressed with Ryobi's 40V battery packs lately. They are fairly energy dense and compatible with a wicked 1800W inverter/power station that I have. Internally they are a 10S lithium pack with some number of cells in parallel depending on the pack capacity. They use the safer flame proof cells and have a sophisticated BMS built-in so I feel pretty comfortable messing with them. They can also be obtained at steep discounts if you are willing to tolerate used packs from eBay. In my experience, they test out fine.

I came to the realization that it would be handy to be able to charge these batteries in the car. I drive electric vehicles and in the event of a power failure, these packs could easily be charged up numerous times. Ryobi doesn't make a 12V-compatible charger for this line of batteries so I took matters into my own hands.

12V Compatible Ryobi 40V Battery Charger

This charger uses an adjustable constant-current/constant-voltage (CC-CV) power supply that steps the 13.8-14.4V that my car provides up to the 42V needed to charge the batteries. I tore apart the OEM chargers that Ryobi provides and found that the weakest link is a blocking diode rated for 3A. The OEM charge adapter ordinarily operates at 1.5A from a wall-powered CC-CV supply. I configured my DC-DC converters to operate at a peak voltage of 42V with 2.75A of current, just shy of the limit of the blocking diode. So far, all has been working well. There is a slight increase in temperature of the case plastics of the charger, but nothing outside of my own personal comfort zone.

Ryobi 40V Batteries Charging in the Car :]

The result is that I can now easily charge my batteries in the car. I decided not to use a 12VDC to 120VAC inverter to avoid any efficiency penalty and have measured this design to be around 90% efficient. In this configuration, the batteries are charging at 100W+ which is pretty impressive. In order to mitigate thermal losses, a small 12V fan is employed to move heat away from the converter.

Ryobi 40V Car Charger Assembly

The assembly consists of a few 3D printed components. I designed a case to accommodate the DC-DC converter, some panel mount XT-60 connectors and the cooling fan. I also designed a backshell for the XT60 connectors to make them easier to manipulate. They turned out great.

XT60 Backshell Custom Design

I used a cheap rubber strain relief to make the cables more robust. The end result is a convenient device that should come in handy for power outages, camping trips or any other time that it makes sense to keep batteries topped off in the field.

Finished Assembly

I sacrificed the wall power supply for the connector to use for the charger. All is not lost though. I fitted the opposite gender XT60 connector and an identical backshell so that the charger can still be used in the factory original configuration or for charging from a car.

I hope you enjoyed this short blog post and small project of mine!

QRTape | Audio Playback from Paper Tape with Computer Vision

Here is a project that I have been tossing around in my head for at least a year or two now. Given advances in audio compression algorithms and computer vision: could reasonably high-quality audio be stored on a paper tape?

This is a fascinating concept to me. When considering the complete history of storage media from early cylindrical engraved records through to magnetic tape it is evident that achieving high quality and reliable data storage is both challenging and expensive. When specifically considering magnetic tape, the challenges surrounding complex mechanical tape transports and sensitive electronics highlight some of the difficulties that engineers have faced in the past.

QRTape

In this blog I demonstrate a system that exploits modern computer vision and audio compression to replace the complex mechanical tape transports of the past. In fact, my tape transport is made entirely from paper and cardboard (excluding some electronics, of course). I have decided to call this system QRTape.

QRTape Player

Under the QRTape system, data is encoded by a series of QR codes that are printed on a continuous strip of paper. This strip of paper is fed from one spool through a crude tape transport, past a webcam and onto a take-up spool. The paper is advanced by a small stepper motor driven by a cheap Arduino. The rest is pure software magic.

nrfnet: Streaming Video over nRF24L01

A couple of days ago I published a blog discussing how I used NRF24L01 radios to implement a point-to-point network between two Raspberry Pi computers. I implemented this as a virtual network device and sent packets between the radios.

Since then, I have made numerous improvements to the software and more than tripled the throughput from ~90kbps to nearly 300kbps. These improvements were through a variety of changes that I will cover in this blog post.

Streaming video from one headless Raspberry Pi to another

Thanks to the higher throughput, I was able to implement streaming video using the h264 HEVC video codec and monaural audio using the Opus codec at 32kbps. The result is great, especially when considering the link.

Continue reading to learn more!

nerfnet: Wireless Networking over nRF24L01 2.4GHz Radios

I recently picked up a set of nRF 2.4GHz radio transceivers. These are low-cost radios with a SPI interface that allow exchanging 32 byte packets across a radio link that can run at up to 2MBit on-air data rates. They are popular among hobbyists who want to introduce wireless to their Arduino-flavored projects. I was able to buy ten of these radios with trace antennas for just $11 and three more with SMA-connected antennas for $18.

NRF24L01 Radios

My first inclination is to try something a bit more extreme with this hardware. There is a GitHub project named RF24Audio that allows transmitting audio data over these radios. I wondered if video could be possible and started brainstorming about how a video transport over this link would look. The further I got into the specifics of streaming video the more convinced I was that an abstract link that could carry any form of data would be more fun.

This led me to build nerfnet: a simple application that allows sending network frames over NRF24L01 radios. This is implemented by exploiting the TUN/TAP virtual network device API under Linux on a Raspberry Pi. The code is available on GitHub for you to review and use.

I was able to demonstrate nearly 90kBit throughput as measured by iperf. I suspect this is the first time that iperf has been used to characterize a link composed of these radios.

andrew@andrew-pi:~/Projects/nerfnet $ iperf -c 192.168.10.2
------------------------------------------------------------
Client connecting to 192.168.10.2, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  3] local 192.168.10.1 port 34490 connected with 192.168.10.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.1 sec   110 KBytes  89.4 Kbits/sec

Continue reading or watch the video to learn more about how I pulled this off.

Plaid Model S Excitement

If you have been following my blog for a while, you probably know that I am an unabashed Tesla fan. My heart lies with the Model S, the only car that I have ever bought twice.

The recent Laguna Seca record-setting lap has me excited. One thing that stood to me was the e-stop stuck to the dash. I wonder what it does. Does it cut the main battery pack contactor? Wouldn't it be too late by the time it needs to be disengaged? Does it dump fire suppressant over the vehicle? Well, I might be too far out of the loop to ever know what function it performs but decided to re-create the look for fun.

The Resistor Network Plaid Model S :]

The total cost was $15 for a pair of dubious e-stop switches and a couple of small squares of 3M VHB. I put on my longboarding attire and snapped a few photos with my new Canon R5. After a little futzing about in Adobe Lightroom, I ended up with this.

My blog has been a little light on projects lately. Work definitely keeps me busy. It is fun to head out and be creative every once in a while though. This was taken near The Computer History Museum in Mountain View, California. There is a Supercharger nearby.

I hope you enjoy my fun and congratulations to Sebastian Vittel on the impressive lap!

Amateur Radio from 2900ft. on Mount Diablo | 146.520MHz FM Simplex

Over the past weekend I took a solo, socially-distanced road trip to Mount Diablo to try out my Amateur Radio station. The results far exceeded my expectations and I ended up making more than 25 contacts total.

View from Mount Diablo with Antenna on Trunk

I decided to put together a video about my adventure. It turned out to be a nice day and a good way to get out of the house. The Amateur Radio Community is incredibly friendly which made for many smiles and fun contacts.

I decided to QSL as many of these contacts as possible. I hope to get some interest cards back from the various operators that I was able to reach.

First QSL Card Design: Lick Observatory

Thanks for reading and I hope you enjoyed the video :]

datvideo: Storing Video on Digital Audio Tape (DAT)

About a year ago I added a Digital Audio Tape (DAT) deck to my home theater system. I was turned onto the format by popular Youtuber Techmoan and a great video that he produced: Digital Audio Tape: The one DAT got away. I had wanted to add magnetic tape to my home theater for some time and this highly unique format seemed like a perfect choice.

Big Buck Bunny from DAT Tape

Shortly after purchasing the deck I realized that I could likely store any arbitrary binary data on the tape with the S/PDIF input/output. It would be really cool to merge some modern high-compression video codec with this antiquated format. Over this weekend I decided to do just that.


I wrote a small tool called datvideo that allows storing arbitrary binary data on the tape. This tool is used to grab raw audio binary data from a sound card, search for frames of binary data, decode them and emit them into another file. This can be assembled into a pipeline to feed video data into a player such as mplayer. How cool is that?

Sony DTC-690, below Marantz Blu-Ray, AV Receiver and New-Old-Stock DAT Tapes

In this article, I will walk you through how I pulled this off.

PortL2 - Portable Electric Vehicle Charger

Over the past year I have been tossing around the idea of extending the electric range of my Cadillac ELR. I have wanted to build a large scale lithium battery system for some time and this seemed like a great way to learn about the technology.

I had originally started with the idea of charging the ELR hybrid pack while driving. The plan was to use a 48V to 390V (96S) lithium battery charger. Accessing the high voltage bus in the car proved to be challenging and made this approach too difficult for my taste. This made me decide build a portable L2 charger using a 240V inverter and an OpenEVSE instead.

Cadillac ELR Charging at Sierra Trail Head, No L2 Chargers for Miles

I used four battery modules from an old Enginer PHEV Conversion Kit, with a design capacity of more than 2kWh each. The total capacity is around 7kWh now as the cells are several years old and have seen some use. The results so far have been great with the system capable of charging the car to more than 60%.



In addition to being a great portable car charger, this has proven to serve well in powerwall applications. I am able to shift on-peak loads to off-peak, thus saving money and reducing reliance on dirty sources of power.

PortL2 Before Transferring to the Car

I used an Orion Jr. BMS to monitor the pack, perform cell balancing and monitor temperatures. The video above has a complete build log and you can find more details in the rest of this article.

Groking the Cadillac ELR

Those who know me well know that I have a passion for electric vehicles (EVs). I own a Tesla Model S, have driven it across the US and Canada twice, and am currently designing an electric bicycle with design cues taken from larger EVs and based on lessons learned from my electric skateboard project. I really can not scratch the itch to drive and learn more about EVs.

Last fall I purchased a Cadillac ELR to keep my Tesla company in the garage. I am a huge fan of this car for the striking good looks and the fact that it is a 2-door coupe. I had also never experienced a PHEV (Plug-In Hybrid Electric Vehicle) and really wanted to try out the GM Voltec platform. The prices of the Cadillac ELR are plummeting due to their relatively short-lived time on the market and the fact that they are a little unknown in the eyes of the average consumer.

Cadillac ELR at Cardinale GMC in Seaside, California

I did not buy this car with the intention of merely letting it sit in the garage though. One of the great features of the Tesla Model S is that it lets you "nerd out" with all kinds of stats about your trip. You get total energy consumption in kWh, efficiency in Wh/mile, range estimates based on change in elevation/speed and a whole lot more that makes driving a true delight.

The Cadillac ELR (and Chevy Volt - they share the same powertrain and a similar UI) tend to shield you from these details. I wanted more and without having to do quick mental math based on the limited information available from the built-in infotainment system.

Speed, Distance, Total Energy and Wh/mi Top | State of Charge (SoC) kWh Left | Speed mph Right

I spent about a week decoding traffic on the CAN bus of the ELR in an effort to find a few signals: speed, state of charge (SoC) in kWh, and odometer or a trip of some sort. Once I found these fields, I built a UI with PyQt to render statistics in real-time.

CAN Bus Interface Cable

I packaged it all up into a neatly organized cable and used a Sony Vaio UX to read data from the CAN buses and visualize it on a small display. Continue reading for more details.

Crossing the Great Canadian Electric Vehicle Desert

I recently returned from a three week trip crossing Canada in a Tesla Model S. The journey began in the heart of Silicon Valley and followed the West Coast of the United States until reaching Vancouver. I then travelled east on the Trans-Canada Highway until reaching the Greater Toronto Area where I was born and raised. After spending some quality time with family and catching up with old friends I returned to California on the Tesla Supercharger Network, crossing through the Rocky Mountains of Colorado.

Green: Charging Stops | Red: Rest Stops | Purple: Trip Start/Finish

It was an incredible trip and certainly takes first place for the longest of my road trips. I faced challenges crossing Canada due to the lack of charging infrastructure. With perseverance, resourcefulness and patience I was able to complete the trip without requiring a tow (though I did have a close call in Northern Ontario).

Tesla Model S in Rocky National Park, Colorado

Along the way I met some incredible people. One fellow performed his own EV conversion on an older model Porsche. Another built his own experimental aircraft. I was graciously invited into the home of a Tesla owner in Thunder Bay, Ontario who was happy to lend a charge and share a meal. The trip was enlightening because it was off the beaten path and allowed an opportunity to meet people who live their life just a little differently than the way I live my own.

Driving through the Rockies in Alberta

This trip encompassed 12000km (7400mi) of driving and consumed more than 2.2MWh of energy spread out over 140+ hours of driving. The trip across from Silicon Valley to Toronto took 8 days, 3 of which were spent navigating Northern Ontario. It was an exercise in patience but the prize of being just one of a few drivers to complete this route in an electric vehicle is unequivocally worth it.

pwrusbctl: A command-line tool for controlling USB-connected power strips

Over the past few days I wrote a command-line tool for controlling and monitoring PowerUSB-branded power strips. These are clever devices that connect to a computer via USB and allow power monitoring and controlling the state of the outlets. I decided to call it pwrusbctl and you can view the code on GitHub.

The company that sells them offer a binary blob driver, but I wanted something that I could extend and modify. I also want it to work with my RaspberryPi.

PowerUSB Basic connected to a RaspberryPi

I used HIDAPI as an abstraction over HID which makes the codebase portable. It has been tested to work on Linux and Mac OS. You can find more documentation in the README on GitHub. Here is an example of the output:

[andrew@andrew-rpi-1 pwrusbctl]$ ./pwrusbctl --reset_charge_accumulator --device_info --log_indefinitely --power --energy --interval 2000000
Found PowerUSB device type: Basic
Power: 32.200001W
Energy: 0.000000kWh
Power: 32.200001W
Energy: 0.000537kWh
Power: 27.599998W
Energy: 0.000537kWh
Power: 27.599998W
Energy: 0.000537kWh
Power: 50.599998W
Energy: 0.000537kWh
Power: 172.500000W
Energy: 0.000537kWh
Power: 342.700012W
Energy: 0.000537kWh
Power: 519.799988W
Energy: 0.000537kWh
Power: 545.099976W
Energy: 0.000537kWh

Extreme Long Range Electric Longboarding: 56km on a Boosted Board

I just returned from a 56km Boosted Board ride. I toured the bike trails along the Upper/Lower Crystal Springs Reservoirs and San Andreas Lake here in the San Francisco Bay Area. This is a new record for me. My previous range record was 27km on a single charge.

Boosted Board with external battery in my backpack :]

To achieve this I used a very large and custom designed lithium-ion battery pack. I tossed it in my backpack and connected it to the Boosted Board where I had previously made a modification to expose the internal battery voltage rail. This pack is arranged in a 10S6P (10-series, 6-parallel) configuration. It has a rated capacity of 21Ah at 37V nominal or roughly 777Wh. The internal battery of the Boosted Board has a rated capacity of 99Wh.

This battery pack is designed to attach to a custom designed board that I have been working on but is not ready for prime time yet. Below is a sneak preview of the board I am working on. I have decided to name it Voyager as one of the design goals has been to prioritize range.

10S6P battery mounted to Voyager with custom charging cable attached

Continue reading to learn more about my new record and construction of this battery pack. There are also more pictures of Voyager at the bottom of this post.

ichargermon: a Linux/Mac iCharger monitoring tool

I use an iCharger 1010B+ to charge a few batteries for my electric skateboard. I also top up the battery in my car from time to time. I thought it would be nice to keep track of the charging progress so I wrote a small tool to parse logs that are output over UART and format them into a human-readable format.

I named the tool ichargermon. Feel free to send a pull request!

The project is written in C/C++ and has no external dependencies at this time. If I decide to support more advanced iChargers I will add libusb as a dependency.

Charging a small 3S1P battery pack

I also have an iCharger 4010Duo that I would like to add support for. The 4010Duo implements a much more rich protocol (MODBUS) that supports both remote monitoring and control.

Anyway, for now here is an example of the output. Thanks for reading!

Input voltage:    15.163V
Battery voltage:  40.165V
Battery amps:     0.990A
Internal temp:    30.300C
Cell 0 voltage:   4.009V
Cell 1 voltage:   4.010V
Cell 2 voltage:   4.012V
Cell 3 voltage:   3.998V
Cell 4 voltage:   4.014V
Cell 5 voltage:   4.014V
Cell 6 voltage:   4.014V
Cell 7 voltage:   4.014V
Cell 8 voltage:   4.017V
Cell 9 voltage:   4.019V

VGA Generation with Freescale i.MX23 + Linux

I have long wanted to become more well acquainted with the Linux kernel and finally decided to bite the bullet. I ordered an iMX233-OLinuXino-MAXI from Olimex to tinker with. The Freescale i.MX23 processor is noteworthy because it is available in an LQFP-128 package which means it can be installed on a PCB by hand with inexpensive tools and a steady hand. It is also a great platform to learn with because it is well supported by the upstream kernel and has documentation available without signing an NDA.

The first thing I did was modify the device tree (DTS files) to enable the LCD controller and tuned it to generate a VGA signal. I built a simple R/R2 DAC on a breadboard and was able to view the image on an LCD monitor.

The projected image next to my workstation

All of this was completed under Arch Linux ARM which provides a minimal base image and root filesystem upon which a large number of packages have been ported to run on ARM. This has been a great learning experience and I plan to continue working with this chip more.

Hackaday, it doesn't quite like the 640 width

Continue reading to see the very minor changes I made to the kernel and how I wired it  all up.

MikMod on STM32F4

Over the past couple of days I have ported libmikmod to run on an STM32F407. This is a very memory constrained environment with only 128kB of RAM but I am able to play some rather complex MOD and XM files that are in the range of 60kB in size. I had to patch libmikmod slightly and write a new audio output driver to make all of this happen.

MikMod on STM32F4 Hardware :]

The audio quality is quite good. I currently have MikMod configured to render at 44.1kHz in mono. I could likely render in stereo but I would be limited to smaller MOD and XM files due to the increased memory usage. Here is a video that mainly shows the audio quality as captured by my camera.

The video below contains a little more technical detail, demonstration of boot and loading different audio files.

Long Range Electric Longboarding

I bought a Boosted Board back in August of this year while living in New York City. It was an awesome purchase. It is fun to ride and I could often beat the subway to my destination on shorter trips. I have since moved to Silicon Valley and find the range to be lacking slightly. The board gets approximately 7 miles of range out of the box and my office is 10 miles from home.

My Boosted Board shortly after purchase.

I added 288Wh of high-discharge lithium-ion batteries to the 99Wh of batteries that came with the board. I was inspired by the Portable Electric Vehicle Youtube Channel.  During the first test ride I was able to ride more than 13 miles before stopping. The fuel gauge on the remote showed more than 20% of battery remaining.

My Boosted Board after the battery upgrade :]

During this process I also designed a custom lighting system based on WS2812 LEDs and an Arduino Mini Pro.

These LEDs are bright!

Flir Lepton Thermal Imaging Sensor + Gameduino 2

I recently got my hands on a pair of Flir Lepton thermal imaging sensors and have spent the last week bringing them online in my spare time. These are absolutely incredible devices that I believe will pave the way to consumer devices incorporating thermal imaging cameras. The footprint of the camera module (and optical assembly) is about the size of a dime. The resolution is 80x60 at 14bpp which is remarkable despite sounding low.

Thermal Andrew :]

I have successfully implemented a driver for the Lepton module and displayed frames on an LCD. This is all running on an STM32F4 processor on a Nucleo board. Attached to it is a Gameduino 2 which incorporates the FT800 graphics processor. I have implemented my own colorization and min/max scaling before uploading the frames to the GPU.

Front System Overview

I have used some simple jumper wires to interface with this camera. This setup is running at 21MHz with no issues. I am using a breakout board provided by Pure Engineering. You can pick up one from Tindie if you are interested. The Lepton module can be ripped out the Flir One iPhone accessory for now.

Rear System Overview and Second Camera (future project ;])

I wrote my own driver for the Lepton core and the FT800 graphics processor. Continue reading for more details!

FT800 with Streaming Video

I like to check out the thrift stores in my area for one of a kind technical gems to add to my collection. A few years ago I came across a Connectix QuickCam. This is one of the earliest webcams that didn't require a separate video capture card. Due to how easy it was to install, it was incredibly popular. So much so that Logitech ended up buying Connectix out and forking the product under their own brand.

Connectix QuickCam

When I saw it in the store I really had no idea what it was aside from the fact that it was a webcam and it had a parallel port. My first instinct was that a parallel port is simply a collection of TTL lines that I could emulate with a modern microcontroller. I finally had some time to put it into action and decided to stream video frames from the camera to a Gameduino 2 and the FT800 video processor that it uses.



The results were great, aside from the low image quality of the camera. This experiment proves the concept of using an FT800 to composite motion video with other visual elements. A better image sensor would provide a really cool experience.

Keep reading to see how I did it!

2048, Embedded

Let me first get something out of the way... I am an embedded nut. I absolutely love taking a foreign piece of hardware and bringing it to life. My most recent excursion into embedded land has been with a Nucleo board designed by STMicroelectronics.

I decided to pair this board with a Gameduino 2 shield designed for the popular Arduino. I wrote a driver for the FT800 graphics processor and implemented Gabriele Cirulli's 2048 game as a first project with this hardware.

2048 Screenshot

The result is an awesome device that emulates the fun 2048 browser game (pretty closely, anyway). It will make a great desk ornament for passersby at work to gawk at.

Nucleo F4 Board

In this article I will give you a demonstration of the game and take you through some of the design concepts I have used to come up with this product. I wrote my own linker script, C-Runtime and driver library for this project and it was an awesome experience.