Tuesday, January 26, 2016

MikMod on STM32F4

Over the past couple of days I have ported libmikmod to run on an STM32F407. This is a very memory constrained environment with only 128kB of RAM but I am able to play some rather complex MOD and XM files that are in the range of 60kB in size. I had to patch libmikmod slightly and write a new audio output driver to make all of this happen.

MikMod on STM32F4 Hardware :]
The audio quality is quite good. I currently have MikMod configured to render at 44.1kHz in mono. I could likely render in stereo but I would be limited to smaller MOD and XM files due to the increased memory usage. Here is a video that mainly shows the audio quality as captured by my camera.

The video below contains a little more technical detail, demonstration of boot and loading different audio files.

A Brief History of this Project


I was initially interested in working on an embedded MOD player approximately 3 years ago. I was working with AVR devices, mainly because I had access to them. I quickly ran up against the resource limitations of those devices. Fast forward to now when I have access to STM32F4 devices with 128kB of RAM and anywhere between 256kB and 1MB of flash memory.

I should note that this experiment began on an STM32F401 which has 96kB of RAM. I was able to load and play small MODs and XMs on a Nucleo board. I moved up to the STM32F407 in order to play much larger and more interesting files.

Compiling the Library


The first step was to compile libmikmod for Cortex-M4 using an arm-none-eabi toolchain. I looked at other examples in the libmikmod source tree for support on unusual platforms such as the Sony Playstation Portable or a Korean Gameboy known as GP32.
# libmikmod Makefile for targetting Cortex-M4.
                                                                                                                                                                                                                   
TARGET   = arm-none-eabi                                                                                                                                                                                           

CC = $(TARGET)-gcc                                                                                                                                                                                                 
LD = $(TARGET)-gcc                                                                                                                                                                                                 
AS = $(TARGET)-as                                                                                                                                                                                                  
AR = $(TARGET)-ar                                                                                                                                                                                                  
RANLIB = $(TARGET)-ranlib                                                                                                                                                                                          
                                                                                                                                                                                                                   
INCLUDES = -I../include

CPPFLAGS = -DMIKMOD_BUILD -DDRV_METAL -DHAVE_LIMITS_H

CFLAGS = -O2 -Wall -mcpu=cortex-m4 -mthumb -mlittle-endian \
         -mno-thumb-interwork -g

ARFLAGS = cr                                                                                                                                                                                                       
                                                                                                                                                                                                                   
COMPILE = $(CC) -c $(CFLAGS) $(CPPFLAGS) $(INCLUDES)                                                                                                                                                               
                                                                                                                                                                                                                   
LIBS = libmikmod.a                                                                                                                                                                                                 
                                                                                                                                                                                                                   
OBJ = load_mod.o load_xm.o mmalloc.o mmerror.o mmio.o mdriver.o mdreg.o \
      mmcmp.o pp20.o s404.o xpk.o mloader.o mlreg.o mlutil.o mplayer.o \
      munitrk.o mwav.o npertab.o sloader.o virtch.o virtch2.o \
      virtch_common.o                                                                                                                                                                                              

HEADER_DEPS = ../include/mikmod.h ../include/mikmod_internals.h
                                                                                                                                                                                                                   
libmikmod.a: $(OBJ)                                                                                                                                                                                                
  $(AR) $(ARFLAGS) $@ $(OBJ)                                                                                                                                                                                       
  $(RANLIB) $@                                                                                                                                                                                                     
                                                                                                                                                                                                                   
clean:                                                                                                                                                                                                             
  rm -f $(LIBS) *.o                                                                                                                                                                                                
                                                                                                                                                                                                                   
drv_nos.o: ../drivers/drv_nos.c $(HEADER_DEPS)                                                                                                                                                                     
  $(COMPILE) ../drivers/drv_nos.c -o drv_nos.o                                                                                                                                                                     
load_it.o: ../loaders/load_it.c $(HEADER_DEPS)                                                                                                                                                                     
  $(COMPILE) ../loaders/load_it.c -o load_it.o                                                                                                                                                                     
load_mod.o: ../loaders/load_mod.c $(HEADER_DEPS)                                                                                                                                                                   
  $(COMPILE) ../loaders/load_mod.c -o load_mod.o                                                                                                                                                                   
load_xm.o: ../loaders/load_xm.c $(HEADER_DEPS)                                                                                                                                                                     
  $(COMPILE) ../loaders/load_xm.c -o load_xm.o                                                                                                                                                                     
mmalloc.o: ../mmio/mmalloc.c $(HEADER_DEPS)                                                                                                                                                                        
  $(COMPILE) ../mmio/mmalloc.c -o mmalloc.o                                                                                                                                                                        
mmerror.o: ../mmio/mmerror.c $(HEADER_DEPS)                                                                                                                                                                        
  $(COMPILE) ../mmio/mmerror.c -o mmerror.o                                                                                                                                                                        
mmio.o: ../mmio/mmio.c $(HEADER_DEPS)                                                                                                                                                                              
  $(COMPILE) ../mmio/mmio.c -o mmio.o                                                                                                                                                                              
mmcmp.o: ../depackers/mmcmp.c $(HEADER_DEPS)                                                                                                                                                                       
  $(COMPILE) ../depackers/mmcmp.c -o mmcmp.o                                                                                                                                                                       
pp20.o: ../depackers/pp20.c $(HEADER_DEPS)                                                                                                                                                                         
  $(COMPILE) ../depackers/pp20.c -o pp20.o                                                                                                                                                                         
s404.o: ../depackers/s404.c $(HEADER_DEPS)                                                                                                                                                                         
  $(COMPILE) ../depackers/s404.c -o s404.o                                                                                                                                                                         
xpk.o: ../depackers/xpk.c $(HEADER_DEPS)                                                                                                                                                                           
  $(COMPILE) ../depackers/xpk.c -o xpk.o                                                                                                                                                                           
mdriver.o: ../playercode/mdriver.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/mdriver.c -o mdriver.o                                                                                                                                                                  
mdreg.o: ../playercode/mdreg.c $(HEADER_DEPS)                                                                                                                                                                      
  $(COMPILE) ../playercode/mdreg.c -o mdreg.o                                                                                                                                                                      
mdulaw.o: ../playercode/mdulaw.c $(HEADER_DEPS)                                                                                                                                                                    
  $(COMPILE) ../playercode/mdulaw.c -o mdulaw.o                                                                                                                                                                    
mloader.o: ../playercode/mloader.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/mloader.c -o mloader.o                                                                                                                                                                  
mlreg.o: ../playercode/mlreg.c $(HEADER_DEPS)                                                                                                                                                                      
  $(COMPILE) ../playercode/mlreg.c -o mlreg.o                                                                                                                                                                      
mlutil.o: ../playercode/mlutil.c $(HEADER_DEPS)                                                                                                                                                                    
  $(COMPILE) ../playercode/mlutil.c -o mlutil.o                                                                                                                                                                    
mplayer.o: ../playercode/mplayer.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/mplayer.c -o mplayer.o                                                                                                                                                                  
munitrk.o: ../playercode/munitrk.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/munitrk.c -o munitrk.o                                                                                                                                                                  
mwav.o: ../playercode/mwav.c $(HEADER_DEPS)                                                                                                                                                                        
  $(COMPILE) ../playercode/mwav.c -o mwav.o                                                                                                                                                                        
npertab.o: ../playercode/npertab.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/npertab.c -o npertab.o                                                                                                                                                                  
sloader.o: ../playercode/sloader.c $(HEADER_DEPS)                                                                                                                                                                  
  $(COMPILE) ../playercode/sloader.c -o sloader.o                                                                                                                                                                  
virtch.o: ../playercode/virtch.c ../playercode/virtch_common.c $(HEADER_DEPS)                                                                                                                                      
  $(COMPILE) ../playercode/virtch.c -o virtch.o                                                                                                                                                                    
virtch2.o: ../playercode/virtch2.c ../playercode/virtch_common.c $(HEADER_DEPS)
  $(COMPILE) ../playercode/virtch2.c -o virtch2.o
virtch_common.o: ../playercode/virtch_common.c $(HEADER_DEPS)
  $(COMPILE) ../playercode/virtch_common.c -o virtch_common.o
This Makefile is inspired by the GP32 and PSP drivers. They gave me insights into the minimum necessary to build libmikmod and then I took it a few steps further.

The first thing I did was remove most of the loaders. I was only interested in MOD and XM. This results in the following diff to include/mikmod.h:
-MIKMODAPI extern struct MLOADER load_669; /* 669 and Extended-669 (by Tran/Renaissance) */
-MIKMODAPI extern struct MLOADER load_amf; /* DMP Advanced Module Format (by Otto Chrons) */
-MIKMODAPI extern struct MLOADER load_asy; /* ASYLUM Music Format 1.0 */
-MIKMODAPI extern struct MLOADER load_dsm; /* DSIK internal module format */
-MIKMODAPI extern struct MLOADER load_far; /* Farandole Composer (by Daniel Potter) */
-MIKMODAPI extern struct MLOADER load_gdm; /* General DigiMusic (by Edward Schlunder) */
-MIKMODAPI extern struct MLOADER load_gt2; /* Graoumf tracker */
-MIKMODAPI extern struct MLOADER load_it;  /* Impulse Tracker (by Jeffrey Lim) */
-MIKMODAPI extern struct MLOADER load_imf; /* Imago Orpheus (by Lutz Roeder) */
-MIKMODAPI extern struct MLOADER load_med; /* Amiga MED modules (by Teijo Kinnunen) */
-MIKMODAPI extern struct MLOADER load_m15; /* Soundtracker 15-instrument */
 MIKMODAPI extern struct MLOADER load_mod; /* Standard 31-instrument Module loader */
-MIKMODAPI extern struct MLOADER load_mtm; /* Multi-Tracker Module (by Renaissance) */
-MIKMODAPI extern struct MLOADER load_okt; /* Amiga Oktalyzer */
-MIKMODAPI extern struct MLOADER load_stm; /* ScreamTracker 2 (by Future Crew) */
-MIKMODAPI extern struct MLOADER load_stx; /* STMIK 0.2 (by Future Crew) */
-MIKMODAPI extern struct MLOADER load_s3m; /* ScreamTracker 3 (by Future Crew) */
-MIKMODAPI extern struct MLOADER load_ult; /* UltraTracker (by MAS) */
-MIKMODAPI extern struct MLOADER load_umx; /* Unreal UMX container of Epic Games */
-MIKMODAPI extern struct MLOADER load_uni; /* MikMod and APlayer internal module format */
 MIKMODAPI extern struct MLOADER load_xm;  /* FastTracker 2 (by Triton) */
I also removed all of the other drivers except for drv_nos which is required as a fallback when no other driver is available.

After some experimentation I was able to produce an archive file to link into my application.

Initializing MikMod


To my surprise the library worked on the first attempt. I decided to use ChibiOS for this project. It comes with a kernel that supports pre-emptive multitasking and a hardware abstraction layer (HAL) with drivers for all of the common peripherals: UART, SPI, I2C, PWM, GPT and more.

Here is the main function for my program. As you can see, it is not terribly complex. Initializing MikMod is very straightforward. I included some logging to make debugging easier.
int main(void) {
  halInit();
  chSysInit();

  initSerialConsole();
  initMikMod();

  MODULE *module = Player_LoadMem(goldenages_mod,
        goldenages_mod_len, 4, false);

  if (!module) {
    SerialLog("MikMod", "Error loading module");
    chThdSleep(TIME_INFINITE);
  }

  SerialLog("MikMod", "Loaded module: %s", module->songname);

  // Play the module.
  Player_Start(module);

  while (Player_Active()) {
    MikMod_Update();
  }

  SerialLog("MikMod", "Playing complete");

  Player_Stop();
  Player_Free(module);

  // Cleanup after MikMod.
  MikMod_Exit();

  while(1) {
    chThdSleepMilliseconds(100);
  }

  return 0;
}
I register an error handler with MikMod and print the logs to a serial console. This is handy when attempting to play a MOD file that just barely fits in RAM. MikMod will happily invoke this function and tell you that it failed to allocate memory.
static void MikModErrorHandler(void) {
  SerialLog("MikMod", "error %d%s: %s",
      MikMod_errno,
      MikMod_critical ? " (critical)" : "",
      MikMod_strerror(MikMod_errno));
}

/*
 * Initialize the serial console for logging.
 */
void initSerialConsole(void) {
  SerialConfig serialConfig = {
    .speed = 115200,
    .cr1 = 0,
    .cr2 = 0,
    .cr3 = 0,
  };

  sdStart(&SD2, &serialConfig);
  palSetPadMode(GPIOA, 2, PAL_MODE_ALTERNATE(7));
  palSetPadMode(GPIOA, 3, PAL_MODE_ALTERNATE(7));
}

/*
 * Initialize and configure libmikmod.
 */
void initMikMod(void) {
  SerialLog("MikMod", "Initialization start");

  MikMod_RegisterErrorHandler(MikModErrorHandler);
  MikMod_RegisterDriver(&drv_metal);
  MikMod_RegisterAllLoaders();

  md_mode = DMODE_INTERP | DMODE_SOFT_SNDFX | DMODE_SOFT_MUSIC;
  md_reverb = 0;
  md_mixfreq = 44100;

  if (MikMod_Init("")) {
    SerialLog("MikMod", "Initialization error");
    chThdSleep(TIME_INFINITE);
  }

  SerialLog("MikMod", "Initialization complete");
}

PWM Audio Output


I contemplated using the CS43L22 audio codec in the early stages but decided it was not worth the effort. The STM32F4DISCOVERY includes this codec with a convenient 3.5mm jack for connection to headphones. I decided not to use the codec due to the effort required to initialize and enable streaming audio output. There are several samples available online, but if I want to work with an audio codec I would prefer to take a deep dive and read the entire datasheet.

I decided to use a PWM output to generate an audio waveform and am very satisfied with the results.
static PWMConfig pwmConfig = {
  .frequency = 16000000,
  .period = 255,
  .callback = NULL,
  .channels = {
    { .mode = PWM_OUTPUT_ACTIVE_HIGH, .callback = NULL },
    { .mode = PWM_OUTPUT_DISABLED, .callback = NULL },
    { .mode = PWM_OUTPUT_DISABLED, .callback = NULL },
    { .mode = PWM_OUTPUT_DISABLED, .callback = NULL },
  },
};

static void timerCallback(GPTDriver *gptDriver) {
  chSysLockFromISR();

  signed char sample = buffer[buf_index][play_pos++];
  pwmEnableChannelI(&PWMD1, 0, PWM_FRACTION_TO_WIDTH(&PWMD1, 255, sample));

  if (play_pos == SAMPLE_LENGTH) {
    play_pos = 0;
    buf_index ^= 1;
  }

  chSysUnlockFromISR();
}

static GPTConfig gptConfig = {
  .frequency = 882000,
  .callback = timerCallback,
};
A PWM peripheral is configured to run at a very high frequency and a general purpose timer is used to update the duty cycle at a rate of 44.1kHz. These two devices working together produce an audio waveform.

This audio driver is double buffered. When playback begins, the first buffer is filled with audio samples and the PWM/timer are enabled. While the first buffer is playing, the second is filled with sample data. The driver then waits until playback from the second buffer has began to replace the contents of the first buffer with new audio data. This ensures that the audio output is never starved for data (buffer underrun). This is based on the assumption that generation of samples is much faster than playback.
static void METAL_Update(void) {
  if (buf_index_update != buf_index) {
    VC_WriteBytes((signed char *)buffer[buf_index_update], SAMPLE_LENGTH);
    buf_index_update ^= 1;
  }
}

static BOOL METAL_IsThere(void) {
  return 1;
}

static int METAL_Init(void) {
  if (VC_Init()) {
    return 1;
  }

  // Initialize the PWM peripheral.
  pwmStart(&PWMD1, &pwmConfig);
  palSetPadMode(GPIOA, 8, PAL_MODE_ALTERNATE(1));
  pwmEnableChannelI(&PWMD1, 0, PWM_FRACTION_TO_WIDTH(&PWMD1, 255, 127));

  // Initialize the GPT peripheral.
  gptStart(&GPTD3, &gptConfig);

  return 0;
}

static void METAL_Exit(void) {
  VC_Exit();
}

static int METAL_Reset(void) {
  VC_Exit();
  return VC_Init();
}

static int METAL_PlayStart(void) {
  VC_PlayStart();

  VC_WriteBytes((signed char *)buffer[0], SAMPLE_LENGTH);
  gptStartContinuous(&GPTD3, 20);

  return 0;
}

static void METAL_PlayStop(void) {
  gptStop(&GPTD3);
  pwmStop(&PWMD1);
  VC_PlayStop();
}

MIKMODAPI MDRIVER drv_nos;

MIKMODAPI MDRIVER drv_metal = {
  NULL,
  "",
  "",
  0,
  255,
  "",
  NULL,
  NULL,
  METAL_IsThere,
  VC_SampleLoad,
  VC_SampleUnload,
  VC_SampleSpace,
  VC_SampleLength,
  METAL_Init,
  METAL_Exit,
  METAL_Reset,
  VC_SetNumVoices,
  METAL_PlayStart,
  METAL_PlayStop,
  METAL_Update,
  NULL,
  VC_VoiceSetVolume,
  VC_VoiceGetVolume,
  VC_VoiceSetFrequency,
  VC_VoiceGetFrequency,
  VC_VoiceSetPanning,
  VC_VoiceGetPanning,
  VC_VoicePlay,
  VC_VoiceStop,
  VC_VoiceStopped,
  VC_VoiceGetPosition,
  VC_VoiceRealVolume
};
The remainder of the driver is straight forward. I implemented the required callbacks from the MikMod library to control the state of the PWM and timer peripherals. I then spent time tuning the audio output. I initially had misconfigured the period of the PWM peripheral and introduced significant aliasing into the output.

Conclusions!


I may look at libmikmod some more. Some optimizations that I made included marking constant lookup tables as const to move them from RAM to flash. This made improvements in the output of arm-none-eabi-size. This change would likely benefit all platforms so I may consider upstreaming it. I would also like to look at making some dynamically allocated buffers static. Their size is known at compile time and allocating them statically would reduce heap usage and subsequent fragmentation. This is a major concern on embedded platforms.

Thanks for reading!

6 comments :

  1. Interesting !

    Would it be possible to have just a single ring-buffer for output to save some bytes ?

    ReplyDelete
  2. Most likely, yes. The advantage of double buffering is that MikMod can generate a few bytes at a time. You can think of the two buffers as entries in a ring buffer with a length of 2.

    ReplyDelete
  3. Hi, can you share complete code?

    ReplyDelete
  4. Crypto-currency as a modern form of the digital asset has received a worldwide acclaim for easy and faster financial transactions and its awareness among people have allowed them to take more interest in the field thus opening up new and advanced ways of making payments. Crypto.com Referral Code with the growing demand of this global phenomenon more,new traders and business owners are now willing to invest in this currency platform despite its fluctuating prices however it is quite difficult to choose the best one when the market is full. In the list of crypto-currencies bit-coins is one of the oldest and more popular Crypto.com Referral Code for the last few years. It is basically used for trading goods and services and has become the part of the so-called computerized block-chain system allowing anyone to use it thus increasing the craze among the public, Crypto.com Referral Code.

    Common people who are willing to purchase BTC can use an online wallet system for buying them safely in exchange of cash or credit cards and in a comfortable way from the thousands of BTC foundations around the world and keep them as assets for the future. Due to its popularity, many corporate investors are now accepting them as cross-border payments and the rise is unstoppable. With the advent of the internet and mobile devices,information gathering has become quite easy as a result the BTC financial transactions are accessible and its price is set in accordance with people’s choice and preferences thus leading to a profitable investment with Crypto.com Referral Code Code. Recent surveys have also proved that instability is good for BTC exchange as if there is instability and political unrest in the country due to which banks suffer then investing in BTC can surely be a better option. Again bit-coin transaction fees are pretty cheaper and a more convenient technology for making contracts thus attracting the crowd. The BTC can also be converted into different fiat currencies and is used for trading of securities, for land titles, document stamping, public rewards and vice versa.

    Another advanced block-chain project is Ethereumor the ETH which has served much more than just a digital form of crypto-currency Crypto.com Referral Code and its popularity in the last few decades have allowed billions of people to hold wallets for them. With the ease of the online world,the ETH have allowed the retailers and business organizations to accept them for trading purposes, therefore, can serve as the future of the financial system.

    ReplyDelete