Sunday, April 14, 2013

Designing a Window Manager for an AVR Microcontroller

I have been experimenting with the uVGA-II VGA controller for the past couple of weeks. It is an amusing piece of hardware that is capable of drawing graphics onto a VGA framebuffer. The VGA controller takes care of line drawing algorithms and helps to hardware accelerate the drawing of geometric primitives (squares, circles, triangles, polygons, lines).

Once I realized the power of this hardware I decided to implement a window manager like you would expect on any standard desktop PC. I have used a mouse for user input to the system.

Close-up of the Default Configuration
The default system boots with three applications: Theme Manager, Audio Player and Window Factory. The Theme Manager is used to modify the colors of the system theme, the Audio Player is used to playback some audio files stored on an SD card and the Window Factory is used to create new windows for the purposes of demonstration.

Slightly Blue Theme, More Windows
Here is a video demonstrating the system.



Software Overview

I have tried to maintain layers of abstraction within this software. I will start by explaining the layered approach that I have taken for the drivers, window manager and user application.

Layered Approach
At the bottom of the stack is the UART driver. My MCU has two UART modules so I can interface with both the uVGA-II and the serial mouse simultaneously.

Unfortunately I have fallen into the trap of trying to write good code and there is a blur between the layers. The Window Manager makes calls directly to the uVGA-II layer which means that it would be difficult to port this window manager to another VGA controller. This can be resolved by creating an intermediate, platform independent hardware interface layer. If a user wished to run AVRDE using another VGA controller they would change the hardware abstraction layer rather than the window manager itself.

Despite the blur between the layers, I find it easy to maintain. I was able to add a new widget (the slider) within a couple of hours.

Modelling a GUI Toolkit in C

This is my first attempt at using C to model complex abstraction. Typically I am using C to control hardware, registers and other low-level hardware concepts. This project takes a completely different direction and attempts to model the desktop metaphor. I do not claim to know C++ so I have decided to stay within my element and use structs and unions in C.

All On-Screen objects are descendants of the Widget
The dWidget_t has an anonymous union of all child types. This allows the dWidget_t object to be the parent type of all desktop widgets. The mouseDown, mouseMove and mouseUp callbacks of the parent dWidget_t object are handled by the window manager itself. The function pointers within the "inherited" types are for the user applications. I bind to the valueChange() callback to handle theme changes.

Managing Windows

In addition to the dWidget_t type, I have implemented a dManager_t. This struct is used to maintain the windows and associated function callbacks.

Window Manager
This manager struct keeps track of the size of the window manager (to avoid hard-coded constants), some aspects of the cursor, a list of window pointers and the positions and sizes of the buttons on the taskbar.

Drawing the Desktop

I have employed the Painter's algorithm to handle the drawing of windows. This means that I sort the windows by Z-Index and then draw them from lowest to highest such that the foreground window is always on top.

I have borrowed a concept from Android that I am quite fond of. I am using a repaint flag to handle repainting when necessary. This means that if I make a modification to a widget (change the text of a Label, for example), I must call dInvalidate() on that widget to have it painted onto the screen. This minimizes the amount of total repaints necessary.

I also have a repaint flag in the manager. This flag is set when a window is moved, the theme is changed or a window is minimized. The manager.repaint flag results in a complete repaint of the desktop environment.

Lots of Windows!
System Theme

Each widget contains a pointer to a theme. This makes it possible for different windows to have different themes or for applications to maintain their own appearance. I have not exploited this functionality in the demonstration video.

The manager also has a reference to the system theme for drawing the taskbar.

The Taskbar

The taskbar is quite simple. It is not implemented as a widget, it is simply drawn on top of the window manager when necessary and the locations of the buttons are stored for window switching.

When the titles of windows are too long for the button, the final 2 characters are replaced with "..".

Shortened Titles
Hardware

There isn't too much to the hardware. I have an ATmega1284p microcontroller, a uVGA-II VGA controller, a MAX233 level converter and a Microsoft Serial Mouse.

Hardware Overview w/ Headphone Splitter-Coupler :)
Hardware Close-up
Show me the Code!

I have maintained a Git repo on my laptop containing all of this code. I am not 100% happy with it yet. I need to work on the UART driver some more and polish up some documentation before uploading it.

I will post another blog soon with a link to the Github and a bit of a tour through the repo.

Thanks for reading!

8 comments :

  1. I'm wondering if the window management part of the project can be adapted for non-VGA based displays. Something like a small 320x200 SPI graphic LCD or slightly larger. I realize that 320x200 may not be much in a way of screen real estate but small windows can still be useful for various option settings, some alert popups and other things that you may want to do on an Arduino (nothing heavily graphical obviously).

    Do you think it's possible? What changes would be needed in the software?

    Thanks!

    ReplyDelete
    Replies
    1. This absolutely could be adapted to a non-VGA based display. Resolution is no concern with the current setup. I believe you will have two problems:

      1. The code is specific to the uVGA-II controller. In the AVRDE layer, I make function calls directly to the driver rather than through a HAL (Hardware Abstraction Layer). I did this over the course of a week and a half, mostly as a proof of concept.

      2. The uVGA-II controller hardware accelerates a lot of things involved in doing something like this. As an example, drawing a window consists of drawing a box and then drawing 4 lines on the edges. I simply call the DrawBox command and the hardware takes care of it for me. If you were to do this in software, you would need to implement efficient line drawing algorithms (http://en.wikipedia.org/wiki/Bresenham's_line_algorithm). You would also need to implement fill algorithms for filling in various shapes.

      This is definitely possible, but with some additional modifications and at a loss of performance.

      Thanks for reading! :]

      Delete
  2. Thank you for the explanation. Yes, I did not consider the performance loss - this may be an issue (but perhaps the smaller size might make it a bit easier on the MCU). I'll think about it. Perhaps it would be easier to redraw the entire screen for each change in the program.
    Thanks!

    ReplyDelete
  3. Just a thought, but it might just be faster to save the images pre-drawn to memory for smaller devices. Just copy them to draw. I mean most buttons are probably the same size, maybe a border etc... so you can use a way that works more like a game boy etc... and draw pseudo sprites. I'd imagine it would be pretty fast too. I've done a visualizer for my car pc (before stupid things like the iPod) that hooked up to the parallel port + mpxplay in DOS on a Pentium 100/133Mhz. Booted up in < 5 secs. All play controls were controlled via a PSX like analogue pc controller. Dynamic stereo expansion during play. SoundBlaster AWE32 Goodness.

    CopyDrawBorder()
    CopyDrawButtonSmall(x,y,text)

    etc...

    ReplyDelete
  4. Awesome work. I cant wait to dig into the GIT repo.

    ReplyDelete
  5. frazzledjazz[at]gmail. Hit me up. looking to do this on a Gameduino, which is similar hardware to what you have on the VGA side.They stopped production but GAMEDUINO add-on for arduino boards are available.HIGH DEMAND part, BTW. Use the SPI pins,you get faster buffered I/O.I also have a MAPLE.Can this be done on a 328P(window buffer compression?)?

    ReplyDelete
  6. Awesome work and thanks for posting the video and explanation.

    I have been playing around with uVGA from 4d systems for sometime now using an Arduino. I am interested in porting this to Arduino. (I think Uno may not be able to run this, but Arduino Mega might be able to). If you are interested then do contact me at sudar {at} sudarmuthu.com and we can see how we can collaborate on this.

    Really looking forward to look at your code (if you are interested in sharing it)

    Also, the url to the blog post in the youtube video is wrong. I found the url by searching your blog.

    ReplyDelete
  7. Any update on this project?

    I would like a look at your code.

    ReplyDelete