Tuesday, September 11, 2012

The State of the World, Targa Edition

Recently, for my game project I thought I'd read in my little pixel people using Targa files.  That coding problem is, in miniature, a good example of many issues facing free software today: standards vs implementations, reinventing the wheel, and too much code.

First off, why TGA files?  Well my game is meant to have the look and feel of a Nintendo DS or Gameboy Advance game.  These systems draw using a 16-bit colorval format: 5 bits each of red, green, and blue, plus one alpha bit, or 'ARGB1555' for short. The alpha bit indicates either completely transparent or completely opaque.  I vaguely remembered that eons ago, TGA also had a 16-bit ARGB1555 format, so it seemed like a good fit.

So I checked the latest spec for TGA v2.0, and sure enough it was a format that TGA supported.  This was going to be great, because it was going to save me 50% on memory storage of uncompressed images.

standards vs implementation

There's a lot of cool features in that spec.  Plenty of ways to store your own custom metadata inside of each image.  There is plenty of flexibility over color formats and the like.  Color correction tables.  Gamma correction.  Key colors.  It is simple enough for a single programmer to implement in a couple of days.  Looks like a slam dunk.

There are dozens if not hundreds of implementations of TGA encoding and decoding.  I looked through a couple dozen of them.  Notably libav, imagemagick, netpbm, freeimage, SETI@home, GIMP, Quake, Free Space, and UWisc's libtarga.

None of them have has implemented that specification in full.  As far as I can tell, everyone uses a subset of features that is the 'de facto' standard.  All the fun metadata is never used.  The many varieties of bit depths are never used.  The de facto standard is a stripped down version that stores RGB888 and ARGB8888 with a minimalistic header.

And since recent drawing programs will rarely output any bit depth other than RGB888 or ARGB8888 or will output any of that new header data or use any of those features, the fact that other features and bit depths are valid is moot.

It isn't about what the standards says, it is about what you can get people to implement.

in C, portability implies reinventing the wheel

One bit of comedy when looking at all these implementations is how each one has defined its own type for unsigned 8-bit and 16-bit unsigned little-endian integers and each one has defined its own nomenclature for functions to read them from disk. Each one, in trying to be portable, is built upon little more than libc.  And in so doing, each one reinvents the wheel.
  • Free Space: ubyte, short, cfread_ubyte(), cfread_ushort()
  • ImageMagick: unsigned char, unsigned short, ReadBlobByte(), ReadBlobLSBShort()
  • libav: uint8_t, int, bytestream2_get_byte, bytestream2_get_le16()
  • Blender: uchar, short
I'm not claiming that it is hard to typedef a uint8_t or that it is hard to make a function fread an LE uint16_t.  These are just what I noticed when reading those implementations.  But there's so much more than that that is reimplemented every day: linked lists, trees, etc.  Because once you invoke 'portability' when talking about C, you end up stripping all the way back to the metal, all the way back to libc, and then cut-and-pasting yourself back up to something more manageable.

Why did C never develop a thin layer on top of libc (like glib or gnulib tries to be) that became broadly accepted?  Where is C's equivalent of Boost?

too much code

When I look at Savannah, Github, Gitorious, Google Code, Sourceforge, I realize that there is an implementation of every existing idea of simple to moderate complexity. (With the obvious exception of a feature-complete TGA 2.0 library.)  There is an ocean of code.  If it all could be cleaned up, sorted, indexed, documented, collated, then we could create the Grand Unified SDK.  Any new project would be trivial or moot. But, more often than not, it is easier to begin again than wade through it all, try to find the gems, understand their logic, work around their lack of documentation and fix their bugs.

I envy PHP coders sometimes.  Their standard lib is quite amazing.

and so

I chucked the Targa idea and just used GDK to read in .png's as GDK pixbufs and then converted them to Cairo contexts.  If I want retro art, I'll just draw it retro.  It doubles the memory storage size, but, ultimately a unoptimized program is better than a highly optimized idea.

Monday, August 27, 2012

Writing a game for GNU and GNOME

Hello All,

I've decided that I'm going to write a game.  I've always wanted to do it. I've held off because the world is full of homebrew games that aren't very good, and I'm not sure I can do better.  But, whatever, let's see what happens when I try.  Programming is my golf, I guess.

But I know myself, and I know that when faced with the awesome ambiguity and limitlessness of "making a game", I'll get lost in the planning and never bring anything to execution.  I've always been the type of guy that loses the big picture by looking at details. So first, I need to set some limits, some serious restrictive boundaries on the problem so that I don't get lost in the weeds.

So the first consideration was what SDK was I going to write it in.  I looked at the various game SDKs, but, I wanted to do something to help the GNU + GNOME stack.  This is one of the problems with GNU and GNOME: it is still wicked hard to develop anything, especially coming from a work background of C# and .NET.  C# and .NET form a complete well-documented SDK.  There is no need to look outside of .NET for a solution.  Whenever I get back to EMACS + gcc + GNU + Linux + GNOME, the whole devel experience is much tougher, primarily because first you need to basically roll your own IDE and SDK.  You choose a bunch of libraries and integrate them with autotools glue.  In the integration of several libraries there is always varying quality and detail of the documentation.

This is where my productivity always gets derailed, this process of rolling your own SDK from the dozens of great libraries out there, and rolling your own IDE from the dozens of EMACS libraries out there.  What should I use?  What shold be in my own personal GNU IDE and SDK, exactly?

So I decided to let some one else make that decision.  I decided that I would only use the libraries listed at Gnome Developer Center in their Platform Overview, and I would only use API that was documented and languages that it supported.  I'm not going to add any other dependencies.

So this simplified the decision making trememdously.  For audio, there is only GStreamer, so I'll use that.  Its sample application plays Ogg audio, so I'll use that.  For graphics, I can use either Cairo or Clutter.  But I can't use Clutter, because my graphics card's free driver doesn't have any hardware acceleration, so OpenGL won't run.  Cairo will have to do.

Cairo is a 2D library, so my game will be in 2D.

Since I'm limiting myself to fully documented interfaces, my game will be in C.  It is the only language that spans the whole GNOME SDK stack and that has proper documentation.

Since I'm limiting myself to the GNOME stack, I guess I'm writing my own engine.  For inspiration on how to do that, I've been reading a lot on how the Game Boy Advance and Nintendo DS work.  There is a big homebrew Nintendo DS scene, and some great free developer tools to write Nintendo DS games, like DevkitPro.  So I started writing an engine inspired by the description of the 2D stack of the Nintendo DS hardware.  Not an emulator of course; an engine inspiried by a description of the workings of the DS's 2D stack.

What's great about the Nintendo DS as an inspiration is that it is very specific in how it works.  Objects are sprites.  Sprites are made of collections of 8x8 pixel tiles.  Sprites can only be specific sizes, 2x2 tiles (16x16 pixels) or 4x4 tiles (32x32 pixels).  Backgrounds are either maps of 8x8 pixel tiles or they are single bitmaps.  The screen size is small 256x192 pixels. There's a lot of structure there to keep me from getting off track.

I'm also setting myself some resource restrictions based on the Nintendo DS as well.  The memory footprint of the application will be under 4.6 MB. 4 MB for the in-memory application and the heap, plus another 0.6 MB specifically reserved for the video resource cache. The total disk space of the game executable and resources has to be 32 MB or less.

So along the way, I hope to learn and describe a few fun things
  • Setting up EMACS to be more like an IDE
  • Using Cairo to make a tile and sprite rendering engine
  • Using GIMP to draw some graphics
  • Generating audio with free tools
  • Playing game sound with GStreamer
And I hope to file lots of bugs and documentation clarifications

So, my very first task is to assemble some graphics into a resouce file and then implement my 600kb resource cache.