Episode 9 - Baptizing Bots, Cursing C++ Updates, Shrinking Songs

Awesome bot names

Team Trivia features, at least in the initial testing phase, bots. These cool little automatons will fill up your team to 4 players whenever there are not enough human players available in the lobby.

To give the bots a bit of personality I wrote a simple syllable-based random name generator. The generation is based on three arrays with syllables taken from the characters of the Oz books. Some of the entries are deliberately empty to create two-syllable words. All in all, nothing fancy, but I really like the names it spits out.

Update from hell - Visual Studio SP1

Time for a rant. I wanted to try building http://1337haxorz.de/products.html and then using it (one way or another) from Delphi. I don't have a decent C++ compiler on this laptop yet, so I downloaded Visual C++ 2010 Express and installed it. Which was a nice and smooth ride, but then... I run it.

First off, it complained about finding other Visual Studio 2010 components running on the system, but ooooh those were SP1, so Visual C++ couldn't run until it was updated as well. Excuse me? I just want to compile some standalone C++ code, do I really care about the exact update version of ASP.net or SQL Server? If it is so darn important that "everything runs on the exact same version", why have me download an outdated version in the first place? But ok, I'll click the button on the dialog that downloads SP1 and install it...

... which apparently takes 4x as long as doing the full Visual C++ 2010 Express installation in the first place. Oh, and it requires a reboot too...

... which then ends in a black-screen-with-mouse-cursor, where it is installing stuff without any visual feedback (i.e. "it just sits there"), for well over an hour! Dear developer people designing this install process, please realize that this sort of user experience is Unacceptable. I can't use my machine (it zapped about _all_ the personal coding time I get on a weeknight), I haven't got a clue how long it is going to take, and most importantly: I'm seriously worried! Is it still running? Is it breaking just my Visual Studio, or trashing my hard disk completely? Will my machine ever come out of it, again?

If you are going to lock me out of my own machine for this amount of time, at least have the decency to warn me beforehand, but even then... what the heck are you installing on my machine that takes so frighteningly long? The entire MSDN?

 

Building a tiny V2 executable with VS2010

As Fabian "Ryg"Giesen just published loads of Farbrausch tools on GitHub, all of this will probably become irrelevant really fast, but here is how you do it:

Prerequisites

- Install Visual C++ 2010 express. Gnash teeth whilst enduring any SP1 updates.

- Get nasm. Install it anywhere, but make sure to put the location into the path.

- Copy nasm.exe to nasmw.exe. This is probably not the best way to do this, but it works.

- Get the official kkrunchy binary from ryg's Homepage . Put anywhere on your disk.

- Get the V2 sources . Again, the project is now on Github, so you might want to pull it from there.

Building

- Open up example/tinyplayer.sln from the V2 source folder. Visual C++ will automatically upgrade the solution to 2010

- Open up the project property pages.

- Change the Base Address to 0x800000 (or anything significantly bigger than the default 0x400000, really). This will suppress kkrunchy's warning about the resulting executable not working on win 9x.

- Change the Randomized Base Address to No (/DYNAMICBASE:NO). If you leave this setting on th default Yes, the kkrunched executable will immediately crash when started.

- Close the property pages and build the project (F7).

- Open up a CMD prompt, run kkrunchy on the resulting tinyplayer.exe . Done. 16kb of tiny music madness.

Episode 8 - A better stopwatch, character encoding

A better Windows 7 Stopwatch - Stopwatch7

To celebrate Easter, my trusty XNote Stopwatch decided to malfunction (no, I didn't actually take 1193 hours to write this blog post). Instant panic! I absolute need some sort of time measurement at home, because with all the interruptions in a day (kids, kids, wife, chores, kids, more kids) my sense of time goes all funny, and I start to feel like accomplishing nothing at all.

My demands for such a tool are simple: it should be simple, trivial to start, trivial to stop, and should always be visible, yet not in an immensely intrusive way. I dare you to look for such a timer on the internet, I bet you won't find it. I know I haven't.

So I happily jumped into the most alluring of developer pitfalls, and rolled my own. Against all Windows guidelines I write the time progress straight onto the taskbar (heck, my taskbar icon is prime screen estate!) At a later phase I'll probably add a better peek thumbnail, including some nice thumbbuttons and global hotkeys... but for now this solves my problem.

If you want to play around with it, you can download it here. The lovely icon is by Oliver Scholtz .


Character encoding - always a worry

Some of the questions in the Team Trivia database contained evil Word-style quotes. Those questions would be displayed at the client as a lovely "null"... which makes providing the proper answer rather difficult.

After a quick scavenging hunt across the different application layers I learned how PHP's json_decode will silently and happily replace a string of text with null if it contains anything other than UTF-8 characters. As MySQL defaults to latin1 (ISO 8859-1), this explained the troubles.

Adding a mysql_query('SET CHARACTER SET utf8') at a tactical location fixed things up nicely.


Misc

- Registered ShinyHammerGames.com, registered myself as a Google Play Developer .
- This blog is now hosted on blog.shinyhammergames.com .
- NetBeans has integrated MySQL management ... but it refused to show me the table data of some of my tables, was horrible for proper editing of text data, and is rather short on functionality anyway. I've fallen back onto my trusted friend SQLyog Community Edition .
- Got bitten by the backticks-versus-single-quote semantics of PHP/MySQL, referring to random columns in a WHERE clause. That's what you get for developing on a tiny 13" laptop screen.

Episode 7 - Coordinates, New Project, PHP Setup

Revised Coordinate System

Ok, so it turned out my coordinate woos from Episode 4 were indeed "woos", it turned out the coordinate system I implemented before was totally impractical. Here is what I do now:

• I simply provide the render system with a desired virtual resolution (800x480 in my case) for the view
• The size of all sprites is interpreted relative to this resolution, i.e. a 80x48 bitmap sprite is 1/10th of the view
• The render engine automatically applies letterboxing and pillarboxing on systems running a different aspect ratio, to prevent stretching
• The upper left corner of the screen is at (0, 0), because this is the traditional approach I've been comfortable with since the days of the C64. There will be a flexible camera at a later stage, so this is not set in stone.

Letterboxing on an extra wide viewport

Kicked off a New Project.

I'm trying to get into the habit of starting and finishing short, smaller-scoped projects... so after a (fun!) little brainstorm session with Ye Balde Scribe I've set aside Seesaw Kiwi for the moment, and started a Brand New Shiny Project (tm).

The awesome part about this particular project is that it will be shippable at a very early stage, but still leaves room for massive amounts of improvement. Awesome!


Setting up PHP Development - NetBeans, XAMPP


The new project requires a server component. PHP seems the obvious choice for now, simply because it is the easiest/cheapest to setup... I've got more than a couple of Shared Hosting LAMP websites sitting around, doing mostly nothing.

After installing XAMPP, I hit a small hurdle getting Apache to run, because a running TeamViewer service (which I was pretty sure I'd never enabled) hijacked port 80. On a modern Windows 7 developer system it's quite hard to figure out which of the 70-odd running services is the culprit, especially because netstat -ano returned PID 4... which is the System process. Very helpful, netstat.

This being my first experience with XAMPP and the NetBeans IDE, I was a bit disappointed that I had to manually edit php.ini to get the debugger to work (why bundle it with XAMPP but not configure it?). And why do I have to manually edit httpd.conf to get the active NetBeans project running in Apache? Can I have my IDE a bit more Integrated, please? Good news: the debugging experience is really good thus far.

Rant: it's kinda unfunny how windows projects that originate as Cross Platform Open Source never seem to get UAC right. Same with XAMPP: You have to make sure you don´t install it inside Program Files (because it drops read/write config files inside it's install directory, which is a big no-no in Windows-land), and you need to explicitly launch the XAMPP control panel with elevated rights because it needs to manipulate windows services, and doesn't include a manifest of its own.

Oh well, enough complaining, it's free and it works. Hopefully the Mac/Linux users get a bit more love for their respective platform guidelines :-).


Misc

• Learned you have to do a glScissor if you don't want the glClearColor to leak outside of the glViewPort at each glClear.

Episode 6 - Camera controls and exceptions

Implementing camera controls for Board Game Machine

Worked on some camera controls for our upcoming (multi player, multi device) board games simulator Board Game Machine . Control mechanisms demonstrated in the video below:

• 90 degree spherical (elliptical) rotation of the camera position to view each side of the game board
• Continuous vertical rotation of the camera position to get the desired view angle onto the board
• 90 degree camera rolling when the camera is facing straight down

Notice (or rather, please do not notice) how there is a myriad of debugging text on the screen (the video is recorded straight from the dedicated development tool), and how the graphics are still very much in development.

Making Eclipse stop on Null Pointer Exception (and friends)

By default, debugging an exception in your Android application goes something like this:

1. Watch the IDE stop into an empty page somewhere in the Dalvik code (no source code available unless you installed the full Android sources) with a rather meaningless call stack.
2. Hit F8 to continue running.
3. Look at LogCat to find out what was actually wrong. Find the red lines in the log, scroll down until you find the "caused by" line, doubleclick, and finally be transported to the offending line in your code.

I just learned you can force Eclipse to actually behave like a sane IDE by performing the following simple steps:

1. From the menu, select Run, Add Java Exception Breakpoint
2. Find NullPointerException (or whatever exception you are debugging)
3. Verify Suspend On Caught Exceptions is checked. Click OK.

The debugger will now immediately break (suspend) where the exception occurs. Like, you know, every other (proper) debugging environment on the planet. I'm kinda stumped as to why they didn't make this the default debugging behaviour, but at least you can configure it to work properly.

Mobile client for DMSforSharePoint

Since after a fair bit of swearing and cursing I've got SOAP + NTLM up and running now (hurray, detailed blog post on that topic later), ít's time to develop a little prototype of a mobile client for our SharePoint-based DMS, DMSforLegal / DMSforSharePoint. Here is the first screenshot of this work in progress:

 

Misc

- The Android game engine now plays sound & music through some convenient manager objects. This was really easy to get up and running on android (thanks, MediaPlayer and SoundPool!). Still unsure about the best engine design for handling sound effects, but we'll see how it works out.

Episode 5 - Input/Output

Accelerometer... cool in theory, ok-ish in practice

I'm implementing basic game input control, so I needed simple tilt values. Android supports this through a rather heavy handed interface called SensorManager and a couple of SensorEvents providing you with such information as acceleration forces and geomagnetic field information, from which a rotation matrix can be deduced. Extracting the right column from that rotation matrix will then yield the tilt.

Which is all nice and dandy, but with such an advanced and complex API I'd expect the sensor output to be phenomenal... and it simply isn't.The accuracy and response times are a disappointment. Disclaimer: I am working on the assumption that my Samsung Galaxy SII is representative of most of the hardware that is currently out there.

Ah well, that just means that direct rotational control (anyone remember the paddles on the C64?) is out of the question, so now I've opted for tilt-controlled velocity. Cool.


Android and SOAP ... not so clean

The de facto standard library for handling SOAP in Android is KSoap 2 (I'd advise using the -android branch from the link, as it is actively maintained). And while it is a pretty extensive library (all due respect to the contributors to this the project!), it most definitely lacks in the ways of KISS.

If you come from a .NET background, or any other platform where SOAP is a serious player (we have run our own custom implementation in Delphi), the thing just reeks of a lack of abstraction. If I just want to make a simple remote call, I sure don't want to be instantiating an "envelope" object (really, I don't want to know). On the other hand, if I do care about soap intrinsics, I sure don't want to get the response back as a property of that same envelope... That's just wrong. But enough ranting, it gets the job done.

I more or less hit a roadblock then, as I discovered doing (Windows 2008 compatible) NTLM authentication isn't exactly trivial, and lots of existing Android/Java projects seem to think it's ok to simply drop support for it anyway because it is deprecated (Kerberos is superior anyway, etc). Hello? People living inside a glass box? There are millions and millions of Windows (business!) users on NTLM networks.... I'm sure they wouldn't mind using their android devices to access their stuff.

Oh well, I guess I'll just have to bite the bullet and modify KSoap 2 a bit to hack the necessary support in. Unfortunately, this is all just a side-project, so it might be a while before I get around to it.


Misc

 - Have been tweaking the camera controls of our Board Game Machine project a bit. It's funny how low frame rates (still on the to-do list) can sometimes make it hard to judge whether your implementation is actually correct.

- Added activity switching and a single option screen (input through touch or accelerometer) to the Android game.

- The barebones Android game engine now has automatic texture reloading on surface change. No rocket science, but the textures disappearing was getting kind of annoying.

This is what lost textures look like. It's not pretty.

Episode 4 - Grinding through a Sprite System Prototype

Coordinate woos

Picking a convenient coordinate system for a 2D game environment turned out to be a bit harder than expected. Working purely pixel-based sounds nice in theory (coming from an ancient C64 / VGA background), until you realize that you are designing things to run on a wild variety of screen dimensions.

So then I tried working with pure abstract floating point coordinates, assigning the sprites a completely free scale, only maintaining their ratio.  As it turns out, this doesn't work as well as in a pure 3D environment: rescaling bitmap-based sprites to arbitrary sizes does not always come out pretty.

I ended up with a hybrid solution where viewport coordinates are based around the ratio-adjusted unit square, but individual sprite size is in pixels. We'll have to see if it holds up once I have to implement real game stuff.

Alpha woos

All my alpha-blended sprites had ugly black edges around the corner. As it turns out, Android's default approach to texture loading (GLUtils.texImage2D) premultiplies the alpha channel.

Fixing the texture loading seems a bit tricky to implement efficiently, but as a workaround I simply changed the rendering blend function into (GL_ONE, GL_ONE_MINUTES_SRC_ALPHA). This might come back to hunt me, as premultiplying alpha kills information in the other channels that might be important when using a texture for other purposes than simple alpha-blended display.

Proper alpha-blended sprites go in...

...weird borders come out!

The final result looks like the screenshot below. Notice how the pixel aspect ratio of the physical phone is different, just pretend the screenshot is in proper 16:9 ratio. Also notice how the graphics are all placeholders, just building the technology here...

These sprites move around. That's kinda the point.

Misc
- Created a local Subversion repository for my Android work-in-progress using TortoiseSVN. Man, I'd almost forgotten how Subversion is cool that way. Considered experimenting with Mercurial in stead, but decided not to shoot myself in the (distributed) foot at this point.
- Replaced the default TortoiseSVN merge/diff tool with WinMerge. Seriously, if you are still on TortoiseMerge, make sure to do the same... cause you are missing out! Make sure to set the WinMerge font to something sensible (like Consolas 10), and you'll never want to go back. The syntax highlighting alone is worth the switch.
- Learned that it is really hard to predict how colors will come out on the Samsung Galaxy SII AMOLED screen. The screen of my little Dell Inspiron Laptop literally pales in comparison.
- Learned that it is rather difficult to get a proper movie recorded from an Android application without dedicated hardware. Might end up recording the actual phone with a camera.

Episode 3 - A foray into Android-Land

Android is awesome!

After my first baby steps into the Android Tutorials, I now took the next step and managed to deploy a home brew Hello World Android application from Eclipse (ADT) onto my physical Samsung Galaxy SII phone. Things just got real!

Getting this to work involved installing Samsung Kies on the laptop, as it seems the only way to get the correct USB drivers. The hardest part was to actually get access to the Kies download from the extremely "helpful" Samsung website.

The debugging experience on the physical device is great, much more responsive than the emulator. Once in a while I do need to restart Eclipse because the debugger will not connect to the device... but even then I can still run on the device, and logging works fine. All in all, not too bad.

Debugging OpenGL ES 2.0 is not awesome!

I'll be making games on android, and for that I'll need sprites. And for performant (yes, I'm aware that is not a real word) sprites, I'll need OpenGL. According to the Device Dashboard, ES 2.0 is very wide spread amongst current Android devices, so it's a safe bet to require it.

Now, I've spent the last couple of years doing mostly DirectX, and I'd forgotten how poor the debugging experience of OpenGL really is... especially on the android device. First of all, you don't get to use the emulator if you want to use ES2.0. Then, everything fails silently without so much as a return code, so the only way to pin point silly mistakes in the calls is by liberally sprinkling your code with glGetError-based checks... and even then the errors reported are not very informative. How about proper GLSL debugging? When hell freezes over, probably.

After a fair bit of sweating and cursing, finally I got the first textured quad up and running... free drinks for everyone!

Now my fingers are itching to abstract away the big fat pile of OpenGL calls it took to get this far. I am trying to focus on moving forward and stay way from over-engineering, but man... talk about a high barrier to entry for new programmers! No wonder the internet is full of badly copy-pasted malfunctioning code fragments.

Misc
- As usual, the freshly installed GIMP insisted to have a Dutch User Interface... because I use Dutch formatting in the Windows Regional Settings. This annoys me every single time I run a GTK application, that is not how locales are handled in Windows. Please show some respect for my platform, guys! Rather than overriding the locale through a LANG environment parameter (again... not how things are done in a Win32 environment), I simply deleted the NL folder from [GIMP Install Folder]\share\locale. Works like a charm, and this way I'm pretty sure it will not revert to Dutch anytime soon.
- Learned how to take proper Android screenshots using Dalvik Debug Monitor (ddms.bat). Tried to spice them up with a Android Phone Template .PSD, but the end result looked too cheap and cheesy. Pure screenshots it is!

Episode 2 - Death of a Hard Drive

This week, out of the blue, the hard disk in my trusty Vaio decided to die on me in several subtle ways. So in between the figuring out why the heck it kept locking up, heavy cursing, reinstalling my work environment on my travel laptop, and nightly tours of the house walking off the little cries of my 3-week old youngest daughter... I've not exactly been very productive.

The good news is my little Dell Inspiron on-the-train laptop, despite having the whole cardboard look-and-feel going on, is happily running my Delphi XE2 environment. Even the shoddy HD 4225 is more or less handling the trivial 3D stuff I'm throwing at it. For now it seems I'm sorted.

Rounded cube
For our Board Game Machine implementation of Maria, we needed simple cubic pieces. As with many real life objects, the pieces have a nice little rounded bevel going on at the edges.

After a fair bit of fiddling around with the geometry generation code, the final result looks quite acceptable in the prototyping app: 

From comparing the screenshot to the real deal, it is immediately clear that (apart from a higher resolution texture of the game board), a realistic visualisation of the game pieces will require:

• A wood-like texture on the block, perhaps a subtle normal map to accentuate the wood grains would suffice
• Fading, glossy reflections on the game board
• Very subtle shadows directly under the block, mostly to enhance contrast.

Misc
- Added a debugging setting to the 3D engine that renders all vertices as 4.0 sized points. A real must-have when painstakingly handcrafting geometry from vertices and indices, as I can now debug vertex and index buffer setup individually.

- Tried to use Crinkler to link & compress a trivial FPC application, with minimized system "rtl"units. So far no luck, can't get the linker to resolve DLL binding symbols.

Episode 1 - Reflections and more

2012, Week 3 - No matter how passionate a coder you are, at some point in your life you'll find that your home programming hours have become so scarce and far between that it becomes hard to keep the flow going of whatever hobby projects you are working on (and your are working on hobby projects, right?). 

For me, detailed planning and structuring really kill motivation if I know my next opportunity to put another half hour into it is at least a week away. So now I'm trying something new: I'm just going to write down whatever I'm working on this week, add some imagery, and hopefully when rereading it'll look enough like I'm actually Getting Things Done to keep me going.

Skybox
Added convenient SkyBox support to the office 3D engine, which I also use to do at-home 3D prototyping. Currently only supports a cubemap texture, for this example I've used one of the great sets by Emil 'Humus' Persson. I used the DirectX Texture Tool from the SDK to stitch the texture together into a single cubemap, which took a surprising amount of processing time to do its job and then spit out a huge 98mb cubemap. I'll sure I'll get around to fixing that huge file size later, but for now it works... and it sure seems to load quickly enough.

Reflection & Refraction

Time to get some basic shaders up and running. Did a basic reflection shader, then a basic refraction shader. Rather unsure about picking the "best" refraction ratio, settled for something that results in the standard "backdrop looks upside down when seen through a sphere" look.

Finally, I combined the two shaders into one and included a tint color. It only colors the refraction bit, not the reflection. Not entirely correct, but in general it gives the look I was after, and isn't that what graphics programming is all about?

Misc

• Started a new test project that should give me some legroom to do some experiments. Revived support for full frame rate vsync timing in the 3D engine, because the fixed rate timer was getting in the way of kkapture capturing... or so I thought. Turned out to be the aweful high resolution uncompressed AVI video playback on my laptop.
• Worked out a very awkward way of recording YouTube video using kkapture to do the uncompressed capture, VirtualDub do stitch the AVI segments back together, and finally x264 to compress into MP4. At least two of the components in this workflow refuse to be properly automated from the command line, but for now it works.
• Played around with the Protrekkr playback source, planning to research options for getting it to run with Delphi, perhaps through the Agner.org Object File Converter?
• Played around with kkrunchy, verified that for <64kb C++ applications it does indeed run circles around any other cruncher. Need to figure out how to get it to properly pack a Delphi application.