The OSX equivalent of simple batch files

As a Windows developer, one of the biggest hurdles in getting into iOS (Xcode) development is getting used to a whole new OS. Silly little tasks suddenly take up a significant amount of time, and because you mentally consider them "trivial", you completely underestimate the impact this has on daily development. Because zero minutes times a lot is still zero, right?

One of those tasks is using batch files for simple stuff. Stuff like zipping up a bunch of folders and copying them somewhere else. Deleting some files with specific names. Kids' stuff, basically.

Enter Mac OSX. It is based on Unix, meaning it has a terminal and does shell scripts. As any Unix guy will tell you, Unix shell script runs circles around  Windows batch script. It's like a full-fledged class IV intergalactic battle cruiser to our granny bike. The problem is: I just want to get to the grocery store today.

The first thing you need to realize is that Windows batch files execute from the Windows Explorer really nicely. We consider this a given, but you don't get this stuff for free on OSX.

Here is a little step-by-step on how to create the equivalent of a batch file on OSX. Make sure to open up a terminal and navigate to the required folder first.

Step 1) Create a new file (copy con test.bat).

 touch test.command  

Touch can be used to create a file out of thin air. Do note the .command extension: this is a known file association for Finder, allowing you to actually click it to run it. Using the regular .sh for shell scripts will not work.

Step 2) Mark the file as executable (no batch equivalent)

chmod +x test.command  

Step 3) Open up the file in the text editor of choice of the OS (notepad test.bat)

open -a TextEdit test.command  

Step 4) Start the script in the current folder (the default in batch ), by opening with

cd "$(dirname "$0")" 

The double quotes take care of any spaces in filenames. The $( ) performs an inline command execution. It executes the dirname command to parse the directory from the full path argument passed in $0 (by Finder)

Step 5) Fill out the rest of the script to do something useful

cd "$(dirname "$0")"
[insert useful stuff here] 

Step 6) Save the script using command-s.

You now have a nice little script that can be execute from Finder. If you want to execute it from the terminal directly, make sure to prefix it with the current folder (./test.command).

Safely accessing Named Properties in Outlook

Whenever you want to access properties in Outlook that are not exposed by the Outlook Object Model, there is the PropertyAccessor object.

Not only does this powerful class allow you to retrieve any of the standard MAPI properties, it also gives you access to any custom Named Properties using a convenient string notation.

This class does have one minor problem though: it doesn't offer any way to explicitly check for the existence of a property. So when one half of your applications marks certain emails with a certain named property, and the other half needs to read out that marker, you end up with code like this:

var
  mailItem: OleVariant;
  propertyName: String;
  marker: String;

begin
  ...

  propertyName := 'http://schemas.microsoft.com/mapi/string/someGuid/my.secret.marker';

  try
      marker := mailItem.propertyAccessor.getProperty( propertyName );

  except
      // Apparently, the marker isn't set.
      marker := '';
  end;

end;

Ouch. Not so much fun when doing this in a loop over a few thousand email. And turning off Break on OleException in the IDE is not the best of ideas when you are doing office automation, either.

Moving from late to early binding (and you should, anyway) makes this problem blatantly apparent; here is the interface definition of PropertyAccessor (renamed IPropertyAccessor for compliance with Delphi naming standards):

  IPropertyAccessor = interface(IDispatch)
    ['{0006302D-0000-0000-C000-000000000046}']
    ... snip ...
    function  GetProperty(const SchemaName: WideString): OleVariant; safecall;
    procedure SetProperty(const SchemaName: WideString; Value: OleVariant); safecall;
    ... snip ...
  end;

See those safecall markers? Those means that Delphi has wrapped the calls in a little check, raising an exception whenver the result is not ok. Which is nice and error proof, but brushes over the fact that there are several different results possible from the underlying call, and not all of them are crucial errors. In our case the returned result is MAPI_E_NOT_FOUND, which just tells us the mailItem at hand wasn't marked... not exactly an "exceptional" situation in this particular application.

The trick is that we can simply redefine this interface, using HResult stdcalls directly, as opposed to the magically wrapped safecalls. This conversion is easy enough to do by hand (everything needs to return a HResult, move all function results to out parameters). However, for even more convenience, we can use use tlibimp to do this. From a command line, execute:

tlibimp -P -Ps- "C:\Program Files\Microsoft Office\Office14\MSOUTL.OLB"

(make sure to correct that path if office is installed elsewhere). This will output a nice Outlook_TLB where we can copy the property from. I aptly (although perhaps slightly confusing) named my copy ISafePropertyAccessor.

ISafePropertyAccessor = interface(IDispatch)
    ['{0006302D-0000-0000-C000-000000000046}']
    ... snip ...
    function GetProperty(const SchemaName: WideString; out Value: OleVariant): HResult; stdcall;
    function SetProperty(const SchemaName: WideString; Value: OleVariant): HResult; stdcall;
    ... snip ...
  end;

Here is how we'd use it in the example above:

var
  mailItem: OleVariant;
  propertyName: String;
  propertyAccessor: ISafePropertyAccessor;
  value: OleVariant;
  marker: String;

begin
  ...

  propertyName := 'http://schemas.microsoft.com/mapi/string/someGuid/my.secret.marker';
  propertyAccessor := IUnknown(mailItem.propertyAccessor) as ISafePropertyAccessor;

  hr := mailItem.propertyAccessor.getProperty( propertyName, value );

  if hr = S_OK then
  begin
      marker := value;
  end
  else
      marker := '';

  ...
end;

Slightly more verbose, but that can always be wrapped away in some convenience functions... and it sure is nice to get rid of the exception!

Episode 9 - Baptizing Bots, Cursing C++ Updates, Shrinking Songs

Awesome bot names

Team Trivia features, at least in the initial testing phase, bots. These cool little automatons will fill up your team to 4 players whenever there are not enough human players available in the lobby.

To give the bots a bit of personality I wrote a simple syllable-based random name generator. The generation is based on three arrays with syllables taken from the characters of the Oz books. Some of the entries are deliberately empty to create two-syllable words. All in all, nothing fancy, but I really like the names it spits out.

Update from hell - Visual Studio SP1

Time for a rant. I wanted to try building http://1337haxorz.de/products.html and then using it (one way or another) from Delphi. I don't have a decent C++ compiler on this laptop yet, so I downloaded Visual C++ 2010 Express and installed it. Which was a nice and smooth ride, but then... I run it.

First off, it complained about finding other Visual Studio 2010 components running on the system, but ooooh those were SP1, so Visual C++ couldn't run until it was updated as well. Excuse me? I just want to compile some standalone C++ code, do I really care about the exact update version of ASP.net or SQL Server? If it is so darn important that "everything runs on the exact same version", why have me download an outdated version in the first place? But ok, I'll click the button on the dialog that downloads SP1 and install it...

... which apparently takes 4x as long as doing the full Visual C++ 2010 Express installation in the first place. Oh, and it requires a reboot too...

... which then ends in a black-screen-with-mouse-cursor, where it is installing stuff without any visual feedback (i.e. "it just sits there"), for well over an hour! Dear developer people designing this install process, please realize that this sort of user experience is Unacceptable. I can't use my machine (it zapped about _all_ the personal coding time I get on a weeknight), I haven't got a clue how long it is going to take, and most importantly: I'm seriously worried! Is it still running? Is it breaking just my Visual Studio, or trashing my hard disk completely? Will my machine ever come out of it, again?

If you are going to lock me out of my own machine for this amount of time, at least have the decency to warn me beforehand, but even then... what the heck are you installing on my machine that takes so frighteningly long? The entire MSDN?

 

Building a tiny V2 executable with VS2010

As Fabian "Ryg"Giesen just published loads of Farbrausch tools on GitHub, all of this will probably become irrelevant really fast, but here is how you do it:

Prerequisites

- Install Visual C++ 2010 express. Gnash teeth whilst enduring any SP1 updates.

- Get nasm. Install it anywhere, but make sure to put the location into the path.

- Copy nasm.exe to nasmw.exe. This is probably not the best way to do this, but it works.

- Get the official kkrunchy binary from ryg's Homepage . Put anywhere on your disk.

- Get the V2 sources . Again, the project is now on Github, so you might want to pull it from there.

Building

- Open up example/tinyplayer.sln from the V2 source folder. Visual C++ will automatically upgrade the solution to 2010

- Open up the project property pages.

- Change the Base Address to 0x800000 (or anything significantly bigger than the default 0x400000, really). This will suppress kkrunchy's warning about the resulting executable not working on win 9x.

- Change the Randomized Base Address to No (/DYNAMICBASE:NO). If you leave this setting on th default Yes, the kkrunched executable will immediately crash when started.

- Close the property pages and build the project (F7).

- Open up a CMD prompt, run kkrunchy on the resulting tinyplayer.exe . Done. 16kb of tiny music madness.

Episode 8 - A better stopwatch, character encoding

A better Windows 7 Stopwatch - Stopwatch7

To celebrate Easter, my trusty XNote Stopwatch decided to malfunction (no, I didn't actually take 1193 hours to write this blog post). Instant panic! I absolute need some sort of time measurement at home, because with all the interruptions in a day (kids, kids, wife, chores, kids, more kids) my sense of time goes all funny, and I start to feel like accomplishing nothing at all.

My demands for such a tool are simple: it should be simple, trivial to start, trivial to stop, and should always be visible, yet not in an immensely intrusive way. I dare you to look for such a timer on the internet, I bet you won't find it. I know I haven't.

So I happily jumped into the most alluring of developer pitfalls, and rolled my own. Against all Windows guidelines I write the time progress straight onto the taskbar (heck, my taskbar icon is prime screen estate!) At a later phase I'll probably add a better peek thumbnail, including some nice thumbbuttons and global hotkeys... but for now this solves my problem.

If you want to play around with it, you can download it here. The lovely icon is by Oliver Scholtz .


Character encoding - always a worry

Some of the questions in the Team Trivia database contained evil Word-style quotes. Those questions would be displayed at the client as a lovely "null"... which makes providing the proper answer rather difficult.

After a quick scavenging hunt across the different application layers I learned how PHP's json_decode will silently and happily replace a string of text with null if it contains anything other than UTF-8 characters. As MySQL defaults to latin1 (ISO 8859-1), this explained the troubles.

Adding a mysql_query('SET CHARACTER SET utf8') at a tactical location fixed things up nicely.


Misc

- Registered ShinyHammerGames.com, registered myself as a Google Play Developer .
- This blog is now hosted on blog.shinyhammergames.com .
- NetBeans has integrated MySQL management ... but it refused to show me the table data of some of my tables, was horrible for proper editing of text data, and is rather short on functionality anyway. I've fallen back onto my trusted friend SQLyog Community Edition .
- Got bitten by the backticks-versus-single-quote semantics of PHP/MySQL, referring to random columns in a WHERE clause. That's what you get for developing on a tiny 13" laptop screen.

Episode 7 - Coordinates, New Project, PHP Setup

Revised Coordinate System

Ok, so it turned out my coordinate woos from Episode 4 were indeed "woos", it turned out the coordinate system I implemented before was totally impractical. Here is what I do now:

• I simply provide the render system with a desired virtual resolution (800x480 in my case) for the view
• The size of all sprites is interpreted relative to this resolution, i.e. a 80x48 bitmap sprite is 1/10th of the view
• The render engine automatically applies letterboxing and pillarboxing on systems running a different aspect ratio, to prevent stretching
• The upper left corner of the screen is at (0, 0), because this is the traditional approach I've been comfortable with since the days of the C64. There will be a flexible camera at a later stage, so this is not set in stone.

Letterboxing on an extra wide viewport

Kicked off a New Project.

I'm trying to get into the habit of starting and finishing short, smaller-scoped projects... so after a (fun!) little brainstorm session with Ye Balde Scribe I've set aside Seesaw Kiwi for the moment, and started a Brand New Shiny Project (tm).

The awesome part about this particular project is that it will be shippable at a very early stage, but still leaves room for massive amounts of improvement. Awesome!


Setting up PHP Development - NetBeans, XAMPP


The new project requires a server component. PHP seems the obvious choice for now, simply because it is the easiest/cheapest to setup... I've got more than a couple of Shared Hosting LAMP websites sitting around, doing mostly nothing.

After installing XAMPP, I hit a small hurdle getting Apache to run, because a running TeamViewer service (which I was pretty sure I'd never enabled) hijacked port 80. On a modern Windows 7 developer system it's quite hard to figure out which of the 70-odd running services is the culprit, especially because netstat -ano returned PID 4... which is the System process. Very helpful, netstat.

This being my first experience with XAMPP and the NetBeans IDE, I was a bit disappointed that I had to manually edit php.ini to get the debugger to work (why bundle it with XAMPP but not configure it?). And why do I have to manually edit httpd.conf to get the active NetBeans project running in Apache? Can I have my IDE a bit more Integrated, please? Good news: the debugging experience is really good thus far.

Rant: it's kinda unfunny how windows projects that originate as Cross Platform Open Source never seem to get UAC right. Same with XAMPP: You have to make sure you don´t install it inside Program Files (because it drops read/write config files inside it's install directory, which is a big no-no in Windows-land), and you need to explicitly launch the XAMPP control panel with elevated rights because it needs to manipulate windows services, and doesn't include a manifest of its own.

Oh well, enough complaining, it's free and it works. Hopefully the Mac/Linux users get a bit more love for their respective platform guidelines :-).


Misc

• Learned you have to do a glScissor if you don't want the glClearColor to leak outside of the glViewPort at each glClear.

Episode 6 - Camera controls and exceptions

Implementing camera controls for Board Game Machine

Worked on some camera controls for our upcoming (multi player, multi device) board games simulator Board Game Machine . Control mechanisms demonstrated in the video below:

• 90 degree spherical (elliptical) rotation of the camera position to view each side of the game board
• Continuous vertical rotation of the camera position to get the desired view angle onto the board
• 90 degree camera rolling when the camera is facing straight down

Notice (or rather, please do not notice) how there is a myriad of debugging text on the screen (the video is recorded straight from the dedicated development tool), and how the graphics are still very much in development.

Making Eclipse stop on Null Pointer Exception (and friends)

By default, debugging an exception in your Android application goes something like this:

1. Watch the IDE stop into an empty page somewhere in the Dalvik code (no source code available unless you installed the full Android sources) with a rather meaningless call stack.
2. Hit F8 to continue running.
3. Look at LogCat to find out what was actually wrong. Find the red lines in the log, scroll down until you find the "caused by" line, doubleclick, and finally be transported to the offending line in your code.

I just learned you can force Eclipse to actually behave like a sane IDE by performing the following simple steps:

1. From the menu, select Run, Add Java Exception Breakpoint
2. Find NullPointerException (or whatever exception you are debugging)
3. Verify Suspend On Caught Exceptions is checked. Click OK.

The debugger will now immediately break (suspend) where the exception occurs. Like, you know, every other (proper) debugging environment on the planet. I'm kinda stumped as to why they didn't make this the default debugging behaviour, but at least you can configure it to work properly.

Mobile client for DMSforSharePoint

Since after a fair bit of swearing and cursing I've got SOAP + NTLM up and running now (hurray, detailed blog post on that topic later), ít's time to develop a little prototype of a mobile client for our SharePoint-based DMS, DMSforLegal / DMSforSharePoint. Here is the first screenshot of this work in progress:

 

Misc

- The Android game engine now plays sound & music through some convenient manager objects. This was really easy to get up and running on android (thanks, MediaPlayer and SoundPool!). Still unsure about the best engine design for handling sound effects, but we'll see how it works out.

Episode 5 - Input/Output

Accelerometer... cool in theory, ok-ish in practice

I'm implementing basic game input control, so I needed simple tilt values. Android supports this through a rather heavy handed interface called SensorManager and a couple of SensorEvents providing you with such information as acceleration forces and geomagnetic field information, from which a rotation matrix can be deduced. Extracting the right column from that rotation matrix will then yield the tilt.

Which is all nice and dandy, but with such an advanced and complex API I'd expect the sensor output to be phenomenal... and it simply isn't.The accuracy and response times are a disappointment. Disclaimer: I am working on the assumption that my Samsung Galaxy SII is representative of most of the hardware that is currently out there.

Ah well, that just means that direct rotational control (anyone remember the paddles on the C64?) is out of the question, so now I've opted for tilt-controlled velocity. Cool.


Android and SOAP ... not so clean

The de facto standard library for handling SOAP in Android is KSoap 2 (I'd advise using the -android branch from the link, as it is actively maintained). And while it is a pretty extensive library (all due respect to the contributors to this the project!), it most definitely lacks in the ways of KISS.

If you come from a .NET background, or any other platform where SOAP is a serious player (we have run our own custom implementation in Delphi), the thing just reeks of a lack of abstraction. If I just want to make a simple remote call, I sure don't want to be instantiating an "envelope" object (really, I don't want to know). On the other hand, if I do care about soap intrinsics, I sure don't want to get the response back as a property of that same envelope... That's just wrong. But enough ranting, it gets the job done.

I more or less hit a roadblock then, as I discovered doing (Windows 2008 compatible) NTLM authentication isn't exactly trivial, and lots of existing Android/Java projects seem to think it's ok to simply drop support for it anyway because it is deprecated (Kerberos is superior anyway, etc). Hello? People living inside a glass box? There are millions and millions of Windows (business!) users on NTLM networks.... I'm sure they wouldn't mind using their android devices to access their stuff.

Oh well, I guess I'll just have to bite the bullet and modify KSoap 2 a bit to hack the necessary support in. Unfortunately, this is all just a side-project, so it might be a while before I get around to it.


Misc

 - Have been tweaking the camera controls of our Board Game Machine project a bit. It's funny how low frame rates (still on the to-do list) can sometimes make it hard to judge whether your implementation is actually correct.

- Added activity switching and a single option screen (input through touch or accelerometer) to the Android game.

- The barebones Android game engine now has automatic texture reloading on surface change. No rocket science, but the textures disappearing was getting kind of annoying.

This is what lost textures look like. It's not pretty.