Building a Hackintosh

November 24th, 2011

For the past 4 years, I’ve been using a Mac Pro as my primary desktop.  It’s been a fantastic machine:  Ultra stable, fast, and reasonably expandable.  It’s getting old, though.  The whole Mac Pro line is getting old; Apple’s last update was almost 500 days ago (as of this writing, check the MacRumors Buying Guide for current info).  Intel has repeatedly delayed the Xeon CPUs used by Apple in the Mac Pros.  Today, an i7 MacBook Pro can easily outperform an entry-level Mac Pro.

So Use an iMac or MacBook Pro?

The writing is pretty clearly on the wall, and there are indeed rumors that Apple will indeed kill the Mac Pro line soon.  Thunderbolt, which is basically external PCI Express, is meant to supplant most use cases for a burly desktop.  iMacs and MacBook Pros both ship with i7s, and the new iMacs can support 16GB of memory.

I actually have an i7/SSD/HD 27″ iMac, which I use for work, and an i7/SSD/HD MacBook Pro, which I use for travel.  They’re both awesome devices, but they fail to meet my needs for a desktop in three primary areas:

  • Storage (my desktop has 4 internals drives and 8 external drives)
  • Gaming performance (the iMacs use laptop graphics cards)
  • >2 Display support

Enter Hackintosh

I use OS X as my primary operating system–I only boot to Windows for games–which puts me in a bit of a bind. Do I wait for a Mac Pro upgrade, and shell out a ridiculous premium once they launch?  Part of the price problem with the Pro line is that you’re paying for huge amounts of upgradability that you’re probably not going to use (massive power supply, dual socket motherboard, ECC memory, etc).  The Mac Pro line is meant to accomodate serious scientific computing and video production needs, but all I really want is a faster photo editing machine with decent graphics performance.

“Hackintosh” refers to running OS X on commodity hardware.  This has serious price advantages.  Let’s take a look at a higher-end build, as of November 2011.  Check the Kakewalk Builds page for current pricing:

Up agains a low-end Mac Pro, this is awfully appealing:

Hackintosh Downsides

Modern Hackintosh machines are actually pretty good.  You can certainly achieve stability with minimal hassle.  Still, the Hackintosh route is not nearly as painless as owning actual Apple hardware.  In short, you’ll be facing:

  • Complicated initial setup
  • Inconvenient minor/point updates for OS X
  • Painful major updates (ie Snow Leopard -> Leopard)
  • Potential compatibility problems with your workflow (ie MainStage)

My Build

Apple maintains a fairly minimal set of drivers in OS X itself.  A lot of hardware has 3rd-party support, but this means adding kernel extensions after initial install (.kext packages).  These 3rd-party kext files are what make updates tricky, especially since some hardware compatibility relies on replacing or patching existing Apple extensions.

I tried to minimize kernel hacking by selecting hardware with known/stable OS X compatibility.

(Note that I changed my network card configuration after taking this video)

Hardware List

These link to Amazon:

I moved most of my drives, but wanted new SSDs so I could sell the old Mac Pro with boot SSD intact:
GPU-wise, I went with two of these:
I needed FireWire 800 for my Drobo.  This card works out of the box:
Post-setup, onboard audio had a slight-but-annoying noise floor in OS X (but not Windows).  I added this USB 2.0 audio interface, which has been super awesome.  I run Audioengine A5 speakers and can definitely hear the difference:
Finally, this case is awesome and contains everything:

Setup Process

Now that I had a pile of hardware I had to get everything running!

Motherboard/BIOS Settings

Taking some steps from this guide, I made the following BIOS changes:

  • Reset to “Optimized Defaults”
  • Set SATA controller modes to ACHI instead of IDE
  • Changed HPET to 64-bit in power management
  • Disabled onboard FireWire
  • Later, I overclocked to a 42x multiplier

UniBeast (Initial Lion Install)

UniBeast is an amazing tool that creates a bootable USB drive with the Mac App Store version of Lion.  You need a working copy of OS X to create the bootable drive, and a purchased copy of Lion from the App Store in the form of the “Install Mac OS X Lion.app”.  Since I was already running Lion on all my machines, I need to redownload this.  Turns out you can hold option and click on “Purchases” to redownload Lion in the App Store application.  The App Store version of Lion was current (10.7.2).

The UniBeast guide on tonymac’s side was comprehensive and worked flawlessly.  As soon as my install was complete, I had a working version of Lion with 1024×768 display, bootable via the USB drive’s copy of Chameleon (a boot manager by the Hackintosh community).

MultiBeast (Post-Lion Drivers)

MultiBeast is an all-in-one installation tool designed to modify a clean Lion install with the necessary files, drivers, and settings to self-boot your copy of OS X.  I used the settings from the same guide I used for BIOs settings.  Note that you’ll need to download a DSDT file for your motherboard, which contains the proper lookup tables to map hardware features to the OS.

Dual Video Cards

If you’re looking at a Hackintosh build, and don’t need crazy multi-monitor support or high-end gaming performance, you should stick to one of the video cards supported by OS X out of the box (there’s a great list on the tonymacx86 blog).

This is where things got tricky.  The GTX5xx series of cards are not officially supported, which means you need to do a bit of technical muckery to get things working.  I booted with a single GTX560Ti connected and ran this GT5xx Enabler.  This produced native resolutions with the one card.

To support two cards, I poked around this thread.  This gist is that you must pass in all graphics cards parameters into your boot.plist file as a hex-encoded EFI string, since auto-detection will fail.  You also need to probe your PCI addresses for both slots independently (boot one card, probe, boot second card, probe).

My input plist looks like this.  The first two addresses are for my onboard Ethernet and PCI slot.  I’m not sure if I still need these, to be honest.  After encoding hex with gfxutil, here’s my full /Extra/org.chamaleon.Boot.plist.

Network Setup

My initial setup used an out out of the box-supported network adapter (this Rosewill PCIe card).  However, this meant I had to use a PCI FireWire adapter, since the two GPUs cover some of my slots.  It turned out that whatever was in this PCI slot failed to re-initialize after sleep.  I wanted to preserve sleep, so I rearranged my hardware to use a PCIe FireWire card and onboard network.  Apparently onboard ethernet is flakey for some people, but it’s been fine for me.  I used the official RealTek drivers available in MultiBeast.  Gigabit speeds are great, it recovers from sleep; everything seems fine.

Juggling my network cards around meant changing my MAC address.  Definitely do all App Stores purchases after you’ve figured your network setup out.  I had to reinstall Apps due to Apple’s DRM.  I was unable to login to the App Store initially, too, with a “could not be verified” error (this has to due with default calls to discovering your MAC address failing).  If you find yourself in that position, this guide worked perfectly!

Sleep and CMOS Reset

Apparently this motherboard will reset CMOS/BIOS settings on sleep.  I applied this fix without testing that I had the problem.  Sleep has been fine, so it was either a solution or a benign change.

TRIM Support

OS X now supports TRIM for SSDs, although Apple has hardcoded specified device IDs into their support.  You can patch in general support for all SSDs.  I did this (and was running this before on my Mac Pro without any ill effects).

“Fermi Freeze”

Apparently Lion has much better overall support for NVIDIA’s “Fermi” GT100 architecture (via its official GTX4xx drivers, which is what you’re patching into with GTX5xx support).  Still, I experienced two graphics freezes/corruption.  Most of the information out there appears to be people experiencing issues with Snow Leopard.  Two fixes were suggested that I applied:  Disabling hardware acceleration in Flash, and upping my base clock/bus speed from 100MHz to 103Mhz.  The theory on the clock change has something to do with GPU idle states.

I’m not sure if my freezes were random, and will return, but as of this writing I’ve had 4 days of uptime (partly to run a gigantic Time Machine backup), during which I’ve edited a photo set in Lightroom/Photoshop, encoded videos, watched a movie, and did some work in Unity.  Seems very stable!

Still, there are rumors Apple is returning to NVIDIA architecture for its next refresh, which will hopefully bring additional support for the GTX5xx GPUs.  Patching graphics card drivers is the only real Hackintosh requirement that makes me nervous.

Windows

Windows install was straightforward.  Gaming performance has been great with the GTX560s in SLI mode.  Service Pack 1 install was unhappy due to being booted from Chamelon, since it wanted to screw with the reserved partition, but booting directly via the Motherboard F12-key boot selector fixed that.  In general, Windows has been super stable, and much more stable than Windows 7 on my Mac Pro.  Apple doesn’t give a shit about driver support on Windows.  I was bluescreening pretty regularly after updating to the Boot Camp driver version that accompanied Lion.

Hiding Partitions

My Windows 7 install actually created two partitions:  My actual Windows partition, plus a “System Reserved” NTFS partition.  This partition shows up in Mac OS X.  To hide it, I created an /etc/fstab file with:

LABEL=System\040Reserved none ntfs rw,noauto

The partition is still there; it just no longer mounts on boot.

I also use Carbon Copy Cloner to maintain a hot copy of my OS X boot drive.  To hide this drive from Finder, I used:

sudo SetFile -a V /Volumes/Backup

I think CCC is actually resetting this flag after it runs, so I guess I need to reset it immediately after a backup.

Impressions and Conclusions

I’ve been running the new build for a few days now, and have completed a some real-world project work on it:  Lightroom/Photoshop photo editing, video encoding, Unity game development, and general browsing.  It’s fast!

Benchmark-wise, my old Geekbench score was 9,800.  The new system is currently 13,800.  My old Mac Pro was an 8-core machine, though, and the new rig is 4-core.   General “speediness” is way up, since each individual core absolutely crushes my old box. My cooling setup definitely accommodates more overclocking, although I strongly favor stability over speed.

Gaming is awesome again!  My old setup was struggling to perform well on a DX10 code path with its GTX285.  Now I can run max settings at 60+ FPS on all modern games.  It’s crazy to see the difference in graphics fidelity in a game like Battlefield 3.

I’m a little wary about system fragility with updates.  Cloning your boot drive is a very wise move with a Hackintosh (I’ve already used it once when I broke my boot with a kernel panic).  If you’re going the Hackintosh route, spend the $50 to get a cheap drive to clone to!

I spent around $2k on this build.  A huge portion of that was SSD/GPU, so you can definitely go cheaper.  I’m going to offset much of the cost when I sell my old Mac Pro after another week of testing.  All told, I’m very happy with this upgrade!  I’d recommend an out of the box-friendly Hackintosh build to most people, with the caveat that you need to be quite technical to manage unsupported GPU hardware.  Good luck!

Photographic Favorites From the Year

August 11th, 2010

This last Christmas I gave myself a Pentax K-x entry-level DSLR as a Christmas present. I’m enjoying learning more about photography, and it’s been a lot of fun capturing events in my life.  Here are some of my favorites from this year (each of these photos is a link into the set, if you want more context):

Andy & Tierney’s Wedding

Due to a clerical error, I was invited to the Schatz wedding.  It was great!

Dip

The Band

Champagne

Lighting Fun

I picked up some lighting gear–Elinchrom D-Lite 2s and YN-460 II speedlights–which have been a lot of fun to play around with.  Most of the resulting photos aren’t posted, but to give you an idea:

Head Shot

Crystal is Reading Your Mind

Droplets

CrossFit

I’ve taken a handful of CrossFit-related photos. I’d love to do a full photo shoot at the gym sometime, with a portrait/lighting setup to catch people post-workout.

Awesome Double Unders!

Tear!

Hammertime

Dodgeball

I tried my hand at some action shots for dodgeball. Boo, florescent lighting! The last shot is from trampoline dodgeball, which is the best thing ever.

Jeff's Throw

Opening Rush

Intimidation

Trips

I usually take the camera and snap some random shots on trips. A sampling!

Playing

Smell the Flowers

Bowling Pose

Arizona Life

And a variety of photos from life in Arizona, including cats. No gallery is complete without cats.

Licking Chops

Sugary Sweets

Dad and Vicky

Phoenix as Ocean

Office Sunset

Dave Hard at Work

Swinging

Filthy Kitty

Orange Goatee

Windows 7 For Mac Users (Switching Back)

January 31st, 2010

I recently switched my primary OS to Windows 7, after nearly two years of exclusively using OS X.  I really, really like OS X!  It’s a great dick-around-and-browse-the-web operating system, but it’s even better when it comes to “real” work.  We developed all of our Blurst games on a Mac.  So why switch back?

It’s just painfully inconvenient to use OS X if you’re a game developer.  Our workflow and tool chain were fine.  Unity, Photoshop, Maya all run well on a Mac.  Rather, the inconvenience stems from interacting with the larger game development ecosystem.  Everything adds up:  Fellow developers would send us Windows-only builds for feedback, I’d want to quick reference an effect in another game, or I’d have to spot-check some problem with an IGF entry.  As a gamer, Steam by itself is a big reason to use Windows. It’s just more convenient for me to be in Windows by default.

This article will outline the services and software I used to make my transition to Windows 7 less painful and a little more Mac-like.

Cloud Control

Over the last few years almost all of my work-related stuff–email, documents, project planning–have moved into cloud services.  This has been fantastic!  Getting ready for a conference used to suck, because I had to ensure my laptop had every little piece of information I needed.  Today everything is fully accessible from any machine, anytime.

This makes operating system choice extremely portable, too.  It doesn’t really matter if I’m booted into Windows or OS X, because everything is always available.  Here’s a quick list of what we use:

  • Google Apps – Google-hosted Gmail, Google Docs, Google Calendar.   Calendar is two-way sync’ed to my iPhone for appointment reminders.
  • Dropbox – Amazing file synchronization service.  Files in a designated folder are automatically uploaded and downloaded from the web.  I keep all of my mindmaps and presentations in here, along with misc work files.
  • Xmarks – Syncs my browser bookmarks.  I tried Delicious multiple times, but never had the discipline to make it work.  This lets me use Firefox’s “click the star” convenience bookmarking.  Also syncs my saved browser passwords.
  • Pivotal Tracker – Web-based project management.  Super simplistic drag-and-drop interface makes prioritization and scheduling stupid obvious, instead of buried in a multiple screens.

Ninite for Initial Install

Windows 7 works surprisingly well out of the box.  I skipped all of Vista, so my last experience with Windows as a development environment was XP, which required a lot of tweaking.  Nearly everything that I did need was taken care of by Ninite, which is completely amazing.  Simply check the programs you want installed, and Ninite will generate a custom installer that does the work for you (opting out of toolbars and crap):

Ninite supports dozens of different pieces of software, which covered my bases nicely.

Adium -> Digsby

Adium is a great multi-network IM client for OS X.  Fortunately, Digsby has come around in recent times for Windows, and gets the job done just as well (and thankfully replaces Trillium as the IM front-runner).  My only complaint with Digsby is that it won’t show “OtherPersonName disconnected” in a chat window if they log off.  It lets you go on typing to them, oblivious of their new offline status.  Bonus points to Digsby for requiring a global Digsby account.  Yes, slightly annoying at first, but really saves a lot of configuration time on multiple machines (your various account information is stored on their servers).

Exposé -> Switcher

I useOS X’s Exposé feature frequently.  I have a Logitech mouse with two buttons immediately below the scroll bar, which I use to show all windows or the active application’s windows.  Fortunately, Switcher for For Windows mimics the exact same functionality.  I had to turn off window animation at work, since it was kind of jerky, but it’s buttery smooth on my Mac Pro at home.

Switcher is a mixed bag.  It actually adds features, compared to Exposé.  You can middle-click on windows to close them, which is great for cleaning up errant Explorer windows, and you can add the process name and memory consumption to each window preview.

It falls down a bit on complexity.  You can filter windows by active application, but only after you trigger it (there isn’t any way to have a second hotkey trigger with the initial filter set to the current application).  That’s needlessly completely, but overall this is a great replacement for OS X’s Exposé.

Growl -> Growl for Windows

We make heavy use of Growl, the universal notification system.  All of our source control and scheduling systems trigger notifications via Growl over the network.  Fortunately for us, Growl for Windows has matured to the point where it works just as well.  Growl for Windows supports the same network protocol as Growl for OS X, which means things just keep working like they’re supposed to.

iStat Menus -> Moo0 SystemMonitor

This is one replacement that I’m not too happy with.  iStat Menus is a fantastic Swiss army knife of system monitoring for OS X.  Tiny graphs show just enough information to tell if your system is thrashing on something, and can be clicked on to show more details if needed.

The Windows world seems to hate that kind of elegance, instead opting to show as many thing as possible persistently.  Moo0 SystemMonitor is the best solution I could find.  Its monitors are highly configurable, and I was able to tuck it away on one of my monitors fairly innocuously.  I dislike constantly-changing numbers, but fortunately you can drop its transparency to the point where it blends in with your wallpaper.

Minutes -> Orzeszek Timer

I will occasionally timebox my attention by working on something uninterrupted for 48 minutes.  Minutes is a simple Dashboard widget for OS X that will act as a countdown timer.  The best simplistic replacement I could find on Windows was Orzeszek Timer:

Volume Popup -> 3RVX

I only want two things out of my Microsoft multimedia keyboard:  Working volume/mute buttons, with on-screen display, and a working play/pause button.  Microsoft 64-bit IntelliType drivers are supposed to provide this, but they fail–volume works, with OSD, but the play/pause button just brings iTunes to focus.  It doesn’t actually work (and some Googling seems to confirm this).

The drivers themselves aren’t required for the media keys to work–you only need drivers if you’re using the other buttons–so I just uninstalled them.  Now my keys work, but I don’t get volume OSD, since Windows 7 doesn’t provide this itself.  If you’re on a laptop, or using an Apple keyboard, Apple’s Boot Camp drivers should provide this.

Fortunately, I discovered 3RVX, which is a skinnable volume OSD, initially designed to mimic Mac OS X.  If you want the volume bars to line up, one per button press, make sure you set up your volume up/down keys as hotkeys in 3RVX itself, and check “custom volume change amount” with the default of 6.25%.  Voila, Mac-like volume OSD!

Butler iTunes Popup -> Custom AHK/iTunes Plugin/Growl Setup

I used Butler on OS X for two things:  text snippets and its “now playing” iTunes popup.  Growl for Windows has an iTunes plugin that will growl on track changes, but it’s too noisy for me.  I don’t want to know each and every track that I play (or that the Shoutcast station I’m listening to is playing).  Rather, I hear a catchy part of a song and want to quickly know what’s playing without switching out my active window.  My ideal setup is a global hotkey triggering an on-demand now playing popup.

Growl for Windows is an open-source project, so I quickly looked at modifying their Growl for iTunes plugin to optionally trigger from a global hotkey.  That’s a bit out of my depth, so I went with one of my strengths instead:  Cobbling together a bunch of systems!

First, I use the AMIP iTunes plugin.  This plugin can do a few things, but all I have it do is maintain an iTunes.txt file in My Documents.  This file is either blank, if nothing is playing, or it contains the name of the currently-playing track.  Note that the Now Playing iTunes plugin does the same thing, but doesn’t handle Shoutcast (it will just tell you the station name, not the song name from the Shoutcast metadata).

Next, I simply use an AutoHotkey script to pipe this file to Growl for Window’s provided growlnotify.exe utility when I press a global hotkey.

^+i::
FileRead, Song, %A_MyDocuments%\itunes.txt
Run “C:\Program Files (x86)\Growl for Windows\growlnotify.exe” /i:”.\icons\itunes.png” /t:”%Song%” /r:”Now Playing” /n:”Now Playing” /a:”Now Playing” “iTunes”
return

This is a bit of a hack, but at the same time it’s a useful setup–it’s nice to have a quick AHK->Growl pipeline working, which I can use for all kinds of quick stat checking.

Multi-Monitor Wallpaper

Windows 7 is too dumb to handle multi-monitor wallpapers out of the box.  I used DisplayFusion to do this.  It has its own hooks/hotkeys, too, but you don’t have to run it persistently to keep your multi-monitor wallpaper working.

I run a 37″ Westinghouse in 1080p, with a 24″ Dell in portrait mode next to it.  Great for coding/IM plus media/games!  My wallpaper images are from the excellent Blatte’s Backgrounds.

Useful Hotkeys

  • Windows-1, Windows-2, Windows-3, etc will launch programs pinned to that position in your dock.  This is pretty nice.
  • Shift-Windows-X will launch another instance (ie another Explorer window, another Firefox window), as opposite to bringing it back into focus.
  • Ctrl-Shift-Windows-X will launch a pinned dock application with administrative privileges.
  • Ctrl-clicking on a dock icon will cycle through that application’s open windows.  This solved a big annoyance for me with Digsby; I know I have a new IM waiting, but to get to it I would have to click twice (once to bring up the thumbnails to all Digsby windows, and once to click on my IM window).  Control-clicking on the Digsby icon will bring you to the last Digsby window!
  • Windows-Left, Windows-Right will align the current window left/right, maximum height.  Handy to throw two windows side by side.
  • Windows-Up, Windows-Down will maximize or minimize the current window.
  • Shift-Windows-Left, Shift-Windows-Right will move the current window between monitors (in a multi-monitor setup).  Works on maximized windows, too, which is great.
  • Shift-Windows-Up will maximize the height of the current window (but not its width).
  • Shift-Right-Click a folder or file will show additional context menu items.  For a folder this includes “open command window here”, and for a file “copy as path”.  Both are great if you do any command-line work.
  • Windows-Space will show your desktop.  I guess this is to peek at any Gadgets you have installed, but so far I haven’t found any compelling gadgets.
  • Windows-+ and Windows– (minus) will zoom in and out at the operating system level.
  • Screenshots can be taken with the “Snipping” tool.  Just use the start menu search to whip it open.

Things I’ll Miss – Spotlight Menu Search

Spotlight menu search is a massive time saver on a Mac.  OS X enforces a system-wide paradigm for menu items.  There is only one active menu bar, and it’s for the active application.  Because of this homogeneous setup, Apple can do things like search this menu.  This is amazing for big applications–want to do a hue shift in Photoshop, but can’t remember where it’s hiding?  Hit cmd-shift-?, type ‘hue’, and voila–it’ll even show you where in the menu it’s located if you arrow down to it:

I would surprised if something this universal is even possible on Windows 7.

Things I’ll Miss – Quick Look

Quick Look is a feature of the Mac Finder–press space, and whatever file you have highlighted will preview.  Look at PSDs, PDFs, listen to audio files, watch videos, and generally get an idea of what it is you’re dealing with.  Super-fast, super useful.  Windows 7 thumbnails and previews are getting better, but Quick Look’s reliability and speed is something I’ll miss.

Is there a universal preview application for Windows that can compete with this?

Gripes – File System

The only thing that really bugs me about Windows itself is how slow file operations can be.  I guess NTFS is in dire need of an overhaul–that “new technology” moniker is pretty old now–because Mac OS X can do much, much faster file/folder manipulation than Windows (and I’m running my Windows 7 install on a Intel X-25 SSD).  Duplicating a 500MB, several-thousand-file Unity project is quite fast on OS X, but super slow on Windows.  Even worse, Windows actually sanitizes folder names, which breaks the project.  We space-prefix some folders, which makes perfect sense inside of Unity, and Windows will actually scrub those for you.

Gripes – Unity

This is pretty specific to game developers using the excellent Unity engine (and for the record, we absolutely love Unity).  But Unity’s Windows version is noticeably slower than its Mac counterpart.  This makes sense, since Unity was Mac-only for much of its life.  Going from clicking play to interacting with your content is probably 50% or more slower, on the exact same hardware, and pressing stop will actually lock your entire system while it thinks (no more clicking stop and tabbing over to your code editor while Unity cleans itself up).  In general things are slower, quirkier, and crashier.  Hopefully Unity 3.0 will see more speed/stability parity between the two operating systems, since Windows is now in on the ground floor.

The tipping point for me on the OS transition, by the way, was the ease in which we added XInput support to Unity for Off-Road Velociraptor Safari development.  I really wanted rumble for our publisher demo version, and it was actually easier to move my desktop to Windows than to bother with a custom Mac plugin that somehow supported vibration.

What Else?

That sums up my discoveries a few days into my Mac-to-Windows switch.  There are some missing elements still, but so far I’m happy with the functionality I’ve been able to put together.  What must-have Windows 7 applications have I missed (particularly anything from any one-time Mac users)?

What to Do If: Bookworm Makes You Feel Stupid

August 2nd, 2009

Bookworm LexPopCap’s Bookworm is a fantastic franchise. I’ve played a lot of Bookworm in my day, particularly because my company’s first game, years and years ago, was a spelling word game.  But I never became very good at word games, despite playing dozens of them while surveying the genre’s design landscape.

Word Game Players

Other people are insanely good at word games.  We spent a few days at the Independent Games Festival in 2004 demoing Beesly’s Buzzwords, and the occasional word game wizard would come forth from the crowd to spell 7- and 8- letter words with alarming rapidity.  I’ve also witnessed people who play word games daily but never seem to get any better, but still seem to enjoy the experience.

I don’t get any better.  And Bookworm is now available on the iPhone–at the everything-on-the-iPhone-is-cheap-as-dirt price of $0.99–so I thought I’d check it out.  It’s the usual PopCap fare, with fantastic polish, some new features, solid production…but it still makes me feel dumb.

How to Play Spelling Games

The skill of a word game is “chunking” your mental searches into abstract-but-useful combinations of letters.  They’re hard to see at first because we rarely build mental associations with them.  The beginner searches for consonant-vowel-consonant patterns:  pot, dog, cat, tip, etc.  However, most spelling games begin to punish short words, and Bookworm is no exception.  To expand their search the intermediate player begins to see prefixes and suffixes, like “ing” or “ers”.  More advanced players see all kinds of subtle pairs, like “av”, “dg”, “ch”, “dr”, “gr” (think changing, dodge, greater, avalanche).

Automating Bookworm

I get all of this, but I still have a hard time increasing my mental lookup times for bizarre letter combinations.  I don’t like the feeling of floundering at a task that I know is more well-suited to a computer.  So I thought:  Why not have a computer do the actual work?  I saw a great article on HN about a guy who wrote an automated Sudoku solver, where you take photos of printed puzzles and it solves them.  Doing the same with Bookworm should be much easier, since I can screenshot the game and deal with pixel-perfect data.

And all of the words for this board:

8-Letter Words:
Briskets B(0,2) R(1,2) I(1,1) S(2,1) K(3,1) E(4,2) T(5,2) S(6,3)
Sirvente S(2,1) I(1,1) R(1,2) V(2,2) E(3,2) N(4,3) T(5,2) E(4,2)

7-Letter Words:
Branner B(0,2) R(0,3) A(0,4) N(0,5) N(1,4) E(2,5) R(1,5)
Birdmen B(0,2) I(1,1) R(1,2) D(1,3) M(2,3) E(3,2) N(4,3)
Brisker B(0,2) R(1,2) I(1,1) S(2,1) K(3,1) E(4,2) R(4,1)
Brisket B(0,2) R(1,2) I(1,1) S(2,1) K(3,1) E(4,2) T(5,2)
Reynard R(1,5) E(2,5) Y(2,4) N(1,4) A(0,4) R(0,3) D(1,3)
Becrime B(2,6) E(2,5) C(3,4) R(4,4) I(3,3) M(2,3) E(3,2)
Reynard R(3,5) E(2,5) Y(2,4) N(1,4) A(0,4) R(0,3) D(1,3)
Parents P(4,0) A(3,0) R(4,1) E(4,2) N(4,3) T(5,2) S(6,3)

6-Letter Words:
Branny B(0,2) R(0,3) A(0,4) N(0,5) N(1,4) Y(2,4)
Brandy B(0,2) R(0,3) A(0,4) N(1,4) D(1,3) Y(2,4)
Arrive A(0,4) R(0,3) R(1,2) I(1,1) V(2,2) E(3,2)
Risker R(1,2) I(1,1) S(2,1) K(3,1) E(4,2) R(4,1)
Driven D(1,3) R(1,2) I(1,1) V(2,2) E(3,2) N(4,3)
Sirdar S(2,1) I(1,1) R(1,2) D(1,3) A(0,4) R(0,3)
Skeets S(2,1) K(3,1) E(3,2) E(4,2) T(5,2) S(6,3)
Veiner V(2,2) E(3,2) I(3,3) N(4,3) E(4,2) R(4,1)
Venter V(2,2) E(3,2) N(4,3) T(5,2) E(4,2) R(4,1)
Minter M(2,3) I(3,3) N(4,3) T(5,2) E(4,2) R(4,1)
Brewer B(2,6) R(3,5) E(4,6) W(5,6) E(5,5) R(6,6)
Entera E(3,2) N(4,3) T(5,2) E(4,2) R(4,1) A(3,0)
Crewel C(3,4) R(3,5) E(4,6) W(5,6) E(5,5) L(5,4)
Cymene C(3,4) Y(2,4) M(2,3) E(3,2) N(4,3) E(4,2)
Reeler R(3,5) E(4,6) E(5,5) L(5,4) E(6,5) R(6,6)
Crewel C(3,6) R(3,5) E(4,6) W(5,6) E(5,5) L(5,4)
Parent P(4,0) A(3,0) R(4,1) E(4,2) N(4,3) T(5,2)
Mercer M(4,5) E(4,6) R(3,5) C(3,4) E(2,5) R(1,5)
Kraken K(5,1) R(4,1) A(3,0) K(3,1) E(3,2) N(4,3)
Kraken K(5,1) R(4,1) A(3,0) K(3,1) E(4,2) N(4,3)
Terais T(5,2) E(4,2) R(4,1) A(3,0) I(2,0) S(2,1)
Steeks S(6,3) T(5,2) E(4,2) E(3,2) K(3,1) S(2,1)

Processing Bookworm

The first step is to get data out of Bookworm itself.  The iPhone provides a screenshot function–press home and power simultaneously–and you can email the resulting JPG to yourself from Camera Roll (you can also pull it out of Xcode Organizer if you’re a developer).  Bookworm saves your board when you exit, so it’ll be waiting upon your return.  This gives you a nice clean JPG:

Bookworm Board

Next, you’re going to want to clean up some of the data.  This is especially important for boards where you have flaming tiles going on. The simplest solution is to threshold the pixels.  I used the fantastic ImageMagick suite of command-line image tools for this, with:  convert -threshold 25% input.jpg output.png

Bookworm OCR

From there it’s a simple matter of identifying the letters.  You can’t rely on pixel-perfect matches, due to the flaming tiles and other effects, but ImageMagick provides a handy compare tool in their suite.  I’m using: compare -metric mae thisTile.png letters/someLetter.png difference.png 2> compare.txt (for whatever reason ImageMagick on OSX outputs the numerical analysis to standard error, so I have to pipe it to get it out).  It even gives you a nice display of the difference between the two files:

You do need to capture each individual letter, but this is pretty easy to do, since you’re already isolating each letter to compare it against all known letters.  It took two minutes to go in and rename the temporary grid output to build my identification library:

Training Letters

Edit: Looks like I forgot to save an ‘H’ when I did the letter training. In the above images they’re matching as ‘R’, which is the next closest visual match. Whoops!

From there it’s a simple matter of recursing the entire board, looking for matches.  I don’t do anything particular clever on the search (it’s actually the fastest part of the process at just a few seconds).  I am checking for premature search death.  For instance, if the first few letters of a match are xyt, you can kill off the entire tree instead of continuing to search for words.  My enable2k dictionary of 180k words results in 380k possibilities (“strawberry” is stored as “str”, “stra”, “straw”, etc).

What’s Next?

I did this whole thing as one big experiment.  My thought process went from “man, I feel like a shitty computer when play word games–a computer could easily do this” to “oh yeah?  prove it, talky talk-man”.  The three hours that it took to make this were certainly more fun for me than three hours of actual Bookworm play would have been.  That was just my goal all along:  To find some way to turn a frustrating experience into a fun one.

I may launch it as a service, actually.  It would be amusing to unleash this on the hardcore Bookworm fans.  You would take a screenshot, email it to some address, and get your best words emailed back in a minute or two.  I wonder if people would actually use that?  Only one way to find out…

P.S. Trademarks, Cheating

Bookworm and the Lex character are registered trademarks of PopCap Games.  For the record, Bookworm for the iPhone has no online scoring.  I can’t do anything with this except feel less retarded when I play.


P.P.S. Wurdle

I did Wurdle, too. It uses an 8-neighbor grid, so it actually has more words (and the max grid size is 8×8 compared to Bookworm’s 7×7). The average 8×8 Wurdle board has ~1500 possible word solutions. Word lengths over 9 are rare and seem to appear in 25% or fewer layouts. Have a look:

Wurdle Clean
Wurdle OCR

Words:

12-Letter Words:
Criticasters C(3,2) R(4,2) I(3,3) T(2,4) I(1,5) C(2,5) A(3,6) S(3,5) T(4,4) E(5,4) R(5,5) S(4,5)

11-Letter Words:
Criticaster C(3,2) R(4,2) I(3,3) T(2,4) I(1,5) C(2,5) A(3,6) S(3,5) T(4,4) E(5,4) R(5,5)

9-Letter Words:
Freewares F(1,1) R(2,2) E(2,1) E(3,1) W(4,0) A(4,1) R(5,1) E(6,0) S(6,1)
Fricassee F(1,1) R(2,2) I(3,3) C(3,2) A(4,1) S(5,2) S(6,3) E(5,4) E(6,5)
Neuritics N(1,2) E(0,2) U(1,3) R(2,2) I(3,3) T(2,4) I(1,5) C(2,5) S(3,5)
Reasserts R(2,2) E(3,1) A(4,1) S(5,2) S(6,3) E(5,4) R(5,5) T(4,4) S(4,5)
Rectrices R(2,2) E(3,1) C(3,2) T(2,3) R(3,4) I(3,3) C(4,3) E(5,4) S(6,3)
Crassness C(3,2) R(4,2) A(4,1) S(5,2) S(6,3) N(6,4) E(5,4) S(4,5) S(3,5)
Reasserts R(4,2) E(3,1) A(4,1) S(5,2) S(6,3) E(5,4) R(5,5) T(4,4) S(4,5)
Rectrices R(4,2) E(3,1) C(3,2) T(2,3) R(2,2) I(3,3) C(4,3) E(5,4) S(6,3)
Crassness C(4,3) R(4,2) A(4,1) S(5,2) S(6,3) N(6,4) E(5,4) S(4,5) S(3,5)
Stricture S(4,5) T(4,4) R(3,4) I(3,3) C(3,2) T(2,3) U(1,3) R(2,2) E(2,1)
Frenetics F(5,6) R(5,5) E(6,5) N(6,4) E(5,4) T(4,4) I(3,3) C(4,3) S(5,2)

8-Letter Words:
Efferent E(0,0) F(1,1) F(2,0) E(3,1) R(2,2) E(2,1) N(1,2) T(2,3)
Generics G(0,1) E(0,2) N(1,2) E(2,1) R(2,2) I(3,3) C(4,3) S(5,2)
Gentrice G(0,1) E(0,2) N(1,2) T(2,3) R(2,2) I(3,3) C(3,2) E(3,1)
Efferent E(0,2) F(1,1) F(2,0) E(3,1) R(2,2) E(2,1) N(1,2) T(2,3)
Enticers E(0,2) N(1,2) T(2,3) I(3,3) C(3,2) E(3,1) R(4,2) S(5,2)
Dicastic D(0,4) I(1,5) C(2,5) A(3,6) S(3,5) T(4,4) I(3,3) C(3,2)