rohan singh

software engineer. bicyclist & rock climber. craft beer addict.

What I’ve Been Reading (Dec. 26, 2011)

What follows are the most insightful, interesting, or otherwise noteworthy articles I’ve stumbled across over the past couple weeks.

  • Democracy is not a truth machine. One of the best articles I’ve read all week. In it, a bearded philosopher eloquently demonstrates why democratic processes can’t be trusted to find the truth about objective facts about reality. For example, while a democratic process can be used to decide what we should do about climate change, the underlying scientific reality is not up for debate.

  • Why we invented monsters. Paul Carr start with the idea that monsters grew into the human consciousness in the image of predators, such as felines and snakes. Eventually, as protohumans transformed from a prey species into predators, our imaginations transformed as well.

  • The scandal of the Alabama poor cut off from water. It really kills me that we need a foreign news source to report about the fact that America’s infrastructure is failing and the promises of citizenship aren’t panning out for some of those who have the greatest need. Reading this article, others like it, and watching the documentary Waiting for “Superman” over the weekend has left me with a lot of questions in a place where there are few good answers.

  • Why America’s Death Penalty Just Got Us Sanctioned by Europe. The European Union has decided that it will have no part in the barbarism involved in the methodical killing of human beings by their peers. I completely agree with Ford Vox’s characterization of the situation:

    The death penalty is a blind spot in our democracy, our own peculiar national anosognosia. It probably will take the rest of the world shouting us down in order to recognize our impairment.

  • A tale of two systems. The folks at Remapping Debate contrast a failing labor system in America with a robust, unionized system in Germany.

  • Hitch. Sam Harris, eloquent as always, on Christopher Hitchens’ death:

    Needless to say, he was effortlessly lucid and witty—and taking no prisoners. There should be a name for the peculiar cocktail of emotion I then enjoyed: one part astonishment, one part relief, two parts envy; stir. It would not be the last time I drank it in his honor.

  • Freakonomics: What Went Wrong?. Andrew Gelman and Kaiser Fung examine many of the conclusions in Freakonomics and Superfreakonomics which have worked themselves into the cultural consciousness and which are now taken for granted. Specifically, they show that many of these conclusions are just plain wrong.

Thoughts on Home Walls & Setting

I think of all the things I love about where I live, having a climbing wall in our studio is definitely in my top three. We had a couple of constraints in constructing the wall:

  • Our lease doesn’t allow pounding anything into the wall, so the structure had to be entirely free-standing.
  • I love bouldering on overhanging walls — the steeper the better. But we wanted to be sensitive to the fact that overhangs are not everyone’s cup of tea.

The Wall

In the end, we settled on an adjustable 10’ x 10’ lean-to design.

My roommate Chris and I have both found that mentioning the dimensions of the wall is a great heuristic for identifying climbers and non-climbers. The usual response from a non-climber is along the lines of, “Really? Isn’t that too small?”

Everyone always changes their mind on that after a good workout on the wall.

Chris and a friend handled most of the construction of the structure, which took about 40 hours in total. I think the most time-consuming part of the process — aside from waiting for the deck paint to dry – was drilling each of the dozens upon dozens of holes and installing a t-nut in each.

As I mentioned earlier, the incline is adjustable. Originally we had a 15° overhang, but have since dropped the wall down a bit further to 30°. This seems to be a nice compromise given that our friends are climbers with varying amounts of experience, specialties, and skill levels.

Setting — First Try

Initially, I tried a couple of traditional setting strategies. The specifics varied, but the basic idea was to visualize the outline of a problem and then set it piece by piece. I quickly learned a couple of things:

  1. This takes skill. Just because you are good at climbing a certain type of problem does not mean you will be able to set similar satisfying problems. It’s a completely different thought process and creative skillset.

  2. Setting is hard work! At first I was trying out the problem as I set each piece, which was very tiring. I heard a better strategy was to set the whole problem before trying to test and tweak it. This was great advice, but climbing up and down with holds and screwing them in is still a lot of work.

I came out with a newfound respect for routesetters at the gyms I go to, and it gave me some small amount of additional insight into bouldering problems in general.

Setting — Second Try

What ultimately made me try a different strategy was thinking more deeply about how we were using the wall. Given that the friends I climb with are all at different levels, we would generally just try climbing around or playing add-on rather than climbing specific problems.

Add-on is a very simple game. In a sense, it’s the bouldering version of H.O.R.S.E. The first climber chooses a starting position and makes a single move. The next climber copies that move, and then adds on another move. This is a great game when you have climbers at different levels, since everyone can play along and take part in some friendly competition.

Given that we were using the wall this way, I started thinking more about where to place holds on an individual basis, rather than where to place problems. Instead of trying to visualize an entire problem, I now just think about where a single hold might be interesting, and what orientation might make it fun to pull on.

Since I only have to consider one hold at a time, the cognitive load required is a lot lower. This is nice, since I’m not a very good routesetter. By just throwing up each hold in an interesting position, it’s a lot easier for me to fill the whole wall.

Here’s one of my favorite holds from So Ill, brilliantly entitled The Picking:

As a field test, we played quite a few games of add-on last Saturday, and it was tons of fun!

New Holds & Resetting

I’ve subscribed to So Ill’s hold of the month club, and it gives me an excuse to reset things and try something new every month or so. These just arrived recently:

I took the opportunity to move things around a fair bit. I moved The Picking much higher up, and rotated it to a fun new orientation. I also threw on a smattering of underclings and other new stuff.

I had actually tried putting The Picking in a similar orientation originally, but found that the it would just spin when pulled sideways — no matter how tightly I tried to bolt it in. This time I wised up and put a small hold below it to give it some reinforcement.

New game of add-on tomorrow, and we’ll see how it goes!

Installing Pgmagick on OS X

pgmagick is a Python wrapper for for ImageMagick (or GraphicsMagick). I needed it for something I was trying to run yesterday, but the instructions for OS X seem to have gotten lost in the depths of the Internet.

I’m assuming you have a working Homebrew, Python, and pip. You’ll also need my Homebrew formula for Boost.

Once you’ve got that, it’s pretty straightforward:

$ brew install imagemagick --with-magick-plus-plus
$ brew install boost --with-thread-unsafe
$ pip install pgmagick

Building Boost takes a long time so don’t fret if you don’t see anything for a while. Try it out and it should work:

$ python
Python 2.7.2 (default, Oct 20 2011, 17:33:50) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pgmagick
>>> pgmagick.gminfo.library + ' ' + pgmagick.gminfo.version
'ImageMagick 6.x.x'

For extra credit, you should also be able to install the Python Imaging Library (PIL) as well:

$ pip install -f http://effbot.org/downloads Imaging==1.1.7

This all worked for me on OS X 10.7 (Lion). Your mileage may vary on other versions.

Optimism

Software engineers are some of the most optimistic people I know.

Case in point:

Don’t Break the Back Button

Recently, I’ve worked on/with or used a few Android applications that “disable” the back button in some cases.

If you haven’t used Android, the back button is a prominent feature. Nearly every Android device has a hardware back button. The system maintains a stack of all the activities the user has navigated to, and when the user presses the back button, it takes the user to the previous activity.

This works both inside individual apps and also across process boundaries. For example, when I’m reading an email and push the back button, I end up back at my inbox. If I push the back button again, I’ll end up at whatever app I was in before I opened Gmail.

This functionality is all provided by the OS, so all you have to do to get it right is to not break anything.

Nonetheless, many developers have taken it upon themselves to intercept the back button and swallow it. Here’s what this is like when you intercept the back button on the web:

In other words, it’s completely obnoxious and unacceptable. Unfortunately, it seems that which we know to be obviously true in web development, is quickly forgotten when it comes to a mobile device.

I remember reading about a study at the Rochester Institute of Technology a few years ago, wherein researchers mounted eye-tracking devices on subjects who were washing their hands. A very simple task: washing your hands.

What the study found was a new type of eye movement, the “planful” movement. In short, before interacting with any object, we build a model of that interaction in our minds and look at the object, without being consciously aware that we are doing so. For example, your eyes and your brain are glancing at and scouting out the paper towel dispenser while you are still rinsing your hands.

The same concept that applies to navigating your bathroom sink applies to navigating a mobile operating system. Your app may only be displaying one specific activity on the screen, but you can be sure that the user has constructed a mental model of how they are going to navigate through your app, through their device, to accomplish whatever they need to accomplish today.

Most of the time, these mental models aren’t fully conscious or completely articulated. Ask someone how they would get from Angry Birds to Gmail, and you’ll hear something along the lines of, “I’ll save and back out of the game and then open up Gmail.” Their mental machinery has abstracted away the minutiae of what they would specifically push or click on.

Instead, users just rely on their intuitive experience of how to navigate a system. That intuitive experience makes certain assumptions regarding how this virtual world is structured. These assumptions are often as strong as your assumptions regarding the physical world — up is up, down is down, and if you retrace the way you came, you’ll always end up where you started.

When you break these unconscious and conscious assumptions regarding how things work, you are basically giving your users a mental segfault:

Don’t break the back button.

Getting ASP.NET to Play Nice With Opera & Wget

A few weeks ago we noticed an issue with some crawlers and search engines being unable to crawl Starbucks.com. Around the same time, Karl Dubost from Opera discovered that visitors using Opera were getting a parse error. In fact, a number of high-profile ASP.NET sites seem to be impacted by the latter issue.

If you’re just looking for the quick fix, skip ahead to the bottom of this post.

To sum up the issue, our server was responding with a Content-Type header of application/xhtml+xml to both Opera and wget. In Opera, this triggers an XML parser, which would fail since our content is actually text/html.

Of course, nowhere in our code were explicitly setting the Content-Type header to anything other than text/html, so the behavior was puzzling. Additionally, the server was responding with an application/xhtml+xml type even when the Accept header from the browser specified */*. This made no sense at all, since if the browser was willing to accept anything, we should be sending the content in its default text/html.

ASP.NET Browser Detection

As it turns out, ASP.NET has a somewhat questionable feature that allows you detect browser types and capabilities, largely based on the browser’s request headers. This is mediated through a browser definition file (*.browser), which is just a bunch of XML that matches up request header patterns to browser types and know capabilities. This file lives in your ASP.NET applicaiton’s App_Browsers folder.

The known capabilities for the current user agent are all available through the HttpContext.Request.Browser object.

As I said, all very questionable. The idea of having a giant database of browsers and what they are like just rubs me wrong and strikes me as unmaintainable. In fact, we only tried out the whole browser definition file as part of a proof-of-concept for some mobile pages. The feature never quite worked correctly, so we abandoned it and thought that was the end of that.

The insidious preferredRenderingMime

Of course, that wasn’t the end of that. For each browser definition in a browser definition file, you can define a preferredRenderingMime value for a browser. For example:

<capabilities>
  <capability name="preferredRenderingMime" value="application/xhtml+xml" />
</capabilities>

Most of the larger *.browser file compilations floating around the Web have Opera set to preferring a MIME type of application/xhtml+xml. A lot of files will default to a preferred type of application/xhtml+xml for all browsers that pass an Accept: / request header.

And even though we use ASP.NET MVC and no longer are using any of the browser detection stuff at all, the System.Web.UI.Page class has this fun code that runs when the page is processed:

1
2
3
4
5
6
7
8
9
private void SetIntrinsics(HttpContext context, bool allowAsync)
{
    ...

    HttpCapabilitiesBase browser = this._request.Browser;
    this._response.ContentType = browser.PreferredRenderingMime;

    ...
}

Brilliant, right? Since Opera prefers XHTML, this code does you the favor of automatically setting your response content type to application/xhtml+xml. Of course, since your actual content is still HTML, this causes an XML parse error and all your Opera visitors are hosed.

This might make sense if you are doing the classic ASP.NET Web Forms thing with server controls that adapt their rendering based on the browser’s preferred MIME. But even if you are using ASP.NET MVC, your .aspx views are still essentially pages, and this old code will still run.

Broken wget

Unfortunately, this SetIntrinsics code has another nasty side effect. If your application has already sent out response headers or content, it will just throw an exception since Response.ContentType can’t be set after response headers have already been sent.

In the case of Starbucks.com, this meant that issue wasn’t just that Opera visitors experienced a parse error, but that an exception would be thrown for any browser for which ASP.NET tried to switch the Response.ContentType. This would result in absolutely no content being served, resulting in a blank page for Opera visitors.

Furthermore, a large percentage of crawlers and search engines use wget to grab pages. wget sends an Accept header of */*, and runs into the same no-content issue. A fine mess all around.

The fix

The simplest fix, of course, is to get rid of any *.browser files you may be using in your application. I understand redirecting to a mobile version of your site for mobile browsers or the like, but basing any major functionality on guesses about the user’s browser is a great path to future pain.

If, however, you want to keep your browser definition files around, consider removing any uses of the preferrendRenderingMime capability. Here’s a regex that should be able to find those instances for you:

^.+"preferredRenderingMime".+$

Just do a find & replace on that and you should be good to go. This is what we ended up doing for Starbucks.com and Starbucks.co.uk, which I’m happy to say now work perfectly for Opera and wget users alike!

Outlier Is Pretty Awesome

We love our bikes not because we race them, ride trails or perform tricks with them, although sometimes we do all of that. We love our bikes mainly because they are our transportation, our freedom, our way of moving through the spaces we call our home.

Posting JSON With jQuery

Lately, I’ve been a big fan of JSON via HTTP POST to send data to a Web service. It’s awesome for a bunch of reasons:

  • Works equally well for simple and complex data. Regular HTTP POST works great for simple data, but gets unwieldy for anything hierarchical. XML works for complex data, but is too wordy for something simple.

  • Compact and human-readable. Really, this just comes with the territory when you’re using JSON for anything.

  • Easy to debug using Fiddler or related tools. You just have to type in or modify a JSON string when you want to do some ad-hoc testing.

  • Easy to add support for to any server. In fact, support for receiving JSON POST is built into the latest versions of CherryPy and ASP.NET MVC.

That said, JSON POST still doesn’t seem to be all that popular compared to an old-style URL-encoded HTTP POST. One explanation I’ve heard proposed for this is the lack of support in jQuery. By default, jQuery goes with the old-style POST. You know:

foo=bar&abc=xyz&x=123

Sure, you can pass in a complex type and jQuery will gladly serialize it into this format for you, but good luck trying to read that with your human eyes, or trying to piece it back together on the server in any sane way.

The really sad thing is that jQuery has a great getJSON() function that lets you receive JSON output and parses it appropriately. But there is no built-in support for posting raw JSON.

To that end, I threw together this little plugin that I use all over the place:

1
2
3
4
5
6
7
8
9
10
11
12
13
  jQuery.extend({
    postJSON: function(url, data, callback) {
      return jQuery.ajax({
        type: "POST",
        url: url,
        data: JSON.stringify(data),
        success: callback,
        dataType: "json",
        contentType: "application/json",
        processData: false
      });
    }
  });

A Little Trick

Here’s a little trick I use. I figured I’d share it just in case anyone else finds it useful.

Whenever I need to decide whether something is good, whether I support it or I don’t, it all comes together in one simple question. In the ideal world that I want to live in — the ideal, peaceful, happy, loving, accepting world — would this thing be or would it not be?

Once you start putting things into that perspective, everything becomes quite straightforward. Whatever the issue or thing is, all you have to do is ask that one simple question, and then the correct path of action often becomes quite clear.

Whether the issue you are considering is one of those big values-based divides of our day — war, abortion, gay marriage, religion — or whether it is just a simple matter of your daily dealings with other people, all you have to do is ask yourself: is this the world I want to live in?