Errant Signals

In the Spirit of “Do, Don’t show”, I’d like to open this series on the XBox 360 controller to let you feel what’s really happening when you use a half decent joystick. And the truth is, what goes on under the hood is uglier than you think. After this, you’re going to run up and hug the first developer you know who has gotten their control scheme just right, and whisper tearfully in their ear “I’m so sorry… I didn’t know!”

First off, grab a USB 360 controller, and plug it in. No wait! Install the Unity Plugin if you haven’t already. Actually, the order’s not important. Do not freak out! It’s not a big deal! Once you’ve got that sorted, click on the thing below:

If you’ve already got to manipulating your inputs (and there is no way to say that which doesn’t sound completely sexy; sorry, I tried), you’ll see that you’re plastering these two textures with white pixels. There’s no special code distorting the stick’s inputs: this is the raw signal straight from the source. Or, at least, as raw as Unity will give it to you (which, I’m guessing, is as clean as any other API).

Try playing around with the sticks, almost meditating on what you’re doing, and how it feels.

At some point, you’ll find yourself tracing the extremities of the analogue thumbs, moving the sticks in a circular motion, resting on the edges of the controller case’s gate.

Feeling the outside edges of inputs of an XBox 360 Controller.

In deflecting the sticks to the physical limit of the circular gate housing each stick, you’d expect a perfect circle to be drawn; but something’s off. It seems as though we’re missing the corners of the input! Our interface bandwidth is clipped into a sort beveled square. And to rub it in, the shapes resolved by the left and right sticks aren’t even the same (and this is notably an even worse on the DualShock/SixAxis’s left and right sticks, if memory serves me correctly. Though it probably doesn’t because of this one time when I wasn’t paying attention and I got my head squeezed between two doors on the London Underground, and was totally concussed, and went to work anyway, and wrote hilarious code, and made the coders angry).

Not only are the corners physically out of reach, there’s also some detail at the top, bottom, left and right which is cropped off!

'Sensitive Area'. 'Ignored'. 'Unreachable'. Story of my life.

And yet, while far from perfect, apparently people aren’t throwing their controllers out of windows in disgust. It seems modern video games somehow seem to cope. To figure out why, let’s get you drawing as straight a line as you can. Here, fresh canvas:

You may also notice that it’s pretty difficult to create anything like a straight diagonal line. Lines in cardinal directions (up, down, left, right) are relatively easy to do because the spring pressure exerted to bring the stick back to the center is happening twice: once along the horizontal axis, and once along vertical. You’re essentially finding a “groove” of least resistance in the X and Y axes, and it’s guiding your thumb.

I spent way too long on this.

Probably why artists in the games industry don't draw using XBox controllers.

The fact that you can’t draw a diagonal is, frankly, your thumb’s fault (assuming you are using thumbs – I’ve seen incredible streetfighter players with motor neuron disease use only their tongues poking around under their cheeks to pull off amazing combos that I can’t even do). Your thumbs are not as accurate as you hoped: if you try to draw a circle without leaning on the guide rails, you end up with concept art from the now canned PacMan World 4: PacMan World Universe Land (see fig. whatever: right hand side).

You may be able to “lock” your thumb and maintain a position, but the act of moving with finesse feels kind of drunken. Your thumb is trying to balance its force against the reactive force of two springs, and those springs are acting perpendicularly to one another, because that’s what seemed to make sense to the engineers behind the sticks. Your thumb itself doesn’t naturally move better along cardinal axes, because of… evolution I guess?

In fact, I’ve heard some suggest that games should rotate their inputs such that the thumb is regarded on its rotation and bendiness for X and Y inputs, respectively. That might be an idea, but it’s incompatible with the natural grooves found in the XBox controller, due to the axis aligned spring setup. And it also requires calibration per user, so that’s just irritating.

As you move your thumb around, notice the different pressures that feed back. When you try to move through the full deflection range very slowly and gradually, you’re actually moving in little hops as your thumb’s force overcomes the resistant force of the spring, but slightly overcompensates as it beats the static friction, and finds a new equalibrium. The fact that you feel different pressures at different degrees of displacement, or “throw” is really important: That force provides the haptic feedback to feel where your thumb is in the entire range of motion of the stick. You don’t have to look down to see where your thumb is, because you feel the counter-force from the stick, pushing as hard against you as you are against it, in the direction you’re pushing it. That’s sort of the genius of this interface. Compare using an old fashioned “yoke” style joystick with no return springs what so ever, and you immediately grasp how important the gradual return force is to the haptic feedback of the controller.

And when you push hard out to the outer gate/rail of the stick housing, you really feel that you’ve hit the limit – no amount of extra force will yeild further displacement… unless you’re good at tearing phonebooks in half with your bear hands. Yeah I just said “bear hands”. What you gonna do? Tear my phonebooks in half? I don’t even have any, you bear handed buffoon! Use bing ya idiot!

The lines you try to create also seem spotted. This is partly down to  to how frequently the pad is sampled or “Polled” (If any unity users could tell me the default polling, it’d fill in this gap right here: ________________). But when filling in the entire bevelled square (which will take some time!) you may find that some values inside the detectable area are unaccounted for. This might be the result of crunching numbers within drivers and losing detail along the way but it’s mostly that your thumbs (the best thumbs of any animal!) are still not the perfect herald of your intent.

And that’s really the first lesson here: human output, itself, is fuzzy, and interfaces rarely read a pure signal. And depending on how it’s crafted, the interface itself could feed noise back into the signal it’s trying to read – that war of spring vs. thumb you saw as you tried, in vain, to draw a straight line steadily.

There’s easily enough data from the stick to feed into a mechanic which reacts properly to analogue inputs, but we have to accept that there’s some work to do in filtering out the noise from both the player, and the interface.

Good Interface design aims to clean the broad and noisy signals that users can potentially give down to a limited set that they can easily and intentionally give. Often this is by funneling that broad range of inputs into a limited set of actions, cutting back what you can do in the game to what you must do, and ignoring, or at least smoothing, the noisy inbetweens.

On the hardware side, there are also intentional limits to the player’s range of inputs: by physically limiting our interface freedoms (as with the restrictor plate on a joystick), a very clear, but unspoken contract is established between player and game: our unconscious physical ticks, or habits like picking our nose aren’t registered within the game, embarrassing us, making us feel self conscious about the weird ways we move when we’re not watching. Or, I dunno, maybe that’s just me.

Thanks to these physical restrictions (to throw, or to the limit a button must be pressed before registering etc) we can also tell if we gave a signal by mistake, because we feel ourselves physically change the interface, and get a miniature feedback loop from its clunking and clattering about. Every time we instinctively scratch our nose, or cough, our game doesn’t cause our avatar to jump off the closest cliff, because physical interfaces don’t really watch for such things. But if we missed that quick scope shot because we sneezed and pulled the stick off target accidentally, we know it because our thumbs tell us pretty much instantly.

Interpretive interfaces like the Kinect, voice recognition and (to some degree) touch screens, don’t give this naturally haptic feedback, where we intuitively know what signal we’re expressing: our own bodies are almost too intuitive to us for us to really know what they’re doing: try watching yourself dancing on video and try to come away from that feeling better about yourself. If you think you’re great before the video, and awful after, you suffer from proprioceptive self delusion… like the rest of us.

Interfaces are a translator for your intent into machine manageable form. It just so happens that you have horrible grammar, you rarely say what you mean, and they are confused tourists trying to understand your directions through your loud, fake frensch achhsont.

Game developers have a duty to establish a “contract” with the player by giving them an interface, and teaching them the interactions which are actually being listened for in the game. Knowing this, the player accepts that their fuzzy signal is filtered down into the agreed upon in-game action, and, having established this contrivance, they player can feel immersed through the established channels of interaction and feedback, and start to obtain flow.

The key to doing this successfully is to show the player that they are being listened to, how they are being listened to, and by either being successfully devious about filtering the noise in hidden ways, or at least up front about how the noise is overcome via acknowledgement of the interface’s shortcomings.

I’ll be going into various little techniques to do this, hopefully in a useful amount of detail. Next time we continue with the 360 controller to see these principles in action, starting with DeadZones! If it is exciting, I will be the first to write to my minister in mock outrage. Writing about kinaesthetics is awful, and I intend to keep it that way.

Here’s a bonus link to Chris Franklin’s take on Kinaesthetics (I did some co-writing. You can tell my bits from where the sentences just sorta go off like in some meandering direction and you don’t actually think Chris is even going to draw breath because seriously he’ll just read out any old crap I write because he’s a really nice guy and he doesn’t want to offend me even though I thrive on that kind of negative attention since I can’t seem to get any other kind of attention from anyone not even my parents and i’m pretty sure they love me?)

10 thoughts on “Errant Signals

  1. This blog is turning out awesome. Nice work dude :)

    I don’t have an Xbox controller, but I can imagine that your demos are informative.

    One thing I’d love to talk about is the kinaesthetics of touch interfaces. You seem to be biased against them. I’ve seen first hand how a game controller can be incredibly intimidating to many (most?) people, but most people seem to be able to pick up touch interfaces and just start using them immediately. It’s kind of sad because of how satisfying joysticks and buttons are, but that just seems to be the way it is.

    Clearly both interfaces have their pros and cons, and therefore likely have a “grain” to them in terms of gameplay. The game controller seems to favor, for lack of a better term, arcade games. What about touch interfaces?

    • Biased? I hope not, as I’ve been working with them for a while now. You’re exactly right that they have their own grain and strengths and weaknesses. Coming from physical interfaces to touch screens can definitely be a bit confusing and frustrating, as you have to ditch a lot of go-to concepts that you had previously, but if your principles are good, there’s ultimately nothing stopping you making a game which suits the interface.

      However, to figure out what suits it, one has to acknowledge the problems it faces (in the case of touch screens there are a LOT of ergonomic issues, quite apart from input latency, haptic feedback etc. For instance, you cannot man-handle an iPad or android phone for fear of your hand counting as an errant touch). That’s what I’ve been attempting to do with this XBox controller write up: expose its flaws, go through the 5 stages of grief, reach acceptance, and work with it.

      Once the limitations are exposed and accepted, we avoid the design landmines and get on with making the right game for the interface – which is how it has always been. Embrace the limitations. Let the interface tell you what game it wants.

      So, any frustration I have is with the issues or limitations of the interface, rather than the challenge of designing for it. I do believe touch screens are a bit of a transitional technology. I think we’ll look back at them as a bit quaint. But while they’re here, there’s still fun things to do with them.

      • You’re spot on on everything. I hate that an analog stick with a bunch of buttons is all we have to manipulate the game. And with the increasing complexity on both the sides of this interface – the system running on the computer and consequently in the model in a players mind – its a pity that there is such a huge bottleneck of information flow. Its like using a telegraph line instead of cables for the internet backbone.
        Employing pieces of intent to interact with a system is not the problem(All our body movements including the vast set of facial expressions can after all be broken down into chunks of muscular intents ). Its just that they are all preset in a way that limits the outcomes to triggering animations, setting parameters(“move to this point” or “shoot!” or “run for your life” ), creating inflection points(in the story) etc. After subjecting the system to a series of these actions, the player then has to reconcile his model with the game instance by spinning amusing little anecdotes.
        I was heartbroken when realized that my favorite game at a point, SimCity 3000, was actually just 2D cellular automata with a bit of fluff on top pretending to behave like a city( i was unaware of its origins). I have never since been able to play that game with the same level of trust. Its like the game is saying “ill keep on doing my thing, you can think up of an explanation for it” ( i used to blame my own model for any anomaly earlier). And in a way the controller too renders the model an inconsequential mental masturbation .

        • On a second reading this sounds like i’m dissing or something but its a pet peeve that iv’e been harboring for a long time and think about a lot. In fact i have never used a controller except for once when i played Gish with a generic controller. So it’s more of a fancy “they’re all doing it wrong” notion that is misguided but still drives me to learn more about games.

          • Well, what I guess I’m arguing in the piece is that a bottle neck is actually important. It’s important for the player to know when they are and aren’t “connected” to a system. We make so many gestures we don’t intend to. It’s really GOOD when a haptic interface tells us that we are transmitting a signal (implicitly, by us feeling its resistance). If we don’t want to generate a signal, we can just “let go”, and we can FEEL that we’ve let go. This can be done in other ways with non haptic interfaces (maybe a light going on on a kinect when you leave its vision cone) but it’s a fuzzier notification to our senses.

            On top of this, having physical resistance is a great modulator of our intended signal – it reduces vibration and unintentional signals. It’s like leaning on rail when walking up and down stairs. Some of us really need that railing to avoid unbalancing and falling.

            I’d say it’s only a restrictive bottle neck if you need the player to say more than is possible. But that’s not so much an issue of the interface as an unrealistic design expectation. This is why it’s important to regard interface as one of your hard restrictions – as hard a restriction as the capability of your target hardware. Design around the interface’s strengths and weaknesses – if you don’t, you’re not going to get many gains by blaming the controller.

    • …As far as what suits touch screens best, I’ll get into that, but in short, quick swipes are ideal. Taps are physically painful. Holds block your view. Pinches are fairly nice too, but some of apples decisions with the most recent iOS have undermined the applicability of more than 3 simultaneous touches (even though ten can be registered at once).

      But with all of these rules of thumb there’s room for skillful subversion. As with anything, it’s worth learning the rules before you break them.

      • Yeah I certainly have my own ideas, but I wanted to hear yours, heh.

        My intention with a future project is to use swipes, but for slower, more intentional gestures. And potentially pinches as well, not sure yet.

        “Let the interface tell you what game it wants.”

        I guess what surprises me so much is that the interface is telling me “simulated physical touch,” and yet I’ve seen very few videogames based on so intimate a concept.

        In the same way a gamepad seems to be designed specifically for arcade experiences, the touch interface seems to be designed specifically for romance.

        No wonder iDevices are such desire-based objects. 😛

  2. What if you assume the player has access to a flat planar surface near the gaming device? You might be able to piggyback your controls off this plane to provide accurate and reliable two-dimensional input.

    I’m picturing some kind of haptic device using high frequency visual polling (maybe even a laser) to determine its relative motion across this surface. I imagine you could get several orders of magnitude more accuracy than the spring loaded sticks you describe, perhaps even into the hundreds of DPI.

    Then all these inelegant hacks would be unnecessary.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>