Friday, 14 September 2012

Intuitive flick gestures in Unity

One thing I implemented was flick gestures for iOS in Unity.  This turned out to be a little trickier than anticipated, because it must be carefully designed to feel good.  In our project, flicking is the primary method of interaction, so it's crucial that it's comfortable.

The challenge

Our project has the player throwing objects from a first-person perspective.  We use flick gestures to control the throws, so we have to translate a 2D gesture into a 3D velocity. So to get started, we'll consider what we have to work with.

The 2D gesture has a start position and end position, both in screen coordinates, as well as the time it took to complete the gesture.  In other words, we have the distance, time, and direction of the swipe in screen space.

Getting velocity

The first thing I did was turn the screen space velocity into world space velocity.  To do this, I took the swipe time divided by the swipe distance and multiplied it by a fudge value.  Although this strikes me as a fairly naive approach, it turned out to be surprisingly effective.  To finish, I spent some time tweaking the fudge value until the velocity of the thrown stones aligned with my flick gesture in a way that felt comfortable and intuitive.

Getting direction

First try

Translating the direction of the flick into a 3D direction took a little more effort.  The first thing I tried was getting the angle of the swipe, creating a rotation matrix, and applying it to the forward vector of the camera. At the time we were only concerned with the rotation of the stone around y, in other words the yaw.  This approach felt terrible.  As a player it was difficult to know where you'd throw your stone based on your flick gesture, which means it was completely unintuitive and deemed no good.

Second try

The next attempt was to throw the stone in the direction that the player's finger stopped.  Luckily, Unity has a function ScreenPointToRay built-in to the camera, so all I had to do was take the final position of the touch, and convert it into a ray.  So when I throw the stone, I take the direction of the ray multiplied by the velocity we calculated earlier.  

Initially I was concerned that this approach would feel awkward.  The user could potentially swipe from the bottom left towards the top right, but if she didn't cross the centre of the screen the stone would still fly leftwards.  It turns out I was dead wrong for two reasons.  First, moves like these are incredibly unlikely.  For example, a user who wants to throw a stone right isn't likely to start from the bottom left of the screen and move only halfway across.  He's much more likely to start from the center of the screen and move right.  Second, control is so good that the player is likely to understand it immediately.  Thus he'll never make some sort of awkward flick and be surprised by the result.

So, as you may have guessed, this is the scheme we stuck with.  In fact, it felt so good that I decided to revisit the velocity equation.  I was curious if using world space would feel better than screen space distance for determining the velocity of the stone.  Although it turned out badly due to the perspective of our camera, I'll explain how this is done for those want to try it.

Getting world space distance of the flick

Using our trusty ScreenPointToRay function, we can calculate the exact distance that the player's finger travelled in the game world.  To do this, turn the starting position and ending position of the flick into rays.  Then, perform a raycast onto your terrain, or other relevant collider, using each of these rays.  If the raycast hits, you can extract the world space coordinates of the collision from the RaycastHit object.  Finally, subtract one set of coordinates from the other one and you've got a vector that'll tell you the distance travelled in the game world.

Finishing touches

As a finishing touch you'll probably want to add some flick gesture recognition.  For example, checking the time it took the player to perform the gesture.  This will allow you to differentiate between a flick and a slow swipe. If the direction of you flick matters you may want to check that as well.  For example, in our game we only want flicks that travel from bottom to top.  To determine the direction of the flick I would use atan2 to get the angle of the flick vector and decide if it's within your range of acceptable values.

The code I used while prototyping can be found here:  Please leave any questions or suggestions in the comments.