Skip to content

Set Up Smooth Pursuit Interaction

Smooth Pursuit Eye Movements

Wikipedia puts it nicely:

Smooth pursuit eye movements allow the eyes to closely follow a moving object. It is one of two ways that visual animals can voluntarily shift gaze, the other being saccadic eye movements. The pursuit of targets moving with velocities of greater than 30°/s tends to require catch-up saccades. From Wikipedia: Smooth Pursuit

Smooth pursuit eye movements are important in eye tracking because they can be used for analytics and interaction on par with gaze interaction.

smooth-pursuits

Vertical linear smooth pursuit movements using linear targets from the Obital Framwork.

Smooth Pursuit Interaction

Interaction with smooth pursuits is built on the principle of:

  1. Display one or more moving objects varying in either phase, amplitude or path.
  2. Record the eye or gaze movement
  3. Correlate the eye- and object movements, and decide which object is being followed, if any.
  4. Perform an action on the selected object, like a regular button.

One advantage of smooth pursuit interaction is that it does not require calibration, making it ideal for quick interaction. Another advantage is in the nature of correlating movements. Since smooth pursuit eye movements cannot be done without stimulus, it more robust than gaze interaction - especially on eye trackers without specialized hardware.

Smooth Pursuit graph displaying correlation between targets and eye gaze

Above is an example of how target correlation could look. The graph depicts x-coordinates (upper graph) and y-coordinates (lower graph) of target and eye movements in normalized space. Notice how the gaze clearly follows target 2. The Obital Framework takes care of correctly classifying the correct target, so that even if the y-coordinates seems slightly off in the beginning, target 2 is still correctly classified.

Limitations of Smooth Pursuits

As any type of eye interaction, smooth pursuit has limitations. First of all, there's an inherent difference in how humans perform smooth pursuits. Some have slight delays, some perform more catch-up saccades, while others have involuntary nystagmus, and there's even a difference in our eye movements whether we're tracking object in or against our reading direction. This means that some people may have issues interacting with smooth pursuits. In our experience, these counts mostly people with severe eye movement disabilities.

As eye movements are individual and varying in several conditions, the correlation between target- and eye movements becomes stochastic, making it difficult to set a pre-defined activation time. The Obital Framework takes a different approac - the activation time is dynamic, making it faster under optimal conditions, and slower in others. This ensures a robust user experience, minimizing errors.

The Obital Framework takes care of properly displaying the targets, animating them and doing the entire interaction pipeline. All you have to do is design them, and decide what to do on selection. Let's set it up!

Create a Smooth Pursuit View Controller

The SPViewController class manages targets, the term we use for moving objects that can be interacted with. It also manages registering all targets to listen to eye tracking updates, and notifying you whenever there's an interaction. See the reference for a full overview of your options. We'll use this to create a simple smooth pursuits based app - a linear pursuit that changes color upon activation:

Smooth Pursuit app

Several things are going on in the video above.

  • The target is moving linearly. We have full control over phase, amplitude and path shape.
  • A ring on the perimiter of the target indicates the progress of an interaction. As it reaches full circle, the target is activated.
  • On activation, the target performs a "pulse", to give feedback to the user.

All of these actions are completely customizable. We'll go through how here.

Start by creating a new view controller "MySPViewController" as a child of SPViewController:

import Foundation
import Obital_iOS

class MySPViewController: SPViewController {

    override func viewDidLoad() {
        // We need to create a session manager object and start it.
        if sessionManager == nil {
            sessionManager = SessionManager()
            sessionManager?.start()
        }

        super.viewDidLoad()
    }
}

SPViewController inherits from SessionStatusViewController, which means it automatically manages all tracking, and provides feedback if the user is not correctly placed realtive to the device.

If you run the project with "MySPViewController" as the app entry point, it configures the default layout which is 8 targets moving circular in the center of the screen. Now we need to configure it.

Configure Smooth Pursuits View Controller

SPViewController is configured similar to swift's own UITableViewController delegate methods. These methods are:

open func numberOfSections() -> Int
open func spType(inSection sectionIndex: Int) -> SPType
open func numberOfTargets(inSection index: Int) -> Int
open func configure(section: Section) -> Section
open func configure(target: TargetNode, inSection section: Int, atIndex index: Int) -> TargetNode

Overriding the return value of these methods configures the relevant parameter in the view controller. These methods are automatically called on viewDidAppear(), and can be reloaded by calling the reload() method on SPViewController. Let's start by changing the number of targets to one:

override func numberOfTargets(inSection index: Int) -> Int {
    return 1
}

If you run the project now, you'll see a single target moving in a circular pattern. To make it move in a linear fashion instead, configure it by overriding the section configuration:

override func configure(section: Section) -> Section {
    // Reduce the height so it stays within the screen bounds
    // and away from the edges.
    let height = self.view.bounds.height * 0.9
    let width = self.view.bounds.width
    // Create the section with a slight offset in y to accommodate
    // the height.
    section.bounds = CGRect(x: 0,
                            y: self.view.bounds.height - height,
                            width: width,
                            height: height)
    // Create start- and end points
    let end = CGPoint(x: width / 2, y: height)
    let start = CGPoint(x: width / 2, y: section.bounds.minY)
    // Set the pursuit type to be linear going from start to end
    // point in "normal" direction.
    section.pursuitType = .linear(p1: start, p2: end, .normal)

    return section
}

A section is like a UIView, but for pursuits. By configuring the bounds, we make sure that the target cannot go outside of the screen space. This should configure the pursuit to be linear, going from the bottom to the top of the screen.

Reacting to User Interaction

Now it's time to react to user interactions. SPViewController includes a handy overridable didActivate(_) method that you can use to react to interactions:

open func didActivate(target: TargetNode,
                      inSection sectionIndex: Int,
                      atIndex targetIndex: Int)

This method supplies a reference to the target node, that you can modify directly by accessing it through here. It also informs you what section the target is placed in, and at what index, making it easy for you to identify which one it is. Let's make it do something!

override func didActivate(target: TargetNode,
                          inSection sectionIndex: Int,
                          atIndex targetIndex: Int) {
        target.react(with: .pulse)
        target.color = .random
}

// Extension to generate a random UIColor.
fileprivate extension UIColor {
    class var random: UIColor {
        return UIColor(red: .random(in: 0...1),
                       green: .random(in: 0...1),
                       blue: .random(in: 0...1),
                       alpha: 1.0)
    }
}

The react method takes a TargetReaction enum as parameter. Try out the different animations!