Unity - Getting started

Adding eye tracking to your application with Pro SDK Unity Prefabs

The Tobii Pro SDK prefabs help you start working with the Tobii Pro eye trackers in Unity. The prefabs’ source code is included, and you can used it as a starting point for an eye-tracking-enabled project. However, there is no guarantee that they will suit a particular purpose; therefore, additional development and testing may be required for your project or use case.

There are two groups of prefabs, screen-based and VR, that cover the same use cases:
The VR prefabs are located in: \Assets\TobiiPro\VR\Prefabs
The screen based prefabs are located in: \Assets\TobiiPro\ScreenBased\Prefabs
The [EyeTracker] or [VREyeTracker] prefabs are required to get the other prefabs to work, but the rest are independent of each other.

A quick guide to adding Pro SDK Prefabs to your Unity project

1. Import the Tobii Pro SDK Unity package as described above
2. Locate the prefabs folder corresponding to the desired eye tracker (VR or ScreenBased) in the Unity Project Window.
3. Start by dragging the [EyeTracker]/[VREyeTracker] prefab and dropping it into your Hierarchy Window.
4. Select the new [EyeTracker]/[VREyeTracker] object and use the Inspector Window to edit it's configuration.
5. If more than one (ScreenBased) eye tracker are connected to your computer, use the "Eye Tracker Serial Start" text box to create a filter that matches the desired eye tracker. If you have only one eye tracker, tick the "Connect to first" box.
6. Drag-drop any of the other prefabs into your hierarchy. Edit their configurations in the Inspector Window.
7. All prefabs lack default keyboard shortcuts (for starting/toggling feature). Set new shortcuts previously unused keys.
8. Please check the readme files in the package for more detailed information about each Prefab!

Note: The VR prefabs require that you import the Steam VR package into your project!

Examples and Demo Scenes

Two demo scenes are located in \Assets\TobiiPro\Examples\PrefabDemo, one for VR and one for screen based eye trackers. Both highlight how you use the eye tracking prefabs in a simple scene. Test them as demos or use them as examples on how to use Pro SDK Unity Prefabs.

Developing your own behaviours using the Tobii Pro SDK API

The Tobii Pro SDK .NET binding is compatible with Unity's version of C#. For more information check the .NET API’s reference guide pages.

A quick guide to a functional application

Most eye tracking applications follow the same pattern in terms of in which order functionality is used. The order is usually as follows:

1. Browsing for eye trackers or selecting an eye tracker with known address.
2. Establishing a connection with the eye tracker.
3. Running a calibration procedure that calibrate the eye tracker to the user.
4. Setting up a subscription to gaze data, and collecting and saving the data on the computer running the application. In some cases, the data is also shown live by the application.

Here's how to do this with the Tobii Pro SDK API in Unity:

Step 1: Browsing

Start with the EyeTrackingOperations, a static class residing in Tobii.Research namespace. Use either the FindAllEyeTrackers function to get a list of available eye trackers or the GetEyeTracker function to get the sole eye tracker specified by the eye tracker URI. Do it in one of Unity's "initialization" event functions, such as Awake() or Start (convenient if you only have one eye tracker), or on demand (if the user is supposed to be able to select eye tracker).

Step 2: Connecting to an eye tracker

The objects returned from the functions FindAllEyeTrackers and GetEyeTracker are instances of the class IEyeTracker. Use these objects to interact with the eye trackers.

Step 3: Performing a calibration

Calibrate the eye tracker by using either a ScreenBasedCalibration or a HMDBasedCalibration object (depending on the type of eye tracker). The ScreenBasedCalibration / HMDBasedCalibration class requires an IEyeTracker object in the constructor. More information about calibrations is available in the section on Calibration.

Step 4: Subscribing to data

When you have the IEyeTracker object and want to subscribe to gaze data, subscribe to either the GazeDataReceived or the HMDGazeDataReceived event (depending on the type of eye tracker). The data is available as GazeDataEventArgs or HMDGazeDataEventArgs.
Note: The events in the SDK are called on a thread internal to the SDK. That thread can not safely set values that are to be read on Unity's main thread. Fix that by enqueuing the data, and dequeuing it on the main thread, e.g. via Update() in a MonoBehaviour. This pattern is used in the examples included in the unitypackage.

Step 5: Unsubscribing from data and cleaning up

In Unity it is important to unsubscribe from any Pro SDK data that you have subscribed to, otherwise Tobii Pro SDK cannot terminate properly, and you risk that your application/game will hang on exit. Unsubscribe to the Tobii Pro SDK events together with your other clean up code, for example in one of Unity's terminating event functions, OnDisable() or OnApplicationQuit().
To be on the safe side, you can also call the Terminate() method on EyeTrackingOperations to ensure that all Tobii Pro SDK resources gets cleaned. Only do this when closing down an application or object (i.e. in OnApplicationQuit(), OnDestroy() or similar).

Useful tips and hints

The following tips are used in the Tobii Pro SDK Prefabs, so disregard this section if you already are using the Prefabs.

• Examples

  You can use the code from the prefabs as examples on how to use Pro SDK in Unity. The [EyeTracker] prefabs provide inspiration for most steps in the guide above, except for Step 3. The [Calibration] prefabs describes Step 3.

• Units

  In Unity, the standard unit is meter, but all "real-world" values in Pro SDK API are in millimeters. the Tobii Pro SDK Prefabs convert the values automatically to Unity units (meters). But if you're not using the prefabs, you must convert all Pro SDK data values yourself from millimeters to meters when using them in Unity (and vice versa). When converting, keep in mind that the x-axis of Unity's coordinate system is inverted compared to Tobii's. See the Coordinate systems section for more information about coordinates and transforms.

• Eye tracker and camera origins in VR

  When you work with VR in Unity, the Camera's position is fixed to a point inside the VR headset (or "HMD"). Likewise, the VR eye tracker has a fixed point inside the HMD, which it considers to be the origin of its coordinate system (0,0,0). Unfortunately, for the Tobii Pro VR Integration, these two points do not always match.
  With the Steam VR Plugin in Unity, the origin of the HTC Vive headset is located 15 mm "behind" the eye tracker's origin. the Tobii Pro SDK Prefabs compensate this -15mm offset automatically, so use them as your guide when you resolve this issue.