Connecting to the gaze tracker and receiving the gaze data
The Haytham_SimpleClient is a simple program that shows how to connect to the Haytham server through TCP/IP, and receive the eye tracking and scene image data.
Download: source code
Interaction with computer displays
Haytham_Monitor allows you to interact with your computer using gaze and gestures detected by the Haytham gaze tracker. Showing the gaze point in the screen, moving the cursor with gaze, performing different actions by commands provided by the Haytham gaze tracker, and exporting the gaze data are the main features of this client.When the Haytham gaze tracker is working in remote gaze tracking mode, the Haytham_Monitor should be executed in the same display that the system is calibrated on it. In the mobile gaze tracking mode, you can run this client in any display connected to the network. The Haytham gaze tracker detects and recognizes the display that you are looking at and automatically transfer the gaze data to that display. It is possible to run multiple instances of the Haytham_Monitor in different displays in the network (in mobile gaze tracking mode). The video below shows a quick demo of how to use the eye tracker for interaction with multiple displays in the environment.
You can also export the gaze data in the coordinates system of the display (in pixels) as a text file.
The Haytham_RecipeDemo is intended for a hands-free interaction with a recipe when cooking. The user interface consists of three areas: the recipe slides frame, a simple music player and the volume frame. The interface is fully operated by looking at these predefined regions while doing head gestures. This demo is intended to show the potential of using gaze trackers for fully mobile hands-free interaction with display. The video below shows a quick demo of the Haytham_RecipeDemo works.
The Haytham_Roomba is an application example of how to use gaze and only eye tracker for controlling a robot in the environment. The roomba vacuum cleaner robot is used because it actually has a useful function (cleaning) and also it is a good example of a mobile robot. With this client you can either control and drive the roomba with head gestures (measured by the Haytham gaze tracker) or with your gaze. In the gaze mode, roomba will go to the point that you are looking at on the floor. A visual marker is used for detecting the roomba in the scene image of the head-mounted eye tracker. There is also a simple application (written in C#) in the source code folder which can be used to understand more about controlling roomba via Bluetooth. You may need that if you are interested to extend the abilities of the Haytham_Roomba. The video below shows a quick demo of how this client works. There is also another demo about gaze-based controlling the roomba in the video made for the Haytham_SerialPortSwitch client.
The Haytham_SerialPortSwitch is intended to show how to use a mobile eye tracker for controlling real objects in the environment when the user is fully mobile. This client receives the eye tracking data and the eye-based commands provided by the Haytham gaze tracker, and uses them for controlling the objects connected to the client computer via a serial port. The first version of this client is designed for switching 10 different appliances in the home. Some visual markers are used for detecting objects in the user’s field of view by the eye tracker. The video bellow shows a quick demo of this client software.
The Haytham_SerialPortTV is a client software for the Haytham gaze tracker that can be used for controlling TV with gaze. TV is an example of a complex appliance that is more challenging to be control only by an eye tracker. This client is actually a combination of the Haytham_Monitor and the Haytham_SerialPort clients. The Haytham gaze tracker detects the TV screen like a computer display and sends the point that the user is looking (in the TV screen) to the Haytham_SerialPortTV. This client uses the eye-based commands and gaze for controlling the TV as a serial device. All the basic actions for controlling the TV can be done in this application e.g., looking at an area in the TV screen and adjusting the volume by continuous vertical head movement.
This version of the software is designed for LG TV panel of model LG55LE550, but you can easily modify the source code to work on you TV (with a serial connection). The video bellow shows a quick demo of this client.