Unblicked: What It Actually Does and When It Falls Apart
Unblicked is a computer vision toolkit built around detecting and tracking eye gaze and blink state in real time. It was designed primarily for gaze estimation pipelines and accessibility research, where you need to know whether someone is looking at a specific region of interest and whether their eyes are open or closed. The core of it runs on a pre-trained model that processes a cropped face region and outputs blink classification plus gaze coordinates. It's not magic. It's a stack of media pipe face mesh combined with a lightweight classifier on top. I'll assume you already have Python 3.9 or higher and a basic understanding of pip environments. Create a virtual environment first because Unblicked's dependency chain conflicts with a lot of common packages if you don't isolate it. Run pip install unblicked after activating your environment. The package itself pulls in mediapipe, numpy, opencv-python, and a few other standard vision libraries. If you're on a headless server without a GPU, it still works but inference will be slow because the default model runs on CPU at roughly 8 to 12 frames per second on a modern desktop CPU. The GitHub repo is the main distribution point. Clone it and run setup.py install from the source if the pip version isn't keeping up with the latest commits. That happened a couple of times when I was running tests. The PyPI release can lag behind the master branch by a few weeks.
One important thing most people miss: Unblicked expects a specific face landmark input format from mediapipe. If you feed it raw camera frames without running the face mesh detection step first, it silently fails or throws a shape mismatch error that isn't very descriptive. You need to pass the landmark coordinates, not the pixel image, into the gaze estimation function. I spent an afternoon debugging that exact issue before realizing the API documentation doesn't emphasize this clearly enough.
How the Blink Detection Pipeline Works
The blink detection in Unblicked relies on the Eye Aspect Ratio, commonly called EAR. It computes a ratio based on the vertical to horizontal distances between eye landmarks. When the ratio drops below a configured threshold, it registers a blink. The default threshold is around 0.25 but you can adjust it depending on your camera resolution and subject distance. Gaze estimation works similarly but uses the pupil position relative to the eye corners and the iris boundary. The model outputs a direction vector in 3D space. You then project that onto a 2D screen plane if you need screen-based gaze tracking. The projection step introduces its own error, which compounds when the subject moves their head.
Practical Implementation Example
Here's how I set it up for a typical session-based experiment. Initialize the video capture, run mediapipe face mesh on each frame to get landmarks, extract the left and right eye regions, compute EAR for blink detection, and feed the landmarks into the Unblicked gaze estimator. The whole loop runs at about 15 FPS on my machine with a decent CPU. If you downsample the frame to 480p before feeding it into mediapipe, you gain roughly 5 to 7 additional frames per second without noticeable accuracy loss for most use cases. For blink counting, the tricky part is debouncing. A single real blink can register as two or three detections because EAR dips below threshold and bounces back during the closure and reopening phases. Unblicked includes a simple frame-based filter, but I found it insufficient for subjects who have a slow blink pattern. My workaround was adding a minimum inter-blink interval check of 200 milliseconds in my own post-processing layer. Any blink detection within that window gets suppressed. It's not part of the default config and you have to implement it yourself.
Common Pitfalls and Where Unblicked Struggles
The biggest limitation is face angle. Once the head rotates more than about 30 degrees off the frontal plane, gaze estimation accuracy degrades significantly. The model was trained primarily on relatively frontal faces. Side profiles produce unreliable coordinates. I tested this on a dataset of naturalistic viewing behavior and the error rate jumped from roughly 2 degrees to over 8 degrees past that 30 degree mark. Lighting matters too. Low light causes the eye landmarks to shift because the iris boundary becomes harder to detect. The mediapipe face mesh model, which Unblicked depends on, is decent with lighting variations but not immune. I've seen gaze coordinates drift by several degrees in dim conditions. Adding a fill light or infrared camera helps if you're doing serious research work. Another issue is glasses. Standard polarized or tinted sunglasses break pupil detection entirely. Even clear prescription glasses with strong anti-reflective coating can confuse the iris boundary detection. Unblicked doesn't have a built-in glasses handling mode. I worked around this by falling back to eye corner based gaze estimation instead of pupil tracking when the confidence score dropped below a threshold. It's less accurate but more stable in those conditions.
For subjects with certain eye conditions like strabismus or very large iris-to-sclera ratios, the default model produces systematically biased results. I encountered this with a participant in an accessibility study and had to recalibrate per individual. The calibration routine takes about 30 seconds but it's necessary if you need sub-degree accuracy.
When to Use Something Else
If your use case requires head-tolerant gaze tracking or you're working with diverse populations including people with visual impairments, consider looking into alternatives like deep Gaze or custom-trained models using a dataset like MPII or Gaze360. Unblicked is fine for controlled lab settings with cooperative participants. It's not a general purpose solution for every scenario. The tool also doesn't handle multi-face setups well. If you have more than one person in frame, you need to manage face ID across frames yourself. There's no built-in tracking association. This matters more than the documentation suggests if you're doing group studies.
Unblicked for Real World Research Work
Despite the limitations, it's one of the more straightforward open source options for blink and gaze detection if you understand its boundaries. The codebase is readable, the API is consistent, and the documentation covers the happy path well enough. The problems arise in the edges, which is where any vision library reveals its actual quality. I'd recommend starting with the sample scripts, validating on your own hardware and lighting conditions before building anything production grade. Benchmark your own setup. Run a quick calibration test with five to ten participants and measure error rates before committing to this stack for a larger study. The time you spend there usually pays off because fixing detection issues mid-experiment is worse than almost anything else.
