Hand Man: A Practical Guide
Hand Man is a hand-tracking software utility that uses your webcam to map hand position and finger articulation in real time. It outputs data in a format that other applications can consume, which makes it useful for gesture control, prototyping interaction designs, or building custom input devices without buying specialized hardware. The latest version runs on Windows 10 and later, and it supports both built-in cameras and most USB webcams. The program captures video from your camera, runs a neural network model to detect key points on your hand, and outputs coordinates for each joint plus grip state. You can configure which output protocol you want — OSC, HID keyboard/mouse events, or a raw TCP socket. Most people who end up using it don't need all of that; they just want the tracking to drive something else. One thing beginners get wrong is assuming the camera needs to be perfect lighting. It does not. Hand Man's model was trained on a wide variety of conditions, and in my testing it tracked reasonably well at a desk lamp level. Direct sunlight or complete darkness will break it, obviously, but most indoor environments work fine without.
I once spent two hours troubleshooting why my tracking data was jumping around unpredictably. Turns out my webcam driver was reporting inconsistent frame rates, and Hand Man's internal smoothing kept compensating incorrectly. The fix was switching to a different camera in the Windows camera settings and setting the feed to a fixed 30fps rather than letting it auto-negotiate. After that, jitter dropped to nearly nothing. This is worth checking early if your data looks noisy.
Installation and Setup
Download the latest release from the official GitHub repository at handman-tool on GitHub. The installer is straightforward — it places everything under Program Files and registers a service that launches on startup if you choose that option. I recommend skipping the startup registration if you only use it occasionally, because it adds a background process that some antivirus tools flag unnecessarily. The model files alone take up about 200MB, so make sure you have space. Once installed, open Hand Man and select your camera from the dropdown. The preview window should show detected keypoints overlaid on your hand. If you don't see keypoints, check that the camera is not being used by another application — Hand Man will not share the feed. Adjust the distance between your hand and the camera. The sweet spot is roughly 30 to 60 centimeters. Too close and fingers occlude each other. Too far and the model loses precision on individual joints. For output configuration, pick your protocol. OSC is the most flexible and works with most creative tools like TouchDesigner, Unity, and notational programs. HID mode maps hand data to keyboard and mouse events, which is useful if you need to drive legacy software that doesn't support OSC. TCP socket output gives you raw coordinate data that you can parse yourself.
Get the Full Details
Common Pitfalls and Workarounds
Hand Man struggles with certain hand orientations. When your palm faces directly toward the camera with fingers spread flat, the model has trouble distinguishing left from right because the symmetry removes depth cues. I found this out the hard way while building a gesture-controlled UI prototype. The workaround is to keep a slight angle on your wrist — even 15 degrees makes a measurable difference in tracking accuracy. It is not dramatic, but it matters. Another issue is rapid movement. The model processes each frame independently, so fast gestures can produce lag or missed detections. The default frame processing rate is around 15fps for the tracking model, which is enough for casual use but not ideal for anything requiring precision. You can improve responsiveness by reducing the tracking resolution in settings, which cuts processing time per frame. This trades off some accuracy for speed, and the balance point depends on your GPU. On a mid-range laptop GPU, I saw responsiveness improve from about 200ms latency to roughly 80ms with a modest drop in joint accuracy. Usually acceptable. The software does not handle multiple hands well. If two hands appear in frame simultaneously, the model assigns them IDs inconsistently and you will see data swapping between them. There is no built-in solution for this. If you need dual-hand tracking, you are better off looking at OpenPose or MediaPipe directly. Hand Man is designed for single-hand use, and trying to push it beyond that will just waste your time.
Integration Examples
Using Hand Man with OSC in TouchDesigner is straightforward. Set Hand Man to output on port 9000, then in TouchDesigner add an OSC In CHOP and connect it to whatever you are driving. The channel naming convention is consistent — hand.x, hand.y, hand.z for wrist position, then finger.0 through finger.4 for each digit, with sub-channels for each joint. It takes about five minutes to get a basic cursor-following setup working. For Unity projects, there is no official plugin, but the TCP output works well with a simple Cclient. I wrote a lightweight wrapper that reads the coordinate stream and applies it to a hand rig in about 30 lines of code. The wrapper handles reconnection and basic smoothing. If you need something more robust, the community has a few alternatives on GitHub, but they tend to be poorly maintained. Your own wrapper is usually worth the effort since you control the behavior.
Limits You Should Know About
Hand Man does not track force or grip pressure. It estimates grip closure based on finger joint angles, which is close enough for most use cases but not accurate if you need quantitative grip data. I learned this when someone asked me to use it for a rehabilitation assessment tool. The angle-to-pressure correlation is too variable between individuals. That project required a different sensor entirely. The software also has no built-in calibration for hand size. It normalizes coordinates to a relative space, which works fine for most interactions but breaks if you need absolute measurements. People with larger or smaller hands may find that their range of motion does not map cleanly to the output scale. You can adjust sensitivity in settings, but it is a coarse correction. If you are building something that requires precise scaling, account for this from the start. Battery-powered laptops running on power saver mode will see reduced tracking performance. The model is CPU-light but not free, and throttling cuts processing throughput. If you notice frame drops or increased latency, plug in or switch to a high-performance power plan. This is a small thing that people overlook.
