A student came by my office the other day complaining about a lab he was working on. For his junior lab, he was manually tracking the angular position of a damped torsional pendulum in each frame of a video to find the damping rate. I recently read Andy Rundquist’s post about tracking objects with Mathematica, and this was a perfect scenario to test it out. What follows is my experience with ImageFeatureTrack in Mathematica and a few things to watch out for. The full notebook I used can be found here.

*Note: I know Tracker can do this as well, but I believe each frame has to be manually marked, which can take time for long videos. If the video frame rate and quality are high enough, Mathematica does a good job. If the frame rate and quality are too low, it will not follow (fast) objects. There are a few options to tweak, but ImageFeatureTrack is largely a black box. *

The video I used is embedded below at half speed (download raw video). I shot this with my iPhone 5s in Slo-Mo mode (120 fps), which we found worked better than a lower frame rate. Bryan tried taking video later with a 20 fps camera, which didn’t work as well. I then downsized the video to 640×360 with Quicktime since 720p was taking a long time to analyze.

Loading the video into Mathematica is as simple as:

videopath = "/Users/matt/Dropbox/Work/damped_small.mov”; all = Import[videopath, {"Frames", All}]; // Timing

The import takes about 2 seconds for the full video. A few pieces of metadata about the video (framerate, dimensions, etc.) can be retrieved at the same time. At this point, there are two ways to approach the tracking.

## Automatically Tracking

You can simply do:

tracks = ImageFeatureTrack[all]; // Timing

This took about 25 seconds on my Xeon E3-1240, and Mathematica will detect and track “interesting” objects by itself, returning a list of lists of x,y points for each object it thinks is interesting. Here are the traces of the objects it found:

The axes are x and y pixel locations. Notice that it thinks several other objects might be of interest (the ruler in the first few frames, etc.). Now we have to figure out which one we want to follow. The notebook (included below) includes a section to help with this, but I prefer the second method.

## Manually Specifying a Point to Track

ImageFeatureTrack also allows you to manually track a specified point. This simplifies the analysis a bit and gives you more control over what to track. You can pick a point in the frame (x pixel and y pixel locations) and go from there. Below I specified I want to track whatever object appears to be at x = 396, y = 140 in the first frame.

tracks = ImageFeatureTrack[all, {{396, 140}}];

Here is the result:

Now we just have to choose an origin to calculate the angle, and we’re set. Below is the angle plotted as a function of time, with a fit line showing a damped oscillator equation fit:

Mathematica reports that ω = 3.17 rad/s and β = 0.48 s^{-1}. See the notebook for the other parameters.

## Thoughts

- This works well if the video is clear and high enough framerate. You may have to play with the Tolerance setting for ImageFeatureTrack if it doesn’t follow your object by itself.
- ImageFeatureTrack is somewhat of a black box — if it doesn’t work, play with some of the tolerance settings. I wasn’t able to get it to track the same damped pendulum when the video was at < 20 fps with some duplicated frames.
- This is somewhat CPU and memory intensive. On opening and running the notebook linked above, it takes about a minute to process the 8 second, 120 fps clip, and the kernel uses about 500 MB of memory. I imagine longer videos could become unwieldy quickly.