Accelerometer Localization dataset

With accelerometers becoming increasingly ubiquitous (through, e.g., smartphones and tablets) there is a growing interest in fusing accelerometer data with visual data. Combining these sensor types has strong potential as they provide complementary information. While accelerometers capture subtleties in the movement of the device, computer vision may, for example, put this information into spatial context and into relation with other entities. This dataset captures the problem of localizing an accelerometer in the visual field of a stationary camera in the context of food preparation, where accelerometers are attached to kitchen utensils.

SUMMARY

The dataset contains

RGB video data 640×480 pixels at 30 Hz
Depth video data 640×480 pixels at 30 Hz
3-axis accelerometer data at 50 Hz of devices attached to the handles of the knife and the spoon, and the rim of the bowl
Manually annotated (point and click) ground-truth point locations of all accelerometers in every frame of the video
Synchronization parameters for temporal alignment of video and accelerometer data

Download the full length summary video: MPEG4 .avi (448 MB)

LICENSE

This dataset is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. In addition, reference must be made to the following publication whenever research making use of this dataset is reported in any academic publication or research report:

Sebastian Stein and Stephen J. McKenna
Accelerometer Localization in the View of a Stationary Camera.
The 9th Conference on Computer and Robot Vision (CRV’12),
Toronto, Ontario, Canada, 2012.
.bib .pdf

RGB AND DEPTH VIDEO DATA

Frame index .txt (622 KB)
Frames .rar compressed (7.6 GB): part01, part02, part03, part04, part05, part06, part07, part08
Frames .tar archive .gz compressed (10.9 GB): part01, part02, part03, part04, part05, part06, part07, part08, part09, part10, part11
License (1 KB): LICENSE .txt

All frames are stored in separate image files. Filenames encode the timestamp of frames as captured by the camera, and the type of image (“-i” for RGB frames and “-d” for depth frames). RGB frames are stored as .ppm and depth frames as 16-bit .pgm files.

Loading depth in C/C++

Our code for loading depth files in C++/OpenCV can be found here.

Loading depth in MATLAB

The depth-data is stored in little-Endian byte-order. In order to use it in MATLAB, use
depthImage = swapbytes(imread('depthImage.pgm'));

ACCELEROMETER AND SYNCHRONIZATION DATA

The accelerometer data describes a single acceleration reading per row. Each row consists of (from left to right):

data info field (“ACCEL”)
time-stamp
device ID
sequence number
x-acceleration in g
y-acceleration in g
z-acceleration in g

The ground-truth data describes the locations of all accelerometers in a single frame per row. Row i consists of (from left to right):

x-location of the bowl-accelerometer in frame i
y-location of the bowl-accelerometer in frame i
x-location of the spoon-accelerometer in frame i
y-location of the spoon-accelerometer in frame i
x-location of the knife-accelerometer in frame i
y-location of the knife-accelerometer in frame i

ACKNOWLEDGEMENTS

This research is funded by an EPSRC studentship from the SiDE hub in Newcastle from December 2010 until May 2014. We would like to thank Patrick Olivier and his group for providing accelerometers and for useful discussions.

CONTACT:

For comments, suggestions or feedback, or if you experience any problems with this website please contact Stephen McKenna.