Human Pose, Shape and Action
3D Pose from Images
2D Pose from Images
Beyond Motion Capture
Action and Behavior
Body Perception
Body Applications
Pose and Motion Priors
Clothing Models (2011-2015)
Reflectance Filtering
Learning on Manifolds
Markerless Animal Motion Capture
Multi-Camera Capture
2D Pose from Optical Flow
Body Perception
Neural Prosthetics and Decoding
Part-based Body Models
Intrinsic Depth
Lie Bodies
Layers, Time and Segmentation
Understanding Action Recognition (JHMDB)
Intrinsic Video
Intrinsic Images
Action Recognition with Tracking
Neural Control of Grasping
Flowing Puppets
Faces
Deformable Structures
Model-based Anthropometry
Modeling 3D Human Breathing
Optical flow in the LGN
FlowCap
Smooth Loops from Unconstrained Video
PCA Flow
Efficient and Scalable Inference
Motion Blur in Layers
Facade Segmentation
Smooth Metric Learning
Robust PCA
3D Recognition
Object Detection
Neural Control of Grasping

In this project we ask how objects are represented for grasping, how this representation evolves over time and how it is affected by context. This work addresses questions in basic neuroscience but has implications for machine vision and robotics as well as in brain-computer interfaces. Our work focuses on naturalistic grasping activities in a range of contexts from free-reach-to-grasp scenarios to instructed-delay tasks. We use motion capture to record behavior during complex grasping tasks with the aim of elucidating the underlying neural code enabling natural grasping.
Previous work has shown that ventral premotor cortex (PMv) is a key node in the parieto-frontal network specialized in transforming visual information representing object shape into hand postures best suited for grasping. These previous studies, however, use task conditions that do not separate two main factors: the visual features of the object and the motor strategy chosen to grasp it. It is not clear whether PMv operates in a primarily visual or motor regime, or if it transitions from one into the other at different times.
In order to address this question, we have developed an instructed delay task that includes objects that can be grasped using two different types of grips. All the objects share one common grip (power grip), but additionally can be grasped using an additional strategy unique to each object. We have recorded PMv ensemble activity in monkeys trained to perform different grips on cue using chronically implanted microelectrode arrays.
Grip selectivity has been described as a feature of PMv, but comparison of the same grips in tasks with differing temporal presentation of cues has not been pursued. We examined the differences in neural activity associated with movements cued after an instructed delay (sequential condition) and movements cued at the moment of motor execution (simultaneous condition). While the sequential condition allows for planning in advance of the movement, movement planning and execution must be performed at the same time in the simultaneous condition.
Our results suggest that grip selectivity in PMv may be influenced by the prior conditions that set up the task, including the ability to plan upcoming actions. Further, these results suggest that the neural population activity during a movement is affected by the timing of the cues to lead up to the action. Finally we find support for the hypothesis that objects may be characterized by their affordances - that is, their neural "representation" may "describe" how they may be grasped.
Members
Publications