Ernesto Brau
Email: brauavil AT bc DOT edu
Office: St. Mary's Hall, Rm. S256, Boston College
CV: [pdf] [ps]

I am currently a postdoctoral researcher in the computer science department at Boston College, working under the supervision of Prof. Hao Jiang. My current research focuses around 3D human pose estimation from images and human action recognition and matching. Previously, I was a postdoctoral research associate in the School of Information: Science, Technology, and Arts at The University of Arizona, working on Bayesian modeling and inference of group activity from video under the supervision of Prof. Clayton Morison. I completed my Ph.D in computer science on Bayesian data association for temporal scene unerstanding, under the advisement of Prof. Kobus Barnard in the Interdisciplinary Visual Intelligence lab.
In general, my research interests are 3D scene understanding, tracking, human action recognition, and Bayesian modeling and inference.

Human pose estimation
Bayesian 3D human pose estimation
We develop a method to fit a 3D volumetric model of the human body to pixel-level body part labels obtained via a fully convolutional neural network trained on the Human80K dataset, which provides pixel-level annotations. We represent a human body as 3D cylinders, and use priors to constrain the size and orientation of the body parts, as well as to enforce non-intersection. We then use a graphics engine (e.g., OpenGL) to efficiently render the model onto the image plane -- using a camera model, which we also estimate -- resulting in pixel-level part labels.

Temporal action matching
Common action matching
Imagine having two videos known to contain similar content (e.g., sports) and wanting to extract clips in which both videos feature the same semantic action (e.g., a golf swing). In this research, we approach this problem as a linear integral optimization problem, and use the primal-dual method to solve it.

Activity recognition
Recursive sequences of group activities from video
Human activity recognition comprises a range of open challenges and is a very active research area, spanning topics from visual recognition of individual behavior, pairwise interactions among individuals participating in different roles in a joint activity, coordinated sequences of actions as expressions of planned activity, and multiple groups of individuals interacting across broad time scales. In this research, we address the last of these, presenting a framework for automatically constructing an interpretation of high-level human activity structure as observed in surveillance video, across multiple, interleaved instances of activities.

3D tracking
Bayesian 3D tracking and video understanding
We wish to obtain a 3D understanding of a moving scene from videos. This includes tracking everyone's position, as well as inferring their size and orientation. Since the camera is unknown, we must also infer it. The picture shows the result of running our algorithm on a well-known dataset.

Pollen tubes
Tracking multiple smooth trajectories
Simultaneously tracking many objects with overlapping trajectories is hard because you do not know which detections belong to which objects. We have developed a new approach to this problem and has applied it to several kinds of data. For example, the image to the left shows tubes that are growing out of pollen specs (not visible) towards ovules (out of the picture) in an in vitro plant fertilization experiment. The paths of the pollen tubes are green, and the tracks automatically found are superimposed in red.

Ernesto Brau, Colin Dawson, Alfredo Carrillo, David Sidi, and Clayton T. Morrison: "Bayesian Inference of Recursive Sequences of Group Activities from Tracks," To appear in AAAI-16, 2016.

Jinyan Guan, Kyle Simek, Ernesto Brau, Clayton T. Morrison, Emily Butler, and Kobus Barnard: "Moderated and drifting linear dynamical systems," Proc. International Conference on Machine Learning (ICML), 2015. [pdf]

Ernesto Brau, Jinyan Guan, Kyle Simek, Luca del Pero, Colin Dawson, and Kobus Barnard: "Bayesian 3D tracking from monocular video," Proc. International Conference on Computer Vision (ICCV), 2013. [pdf]

Luca Del Pero, Jinyan Guan, Ernesto Brau, Joseph Schlecht, Kobus Barnard: "Sampling bedrooms". IEEE CVPR 2011: 2009-2016   [pdf]

Ernesto Brau, Damayanthi Dunatunga, Kobus Barnard, Tatsuya Tsukamoto, Ravi Palanivelu, Philip Lee: "A generative statistical model for tracking multiple smooth trajectories". IEEE CVPR 2011: 1137-1144   [pdf]

Tracking smooth trajectories - C++ code and UNIX binaries used in this work. The data used in the paper is also available.

libergo - C++ library for MCMC sampling; developed with Kyle Simek.

© Ernesto Brau   |   Valid HTML & CSS