PI: Jason Corso
Graduate Students: Albert Chen, Kevin Keane, David Johnson, Xin Li, Harsh Shah
Undergraduate Students Alexander Haynie, Brian Borncamp

Funding: NSF IIS 0845282

From representation to learning to inference, effective use of high-level semantic knowledge in computer vision remains a challenge in bridging the signal-symbol gap. This research investigates the role of semantics in visual inference through the generalized image understanding problem: to automatically detect, localize, segment, and recognize the core high-level elements and how they interact in an image, and provide a parsimonious semantic description of the image.

Specifically, this research examines a unified methodology that integrates low- (e.g., pixels and features), mid- (e.g. latent structure), and high-level (e.g., semantics) elements for visual inference. Adaptive graph hierarchies induced directly from the images provide the core mathematical representation. A statistical interpretation of affinities between neighboring pixels and regions in the image drives this induction. Latent elements and structure are captured with multilevel Markov networks. A probabilistic ontology represents the core knowledge and uncertainty of the inferred structure and guides the ultimate semantic interpretation of the image. At each level, rigorous methods from computer science and statistics are connected to and combined with formal semantic methods from philosophy.

A symbiotic education plan involving graduate and undergraduate mentoring and education, professional tutorial courses at the boundary of vision and ontology, and K-12 outreach is incorporated into the research plan. The research and education, disseminated broadly through both the applied science and semantics/philosophy literatures, lays a foundation on which to both utilize and automatically extract rich semantic information from images and other signal data for critical application areas such as internet vision, autonomous navigation, and ambient biometrics.

A key enabler for reasoning about the relations between elements in a scene is a robust object detection mechanism. During the Summer of Code 2010, I have designed an experential learning summer workshop for a group of masters students and undergraduates in which we are implementing and evaluating diverse state of the art methods in object detection on large real-world image datasets. Information is available a the SOC2010 page. Results and code will be posted here.

• Quad-Chart describing the overall CAREER project.

[1]	C. Xiong, S. McCloskey, and J. J. Corso. Latent domains for visual domain adaptation. In Proceedings of AAAI Conference on Artificial Intelligence, 2014. [ bib ]
[2]	S. Kumar, V. Dhiman, and J. J. Corso. Learning compositional sparse models of bimodal percepts. In Proceedings of AAAI Conference on Artificial Intelligence, 2014. [ bib | code | .pdf ]
[3]	W. Chen, C. Xiong, R. Xu, and J. J. Corso. Actionness ranking with lattice conditional ordinal random fields. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2014. [ bib ]
[4]	C. Xiong, D. M. Johnson, and J. J. Corso. Active clustering with model-based uncertainty reduction. Technical Report 1402.1783, arXiv, 2014. [ bib | .pdf ]
[5]	V. Dhiman, A. Kundu, F. Dellaert, and J. J. Corso. Modern MAP inference methods for accurate and faster occupancy grid mapping on higher order factor graphs. In Proceedings of International Conference on Robotics and Automation, 2014. [ bib | code | .pdf ]
[6]	C. Xiong, W. Chen, G. Chen, D. Johnson, and J. J. Corso. Adaptive quantization: An information-based approach to learning binary codes. In Proceedings of SIAM International Conference on Data Mining, 2014. [ bib | code | .pdf ]
[7]	C. Xu, R. F. Doell, S. J. Hanson, C. Hanson, and J. J Corso. A study of actor and action semantic retention in video supervoxel segmentation. International Journal of Semantic Computing, 2014. (In Press) Selected as a Best Paper from ICSC; an earlier version appeared as arXiv:1311.3318. [ bib | .pdf ]
[8]	W. Wu, A. Y. C. Chen, L. Zhao, and J. J. Corso. Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. International Journal of Computer Aided Radiology and Surgery, 9(2):241-253, 2014. [ bib | http ]
[9]	J. A. Delmerico, P. David, and J. J. Corso. Building facade detection, segmentation, and parameter estimation for mobile robot stereo vision. Image and Vision Computing, 31(11):841-852, 2013. [ bib | project | data | .pdf ]
[10]	C. Xu, S. Whitt, and J. J. Corso. Flattening supervoxel hierarchies by the uniform entropy slice. In Proceedings of the IEEE International Conference on Computer Vision, 2013. [ bib | poster | project | video | .pdf ]
[11]	C. Xu, R. F. Doell, S. J. Hanson, C. Hanson, and J. J Corso. Are actor and action semantics retained in video supervoxel segmentation? In Proceedings of IEEE International Conference on Semantic Computing, 2013. [ bib | .pdf ]
[12]	V. Dhiman, J. Ryde, and J. J. Corso. Mutual localization: Two camera relative 6-dof pose estimation from reciprocal fiducial observation. In Proceedings of International Conference on Intelligent Robots and Systems, 2013. [ bib | slides | code | .pdf ]
[13]	L. Zhao, W. Wu, and J. J. Corso. Semi-automatic brain tumor segmentation by constrained MRFs using structural trajectories. In Proceedings of Medical Image Computing and Computer Aided Intervention, 2013. [ bib | .pdf ]
[14]	C. Xiong, D. M. Johnson, and J. J. Corso. Uncertainty reduction for active image clustering via a hybrid global-local uncertainty model. In Proceedings of AAAI Conference on Artificial Intelligence (Late-Breaking Papers Track), 2013. [ bib | .pdf ]
[15]	D. M. Johnson, C. Xiong, J. Gao, and J. J. Corso. Comprehensive cross-hierarchy cluster agreement evaluation. In Proceedings of AAAI Conference on Artificial Intelligence (Late-Breaking Papers Track), 2013. [ bib | code | .pdf ]
[16]	P. Das, C. Xu, R. F. Doell, and J. J. Corso. A thousand frames in just a few words: Lingual description of videos through latent topics and sparse object stitching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2013. [ bib | poster | data | .pdf ]
[17]	J. A. Delmerico, D. Baran, P. David, J. Ryde, and J. J. Corso. Ascending stairway modeling from dense depth imagery for traversability analysis. In Proceedings of IEEE International Conference on Robotics and Automation, 2013. [ bib | project | .pdf ]
[18]	Y. Miao and J. J. Corso. Hamiltonian streamline guided feature extraction with application to face detection. Journal of Neurocomputing, 120:226-234, 2013. Early version appears as arXiv.org tech report 1108.3525v1. [ bib | http ]
[19]	J. J. Corso. Toward parts-based scene understanding with pixel-support parts-sparse pictorial structures. Pattern Recognition Letters: Special Issue on Scene Understanding and Behavior Analysis, 34(7):762-769, 2013. Early version appears as arXiv.org tech report 1108.4079v1. [ bib | .pdf ]
[20]	J. A. Delmerico, J. J. Corso, D. Baran, P. David, and J. Ryde. Ascending stairway modeling: A first step toward autonomous multi-floor exploration. In Proceedings of IEEE/RSJ Intelligent Robots and Systems (Video Proceedings), 2012. [ bib | project | video ]
[21]	J. Ryde and J. J. Corso. Fast voxel maps with counting bloom filters. In Proceedings of International Conference on Intelligent Robots and Systems, 2012. [ bib | code | .pdf ]
[22]	C. Xu, C. Xiong, and J. J. Corso. Streaming hierarchical video segmentation. In Proceedings of European Conference on Computer Vision, 2012. [ bib | code | project | .pdf ]
[23]	C. Xiong and J. J. Corso. Coaction discovery: Segmentation of common actions across multiple videos. In Proceedings of Multimedia Data Mining Workshop in Conjunction with the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (MDMKDD), 2012. [ bib | .pdf ]
[24]	M. A. Bustamante and J. J. Corso. Using probabilistic ontologies for video exploration. In Proceedings of the Eighteenth Americas Conference on Information Systems, 2012. [ bib ]
[25]	C. Xiong, D. Johnson, R. Xu, and J. J. Corso. Random forests for metric learning with implicit pairwise position dependence. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012. [ bib | slides | code | .pdf ]
[26]	R. Xu, P. Agarwal, S. Kumar, V. N. Krovi, and J. J. Corso. Combining skeletal pose with local motion for human activity recognition. In Proceedings of VII Conference on Articulated Motion and Deformable Objects, 2012. [ bib | slides | .pdf ]
[27]	G. Chen, C. Xiong, and J. J. Corso. Dictionary transfer for image denoising via domain adaptation. In Proceedings of IEEE International Conference on Image Processing, 2012. [ bib | .pdf ]
[28]	S. Sadanand and J. J. Corso. Action bank: A high-level representation of activity in video. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012. [ bib | code | project | .pdf ]
[29]	C. Xu and J. J. Corso. Evaluation of super-voxel methods for early video processing. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012. [ bib | code | project | .pdf ]
[30]	C. S. Lea and J. J. Corso. Efficient hierarchical markov random fields for object detection on a mobile robot. Technical Report 1111.1599v1, arXiv, November 2011. [ bib ]
[31]	Y. Miao and J. J. Corso. Hamiltonian streamline guided feature extraction with applications to face detection. Technical Report 1108.3525v1, arXiv, August 2011. [ bib ]
[32]	J. A. Delmerico, P. David, and J. J. Corso. Building facade detection, segmentation, and parameter estimation for mobile robot localization and guidance. In Proceedings of International Conference on Intelligent Robots and Systems, 2011. [ bib | project | data | .pdf ]
[33]	D. R. Schlegel, A. Y. C. Chen, C. Xiong, J. A. Delmerico, and J. J.  Corso. AirTouch: Interacting with computer systems at a distance. In Proceedings of IEEE Winter Vision Meetings: Workshop on Applications of Computer Vision (WACV), 2011. [ bib | .pdf ]
[34]	D. Gagneja, C. Xiong, and J. J. Corso. Towards a parts-based approach to sub-cortical brain structure parsing. In Proceedings of SPIE Conference on Medical Imaging, 2011. [ bib | .pdf ]
[35]	A. Y. C. Chen and J. J. Corso. Temporally consistent multi-class video-object segmentation with the video graph-shifts algorithm. In Proceedings of the 2011 IEEE Workshop on Motion and Video Computing, 2011. [ bib | code | project | .pdf ]
[36]	R. S. Alomari, J. J. Corso, and V. Chaudhary. Labeling of lumbar discs using both pixel- and object-level features with a two-level probabilistic model. IEEE Transactions on Medical Imaging, 30(1):1-10, 2011. [ bib | .pdf ]
[37]	A. Y. C. Chen and J. J. Corso. Propagating multi-class pixel labels throughout video frames. In Proceedings of Western New York Image Processing Workshop, 2010. [ bib | .pdf ]
[38]	J. A. Delmerico, J. J. Corso, and P. David. Boosting with stereo features for building facade detection on mobile platforms. In Proceedings of Western New York Image Processing Workshop, 2010. [ bib | .pdf ]
[39]	A. Y. C. Chen and J. J. Corso. On the effects of normalization in adaptive MRF hierarchies. In Proceedings of CompImage '10-Computational Modeling of Objects Presented in Images, 2010. [ bib | .pdf ]

This project is supported under NSF IIS 0845282: "CAREER: Generalized Image Understanding with Probabilistic Ontologies and Dynamic Adaptive Graph Hierarchies". Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.