Publications

Highlights

(For a list of recent papers see below or for full list go to Google Scholar)

Camera Tampering Detection using Generative Reference Model and Deep Learned Features

An unauthorized alteration in the viewpoint of a surveillance cameras is called tampering. We propose to employ a generative model that learns the distribution of images from the surveillance camera under normal operating conditions, by training a generative adversarial network (GAN). The GAN is capable of sampling images from the probability density function, which are used as reference. We train a Siamese network that transforms the images into a feature space, so as to maximize the distance between the generated images and tampered images (while minimizing the distance between generated and normal images). The distance between the generated and the surveillance camera image is classified as either normal or tampered.

P. Mantini and S.K. Shah

Understanding How Video Quality Affects Object Detection Algorithms

Video quality is an important practical challenge that is often overlooked in the design of automated video surveillance systems. Commonly, visual intelligent systems are trained and tested on high quality image datasets, yet in practical video surveillance applications the video frames cannot be assumed to be of high quality due to video encoding, transmission and decoding. Recently, deep neural networks have obtained state-of-the-art performance on many machine vision tasks. In this paper we provide an evaluation of 4 state-of-the-art deep neural network models for object detection under various levels of video compression. We show that the existing detectors are susceptible to quality distortions stemming from compression artifacts during video acquisition. These results enable future work in developing object detectors that are more robust to video quality.

M. Aqqa, P. Mantini and S.K. Shah

SHaPE: A Novel Graph Theoretic Algorithm for Making Consensus-based Decisions in Person Re-identification Systems

Person re-identification is a challenge in video-based surveillance where the goal is to identify the same person in different camera views. In recent years, many algorithms have been proposed that approach this problem by designing suitable feature representations for images of persons or by training appropriate distance metrics that learn to distinguish between images of different persons. Aggregating the results from multiple algorithms for person re-identification is a relatively less-explored area of research. In this paper, we formulate an algorithm that maps the ranking process in a person re-identification algorithm to a problem in graph theory. We then extend this formulation to allow for the use of results from multiple algorithms to make a consensus-based decision for the person re-identification problem.

Arko Barman and Shishir K. Shah

IEEE International Conference on Computer Vision (ICCV), 1124-1133, (2017)

A signal detection theory approach for camera tamper detection

Camera tamper detection is the ability to detect faults and operational failures in video surveillance cameras by analyzing the video. Researchers have increasingly focused on such techniques attributing to the ubiquitous deployment of large scale surveillance systems. In this paper, a signal detection theory approach is proposed to quantitatively analyze the information being captured by the camera and to detect tampers. Signal activity is used as a feature to measure the amount of information in the image. The distribution of features representing the normal operation of a camera are modeled as a Gaussian mixture model (GMM). The GMM is trained using synthetic data. To reduce the effects of noise, a Kalman filter is used to model changes in signal activity in the video.

Pranav Mantini and Shishir K. Shah

IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1-6, (2017)

End-to-end 3D face reconstruction with deep neural networks

Monocular 3D facial shape reconstruction from a single 2D facial image has been an active research area due to its wide applications. Inspired by the success of deep neural networks (DNN), we propose a DNN-based approach for End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different from recent works that reconstruct and refine the 3D face in an iterative manner using both an RGB image and an initial 3D facial shape rendering, our DNN model is end-to-end, and thus the complicated 3D rendering process can be avoided. Moreover, we integrate in the DNN architecture two components, namely a multi-task loss function and a fusion convolutional neural network (CNN) to improve facial expression reconstruction.

Pengfei Dou, Shishir K. Shah, and Ioannis A. Kakadiaris

IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 1503-1512, (2017)

 

Recent Publications

Camera Tampering Detection using Generative Reference Model and Deep Learned Features
P. Mantini and S.K. Shah

Understanding How Video Quality Affects Object Detection Algorithms
M. Aqqa, P. Mantini and S.K. Shah

Monocular 3D facial shape reconstruction from a single 2D image with coupled-dictionary learning and sparse coding
Pengfei Dou, Yuhang Wu, Shishir K. Shah, and Ioannis A. Kakadiaris
Pattern Recognition, 81, 515-527, (2018)

GoDP: Globally Optimized Dual Pathway deep network architecture for facial landmark localization in-the-wild
Yuhang Wu, Shishir K. Shah, and Ioannis A. Kakadiaris
Image and Vision Computing, 73, 1-16, (2018)

Annotated face model-based alignment: a robust landmark-free pose estimation approach for 3D model registration
Yuhang Wu, Shishir K. Shah, and Ioannis A. Kakadiaris
Machine Vision and Applications, 29(3), 375-391, (2018)

Tackling the Optimization and Precision Weakness of Deep Cascaded Regression for Facial Key-Point Localization
Yuhang Wu, Shishir K. Shah, and Ioannis A. Kakadiaris
Deep Learning in Biometrics, CRC Press, 85, (2018)

SHaPE: A Novel Graph Theoretic Algorithm for Making Consensus-based Decisions in Person Re-identification Systems
Arko Barman and Shishir K. Shah
IEEE International Conference on Computer Vision (ICCV), 1124-1133, (2017)

Hierarchical multi-label classification using fully associative ensemble learning
Lingfeng Zhang, Shishir K. Shah, and Ioannis A. Kakadiaris
Pattern Recognition, 70, 89-103 (2017)

A signal detection theory approach for camera tamper detection
Pranav Mantini and Shishir K. Shah
IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1-6, (2017)

End-to-end 3D face reconstruction with deep neural networks
Pengfei Dou, Shishir K. Shah, and Ioannis A. Kakadiaris
IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 1503-1512, (2017)

Identification of vascular disruptor compounds by analysis in zebrafish embryos and mouse embryonic endothelial cells
Catherine W. McCollum, Javier Conde-Vancells, Charu Hans, Mercedes Vazquez-Chantada, Nicole Kleinstreuer, Tamara Tal, Thomas Knudsen, Shishir K. Shah, Fatima A. Merchant, Richard H Finnell, Jan-Åke Gustafsson, Robert Cabrera, Maria Bondesson
Reproductive Toxicology, 70, 60-69 (2017)

Distance aggregation based score fusion for improving person re-identification
Arko Barman and Shishir K. Shah
IEEE International Symposium on Technologies for Homeland Security (HST), 1-8, (2017)

Indexed Captioned Searchable Videos: A Learning Companion for STEM Coursework
Tayfun Tuna, Jaspal Subhlok, Lecia Barker, Shishir K. Shah, Olin Johnson, and Christopher Hovey
Journal of Science Education and Technology, 26(1), 82-99, (2017)

Human Activity Recognition using Deep Neural Network with Contextual Information
Li Wei and Shishir K. Shah
12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), 34-43, (2017)

3D-2D face recognition with pose and illumination normalization
Ioannis A. Kakadiaris, George Toderici, Georgios Evangelopoulos, Georgios Passalis, Dat Chu, Xi Zhao, Shishir K. Shah, Theoharis Theoharis
Computer Vision and Image Understanding, 154, 137-151, (2017)

A Semi-Automated Method for Object Segmentation in Infant’s Egocentric Videos to Study Object Perception
Qazaleh Mirsharif, Sidharth Sadani, Shishir K. Shah, Hanako Yoshida, Joseph Burling
International Conference on Computer Vision and Image Processing, 59-69, (2017)