Deep Neural Network Compression for Object Detection and Uncertainty Quantification

Abstract

Neural networks have been notorious for being computational expensive. Their demand for hardware resources prohibits their extensive use in embedded devices and puts restrictions on tasks like real time tracking. On top of that, neural networks are usually deterministic and provide no uncertainty which is crucial on safety decision tasks and physical sciences. In this work techniques were developed to reduce the computational cost of neural networks, such as Model Compression infused with a novel dynamical clustering and Knowledge Distillation while estimating the impact of such techniques on the uncertainty of the model by using Bayesian neural networks. A brief introduction is made on deep learning and the tools used. Furhtermore the ideas of Model Compression, Knowledge Distillation and Bayesian Deep Learning were extended analytically. All three approaches were tied together followed by the final discussion.