Date of Award


Document Type

Campus Access Thesis

Degree Name

Master of Science (MS)


Computer Science

First Advisor

Wei Ding

Second Advisor

Dan Simovici

Third Advisor

Nurit Haspel


Despite their impact on computer vision and face recognition, the inner workings of convolutional neural networks (convnets) are considered uninterpretable. To counter this, we propose prediction gradients to understand how convnets encode concepts. Existing efforts to understand convnets focus on visualizing units and classes in pixel space. In contrast, prediction gradients measure how individual units in convnets contribute to prediction, and is thus more general than pixelization methods. Calculation is also extremely efficient. Experiments verify the ability of the prediction gradients to measure performance impact of units in a convnet. Using a standard face recognition data set, we also show concepts are distributed in convnets. Finally, we show how the prediction gradient can find the most distinguishing features of faces, what part of one image looks like a target class, and how to use that information to fool classifiers. These use cases demonstrate that prediction gradients can provide unique insight into how neural networks operate.


Free and open access to this Campus Access Thesis is made available to the UMass Boston community by ScholarWorks at UMass Boston. Those not on campus and those without a UMass Boston campus username and password may gain access to this thesis through resources like Proquest Dissertations & Theses Global or through Interlibrary Loan. If you have a UMass Boston campus username and password and would like to download this work from off-campus, click on the "Off-Campus UMass Boston Users" link above.