Human behavior alignment to improve the robustness of deep neural networks

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Deep neural networks (DNNs) are known to produce erroneous results under real-world noisy inputs, presenting a major bottleneck to their use in applications where lives, safety, or significant resources are at stake. It has been commonly observed that humans are highly resilient to the noisy inputs that are challenging for DNNs. However, very few efforts have translated this observation into techniques to improve DNN robustness. We hypothesize that statistically aligning DNNs to human behavior during training could improve robustness. Based on this insight, we propose BrainTrain, a framework to create more robust DNNs through human behavior alignment and demonstrate its utility in the context of object recognition. BrainTrain captures human behavior in the form of a confusion matrix constructed from human responses to object recognition challenges and uses a composite loss function to co-optimize accuracy and human behavior alignment during stochastic gradient descent (SGD) based training. We also propose Similarity Driven Label Smoothing (SDLS), a regularization method that scales BrainTrain to applications where it is difficult or expensive to collect human behavioral data. DNNs trained with BrainTrain showed up to 26% higher accuracy under a wide range of noisy inputs and 2.1 times lower calibration error with negligible increase in training time. We also demonstrate that SDLS leads to improvements in noise robustness in ResNets trained on the ImageNet-1 K image classification dataset. We show that BrainTrain is complementary to conventional techniques like noise-added training and can provide further improvements over and above these techniques.