A Novel Activation Function for Enhancing Deep Neural Network Stability and Performance

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Activation functions play an important role in the performance of deep learning models by enabling them to learn complex and nonlinear relationships. However, widely used activation functions, such as ReLU, ELU, Swish, and GELU, have certain limitations. Issues such as exploding and vanishing gradients and dead neurons can negatively affect stability and performance, especially in very deep architectures. In this study, a novel activation function called RoCLU (Robust Cauchy Linear Unit), inspired by statistical constraint relaxation theory, is proposed to overcome these limitations. By combining logarithmic and arctangent components, RoCLU promotes smoother activation transitions and improves numerical robustness by preserving finite‐valued computations during training, while facilitating stable optimization across deep architectures. The effectiveness of RoCLU was tested on a wide range of datasets with varying levels of complexity and state‐of‐the‐art deep learning architectures. The competitor models were chosen from both widely used proven models and state‐of‐the‐art models. The models were evaluated based on classification metrics, as well as object detection, segmentation, pose estimation, and oriented bounding boxes. RoCLU delivers competitive or higher accuracy compared to established and state‐of‐the‐art baselines while maintaining numerical robustness in networks of even 700 layers. Moreover, time‐based performance analyses showed that, despite a modest increase in computational time compared to some built‐in optimized functions, the accuracy and stability gains offered by RoCLU effectively balance this cost. Successful integration into YOLOv8‐based object detection tasks further supports its practical applicability. The findings suggest that RoCLU may serve as a robust alternative activation function within the scope of the evaluated tasks and architectures.