A biomimetic learning acceleration strategy on reinforcement learning for obstacle-aware end-to-end path planning of USV

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Reinforcement learning (RL) can be leveraged to implement end-to-end path planning with obstacle avoidance, owing to its strong ability to learn from interactions with the environment without manual intervention. However, training RL models can be highly time-consuming, as they must capture the complex dynamics of the unmanned surface vehicle (USV) operating on water. Hence, this paper proposes a biomimetic learning acceleration (BLA) strategy that can scale up the testing success rate on the initial-stage training for the obstacle-aware end-to-end path planning in USV. The strategy consists of three modules: the learning-process segmentation, the reward reinforcement, and the exploration-efficiency enhancement. The learning-process segmentation module extracts the sub-region, sub-target-point, etc., to enforce reward proximality. In addition, it can reduce the low-efficiency exploration in the vast region away from the optimal path. The reward reinforcement module reinforces the reward of the redundant failure actions and potential successful actions through setting the sub-risk-region and prior feasible region. This module aims to boost the transmission speed of reward values between successful or failed actions in RL model, thereby enhancing training efficiency. The exploration-efficiency enhancement module establishes a high-dimensional matrix storing potential failure actions based on each failure trial and sub-risk-region yielded by the preceding module. Subsequently, the redundant exploration of actions that may lead to the collision can be avoided. Moreover, explored paths that are approaching to the target point can be recorded and replayed to elevate the probability of reaching the target point. The effectiveness of the proposed strategy is verified through simulation. The proposed algorithm ameliorates the limitation of low model testing accuracy during the initial training stage of Implicit Quantile Network (IQN). Specifically, under 120 s training, the success rate increases from 60.0% to 93.3%. The biomimetic learning acceleration algorithm code is publicly available at https://github.com/bjmyX/Improved-IQN-for-USV/tree/main .