Abstract

It is an enormous challenge for intelligent robots or vehicles to detect and avoid collisions at night because of poor lighting conditions. Thermal cameras capture night scenes with temperature maps, often showing different pseudo-colour modes to enhance the visual effects for the human eyes. Since the features of approaching objects could have been well enhanced in the pseudo-colour outputs of a thermal camera, it is likely that colour cues could help the Lobula Giant Motion Detector (LGMD) to pick up the collision cues effectively. However, there is no investigation published on this aspect and it is not clear whether LGMD-like neural networks can take pseudo-colour information as input for collision detection in extreme dim conditions. In this study, we investigate a few thermal pseudo-colour modes and propose to extract colour cues with a triple-channel LGMD-based neural network to directly process the pseudo-colour images. The proposed model consists of three sub-networks—each dealing with one specific opponent colour channel, i.e. black-white, red-green, or yellow-blue. A collision alarm is triggered if any channel’s output exceeds its threshold for a few successive frames. Our experiments demonstrate that the proposed bio-inspired collision detection system works well in quickly detecting colliding objects in direct collision course in extremely low lighting conditions. The proposed method showed its potential to be part of sensor systems for future robots or vehicles driving at night or in other extreme lighting conditions—to help avoiding fatal collisions.