BehaveAI is a user-friendly tool that identifies and classifies animals and other objects in video footage from their movement as well as their static appearance. The framework converts motion information into false colours that allow both the human annotator and convolutional neural network (CNN) to easily identify patterns of movement. The framework integrates both motion streams and static streams in a similar fashion to the mammalian visual system, separating the tasks of detection and classification.
The framework also supports hierarchical models (e.g. detect something from it's movement, then work out what exactly it is from conventional static appearance, or vice-versa); and semi-supervised annotation, allowing the annotator to rapidly correct errors made by initial models, making for a more efficient and effective training process.
- Identifies objects and their behaviour based on motion and/or static appearance
- Fast user-friendly annotation - in under an hour you can create a powerful tracking model
- Identifies small (2 px!), fast moving, and motion-blurred targets
- Can track any type(s) of animal/object
- Tracks multiple individuals, classifying the behaviour of each independently
- Semi-supervised annotation allows you to learn where the models are weak and focus on improving performance
- Tools for inspecting and editing the annotation library
- Live (edge) support - record videos, train, then run live video processing using the user interface
- Built around the versatile YOLO (You Only Look Once) architecture
- Computationally efficient - runs fine on low-end devices without GPUs
- Intuitive user interface with installers for Windows and Linux (including Raspberry Pi) and full user-interface. Also works on MacOS but currently no installer.
- Free & open source (GNU Afferro General Public License)
See the project wiki here for detailed user guide and installation instructions.
If you use BehaveAI please cite:
- Troscianko, Jolyon, Thomas A. O’Shea-Wheller, James A. M. Galloway, and Kevin J. Gaston. (2026) 'BehaveAI Enables Rapid Detection and Classification of Objects and Behavior from Motion’. PLOS Biology 24, no. 2 (2026): e3003632. https://doi.org/10.1371/journal.pbio.3003632.
See the project Wiki for detailed instructions.