flybody is an anatomically-detailed body model of the fruit fly Drosophila melanogaster for MuJoCo physics simulator and reinforcement learning applications.

The fly model was developed in a collaborative effort by Google DeepMind and HHMI Janelia Research Campus.

We envision our model as a platform for fruit fly biophysics simulations and for modeling neural control of sensorimotor behavior in an embodied context; see our accompanying publication.

The fruit fly body model lives in this directory. To visualize it, you can drag-and-drop fruitfly.xml or floor.xml to MuJoCo’s simulate viewer.

Interacting with the fly via Python is as simple as:

import numpy as np
import mediapy

from flybody.fly_envs import walk_imitation

# Create walking imitation environment.
env = walk_imitation()

# Run environment loop with random actions for a bit.
for _ in range(100):
   action = np.random.normal(size=59)  # 59 is the walking action dimension.
   timestep = env.step(action)

# Generate a pretty image.
pixels = env.physics.render(camera_id=1)
mediapy.show_image(pixels)

The quickest way to get started with flybody is to take a look at a tutorial notebook or .

Also, this notebook shows examples of the flight, walking, and vision-guided flight RL task environments.

To train the fly, try the distributed RL training script, which uses Ray to parallelize the DMPO agent training.

Follow these steps to install flybody:

1. Clone this repo and create a new conda environment:

   git clone https://github.com/TuragaLab/flybody.git
   cd flybody
   conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0
   conda activate flybody

   flybody can be installed in one of the three modes described next. Also, for installation in editable (developer) mode, use the commands as shown. For installation in regular, not editable, mode, drop the -e flag.

2. Core installation: minimal installation for experimenting with the
   fly model in MuJoCo or prototyping task environments. ML dependencies such as Tensorflow and Acme are not included and policy rollouts and training are not automatically supported.

3. ML extension (optional): same as core installation, plus ML dependencies (Tensorflow, Acme) to allow running
   policy networks, e.g. for inference or for training using third-party agents not included in this library.

4. Ray training extension (optional): same as core installation and ML extension, plus Ray to also enable
   distributed policy training in the fly task environments.

1. Create a new conda environment:

   conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0
   conda activate flybody

   Proceed with installation in one of the three modes (described above):

2. Core installation:

   pip install git+https://github.com/TuragaLab/flybody.git

3. ML extension (optional):

   pip install "flybody[tf] @ git+https://github.com/TuragaLab/flybody.git"

4. Ray training extension (optional):

   pip install "flybody[ray] @ git+https://github.com/TuragaLab/flybody.git"

1. You may need to set MuJoCo rendering environment varibles, e.g.:

   export MUJOCO_GL=egl
   export MUJOCO_EGL_DEVICE_ID=0

2. Also, for the ML and Ray extensions, LD_LIBRARY_PATH may require an update, e.g.:

   CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
   export