FLY SENSORIMOTOR FLIGHT LAB

Construction & limitations

This page explains what the browser experiment represents, what it leaves out, and how the current demonstration can grow toward a more biological model.

01 / CONSTRUCTION

What the replay does

The game renders a forward flight course. Each frame is reduced to a small 48 × 27 image that stands in for a fly-facing visual sensor. Independent horizontal and vertical categorical policies choose movement from retinal ring error, confidence, apparent size, motion, and ring phase. A separate binary policy maps retinal target alignment to a front-limb firing event. Ring progress and target hits train their respective policies without sharing rewards. Stationary mode repeats one course for learning tests; randomized mode varies the ring layout to test transfer.

The 3D body is a browser display based on simplified NeuroMechFly body meshes. The small brain panel now uses the low-poly JRC2018U adult Drosophila atlas surface, with grouped visual, motor, front-limb, and dopamine-like activity markers layered on top.

02 / LIMITS

What it does not claim

  • It is not a full Drosophila connectome simulation.
  • The four visual channels are aggregated signals, not individual ommatidia or reconstructed visual neurons.
  • The JRC2018U surface is an atlas average; the activity locations are still schematic pathway overlays, not measured firing locations.
  • The clipped policy-gradient learner, value baselines, elite memory, and dopamine-like reward display are machine-learning abstractions, not demonstrated biological dopamine plasticity.
  • The firing policy is a machine-learning abstraction of a front-limb decision, not a reconstructed leg premotor circuit. A small target-alignment movement is enabled only in course gaps and is intentionally capped to limit interference with ring steering.
  • Training episodes remain stochastic. The moving average should trend upward, but individual episode scores are not expected to increase monotonically.
  • The body animation has no aerodynamic, muscle, contact, or wingbeat physics.
  • The body and brain references combine assets from different fly datasets and should not be interpreted as one exact individual.

03 / NEXT LAYER

What would make it more authentic

A stronger experiment would map retinal samples onto selected visual neurons, propagate activity through a curated visual-to-descending pathway, and drive the game from recorded or simulated motor outputs. A browser-friendly version should use a carefully selected subset of neurons and cache only the geometry needed for display.

The full connectome is better suited to an offline or server-side simulation, with the website receiving compact activity traces and replaying them alongside the retinal image.

Sources and attribution

NeuroMechFly / FlyGym documents the biomechanical fly model, compound-eye inputs, hierarchical brain/VNC control, and mechanosensory interfaces.

MaleCNS provides connectome annotations, neuron skeletons, synapses, and connectivity data. Its full tables are far too large for this lightweight browser demonstration.

The browser body assets are included with attribution in assets/neuromechfly/NOTICE.

The low-poly brain surface is the JRC2018U template mesh from navis-flybrains, included with its GPL-3.0 license in assets/brain/NAVIS_FLYBRAINS_LICENSE.txt. The template is associated with the JRC2018 atlas documented by Virtual Fly Brain.