Skip to the content.


We study the design of learning architectures for behavioural planning in a dense traffic setting. Such architectures should deal with a varying number of nearby vehicles, be invariant to the ordering chosen to describe them, while staying accurate and compact. We observe that the two most popular representations in the literature do not fit these criteria, and perform badly on an complex negotiation task. We propose an attention-based architecture that satisfies all these properties and explicitly accounts for the existing interactions between the traffic participants. We show that this architecture leads to significant performance gains, and is able to capture interactions patterns that can be visualized and qualitatively interpreted.

A social attention architecture.


(Only supported on the github page)

Full episode

Attention and distance to vehicles

Sensitivity to vehicles states

Effect of road priorities

Supplementary videos

Reproduce the experiments Open In Colab

  1. Install the highway-env environment

pip install highway-env

  1. Install the rl-agents implementations, and clone the repository.

pip install --user git+

  1. Train the agents (repeat for several seeds)

cd <path/to/rl-agents>/scripts

python evaluate configs/IntersectionEnv/env.json \
                               configs/IntersectionEnv/agents/DQNAgent/baseline.json \
                               --train --episodes=4000 --name-from-config
python evaluate configs/IntersectionEnv/env_grid_dense.json \
                               configs/IntersectionEnv/agents/DQNAgent/grid_convnet.json \
                               --train --episodes=4000 --name-from-config
python evaluate configs/IntersectionEnv/env.json \
                               configs/IntersectionEnv/agents/DQNAgent/ego_attention_2h.json \
                               --train --episodes=4000 --name-from-config
  1. Visualize the results

tensorboard --logdir out/IntersectionEnv

or python run out/IntersectionEnv/DQNAgent/