AAAI 2021 | PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception

type

status

date

slug

summary

Contribution

A joint physical-social simulation for procedurally generating abstract social event grounded in physical environments.

Using this engine to generate a first-of-its-kind abstract social events dataset.

Proposing two social perception tasks

recognizing goals and relationships of agents
predicting the future trajectories of agents

A benchmark including state-of-the-art methods and a Bayesian inverse planning-based approach.

💡

这篇笔记重点总结第 2 条，即这篇工作如何生成 abstract social events。

Prerequisite—Dec-POMDP

Decentralized partially observable Markov decision process, Dec-POMDP.

Definition：

where

is the set of agents.

is a set of states.

is the set of joint actions

is the transition probability function, i.e. .

is the set of joint observations.

is the observation probability function, i.e. .

is the immediate reward function.

is the horizon.

is the initial state distribution.

Belief

For simplicity, take POMDP as an example.

means in a trajectory , the agents believes how likely the world is being at state in the step , based on his last action and his new observation .

where can be computed through , and is the normalizing factor.

Policy

Definition:

A deterministic policy for agent is a mapping from observation histories to actions, .

Method

The planner maintains a set of “particles” to approximate the belief of each agent at each step, i.e. .

At each step, they first update particles by simulating one step in the physics engine assuming that other agents will maintain a constant motion.

They use a high-level planner to generate subgoals (immediate goals).

Subgoals are represented by predicates.

Let be the best subgoal for agent at step on its belief state in particle .

Define the value of each subgoal by

where is a heuristic-based estimation of cost to reach goal state based on belief state defined as the estimated distance that the agent need to travel before reaching the final goal state.

Select the most valuable subgoal .

Feed the subset of the particles that yield to the low-level planner, which will search for the best action to reach that subgoal.

Supplementary

Belief Representation

The state in a particle includes the environment layout, and physical properties of each entity—shape, size, center position, orientation of the body, linear and angular velocity, and attached entites.

Belief Update

Each particle is updated with the ground truth properties of observed entities.

Unobserved entity:

First randomly sampling possible initial positions and setting other properties (orientation and velocity) to 0.
To update a belief particle from to , they first apply the physics engine to simulate one step, where they assume constant motion for entities.
Then check the consistency based on observation. For entities that contradict the observation, they resample their positions and orientations. Repeat until there is no conflict.