Active Perception

Active visual perception with realistic and complex imagery can be formulated as an end-to-end reinforcement learning problem, the solution to which benefits from additionally exploiting the auxiliary task of action-conditioned future prediction.

By exploiting human-uploaded web videos as weak supervision, we may train a system that learns what good videos look like, and tries to automatically direct a virtual camera through precaptured 360-degree videos to try to produce human-like videos.

An agent’s continuous visual observations include information about how the world responds to its actions. This can provide an effective source of self-supervision for learning visual representations.