The Future is in Seeing the Past