Our visual experience is determined not only by extrinsic properties (such as colours and shapes), but also by a type of peer pressure: We constantly monitor (and follow) where others are looking, and many past studies have emphasised the importance of others' eyes as uniquely powerful stimuli. In this talk, I will argue that perception is socially sophisticated, and is driven not just by the brute presence of others’ eye and head movements, but rather by the perception of underlying mental states such as others’ attention and intentions. I will show how gaze effects are attenuated when the eyes do not signal any underlying pattern of attention and intentions (as in the phenomenon we have dubbed 'gaze deflection'), and how social impressions can influence one of the most foundational aspects of our visual experience: the temporal order in which events unfold. These studies have also paved the way for new explorations of how we perceive attention in others, e.g. how the visual system spontaneously prioritizes others’ degree of attention (vs. distraction). And in more recent work, we have been exploring the perception of others’ metacognition, e.g. how we attribute confidence to others when watching them act. This body of work suggests that ultimately, what matters for visual experience is not just perceiving and attending to the relevant physical features, but rather ‘perceiving perception’, and ‘attending to attention’.