Generating predictions during action observation is essential for efficient navigation through our social environment. With age, the sensitivity in action prediction declines. In younger adults, the action observation network (AON), consisting of premotor, parietal and occipitotemporal cortices, has been implicated in transforming executed and observed actions into a common code. Much less is known about age-related changes in the neural representation of observed actions. Using fMRI, the present study measured brain activity in younger and older adults during the prediction of temporarily occluded actions (figure skating elements and simple movement exercises). All participants were highly familiar with the movement exercises whereas only some participants were experienced figure skaters. With respect to the AON, the results confirm that this network was preferentially engaged for the more familiar movement exercises. Compared to younger adults, older adults recruited visual regions to perform the task and, additionally, the hippocampus and caudate when the observed actions were familiar to them. Thus, instead of effectively exploiting the sensorimotor matching properties of the AON, older adults seemed to rely predominantly on the visual dynamics of the observed actions to perform the task. Our data further suggest that the caudate played an important role during the prediction of the less familiar figure skating elements in better-performing groups. Together, these findings show that action prediction engages a distributed network in the brain, which is modulated by the content of the observed actions and the age and experience of the observer.