Shared and separate semantic systems for text and figure: Converging evidence from eye-tracking and brain imaging

Both text and figure are two basic visual forms to express the meaning of information. However, the relationship between the semantic processing of text and of figure is still not clear among cognitive scientists. This study investigated whether semantic processes in the two forms of expressing the same meaning of information have shared and separate neural systems using eye-tracking and functional neuroimaging. In the eye-tracking study, the results suggest that the text and figure have no significant difference on the reading effect, which supports the common-code theory. In the fMRI study, using conjunction analysis, there were greater common regions as a shared semantic network involving the frontal lobe, parietal lobe, temporal lobe and limbic system, which suggests that the greater shared activiations as a common semantic system for the content of both text and figure; the direct comparisons of the two forms showed that the text was more significant activations of striate and prestriate cortex, whereas figure was more significant activations of the parahippocampal gyrus and extrastriate, which indicate that form-specific activations in visual cortex are involved in the initially semantic processing. Taken together, our results provide the converging evidence for that the text and figure mainly share a common semantic system, and also have the form-specific semantic system in initially processing.