Expert talk in Computer Science and Engineering

Title of the Talk: On Representing Words
Speaker: Prof. Harish Karnick, Professor, Department of Computer Science and Engineering, IIT Kanpur
Date and Time: Monday, August 13, 2018; 1530 hours
Venue: POD 1E - 203


Mitchell et al. [1] showed substantial predictability for generating fMRI images for unseen concrete nouns that were based on a computational model that was constructed using 360 fMRI images per subject for 9 subjects where each subject was shown a stimulus containing a word and a drawing. There were 60 concrete nouns and exposures were repeated 6 times. There was substantial within subject and between subject similarity similarity in the fMRI images.

Their model used an intermediate set of words that functioned as semantic features for the set of concrete nouns and the generation was done as a linear combination of 25 such semantic features used as a basis set. The semantic features were generated using a large text corpus.

We wanted to ask the reverse question. Given the fMRI images how accurately can the stimulus word be predicted? While attempts to do this by using intermediate semantic attributes failed we got surprisingly good results by directly using voxel level activations to predict the stimulus.

The talk will first give some background regarding existing theories for how words/concepts are represented then discuss implications of Mitchell et al's results and end with our attempt to solve the reverse problem.

1. T Mitchell, et al., Predicting Human Brain Activity Associated with the Meanings of Nouns, Science, 320, 1191-1195, 2008.