Using grammars for scene interpretation

A method that employs grammars to direct the inference process of a vision system that does interpretation of dynamic scenes is described. The system uses a set of qualitative image descriptors to drive the interpretation. The result is a 'natural language' description of scene activities. In addition the inference engine generates a set of predictions that can be used to control the interpretation strategy so as to make the processing of new images more efficient. The system has been implemented in an expert system shell to demonstrate the viability of the approach. Results on real images are reported.