AI Looks at Pictures, Tells Stories

AI Looks at PicturesScientists are developing a system that looks at a series of pictures and tells a story about them. The intention is to do more than to say, for example, “A man is getting into a car. The car drives away;” what is planned is that the story will draw conclusions, say who is doing what and WHY (one of the more challenging parts) and – even more difficult – how the participants might be feeling at the time.

If what was happening here was that the computer was looking at pictures and using its own awareness, understanding and empathy, that might truly be called artificial intelligence. (It would also be quite frightening). But that is not what is happening. Instead, the computer is being fed with huge numbers of pictures together with equally huge numbers of short text descriptions by volunteers of: what is happening in the pictures; why it may be happening; and how the protagonists may feel about it. It is from that memory bank that, faced with new pictures, the system creates its “story.”

The computer scientists involved in this research are proud of the work they are doing, so it is left to others to express reservations. The first of these is the amount of power it gives those deciding what information will be stored in the computer – what stories it will be told before it creates its own.

Norman Fairclough, writing in The Discourse Reader (Adam Jaworski and Nicholas Coupland, editors, London: Routledge 1999) says, “It is increasingly through texts… that social control and social domination are exercised (and indeed negotiated and resisted).” To put it plainly, text is political, and the struggle to control what is published and therefore available to read is never-ending. An individual’s personal beliefs and ideology are irrelevant here: any move towards automated text creation is, potentially, a political weapon placed in the hands of those in power. Nor is the quality of the “stories” created likely to reach even the lowest level of entertainment.