Sony Files Patent Application for "Mystery Science Theater 3000"-Like Interactive TV Technology

Sony Computer Entertainment America has filed a patent application for an interactive TV technology that would superimpose viewer-controlled avatars on top of a TV show or movie--thus resulting in an experience which might be described as an interactive version of the cult show, "Mystery Science Theater 3000" (which is actually referenced in the patent application). "In an embodiment of the invention, users are enabled to interact with enhanced interactive productions of television shows or movies, based upon adding game overlays to such movies or television shows," the patent application explains. "In this way, the physics in the game or virtual world meet with the boundaries defined in the television show or movie set. For example, avatars displayed to a user, in response to user gestures in the real world, e.g. in response to manipulation of a game controller or other such expedient, may toss tomatoes that stick to the actor's face or bounce off the actor's face and roll along a table that appears in the movie or television show."

The patent application--which was unearthed by gaming blog, Siliconera--goes on to summarize the technology underlying the user experience it describes: "The movie or television show may be preprocessed to extract a time-variant metadata stream associated with regions in the moving pictures. An overlay having both active regions and added content, such as animation, is composited with the movie or television show. Such preprocessing can include both visual and/or aural components of the movie or television show. User interaction with the enhanced movie or television show via a game controller or the like may be mapped to the movie or television show via the overlay, and game or other active features of the overlay respond during presentation of the movie or television show. As a result, the audience can be brought into, and interact with, the performance depicted in the movie or television show. Such interaction may include, for example, individual user actions, such as throwing a tomato, as discussed above, or group and/or competitive actions, such as shooting a spider that is crawling on an actor's back."

The patent application provides various examples of interactive TV games that could be based on the technology, including--in addition to games in which viewers could compete for points by having their avatars throw tomatoes at or shoot spiders off the backs of characters in the TV show or movie they are watching--games in which viewers would throw a sponsoring brand's products (e.g. Coca-Cola cans) at the screen, have their avatars kick characters in the behind, or receive points for being "the first participant to react to the appearance of a character or object." It also explains how the tomato-tossing game that it envisions could be used to provide producers with scene-by-scene data on viewer reaction to a show or movie.

[itvt] hopes to cover this technology--which would presumably be offered in conjunction with over-the-top TV on the PlayStation 3 games console--in more depth shortly. Meanwhile, the text of the patent application can be found here.