In the same sense that films aren’t plays and records aren’t music performances, multimedia books aren’t really books as we understand them. Depending on how integrated all of their media components are, they are a new art form, different from what has come before. I see them looking a little bit like the web with the advantage of being self contained, so as object of art they can be made more powerful. The question I have is how well they convey or in what ways they are good at conveying narrative, a meaningful story, in comparison to other methods of communicating a story, ‘interactive’ or not. There are going to be formal limitations, as in any art.
If we want to make a multimedia fiction, we honestly don’t have much to guide us. An interesting portrayal in fiction of a ‘Mmook’ or ‘ibook’ or whatever was in ‘The Diamond Age’ by Neal Stephenson. It was almost part of the story’s neo-victorianism though, kind of a ‘magic book’ or might as well have been magic. I don’t remember it being a narrative, but an instruction manual, much like a very real multimedia book, Microsoft’s Encarta encyclopedia. There were a lot of multimedia books that came out on CD-ROM in the 90′s but I don’t remember any fiction. Books enriched with sound and video were great for manuals but what aspects, if any, are good for stories?
Ursula le Guin, in a trend with many writers of her generation, had a book/tape combo. I like the idea of having music to listen to tailored to what you’re reading and they can inform each other without necessarily being integrated. I can do that part in my head, just like my imagination is interacting with the words in any book in order to make the story. If you look at how that works and how it doesn’t, you can begin to figure out how the forms might be more tightly bound with technology and made more effective.
If you’re going to integrate music or art/video into the text it can’t really be something that begins automatically for the end-user. Puting sounds that start when you turn the page etc., might be OK for small children (like a pop up book) but I think it would be obnoxious for adult books, as it is on web pages. You could have a media player at the bottom of the page, though, with music you could turn on and off. I don’t see why, instead of having illustrations on a separate page, you couldn’t have them on a pop up link. Why not have an illustration that you click on and it begins to move? It depends on the type of book.
Not all Multimedia Books would be the same, just like there are lots of different kinds of illustrated books now – obviously a pop up book for little kids uses illustrations differently than Anna Karenina or an illustrated Alice in Wonderland. So, you’d have different amounts of multimedia and different ways it’s embedded in the text (or that the text is embedded in the other media.) Adults in most instances probably desire more control and less of that kind of illustration in general. At the same time, if the media-illustrations outweigh the words then it’s more of a picture-book or pop up book. Or a manual. Different books for different people and purposes.
Multimedia books have a lot of potential, if anyone ever starts making them. The way to make them certainly exists, but the ‘e-stores’ still aren’t set up for them. You can’t sell them as a book, or a video, or a song. I think the only real option is to sell them as an ‘app’. I have a feeling they cost money to make. You won’t see anyone creating them in prison which leaves out a lot of potential writers. I think multimedia books may be largely corporation funded.
In terms of what I can see the big companies doing, you’ll have the big trilogy of movies, then some books with things people could click on that let them hear/see bits of the movies or some stuff they could only get there, then the kid versions that are mostly movies and pictures, and the games with the even more added cut scenes, etc., all tied in together.
I’m not sure it leaves much room for personal expression in the way that so called ‘legacy’ art forms are, forms that are centered around individual rather than group expression. People try to make films into personal art forms, to a certain extent it’s a fools errand because you always need other people.. at least to date. And I think we might have a similar situation evolve with multimedia books.