I meant to write part two of the this story last Thursday already, but then as so often other more important things crossed the plans, delaying its arrival a bit until today. Part three with the final release of software is still on too, and planned to happen in the coming days.
In Part one I described the technical endeavours necessary to turn the Kindle into a software platform that could be tweaked to fit our needs of a device running in kiosk mode to be used as an information device in the exhibition Design Real at the Serpentine Gallery . Once the technical feasibility of this was ensured, many design questions had to be solved.
The materials about the 43 objects in the show to be displayed on the devices were collected by around 40 students from 4 different art schools, and simultaneously edited on design-real.com by multiple editors. The process of editing was ongoing, meaning there was no deadline at which moment all the materials could be taken from the website and turned into a form that suited the display on the Kindle. A solution had to be found that allowed this process to happen repeatedly over the duration of the show, so the devices in the show would be more or less up to date with the content online.
It is a well known fact that the Kindle is a device that emulates the printed book. Its eInk display suites this purpose very well, as its low refresh rate is not a big disadvantage when flipping pages and the very low energy consumption outweighs the slightly slow browsing (in our tests, the Kindle was still running after 24 hours of page flips every 5 seconds). But it is not designed to display long, scrolling webpages such as the pages for each object on design-real.com. The Kindle is a page based medium, the web is not.
The challenge was to find a simple, automated way to turn each of these 43 webpages into a sequence of nicely laid out single pages. We faced additional issues as we have opted for a two column layout on the site that was automatically produced by a script.
The first idea of using the browser support for printing failed in oh so many ways, despite the promising sounding print and page features in CSS. Unfortunately it seems that the browser support for standard CSS features like page-break-after, page-break-before, page-break-inside, orphans and widows is mediocre at best on all major browsers. With increasing support for web fonts, the idea of using browsers, HTML and CSS as a layout mechanism for print did seem very promising – to bad that topic appears to receive so little attention from all the browser developers.
After this route was given up, we decided to give Adobe InDesign's XML support a go. The chapter about XML in the book Real World Adobe InDesign CS4 gives a very good introduction into the topic and is available for viewing online. InDesign allows the creation of template files in which different content types can be marked with different tags through a special tool that is part of the InDesign UI. An XML file then needs to be produced that matches this description. Content can be duplicated and repeated, unused parts of the template can be removed.
What sounds rather simple and straight forward unfortunately was not. After a lot of trial and error we found out that InDesign is very particular about white space, line breaks and paragraph separators. For example what appears to be a normal line break in the initially exported XML file from InDesign is often in fact a so called paragraph separator (U2029), and if this is not strictly followed in the XML used to produce a layout, things can get pretty messed up. Another issue was that the images used on the website came from all kind of sources and had very different DPI settings. There was no way to control at what size the images to be imported through XML would appear inside their template containers in the document, although these had a defined bounding box. Unfortunately InDesign does not offer a set of options to control content fitting.
To make matters worse, images often were imported too big to actually fit the entire page let and therefore did not let the page continue to flow. Such images seemed to break the whole document, as nothing after these images would flow into pages, and the images could not even be selected with the mouse and resized, as they were not visible, they were part of so called overset content. A logical solution here appeared to be Adobe's ExtendScript environment that offers access to objects in the document through scripting. But as these images were not visible, they also did not have bounds that could be modified. Even finding them in the DOM was difficult, for the same reason. In the end, scaling them multiple times using horizontalScale / verticalScale until they eventually would appear in the layout did the trick, as this luckily did not require them to have any dimensions. When they finally did appear, InDesign needed some time to pick up the reflowing of the document until the next image eventually broke the flow and the script had to be executed repeatedly again.
I mention these details here as I would like to prevent anybody else from having to fiddle around with the same problems as we had to. Adobe's suite of tools is powerful and some of these open features very promising, but I have yet to come across one of them that just works out of the box without any major quest for ugly workarounds. The whole Creative Suite seems terribly buggy for such expensive software, and each time I am faced with such a problem, the need for good open source software that can match Adobe's offerings in the design sector becomes more apparent.
So in the end, all oddities of this approach could be worked around, and an extension to the web app was written that formatted each page in this required XML format. The resulting workflow was pretty smooth, except for the repeated execution of the bug-fix script mentioned above. The big advantage of this approach was that it allowed manual tweaks of each layout before the InDesign document was turned into a sequence of JPEG images for the Kindle. Images could be scaled down to allow the layout the better fit a page, or page breaks could be forced to move a title to a new page, etc. For 43 topics / objects and an average of about 10 pages per topic, this process took about 5 hours to complete. This means that a book of 430 pages was formatted in a hand assisted but largely automated process. If the problems in the underlying software would be fixed, the process could be optimised a lot.
Attached two examples of resulting PDFs of the pages about the Shipping Container and the Broom. The pages of these PDFs then were turned into sequences of JPEGs at the native resolution of the Kindle DX screen. This format appeared to be loaded much faster than for example PNGs. But more about this in the upcoming Part three.
27 January 2010: Kindle Kiosk, Part I
Tomorrow Apple will launch its new long anticipated Tablet project, a fact that has made many in the industry nervous, maybe most of all Amazon, who has already taken measures in the last weeks to better be prepared for whatever Apple will be unveiling. One of their decisions was to open up the platform and give developers the possibility to produce Active Content through a soon to be release Kindle Development Kit (KDK).
A few months ago when working together with Konstantin Grcic and Alex Rich on the show Design Real at the Serpentine Gallery, we were looking for exactly such a platform. The aim was to find a technical solution to offer visitors access to the extensive research that product design students from four different schools have conducted about the 43 objects of the show. This research was collected on the website www.design-real.com which was also designed and developed by us.
The problem was that none of the existing computer technologies seemed suitable for such a gallery context, not even anything from Apple. A laptop or a notebook seems to be a too lonely affair, the association to work and the invitation to type on the keyboard misplaced in such a context. The few tablets available appeared to bee too techie, their pivotable screens and detachable keyboards too flimsy for hundreds of visitors per day.
The idea of using the Kindle came up, and its use of the unconventional and somewhat odd eInk display technology seemed to fit the topic of the rest of the show well. The fact that only few in Europe were already confronted with this new technology that feels modern and nostalgic at the same time made us hope that such a piece of technology would be approachable and invite people for longer reading and exploring of the collected information.
But in October 2009 only the small one was available in Europe since a few weeks, and it was quite clear to us that if anything it could only be the Kindle DX. So without conducting any tests about the possibilities of putting our own content on the Kindle trough the PDF or MOBI format, we decided to buy one from the US and find out what the possibilities are.
So a device was bought through a friend from overseas, and a few days later the testing could start. Unfortunately the PDF viewer seemed very basic, and very quickly it became clear that the Kindle was not the kind of technology we imagined it to be. Just like Apple with the iPod, Amazon has designed this device as a storefront as much as a player / displayer of content. The Kindle is the place were content is bought through Amazon's Whispernet, and no content whatsoever is going to prevent users from quitting it and going back to the store where they can browse and buy other content. The Kindle was clearly not a device that Amazon intended to be used for display of information in a show, a fact that made us very sad as its technology and design seemed so suitable otherwise.
But then it came to our attention that the Kindle is based on Linux, and that Amazon has already released the source of its Kernel.
After more research, reports of Kindle hacks were found were people managed to 'break' the devices open and even install other flavours of Linux on it. Eventually a forum was found were a lot of the knowledge of these hacks was gathered and exchanged. The key component seemed to be a file called usbnetwork-0.7.tar.gz which allowed to reactivate a dormant feature on the Kindle DX to use its USB connection for a USB network connection to a computer rather than the mass storage mode that it usually shows up as when connected.
Once this connection was established, the device could be accessed through telnet and the various user interface files could be copied over. They were all jar archives containing compiled Java ME classes. Using the JD-GUI decompiler, these were analysed and various hooks into the event loop and display system were found. It seemed that there were enough starting points making it possible to write a software in Java that would run on top of the Amazon system, locking the whole interface and drawing its own, responding to key events and updating the screen accordingly. The idea of writing our own Mini OS for Design Real was born, and as mad as it sounded, there seemed no better way to achieve our goal.
Many trials and restarts later, a strategy was found that seemed to work well: The software had to be divided into two parts, one that installs itself pretending it is a so called Amazon booklet. When loaded by the underlying system which remains unmodified, this booklet installs its own event queue that intercepts all system events and runs a socket server, waiting for clients to connect to. As soon as a client connects, it sends the events only to the client and does not dispatch them further in the queue, preventing the Amazon system components from receiving them and therefore from updating the screen or responding to the keyboard.
The client software itself uses a JNI library that was found on the Kindle which offers direct access to the eInk frame-buffer and various options over how it is updated (with or without flash, rectangular regions or full screen, etc). The two components combined turned out to offer full control of the system, one blocking the system from receiving any events, the other responding to the events and drawing on top of whatever the Amazon system was already drawing before it was prevented from receiving events. Combining them into one for some reason was not possible. It seemed that booklets loaded by the Amazon system were unable to directly modify the frame-buffer, so the split and the client-server architecture was necessary.
This is part one of the story of how the Amazon Kindle DX became a open reading platform in our show. Part two
07 February 2007: SPDtalks!
The series of talks does not end:
Silvia Sfligiotti invited me to give a lecture about my graphic design related work at SPDtalks! in Scuola Politecnica di Design, Milano.
As allways: If you are interested and in the neighborhood, you might want to come along, as the event is free to the public:
Tuesday, 13.02.2007, 18:30
04 January 2007: Lecture in Basel
Together with Urs Lehni, I will be giving a lecture in the HGK Basel:
Wednesday, 31.01.2007, 17:00
F-Galerie
Hochschule für Gestaltung und Kunst
Vogelsangstrasse 15, 4058 Basel
Free Entry
16 November 2006: Things To Do
Kurt Eckert from HGK Zurich invited Urs Lehni and me to present our work to students in the context of a series of lectures he organizes about people from various design related practices presenting their ways of working.
If you are around and interested, I am sure they'll let you in:
Wednesday, 06.12.2006, 17:15 - 19:00
07 June 2006: Making do and getting by
Adobe has put online David Reinfurt's article about design, software and people who use computer based tools in unconventional ways or create their own.
It is an interesting text in which both Hektor and Scriptographer are mentioned.
You can choose between HTML or PDF.
30 May 2006: Educating Rita, Update
Almost exactly a year after the show «Rita + Hektor» opened at Tensta Konsthall in Stockholm, I finally found time to start putting things online.
The page about Rita is a start, next are all the Hektor pieces, as soon as I find a good way to present them.
The year in between was mostly spent on taking some time off and going to Tokyo to work at a Sony concept lab. While being in Tokyo, I also organized two Hektor exhebitions together with Alex Rich, one at «Trico Information Gallery» in Tokyo and the other at the «By Trico» shop in Fukuoka. Please stay tuned for more of these things to be put online soon.
By the way: Rita is looking for a new home. If you are interested in this installation, please do not hesitate to send a message.
