September 2008, Japan
Robotic mirrors in Orchestra’s, Otomo YoshihideTakamine Tadasu, YCAM 2008.
In September 2008 I traveled from Melbourne, Australia to the Yamaguchi Centre for Art and Media (YCAM), Japan to take part in the InterLab – openFrameworks workshop. The workshop was presented by Zach Lieberman and Theo Watson, who founded and maintains the openFrameworks project, bringing together open source c/c++ libraries for the specific purpose of creating new media art projects. My participation in the workshop, facilitated by the ANAT Professional Development Travel Grant, has greatly strengthened my skills as an artist and programmer. Not only did the workshop provide me with an introduction to programming with openFrameworks, but it also exposed me to new approaches and networks in the media art field.
YCAM
The Yamaguchi Center for Arts and Media was established in November 2003 as a cultural-arts complex that accommodates a hall for stage performances and large installations (Studio A), an exhibition/project space (Studio B), a mini film theatre (Studio C), and the Yamaguchi City Library. A platform for sharing computer-and IT-based media technology, YCAM hosts among other events, a variety of theatre and dance performances, art exhibitions, film screenings, sound-related events, workshops and lectures. YCAM has a strong track record of collaboratively developing technologies and productions with international and Japanese new media, theatre, sound, video and dance artists, including the legendary Dumb Type.
I arrived a day early for the workshop so that I could check out ‘Ensembles’, a series of installations by Otomo Yoshihide celebrating YCAM’s fifth anniversary. Quartets (Studio B), Without Records (lobby/patios), Hyper WR Player (patio) and Filaments (City Library walkway) all explored ideas of collecting and collective creation through performance and sculptural elements.
Orchestras (Studio A) was by far the most impressive and complex installation in ‘Ensembles’, with over 2000 samples from 100+ contributors, more than 70 speakers and 200 objects suspended from the ceiling, 5 robotic mirrors lighting up the objects and an ‘underground’ wonderland of smashed records, guitars, wireless headphones and turntables. It was truly an amazing monument to the fantastic garbage of collected and collective culture.
Screen shot from Matthew Gingold’s registered video application developed with openFrameworks.
Having seen several YCAM performance co-productions before (including Ryoji Ikeda’s C4I and Takayuki Fujimoto’s True) it was great to see such an extensive installation co-production. It was particularly interesting in the context of my arts practice to see how all the works in ‘Ensembles’ meshed performance, installation, sculpture and algorithmic composition, questioning notions of who exactly is the performer and what exactly is being performed.
openFrameworks (oF)
oF was developed for “…folks using computers for creative, artistic expression, and who would like low level access to the data inside of media in order to manipulate, analyze or explore. That audience we felt was significantly underserved by the current crop of C++ libraries”. As such, oF wraps several other open source c and c++ libraries whilst maintaining access to the original source code. It is cross-platform compatible with OS X, Windows and Linux.
The project was initiated, and is maintained by Zach Lieberman and Theo Watson, both with extensive international media arts practices. oF has been used in several significant installation and performance contexts by Zach and Theo (including Laser Tag with the Graffiti Research Lab), as well as by a growing community of user and developers.
I found out about the project in an Open Source for Artists workshop given by Chris Csikszentmih‡lyi (MIT) at ISEA 2008. Over the last 12 months I have been steadily working towards using more open source tools as I believe they provide an excellent alternative to commercial products – both ethically and technically – and provide a platform for art-making that is fundamentally about community, learning and sharing. It was a privilege to ‘meet-the-makers’ and inspiring to see how hard Zach, Theo and the rest of the oF community work, and most importantly, share their knowledge so freely.
Interlab Workshop Day 1
The first day of the workshop started with Zach and Theo giving us a brief background to openFrameworks and the philosophy behind the project, before outlining their own arts practices. Then it was our turn to show a bit of our previous work and talk about what we hoped to get out of the workshop. With 17 participants (15 from Japan, 1 from Korea and myself) the breadth and depth of work shown was amazing (and if truth be known a little intimidating!). Many of the participants were from design companies, IAMAS (see below) or other media art schools, as well as some of the YCAM InterLab staff. Works ranged from installation and performance instruments through to physical-computing, web-design and industrial 3D imaging. After the presentations we split into two groups, so that Zach could give a brief tutorial on the basics of programming, while the rest of us set-up our working environments and started to think about what we were going to make over the next three days.
Day 2
In the morning all of the participants took it in turn to outline what they wanted to make during the workshop. I decided to port the work I had been doing in Processing with registered video. This is a process whereby the silhouetted outline of a performers’ body is used as a mask for projections directly back onto the same performer. The setup usually involves a camera and a projector closely aligned together, and some kind of infrared lighting (either onto the performers, or onto the background behind the performers) in order to get a more distinct silhouette of their outline. For some great examples of this in dance performance take a look at Klaus Obermaier’s work or Chunky Move’s Glow and Mortal Engine in collaboration with Frieder Weiss.
Based on some examples given to me by Theo, I began experimenting with grabbing images from the camera, distorting them (in order to get rid of discrepancies between the camera and the projector) and getting the outline of the image. It became apparent that I was going to have a difficult day. Being new to programming in c/c++, and quite unused to the strictness of the syntax and memory management, I struggled to get my applications to compile. I ended up staying up until 4am reading c++ tutorials on the difference between vectors and arrays; variables, pointers, and dereferences. Fun.
Day 3
Bleary eyed I arrived to a workshop presented by Prof. Etsuko Ueda (Nara Sangyo University), a world expert on openCV (an open source library for computer vision). I’m not sure I got the most out of this presentation (given that it was in Japanese), but strangely it was much easier to follow the code examples than you would think. I commiserated with a number of other participants, who also come from a more Java/scripting/patching oriented practice – we were all struggling with c/c++. With a lot of help from Theo and examples from his work with the Graffiti Research Lab, my application began to take shape. YCAM supplied me with a video projector, an infra-red light and a CCTV camera, so I could make some test footage to work with overnight. I stayed up late. Very late. But it was working •
Day 4 (Presentations)
The next morning everyone was looking pretty tired, yet focused. I tested my application with the camera and projector setup. Then fine-tuned features like full-screen mode and the video and particle systems I would later project. Before lunch we were taken on a tour of YCAM by Miki Fukuda and were given insights by Ito Takayuki on the production of Otomo Yoshihide’s installations. The tour reconfirmed the world-class nature of YCAM’s facilities and practice of not only curating, but also developing and educating the public about art and media.
In the afternoon we presented our workshop projects to the group. The projects were really amazing especially given the very short amount of time to develop them during the workshop. My favorites were Hiroyuki Satake’s facial recognition project (that swapped peoples, heads in real-time); Suguru Sasaki’s extremely beautiful contour finding, particle generating project; Seo Hyojung’s very funny rock-paper-scissors application in which you play the classic game against god (who always wins); and Tomoyuki Mukasa’s (failed but now ongoing) attempt at getting the Parallel Tracking And Motion library to work inside oF (an Augmented Reality project coming out of Oxford University that is amazing). A full list of projects can be found on the oF wiki and edited video of the entire workshop is available courtesy of the YCAM InterLab: Day 1, Day 2, Day 3, Day 4 and presentations: Session 1, Session 2 (I’m in here!), Session 3 and Session 4.
My presentation was successful. With Zach dancing around in the projection, everyone went ‘oooh’ when I switched from video to particle system imagery – it was a very nice moment. I was super happy to have learnt so much. The aesthetic and efficiency improvements made by porting my work from Processing to openFrameworks were significant. Essentially I went from being able to reproject video at around 10-15 frames a second, to 30fps for video and 60(!)fps with 1000 3D particles.
I also tweaked the method by a) using the outline ‘blob’ data to create a smoother mask, and b) improving efficiency by distorting the outline ‘blob’ data, rather than distorting the actual video input. These technical improvements to the method will allow me to better match projections to the movements of performers onstage.
All in all the openFrameworks workshop involved overcoming an extremely steep learning curve, but represents an important development in my creative and professional practice. For many years I have wanted to learn to program in a c/c++ environment – mainly because of the 10 to 100 times efficiency gains over Java based languages – but have failed to get very far due to lack of help and decent examples. openFrameworks, provides exactly that: an excellent community, forums, feedback, examples and more complex code wrapped in a simpler form (but still with the source all there if you want to go deeper).
Conclusion A number of new projects now in development (both solo and collaborative) for 2009 are directly related to the cultural and technical knowledge gained through this Travel Grant. Organizations directly benefiting from my knowledge of openFrameworks include Well Theatre, the Melbourne Fringe Festival and Experimenta Media Arts. Outcomes directly arising out of the workshop include an audio-visual performance with Kelly Ryall at Outpost and Electundra. For this performance I wrote a midi-controlled HD video mixing application in openFrameworks. The software uses 3D particle generators to create video masks and allows direct audio and midi control of the video and particles from external sources. All video used in these performances was generated during my travels in Japan.
Attending the YCAM/oF workshop and my trip to Japan in general, represents a significant professional development: technically and culturally. The multiple language barriers – c++ and Japanese – were significant challenges with significant rewards. The workshop has extended my international artistic networks, through establishing both professional and personal friendships with the developers of oF, individual artists and staff members from YCAM and IAMAS. Exposure to the Japanese culture of performance and installation media art has opened up new possibilities in creative thinking, whilst the knowledge gained about hardware and software development is now providing the means to move forward into new territory for my arts practice.
Other Professional Developments
In addition to attending the workshop I extended my stay in Japan in order to undertake a number of other cultural and professional development activities, including:
Exhibitions and Performances in Tokyo
Highlights included seeing Haino Keiji and Otomo Yoshihide playing together live at SuperDeluxe and seeing Trace Elements: Spirit and Memory in Japanese and Australian Photomedia. It was especially good to see, not only for the opportunity to experience Teiji Furuhashi’s LOVERS in the flesh, but because I was involved (as the Director of Seventh Gallery) with Lieko Shiga’s RAPT! exhibition in Australia and worked on both Alex Davies and Genevieve Grieves’ installations. It was also great revisiting the ICC and their excellent bookshop (even if all the DVD’s and catalogues I bought nearly broke the bank!).
Photography and Video
Although lugging my Panasonic HVX-202 around Japan caused some serious shoulder crunching, it also generated some beautiful video. Concentrating mostly on time-lapse and slow-mo I continued my obsession with filming the urban landscape and human flow: bullet train views at 300km/hr, Shinjuku rush hour, Kyoto back streets and the view from the Mori Museum lookout were all highlights. This footage formed the basis (along with 3D/HD video mixing software written in oF) for my recent performances with Kelly Ryall at Electundra and Loop.
Akihabara and Nipponbashi
Denki heaven in Tokyo and Osaka’s respective electronics markets. I exercised remarkable restraint in the face of much temptation (even if this required furiously scribbling the make and model numbers of precision instruments, high quality CCTV cameras, video transmitters & encoders, RGB LEDs and sundry other components). In particular I spent a great deal of time researching robot manufacturers and suppliers (including visiting the head office of Kondo and checking out Bioloid) – an area I hope to be able to incorporate into my art practice sometime soon.
IAMAS
Many of the participants in the workshop were students or staff from the Institute of Advanced Media Arts and Sciences and/or International Academy of Media Arts and Sciences, including Shigeru Kobayashi a senior lecturer from the Institute, who is a former Roland Electrical Engineer and developer of the Gainer physical computing interface and Funnel, a protocol for serial interface connectivity across multiple platforms (including Pure Data, Processing, Max/MSP, Flash, and after the workshop, openFrameworks). Although I only had one day to visit IAMAS it was a great opportunity to see a truly unique media art school, and meet with several of the staff and students. I gave a presentation of my work, and interviewed Kobayashi about the school, it’s aims and history.
Yokohama Triennial
Although the Triennial was on the whole a little disappointing (feeling a little bit more like a world expo of international art, than a critically curated exhibition), there were still several works that were good to experience first-hand, including Teshigawara Saburo and Sato Rihoko’s Fragment of Time (a corridor of broken glass, light and the sound of dis-embodied feet crunching across the debris), Ulla von Brandenburg’s ‘living picture’, La Maison/Dockingstation, Fischli & Weiss’s latest installment of A Rat and A Bear, and Kuswidananto a.k.a. Jompet’s Java’s Machine: Phantasmogoria.
All photo’s copyright © Matthew Gingold 2008
Tags: open source c/c++ libraries, openFrameworks, programming, sharing platforms
