How does traditional image making merge with coding and data visualization today? Three artists and engineers bring their ideas and experiences to the table.
Dec 21, 2015

Over the past decade, data visualization has played an increasingly significant role in the visual arts. This practice combine traditional disciplines with new technology, and its devotees include people such as Martin Wattenberg, Ben Fry, Jer Thorp or Mishka Henner, whose work can be seen at the current MoMa exhibition New Photography.

Artists who choose this alternative mode of expression, which can take many different forms—installation, video, still images, graphic design, sculpture—sometimes use computer code. The symbiotic result is a complex confrontation between the real and virtual worlds.

On December 2nd, three artists and designers were invited to the ICP: Ekene Ijeoma, a Nigerian-American artist and designer; Igal Nassima, a Turkish “Hacker in Residence” at Betaworks; and Ramsey Nasser, American software engineer, designer and educator.

A screenshot from Ijeoma's "Refugee Project", an interactive map of refugee migrations around the world every year since 1975.
A screenshot from Ijeoma's "Refugee Project", an an interactive map of refugee migrations around the world every year since 1975. 

Ekene Ijeoma’s projects explore the artistic and humanistic properties of data and algorithms. He has worked in multiple media and formats, ranging from commercials and music videos to websites and installations. His early experiments were an attempt to translate a photograph into data, to make drawings and surrealist videos out of computer code and apply them to the human figure, and to create a map visualizing banking transactions and the exchange of images across the world.

At the ICP seminar, Ijeoma presented several projects. The first, “Chrome Web,” which was created in partnership with Google. It was a yearlong exhibition held at the London Science Museum in 2013, featuring a series of interactive Google Chrome Experiments that brought the extraordinary workings of the Internet to life. Visitors to the museum were able to play with five unique installations, while at the same time online participants could visit www.chromeweblab.com and interact with the same installations. Together, in-museum and online visitors brought web technologies to life through five experiments. Universal Orchestra: an Internet-powered eight-piece orchestra creating harmonious music. Sketchbots: custom-built robots able to take photographs and then sketch them in sand. Data Tracer: a tool to trace where the world’s online information is physically stored. Teleporter: a series of web-enabled periscopes through which you can instantly access the world. Lab Tag Explorer: a visualization of all Web Lab visitors from around the world that groups and categorized participants in different ways.

The second one, “Refugee Project,” looked at ordinary people all over the fleeing death and persecution. The Refugee Project is an interactive map of refugee migrations around the world every year since 1975. UN data is complemented with original histories of the major refugee crises of the last four decades, situated in their individual contexts. “This is how can we use data visualization to make something that is both empathetic and insightful,” said Ijeoma. “As Joseph Stalin once said: ‘One death is a tragedy; a million deaths is a statistic.’”

Ijeoma’ third project, “Wage Islands,” expands New York City’s “tale of two cities” by revealing the geographies of access throughout the city based on housing costs and wages. The project is a three-dimensional map of the city where elevations are based on median monthly housing costs from $271 to $4001. In an installation at the Chicago Architecture Biennial in 2015, the map was submerged in water, depicting the peaks of New York as islands of access for minimum wage. When an 'included' button was pressed, the wages slowly increase, showing the area of the islands growing towards the base. When the button was released, the wages slowly decrease, showing the area of the islands shrinking towards the peaks. With this project, Ijeoma hopes to expand the relationships between housing and accessibility and wage and affordability in New York City. “Data needs photographs and photography needs data,” he said.

Ramsey Nasser's "Swordfight" project.
Ramsey Nasser's "Swordfight" project.

Although Ramsey Nasser explores the relationship between coding, electrical engineering and their use in photography, his works are slightly less serious than the rest. “Swordfight,” the first project he presented, is a fighting game where the two players face off wearing a harness strapped around their waists with a sword-like “joystick” extending from the genitals. The goal is to press a small button located next to the opponent’s joystick, pushing the environment of the game to its extreme in an appeal to baser instincts. “Games are my favorite medium,” said Nasser. “I like the social interaction in a game, the feelings and baser instincts that develop. This experience examines the interactions with viewers, so I documented their reactions with photographs. That’s the story of the project.”

555µHz” (2014) is Nasser’s second project, a Twitter bot tweeting every 24th frame of the 1986 film Top Gun with subtitles every thirty minutes. The result is a communal viewing of a silent, slowed down version of the classic action film. The project was run anonymously with no description or link back to Nasser’s Twitter account and site. 555µhz is written in Rubyx with the film processing done using ffmpeg. “For this project, which makes use of the ‘ultimate masculine movie,’” according to Nasser, he drew some 8,000 followers. “It was a different way to assume this piece of culture,” he said. “A celebration of the cinematography through details, a close attention similar to what one finds in paintings or a work of art shorter than the original. The Paramount lawyers contacted me and asked me to take down the project because the film was being ‘distributed illegally.’ The project can still be found on Twitter but the images no longer appear. The code still exists though, waking up every 30 minutes to decide not to initiate a new sequence before going back to sleep.”

Ramsey Nasser's “555µHz” (2014), a Twitter bot.
Ramsey Nasser's “555µHz” (2014), a Twitter bot.

For Igal Nassima, interactive maps offer us an excellent context in which to discuss databases. They are universally understood because they are representations of human data. My Block NYC, created by Nassima between 2009 and 2012, was made in reaction to the development of Google Maps. MyBlockNYC.com was a one-of-a-kind user-generated video map of New York City where 110,000 blocks of NYC were “clickable” and people could upload personal videos to them. Nassima took some census data and created an interactive map for a holistic perception of the city. This project was presented at the MoMa as well as at the Venice Architecture Biennale. Hyper-local videos uploaded by the public allowed visitors to both define and explore a moving city from the human perspective rather than a satellite’s. It was one of the first websites to utilize the collective power of user-generated video to provide a holistic, organized, and immersive portrait of local current events and culture of a city. During its time, MyBlockNYC was critically acclaimed and exhibited at the Museum of Modern Art and the Venice Architecture Biennale.

Nassima's Surveillance PET, an autonomous custom-built flying drone.
Nassima's Surveillance PET, an autonomous custom-built flying drone.

Nassima’s second project, Surveillance PET, examines our desire for surveillance and the uploading of personal data. Surveillance PET is an autonomous custom-built flying drone that followed a single person using GPS location tracking, while taking pictures at 3-second intervals. The pictures were then compiled into a map, overlaying Google maps. The installation was a two-channel video installation simultaneously showing the flying drone from the third person and the created maps from the drone perspective. The result was an uncanny reflection of surveillance using everyday tools such as cellphones, cameras, and maps.

BED, the last project presented by Nassima, is an interactive drawing machine that uses online news data to draw physical maps of conflicts. The custom software parses different news sources online and detects locations where conflicts such as terrorist attacks or wars are reported. A custom machine translates the digital location to a physical location on a map and draws it with red paint. The machine itself replicates the process of painters like Jackson Pollock. It is about visualizing contemporary news data and questioning the role of machines, news, and art and their relationships to modernity. The final piece is a cartographic artifact, revealing both the amount, and lack of data from different parts of the world. ”I’m really interested in finding some human reflection in data,” Nassima said. ”I’m not really concerned about aesthetic results but more in the process of both the underlying technology and the story itself. ”

Nassima's BED interactive drawing machine.
Nassima's BED interactive drawing machine.

The work of all three artists raises questions for those who are new to the approach. Fabrice Nadjari, the moderator of the conference at the ICP, first took time to reflect of the creative process. With a background in game design, Nasser’s process is very prototype-oriented. “The sooner I can make something, the sooner I can have a discussion with the piece,” he said. ”Once I have something tangible, the next steps become obvious. I don’t spend a lot of time scratching my head. ” On the other hand, Ijeoma’s work looked for voices to amplify data when engaging social and political projects, since his creative is first about learning and research. “There is a lot of iteration when you work with technology and code,” said Nassima. “The best projects are sometimes the ones you finish in 6 hours. At the same time, collaboration is useful. It’s important to work with several people. I usually bring together 3 to 5 people from different disciplines.”

It goes without saying that every project of this kind must be fully understood by its audience, leading one to reflect on the perception prior to the realization. According to Nassima, there are two types of audiences. First, the one that will look at both the physical project and the one online. “They have to be treated completely differently,” he says. “I think with projects that are kinetic and moving, we spend a lot of time making sure the people understand what’s going on. As engineers, we tend to get very nerdy about these creations but it might not communicate anything. People need to interact with these pieces within seconds. This interaction process is very important and has to be thought through seriously.”

Images from Nassima's Surveillance PET, an autonomous custom-built flying drone.
Images from Nassima's Surveillance PET, an autonomous custom-built flying drone.

Will knowing how to code one day be compulsory for photographers? Will the democratization of photography lead to the democratization of code? These are two of the most important questions the speakers sought to answer. For Ramsey Nasser, code isn’t magic. “It doesn’t enable you to do anything categorically new,” he says.

“Technology can amplify your actions, thoughts and behaviors. It needs ideas to be there initially, and those can then be subjected to computation. The core of creative projects is always completely human. I don’t think it will become compulsory. My favorite works of art are paintings, photographs, and movies. Do they need code? Maybe not.” - Ramsey Nasser

“I think there’s a problem with the relationship people have with the collaboration between coding technology and simple image making,” said Ekene Ijeoma in response. “Photographers and designers still don’t really understand in the same ways how to integrate different media or formats because they were trained in only one approach. It’s not necessary to learn how to code. Code is just a tool. But yes, image-makers should maybe learn how to code because you can be very creative with it. It’s been stigmatized as being pragmatic but that should change.”

“If you think about photography in an old school way, you use a lot of chemicals,” said Nassima. “A lot techniques that were discovered by playing with chemistry, and I believe digital photography could embrace the same potential with code.”

Code can be liberating. Every participant agreed that it helps one to think differently and find different ways to accomplish different things. There’s no single process, no right answer or way to do it. There are also different coding languages that encourage different ways of thinking. At the end of their discussion, Nasser, Ijeoma and Nassima offered advice to young photographers who want to join the world of data and coding. Ramsey Nasser discussed his favorite YouTube program, “You don’t want to learn how to code.” “It’s actually true, you only want to use code for some projects,” he said. “The best way is to teach yourself about code by looking at other projects and chasing them.” Ijeoma admitted that he hates reading programming books and that nobody has to know mathematics and numbers to learn how to code—a point that Igal Nassima disputed: “I’ll get more pragmatic. If you’re interested in coding, it’s great to pick up a language and just learn it. But if you’re interested in making a creative project, the best option is maybe to think about your ideas and collaborate with someone that is able to help you with the programming part.”

It was a fitting conclusion to the subject, bringing together the essence of their work, a confrontation of diverse disciplines and the ensuing qualifications. Coding, new and easy-to-use programs, and data visualization are accessible tools for a visual storyteller, whose role today is both to communicate both complex information and emotions to a large audience, only now they must perhaps find a more ingenious way to tell their stories.

Jonas Cuénin is a writer, editor, and photographer based in New York. He is the editor-in-chief of the magazine Camera and a regular contributor to the photography magazines The Eye of Photography and Photograph.

Top Image: Installation view of Ekene Ijeoma's "Wage Islands".