How do the latest discoveries in software and hardware challenge the boundaries of image making? Three visual entrepreneurs offer their answers.
Oct 26, 2015

It is now widely accepted that photography is undergoing a revolutionary change brought on by new technologies. Unlike other crucial moments in the medium’s history, such as the appearance of color, this transition has led to a radical change in our behavior: the constant use of photography, new forms of recording our daily lives, reconfiguring our relationship with the image, a need for social representation, new forms of sharing, and faster execution and publication. Does this mass production of images, circulating nonstop both physically and, above all, virtually on the web, contribute to the development of a collective visual memory? This is without a doubt one of the major questions of our time, despite an increased tendency to see photography as more in the service of communication than knowledge.

To discuss these issues and to present their products, on October 21, 2015 ICP and Studio 55 invited three guest speakers responsible for innovative technologies that may prove disruptive: Denis Nazarov, CEO of Mine, a platform for decentralized image annotation and attribution; Matthew Zeiler, CEO of Clarifai, a website offering image-recognition technology; and Jean-Marc Alexia, Senior Director of Product Management at DxO, a company specializing in photography software and the maker of the DxO One, a new pocket-sized camera.

Panelists Jean-Marc Alexia of DxO, Denis Nazarov, CEO of Mine and Matthew Zeiler, CEO of Clarifai
L to R: Moderator Noah Robinson of Fast Company with panelists Jean-Marc Alexia of DxO, Denis Nazarov of Mine and Matthew Zeiler of Clarifai.

The conversation was moderated by Noah Robinson, Executive Editor of Fast Company, who kicked off the event with several observations. The first was about a series of computer-generated images uploaded to Facebook which, not conforming to the platform’s recommendations, were rejected for "errors" probably by an AI. Robinson then shed light on the innovations of the Joint Photographic Experts Group in Brussels, known as the JPEG Committee, and its goal to implant a DRM standard (Digital Rights Management) in every JPEG image so that users will have to pay to open and view them. He also discussed the recent initiatives of the photographer Rachel Hulin, who announced the publication of a photographic novel on Instagram and who plans to use the social network for storytelling. "We’re in a weird place where things are changing rapidly," said Robinson.

"We’re going to push the boundaries of what you know and challenge you with some of the technologies that are being created today." - Noah Robinson

Mine, Nazarov’s project, aims to identify every image of the Internet. More than a billion images are uploaded to the Internet every day, but there’s no reliable way to know who took them or what they represent. Mine aims to permanently link an image with its author, regardless of where it appears and in what form, from Instagram and Google Images to text messages. Nazarov believes that this is the best way to annotate media. "Unfortunately, content shared on the Internet is mostly dumb," he said. "It's just stuff posted on your feed. You can easily imagine seeing a cow chewing cud, a cop at a pool party, a celebrity wardrobe malfunction, the Mona Lisa, or a cool-looking chair. But there's nowhere to learn anything about these things. And that’s because the Internet is blind. It doesn't see images, it doesn't know what's in them, who made them, or even if two images are the same."

How Mine sees images.
How Mine sees images.

"Every time an image is shared, it gets broken—from its maker, its story and even itself. If you take the time to add a credit or an informative caption, the next person probably won’t. You can easily imagine this process regarding a professional photograph published along a moving article on The New York Times website. Sharing has actually decrease human knowledge," he continued.

As an illustration, Nazarov cited the example of this photograph of a Syrian child drowned off the Turkish coast. The image was widely shared on social networks last September, but the discourse around this sharing was fractured and if often lacked the context provided by the articles in which it originally appeared. "When a network is ignorant about what it’s sharing, ignorance becomes a network effect," said Nazarov. "Creators cannot derive value from the spread of their works, because it is impossible to discover their identity. Consumers are disenfranchised, because they cannot have rich interactions with the media or its author."

Nazarov says that the Internet is "stupid." Perhaps it's more accurate to say that it has no memory. It has no universal database of visual content, and no platform serves the crucial role of transmission played by a history book or an encyclopedia. At this rate, we can imagine photography risking becoming an ephemeral material unavailable for consulting to future generations.

What Mine is...

This is why Mine could prove to be important. According to Nazarov, Mine is able to recognize the unique digital fingerprint of any image online, the way Shazam is able to recognize songs. Information regarding the identity of the author is contained within the image and cannot be altered, while other information about the image can be modified by a community of members, in a system similar to Wikipedia. 

"It is a new information layer over the internet," he said. "Any image you see can be mined: attributed, annotated, commented on, or learned about. These contributions make up a history that lives with the image anywhere it exists. Mine is not a social network. It is a community united by curiosity, creativity, and the goal of building a universal knowledge base that is woven into today’s social networks and platforms. Other media platforms and social networks treat users like products and content like property."

"Because Mine is one of the first alternate applications of the bitcoin blockchain, its foundations are decentralized, transparent and predicated on user ownership. Mine is uniquely suited for the future of cultural commerce that is by and for creators. We are looking for people to join the Mine community and help annotate the visual history of the world. We are creators, image experts, collectors and curators." - Denis Nazarov

Clarifai, developed by Zeiler and his team, offer services that seem complementary to Mine. It aims to understand the visual content of a photograph, to extract the value of our images and the memories we capture, with the help of artificial intelligence and algorithms. Clarifai represents a kind of evolution in search engines of other catalogues, like Google Images. "On our homepage, you can drag and drop any image into the page; it is uploaded and becomes automatically understood," he explained. "The algorithms that we've developed only look at the pixels of the image; no metadata is attached to it. Then they predict tags and allow us to discover images similar to the original one. The tags are the top 10 most relevant ones the model thinks are related to the input image. For example, if you upload an image of a heart, you will get tags such as romance, love, Valentine's Day, etc. On the other hand, the 'similar images' are retrieved in a whole data set, out of millions of images."

Clarifai visual search demo.
Clarifai visual search demo.

Clarifai also uses algorithms that try to emulate the way the human brain functions. “We use a neural network,” Zeiler explained. “It’s basically a bunch of neurons that look at the pixels of the image, and try to extract information from it, whether it’s a sun, a clock, a balloon, a road, etc. It runs through multiple layers and gives you a prediction of the content.”

Unlike Mine, Clarifai also operates with video content. The site is also open to everyone and offers its services to developers and companies for a monthly fee and based on the number of operations. Current clients include Vimeo, Vodafone, Groupon, Kodak, and Buzzfeed. The site can be useful for publishing the content of still and moving images, advertising and cleaning up social media feeds. "On social media, people use hash tags but they’re not fully relevant to the objects," he said. "They’re funny words or information about an event that you’re at, for instance. We can understand the whole content and do a lot of use cases, from search to moderation."

Clarifai photo effects tagging demo.
Clarifai photo effects tagging demo.

In addition to these innovations in the recognition, protection and good use of photographs comes DxO. Founded 15 years ago in Boulogne Billancourt, France, the company specializes in the development of photography software, especially post-production, as in DxO OpticsPro and DxO FilmPack. DxO also serves as an expert in image quality, sensors and camera lenses. According to Alexia, its products can be found in over 300 million cameras.

"Image quality is our religion. Among our activities, we are experts in RAW files. We can be compared to the Dolby Digital sound technology, but for photography." - Jean-Marc Alexia

Alexia presented his company’s latest product, the DxO One, a compact camera, which, he said, surpasses the image quality of mobile phones and approaches that of a professional camera. With a 12mm lens, a maximum aperture of f/1.8, and a 20.2-megapixel sensor, it takes incredibly good pictures for a camera its size (6.9cm long, 108 grams) thanks to its Super RAW system, a new standard that combines four RAW images into a single one. What makes it stand out are its bokeh, connectivity with Apple's iPhone, and the ability to share instantly. "This is a kind of camera which is based on the same technology as the one used on smartphones, but put on steroids. So we boost the size of every component. It’s lighter and way smaller than an iPhone for instance."

DxO One camera

The ideas of each participant raise important questions. The first concerns the limits of artificial intelligence in the understanding of human emotions. Zeiler responded to the question by revealing that the Clarifai system can understand 10,000 different concepts, not only the difference between animate and inanimate objects, but also high-level descriptive things such as togetherness. "It understands two persons hugging or a person holding a dog," he said. "This high level of understanding of images is already happening with these systems and it will get better over time. I think there's going to be a lot of fusions between different types of data that will help that out, with reading stories for instance, to give the ability of storytelling to these models, so they can't just predict tags but actually talk about the image like we want to talk about it."

Other questions can be raised, about data protection on Mine, or how to preserve the rights of the original source and how to deal with appropriation of images. Nazarov responded that there would probably be moderators on Mine, as well as a reputation system for each member of the community, which would determine who can modify what information in a photograph. “Regarding appropriation, we could create tools where there is a platform and a form for all versions of an image to exist and that will present the records of that image, as long as the lineage and derivations of the work is preserved and someone that wants to see the whole history of this derived image would have the ability to.”

The DxO One camera's SuperRAW technology
The DxO One camera's SuperRAWTM technology.

More importantly, we can ask ourselves what each of these innovations means for photography as a medium, not only as an object of pleasure, but also as a carrier of visual history available to everyone, and the guardian of free speech useful to the community. 

When Zeiler spoke about Clarifai’s abilities to more efficiently download and sell “the best of photography,” and when Alexia spoke about the need to address the gap between smartphones and DSLRs, Nazarov mentioned the fact that photography today means many things, explaining that he has been questioning how we use our free time today, and that he has been working to help us use our free time in a positive way, “since technology will win regardless.” This was a rather philosophical way of raising more questions than answers, but which seems in keeping with the visual era in which we live, impatient, sometimes delicate, but surely with more opportunities than in previous ones.

Jonas Cuénin is a writer, editor, and photographer based in New York. He is the editor-in-chief of Camera Magazine and a regular contributor to the photography magazines The Eye of Photography and Photograph.