"This workshop focuses on developing computational and mathematical techniques for the analysis of large sets of cultural artifacts beyond text, and includes considerations of material and graphic design, architecture, fashion, interactive media, games, film, photography, music, painting, performance, and the kinesthetic dimensions of culture. The analysis of audio and visual data requires a different set of quantitative techniques than those devised for textual analysis. This challenge has become all the more acute, as every day individuals and institutions produce and publish hundreds of millions of digital cultural artifacts that are not text. The big data revolution is not only a text-based one, and these enormous new resources of non-text culture require equally revolutionary techniques for meaningful analysis.
The event will highlight novel methods for examining the multidimensional aspects of these cultural expressive forms. Aspects include structural configuration, dynamics in time and space, the changing social implications of artistic production and reception, and the cognitive multiplicity of perception and action, from genesis to memetic diffusion. The workshop aims to provide a point of reference for future research. By identifying and addressing pain-points, conceptual differences, and radical opportunities across the disciplines, our conversation has the potential to facilitate new scholarship in the arts, design, computation, information science, applied mathematics, and the physics of culture."
Alfred Bruckstein (Technion - Israel Institute of Technology)
Mauro Maggioni (Duke University, Mathematics and Computer Science)
Lev Manovich, Chair (The Graduate Center, CUNY, Computer Science)
Isabel Meirelles (OCAD University)
Vwani Roychowdhury (University of California, Los Angeles (UCLA), Professor, Electrical Engineering)
Maximilian Schich (University of Texas at Dallas)
This PDF presents concepts, research questions and examples of computational analysis and visualizations of cultural image collections from our research lab (softwarestudies.com) created between 2009 and 2015. Visualized datasets include 20,000 images from MoMA photo collection, 773 Vincent van Gogh paintings, and 2.3 million Instagram images from 13 cities worldwide.
(Note that the original presentation has a few videos that are not part of this PDF document.)
Subjects and Styles in Instagram Photography - abstract:
What are some of the types of Instagram photos today and how they relate to the 20th century photo culture? I analyze three common types of Instagram photos. We call these types "casual," "professional," and "designed." "Casual" photos are similar in function to personal photographers of the 20th century: they are created for friends; they privilege content of photos and ignore the aesthetics. Both “professional” and “designed” photo type are examples of what Alise Tifentale calls competitive photography. The difference is whom the authors compete with for likes and followers. The authors of professional photos aim for “good photo” aesthetics established in the second part of the 20th century, so they compete with other authors and lovers of such “classic” aesthetics including many commercial photographers. The authors of “designed” photos associate themselves with more “contemporary,” hip,” “cool” and “urban” lifestyle choices and corresponding aesthetics, so this is their peer group on Instagram.
The first part of Subjects and Styles in Instagram Photography chapter discusses the casual photo type. The second part discusses professional and designed photo types.
My text is an experiment to see how we can combine traditional “qualitative” approach of media theory and art history and newer quantitative analysis that uses “big cultural data” and computational methods. I draw on the analysis of 15 million images shared on Instagram in 16 global cities during 2012-2015 carried out in our lab (softwarestudies.com); results from other labs; my own informal observations from using Instagram for 3 years; and histories of photography, art and design.
Recently I was working on a series of articles about Instagram photography, and I wanted to find some digitized collection of personal 20th century photography to compare trends between then and now. I assumed that I could easily find at least 5000-1000 digitized images from some decades of the 20th century, and maybe even such samples for every decade. After searching for a while, I realized that nothing like this exists. The rare exhibitions and small collections of amateur photography all seem to reflect personal choices of particular collectors and curators. And they often only exhibit or collect certain amateur photos that look like art photography and modernist photography – in other words, the “usual.” The rest is discarded because it is too “usual.”
The same problem exists everywhere we look. What did hundreds of thousands of artists around the world painted in 1950? Or in 1850? What are the local differences around the world in personal photography in 1970? What were the contents and styles of all illustrations published in every magazine in 1930? Go ahead and ask a similar question about any form of visual culture, any medium, and any period until 2005. The answer will be the same: we have no idea. That is, we know (sometimes a lot) about the “leading” and “most important,” the “avant-garde” cases – which by definition are statistical outliers. (If they were forward looking avant-garde artists, this means that everybody else around them was different, so these artists do not represent general visual trends of the time.) Vincent van Gogh, Pablo Picasso, Berthe Morisot, Lyubov Popova, Diana Arbus, Ilya Kabakov, Ai Weiwei are all such outliers.
The disciplines that study visual culture are the “sciences of outliers.” But what about all other cultural artifacts?
But now situation is – or rather, could be – very different. We have social media. If we are interested in the histories of visual images, design, and photography - including the subjects, the styles, and the processes of creation and reception - social media is an unprecedented resource. In fact, a really broad, inclusive, and systematic study of visual culture becomes possible only after the wide adoption of social media and introduction of APIs by all major social networks (i.e. since 2005).
While modern histories of art, photography, visual design, and other visual media developed many useful concepts and theories, we can now see that they are mostly theoretical or descriptive undertakings that did not have sufficient data to study.
Imagine trying to develop discipline of biology based on only a few small habitats close to where you live, without being able to study the rest of the planet. And yet, this is what disciplines that study visual culture tried to do. They have developed their concepts and methods by studying particular small (and typically western) habitats. Moreover, within these selected habitats, they used very small and subjective samples. Their insights about these habitats based on the samples they used are often brilliant; but we don’t really know if any of this applies to any habitat as a whole and the rest of planet’s visual ecology.
It is equally easy to create social media datasets that cover content shared in one city or in dozens of cities around the world. For example, for our 2013 project Phototrails, we downloaded 2.3 million Instagram photos shared in 13 global cities using a singe laptop. For 2014 project On Broadway we also used a single Mac to download 10.5 million images shared on Instagram over 5 months in New York City.
Because in a number of countries the percentages of social network users with different genders, ethnicities, incomes, educations and other demographic characteristics are close, this means that we can study creations and cultural activities of all such groups equally well. (For the latest data social media U.S., see Social Media Usage: 2005-2015. As of 2015, 68% of women and 62% of men in the U.S. used social media. The rates were 65% for whites, 65% for Hispanics and 56% for African-Americans. The rates for urban suburban and rural use were %64, %68, and %58. For the latest data on numbers of users of top global social networks, see Leading social networks worldwide as of January 2016.)
When we look back from this perspective of today, it becomes clear that almost all of cultural content created before 2005 is either has not been preserved, or, if it has been preserved, it is hard to access for research. The museums, the books, the institutional websites, and archives contain only tiny islands of all created images. Something only remains because somebody (typically elites) thought were important to preserve, or because somebody donated their collection, or because an institution got the grant to digitize, or because they now have market value. The later is often the most important reason.
Thus, Walter Benjamin’s words “history is written by the victors” certainly perfectly apply to history of images. While works of a handful of “great” artists and photographers are studied, exhibited, and widely published, almost everything else has become invisible. To use a different methodology, we can say that only “top 40” are showcased, and “long tail” of visual culture has been cut off.
Even if you devote a few years to visit all collections around the world relevant to your topic – which requires an institutional affiliation and funding - there is no guarantee that they contain a balanced sample of images for this topic. And while some parts of these islands have been now digitized and available via institutional websites, such websites quickly age, the technology changes and after a while they no longer work.
This digitization process so far did not address the key problem of our “historical storage” – that the material related to a given topic, period, style, subject and so on exists across many separate physical archives, making it next to impossible to see everything. (Important large scale projects such as europeana.eu are working on this problem, but it is not clear if it will be ever be solved).
But most importantly, the distribution of preserved history (and consequently, the digitized materials) is very uneven geographically, with richer contrives having preserved (and digitized) more of their visual artifacts than more poor countries. And in terms of what has been preserved, many of our museum collections consists from the objects that came from the houses of the upper classes (typically, aristocracy) as opposed to lower classes. What world museums present today to millions of visitors as “history of art” are the commissions and possessions of the kings and tsars, not the creations of the people. (Similarly, global art world revolves around the taste of only a few thousand collectors who compete among themselves in purchasing artworks for six figure prices.)
Certainly, social media has its own significant limitations. It is not %100 inclusive.
The rates of use vary between geographic areas around the world, and between age groups. Not every kind of images ends up being shared in social networks. Some local visual communities and professional groups tare not using global social networks. Not every network has easy to use API. (However, since we can also scrape content of numerous websites, this gives us additional resources.)
But in comparison to what we preserved from before 2005, and also in terms of ease of access and quality of records, social media is God’s gift to researchers. It allows us for the first time to study visual culture created by many groups in society. In many countries, we can now study images created by %30-%60 of the whole population of a given country - as opposed to only tiny numbers of professional creators privileged by market, cultural elites, or particular cultural values that control what is saved and what is deleted.
People like to talk about how social networks “exclude” many from participating. But if we can study images shared by %30-%60 of people in a given country, this to me looks like a real progress in comparison to what academic disciplines and cultural institutions have studied and exhibited until now: typically only a few hundred artists, filmmakers, photographers per country (or even less).
Usually these were culture professionals who lived in majorly cities and this allowed them to become successful. But social media allows us to study what people create regardless of where they live. And it does not matter if a person is followed by 50,000 other users or by 5 - their content is equally accessible via APIs.
Only now we can start seeing both broad patterns manifested across billions of images, and also numerous smaller patterns that may manifest themselves only across a few hundred thousands. Only now we can start seeing how preferences in content, styles, and cultural sensibilities change gradually over short periods of times. Only now we can study numerous visual subcultures across the world. Only now we can study reliably the patterns of borrowing and circulation, copying and mutation, sameness and variability. Only now we can pose hypotheses and then verify or reject them, as opposed to promoting our intuitions as expert knowledge.
In short, only now art history, photo history, design history and all other image histories become truly possible.
The application form is for those requesting financial support to attend the workshop. We urge you to apply early. Applications received by Monday, February 15, 2016 will receive fullest consideration. Successful applicants will be notified as soon as funding decisions are made. If you do not need or want to apply for funding, you may simply register. IPAM will close registration if we reach capacity; for this reason, we encourage you to register early.
The application form is for those requesting financial support to attend the workshop. We urge you to apply early. Applications received by Monday, March 14, 2016 will receive fullest consideration. Questions and supporting documents should be sent to the email below. Successful applicants will be notified as soon as funding decisions are made. If you do not need or want to apply for funding, you may simply register. IPAM will close registration if we reach capacity; for this reason, we encourage you to register early.
The application form is for those requesting financial support to attend the workshop. We urge you to apply early. Applications received by Monday, March 28, 2016 will receive fullest consideration. Questions and supporting documents should be sent to the email below. Successful applicants will be notified as soon as funding decisions are made. If you do not need or want to apply for funding, you may simply register. IPAM will close registration if we reach capacity; for this reason, we encourage you to register early.
Data Drift is an exhibition curated by Lev MANOVICH, Rasa SMITE and Raitis SMITS at kim? Contemporary Art Centre in Riga, October 10 – November 22, 2015.
Artforum (February 2016) just published a very nice article about the exhibition written by Marta Burskirk, Professor of Art History and Criticism at Montserrat College of Art in Beverly, Massachusetts.
Since Artforum's content is not available online, and a single print issue costs $10 (in U.S.), we offer the scan of the article above. Addition of URLs, underlining and correction for On Broadway details are ours. Click on the images to see them in high resolution on Flickr and read the text.
Introduction to Data Drift written by Manovich:
DATA DRIFT exhibition showcases works by some of the most influential data designers of our time, as well as by artists who use data as their artistic medium. How can we use the data medium to represent our complex societies, going beyond "most popular," and "most liked"? How can we organize the data drifts that structure our lives to reveal meaning and beauty? How to use big data to "make strange," so we can see past and present as unfamiliar and new?
If painting was the art of the classical era, and photograph that of the modern era, data visualization is the medium of our own time. Rather than looking at the outside worldwide and picturing it in interesting ways like modernist artists (Instagram filters already do this well), data designers and artists are capturing and reflecting on the new data realities of our societies.
Full list of projects and artists shown in Data Drift:
COMPUTERS WATCHING MOVIES (Benjamin Grosser, 2013)
ON BROADWAY (Daniel Goddemeyer , Moritz Stefaner, Dominikus Baur, Lev Manovich, 2014)
CINEMETRICS (Frederic Brodbeck, 2011)
CULTUREGRAPHY (Kim Albrecht, Boris Müller, Marian Dörk, 2014)
THE RUN (Kristaps Epners, 2015)
THE EXCEPTIONAL AND THE EVERYDAY: 144 HOURS IN KYIV (Lev Manovich, Mehrdad Yazdani, Alise Tifentale, Jay Chow, 2014)
"#SELFIE–Imag(in)ing the Self in Digital Media," edited by Jens Ruchatz, Sabine Wirth, and Julia Eckel. Marburg, 2016 (forthcoming).
Many discussions of photography and other types of visual culture including user-generated content often rely on professional — amateur distinction. In this article we introduce a different pair of concepts: competitive — non-competitive. We believe that analyzing photography history and its present such as Instagram’s visual universe using these new concepts allows us to notice phenomena and patterns that traditional professional—amateur distinction hides.
The analysis of presentation of self in online digital photography is a case in point. We can now see that the selfie genre is complemented by an “anti-selfie” genre that presents the self in a different way. The two genres correspond to different understanding and uses of Instagram by non-competitive and competitive photographers.
Information about the illustration above:
Examples of Instagram “anti-selfies.” Authors, first row: @vita_century, @vita_century, @deroy_night. Second row: @lavimeer, @merciless_mart, @sex_on_water. Third row: @recklesstonight. Author’ age ranges from 14 to early 20s. (as explicitly indicated in their profile or can be estimated from photos in their galleries). The authors live in Russia or Ukraine. The photos were shared during Summer and Fall 2015.
Artists: Daniel Goddemeyer, Moritz Stefaner, Dominikus Baur, Lev Manovich. Other contributors: members of Software Studies Initiative (Mehrdad Yazdani, Jay Chow), Brynn Shepherd and Leah Meisterlin, and PhD students at The Graduate Center, City University of New York (Agustin Indaco, Michelle Morales, Emanuel Moss, Alise Tifentale).
The project uses software tools developed by Software Studies Initiative with the support from The Andrew W. Mellon Foundation and National Endowment for Humanities (NEH).