SOFTWARE FOR DIGITAL HUMANITIES

Global News group reviewing Obama montages
Software Studies Initiative researchers exploring a video collection using the tools developed in the lab.


NEW: Guide to using ImagePlot in Polish by Radosław Bomba


We have developed a number software tools for working with big image and video collections, including preparing image data, automatically analyzing it, and using visualization for the exploration of the collections. All tools are free and provided as open source.

To see these tools in action, visit the projects page. To read about theory and methodologies behind them, consult our publications.



USING OUR SOFTWARE TOOLS: TUTORIALS AND GUIDES

GUIDE TO VISUALIZING VIDEO AND IMAGE SEQUENCES | How to prepare images and video collections for visualization; use of ImageJ built-in commands and our custom plug-ins.

ImagePlot documentation (English)

ImagePlot tutorial (Polish)

ImagePlot video tutorials


Below we list some of out tools by category, with links for download:


IMAGE ANALYSIS AND VISUALIZATION WORKFLOWS

workflow 1: First, use one of the tools listed in "digital image processing" section to process your image (or video) collection. ImageMeasure is a imageJ plug-in which extracts 6 basic features; QTIP is a stand-alone app which extracts 49 features. (If you want to use our advanced system which can extract 400 features, contact us.) Next, use ImagePlot (listed under media visualization) to explore patterns in your collection using the results of the processsing in step 1.

workflow 2: ImageMontage and ImageSlice (in "media visualization" section) can generate different kinds of visualizations, and they can work with a collection metadata directly (without processing in step 1).

HIPerSpace_Rothko_screenshot_2009


TOOLS FOR DIGITAL IMAGE PROCESSING

Our core method involves analyzing image or video collections using digital image processing, and then use the results of the analysis to create high resolution visualizations which show all images sorted in different ways. This allows us to efficiently "look inside" large visual collections, identifying patterns and seeing the whole "landscape" of the data. (In parallel with "exploratory data analysis", we can call this method "exploratory image analysis.")

Our tools can extract from a few to a few hundred visual features from each image.


ImageMeasure and ImageShapes | These two ImageJ macros we developed are distributed as part of ImagePlot package. ImageMeasure analyzes brightness, saturation and hue of images (median and standard deviation); ImageShape counts numbers of shapes in every image. Together, these two macros extract 7 features.

QTIP (QTImageProcessing) | very fast batch image processing application which uses OpenCV. The application processes all images in a directory, extracting 49 features. On our iMac (Fall 2011) with 16 GB of RAM, the application processed 192 Flickr images per second.

FeatureExtractor | a system for batch image processing which uses Python and Matlab's Image Processing Toolbox. The system extracts 399 features. It also generates PCA of all features, and separate PCA of features for composition, color, texture, and orientation. (Currently we are using this system in our lab, but not distributing the code.)


TOOLS FOR MEDIA VISUALIZATION

ImagePlot | create high resolutions visualizations which show all images in a collection in a form of a scatter plot. Works with collections of any size. You can customize dozens of graphical options. The application can be also used to create high resolution point-based scatter plots and line graphs, and animated visualizations.

ImagePlot is implemented as a macro which works with the open source image processing program ImageJ. ImagePlot was developed by the Software Studies Initiative with support from the National Endowment for Humanities (NEH), the California Institute for Telecommunications and Information Technology (Calit2), and the Center for Research in Computing and the Arts (CRCA).

ImagePlot download includes additionalImageJ macros developed by our lab. These macros automatically measure basic visual features of all images in a collection (brightness, saturation, hue, shapes) and also list information about images.


ImageMontage | Create high resolution visualizations which show all images in a collection organized in a rectangular 2D grid. The source images can be located in one or multiple directories. The images can also have different sizes. ImageMontage implemented as a macro which runs inside ImageJ.


ImageSlice | Create high resolution "slice" view of an image collection. The source images can be located in one or multiple directories. The images can also have different sizes. ImageSlice is implemented as a macro which runs inside ImageJ.


VisualSense | Interactive exploration of image sets using image plots, bar graphs and pie charts. All graphs "talk" to each other (i.e. selecting a set of images in one graph automatically selects the same set in all other graphs.) You can choose different color palettes for the graphs, arrange a number of graphs in a composition, and save the composition as a PDF. You may expect a good interactive performance with 500-1000 images, where each image is a few hundred pixels in one dimension. You can use software with larger image sets, but the interactive response will be slow. Note: (The documentation was written by a stuff member in the Singapore lab is not complete.. email us if you have questions.)



COLLABORATIVE PEER REVIEW

CommentPress 1.4.1 | extention to a blogging software which allows for commenting on individual paragraphs



OTHER SOFTWARE (these tools are not documented and therefore currently not available for download)

Cultural Analytics Research Environment | real-time visual analytics on scalable displays
PowerWall Presenter | using gigapixel displays for rich media presentations
Cultural Analytics Toolkit | various tools for batch digital image processing and visualization of large sets of images and video


ABOUT OUR SOFTWARE TOOLS

At Software Studies Initiative we don't only study the role of software in contemporary societies theoretically. We also develop new computational tools to enable software-driven cultural theory and criticism.

Digital humanities researchers have already done many projects with texts and geospatial data (see New York Times series of articles on Humanities 2.0 for coverage of some of thee projects). However, the computational analysis of large image and video collections so far is only done by computer scientists.

We are developing techniques and software for analysis and visualization of large cultural visual data sets (art, photography, films, animation, comics, manga, magazines and other print publications, user-generated and remixed video, virtual worlds, video games) which can be used by humanities and social science researches who do not have technical background. We are the first (est. 2007) digital humanities lab to specifically target the analysis of big visual culture data from humanities perspective - and so far, the only lab like that in the world.

Our software works both on regular laptops and desktop computers (PC, Mac, Lunix), and on next-generation large-scale displays such as 287-megapixel HIPerSpace visual supercomputer. (HIPerSpace application developed with Gravity lab at Calit2). We use open-source technologies whenever possible in our development.

These tools have been created and tested for our own projects. We are now gradually documenting them and making available to others. Visit this page periodically to get software updates and new tools.

Recently...