This is the second post in a four-part series on user testing for DH projects. Yesterday, I discussed some ways of doing “Quick and Dirty DH User Testing”.

Changing my form of communication* reminds me of my audience. The challenges of communicating through the digital highlight the corresponding issues faced by readers of this non-traditionally presented content. My master’s thesis involved a user study evaluating the use of well-established digital humanities archives by a wider audience, a group I still refer to as “amateurs”, and it’s an awareness of this specific audience for our work that I’m discussing here.

In terms of DH creations with advanced scholarly uses, “amateurs” as I’m defining them are a sometimes under-served group--not the relatively small circle of creators and colleagues at the center of a digital archive’s realization, but the nebulous audience just beyond that inner circle: “amateurs” in the old sense of people whose pursuits aren't undertaken in a professional role, but are followed with passion, competence, and curiosity--the humanist geek latent in all of us*. For example, instead of assessing the Blake Archive’s use by it’s developer-users and other Blake scholars using the site as a primary resource for their professional work, I’m interested in users without this degree of specific content knowledge about Blake (some with overlapping knowledges, e.g. art historians, non-Romanticist literary scholars, artists and printers, and creative writers, and some with no relationship to Blake, e.g. the casual reader of poetry) and/or without the familiarity with DH commonplaces and digital archive design that make navigation and use of such sites more intuitive.

In "Collaboratively Curating Early Modern Texts", Martin Mueller argues (in the context of crowdsourcing) that this audience encompasses anyone motivated by "duty, fame, and love: the amateur scholar, the citizen scholar, and everybody else who would like to be recognized for something useful or splendid they have done”. Whether we’re seeking their assistance or making our scholarly efforts meaningful outside of the people who already have advanced knowledge in our field, the digital placement of our work means that it’s more easily accessible to more people* than thirty years ago--but virtual access doesn’t mean much if new site visitors are bouncing off your archive without using it. (Check out Section 2.1 “ Two Types of Digital Text Users” in my master’s thesis for more on types of users for digital humanities archive projects, or yesterday’s post on how you can tell if people aren’t staying on your DH website for more than a few second.)

Jandl Agency's Wikipedia Ads The Jandl ad agency's Wikipedia series is an eloquent reminder that the academy isn't the only place to nurture a love of learning. (Click image for more credit info and close-ups.)

I don’t mean to argue that every piece of DH work needs to serve a general as well as a scholarly audience. There’s much to be said (e.g. on 4humanities) on the worth of humanities as public service, as the preservers and curators and teachers of culture (something enjoyably expressed by Ernest Cline’s novel Ready Player One). On the other hand, I don’t think it unreasonable that any mid- to large-scale project should acknowledge that people will want to use their content in unexpected ways or without specialized knowledge of either the content area or DH commonplaces of form and navigation. With with everyone's limited funding, though, it’s entirely understandable why we don’t have many DH user tests or sites geared at diverse users--and that acknowledgement of these users might need to be more of a nod than a hearty welcome.

There are ways of serving a wider audience with your DH site that don’t need to take too much time or effort, and can also serve the scholars at the core of your user audience:

tours and introductions to the site’s structure and content (e.g. the Blake Archive has a lovely tour of their offerings behind the intro page),
intuitive interfaces or explanations of making meaningful serendipitous discovery within a set of objects (Uncle Tom’s Cabin and American Culture does a good job at supporting browsing rather than searching; each object is contextualized on the same page), or
focused teaching tools and exercises that help the reader learn to think like a scholar in your field (e.g. the Dickinson Electronic Archive’s “Emily Dickinson Writing a Poem: Manuscript Ordering Exercise”, which seems to no longer work but is a great idea).

Digital texts (online scholarly editions, digital archives, thematic research collections, literary engagements, etc.) promise learning beyond that possible with traditional resources--new connections, serendipitous discovery, themes rising out of giant texts corpora. Purpose-built digital texts (as opposed to general-audience sites like the LOC’s American Memory project) are crafted for specific research purposes, with developer-users and devoted academics comprising their primary, "scholar" audience--but a secondary, "amateur" audience of learners with less digital text or content experience also relies on these purpose-built resources.

In my master’s thesis user study, I asked: Does the promise of new learning from digital texts extend beyond scholars to amateurs, or does the design of purpose-built digital texts, by focusing on more experienced users with direct lines of communication to digital text developers, prevent this extension of benefits? This 2009-2010 study gauged one subgroup of amateur users' perceptions of the value of digital texts in terms of answering self-generated research queries. The participants, graduate students from the University of Michigan's information master's program, worked with a digital archive (either the Blake or Whitman project) and provided narrative and survey data assessing their experience of digital text features and perception of their learning success. An analysis of the survey data produced an introductory understanding of amateur users' perceptions of their digital text use, their design needs, and their success or failure at learning through digital texts, an understanding I hope to deepen as part of my Ph.D. dissertation work. The study gathered data about amateur-audience digital text use in three areas: general use, nature and success of research queries, and the interplay between experience with digital texts and success at achieving new knowledge. I decided to focus on digital archive use, assessing these resources by whether users were answering their research queries when using that resource; questions pertaining to digital text usefulness (i.e. relevance of the digital text to the amateur audience) and usability (e.g. efficacy of individual features, site structure) were avoided as outside this study's scope (see yesterday’s post for more on the differences among use, usability, and usefulness in user studies).

It’s important to be specific about what audience you are targeting with your using testing, as it’s important to know who is your site’s primary audience. I’m particularly interested in ways we can serve both scholars and a wider audience as public service, teaching, and for the love of sharing things we care about deeply. Tomorrow, I’ll discuss my experience collecting exemplars of DH evaluation and user study research for my own user study, providing some suggestions on readings and example metrics for anyone interested in testing how their digital work is being used. Meanwhile--I'd love to hear about ways you've reached out to a wider or amateur audience with your DH work!

* ...And that's any unaccustomed form. Needing to justify the code-heavy format of my Ph.D. dissertation has been a useful exercise for me, but I think it would be a useful tool for any traditional dissertator as well. As a textual scholar, I’m convinced that awareness of medium can only lead to work that more meaningfully harnesses the unique features of that form (as well as recognizes its limitations). * As I define amateur in an unconventionally positive way, here’s my positive definition of “geek”, courtesy of Simon Pegg: “Being a geek is all about being honest about what you enjoy and not being afraid to demonstrate that affection. It means never having to play it cool about how much you like something." * But not to everyone. I'll be blogging in the future about my work as a programmer with the MITH/BrailleSC "Making the Digital Humanities Open" partnership, which will make Wordpress, Omeka, and other web content accessible via braille and screen readers