September 23, 2014 by Amanda Visconti

Infinite Ulysses Technical Update: Summer 2014

The core functionality of my website is the ability to highlight pieces of the Ulysses text and leave annotations on them (questions/answers, interpretations, contextualizations, notes); to achieve this, I'm building off Annotator.js—Javascript code that lets you do what I just described, with a variety of plugin code that lets you add neat features like a heatmap of where the most annotations appear on a text (e.g. on what pages did readers have the most questions? what pages had the most contentious interpretations?). Since the site itself is built with Drupal, I'm using the Drupal modules already built to let Annotator.js work with, for example, Drupal's database as the place to store readers' annotations.

Not as easy as it sounded (is it ever?)

That is the Drupal log under the skull and crossbones.

That's the Drupal logo under the skull and crossbones. This was on my home dry-erase board for a while...

My original project plan was to spend a lot of time coding the look and behavior of the reading interface, with the site functionality not re-inventing various wheels, but instead making connections among existing sets of code that allow textual annotation and social commentary, voting, ranking, etc. In practice, the existing Drupal-module versions of Annotator.js are buggy. Two big examples (for me):

  • the Drupal Annotation modules (Annotation and Annotator) don't store tags added to annotations, so any tags added by a reader disappear forever on refreshing or changing the page (and tags really need to be treated as entities in their own right, and tied into the core Drupal taxonomy); and
  • the code that sets up an individual textual annotation to be an "entity" (a Drupal type of content—such as a blog post, book page, or comment—that other existing Drupal modules automatically know how to act on) isn't completely working either (e.g. you can't add fields—such as a field to add tags to an annotation, fixing the tag storage problem—with the current code).

Having everything not go as smoothly as planned was actually great, though! I had a realistic goal in terms of what I needed to get done by the end of the summer to stay on track (have the site's main functions almost all working, ready to be completely finished at the end of September), and I wasn't worried about doing different work than expected to achieve those goals. Having the Drupal annotation modules need more work than expected meant I got to spend the summer trying various things to solve these issues, and learning a lot in consequence—like creating non-basic custom Drupal entities.

For example, I learned how to make a custom entity fieldable (able to have fields, like a place where you can add tags and have them autocomplete if they match existing tags) and add some fields to it, both by writing code ("programmatically") as well as via the Drupal admin interface (after making the annotation visible to the core Field UI module). I also read through the Annotator code so many times that I became much more comfortable with it, so I'll be able to augment it in the future far more easily.

In the end, it looks like I'll be building off the code fix created by another DH developer who was kind enough to start an online conversation with me about our mutual work improving textual annotation in Drupal (more on that project once they've decided to make their code public). I went back and forth about whether I wanted to build off their solution or keep hacking at my own, but in the end I hadn't been expecting to need to make those code fixes in the first place, so I decided I'd keep my remaining coding time this year free for expected work with the actual reading interface, like making a heatmap of the annotations and porting other existing Javascript/Coffeescript plugins to work with the Drupal version of Annotator.

Other summer work

Screengrab of many many tabs open across the top of my browser.

Screengrab of many many tabs open across the top of my browser. A fairly normal view when I'm coding!

Patching. I submitted an extremely simple patch to the Drupal Annotation module (fixing a misspelled variable) as a way of learning the process; my patch was accepted into the module. Now I know how to create patches when I want to share further work with others using Drupal Annotation.

Installation profiles. I'm currently working on two separate local versions of the Infinite Ulysses website: one to work on improving the Annotation function code, and one for work on everything else (e.g. user profiles, site content, theming). As I work on the site's annotation abilities, I fairly often want to reinstall the site as a slightly quicker alternative to weeding all the references to annotation code out of the database (I should set up uninstall hooks to do this for me eventually...), or to see whether the code works correctly when installing the modules into a new site. Keeping the annotation work on a separate site lets me do this without losing my other work on the site.

But! I don't want to start from a standard Drupal blank site every time I need to reinstall my annotation code; I want a sample page of Ulysses to test the code on, user permissions already set up, modules installed, and various other things configured that I'd rather not do manually every time I reinstall the site. Installation profiles are a way to avoid starting from square one with a Drupal site, instead automatically pre-configuring the site as you'd like it to be (think of it like a module that tells the site to configure and install things a certain way during the site installation process). When I visit install.php, I now have a "Infinite Ulysses Pop-Up" option in addition to standard or minimal Drupal installations. Eventually, this profile will turn into a distribution, such that anyone can use it to install a Drupal site pre-configured to act as a participatory digital edition.

Annotation sidebar. The Annotator.js code I'm working with displays annotations as pop-ups hovering over the text, but from experience with my UlyssesUlysses prototype and usability research, I prefer having annotations authored and displayed in a sidebar. This will also be handy for the tablet version of the website, as you'll be able to slide the annotation sidebar off and on the page like a drawer. I've been playing around with a couple ways of achieving this: modifying this existing CoffeeScript or this extension of that code (demo'd here) to work with Drupal, creating the sidebar by resetting the current CSS on annotation pop-ups and styling them into a sidebar, or a tactic such as this repo which purports to display annotations as Drupal comments.

Book-page styling. I've been vacillating a bit as to the final display structure of the novel: I'm pretty sure I either want to have one page per chapter of the book and infinite scroll to put all the text of a chapter on one page, or have separate pages to match up with each page of the 1922 text of the novel (which I'm using). This is something I might end up doing A/B testing on, showing the text to some visitors one way and to other visitors the other way, and comparing their reactions and stats such as time spent on the site. Here are some questions I'll need whatever solution I use to address:

  • How many annotations can load on a page at a time? How does this limit or non-limit affect site speed?
  • How do annotations load on a very long page that only loads partial sections of the text at a time (when they're scrolled to, i.e. infinite scroll)?
  • How will bookmarking on the site work—will a reader's last visited and furthest page in the book be marked as where they should start next, or will users have the option (or necessity) to bookmark one or more pages themselves?
  • If I do offer one long scrolling webpage per chapter of the novel, can I easily generate bookmarks that jump a reader down to the last line they were reading?
  • What other ways can users chart their progress through the book, such as badges per chapter, or a progress bar for pages read?
  • What if readers want to read it out of order, or only some parts online (and mark parts they read offline as completed)? What if they want to glance ahead without losing the bookmark at where they actually want to pick up reading next?

Long-term thinking. I met with the artist and IP lawyer behind the excellent web/iPad comic of Ulysses (Ulysses Seen). We're all keenly interested in opening Ulysses to more public reading and conversation; they want a way for comic readers to be able to jump from the comic to the text, and I'm also interested in supporting the comic's readers in enjoying the text—so we've been discussing combining our websites into one super-accessible, textual and visual Ulysses edition, after I've defended my dissertation. This is also a good way for me to think about how I'll manage hosting costs and moderation effort in the future.

Other summer activities included lots of reading (as much as I read, I'm always adding more links to my reading list every time I go on Twitter), researching the social modules I want to use on annotations, and drafting site content. In particular, I've been looking into how existing communities (especially online ones where people submit text and/or converse) have succeeded in building friendly spaces, self-moderating, creating community guidelines, and fostering an inclusive and welcoming atmosphere.

I successfully defended my digital humanities doctoral dissertation in Spring 2015. The now-available Infinite Ulysses social+digital reading platform is part of that project; come read with us!