This shows an intelligent attempt to quantify and qualify what DH means and what it should evolve into if it aspires to a stable future.

Thanks for the comment.

I privilege critique over evaluation for several reasons that certainly require more reflection on my part. My first take is that I think probably shot myself in the foot by claiming that “Critique is fundamentally about interpretation.” What i should have said was something more along the lines of “Critique is normally about interpretation, and even though part of this interpretative work is inextricably linked to evaluating it, it goes far beyond it as well–and it’s both of these that we need to bring to our discussion of DH work.” So you’re right: i do mean to return to evaluation to some extent. But I didn’t want to focus on traditional elements of evaluation (and perhaps unfairly elided it) for a few reasons:

Firstly, as everyone knows, evaluating DH projects is inordinately difficult. Part of the reason for this difficulty is in decided what criteria against which value can be judged. As a community, we have not fully articulated those values. As i tried to suggest, some visible efforts toward establishing these have approached it from an evaluative perspectives, but ones that I don’t think are fundamentally helpful (though one understands the need for simplified lists); and i don’t think that such a discourse can come from evaluations alone. In other words, a more substantial critical discourse would make evaluation more useful, if not possible.

Secondly, the appeal to the old-school nineteenth century critics is a deliberate and rhetorical one (as one would hope). Although one could have an entirely different list–i claim no expertise in critical theory or its history–these critics were instrumental in shaping modern notions of (literary) criticism, so i think we can learn from the ways they went about that creative birthing process and decide what to do differently. They are certainly not the only models, but it seems a relevant community to point to (even if their work diminished the evaluative component of criticism). I do not mean to suggest that they should be our model critics in a general sense. References to critics probably needs to be expanded along the lines you suggest (thanks!).

Putting these together, I would say that DH criticism must not be about only interpretation, but the extent to which a work/project is doing something useful/original/[insert other evaluative criteria here] and doing it what seems to be the best way (yes, i have just increased the tautological coefficient of this entire response). In my conception of them, criticism itself, as opposed to mere evaluation, constitutes valuable intellectual work in itself in the way that it sets agendas, determines evaluative criteria, and creates an ongoing discussion about continually shapes our work and how we approach it afterwards.

My confusion b/w peer review and critique is real; I guess I was thinking about how to make critique more a part of peer review, which is its traditional form seems inadequate for DH work.

And yes, collaborative peer review is very different from a crowdsourced variety. I probably should draw a sharper distinction.

Thanks for the insightful critique (and its extension on the google doc).

As I read this, I am thinking about visualization, and it occurs to me that what you have here is the foundation for a network analysis as well, with the compression distance metric output as the strength of ties between your documents.

This is a very useful and comprehensive introduction to some of the practicalities, caveats, and possibilities, of network analysis.

Your comments about the dangers of two-mode networks, for humanists, and the problems of the various metrics not really being meant for two-mode networks should I think be emphasized even more. What methodological or theoretical implications are there for when a two-mode network gets collapsed into a one-mode network? Some sense of what a person must do conceptually (and/or link to tools), in order to collapse a two-mode network down might be in order.

I’ve been using this post as required reading in two of my digital history classes. Thanks!

Shawn