<\/p>\n\n\n\n
Intertextuality has always been at the core of humanistic scholarly research, but it is only very recently that we have been able to leverage computational methods to help us in this endeavor. TextPAIR is a software package built specifically to detect \u201ctext reuses,\u201d a particular manifestation of intertextuality where authors borrow sequences from other texts. Built upon a shingles of n-grams representation of texts,[2]<\/sup>the algorithm that drives TextPAIR relies on the relatively rare re-occurrence of any given n-gram to efficiently find common passages between texts.[3]<\/sup>While TextPAIR and other similar tools have been quite successful in uncovering borrowings between texts,[4<\/sup>]<\/sub>the growing size of digital corpora has led to a problem of scale, where the number of reuses found by the algorithm is such that it makes it difficult to gain an overall perspective of the phenomenon in these corpora.[5]<\/sup>Attempts at tackling this issue of scale have been mostly limited to user interface tweaks, such as faceted browsing \u2014 as seen in TextPAIR \u2014 and clustering methods meant to group together similar text reuses.[6]<\/sub>While these interface tweaks improved the experience somewhat, we have realized the need for rethinking our approach to the exploration of text reuses and developing a new exploratory interface specifically for the purpose of addressing the shortcomings of the standard search and retrieval functionality provided by TextPAIR. This newly developed package, called TextPAIR Viewer, leverages visualization techniques to provide a more scalable and interpretable perspective on the tens of thousands (even millions) of reuses detected by the TextPAIR algorithm.[7]<\/sup><\/p>\n\n\n\n One specific goal of TextPAIR is to provide a mechanism that allows us to move beyond the \u201crelational sociology\u201d based on bibliographic methods (i.e., patterns of citations identified by scholars) described by Long and So[8]<\/sup>(2013) and artfully employed by Nicholl-Johnson in his work on the Sanguozhi<\/em> (\u300a\u4e09\u56fd\u5fd7\u300b) and Shishuo xinyu<\/em>(\u300a\u4e16\u8bf4\u65b0\u8bed\u300b).[9]<\/sup>We sought to create an algorithmic method to produce intertextual analyses driven solely by the sources themselves, essentially a \u201crelational literary analysis,\u201d akin to the \u201cintertextuality\u201d of ideas, signs, and semiotics that Kristeva describes.[10]<\/sup>But with the scaling issue becoming more and more prominent as the size of digital collections \u2014 and with it, the ambition of scholars \u2014 grows, this original goal is no longer met. While TextPAIR has no trouble scaling up to over a hundred thousand texts,[11]<\/sup>it can be difficult for the user to apprehend the relationships between the potentially very high number of detected reuses. We have created the TextPAIR Viewer add-on precisely to overcome this difficulty and fulfill the original promise. By generating a visual representation of reuses, this add-on provides a distant-reading perspective on the \u201cmosaic of quotations\u201d that the TextPAIR algorithm by itself never could.<\/p>\n\n\n\n The current design of the TextPAIR Viewer provides a marked improvement in the way text alignments are presented to users. The key limitation of the standard TextPAIR interface lies in the pairwise relationship between each original passage and its reuse. While grouping results through faceted browsing certainly provides a more compact view of reuses, it still only highlights the relationship between one single text or author with a set of reuses. The main shortcoming of this grouping is that it does not help scholars understand how the authors (or works) that borrow from the same set of passages relate to one another. Indeed, the reality of intellectual influence is that each author or work may have multifaceted relationships with many other authors or works. It is thus important to consider reuses not so much as a strictly one-to-one relationship between the source and the borrower but rather as an intellectual practice that may, for instance, indicate the belonging of an author to a broader literary or cultural community. Therefore, as a distant reading tool, TextPAIR can only, at best, answer questions such as who reused an author\/title the most or which author\/title was most borrowed from, but fails to show whether there are any consistent patterns in the reuses by multiple authors or works. This is where the visualization techniques at the heart of the TextPAIR Viewer come into play. By leveraging the clustering nature of force-directed graphs, the TextPAIR Viewer places texts which tend to borrow from similar sources in specific areas of the visualization, thus highlighting distinct clusters of reuses (see the example below). These intellectual spheres are nearly impossible to identify with frequency tables as provided by faceted browsing.<\/p>\n\n\n\n One of our preliminary case studies in this endeavor was to examine patterns of citation, paraphrase and allusion in the entire corpus of \u201cTwenty-four Chinese Histories\u201d ( \u4e8c\u5341\u56db\u53f2 ), the official historical record of each dynasty in Chinese history, by algorithmically plotting \u2014 graph by graph, from history to history \u2014 changes in aligned passages as detected by the TextPAIR algorithm.[12<\/sup>]As TextPAIR allows for many-to-many relationships throughout any corpus regardless of source, one of the strengths of this approach is that we are not bound by the traditional view that assumes a chronological relationship in the composition of these works. For example, while we think it is certainly valuable to examine questions like how much of Sima Qian’s\uff08\u53f8\u9a6c\u8fc1,c.145\u2013c.86 B.C.E.) original prose in the Shi ji<\/em>\uff08\u300a\u53f2\u8bb0\u300b, the first history, chronologically speaking, dating to 90 B.C.E.) was adopted by Ban Gu(\u73ed\u56fa,32\u201392 C.E.) when the latter composed the Han shu<\/em> (\u300a\u6c49\u4e66\u300b, the second history in the chronology, dating to 111 C.E.), we can equally use TextPAIR to examine passages from the Shi ji<\/em> that have identifiable parallels in any of the twenty-four histories, right through to the Qing shi gao<\/em> (\u300a\u6e05\u53f2\u7a3f\u300b, the last history in the chronology, composed in 1928). And indeed, TextPAIR identified 204 examples of reuse from the Shi ji<\/em> in the Qing shi gao<\/em>, though all are quite short, under 20 characters in length; as one would expect, the detections of reuse with the Shi ji<\/em> as the source are dominated by the Han shu<\/em> (3,933 parallel passages)and Hou han shu<\/em> (\u300a\u540e\u6c49\u4e66\u300b2,282).<\/p>\n\n\n\n In total, TextPAIR identified 322,916 cases of reuse throughout the \u201cTwenty-four Histories.\u201d The table below (Table 1) provides the results of the pairwise identifications, where the number in each cell represents the number of text alignments between the source document in the far left column and each of the target documents in the other columns.<\/p>\n\n\n\n Table 1 Total pairwise reuses for each source and target detected by TextPAIR<\/p>\n\n\n\n Within these hundreds of thousands of alignments identified by TextPAIR, 82 comprised of at least 500 characters. One important aspect of TextPAIR, which drastically simplifies the work of comparing how the reuse of a particular passage differs from the original, is a feature called \u201cShow differences\u201d (Figures 2 and 3). Based on the Myers algorithm for the detection of the longest common subsequence between two strings of text,[13]<\/sup>it is implemented as a front-end plugin which runs on-demand and returns a character-by-character difference between passage pairs: deletions in the later passage are crossed-out in the source passage, and additions in the later passage are highlighted. This type of analysis can be very helpful in determining the strategies behind a particular reuse.<\/p>\n\n\n\n Figure 1 Shi ji<\/em> vs. Han shu<\/em> passage with \u201cShow differences\u201d<\/p>\n\n\n\n Figure 2 Jiu Tang shu<\/em>(\u300a\u65e7\u5510\u4e66\u300b) vs. Xin Tang shu<\/em>(\u300a\u65b0\u5510\u4e66\u300b) passage with \u201cShow differences\u201d<\/p>\n\n\n\n While such a global table certainly helps in understanding the practice of reuse within the \u201cTwenty-four Histories,\u201d this table was compiled manually from results reported in the standard TextPAIR interface, and it also fails to uncover any pattern in the reuses (e.g., the centrality or peripheral location of various nodes). When these results are visualized, the interactive network looks like this (Figure 3).<\/p>\n\n\n\n Figure 3 Force-directed network visualization of the \u201cTwenty-four Histories\u201d[14]<\/sup><\/p>\n\n\n\n Figure 4 Force-directed network visualization of all alignments split by chapter within the \u201cTwenty-four Histories\u201d<\/p>\n\n\n\n As there are only twenty-four works in the corpus, all of which have cases of reuse, this visualization oversimplifies the complexity of the relationships between the works and also the numbers and variety of reused segments. At this point, in order to try to tease out the more fine-grained nuances, we decided to split each work into its constituent chapters and plot them against each other, including parallel passages that fall within different chapters of the same work. The result is a network of great complexity, with thousands of nodes representing the thousands of chapters (Figure 4).<\/p>\n\n\n\n Having attained proof of concept, we are now turning our efforts to transform the 1.0 version of the TextPAIR Viewer into a new 2.0 version with significant advantages. Primary among these will be redesigning the network engine to employ WebGL technology so that users can visualize complex networks (like the network above) in a browser on an average laptop or desktop. Our tests have shown that networks of tens of thousands of nodes tend to render very slowly and responsiveness is generally poor, making it difficult to efficiently explore these networks and the underlying alignments. In addition, we plan to make it possible to generate these networks on demand from within TextPAIR so that users can take any specific subset of the alignments (e.g., by selecting only specific texts or using keywords to narrow down the output) and produce a network of just those results. In other words, users will be able to leverage the Viewer tool to visually explore a predefined set of results and uncover potential patterns that would not be discernible in the standard TextPAIR interface.<\/p>\n\n\n\n We also have planned to update the visualization interface, such as adding a right-side interactive menu listing all the nodes that can be used to programmatically highlight all the communities within the network and the intersections between highlighted nodes. We would also like to include metrics like Laplacian eigenvectors and betweenness centrality for the nodes in the network so that users may easily discover which nodes have the greatest influence upon the overall network and serve as links between clusters of texts. One final change will be to move from two- to three-dimensional visualizations \u2014 yet another reason to move to a WebGL backend capable of rendering more complex scenes \u2014 in order to improve the visual clarity and facilitate the exploration of large-scale, complex networks.<\/p>\n\n\n\n We would like to reiterate that while the examples in this article have all been drawn from our corpus of the \u201cTwenty-four Histories.\u201d We have been producing networks for several large corpora of texts, in languages like French, English and Japanese, all with equal success. Our testing has shown that TextPAIR and the TextPAIR Viewer can be used for any language or Unicode script to identify and visualize text reuse throughout a corpus (or several corpora) of any size or complexity. We expect that these new tools will be of great assistance to scholars interested in intertextuality as well as those who wish to uncover the currents and changes in intellectual history over the longue dur\u00e9e<\/em>.<\/p>\n\n\n\n \uff08\u7f16\u8f91\uff1a\u8096\u723d\uff09<\/p>\n\n\n\n \u6ce8\u91ca\uff1a<\/strong><\/p>\n\n\n\n [1]Julia Kristeva, Desire in Language: A Semiotic Approach to Literature and Art<\/em>, New York: Columbia University Press, 1980.<\/p>\n\n\n\n [2]In this representation, documents are converted to a series of overlapping word n-grams (blocks of n <\/em>number of words) after a series of pre-processing steps which may involve lemmatization, stemming, filtering of non-distinctive words (e.g. stopwords) and so on. While the value of n<\/em> is configurable, we have found that tri-grams offer a good trade-off between rarity and performance of the matching algorithm.<\/p>\n\n\n\n [3]TextPAIR is based heavily on two software packages: PAIR and PhiloLine. For a complete description of the matching algorithm, see Mark Olsen, Russell Horton, Glenn Roe, \u201cSomething Borrowed: Sequence Alignment and the Identification of Similar Passages in Large Text Collections,\u201dDigital Studies\/le Champ<\/em> Num\u00e9rique<\/em>, vol. 2, no. 1, 2011, https:\/\/www.digitalstudies.org\/article\/id\/7224\/.<\/p>\n\n\n\n [4]See also Donald Sturgeon, \u201cUnsupervised Identification of Text Reuse in Early Chinese Literature,\u201dDigital Scholarship in the Humanities<\/em>, vol. 33, no. 3, 2018, pp. 670-684; Paul Vierthaler, Mees Gelin, \u201cA BLAST-based, Language-agnostic Text Reuse Algorithm with a MARKUS Implementation and Sequence Alignment Optimized for Large Chinese Corpora,\u201d Journal of Cultural Analytics<\/em>, 2019, DOI:10.31235\/osf.io\/7xpqe; John Wilkerson, David Smith, Nicholas Stramp, \u201cTracing the Flow of Policy Ideas in Legislatures: A Text Reuse Approach,\u201dAmerican Journal of Political Science<\/em>, vol. 59, no. 4, 2015, pp. 943-956; David A. Smith et al., \u201cInfectious Texts: Modeling Text Reuse in Nineteenth-Century Newspapers,\u201d Proceedings of the Workshop on Big Humanities, Washington, DC: IEEE Computer Society Press, 2013.<\/p>\n\n\n\n [5]See our white paper for the NEH funded Commonplace Cultures project: hcommons.org\/deposits\/item\/ hc:12365.<\/p>\n\n\n\n [6]See the Commonplace Cultures project: Charles Cooney, Clovis Gladstone, “Opening New Paths for Scholarship: Algorithms to Track Text Reuse in ECCO,” Digitizing Enlightenment: Digital Humanities and the Transformation of Eighteenth-Century Studies<\/em>, eds. Simon Burrows, Glenn Roe, Oxford University Studies in the Enlightenment, Voltaire Foundation in association with Liverpool University Press, 2020.<\/p>\n\n\n\n [7]We would like to thank the Neubauer Collegium for its generous support for the development of the TPV as part of the \u201cVisualization for Understanding and Exploration\u201d project: vue.rcc.uchicago.edu.<\/p>\n\n\n\n [8]Richard J. So, Hoyt Long, \u201cNetwork Analysis and the Sociology of Modernism,\u201dBoundary 2<\/em>, vol. 40, no. 2, 2013, pp. 147-182, DOI:10.1215\/01903659-2151839.<\/p>\n\n\n\n [9]Evan Nicoll-Johnson, \u201cDrawing Out the Essentials: Historiographic Annotation as a Textual Network,\u201dJournal of Chinese Literature and Culture<\/em>, vol. 5, no. 2, November 2018, pp. 214-249, DOI: https:\/\/doi. org\/10.1215\/23290048-7256976.<\/p>\n\n\n\n [10]This functionality is analogous to the Chinese Text Project tool outlined by Donald Sturgeon in his 2018 article, but with greater flexibility for analyzing any TEI-encoded corpus in any language and of any size.<\/p>\n\n\n\n [11]See, for example, our work on the \u201cPractices and Legacies of the Enlightenment\u201d project (artfl-project.uchicago.edu\/legacy_eighteenth).<\/p>\n\n\n\n [12]Jeffrey Tharsen, Clovis Gladstone, \u201cUsing Philologic For Digital Textual and Intertextual Analyses of the Twenty-Four Chinese Histories(\u4e8c\u5341\u56db\u53f2),\u201dJournal of Chinese History<\/em>, vol. 4, no. 2, 2020, DOI:10.1017\/ jch.2020.27.<\/p>\n\n\n\n [13]TextPAIR uses the JavaScript implementation of the DiffMatchPatch library developed by Google, Inc.<\/p>\n\n\n\n [14]See https:\/\/users.rcc.uchicago.edu\/~jcarlsen\/TPV\/TPV_histories\/. Network visualizations in the TextPAIR Viewer 1.0 are created using the JavaScript D3 library’s force-directed network layout, with arrows pointing from the source node (a text that borrows from other texts) to the target node (a text that is borrowed from). The direction of the borrowing is inferred from the date of each text. The size of a node reflects the total number of text segments reused from that node; colors for the nodes are generated at random, edges are colored according to the source node.<\/p>","protected":false},"excerpt":{"rendered":" Jeffrey Tharsen \/ The University of Chicago Clovis Glad […]<\/p>","protected":false},"author":3,"featured_media":20457,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[4956],"tags":[155,650,29,94],"class_list":["post-20452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-4956","tag-155","tag-650","tag-29","tag-94"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":6}},"_links":{"self":[{"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/posts\/20452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/comments?post=20452"}],"version-history":[{"count":1,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/posts\/20452\/revisions"}],"predecessor-version":[{"id":20458,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/posts\/20452\/revisions\/20458"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/media\/20457"}],"wp:attachment":[{"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/media?parent=20452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/categories?post=20452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/achieve.dhcn.cn\/en\/wp-json\/wp\/v2\/tags?post=20452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/achieve.dhcn.cn\/wp-content\/uploads\/2023\/06\/Tu_1-9-1024x826.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/achieve.dhcn.cn\/wp-content\/uploads\/2023\/06\/Tu_2-8-1024x860.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/achieve.dhcn.cn\/wp-content\/uploads\/2023\/06\/Tu_3-9-1024x875.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/achieve.dhcn.cn\/wp-content\/uploads\/2023\/06\/Tu_4-8-1024x900.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/achieve.dhcn.cn\/wp-content\/uploads\/2023\/06\/Tu_5-8.png\")
<\/figure>\n\n\n\n