Finding homonyms in a (Latin) treebank

Homonyms and homographs (H & H) in a language are a good thing to master — a lot of confusion goes away once we have understood the differences, and our grasp of the language is significantly improved.

We don't want to run away from the H & H — we want to tackle them at full speed. That means that, if we read a text, we could pick from it a list of phrases with H & H, read them, and see which meaning is employed where.

To do this, we need:

• a list of homonyms and homographs, such as the one Chris Francese and David Kovacs have compiled for Latin
• a text
• tools to find phrases with H & H and collect them in one place

Our task can be done in many ways, including purely "manual" ones. But we would like to use the Perseus Latin Treebank files to find more homonyms and homographs, and to extract phrases from these treebank files as well.

Finding H & H turns out to be a computationally demanding task. On some 50,000+ words my computer chokes and never finishes; 10,000 words is still too much for it. But sets of some 2,500 words are about right, don't take all night to finish.

To an XML file like this:

<uniq>
<w form="vulgo" postag="d--------"/>
<w form="vulgo" postag="d--------"/>
<w form="vulgus" postag="n-s---mn-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnerat" postag="v3spia---"/>
<w form="vulneratum" postag="t-srppma-"/>
<w form="vulnere" postag="n-s---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulnus" postag="n-s---nn-"/>
<w form="vulpes" postag="n-p---fn-"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vultis" postag="v2ppia---"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-s---mg-"/>
<w form="vultus" postag="n-s---mn-"/>
</uniq>

We apply the following XQuery (using BaseX, in my case):

element uniq
{ let $a := //*:w
for $l in distinct-values($a/@form),
$f in distinct-values($a[@form=$l]/@postag)
return

element w {
attribute form { $l },
attribute postag { $f }
}
}

Result:

<uniq>
<w form="vulgo" postag="d--------"/>
<w form="vulgus" postag="n-s---mn-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnerat" postag="v3spia---"/>
<w form="vulneratum" postag="t-srppma-"/>
<w form="vulnere" postag="n-s---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulnus" postag="n-s---nn-"/>
<w form="vulpes" postag="n-p---fn-"/>
<w form="vult" postag="v3spia---"/>
<w form="vultis" postag="v2ppia---"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-s---mg-"/>
<w form="vultus" postag="n-s---mn-"/>
</uniq>

The most interesting cases are those in which @postag attribute begins with a different value for the same @form, e. g:

<w form="vivis" postag="a-p---mb-"/>
<w form="vivis" postag="a-p---md-"/>
<w form="vivis" postag="n-p---mb-"/>
<w form="vivis" postag="v2spia---"/>

Then we look for "vivis" e. g. in the Croatiae auctores Latini text collection.

How to write a BaseX XQuery with RESTXQ

Caution: technical stuff. Over holidays we managed to put up a BaseX XML database instance as a web application on several machines. But how to execute an XQuery there? A simple approach: use an already provided BaseX REST interface. However, the BaseX team seems more interested in RESTXQ, "a set of XQuery 3.0 Annotations and a small set of functions to enable XQuery to provide RESTful services, thus enabling Web Application development in XQuery" (from the unofficial RESTXQ draft). BaseX supports RESTXQ very well, but the existing documentation is somewhat sparse for a non-programmer like me. Conspicuously absent is an example of a "standard" XQuery search directed at a database (or, in XQuery parlance, a collection). This will be provided here.

The task. A BaseX war instance is deployed on a Jetty server (on my machine, which runs Debian Mint, in /var/lib/jetty8/webapps), accessible on the address http://localhost:8080/BaseX772. A database collection crobib was created and populated with several TEI XML files (with FRBR-structured bibliographical data on Croatian Latin authors, works, and manifestations). We want to execute the following query over the internet, finding the text under tei:persName element as child of all eleventh tei:person elements in the collection:

declare namespace tei = "http://www.tei-c.org/ns/1.0";
for $i in collection("crobib")//tei:person[11]
return element p { $i/tei:persName//text() }


The solution. An .xq script should be written and placed (in our case) under the root of the BaseX war archive. If all goes well, it is found and read by Jetty and BaseX when the server is restarted. This is the script (cbxq.xq). Note how the resulting sequence has to be wrapped in a div element:

import module namespace rest = "http://exquery.org/ns/restxq";
declare namespace page = 'http://basex.org/examples/web-page';
declare namespace tei = "http://www.tei-c.org/ns/1.0";
declare %rest:GET %rest:POST %rest:path("person")
function page:person() {
element div {
for $i in collection("crobib")//tei:person[11]
return ( element p { $i/tei:persName//text() } )
}
};
return

The script is requested over the following address: http://localhost:8080/BaseX772/person.

Going further. We want to search not just for eleventh tei:person element, but for whichever we want. The number of the element should be turned into a variable holding an integer, and the variable will be given as part of the HTML address request. The script now looks like this:

import module namespace rest = "http://exquery.org/ns/restxq";
declare namespace page = "http://basex.org/examples/web-page";
declare namespace tei = "http://www.tei-c.org/ns/1.0";
declare %rest:GET %rest:POST %rest:path("person")
%rest:query-param("var", "{$var}")
function page:person($var as xs:integer) {
element div {
for $i in collection("crobib")//tei:person[$var]
return ( element p { $i/tei:persName//text() } )
}
};
return

We had to declare query parameter var: %rest:query-param("var", "{$var}") and to instruct the function page:person to expect it: function page:person($var as xs:integer).

The script is requested with a call such as this (querying the hundredth tei:person): http://localhost:8080/BaseX772/person?var=100.

Text and its links

One of the things that can be done with a TEI XML texts is transforming it into other formats. E. g. into HTML. (Other thing that can be done is including the text in a collection like CroALa, of course.)

A HTML edition can have links of its own. These links can be encoded in XML, so that they "come alive" after the HTML transformation. What is needed is a good idea what to link to.

There are two natural locations. If a text is a transcription of a source, such as a manuscript, and if the source is present on the internet, we can link to page images. And, if text contains quotations of or allusions to other texts (and the texts are present on the internet), we can link to these texts — or hypotexts, as Gerard Genette would call it.

This isn't quite as simple as it seems. What if hypotext is the Bible, with many books, chapters and verses, and our text refers to a precise location in it? Over the centuries, philology developed special techniques just for such referring actions, and the techniques are migrating to the internet. Slowly, though; perhaps not surprisingly, the Bible — with services such as bib.ly — is again first to apply them.

Linking to images and linking to sources are features of our working editions of Andrija Dudić's (Andreas Dudithius, 1533-1589) Latin translation of Dionysius of Halicarnassus' essay on Thucydides, and of a Latin letter written in 1418 by Juraj Jurjević (Georgius de Georgiis, Zadar c. 1400) to Giovanni Battista Bevilacqua.

Edition of Dudić's text refers to local images taken from the archive.org digital facsimile of a 1586 Frankfurt edition. Edition of Jurjević refers both to images of the manuscript (Munich, BSB, Clm 5350) and to (one) passage in Isaiah. We used a Vulgate edition prepared by the Perseus Project, because Perseus uses stable Citation URIs (as developed by Canonical Text Services) for referring to segments of their texts.

Transforming the TEI XML to HTML required slight modification to their set of XSL stylesheets. Technical information about this (written mostly for myself, as I forget it again and again) is here (on klafil dokuwiki).

Asterisks

* * *

Today I had to mark up several Latin poems which used the device above -- three asterisks, even placed as an asterism -- to mark breaks between thematic units.

How to mark a set of asterisks, a typographical asterism, in TEI XML? A TEI-L discussion from 2007 helped, and I decided to use the space element ("indicates the location of a significant space in the copy text").

⁂

Quantification

This morning we had to compile some numbers on the Croatiae auctores Latini collection. Here they are (also on the CroALa developer's blog):

• 143 TEI XML files (including, alas, some duplicates)


• 437.218 words


• 29.637.450 characters


• 16.465,25 Textkarten


• 1029 Druckbogen

Last two strange categories belong to German printing tradition, which was influential in Croatian printing industry; we translated these terms (Textkarte = kartica teksta, Druckbogen = tiskarski arak), and use them still in text accounting.

[Technical note.] Numbers were produced by Linux wc command (cf. recipe) on all XML files currently in CroALa, also available on its Sourceforge page. The Linux one-liner for calculating number of characters and words in multiple XML files was simple:

wc *.xml | awk '{print $3-$1}'

Working on a Work entry

Here's what an entry for a work of a Croatian Latin author — in this case, Beninj Albertini — looks like when it's encoded in TEI XML courtesy of Tamara Runjak, librarian in the HAZU (Croatian Academy of Sciences and Arts):

<bibl xml:id="alb-c01-1825">
   <author><ref target="alb03">Albertini, Beninj</ref></author>
   <title lang="la">Elegia in Arcadum coetu 
       anno jubilaei MDCCCXXV recitata </title>
   <date when="1825">1825</date>
   <note type="metrum">elegijski distih</note>
   <note type="genus">latinski stihovi</note>
   <note type="genus">prigodnice</note>
   <relateditem><ref target="alb1825car">Romae 1825</ref></relateditem>
</bibl>

The whole entry has an identifier in its "bibl" element. The "ref" element in "author" points to the entry in our List of Authors (where all authors' names and known data will be listed).
The date is the creation date (when we know it).
Notes typed "metrum" tell in which metre the poem is (currently this information is in Croatian); others, with type "genus", show different levels of genres. Finally, the "relatedItem" points to the List of Manifestations (i. e. editions, manuscripts etc).

Oh, and by the way, here is "just the text" from this entry, reminding us how much implied information a cultured human being can understand:

   Albertini, Beninj
   Elegia in Arcadum coetu anno jubilaei MDCCCXXV recitata
   1825
   elegijski distih
   latinski stihovi
   prigodnice
   Romae 1825