Showing posts with label digitalHumanities. Show all posts
Showing posts with label digitalHumanities. Show all posts

Saturday, 4 January 2014

Finding homonyms in a (Latin) treebank

Homonyms and homographs (H & H) in a language are a good thing to master — a lot of confusion goes away once we have understood the differences, and our grasp of the language is significantly improved.

We don't want to run away from the H & H — we want to tackle them at full speed. That means that, if we read a text, we could pick from it a list of phrases with H & H, read them, and see which meaning is employed where.

To do this, we need:

Our task can be done in many ways, including purely "manual" ones. But we would like to use the Perseus Latin Treebank files to find more homonyms and homographs, and to extract phrases from these treebank files as well.

Finding H & H turns out to be a computationally demanding task. On some 50,000+ words my computer chokes and never finishes; 10,000 words is still too much for it. But sets of some 2,500 words are about right, don't take all night to finish.

To an XML file like this:

<uniq>
<w form="vulgo" postag="d--------"/>
<w form="vulgo" postag="d--------"/>
<w form="vulgus" postag="n-s---mn-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnerat" postag="v3spia---"/>
<w form="vulneratum" postag="t-srppma-"/>
<w form="vulnere" postag="n-s---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulnus" postag="n-s---nn-"/>
<w form="vulpes" postag="n-p---fn-"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vult" postag="v3spia---"/>
<w form="vultis" postag="v2ppia---"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-s---mg-"/>
<w form="vultus" postag="n-s---mn-"/>
</uniq>

We apply the following XQuery (using BaseX, in my case):

element uniq
{ let $a := //*:w
for $l in distinct-values($a/@form),
$f in distinct-values($a[@form=$l]/@postag)
return

element w {
attribute form { $l },
attribute postag { $f }
}
}

Result:

<uniq>
<w form="vulgo" postag="d--------"/>
<w form="vulgus" postag="n-s---mn-"/>
<w form="vulnera" postag="n-p---na-"/>
<w form="vulnerat" postag="v3spia---"/>
<w form="vulneratum" postag="t-srppma-"/>
<w form="vulnere" postag="n-s---nb-"/>
<w form="vulneribus" postag="n-p---nb-"/>
<w form="vulnus" postag="n-s---nn-"/>
<w form="vulpes" postag="n-p---fn-"/>
<w form="vult" postag="v3spia---"/>
<w form="vultis" postag="v2ppia---"/>
<w form="vultu" postag="n-s---mb-"/>
<w form="vultum" postag="n-s---ma-"/>
<w form="vultus" postag="n-p---ma-"/>
<w form="vultus" postag="n-s---mg-"/>
<w form="vultus" postag="n-s---mn-"/>
</uniq>

The most interesting cases are those in which @postag attribute begins with a different value for the same @form, e. g:

<w form="vivis" postag="a-p---mb-"/>
<w form="vivis" postag="a-p---md-"/>
<w form="vivis" postag="n-p---mb-"/>
<w form="vivis" postag="v2spia---"/>

Then we look for "vivis" e. g. in the Croatiae auctores Latini text collection.

Wednesday, 19 December 2012

One-line concordance in Linux command line

A recipe. To create a "concordance" — actually, a list of forms from a text with frequencies added — using just a command line, skipping programs such as AntConc (which is great, nice and illuminating, but sometimes I just need to prepare a list quickly). It can be done with the following Bash one-liner:

tr '[:punct:]' ' ' < filename1 | tr '[:upper:]' '[:lower:]' | tr '[:blank:]' ' ' | sort | uniq -c | sed 's/ \{1,\}/","/g' | sed 's/^",//g' | sed 's/$/"/g' > filename2.csv (Filename1 is input file, filename2.csv output in csv format.)

Recently there was a discussion on HUMANIST list whether "bash scripting is a worthwhile approach to "tool" development in the Digital Humanities". People tended to reply no, either learn a "real" language (Python was recommended), or develop a GUI ("like most people, humanists have a strong distaste for the commandline"); I think it was 3:1 against the command line.

Obviously, I disagree. Using bash helped me cross the boundary between user and "programmer" — on the command line one just slides from one region into another. Without a formal education: you have a problem, you look for a solution (discovering gratefully that you stand on shoulders of many colleagues), and bam! it's solved.

I think that digital humanists in general should adopt this kind of sliding — from users to programmers as well as from "classical" to "avant-guarde" scholars — as their MO.

Friday, 7 December 2012

Profiling cultural literacy of Croatian Latin writers

A paper to be presented in the Latin, National Identity and the Language Question in Central Europe conference, organised by the Ludwig Boltzmann Institute for Neo-Latin Studies in Innsbruck (12--15 Dec 2012) will apply E. D. Hirsch's concept of cultural literacy --- which is actually German Allgemeinbildung or Bildungsgut (see it at work in these books), and "opća kultura" in Croatian --- to intellectual horizons of Croatian neo-Latin writers, as represented in the Croatiae auctores Latini collection.

Judging from the programme, the conference offers a chance to present digital research to an audience working mostly in "traditional" ways. This is quite an opportunity; too often digital humanities get separated in a room of their own, where they don't get in anybody's way. So, the challenge is to persuade colleagues that a large-scale search of CroALa using e. g. common terms from CAMENA TERMINI can lead to something interesting.

Update, post-conference, 19/12/2012:

My presentation on cultural literacy is here (note to self: never again try to do a presentation in a browser — outdated browser versions always turn up in crucial moments in crucial spots).

The paper itself is here.

Monday, 16 April 2012

Preparing to launch the Mercurius Croaticus




Mercurius Croaticus, a prosopographical and bibliographical collection of data on Croatian Latin writers, their works, editions, and manuscripts, is nearing its launching point.

The starting dataset comprises information on:

  • 269 authors

  • 1808 works

  • 5867 printed editions

  • 58 manuscripts (more to be added soon)

  • digitised copies, where available