❓ What could your fieldwork app do for you?

[lgessler]

Many of us here are interested in developing new tech for supporting fieldwork, and to this end, I thought it’d be nice to start an open-ended discussion about what an ideal fieldwork app would do. Many people use ELAN and FLEx for doing their documentation work, and we must respect the developers of these apps immensely for their groundbreaking and long-lived work in this space, but I know many people whose needs are not entirely served by either app.

So let’s pose the question: how do ELAN, FLEx, and other tools serve your needs at the moment, and in your ideal world, what would your app do for you? While I haven’t conducted serious fieldwork myself, I’ll get this thread started by sharing some thoughts I’ve heard from fieldworkers I know.

ELAN: nice because it has very configurable tiers and time-alignable annotations, but lacks support for “project-wide” data such as orthographies and morphemes/lemmas.

FLEx: has nice support for “project-wide” data like morphemes and lemmas, enabling bulk edits, but morpheme segmentation doesn’t always work as expected depending on orthographies (esp. for tonal languages), syncing data with collaborators doesn’t always work straightforwardly, and it can be difficult to install.

As far as a new app goes, here are some ideas that have been in my head:

• “Free” dictionary: If an annotation app existed that were implemented as a web application, it would use a database (like SQLite), and since dictionary websites/apps, are primarily read-only, they could be implemented on top of the same database the web application is using, giving anyone using the app a low-maintenance way to also provide their communities with a dictionary
• ML integration: there’s been a lot of work recently in the computational community on automating tasks fieldworkers have to do, such as audio transcription and interlinearization. It’s generally hard to integrate these algorithms into existing apps, but a good one would allow machine learning models to be integrated into the app so that e.g. when an audio file is uploaded it is automatically transcribed, or when a word is entered it is automatically interlinearized.
• Import/export API: there are some formats that are quite common, such as EAF or FLEx XML, but people will inevitably have to import or export their data using less common formats, and the best solution for a case like this is to provide a simple Python or JavaScript API for people to use to import or export their data, assuming a minimum of programming knowledge.

[rgriscom]

Everything you mentioned was good! Just to echo what you said: the interlinearization and project-wide features of FLEx, as well as the flexibility and time-alignment features of ELAN, built-in support for the creation and use of machine learning models, support for non-specialist interfaces, and collaborative editing like Google Docs.

In fact, collaboration functions are increasingly a big need, IMO. If we really want to integrate communities into language documentation then we need tools that aren’t designed for solo researchers.

Also, one aspect of digital fieldwork that has yet to be fully explored and developed is the use of mobile technologies, especially for data collection and transcription/translation tasks. Many endangered language communities live in areas with limited electricity, so laptop use may be restricted or not even possible. A mobile app that works well with a computer app would be really nice. I’ve mostly been using spreadsheet apps with a bluetooth keyboard for linguistic data and Open Data Kit for metadata, and then exporting to CSV for use on the computer. An app like Aikuma that allows community members to create transcriptions and translations for data would be great.

ML integration: there’s been a lot of work recently in the computational community on automating tasks fieldworkers have to do, such as audio transcription and interlinearization. It’s generally hard to integrate these algorithms into existing apps, but a good one would allow machine learning models to be integrated into the app so that e.g. when an audio file is uploaded it is automatically transcribed, or when a word is entered it is automatically interlinearized.

ELAN does have the “recognizer” function which can be used for audio transcription, voice activity detection, etc. I don’t know how widely it is used, though, and I don’t think there is support for external interlinearizers (parsers, etc.). You are right that it would be great to have the flexibility to easily integrate ML models, and then to easily use data opened in the app to re-train models. You should be able to simply choose e.g. Montreal Forced Aligner and select a model, press “run”, make some adjustments to the output of the model, and then press “re-train model”. Rinse and repeat!

Import/export API: there are some formats that are quite common, such as EAF or FLEx XML, but people will inevitably have to import or export their data using less common formats, and the best solution for a case like this is to provide a simple Python or JavaScript API for people to use to import or export their data, assuming a minimum of programming knowledge.

Yes, ELAN is pretty good with its export functions (CSV, TextGrid, etc.), but notoriously bad for importing text data. I made a python script to combine timecode data from TextGrid and text data from a CSV to create an EAF file because it took so much clicking to import the data in ELAN. I’m not as familiar with other less common text formats that linguists may be using - what did you have in mind?

Another mobile app idea that I’ve had in mind is specifically for elicitation. It basically would consist of a spreadsheet for entering transcriptions and translations, paired with a timer function that is synced with the audio recorder. You press play on the audio recorder and start the timer simultaneously, and then press a button in the app to manually create start and end times for each elicited item (like a “lap” button on a stopwatch). You could easily delete or change any associated start/end times during and after recording (e.g. if the speaker makes a mistake and you want to repeat an item). This is an alternative to the automated segmentation of audio and hopefully one that would be even more time efficient, accurate, and accessible for non-tech-oriented linguists.

[mayhplumb]

Apologies if it’s uncouth to drop this in an old thread: I want my fieldwork app to work for elicitation data just as well as natural speech. This means it needs to handle ungrammatical & infelicitous sentences!!

[pathall]

It’s super couth!

Glad you bumped this old thread, actually — it’s interesting to reconsider it after a few years of hacking! It occurs to me that if I were to re-post this today, I think I might have titled it What kinds of interfaces do you need in fieldwork?, since I’m pretty convinced that what we really need now isn’t just one application, but lots of user interfaces that can be combined into an application.

Can you share some more detail about what kind of interface you have in mind? Are these grammaticality judgements (is that the right term here?) a yes/no sort of thing at the sentence level? Or are there levels of grammaticality you’d want to annotate? (Multiple asterisks? Some sort of rank?)

Also, in terms of workflow, are you thinking about process where you start with a pre-composed list of things to test (drawn from previous work, presumably), or are you thinking about this annotation would happen as you collect new content?

Here’s a scribbly idea, would something like this do what you have in mind?

[mayhplumb]

In terms of workflow: I’m picturing this as all after-the-fact. I do my elicitation sessions with pen & paper My main goal is having an auto-glossed, searchable corpus which includes both elicitation and text data. One could input elicitation data into FLEx pretty easily, abusing the notes function for all the extra details, but if you start including ungrammatical examples the search function becomes a minefield. I’d want ungrammatical sentences to show up with special tagging/marking/a big red flashing light in search results.

Other thoughts —
A way to indicate elicitation prompt/context
Multiple felicity judgements linked to different contexts, although that’s a messy one
I like to tag my sentences based on whether they are elicited, constructed by researcher, or spontaneously offered

[pathall]

@mayhplumb

I’d want ungrammatical sentences to show up with special tagging/marking/a big red flashing light in search results.

So to be clear, are you thinking about carrying out this annotation outside of flex? I’m afraid I’m not the person to talk to if you’re thinking of modifying Flex itself! (maybe @hp3 would be?)

I could imagine a workflow that goes:

[graphviz]
digraph {
rankdir=LR;
flextext → grammaticality_interface → modified_flextext
}
[/graphviz]

The grammaticality_interface could probably also have a search function without too much trouble, depending on how fancy the search should be.

Another question would be:

@mayhplumb

if you start including ungrammatical examples the search function becomes a minefield

Is this because the asterisk is a special character, or because searching over the notes field in Flex is problematic?

[mayhplumb]

@pathall

So to be clear, are you thinking about carrying out this annotation outside of flex? I’m afraid I’m not the person to talk to if you’re thinking of modifying Flex itself! (maybe @hp3 would be?)

Oh Lordy no. FLEx is just the software I’m most familiar with, so it jumps to mind. I was thinking broadly in terms of “features I want in a ling annotation app”!

@mayhplumb

if you start including ungrammatical examples the search function becomes a minefield

Is this because the asterisk is a special character, or because searching over the notes field in Flex is problematic?

There’s no way (to my knowledge) to search across notes fields in FLEx. But what I’m thinking about specifically is that the search results are just a clipped preview of the text around the example, which may not include a preceding asterisk. This would make it difficult to know (at a glance) whether a particular sentence is grammatical. To be good a fool-proof, I’d want it to be very obvious when an example is ungrammatical. Otherwise, a user might search for two morphemes, see they co-occur in an example, and conclude that’s grammatical!

[hp3]

I feel like I asked a similar question in 2012 (not that the question shouldn’t be asked regularly)… Because I’m big on graphical representations I drew a picture… Do the things in the linked picture belong in this discussion… maybe we are talking about two different things. The Data Management Space for Linguists | The Journeyler