ParaTExt-FLEx Integration Tutorial

 

I Summary/Background

Having access to a well formed dictionary in the target language can be a great help in drafting and checking a translation.  If the dictionary has thesaurus information with synonyms, antonyms, and related words linked through shared semantic domains, it could be used to help translators access the full richness of their language and make a more vivid translation.  Further, if the dictionary could be in electronic form and be accessible while working in Paratext, it would be both fast and convenient for translators to use.

SIL has already developed software to make this type of electronic dictionary with semantic domains.  It is called FLEx which is part of a bigger software program called FieldWorks.  FLEx can also do complex morphological analysis and discourse analysis, but this document will focus mostly on its lexigraphic capabilities.

Paratext and FLEx have been modified to allow Paratext to access a dictionary in FLEx in some ways that can be very helpful to translators.

• Find the definition of words with a right-click
  • Matches based on glosses
  • Can separate affixes from inflected forms to find stems in dictionary (using word analysis from FLEx)
  • If no word is found, provides means to add the word to the dictionary
• Find words related to a word in a Paratext project with a right-click
  • Generates a list based on glosses,  shared semantic domains, defined lexical relations

These Find in Dictionary and Related Words functions can be accessed from the main editing window, inside the Wordlist Tool, and while working with the Biblical Terms Tool.

While working on a translation in Paratext these functions depend on a dictionary of the target language being available.  If you don’t have one, maybe other linguists have one that they could make available.  If the dictionary is in an older format such as ToolBox or LinguaLinks, contact a Language Technology Specialist to help you convert the data.  If no dictionary can be found, consider starting a dictionary project to support your translation.  One good method is the Rapid Word Collection method. (http://www.sil.org/dictionaries-lexicography/rapid-word-collection-methodology) There are other standardized wordlists that can used as a basis for building a lexicon.  There is the Central Africa Wordlist (CAWL) which is about 1,700 words long, and the Multi-lingual Education (MLE) 4000 wordlist.  (http://lingtran.net/resources?structure=Navmenu) And as has been mentioned you can build a dictionary right from inside Paratext, but you would have to create a blank FLEx lexicon to copy the words into.

How+to+associate+a+FLEx+project+with+a+Paratext+project

Once a Paratext project is linked or “associated” with a FLEx project it is easy to import the Paratext data into FLEx for doing sophisticated linguistic analysis.  Morphological analysis can be used to resolve grammatical and spelling issues in the translation. Discourse analysis can be done to increase naturalness and reading comprehension of the translation.  This type of analysis will be most beneficial if other natural texts have already been analyzed in FLEx, and FLEx has already been taught the basics of the grammar of the target language.

II Uninflected words

A) Example of finding the definition of a word while editing a translation

Files needed:

“PTE2015” Paratext project (Vernacular translation)“Beginner’s English” FLEx database (small lexicon with semantic domains and some Lexical relations)

Procedure:

• Download files,
  • restore PTE2015 into Paratext (File > Restore Project > From File)
  • restore "Beginners English" into FLEx, (Choose Restore a project from a backup file from the project picker dialog, or File > Project Management > Restore a Project). 
• Open the “PTE2015” Paratext project, and navigate to Matthew 5:22
• Click on the word “angry” then right-click.  This will open a pop up window.
• Click on Find in Dictionary. (If Find in Dictionary fails after you've just restored the projects, try shutting down FLEx and Paratext and restarting them.

 

• 

► Another pop up window will open with the word and its gloss from the FLEx dictionary “Beginner’s English”.

 

 

B) Example of finding related words while editing a translation

Procedure:

• Highlight or click on the word “angry” then right-click.  This will open a pop up window.
• Click on Find Related Words.

Another pop up window will open with a list of words with similar glosses and words that share semantic domains with “angry” from the FLEx dictionary Beginner’s English.  The translation team can evaluate if any of the words on the list would be better suited for the context of Matthew 5:22.

 

 

 

III Inflected words

A) Example of finding the definition of an inflected word

Files needed:

“TKE_GaA” Paratext project (Vernacular translation)“BT1” Paratext project (Back translation)Takwane Thematic Dictionary FLEx database (Partial lexicon with semantic domains and some Lexical relations)

 

 

 

  

Procedure:

• Download files, 
• restore TKE_GaA and  BT1 into Paratext (File > Restore Project > From File)
• restore "Takwane Thematic Dictionary" into FLEx, (Choose Restore a project from a backup file from the project picker dialog, or File > Project Management > Restore a Project). 
• In TKE_GaA go to Mark 14:30
• Highlight "ogikoodda"
• Right-click and select Find in Dictionary. If Find in Dictionary fails right after restoring the projects, try restarting FLEx and Paratext. 

This is an inflected form and the search will not find it. If you know the morphology of the language deleting affixes manually can help Find in Dictionary function to find the stem in the FLEx lexicon.

• Delete the prefix "ogi" and the lookup will find the stem "koodda" which means "deny" or "reject".
• Click on the Go to  button. This will show the full definition in all the languages in the database.  In this case, you will see English and Portuguese, but you could build a lexicon with whatever other language of wider communication (LWC) that is used in your area. 

If you wanted to see the full entry in FLEx or to see the words contiguous with "koodda" you could click on the Open Lexicon button and it would open FLEx. 

• Click on Close and close the current lookup window.

B) Example of Finding Related Words

Procedure:

• In Paratext go to Mark 1:28.
• Right-click on "Bili" (which means fame).
• Click on Find Related Words.

It will return the Related Words pop up window that will show all words that have the same semantic domains as "bili" and any words that have been linked to “bili” as having a lexical relation with it. Lexical relations includes such things as antonym, synonym, part-whole and so forth.  Notice that in this example the Related Word function found four synonyms for "bili".

 

IV Adding New Words to FLEx Dictionary

When you encounter a word that is not in the FLEx dictionary it can be added to FLEx. This is a way to grow a dictionary as you are translating.  The method of adding words to the FLEx dictionary from within Paratext only enters the lexeme and the gloss in the primary analysis language.  It does not allow for entering a full definition, part of speech or a gloss in other analysis language(s).  If you wish to enter any additional information, you must either do that at a later time, or stop and open FLEx and fill in all the additional information that you wish to enter. 

Now we will find a word in our Paratext project that is not currently in our FLEx dictionary and we will enter that word and its gloss into FLEx.  Start Paratext if it is not open already, and open the project TKE_GaA.  Navigate to Mark 1:16.  Find the word “vowi” and highlight it. Right-click on “vowi” and select Find in dictionary.   You should see a popup like this one:

 

 

We can see from the blank box under Headword that no match was found.  Paratext will find a match if it finds the headword or any of its allomorphs that have been entered into FLEx for that headword--.--  Paratext will not find a match if this is new word or allomorph. As of right now Paratext does not search the variant fields in FLEx, so  it will not match any spelling, dialect, or irregularly inflected variants that have been entered into FLEx. 

 

To create a new entry in the FLEx dictionary, click on the “create” at the bottom of the popup window.

 

 

Type the English gloss for the “vowi” which is “for”.  When you have finished typing in the gloss, click on the “create” button, an entry for “vowi” will be created in FLEx, and the pop up window will close.

 

Now we can go back to Mark 1:16 and lookup the definition of “vowi” and it will be there.  Navigate to Mark 1:16, highlight “vowi” and right-click on it.  Select “Find in dictionary” and now we will see the gloss of “vowi”.

 

We can open the FLEx dictionary and confirm that “vowi” has been added and do any editing that we want to do to that entry.  Click on the “Open Lexicon” button. FLEx will open and display the entry for “vowi”.  We could add grammatical information, full definitions, an example sentence or anything else we would wish to have in our dictionary.

 

 

V Finding a word in the dictionary when Spell Check is turned on

When spell check is turned on (there are red squiggle underlines), then you cannot use the Find in Dictionary or Find related Words function by using a right-click on any words that are underlined by the spell checker. When you right-click on a word underlined by the spell checker, the spell checker opens. To get around this problem there is a keyboard shortcut that allows you to use the Find in Dictionary function on underlined words.  The short cut key is ctrl+d. You may want to look up a word in the FLEx dictionary even if Paratext does not know if the word is spelled correctly or not, because it may already be in the FLEx dictionary with an approved spelling from some other source such as traditional stories that have been collected or other methods of building a dictionary such as the “Rapid Word Collection” method.

We will demonstrate this in action.  Open the Paratext project TKE_GaA, and navigate once again to Mark 1:16.  Click on the spell check tool on the toolbar.  Some of the words in Mark should become underlined.

 

 

Place your cursor inside of the word “anamatota” in Mark 1:16. Press Ctrl+d.  It will invoke the Find in Dictionary function and return the gloss from the FLEx dictionary.

 

(Note: There is no similar keyboard shortcut to invoke the Related Words function while the spell checker is activated.)

VI Adding a word to the dictionary when Spell Check is turned on

It is also possible to add a new word to the FLEx dictionary while ParaTExt’s spell checker is enabled.  We will add the word “vang'ono” from Mark 1:19 to the FLEx dictionary.  Navigate to Mark 1:19.  Turn on spell check if it is not still on.  Place the cursor in the middle of “vang'ono”.  Press Crtl+d.

 

• Click on the Create button on the bottom of the pop up window.
• Type in the gloss “small”. Then click on Create button.  The pop up window will close and there will now be an entry in FLEx dictionary for “vang'ono”.

 

VII Using Find in Dictionary and Related Words from within the Wordlist tool.

The Find in Dictionary and Related words functions are accessible from within the wordlist tool.  Sometimes you may be working in the wordlist tool and you want to know what a word means, or add a word to the FLEx dictionary.  It is not necessary to close the wordlist tool to work with the FLEx dictionary.

In the following example we will be using a word that has a spelling status of “correct” in Paratext. 

• Open the TKE_GaA project navigate to Mark 1:16. 
• Place the cursor inside of the word “seresere” and right-click.  Select Word List….

 

After opening the wordlist tool “seresere” should be displayed in the middle of the wordlist.

 

 

A) Example of Finding in Dictionary in Biblical Terms Tool

The Find in Dictionary and Related Word functions can be found in the Edit menu or you can use the shortcut key Crtl+d for Find in Dictionary. 

Procedure:

• Click on Edit
• Click on Find in Dictionary.

 

A pop up will give the glosses for all senses of “seresere”.  In this case, we can see that “seresere” has two senses.

 

B) Example of finding Related Words in Biblical Terms Tool

Now let’s try the Related Word function.  There is no keyboard shortcut for this function.

Procedure:

• Click on the Edit menu, then choose Related Word.

 

A pop up will appear with any words with the same gloss as “seresere” which in this case is none, and it will list any words that share a semantic domain with “seresere”.

 

VIII Power from added Prefixes and Suffixes in FLEx dictionary.

If you have a word form analysis of a word in FLEX then the Find in Dictionary function will be able to find the stem and affixes of an inflected word form.  For example in project TKE_GaA in Mark 1:16 the word “mathavi” is plural and means “nets”.  If we try and find “mathavi” in the dictionary it cannot find it.

 

 

This problem stems from the standard practice of using the singular form as the entry in the FLEx dictionary, so the Find in Dictionary function will not find a match.  If we parse the word into morphemes in Analyze tab of FLEx’s interlinearizer, or approve an analysis of the word in Word Analyses area,  then the Find in Dictionary function can identify the plural prefix  “ma-“ and the stem “thavi” and return a match.

A) Example of parsing “mathavi”:

 

B) Example using Find in Dictionary function on a word with a FLEx word analysis:

Once we have successfully analyzed the morphemes of “mathavi” and entered a word analysis into FLEx, the Find in Dictionary function can identify the stem and prefixes and returns matches on them from the FLEx lexicon.

 

 

It is beyond the scope of this module to teach how to parse/analyze words in FLEx.  That is a very complicated topic and is a whole other skill that needs adequate time to study and master, but may well be worth your while to learn.  There is information in Help, and the resource “Introduction to Parsing” found at HelpàRecourses.

IX Translation Suggestions taken from FLEx dictionary

Translation suggestions - FieldWorks looks through the lexicon to recommend words

• PT — Tools->Biblical Terms
• You must then make sure you are using Major Biblical Terms (SIL Semantic Domains)
  • Do this by selecting (in the Biblical terms tool) File->Select Biblical Terms->Select Major Biblical Terms (SIL Semantic Domains) and press OK in the dialog box
• You then select a Greek term
• You can then click the “Translation suggestions” button  (at the top of ParaTExt’s Biblical term tool) and FLEx will look up any words with: 
  • the selected domain or
  • gloss in an entry

 

 

That might lead you to alternate renderings. You can then add the “suggestion” as one of the renderings for that key term.

A) Example of searching for possible renderings for a Greek word

Our example will be ἀδελφή --- sister; sister.believer in Matthew 12:50.

Procedure:

• Open project TKE_AaG and navigate to Matthew 12:50.
• Right-click on any word in the verse.
• Click on View Biblical Termsà Current verse(s). Biblical terms tool will open to Mat 12:50.
• Click on FileàSelect Biblical Terms List, and select Major Biblical Terms List (SIL Semantic Domains) if it is not the current Biblical terms list.
• Single click on ἀδελφή, then click on the Translation Suggestion button on the tool bar. It will return a window with words related by semantic domains, or similar glosses.

 

 

B) Example of adding a Translation Suggestion as an approved rendering of a Greek word

To insert a result of a Translation Suggestion look up, click on the desired rendering, and a new menu will appear at the bottom of the pop up box, click on the Insert option and it will be inserted into the Biblical terms list as an approved rendering for that Greek word.

Procedure:

• Single click on “murogora” in the pop up window.  Additional buttons will become active at the bottom.
• Click the Insert button. The pop up window will close, and “murogora” will appear as an approved rendering for ἀδελφή.
• Close the Biblical terms tool.

 

 

X Using FLEx Interlinearizer on Paratext data

Bible books from Paratext can easily be imported into FLEx.  After associating a Paratext project to a Fieldworks lexicon, you can go to the Text & Words area of FLEx and  select the Bible books you want to import.

 

 

Click on the Choose Texts button and you will get a list of the Bible books that are available in Paratext.

 

 

Check off the Bible books that you want to import and then click the OK button.  The books will be imported into FLEx in sections like these below. You can now begin analyzing your translation in FLEx.

 

Example of Gospel of Mark (Maruku) after it was imported into FLEx

 

 

Flex will only import from Paratext if it does not already have a current version of the text. We calculate a checksum on the PT book and store that when we load it into Flex. The next time you choose anything in this book, it sees the checksum hasn’t changed, so it doesn’t reload it. But as soon as you make a change to the book in PT and save it, it will load it when you go to the Flex chooser, or if it’s already checked in the chooser.

 

XI Tips and Warnings

 

1. Does not seem to search variant fields. 
2. If you attempt to change writing systems or other configuration settings in FLEx after you have launched it from within Paratext, you are given stern warnings of possible bad things that will happen.  It is recommended that you close Paratext before making any configuration changes to FLEx.
3. The Related Words function does not currently work on inflected words in the same way as the Find in Dictionary functions does, which is to say that manually deleting affixes will not help the software to find the stem and return any related words.
4. When using the “Translation Suggestion” function in the “Biblical Terms” Tool be aware that your Biblical terms list will revert to the default Biblical terms lists as specified in “Project Properties” every time you open the Biblical Terms Tool.  It is recommended that you make “Major Biblical Terms (SIL Semantic Domains)” your default Biblical terms list if you plan to use the “Translation Suggestions” function frequently.
5. Both Paratext and Fieldworks have Send and Receive capabilities but they are separate from each other.  Anyone using S/R with both programs should read section 3.5 in Help...Resources…Technical Notes on FieldWorks Send-Receive, or check the equivalent   http://fieldworks.sil.org/wp-content/TechnicalDocs/Technical%20Notes%20on%20FieldWorks%20Send-Receive.pdf.Failure to heed these instructions may result in corrupted scripture inside of Flex that requires technical help to try to unravel.
6. If you look up information about FLEx in ParaTExt’s help you may find references to exporting interlinearized texts from Paratext and importing it into FLEx using a “FlexText” format.  Currently FLEx cannot correctly import such files from Paratext.
7. If you have problems getting the association between Paratext and FLEx to work, try restarting one or both programs.

 

XII Paratext Help File for Reference

 

How do I find a word in the dictionary for a project?

In a Paratext project for which you have associated a FieldWorks lexical pr...

In a Paratext project for which you have associated a FieldWorks lexical project, you can display information about a word or display words related by semantic domain.

Display information about a word

If Display Spelling is not selected (on the Checking menu, the icon to the left of the Display Spelling menu item does not have a box around it), do one of the following:

• Right-click a word, and then click Find in Dictionary....
• Click a word. Then, from the Edit menu, select Find in Dictionary....

If Display Spelling is selected (on the Checking menu, the icon to the left of the Display Spelling menu item has a box around it), do the following:

• Click a word and then press Ctrl + D.

A dialog box appears. If FieldWorks Language Explorer is open, you might have to minimize the Paratext window to see the dialog box.
If the word exists in the dictionary, the Dictionary Entry dialog appears with information about it.
If the word does not exist in the dictionary, the Find in Dictionary dialog appears. Click Create to open the New Entry dialog and add that word to the dictionary. Then, you can enter information for that word.

A dictionary generally has a mix of whole words and morphemes which make up the words. In contrast, the Wordlist command on the Tools menu makes a list of whole words found in the text of the project.

To display information about a word, you must have FieldWorks 7.1 (or later) installed on your computer.

Display words related by semantic domain

• Right-click a word, and then click Find Related Words....
• Click a word. Then, from the Edit menu, select Find Related Words....

A dialog box appears. If FieldWorks Language Explorer is open, you might have to minimize the Paratext window to see the dialog box.
If the word is in the dictionary and shares semantic domains with other words, the dialog box will display the related words.
Click a word to see its dictionary definition and words related to it.

To display words related by semantic domain, you must have FieldWorks 7.2 (or later) installed on your computer.

In Paratext 7.6 (and later), the "Find in Dictionary..." and "Find Related Words..." commands are also on the Edit menu of the Wordlist.

Other Paratext help topics (search for them in Paratext help)

• How do I associate a lexical project with a Paratext project?
• What is FieldWorks Language Explorer?

XIII Harvesting words from Paratext for a Fieldworks Dictionary

If you want to harvest words from Paratext to add words or morphemes to a new or existing Fieldworks database you can use the utility  ParatextLexicon to convert data from the Paratext interlinearizer into an SFM (Standard Format Marker) file that can then be imported into a Fieldworks database.  The basic steps are listed below.
 

• Step 1:  Interlinearize text in Paratext project. 

(If you want morphemes in the dictionary, then break the surface forms into morphemes and gloss them as well as the surface/inflected forms.  If you have not been breaking words into morphemes in the interlinearizer prior to this, breaking words into morphemes in the Word List tool may be faster.

• Step 2:  Download and install ParatextLexiconSetup.exe 
• Step 3: Create a new Fieldworks project (if one does not exist). 

It should have writing systems for the vernacular and one for all the glossing languages used.  If you have interlinearized the vernacular in more than one language (for Example, French and English), you will need to already have writing systems for French and English configured as analysis languages in the FIeldworks project. See Help in Fieldworks for more information. If you don't want all the glossing languages from Paratext in your Fieldworks project, you can exclude those fields when importing. 
 

• Step 4:  Convert Lexicon.xml to a SFM file

Click on the button on the end of the “Paratext Lexicon” line.
Navigate to the lexicon.xml file for the Paratext project you wish to convert. Typically “c:\my Paratext projects\xyz\lexicon.xml”.
Click on the button on the end of the “SFM File” line.  Navigate to the folder where you want to store the SFM that will be created, enter a meaningful name for that file and click Open.
Click convert
A pop-up will report a “count of analysis”, click “OK”
A pop-up will report a “count of entries”, click “OK”
A pop-up will report “conversion complete”, click “OK”.

• Step 5: Import the SFM file into the desired Fieldworks database. 

(An existing one or the one you created in step 3.
File-->Import Standard Format data file
Map data to the appropriate writing systems.  To exclude a gloss language from the import, click on the marker in the Content mapping list, click Modify and choose Exclude from Import.
Assign converters if necessary, and import the SFM file into Fieldworks.  
See Help or the document under the help menu, “Technical notes on SFM Database import” for more detailed information.

• Your words and morphemes and their glosses are now in your FIeldworks project. Add word categories and other desired information.