Automap Graphical User Interface (GUI)
Load Input5.2 Create
a Delete List
5.3 Edit a Delete List
5.4 Apply a Delete Lis
5.4.1 Direct Adjacency
5.4.2 Rhetorical Adjacency
5.5 Un-apply
Delete List
5.6 Save a Delete List
5.7 Save text(s) after application of Delete List
5.8 Example for the Application of a Delete List
Further
Information
How
to cite AutoMap
Questions, Bugs, and Comments
References
AutoMap is a software tool to analyze text using the method of Network Text Analysis. It performs a specific type of Network Text Analysis called Semantic Network Analysis. Semantic analysis extracts and analyzes links among words to model an authors “mental map” as a network of links. Additionally, Automap supports Content Analysis.
Coding in AutoMap is computer-assisted; the software applies a set of coding rules specified by the user in order to code the texts as networks of concepts. Coding texts as maps focuses the user on investigating meaning among texts by finding relationships among words and themes.
The coding rules in AutoMap involve text pre-processing and statement formation, which together form the coding scheme. Text pre-processing condenses data into concepts, which capture the features of the texts relevant to the user. Statement formation rules determine how to link concepts into statements.
Listed below are the steps a user would follow (in typical order) to use AutoMap:
NTA theory is based on the assumption that language and knowledge can be modeled as networks of words and relations. Network Text Analysis encodes links among words to construct a network of linkages. Specifically, Network Text Analysis analyzes the existence, frequencies, and covariance of terms and themes, thus subsuming classical Content Analysis.
In map analysis, a concept is a single idea, or ideational kernel, represented by one or more words. Concepts are equivalent to nodes in Social Network Analysis (SNA). The link between two concepts is referred to as a statement, which corresponds with an edge in SNA. The relation between two concepts can differ in strength, directionality, and type. The union of all statements per texts forms a semantic map. Maps are equivalent to networks.
Social
Network Analysis is a scientific area focused on the study of relations,
often defined as social networks. In its basic form, a social network
is a network where the nodes are people and the relations (also called
links or ties) are a form of connection such as friendship. Social
Network Analysis takes graph theoretic ideas and applies them to the
social world. The term "social network" was first coined in 1954 by J.
A. Barnes (see: Class and Committees in a Norwegian Island Parish). Social
network analysis is also called network analysis, structural analysis,
and the study of human relations. SNA is often referred to as the science
of "connecting the dots."
Today, the term Social
Network Analysis (or SNA) is used to refer to the analysis of any network
such that all the nodes are of one type (e.g., all people, or all roles,
or all organizations), or at most two types (e.g., people and the groups
they belong to). The metrics and tools in this area, since they are
based on the mathematics of graph theory, are applicable regardless of the
type of nodes in the network or the reason for the connections.
For most researchers,
the nodes are actors. As such, a network can be a cell of terrorists,
employees of global company or simply a group of friends. However,
nodes are not limited to actors. A series of computers that interact
with each other or a group of interconnected libraries can comprise a network
also.
Where to find out more on SNA?
Dynamic Network Analysis (DNA) is an emergent scientific field that brings together traditional social network analysis (SNA), link analysis (LA) and multi-agent systems (MAS). There are two aspects of this field. The first is the statistical analysis of DNA data. The second is the utilization of simulation to address issues of network dynamics. DNA networks vary from traditional social networks in that are larger dynamic multi-mode, multi-plex networks, and may contain varying levels of uncertainty.
DNA statistical tools are generally optimized for large-scale networks and admit the analysis of multiple networks simultaneously in which, there are multiple types of nodes (multi-node) and multiple types of links (multi-plex).In contrast, SNA statistical tools focus on single or at most two mode data and facilitate the analysis of only one type of link at a time.
DNA statistical tools tend to provide more measures to the user, because they have measures that use data drawn from multiple networks simultaneously. From a computer simulation perspective, nodes in DNA are like atoms in quantum theory, nodes can be, though need not be, treated as probabilistic. Whereas nodes in a traditional SNA model are static, nodes in a DNA model have the ability to learn. Properties change over time; nodes can adapt: A company's employees can learn new skills and increase their value to the network; Or, kill one terrorist and three more are forced to improvise. Change propagates from one node to the next and so on. DNA adds the critical element of a network's evolution and considers the circumstances under which change is likely to occur.
Automap Graphical User Interface
AutoMap's graphical user interface (GUI) is divided into four primary quadrants (or panels): they will be referred to as P1 (top left), P2 (bottom left), P3 (top right) and P4 (bottom right).
The drop-down menu bar provides access to various analysis tools and utilities. The browse menu bar allows you to quickly navigate between loaded texts.
Index cards, or "tabs," provide a tabular interface allowing you to navigate each panel respectively. The GUI reads any changes dynamically from the XML file. To do that, the user needs to refresh the tool.
The screen shot below highlights primary features of the Automap GUI and where to find them:
It is important to note that P2 can be edited. The other panels (P1, P3, P4) cannot be edited. Information displayed on P1 to P4 always relates to each other. The Text Browse Menu relates to all panels at the same time.
Window sizes do not have an upper threshold. AutoMap will automatically set window size to largest text size upon user’s request. This is a new button on the Analysis Settings panel. This enables text set specific maximum window sizes, which also enhances efficiency.
The Action Tracer Panel in P4 will log preprocessing utilities applied to your text. This is a handy way to keep track of changes and actions relating to your text.
In the various pre-processing panels, such as utilities, "tool tips," provide more information on certain routines. Tool tips become visible when you slide he mouse over that particular tool.
This user's guide provides illustrative examples for all AutoMap functions. The sample texts below will be used throughout this guide.
Tip! it is suggested you follow along using the examples below. To do so, simply copy and paste the texts below into wordpad or notepad and save as a .txt file in the same folder.
| Our Text Example |
|
| Our Text Example II |
|
Load Input
To open a single text file into Automap, proceed as follows from the Automap menu bar:
File Open > Open single file
A file chooser will pop up (screen shot below).
Double click on the file that you wish to analyze > Select the Open button.
The text will be displayed in P1 on tab No. 1. Original Text.
The loading of .TXT files (caps) is now enabled.
Should you wish to analyze multiple texts at the same time, they must be stored in one folder. To do so, proceed as follows from the Automap menu bar:
File menu > Open multiple files
After you select "Open multiple files" a folder chooser will pop up. Again, be sure that correct folder is selected in the folder chooser. The black ellipses in the screen shot below highlight where you should be looking in the file chooser:
Choose the folder that contains the texts you wish to analyze and single click on it. The folder will be highlighted. Do not double click on the folder. Select the Open button (see above screen shot). The first text will be displayed in panel P1, tab no. 1. Original Text . You can browse through the texts by using the Browse Menu.
Text Analysis Utilities
Automap contains text analysis utilities to help you in the pre-processing and data analysis of your text examples. Take time to become familiar with them as they allow you to quickly work with your text examples.
1. Browse through texts
This function enables you to quickly jump from text example to another. All panels are
synchronized in the Browse Menu. A series of screen shots below the instruction ad emphasis on how to access text Automap's text browsing features.
How to use the Browse Menu:
To go backward or forward text by text:
Click the ">" button or the "<" button in the browse menu bar.
To go to first or last text in the text set:
Click the ">>" button or the" <<" button.
To go to a specific text:
Enter the text number in text field right next to the Go to command and hit OK.
The name of the currently selected text is displayed on the Browse menu.
These files can be browsed:
The example shows a part of the Our Text 1.txt in panel P1
tab no. 1 Original Texts.
The browse menu tells you several important facts:
2. Concept List
The Concept List is displayed in panel P2 tab No. 1 Concept List.
The Concept List is created automatically once a text or a set of texts are
loaded or modified in Automap.
The concept list tells you several important facts about your text:
Tip! The number of unique concepts considers each concept only once, whereas the number of total concepts considers repetitions of concepts.
By default, the Concept List is sorted by decreasing frequency of concepts. To sort the list alphabetically, click on the first-column header Concept. In order to resort the list, click on the header of the second-column header Frequency.
2.1 Example for Concept List
The example below shows a part of the Concept List for the text displayed
in the browse menu. The Table
is ordered by Frequency. The concept list contains more entries than the interface
can display:
3. Create and refresh Union Concept List
The Union Concept List, found in panel P2, differs from the Concept List (tab no. 1) in that it considers concepts across all texts loaded in Automap, rather than one single text file. There are several key pieces of information the Union Concept List tells you:
Union Concept List results are displayed on tab No. 2 Union Concept List in P2. However, you must first refresh the union concept list from the file menu, before viewing your results on the o. 2 Union Concept tab. The union concept list can be refreshed after each step of pre-processing in order to visualize the impact of pre-processing operations on the union of concepts.
To refresh the Union Concept, from the drop-down menu bar:
File menu > Refresh Union Concept List.
The ellipse in the screen shot below shows were to access the Refresh Union Concept List from the drop-down menu bar:
The call out box in the screen shot below, shows where to locate the Union Concept tab, which will contain the results of the Refresh Union Concept List analysis:
Your results will be displayed in tab No 2. Union Concept List. The black ellipse in the screen shot below highlights where to find this tab in the Automap GUI.
By default, the list is sorted by decreasing frequency of concepts. In order to sort the list alphabetically click on the first-column header Concept. To re-sort the list, click on the second-header column Frequency.
Note: The number of unique concepts considers each concept only once, whereas the number of total concepts also considers repetitions of concepts.
To save a Union Concept List follow these steps:
File menu > Save Union Concept List.
A file chooser will pop up. The black ellipse below highlights how to save a Union Concept List from the drop-down menu bar.
3.2 Example for Union Concept List
Let us walk through an example of creating a Union Concept List working with our text examples from above. They are restated below for you convenience:
1) Mr. Cray's brown dog ate the lotus blossom at 10 am. Mrs. Brown was unhappy with the dog. She yelled at it saying "You impossible dog!" But the dog kept eating the flowers and weeds. She asked Mr. Cray to stop the dog. He couldn't. Mrs. Brown planted roses and weeded the garden. The silly dog % dug up the roses looking for a vole on June 12, 1880. Weeding was no longer needed.
Prof. Darren, Mrs. Brown & Mr. Cray met the next day to concoct a plan. John Darren and Mrs. Brown put up a scarecrow. She thought it would scare the dog. Mr. Craye putup a fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, and roses.
2) Mr. Cray's brown dog stopped eating the lotus blossom at 12 pm the next day. Mrs. Brown was now happy with the dog. She said "You good dog!" The dog no longer ate the flowers and weeds. Mr. Cray was pleased too.
Mrs. Brown watered the roses and fertilized the garden on June 13, 1880.
Prof. Darren, Mrs. Brown & Mr. Cray met over dinner and discussed how the plan had worked. John Darren and Mrs. Brown would take down the scarecrow the following week. She thought it was too scarry for the dog. Mr. Craye painted his fence. Then Mrs. Brown watered lotus, carnations, daffodils, and roses.
The first step is to load in your multiple texts from which we will create and save a Union Concept List. An empty AutoMap GUI is shown below before any text is loaded into it. This is what you will see when you first run AutoMap:
From the drop-down menu:
File > Open Multiple Files > (select location of folder on applicaple drive containing your text examples)
Select "Open"
The screen shot below displays the Automap GUI after our text examples have been loaded into it.
To run the Create and Refresh Union Concept Utility based on multiple texts:
File > Create and Refresh Union Concept List
The following series of screen shots present a step-by-step on how to create an refresh a union concept list:
Select "Save Union Concept List" from the drop-down menu:
The result is shown in the screen shot below:
The table is ordered by Frequency. The concept list contains
more entries than the interface can display.
The displayed Union Concept List indicates there are 100 unique concepts
and 229 total concepts in the data set.
Text Pre-Processing
1. Introduction to Text Pre-Processing in AutoMap
Pre-processing reduces the data to terms relevant
to you.
Tip! All pre-processing techniques in AutoMap are optional.
There are some points to consider before you begin Pre-Processing:
Pre-processing is semi-automated and iterative and involves several key processes:
Named-Entity Recognition is an Automap feature that allows you to retrieve proper names (e.g. names of people, organizations, places), numerals, and abbreviations from texts (Magnini, Negri, Prevete & Tanev, 2002). The AutoMap Named-Entity Recognition functionality detects:
Stemming detects inflections and derivations of concepts in order to convert each concept into the related morpheme (Jurafsky & Martin, p.83, 654). AutoMap offers 2 stemmers:
A word's collocates are words appearing next to or near to it.
Tip! Collocations occuring with high frequency are powerful indicators of a pattern of meaning in a text.
Collocations are helpful to construct thesauri in AutoMap. AutoMap can identify collocations of size 2 (Bigrams) as shown below:
Deletion removes non-content bearing conjunctions and articles from texts (Carley, 1993). Non-content bearing concepts to be deleted from the texts are denoted in a Delete List. When applying a Delete List, AutoMap searches the text(s) for concepts specified in the Delete List and delete matches from the text(s). Example:
A thesaurus associates concepts with more abstract concepts. When applying a thesaurus, AutoMap searches the text set for the text-level concepts denoted in the thesaurus and translates matches into the corresponding concept. The terminology of a thesaurus depends on the content and the subject of the data set (Burkart, 1997: 163; Zuell & Alexa, 2001: 313).
Generalization Thesaurus.
A generalization thesaurus typically is a two-columned collection that associates text-level concepts with higher-level concepts. The text-level concepts represent the content of a data set, and the higher-level concepts represent the text-level concepts in a generalized way (Burkart, 1997; Klein 1997: 256; Popping & Roberts 1997: 382).
A Meta-Matrix Thesaurus associates text-level concepts with meta-matrix categories. Since one concept might need to be translated into several meta-matrix categories, a meta-matrix thesaurus can consist of more than two columns. For example, the concept “commander” corresponds with the categories agent and knowledge.
For the meta-matrix thesaurus, column headers start with concept knowledge. The order AND naming of column headers of the meta-matrix thesaurus can be changed in the XML file.
Sub-Matrix Selection. The Sub-Matrix Selection denotes which Meta-Matrix Categories should be retranslated into concepts used as input for the meta-matrix thesaurus.
2. Hierarchy of Pre-Processing Techniques
If you apply a pre-processing technique of a lower order prior to a technique of higher order, the pre-processing will be maintained through all following procedures of higher order. You can un-apply each technique after applying it, if needed.
Tip! All pre-processing techniques are optional.
If you wish to apply multiple pre-processing techniques, do this in the following order:
| Numbering of index card tabs on P1: | |
| Numbering of index card tabs on P2: |
3. NLP Utilities
3.1 Named-Entity Recognition
More information about Named-Entity Recognition in AutoMap.
To create a list of all Name-Entities that are contained in the data set opened,
go to Utilities (tab no. 3) in P2 and
click the Create
and save Named Entities List button in the Named-Entities Field. The
resulting list will be automatically saved under NamedEntities.csv in the root
directory of AutoMap.
| The Named-Entity Recognition interface: |
The black ellipses in the screen shot below below highlight where to find the NamedEntities.csv file in your root directory.
3.1.1 Example for Named-Entity Recognition
Resulting NamedEntities.csv
file a
|
|
Redundant concepts can be converted to one word by stemming. Concepts not relevant to the user can be eliminated by deletion.
In the lower left panel (P2) you will find an option under tab no. 3 Pre-Processing Utilities labeled on tab no. 1 as Symbol Removal. This routine removes or strips off all characters that are neither a letter nor a number. It maintains sentence marks. It converts question marks and exclamation marks into sentence marks. This helps replace the delchar option on the delete list in a more user-friendly fashion. The overall purpose of this routine is to do a very thorough cleaning of the data in a fully automated, easy to use fashion. This routine can be unapplied by using the “un-apply” button, which is located close the “apply” button.
3.21 N-gram Identification: Bigrams
More information about Collocation Identification
in AutoMap.
To create a list of all bigrams that are contained in the data set opened,
go to the Utilities, tab no. 3, in P2 and select the Create
Bigram (Correlation)
List button in the N-gram Detection. The
resulting list will be automatically saved under CorrelationList.csv in the
root directory of AutoMap.
4. Stemming
More information about Stemming.
To stem a text (or text set), go to the Stemming (tab no. 4) in P2.
Porter Stemmer:
To apply the Porter Stemmer select the Apply button next
to Porter Stemmer and stemming of irregular verbs for English. The stemmed
text(s) will be displayed on the tab no. 2 Stemmed Text in P1.
To unstem the texts, go to tab no. 4. Stemming
in P2 and select the Un-Apply button. Tab
no. 2. Stemmed Text in P1 will be cleared.
Krovetz Stemmer:
For the Krovetz stemmer, several customization options are offered:
Use radio buttons in the interface to make your selection. By default, capitalized words are stemmed.
These words are collected in a protection list, named selfdefined_protected_concepts.txt, stored in the AutoMap root directory under utilities\KStem. To avoid stemming certain words put them in this list, one word per line, without any line delimiter.
These words are collected in a list of pairwise associations, named selfdefined_pairs.txt, which are stored in the AutoMap root directory under utilities\KStem. To stem a certain word into a pre-defined term, put the pair (first word / pre-defined stem) in the list, one pair per line, without any line delimiter. The selfdefined_pairs.txt list that comes along with AutoMap contains already such pairs, which handle the correct stemming or irregular verbs in English.
To apply the Krovetz stemmer:
Select the Apply button next to K-stem. The stemmed text(s)
will be displayed on tab no. 2.
Stemmed Text index card in P1.
To unstem the texts, go to the tab no. 4 Stemming in P2 and
select the Un-Apply button. The tab no. 2 Stemmed Text in P1 will
be cleared.
4.1 Example for Stemming (Porter)
| Stemmed text in P1 and interface of Stemming index card on P2: |
5. Deletion
The Delete List is not case sensitive.
More information about Deletion in AutoMap.
You can use the predefined Delete Lists that AutoMap offers or create your
own Delete List. All lists can be edited.
5.1 Open a Delete
List
Click the File menu, select Open Delete List and choose
one of the following options:
Open from file. A file chooser will appear. Select a delete list and hit the Open button.
Open small predefined Delete List. AutoMap's predefined small delete list will be opened.
Open extensive predefined Delete List. AutoMap's extensive small delete list will be opened.
The black ellipses in the screen shot below shows where to access the Delete List utility:
The Delete List will be displayed in P2, tab no. 5. Delete List index card:
| Interface of the Delete List index card: |
The Delete List can be edited.
5.1.1 Small predefined Delete List
The Delete List is compiled of words that occur most
frequently in English: a, an,
and, some,
many,
this,
that,
these,
those,
the,
all,
one,
every.
The Small Delete List can be edited.
5.1.2 Extensive predefined Delete List
An Extensive Delete List is based on words occurring most
frequently in English: a, an, and, as, at, but, for,
he, her, hers, him, his, i, it, its, me, mine, my, nor, of, or, our, she, so,
that,
the, their, theirs, them,
they,
to, us, we, who, whoever, whom, whomever, will, would, you,
your, yours, yourself. As the name indicates, the Extensive Delete list contains more words than the Small Predefined Delete List.
The Extensive Delete List can be edited.
5.2 Create a Delete List
There are two ways to create a Delete List:
1. Within AutoMap:
Go to the Delete List index
card.
The general structure of a Delete List is one single concept per
line. Add concepts by typing one concept per line. Hit enter after entering
a concept. Avoid empty lines. See the example for
more information.
2. Outside of AutoMap:
Use a text editor to create a Delete List.
Please consider these instructions to create a Delete List:
5.3. Edit a Delete List
On the Delete List index card you can:
5.4 Apply a Delete List
If you wish to apply a Delete List and a Thesaurus we recommend
first applying a Delete List and then
a Thesaurus. Next, follow these steps:
When applying a Delete List AutoMap does three things:
If rhetorical adjacency was chosen placeholders (xxx) are inserted where a concept was deleted. The placeholders retain original distances of maintained concepts for purposes of visualization and analysis.
To apply multiple delete lists load the first one in, apply it, then load in the next, apply it, and so on.
5.4.1 Direct Adjacency
If direct adjacency is chosen, concepts in the text that match concepts specified
in the delete list will be
deleted from texts. As a result concepts
left
and right
of a deleted
concept move together and will be treated
as directly
adjacent to each other for visualization
and analysis.
To apply direct adjacency check
the radio button in the Delete List index
card. Then apply the
delete list.
If the user does not change the adjacency
option, AutoMap uses direct adjacency for
deletion and analysis.
5.4.2 Rhetorical adjacency
If rhetorical adjacency is chosen placeholders "xxx" are inserted
where a concept was deleted. The
placeholders retain the
original distances
of the maintained concepts visually for
analysis.
To apply direct adjacency check the button on the Delete List tab.
Then apply the
delete list.
If the user does not change the adjacency
option, AutoMap uses direct adjacency for
deletion and analysis.
5.5 Un-Apply a Delete List
To un-apply a Delete List that was applied to the data, in P2
go to the Delete List (tab no. 5) index card and
select the Un-Apply button.
The tab no. 3 Delete List index
card on P1 will be cleared.
5.6 Save an applied Delete List
To save a Delete List that you have applied
to the data, click the File menu, select Save
Delete
List as. A file chooser will pop up.
5.7 Save text(s) after application of Delete List
To save the text(s) after the application of the Delete List, click
the File
menu,
select Save
Text(s) after Delete List applied. All texts are automatically saved
in a folder called "preprocessed" in the root directory of AutoMap. The filename
will be "after_DL_NameOfYourText.txt".
5.8 Examples for the application of a Delete List
| Input text | Tool used | Setting | Resulting text |
Mr. Cray's brown dog ate the lotus blossom at 10 am. Mrs. Brown was unhappy with the dog. She yelled at it saying "You impossible dog!" But the dog kept eating the flowers and weeds. She asked Mr. Cray to stop the dog. He couldn't. Mrs. Brown planted roses and weeded the garden. The silly dog % dug up the roses looking for a vole on June 12, 1880. Weeding was no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met the next day to concoct a plan. John Darren and Mrs. Brown put up a scarecrow. She thought it would scare the dog. Mr. Craye put up a fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, and roses.
|
AutoMap's extensive Delete List: a, an, and, as, at, but, for, he, her, hers, him, his, i, it, its, me, mine, my, nor, of, or, our, she, so, that, the, their, theirs, them, they, to, us, we, who, whoever, whom, whomever, will, would, you, your, yours, yourself |
Direct Adjacency (default) | Mr. Cray's brown dog ate lotus blossom 10 am. Mrs. Brown was unhappy with dog. yelled saying "You impossible dog!" dog kept eating flowers weeds. asked Mr. Cray stop dog. couldn't. Mrs. Brown planted roses weeded garden. silly dog % dug up roses looking vole on June 12, 1880. Weeding was no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met next day concoct plan. John Darren Mrs. Brown put up scarecrow. thought scare dog. Mr. Craye put up fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, roses. |
| Rhetorical Adjacency | Mr. Cray's brown dog ate xxx lotus blossom xxx 10 am. Mrs. Brown was unhappy with xxx dog. xxx yelled xxx xxx saying "You impossible dog!" xxx xxx dog kept eating xxx flowers xxx weeds. xxx asked Mr. Cray xxx stop xxx dog. xxx couldn't. Mrs. Brown planted roses xxx weeded xxx garden. xxx silly dog % dug up xxx roses looking xxx xxx vole on June 12, 1880. Weeding was no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met xxx next day xxx concoct xxx plan. John Darren xxx Mrs. Brown put up xxx scarecrow. xxx thought xxx xxx scare xxx dog. Mr. Craye put up xxx fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, xxx roses. | ||
The following
concepts were added to the Delete List. Then the Delete List was applied
again. |
Direct Adjacency (default) | Mr. Cray's brown dog ate lotus blossom 10 am. Mrs. Brown unhappy dog. yelled saying "You impossible dog!" dog kept eating flowers weeds. asked Mr. Cray stop dog. couldn't. Mrs. Brown planted roses weeded garden. silly dog % dug up roses looking vole on June 12, 1880. Weeding no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met next day concoct plan. John Darren Mrs. Brown put up scarecrow. thought scare dog. Mr. Craye put up fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, roses. | |
| Rhetorical Adjacency | Mr. Cray's brown dog ate xxx lotus blossom xxx 10 am. Mrs. Brown xxx unhappy xxx xxx dog. xxx yelled xxx xxx saying "You impossible dog!" xxx xxx dog kept eating xxx flowers xxx weeds. xxx asked Mr. Cray xxx stop xxx dog. xxx couldn't. Mrs. Brown planted roses xxx weeded xxx garden. xxx silly dog % dug up xxx roses looking xxx xxx vole on June 12, 1880. Weeding xxx no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met xxx next day xxx concoct xxx plan. John Darren xxx Mrs. Brown put up xxx scarecrow. xxx thought xxx xxx scare xxx dog. Mr. Craye put up xxx fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, xxx roses. |
6. Generalization
Thesaurus
More information about thesauri.
The Generalization Thesaurus is not case sensitive.
6.1. Open a Generalization Thesaurus
Click the File menu, select
Open Generalization Thesaurus. A file chooser will pop up. Double click
the thesaurus you wish to wish
to open or single click the thesaurus and then hit the Open button.
The
thesaurus will
be
displayed
on P2, 6. Generalization
Thesaurus index card.
6.2. Create a Generalization Thesaurus
There are two ways to create a Thesaurus:
1. Within AutoMap:
Go to P2, (tab no. 6) Generalization Thesaurus (see
also the interface of the Generalization
Thesaurus index card for an example).
Use the Text Area on this Index card.
Build and edit a thesaurus.
AutoMap supports users in building a generalization thesaurus by loading the union of concepts from the highest level of pre-processing applied into the Generalization Thesaurus field. This is found on the Generalization Thesaurus index card and can be used upon demand.
Follow these steps to load the union concept list into the Generalization Thesaurus field:
This concept list loaded into AutoMap can be refined by applying Named-Entity
Recognition and Deletion prior to Generalization.
Here is an example for multi-step
pre-processing:
To further illustrate multi-step pre-processing techniques, copy the text passages below, then save as a TXT file as "Our Text I.txt" and "Our Text II.txt " respectivley. Load these examples into Automap to follow along.
Input texts:
Our Text I.txt
Mr. Cray's brown dog ate the lotus blossom at 10 am. Mrs. Brown was unhappy with the dog. She yelled at it saying "You impossible dog!" But the dog kept eating the flowers and weeds. She asked Mr. Cray to stop the dog. He couldn't. Mrs. Brown planted roses and weeded the garden. The silly dog % dug up the roses looking for a vole on June 12, 1880. Weeding was no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met the next day to concoct a plan. John Darren and Mrs. Brown put up a scarecrow. She thought it would scare the dog. Mr. Craye put up a fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, and roses.
Our Text II.txt:
Mr. Cray's brown dog stopped eating the lotus blossom at 12 pm the next day. Mrs. Brown was now happy with the dog. She said "You good dog!" The dog no longer ate the flowers and weeds. Mr. Cray was pleased too. Mrs. Brown watered the roses and fertilized the garden on June 13, 1880. Prof. Darren, Mrs. Brown & Mr. Cray met over dinner and discussed how the plan had worked. John Darren and Mrs. Brown would take down the scarecrow the following week. She thought it was too scary for the dog. Mr. Craye painted his fence. Then Mrs. Brown watered lotus, carnations, daffodils, and roses.
| Pre-processing technique applied | Entries |
Result | ||||||||||||||||||||||||||||||||||||||||
1. Create Named-Entity List. |
|
|||||||||||||||||||||||||||||||||||||||||
| 2. Create Named-Entity List and use it to build a generalization thesaurus. | Mr. Cray's/Mr. Craye Mr. Craye/Mr. Cray's |
|||||||||||||||||||||||||||||||||||||||||
| 3. Add further words that belong together to the generalization thesaurus. | Prof. Darren/Prof_Darren |
|||||||||||||||||||||||||||||||||||||||||
| 4. Apply generalization thesaurus (no thesaurus content only). | mr. craye brown dog ate the lotus blossom at 10 am. mrs. brown was unhappy with the dog. she yelled at it saying "you impossible dog!" but the dog kept eating the flowers and weeds. she asked mr. cray to stop the dog. he couldn't. mrs. brown planted roses and weeded the garden. the silly dog % dug up the roses looking for a vole on june 12, 1880. weeding was no longer needed. mrs. brown & mr. cray, mrs. brown & mr. cray met the next day to concoct a plan. john darren and mrs. brown put up a scarecrow. she thought it would scare the dog. mr. cray's put up a fence. problem solved. brown planted lotus, carnations, daffodils, and roses. |
|||||||||||||||||||||||||||||||||||||||||
| 3. Deletion (rhetorical adjacency). | a |
Mr. Cray's brown dog ate xxx lotus blossom at 10 am. Mrs. Brown was unhappy with xxx dog. She yelled at it saying "You impossible dog!" But xxx dog kept eating xxx flowers and weeds. She asked Mr. Cray to stop xxx dog. He couldn't. Mrs. Brown planted roses and weeded xxx garden. xxx silly dog % dug up xxx roses looking for xxx vole on June 12, 1880. Weeding was no longer needed. |
||||||||||||||||||||||||||||||||||||||||
| 4. Create Union Concept List. | (Coincides with Union Concept List loaded into AutoMap, see cell below) | |||||||||||||||||||||||||||||||||||||||||
| 5. Load Union Concept List into AutoMap. |
|
Outside of AutoMap:
Use a text editor to create a Thesaurus.
Build and edit a thesaurus.
Save the Thesaurus.
Open the
Thesaurus in AutoMap.
You can edit the Thesaurus in AutoMap if
you wish.
6.3 Edit a Generalization Thesaurus
You can add, change or drop the lines of a thesaurus on P2, (tab no. 6) Generalization Thesaurus.
The general structure of a Thesaurus follows the five points below (see also the interface of the Generalization Thesaurus index card for an example):
6.4 Apply a Generalization Thesaurus
If you wish to apply a Delete List and a Generalization Thesaurus please be sure to use the Delete List first and then the Thesaurus. Then go through the following process:
6.4.1 Thesaurus content only
If the Thesaurus content only option is chosen AutoMap performs the
following steps:
To select the Thesaurus content only option check the Thesaurus content only item on P2, (tab no. 6.) Generalization Thesaurus index card. You can now choose to either use direct or rhetorical adjacency for the application of the Generalization Thesaurus. Then apply the Generalization Thesaurus. To switch from not using the Thesaurus content only option uncheck the Thesaurus content only item on P2, (tab no. 6.) Generalization Thesaurus and apply the Generalization Thesaurus again.
If the Thesaurus content only option is NOT chosen AutoMap performs the following steps:
The Thesaurus content only item on P2, (tab no. 6) Generalization Thesaurus index card by default is not checked because AutoMap does not apply the Thesaurus content only option. To switch to using the Thesaurus content only option check the Thesaurus content only item on P2, (tab no. 6) Generalization Thesaurus and then apply the Generalization Thesaurus again.
Direct adjacency means that original distances of concepts that represent the key concepts will neither be visualized nor considered for analysis.
To choose the direct adjacency click the Direct button in the Adjacency field on P2, (tab no. 6) Generalization Thesaurus index card. Then apply the Generalization Thesaurus. If the user does not change the adjacency option, AutoMap uses direct adjacency for generalization and analysis.
Rhetorical adjacency means that the original distance of key concepts will be considered for the analysis. Original distances of concepts that represent the key concepts will be visually symbolized by placeholders (xxx) and considered for analysis. Rhetorical adjacency can only be applied if the Thesaurus content only option was not chosen.
To choose the rhetorical adjacency
click the Rhetorical button
in the Adjacency field on P2, Generalization
Thesaurus (tab no. 6). Then apply the
Generalization Thesaurus.
If the user does not change the adjacency option, AutoMap uses direct adjacency
for analysis.
6.5 Un-Apply a Generalization Thesaurus
To un-apply a Generalization Thesaurus that was applied to the data, go to P2, Generalization Thesaurus (tab no. 6) and hit the Un-Apply button. The Generalization Thesaurus (Tab no. 4) on P1 will be cleared.
6.6 Save an applied Generalization Thesaurus
To save a Generalization Thesaurus that you have applied to the data, click
the File menu, select Save
Generalization Thesaurus As ( a file chooser will
pop up).
6.7 Save text(s) after application of Generalization Thesaurus
To save the text(s) after the application of the Generalization Thesaurus, click the File menu, select Save Text(s) after Generalization Thesaurus applied. All texts are automatically saved in a folder called "preprocessed" in the root directory of AutoMap. The filename will be after_general_thes_NameOfYourText.txt".
6.8 Example for the building and applying a Generalization Thesaurus
Applying a thesaurus to text that was not pre-processed
| Input text | Tool used | Setting | Resulting text |
Mr. Cray's brown dog ate the lotus blossom at 10 am. Mrs. Brown was unhappy with the dog. She yelled at it saying "You impossible dog!" But the dog kept eating the flowers and weeds. She asked Mr. Cray to stop the dog. He couldn't. Mrs. Brown planted roses and weeded the garden. The silly dog % dug up the roses looking for a vole on June 12, 1880. Weeding was no longer needed. Prof. Darren, Mrs. Brown & Mr. Cray met the next day to concoct a plan. John Darren and Mrs. Brown put up a scarecrow. She thought it would scare the dog. Mr. Craye put up a fence. Problem solved. Then Mrs. Brown planted lotus, carnations, daffodils, and roses. |
Mr Craye/Mr_Cray |
Not Selected - Thesaurus content only (default) | mr. cray's brown dog eating the lotus blossom at 10 am. mrs. brown was unhappy with the dog. she yelling at it saying "you impossible dog!" but the dog kept eating the flowers and weeds. she asked mr. cray to stop the dog. he couldn't. mrs. brown planted roses and weeded the garden. the silly dog % dug up the roses looking for a vole on june 12, 1880. weeding was no longer needed. prof. darren, mrs. brown & mr. cray met the next day to concoct a plan. prof_john_darren and mrs. brown put up a scarecrow. she thought it would scare the dog. mr. craye put up a fence. problem solved. then mrs. brown planted lotus, carnations, daffodils, and roses. |
Thesaurus content only,
Direct Adjacency |
. eating... yelling......,.. .,... prof_john_darren.......,,,. | ||
| Selected - Thesaurus content only, Rhetorical Adjacency | xxx. xxx xxx xxx eating xxx xxx xxx xxx xxx xxx. xxx. xxx xxx xxx xxx xxx xxx. xxx yelling xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx. xxx xxx xxx. xxx xxx xxx xxx xxx. xxx xxx. xxx. xxx xxx xxx xxx xxx xxx xxx. xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx, xxx. xxx xxx xxx xxx xxx. xxx. xxx, xxx. xxx xxx. xxx xxx xxx xxx xxx xxx xxx xxx. prof_john_darren xxx xxx. xxx xxx xxx xxx. xxx xxx xxx xxx xxx xxx xxx. xxx. xxx xxx xxx. xxx xxx. xxx xxx. xxx xxx xxx, xxx, xxx, xxx xxx. |
Applying a thesaurus to text that was pre-processed with a Delete List, direct adjacency:
| Input text | Tool used | Setting | Resulting text |
Text after application of customized extensive delete list, direct adjacency: Mr. Cray's brown dog ate lotus blossom 10 am. Mrs. Brown was unhappy with dog. yelled saying "You impossible dog!" dog kept eating flowers weeds. asked Mr. Cray stop dog. couldn't. Mrs. Brown planted roses weeded garden. silly dog % dug up roses looking vole on June 12, 1880. Weeding was no longer needed.
|
Thesaurus (same as above) |
Not selected - Thesaurus content only (default) | mr. cray's brown dog eating lotus blossom 10 am. mrs. brown was unhappy with dog. yelling saying "you impossible dog!" dog kept eating flowers weeds. asked mr. cray stop dog. couldn't. mrs. brown planted roses weeded garden. silly dog % dug up roses looking vole on june 12, 1880. weeding was no longer needed. prof. darren, mrs. brown & mr. cray met next day concoct plan. prof_john_darren mrs. brown put up scarecrow. thought scare dog. mr. craye put up fence. problem solved. then mrs. brown planted lotus, carnations, daffodils, roses. |
Selected - Thesaurus content only, Direct Adjacency |
. eating... yelling......,.. .,... prof_john_darren.......,,,. |
||
| Thesaurus content only, Rhetorical Adjacency | xxx. xxx xxx xxx eating xxx xxx xxx xxx. xxx. xxx xxx xxx xxx xxx. yelling xxx xxx xxx xxx xxx xxx xxx xxx xxx. xxx xxx. xxx xxx xxx. xxx. xxx. xxx xxx xxx xxx xxx. xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx, xxx. xxx xxx xxx xxx xxx. xxx. xxx, xxx. xxx xxx. xxx xxx xxx xxx xxx xxx. prof_john_darren xxx. xxx xxx xxx xxx. xxx xxx xxx. xxx. xxx xxx xxx. xxx xxx. xxx xxx. xxx xxx xxx, xxx, xxx, xxx. |
Applying a thesaurus to text that was pre-processed with a Delete List, rhetorical adjacency:
| Input text | Tool used | Setting | Resulting text |
Text after application of Customized extensive delete list, rhetorical adjacency: Mr. Cray's brown dog ate xxx lotus blossom xxx 10 am. Mrs. Brown was unhappy with xxx dog. xxx yelled xxx xxx saying "You impossible dog!" xxx xxx dog kept eating xxx flowers xxx weeds. xxx asked Mr. Cray xxx stop xxx dog. xxx couldn't. Mrs. Brown planted roses xxx weeded xxx garden. xxx silly dog % dug up xxx roses looking xxx xxx vole on June 12, 1880. Weeding was no longer needed. |
Thesaurus (same as above) |
Not Selected - Thesaurus content only (default) | mr. cray's brown dog eating xxx lotus blossom xxx 10 am. mrs. brown was unhappy with xxx dog. xxx yelling xxx xxx saying "you impossible dog!" xxx xxx dog kept eating xxx flowers xxx weeds. xxx asked mr. cray xxx stop xxx dog. xxx couldn't. mrs. brown planted roses xxx weeded xxx garden. xxx silly dog % dug up xxx roses looking xxx xxx vole on june 12, 1880. weeding was no longer needed. prof. darren, mrs. brown & mr. cray met xxx next day xxx concoct xxx plan. prof_john_darren xxx mrs. brown put up xxx scarecrow. xxx thought xxx xxx scare xxx dog. mr. craye put up xxx fence. problem solved. then mrs. brown planted lotus, carnations, daffodils, xxx roses. |
Thesaurus content only, Direct Adjacency |
. eating... yelling......,.. .,... prof_john_darren.......,,,. |
||
| Thesaurus content only, Rhetorical Adjacency | xxx. xxx xxx xxx eating xxx xxx xxx xxx xxx xxx. xxx. xxx xxx xxx xxx xxx xxx. xxx yelling xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx. xxx xxx xxx. xxx xxx xxx xxx xxx. xxx xxx. xxx. xxx xxx xxx xxx xxx xxx xxx. xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx, xxx. xxx xxx xxx xxx xxx. xxx. xxx, xxx. xxx xxx. xxx xxx xxx xxx xxx xxx xxx xxx xxx. prof_john_darren xxx xxx. xxx xxx xxx xxx xxx. xxx xxx xxx xxx xxx xxx xxx. xxx. xxx xxx xxx xxx. xxx xxx. xxx xxx. xxx xxx xxx, xxx, xxx, xxx xxx. |
7. Meta-Matrix Thesaurus
More
information about Meta-Matrix Thesaurus.
A Meta-Matrix Thesaurus has to be applied if Meta-Matrix
Analysis should be performed.
A Meta-Matrix Thesaurus associates concepts with meta-matrix categories:
When applying a Meta-Matrix Thesaurus, AutoMap searches the text(s) for the entries specified in the Meta-Matrix Thesaurus and translates matches into related Meta-Matrix categories. If you also want to apply a Delete List or/ and a Generalization Thesaurus you will need to apply these pre-processing tools before the Meta-Matrix Thesaurus.The Meta-Matrix Thesaurus is not case sensitive.
You might also see the meta
matrix model as
implemented in AutoMap to better understand the meta-matrix.
7.1 Open a Meta-Matrix Thesaurus
Click the File menu, select Open Meta-Matrix Thesaurus and choose Open from highest level of pre-processing.
The union of concepts after the highest level of pre-processing applied so far will be displayed in alphabetical order on P2, Meta-Matrix Thesaurus (tab no. 7) index card.
Note: The Meta-Matrix Thesaurus can be edited.
If you have a pre-defined Meta-Matrix Thesaurus available that matches
(some of) the concepts contained in the loaded Meta-Matrix Thesaurus you
can open
this file.
To do so, click the File
menu, select Open Meta-Matrix Thesaurus and
choose Open
from file.
If a concept contained in the pre-defined file matches a concept
in the currently opened Meta-Matrix Thesaurus the meta-matrix
categories assigned to this concept in the pre-defined file will
be automatically
assigned to the concept in the currently opened Meta-Matrix Thesaurus.
Note: The pre-assigned Meta-Matrix Thesaurus can be edited.
7.2 Edit a Meta-Matrix Thesaurus
To each concept that appears in the Concept column of the Meta-Matrix Thesaurus you can assign special categories: