AutoMap: Version History

• Fixed unicode handling
• Improved n-gram extraction

• Preserve umlauts during auto-cleaning of data
• Option: Parts of Speech tags as node attributes in XML outputs
• Output List with parts of Speech per Word
• Anaphora Resolution

• computation of semantic entropy
• integrated installer for GUI and batch mode

• the generalization thesaurus interface is back to a list style
• updated visualizer and matrix editor
• corpus editing: under tools, there is a new tool that allows for removing text after certain words phrases

• changes in union concept list and concept list to facilitate adding to delete list
• refine table functions in union concept list and concept list for sorting

CASOS Email Parser (CEMAP)

• The CASOS Email Parser (CEMAP) that is launched through AutoMap enables the extraction of different types of network information from emails.

Parts of Speech Tagging

• This routine associates every word after the highest level of pre-processing applied so far with its Parts of Speech.

Changes in DyNetML outputs for XML files per Text

• The DyNetML outputs per Map (Semantic Network Analysis) and per Text (Sub-Matrix Analysis) have been changed to reflect:
  • The new DyNetML specification (http://casos.isri.cmu.edu/dynetml/index.html)
  • Isolated nodes
  • Frequency of nodes
  • A set of attributes on the data and statistical results

2.6.30

AutoMap to Database Connector:
• Purpose: Data management.
• Store thesauri and AutoMap analysis outputs (XML files, DyNetML files) in a MySQL based relational database of your choice (Microsoft Access, MySQL database, Microsoft SQL server, PostgreSQL, Oracle).
• If multiple thesauri or outputs are written to the database, no previous data gets overwritten. This is because the database connector associates each data file a unique serial ID.

Feature Selection:
• Purpose: Find the most important terms in a dataset, e.g. in order to build a generalization thesaurus or to get an idea of what is going on in the data.
• Routine that identifies the most important terms in the text set (language independent, dataset dependent). This functionality based on the TF*IDF algorithm.
• Output stored as .csv list.

Standard format for pre-processing material changed from .txt to .csv:
• Purpose: Easier to work with these files in Excel or other csv based editors.
• All pre-processing material (delete list, thesauri, sub-matrix selection file, etc.) that used to be in .txt format is now in .csv format. Thus, the coma as delimiter got replace with a forward slash. If you would like to used previous thesauri, please replace all , with //. If you have any trouble doing this, please let me know.

Thesauri are displayed in tables that can be sorted by each column in the graphical user interface of AutoMap:
• Purpose: Easy sorting per column, easy copy and paste from or to Excel.

Automated removal of noise symbols and/or numbers:
• Purpose: Fast cleaning. More occurrences per term.
• Remove symbols (this is basically any non-significant noise, such as ^, # etc.) while maintaining end-of-the-sentence identifiers (. ? !)
• Remove symbols and numbers (this is basically any non-significant noise and all numbers, such as ^, # etc.)
• Text files after applying either routine can be stored to a directory of your choice.

CompareMap:
• Panels displayed as tables that can be sorted by each column.
• Inputs other than DyNetML files changed from .txt to .csv file format.

2.6.20

• Move from java 1.4.2 to jav 5.0
• New tool suite for loading dynetml file, see only the attributes and what meta-matrix categories they are associated with, edit these associations, and write the changes back to the dynetml file (only applies to GUI version).

2.6.10

New features mainly designed for speeding up batch mode processing as well as coding of large data sets with any version:

• Option for automated edge attribute extraction in GUI version, single batch mode version and multi batch
• Option for storing map files and stat files for GUI version, single batch mode version and multi batch

Other features:

• Language dependent stemmers for Danish, Dutch, English, Finnish, French, German, Italian, Norwegian, Portuguese, Russian, Spanish and Swedish in GUI version.
• Feature for users to customize stemming for all foreign languages and English.
• Move concepts from Union Concept List interface to Delete List (only applies to GUI version). Designed and supported as an iterative process.

2.6.00

• Completely new multi batch mode processing tool – use one script for processing multiple input folders
• Logical inference engine: Fully flexible rule set. This enables the user to chose from pre-defined rules and/ or to build their own rule sets.
• Post Processor: Attribute-value thesauri functionality .
• New tool for merging BiGram file
• New tool for merging Named Entity Recognition (with threshold that users can define)
• BiGram generation: Faster now.

2.5.19

• Routine for creating ego networks from existent DyNetML or XML files. The
  ego node can be an existent or new node in an existent or new meta-matrix
  category.

2.5.18

• New batch mode version for script based back end processing
• Network Reasoner Tool (Rule-based inference of nodes and edges given information existant in a graph)
• Refined Krovetz Stemmer

2.5.17

• New batch mode version for script based back end processing
• Output quantitative information on data reduction achieved via deletion and generalization
• Show and track coverage of input concepts in delete list and thesauri
• New routine in Map Post Processor that enables the association of different attributes with different meta-matrix categories

2.5.16

• Trace window that tracks all user's actions
• More default thesauri for generalization and meta-matrix coding

2.5.15
script version

• Enhanced adding end editing of node attributes in the Post Processor

2.5.14
script version

• Batch mode version of AutoMap.
• Runs under Windows, Linux and on Macs.
• Uses an XML script for specification of coding settings.

• New editors for Node attributes and Edge Attributes in Post Processor (add or modify attributes and related ontological categories)
• Enhanced functionalities for DyNetML editing for all Thesauri in the Post processor

• Add Date Extractor
• Add general time coding to interpretation thesaurus (Map Post Processor)
• Updated Matrix Editor
• Updated Social Insight Visualizer

1. Automated attribute interpretation:
Enumerated attributes (property name, typically attribute 1, attribute 2, etc.) in the property specification section of edges and/ or nodes in a DyNetML will be interpreted. Interpretation here means that the property name will be translated into a meaningful category. If the data from the resulting DyNetML file is stored in a relational database the meaningful category could serve as the key. The type and value of an attribute remain unchanged. The value could serve as the result when searching relational data that is stored in a database. Attributes that were not covered with the translation thesaurus will be renamed into "nyi" (not yet interpreted).
2. Add predefined interpretation thesaurus for Map Post Processor.

• New Tool (under Tools): Post Processor for Relational Data
  • Purpose: Refine or edit attributes of nodes and/ or edges represented in XML/ DyNetML files that were generated with AutoMap or any other software.
  • Five different sub-components that support different levels of details for editing different portions of XML/ DyNetML files.

• Add Output Storage Manager
  • Outputs from pre-processing and different types of analysis can now be stored with user-defined file names at user-defined directories (optional
  • Remembers user-defined output file named and directories when AutoMap is exited and used again later on
• Pre-Procressing and Statement Formation routines are now located are separated into two tabs - pre-processing and analysis
• Add some more pre-defined material for helping to create thesauri:
• List of common punctuation as a possible delete list (can be customized)
• Generalization Thesaurus for converting common contractions in English into non-contracted form (can be customized)
• Generalization Thesaurus for converting common abbreviations in English into non-abbreviated form (can be customized)
• Generalization Thesaurus for known countries (can be customized)
• Meta-Matrix Thesaurus for known countries (can be customized)
• For meta-matrix coding, the entity "tasks-events" is split up into 2 separate entities: task and event. DyNetML files generated in AutoMap will reflect this change.
• For counting unique and total concepts for the union list per text and per corpus all words are treat ed as case insensitive

• Add limited Parts of Speech Extraction functionality for English.
• Add Extractor for Numericals. Add functionality to use output baseline for delete list and/ or meta matrix thesaurus. Language independent feature.
• Add functionality for Bi-Gram Detector to use output as baseline for generalization thesaurus.
• Add functionality for Named Entity Extraction to use output as baseline for generalization thesaurus.
• Add functionality to generalization thesaurus panel to create a positive list thesaurus.
• Add information about workflow, language dependency, automation, customization and appropriateness of repetition of procedures for all pre-processing techniques to User's Guide.
• Mark pre-processing utilities in the interface as being language independent or not.
• User's Guide also contains FAQ section now. This contains information on how to output excel files from AutoMap, how to use Excel files as AutoMap input, how to view and print the meta-matrix thesaurus by entity class

• Add Text File Splitter
• Add customization features for Krovetz Stemmer

• upgraded Java (1.4.2._08)
• batch file for starting script version of AutoMap
• batch file for starting GUI version of AutoMap

• Output option: XML (DyNetML) outputs of mental models (result of Map
  Analysis). This enables the creation of meta-matrix data when no meta-matrix
  coding has been performed. All nodes will be of type knowledge. This enables the
  automated extraction of knowledge networks.
• Updated Visualizer, Matrix Editor, Network Data Format Converter, DyNetML
  file Merger
• CompareMap: DL outputs from map consolidation

• Ontology handling:
  • use any ontology the user provides
  • generate XML (DyNetML) files that represent user-defined ontologies
  • user-defined exclusion of any (multiple) entities given in the ontology from node representation and thus edge formation, but considering them as attributes of entities (e.g. attributes, roles)
• Give user option to decide whether they measure textual networks or social networks that are represented in texts
• Knowledge grouping (cluster identification) in CompareMap
• Speed up KStem

• Provides KStem, a inflection and derivation based stemmer invented by Bob Krovetz

• Add CASOS tools for:
  • Merging XML files
  • Converting network data
  • Visualizing networks
  • Editing network data
• Enhance creation of binary maps

• Enhancement of Thesaurus Application
• Installer for Linux and Unix

• Utilities: Bi-Gram Identification
• Visualization: CASOS visualizer integrated
• Network Conversion: Updated Network Converter
• DyNetML: Inference of Edge Attributes
• CompareMap: .cvs output files for Hamming Distances

• CompareMap: Computation of Hamming Distances:
  • for Map Comparison the Hamming Distance between each given pair of maps is computed
  • for Map Consolidation Hamming Distances between the consolidated map or central graph and each given map is computed

• Symmetrize DL files by default
• Automatically store outputs from CompareMap in .map format
• Convert outputs from CompareMap to DyNetML upon user's request

• Dynamic generation of full meta matrix for sub matrix selection from meta matrix model as specified be the user (including user-defined categories)

• How to on Collocation Identification (Bigrams) with AutoMap
• Network Converter tool added that converts data from to:

  • CVS, DL, UCINET, DyNetML, VNA

  • VNA files can be visualized in NetDraw