Monday, April 28, 2014

History of GTMetaExp, GTech Loader, and Oracle Spatial Loader

GTI has provided a product called GTMetaExp, short for GT/Metadata Explorer, since 2008 and it currently comes in two flavors:  Intergraph G/Technology and Bentley.  The G/Technology version is significantly more advanced and provides the most functionality, but the Bentley version as well as the Oracle Spatial Loader are related products that are also available that also deserve mention.  I wanted to provide a post that covered the history of the these products and how they  may be useful to you.

G/Tech Loader

Before I can talk about GTMetaExp, the history of the G/Tech Loader for GTViewer must be discussed since it is the origin of most of the technology we use to deal with G/Tech data.  GTI has been exporting Intergraph’s G/Technology data for viewing in GTViewer since 2002, and we have had 4 generations of G/Tech Data Loaders.  Utilizing only the Oracle data, the GTech Loader will query the relevant data to create all of the GTViewer data files (.gtg, .gtn, data.txt, data.idx, etc.).  

The First Generation loader was fairly simple and was essentially a custom VB6 app that we created for each separate dataset we exported.  We did use the G/Tech metadata to style the geometry, but it was a semi-manual task and the style rules were basically converted to VB6 syntax and processed by the application.   As primitive as this application was, we only recently upgraded an early G/Tech user to the current version of the Loader, so it turned out to be very capable at what it did. 

In 2004, we introduced a new version of the G/Tech Loader.  This version is more correctly classified as a CASE (Computer Aided Software Engineering) tool because instead of loading the G/Tech data, it used the G/Tech metadata to generate the code needed to build the original VB6 custom G/Tech Loader application.   The end result was essentially the same as the First Generation G/Tech Loader, but the effort required to implement it for a new dataset was radically reduced.

In 2007, we introduced the Third Generation G/Tech Loader.  It was completely rewritten from scratch and utilized .NET instead of VB6.   This iteration of the loader ditched the CASE tool approach and dynamically integrated the use of the G/Tech Metadata to produce a higher quality exports for GTViewer.  There were many design changes in this generation of the loader, primarily the G/Tech Style Rules were evaluated internally by the loader as it exported the data using the G/Tech Style Rule name for each feature as a GTViewer Style Rule.  This change allowed a much easier way to tweak the styles used in GTViewer without having to reload the data.   Somewhere in this time frame, the G/Tech Styles switched from blobs in the database to relational definitions, and the loader adapted to this change as well.   Tools were also provided to generate GTViewer Style Definition files form the G/Tech Style entries.

In 2011, we introduced the Forth (and current) Generation G/Tech Loader.   While several new features were added, the main changes were centered on switching from the G/Tech 9 Relational Geometries to G/Tech 10 Object Geometries.


So what does the GTech Loader have to do with GTMetaExp?  Well, all of our work that went into building the G/Tech Loader produced a set of tools that did many different things with G/Tech data:  dealing with Metadata, reading geometries (relational and object), computing styles for features, and composing and decomposing features.

Over the many year span of GTech Loader development, several tools were created to help us internally validate converted data or sift through the mounds of metadata to see what needed to be loaded.
The first tool created was the Legend to Style Tool.  It basically provided a GUI to let you point and click navigate from a Legend all the way to a Style.

Another tool was the Feature/Component Tool.  It let you point-and-click browse G/Tech Features and Components along with Database attributes for the tabular records.

Another tool was the Style Definition Tool which allows you to browse GTech styles and convert them to GTViewer style definitions.

Introducing GTMetaExp

All of these tools described above were very useful for their specific purpose.   In 2008, these tools along with some additional functionality were bundled together to create the GTMetaExp Version 1.   This tool was primarily for internal use, but interest from our users prompted us to productize it and make it available to others.  In 2011, GTMetaExp Version 2 was made available to support both G/Tech 9’s relational geometry and G/Tech 10’s object geometries as well as other differences between the two.

The individual tools we had created to support the development and operation of the G/Tech Loader became tabs in the GTMetaExp application:  Feature to Style, Feature and Components, and Styles.   We added new tabs for Picklists which let you browse and modify picklist values in the metadata; Validation which lets you perform various validation routines against the metadata and data, and Export which will export various parts of the metadata to a .MDB or .CSV file (so it can be used in other applications).  

Find Feature

The most powerful feature we have added to GTMetaExp is the Find Feature tab which is technically dealing with the Feature Data instead of the Metadata.   Find Feature allows the user to query for a feature by its attributes or its G3E_FID value.   Once a feature is found, you can perform a variety of tasks with it:   Attribute Info, Style Rule Info, Geometry Info, Connectivity Search, Trace Info, and Ownership Info.

Attribute Info – This tool will show an Attribute Info Dialog for the selected feature much like the one in GTViewer.  One nice feature of this tool is that it can show you the User Name for the Table and Attributes, and it can show you the actual database names as well.

Style Rule Info – Understanding why a style rule is firing for a particular feature can be a particularly useful piece of information.  This tool shows all of the style rules used for a specific feature’s components and highlights the rules that were used.   If you believe that one of the previous rules should have been used, you can select it and press Explain and it will give a detailed analysis of why that rule was or was not selected.

Geometry Info – This tool will show you all of the Geometry information for a specific feature including the elements in a geometry, oriented points for labels and symbols, and can provide the SDO_GEOMETRY as ASCII for a given feature (if you wish to manually make modification in Oracle).   This tool is extremely useful for isolating problems with geometries and also provides a graphical preview of the element.  While the screenshot below shows an Object Geometry used with G/Tech 10, the relational Geometry is also supported:

Connectivity Search and Trace Info– In FRAMME and G/Technology, connectivity is maintained in a separate connectivity table and really has no relationship to the actual graphics other than the nodes associated with the graphics and connectivity info.  Therefore, a variety of connectivity problems can arise where the data looks graphically correct, but the connectivity is not and tracing will not operate correctly.   Two tools are provided to assist.   Originally the Connectivity Search was provided to allow you to graphically illustrate all features connected to the selected feature and then allows you to step out one feature in all directions as many times as you want.   This tool can show a node view and a feature view to easily determine how the features are actually connected in the connectivity table. 

Recently, a new feature called Trace Info has been added that will let you Trace from the selected feature step by step in a particular direction.  If there is a decision in the trace direction, you can choose the path to take.   This tool shows both a tabular view of the connectivity information (with color coded nodes) and a corresponding geometry view (showing the geometry of the actual feature) which makes identifying problems much easier.   You also have the choice of a Feature View and a Node View.

Ownership Search – Like the Connectivity Search, the Ownership Search starts at the connected feature and traces the ownership of the related features as far as they will go and draws a diagram of the relationships. 

Additional GTMetaExp Features

GTMetaExp provides such a large array of features it is sometime hard to categorize them.  Here are a couple of useful features that are also available:  Style Usage Reporting and GTViewer Query Builder.

Style Usage Reporting – Select a Legend, and Feature (or even a specific component), then get a summary of all styles used.   Each feature in the dataset will be evaluated against the Style and Label Rules while statistics are kept and then used to produce a summary report.  These reports can identify anomalies that are hard to find otherwise.

GTViewer Query Builder – prior to the Query Builder add-on app for GTViewer, a query builder was added to GTMetaExp to build GTViewer queries with the G/Tech metadata.   While not as sophisticated as the Add-On app, this feature may still be useful.

GTMetaExp for Bentley Data

There is also a GTMetaExp for Bentley data that will read the metadata .XML files and Oracle Spatial.   While this application is significantly less feature-filled than the G/Tech version, it does provide a great deal of information for Features, Criteria, and Domains.   Also, the Find Feature functionality is available and includes Geometry Info and Symbology Info (which are very similar to those in the G/Tech version).

Oracle Spatial Loader

After the addition of Object Geometry support to the GTech Loader and GTMetaExp, a new Loader for Oracle Spatial data was created from some of the already developed components.    The Oracle Spatial loader is a scaled down version of the GTech Loader (no metadata), but it has the ability to define styles based on expressions of feature attributes, and it also provides a Query feature that is similar to the Find Feature in GTMetaExp.   If you have Oracle Spatial data that you need to use with GTViewer, this application is an alternative to FME.


The product timeline for GTMetaExp, GTech Loader, and the Oracle Spatial Loader and the related projects are shown below:

       2000 - GTViewer
       2002 – G/Tech Loader - Gen 1
       2004 – G/Tech Loader - Gen 2
       2007 – G/Tech Loader - Gen 3
       2008 – GTMetaExp ver 1 (Relational Geo)
       2010 – Oracle Spatial Loader
       2011 – G/Tech Loader - Gen 4
       2011 – GTMetaExp ver 2 (Object Geo)
       2012 – GTBentleyMeta Explorer

No comments: