Tuesday, February 26, 2008

Incrementally Updating GTViewer Data – The Next Generation

For more than two years, GTData has provided two utilities for performing incremental updates to GTViewer data: GTUpdateData and GTUpdateGtg. These utilities integrate graphic and tabular changes into a GTViewer dataset. While these utilities can be used to provide a substantial reduction in the time required to update a GTViewer dataset and in the amount of data that must be distributed to the clients, they are just two pieces of a much bigger puzzle. GTI is currently testing its second generation Increment Update Solution. While the end result is the same, this new approach provides a much more sophisticated set of tools and further reduces the client-side processing required to apply an incremental update .

Instead of providing a couple of utilities to perform the heavy-lifting, the new Incremental Update utilities work together forming a complete solution to the incremental data update process. One of the new utilities is the Incremental Update Manager (called GTIncUpdMgr) which controls the workflow aspects of the incremental update process. This tool is responsible for:
  • Compiling and compressing the update file that will be distributed to the clients

  • Managing the series information in the update file to ensure that a client can only apply incremental updates in the proper sequence

  • Decompressing the update file on the client and applying the incremental changes to the client’s GTViewer dataset

While the Update Manager is not required to perform an incremental update, it streamlines the process and provides assurances that update files are properly created, and that a client cannot accidentally skip intermediate updates which would likely corrupt their local dataset. The Update Manager also orchestrates the process of applying all of the different update components to the appropriate GTViewer data (graphics, linkage, tabular, and query). The Update manager uses the rest of the new utilities to perform its job: GTUpdateGtg1, GTUpdateData, GTQuery, GTDelQueryRec, and GTAppendQuery.

The original GTUpdateGtg utility has been replaced with the new GTUpdateGtg1 which not only applies a graphics update to a GTViewer graphics file (.gtg) but also to the Spatial Index file (.gtn) and Linkage File (.key). Simultaneously updating the graphics, index, and linkage files prevents the client from having to regenerate the spatial index and linkage files from an updated graphics file which saves a considerable amount of time and processing on the client.

The GTUpdateData utility has been enhanced to update not only the GTViewer tabular file (data.txt), but also the tabular index file (.idx). The client no longer has to regenerate the tabular index file from an updated tabular data file. The GTUpdateData utility also creates some temporary files used to accelerate the Query update process.

The GTQuery utility has been updated to more easily generate incremental query files from the Incremental tabular files used by GTUpdateData. These incremental query files are then integrated into the existing query files (.qry) with the GTDelQueryRec and GTAppendQuery utilities. Updating a query file is significantly faster than regenerating a query especially if only a fraction of the records used by the query have changed since the last update. The previous incarnation of the Incremental Update utilities required that each query be regenerated from the updated tabular data file which could take some time to complete on the client.

While all of these additional utilities and steps may seem to complicate the incremental update process, this second generation of the Incremental Update Process is actually much simpler to implement and use because the Update Manager handles all of the details.

Incremental Updates may not be for everyone, but if your GIS can support the creation of the incremental update files (by knowing which features have changed since the last update), then the next generation of the Incremental Update Solution maybe an option for you.

No comments: