Data Collection/Migration Components

Data collection/migration components are used to collect data from various sources and ingest the data into a Historian server (or onto cloud). This topic provides a list of these components.
Data Collectors
Data collectors gather data from a data source based on a schedule or an event, process it, and forward it to the Historian server or a web socket for archiving. The following collector functions are common across all types of collectors (except the File collector):
  • Automatically discovering available tags from a data source and presenting them to Historian Administrator.
  • Options to send data to an on-premises Historian server or to cloud through a web socket connection
  • Performing a first level of data compression (collector compression).
  • Responding to control requests, such as requests to pause or resume collection.
  • Maintaining a local cache of tag information to sustain collection while the server connection is down.
  • Buffering data during loss of connection to the server and forwarding it to the server when the connection is restored.
  • Optionally, automatically adjusting timestamps for synchronizing collector and archiver timestamps.
  • Supporting the timestamp of both the collector and the device, as applicable.
  • Scheduling data polling for a polled collection.

For mission-critical data collection, you can set up redundant collectors. Historian includes a mirroring option for high availability and load balancing, so the data is available all the time.

For more information, refer to About Historian Data Collectors.

Historian File collector
File collectors import .csv and .xml files into Historian. These files can contain data, alarms, tag names, or other configuration information, and messages that you can import with a File collector.

For more information, refer to About Historian Data Collectors.

Historian Extract, Transform, and Load (ETL) Tools
Transferring data from one Historian server to another is typically performed by the data collectors. These tools provide a connected streaming data transfer mechanism (except the calculation and file transfer collectors). In a system where a steady network connection is not possible or not cost-effective, a periodic file-oriented data transfer is preferred. The Historian ETL tools consist of a comprehensive set of file-oriented data extraction, transfer, and loading tools.

For more information, refer to Overview of the Historian Extract, Transform, and Load (ETL) Tools.

Historian Migration Tools
Migration tools are used to migrate your existing classic Historian configuration and data and your iFIX Alarms and Events collector data to the Historian server. Tags, collection rates, and deadbands for tags configured in Classic Historian can be transferred into Historian by the migration tools.

For more information, refer to Migrating Classic Historian Data to Your Historian Database and Migrating Advance Historian DataMigrating Advanced and Classic Historian Data.

Collector Toolkit
The Collector Toolkit is used to develop a customized collector. To do so, you can use the Collector Toolkit to write programs that integrate with Historian and leverage the same configuration tools, redundancy schemes, and health monitoring as the Historian data collectors. It collects data and messages from a data source and writes them to a data archiver. Each deployment of a collector developed on the Collector Toolkit consumes a CAL.

For more information, refer to Overview