Work with Data Archives

Configure Data Archives

About Data Archives

Historian archives are stored data files, each of which contains data gathered from all data sources during a specific period of time. There are two types of archive files in an Historian archives directory:
  • Machinename_Config.ihc the single .IHC file contains information about the archiver, tag configuration, and collector configuration.
  • Machinename_ArchiveXXX.iha archive data files where x is a number indicating the place of the file in a time-based sequence.
Since archived data files can be quite large, adjust system parameters carefully so that you limit data collection to meaningful data only and so that you minimize the required size of system storage. This chapter describes techniques you can use in your application to accomplish these goals.
Note: You must have a minimum of 10 GB free space available for the Data Archiver to start.
Historian now supports a maximum archive size of 256 GB per archive. When you start Historian, it may take a longer time to start an archiver depending on the number of archives online, number of tags, and number of connections.
Note: The limit for the number of LUNs is 100.

Archive Creation

Archive files are created to store data as the Historian server receives it until they reach their duration limit. When the limit is reached, a new archive is created and the data is loaded into that archive. These archives can be created based on number of days or hours. You can observe the archives on the left side of the Data Store page, under Archives, with the name of the archive and start time for each archive
Note: As of Historian 7.0, Historian archives are time-based only. Historian will asynchronously create a new empty archive when data starts loading into an existing archive. If the option to automatically create archives is not enabled, however, you must open a new archive manually.
To create archives based on days:
  1. Open the Data Stores page and then select Edit. The Archive Configuration page appears.
  2. Select Configuration.
  3. In the Archive Duration field, select the Days option from the drop-down list.
  4. Enter the number of days for which you want to create archives.
  5. Select Update.
Setting Days to 1 means that a new archive will be created every day starting from the time your first archive is created. The next archive is created after one day (24 hours) from the time the first archive was created.
To create archives based on hours:
  1. Open the Data Stores page, and then select Edit. The Archive Configuration page appears.
  2. Select Configuration.
  3. In the Archive Duration field, select the Hours option from the drop-down list.
  4. Enter the number of hours for which you want to create archives.
  5. Select Update.
Setting Hours to 1 means that a new archive will be created after every hour starting from the time your first archive has been created.

The Archive Configuration Screen

To access the Archive Configuration page, select Edit on the Data Stores page.

The Archive Configuration page lets you read and modify the parameters of archives and data stores. In this page, you can see the list of all archives of the selected data store.

To examine a particular archive, select the archive and then select Edit. The details of the archive are displayed in the Archive Details section.

  • Action Buttons
  • Statistics Section
  • Archives Section
  • Archive Details Section

Points To Remember

  • You may need to add an archive when the current archive is almost full and you have not enabled automatic creation of archives.
  • You may need to restore an archive when you start up after an unplanned shutdown or when you need to retrieve data from an old, inactive archive.
  • You may need to back up an archive before a planned Historian software product upgrade.
  • You may need to manually resynchronize archives when the archives in the mirrored environment are not synchronized.

Action Buttons

Select a button to perform the action indicated by the name. The following table describes these buttons.

ButtonAction
Add Archive Select this button to add a new archive to the data store.
Remove Archive Remove an archive. First select an archive name to select it, and then select Remove. Selecting OK removes the archive file from the list of archives for the system, and places it in the \Archives\Offline directory. This does not delete the archive file from the system. An archive must be closed before it can be removed.
Close Archive Manually close the current archive. An archive must be closed before it can be removed.
Backup Archive Back up a selected archive. Verify the file name and path and then select OK to save the file.
Restore Archive Restores an archive from backup.
UpdateApply all parameter changes that you have made in this page. If you want to cancel changes and return to the original values or settings, open a different page and then return to the Archive Configuration page.
EditSelect this to edit the Archive configuration details.

Statistics Section

Indicates the current status of the collector.
  • Running indicates that the collector is operating.
  • Stopped indicates that it is in pause mode and not collecting data.
  • Unknown indicates that status information about the collector is unavailable at present, perhaps as a result of a lost connection between collector and server.
Table 1. Configuration Tab
FieldDescription
Default Archive PathThe path name that will be used for any newly created archives. If you change the path, the change takes effect the next time a new archive is created.
Note: Do not use a period in the default archive path field. If a period is present in the default archive path, you will not be able to specify a default archive name.
Default Backup Path The location to which the backup file will be saved.
Archive Duration (Days/Hours)Specifies the duration of a newly created archive in days or hours. A new archive will be created after the selected number of days or hours.
Note: When the Archive Duration property is changed in a mirrored environment, the changes will take effect only after a time gap of 15 minutes.
Data is Read-only After (Hours) The number of hours, prior to now, for which data can be stored in a read/write archive. After the time expires, that portion of the archive file is automatically made read-only. Incoming data values with timestamps prior to this time are rejected.

A single archive file, therefore, may contain a read-only section, another read-write section containing recently written data, and unused free space.

Note: A read-only archive file cannot be moved using Windows Explorer. To move a read-only archive file, select the file and select the Remove button on the Details section of the Archive Maintenance page. The Archiver then releases its locks, which permits you to move the file at will.
Base Archive NameA prefix that is automatically added to the file name of all created archives. To change the prefix, enter a new text string and select Update.
Free Space Required (MB) Indicates the remaining disk space required after a new archive is created. If the available space is less than the requirement, a new archive is not created. The default is 5000 MB.

The Free Space Required field does not apply to alarms and events archives. The alarms and events archiver will continue writing to the alarms and events archive until the drive is full. If this occurs, the alarms and events archiver will buffer incoming alarms and events data until the drive has free space. An error will also be written to the Historian message log.

Automatically Create Archives (Enable/Disable) Select the appropriate button to enable or disable this function. When enabled, the server automatically starts a new archive in the default path directory whenever the current archive fills up. If disabled, no new data will be written to the archives once the default size has been reached.
Note: To create multiple archives at the same time, Automatically Create Archives must be Disabled.
Overwrite Old Archives (Enable/Disable)Select the appropriate button to enable or disable this function. When enabled, the system replaces the oldest archived data with new data when the default size has been reached.
  • To create multiple archives at the same time, Overwrite Old Archives must be Disabled.
  • If you enable both Automatically Create Archives and Overwrite Old Archives, then you must set ihArchiveFreeSpaceHardLimit to TRUE using APIs.
CAUTION: Since this action deletes historical data, exercise caution in using this feature. Be sure that you have a backup of the archive so that you can restore it later.
SCADA Buffer Duration (Days)Indicates the maximum number of days the trend data can be stored. The maximum number of days is 200 days.

This field applies only to SCADA buffer data stores.

Use Caching (Enabled/Disabled) When reading data from the archiver, some data is saved in the system memory and retrieved using caching. This results in faster retrieval as the data is already stored in the buffer. Enable the Use Caching option to retrieve data faster.

This option is not available for SCADA buffer data stores.

Generate Message on Data Update (Enabled/Disabled)If this option is enabled, an audit log entry will be made any time the value of a previously archived data point in the Historian archive is overwritten. This log entry will contain both the original and new values.

To create multiple archives at the same time, Generate Message on Data Update must be Disabled.

This option is not available for SCADA buffer data stores.

Store OPC Quality (Enabled/Disabled) Stores the OPC data quality.

To create multiple archives at the same time, Store OPC Quality must be Disabled.

Stale PeriodSpecifies the time period after which tags are considered stale for this data store. The value is defined in days. Valid values are:
  • 0 (zero): This default value means that tags are never considered stale. This effectively disables stale tag management.
  • 7 days (1 week) to 36500 days (100 years)
Stale Period CheckSpecifies the frequency with which the staleness of the tag is checked. The value is defined in days. Valid values are 1 day (the default) to 30 days.

Archives Section

The Archives section displays the list of archives available with the selected data store. To edit an archive, select the archive and select the Edit button. The details of the archive are displayed in the Archive Details section.
Field Description
Name The name of the archive.
Start Time The time of the oldest sample in the archive.
End Time The time the archive is automatically or manually closed.

Archive Details Section

Archive Details section of the page lets you read and modify all archiving parameters for the Historian system.
Table 2. Archiving Parameters
FieldDescription
StatusThe current operating state of the archive: Active, Current, Empty.
  • Current: Archive is actively accepting data.
  • Active: Archive contains data but is not currently accepting data.
  • Empty: Archive was created but has never accepted data.
Start TimeThe time of the oldest sample in the archive.
End Time The time the archive is automatically or manually closed.
Last Backup OnThe date and time the last backup was performed on this archive.
Backup ByUser name (at time of login to Historian Administrator) of the person who performed the last backup of the archive.
File Location The path and name of the archive file.
File Size (MB)The size (in MB) of the archive file.
Note: Historian now supports a maximum Archive Size of 256 GB per archive.
File Attribute The attribute to set a closed archive to Read-only or Read/Write.
Note: If you plan to create multiple archives at the same time, then you must set File Attribute to Read/Write.

Calculate Required Archive Size

Historian will asynchronously create a new empty archive when data starts loading into an existing archive. Whenever the current archive becomes full, Historian will immediately serve data to a newly created archive. This significantly reduces archive creation and transition time. If the option to automatically create archives is not enabled, however, you must open a new archive manually. As of Historian 7.0, Historian archives are time-based only.
CAUTION: When the default size limit is reached, if automatic archive creation is disabled and you do not manually create a new archive, new data will not be written to the archives .
CAUTION: If the available disk space is less than the configured amount of free disk space, Historian cannot automatically create new archives.
If you enable the Overwrite Old Archives option, the system replaces the oldest archived data with new data. Since this action deletes historical data, exercise caution in using this feature. Be sure that you have a backup of the archive so that you can restore it later.

If you enable the Overwrite Old Archives option and if you want to retrieve time-based information, create an additional archive to overcome the early loss of data due to archive preparedness. For example, if you want to save 12 months of data into 12 archives, create 13 archives.

During archiver startup and every 60 seconds while the server is running, Historian checks to make sure that you have configured enough free disk space to save the archives, buffer files, and log files. If there is insufficient disk space, the Data Archiver shuts down and a message is logged into the log file. For each archive, you need approximately 1MB of archive space for every 1000 tags, for tag information.

By default, you can view the Historian archiver log file in Historian Data\LogFiles
[03/03/10 15:28:41.398] Insufficient space available in [d:\Historian\Archives\] 
[03/03/10 15:28:41.399] The server requires a minimum of [5000 MB] to continue 
[03/03/10 15:28:41.679] USER: DataArchiver TOPIC: ServiceControl MSG: DataArchiver(DataArchiver) Archiver s 
[03/03/10 15:28:41.807] DataArchiver Service Stopped. 
[03/03/10 15:28:41.809] [d:\Historian\LogFiles\DataArchiver-34.log] Closed.
Archive size is a function of the rate at which you archive data and the time period that you want the archive to cover. You may want the archive to cover a time period of perhaps, 30 days.
Factors that affect the rate at which you archive data are:
  • Number of tags a large number of tags increases the data rate.
  • Polling frequency of each tag a high polling frequency increases the data rate.
  • Compression settings disabling compression or setting narrow deadband parameters increases the data rate.
  • Data types choosing data types that increase the number of bytes per value increases the data rate.
The following is an example of a manual calculation of required archive size, using typical parameter values.
Table 3. Assumptions
Number of tags5000
Polling rate1 value/5 seconds
% Pass Compression5% (Pass Compression is the number of data values archived relative to the number of values read, expressed as percent.)
Bytes/value: 4
Duration: 1 month (30 days)

Calculation





The calculation shows that a file size of 500 MB is adequate for archiving one month of data for this application.
If you believe the calculated size is too large for your application, you can modify parameters as follows:
  • Decrease the polling frequency.
  • Increase compression deadband, reducing the pass percentage.
  • Reduce the number of tags.
  • Add more disk capacity to your computer.

Archive Size Calculator

An Archive Size Calculator tool is available to estimate archive size based on your input and estimates the archive size and collector compression based upon a tag that has already been configured. Log on to http://support.ge-ip.com/devsupport/ to download this and other GE Intelligent Platforms freeware product solutions.

Prepare for Multiple Archive Creation

If you plan to create multiple archives at the same time, set the following parameters. These parameters apply only when creating multiple archives at the same time.
  • In the Archive Details section, set File Attribute to Read/Write.
  • In the Configuration Tab:
    • set Automatically Create Archives to Disabled.
    • set Overwrite Old archives to Enabled.
    • set Store OPC Quality to Disabled.
    • set Data is Readonly After (Hours) to 1 month.
    • set Generate Message on Data Update to Disabled.

Before you begin creating multiple archives on a remote machine, ensure that you have enough hard disk space on that machine. The Allocate Space slider does not display a remote machine's hard disk space; the r;percentage of available disk space will be used message displayed by the Allocate Space slider will be inaccurate if it appears at all.

If you receive the error message Runtime error 330 Invalid Property Value while creating multiple archives on a remote machine, it is probably because you did not have enough hard disk space on that machine. When you select OK on the error message, Historian Administrator may disappear. You must now clean up the remote machine's hard disk space and restart Historian Administrator.

Adding One or More Archives

About this task

You may need to add an archive when the current archive is almost full and you have not enabled automatic creation of archives.
Note: Historian now supports a maximum archive size of 256 GB per archive. When the current archive is full, the system will write to the next archive in the sequence in which it was created. As of Historian 7.0, Historian archives are time-based only.

Procedure

  1. In the Archives section, select the icon.
    The Add New Archive(s) window appears.

  2. In the Archive Name field, enter the name of the archive. The archive name must be the same as the filename.
  3. In the File Location field, enter the path of the archive from a local drive or specify a UNC path.
  4. In the Archive Size (MB) field, enter the size of the file in MB that you want to create.
  5. Select OK.
  6. Select Cancel to stop the operation.
    If you cancel the operation, any archives already created during this operation will be deleted.

Back up Historian Archive Files

Back up your Historian archive files periodically to ensure your data is protected. Historian bundles alarms and events data with tag data in its backup files, and stores them as ZIP files. After an archive has been backed up, it can be stored to a shared network location, stored off-site, or written to physical media.

Note:

The .IHC file is automatically backed up when, and only when, you back up the "current" archive .IHA file. By default, the .IHC backup path is the same as the archives path. The .IHC uses the following naming convention: ComputerName_Config-Backup.ihc If the default backup path is different than the archives path, the .IHC file is copied to the backup folder with the standard .IHC naming convention ComputerName_Config.ihc.

If you back up an archive more than once, the backup tool will (by default) attempt to use the same name for the backup file and will detect that an archive with the same name already exists. Rename the backup archive file or move the original backup archive file from the target backup directory.

Backing up Archives using Historian

About this task
Best practice is to store archive backups in a different location than the archive files.
Procedure
  1. Open the Archive Configuration page.
  2. In the Archives section, select an existing archive.
  3. Select the Backup button.
    The Backing up Archive window appears.
  4. Enter the Archive Name.
  5. Save the backup file to the archive backup file location.
    A new Job Id is created and the details of the status are displayed in the Jobs Page.

Including Alarm Data in Archive Backups

About this task
When backing up your Historian archives, any alarms that have a life cycle that overlaps the data archive being backed up will be included. This means that an alarm with a long life cycle can be included up in multiple backups. For example, say the following alarm and archive dates were the following:
Alarm/Data ArchiveStart TimeEnd Time
Alarm1 09/02/2004 09/06/2004
Archive109/01/2004 09/03/2004
Archive2 09/03/2004 09/04/2004
Archive3 09/04/2004 09/06/2004
If any or all of these archives are backed up, Alarm1 will go into the backup for each one. When the archives are restored, Historian will analyze the included alarm data and, if the data is already in the Historian archive, is intelligent enough to know it already has the alarm.

Use the following procedure to change alarm timestamp checking.

Procedure
  1. From the Start menu, select Run and enter Regedit.
  2. Open the following key folder HKEY_LOCAL_MACHINE\SOFTWARE\Intellution, Inc.\iHistorian\Services\DataArchiver\
  3. Create a new DWORD called AlarmTimestampCheck and set its value to 1.
    Set AlarmTimestampCheck to 2 for slower timestamp checking. Set AlarmTimestampCheck to 0 to disable timestamp checking entirely.
  4. Select OK.
  5. Close the Registry Editor
  6. Open Historian Administrator.
  7. Restart the Data Archiver for the changes to take effect.

Backing up Archives Using Volume Shadow Copy Service

Historian can use the Microsoft Volume Shadow Copy Service to back up and restore large archive files reliably and in a short period of time without affecting the data collection. The Historian Data Archiver uses ihArchiverBackup.exe as the default backup system. If you want to back up your archiver files regularly, you can set the scheduler to back up your files automatically.

VSS provides fast volume capture of the state of a disk which is called a snapshot or shadow copy. When the snapshot is taken, disk writes are suspended for a brief period of time, typically on the order of milliseconds. After the snapshot, disk writes can resume, but the original state of the files are maintained by a difference file. The different file allows the state of the original file at the time of the snapshot to be reconstructed. This behavior allows files to be backed up while new data is being written to files.

If you are using ihArchiveBackup.exe before the upgrade, your backup will continue to work in the same or similar manner as it did before the upgrade. There is no change in the backup procedure and the Auto Recovery Backup Files option remains unchanged.
Note: You can use both ihArchiveBackup.exe or VSS for backup, however, VSS is a better choice for larger archives to reduce the load on the Data Archiver service.

The Volume Shadow Copy feature is provided by Windows Operating System, and the instructions to use backup and restore vary depending on the backup application that is used in the Windows operating system.

Historian supports using the Volume Shadow Copy Service in the following operating systems.
  • Microsoft Windows Server 2019 (64-bit)
  • Microsoft Windows Server 2016 (64-bit)
  • Microsoft Windows Server 2012 R2 (64-bit)
  • Microsoft Windows 10 IoT (32-bit or 64-bit)
  • Microsoft Windows 10 (32-bit or 64-bit)
  • Microsoft Windows 8.1 Professional (32-bit or 64-bit)
Microsoft uses a backup format called Virtual Hard Disk (VHD) to back up files. If you create an archive backup using Microsoft Volume Shadow Copy Service, you must first restore the archives files (that is, convert .bkf or .vhd into .iha) using the Windows Restore wizard, and then restore the archives (.iha) into Historian. For more information on restoring an archive (.iha) into Historian, refer to the Restoring an Archive topic.
In addition to using the Backup and Restore wizard, you can also use the command line utilities:

Restore or Resynchronize Historian Archive Files

Under certain circumstances, you may want to restore tag and alarms and events data to Historian. This may be after an unplanned shutdown, or you may need to retrieve data from an old, inactive archive. You can restore only time-based archives.
Warning: Never restore an archive to a production Historian server without a current archive already online.
CAUTION: Restoring an archive is a resource-intensive operation and should be scheduled for non-peak usage times.

If the Archive files in the mirror environment are not synchronized, manually synchronize the archive files from one node to another node by using the Resync Archive option.

Restoring Archives from Historian Backup Files

Before you begin
If it is not already present, copy the archive file to the default archive path. Leave the original backup file where it is. Archives that have been previously removed from Historian can be found in the \Archives\Offline directory. Archives on removable media must also be copied into the default archive path.
Procedure
  1. Open the Archive Configuration page.
  2. Select the Restore Archive icon.
    The Restore Archive window appears.
  3. In the Archive Name field, enter the name of the archive you want to restore.
  4. In the File Location field, enter the path name of the archive from a local drive or specify a UNC path.
  5. Verify that the file name and path are correct.
  6. In the Data Store field, select the name of the data store to load the archive file into.
    If Historian is unable to find the specified data store, the file will be loaded to the default data store.
  7. Select OK.
    The restored archive is moved to the \Archive directory and is made available for querying.

Resynchronizing Archives

About this task
Note: If one of the mirror node is crashed or is down, and you replaced it with a new machine with the same host name, ensure that you manually resync all the archive files.
Procedure
  1. Select the archive in the Archive Configuration page.
  2. Select the Resync Archive button.
    The Resync Archive window appears.
  3. Enter the Source Node and the Destination Node.
  4. Select OK.
    A Job ID is created for the action and the progress can be seen in the Jobs Page.