Working with Python Expression Tags

Important: You do not have the latest version of Historian! You are missing out on the newest capabilities and enhanced security. For information on all the latest features, see the Historian product page. For more information on upgrades, contact your GE Digital sales agent or e-mail GE Digital Sales Support. For the most up-to-date documentation, go here.

Python Expression Tags in Historian

A Python® expression is typically a single line of Python code which, when executed, evaluates to a single value. It can also be thought of as the right-hand side of a Python assignment operation.

Python Expression Tags are used in cases where you do not want to store a raw data value, but wish to store only derived or calculated values. Historian allows raw data to be pre-processed with Python expressions during collection, so that the data collected for the expression tags contain these derived values.

See www.python.org/ for complete Python documentation.

Note: This functionality is available only for the Enterprise edition of Historian.

To use a Python Expression Tag in Historian, you must configure a tag as a Python Expression Tag using either Historian File Collector or Historian Excel Add-in. You can then use the Python expression on this tag to pre-process raw data before the result is inserted into Historian.

The following collectors support Python Expression Tags:
  • Simulation Data Collector
  • iFix Data Collector
  • OPC Data Collector
  • OSI PI Collector
  • Windows Performance Collector
  • Collector toolkit

Supported Python Modules

This table lists the available Python modules. Within the approved modules below, certain Python classes, functions, and keywords may not be available for use, for security reasons. For example, the exec function cannot be used.
Note: Python modules not listed in the table are not supported.
Standard Python Modules Documentation Needs Importing
builtins Standard Python documentation No
datetime Standard Python documentation Yes
math Standard Python documentation Yes
statistics Standard Python documentation Yes
Module part of Historian install

uom (unit of measure conversion module)

 Unit of Measure Python Module.pdf

(This document can also be found in the help directory for your Historian installation.)

Yes

Constructing and Adding Python Expression Tags

To configure and use the Python Expression Tag in Historian:
  1. Construct a Python Expression.
  2. Construct the JSON configuration.
  3. Add the Python Expression Tag to Historian.
    For examples, see the Python Expression Tag Examples.

Constructing a Python Expression

  1. Open a text editor.
  2. Construct an expression in Python that takes raw data and produces a calculated value you want to store in Historian.
    • The raw data in your expression is exposed in the Python script as an object with the following properties: value, is_quality_good, is_quality_bad, timestamp
    • You can give this object any name you want..
  3. Ensure that the expression you create is a valid Python expression (that is, it evaluates to a single value).
    For example, we call our object temp and we use the value property. We create a simple, valid Python expression, taking raw data (temp.value) and create a calculated value.
    temp.value + math.pow(10,temp.value/70)

    To use this example, temp.value must be defined and the Python math module must be imported.

Constructing the JSON Configuration

Construct a JSON configuration object that contains the Python expression in the same text editor. This allows you to specify Python modules that you require and to link the variables (parameters) used in the expression to a raw data source.

It is easier to edit your JSON in the Historian Excel Add-In, which you will use later to add the tag to Historian, see Adding a Python Expression Tag to Historian.

The JSON needs to be “minified” and it needs to conform to a particular structure. Example of JSON containing a Python Expression in "Minified" format

{"imports":["math"],"script":"temp.value + 
math.pow(10,temp.value/70)","parameters":[{"name":"temp","source":{"add
Here is an example of JSON structure in “Beautified” format. The highlighted entries are placeholders.
{
"imports":["<imported_module0>", "<imported_module1>", 
"<imported_module2>",...], "script":"<python_expression>",
"parameters":[
{
"name":"<variable_name>", "source":{"address":"<source_address>",
"dataType":"<datatype_of_parameter_value_field>"}
},...
]
}

After "imports", list the Python modules you need to import for your Python expression. See Supported Python Modules.

After "script", enter the python expression that you created in Constructing a Python Expression.

Provide values for the following parameters:
Parameter Description
name The variable name you gave to the object representing the raw data which will be pre-processed by the expression you created. This variable is exposed in the Python script as an object with the following properties: value, is_quality_good, is_quality_bad, timestamp.
address The source address used by the collector to access the raw data. This parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags.
dataType The datatype of the \value field for the variable representing the raw data. This allows the Python expression to know the data type on which it is operating.

The datatype options are: SingleFloat, DoubleFloat, SingleInteger, DoubleInteger, QuadInteger, UnsignedSingleInteger, UnsignedDoubleInteger, UnsignedQuadInteger, FixedString, VariableString, Byte, Boolean. These are the same Historian data types that are supported by the File Collector.

Example of JSON containing a Python Expression in “Beautified” format

This example uses the Simulation Collector. Your collector might use a different source address.
{
 "imports":["math"],
 "script":"temp.value+math.pow(10,temp.value/70)",
 "parameters":[{"name":"temp","source":
{"address":"Simulation00001","dataType":"SingleFloat" } } ]
}

It is important to check that your JSON is valid, since no validation will be performed on the JSON at tag creation.

Minified JSON

Once you have constructed this JSON, you need to format it as a “minified” string, so that it can be processed in later steps. Minified JSON has no newline characters or comments. There are tools which can help you minify JSON.

For our example, the minified JSON would look like this:
{"imports":["math"],"script":"temp.value + math.pow(10,temp.value/70)","parameters":
[{"name":"temp","source":{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Pay attention to escape characters in your JSON. If your JSON contains a \ character, you need to escape it. So, \ becomes \\ (since \ is used to escape another \).

Adding a Python Expression Tag to Historian

You can now add a Python Expression Tag to Historian. Ensure the following for your Python Expression Tag:
  • CalcType is set to "PythonExpr".
  • SourceAddress contains the JSON configuration.

Adding the Python Expression Tag can be done using Historian File Collector or Historian Excel Add-in in the same way that you would add a regular tag.

For more information on adding tags, refer to Historian File Collector or Historian Excel Add-in.
  • File Collector (especially the CSV File Formats and XML File Formats sections)

    If you add your tag to the Historian via the File Collector and using the CSV format, you must enclose the JSON in quotation marks (") to satisfy CSV requirements for a column value containing commas (,).

    This means that you will also need to escape any quotation marks (") in the JSON. That is, " becomes "" (as " is used to escape another ").

  • Using the Historian Excel Add-in

    The Historian Excel Add-In has the advantage of being easiest to use for editing your JSON.

Example of Adding a Tag Using the File Collector

For this example, we choose to add a Python Expression tag to the Historian using the File Collector to import a CSV file.

The file contents look like this:
[Tag]
Tagname,CollectorName,CalcType,SourceAddress,DataType,Description
ExampleTag,SimulationCollector,PythonExpr,
"{""imports"":[""math""],""script"":""temp.value + 
math.pow(10,temp.value/70)"",""parameters"":[{""name"":""temp"",""source"":{""address"":""Simulation00001"",""dataType"":""SingleFloat""}}]}",
SingleFloat,Python Expression Tag example
Note the following:
  • The CalcType header is included and set to PythonExpr.
  • The Source Address is set to the minified JSON created in the previous step.
  • The CollectorName is set to SimulationCollector, which is a Simulation Collector. This collector is on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
  • The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.

    For more information, refer to File Collector > CSV file format.

Viewing the Python Expression Tag

When viewing a Python Expression Tag using the Historian Administrator, note that:
  • The Source Address field contains the full applicable JSON configuration, which includes an indication of the source address. Changing this is not recommended.
  • The Browse button for configuring the Source Address is disabled for Python Expression Tags.

Python Expression Tag Examples

This section provides examples for using Python® Expression Tags.

Example Using No Python Modules or Functions

Problem Definition

In this example, we want to perform gross error detection on a signal "Signal" and clip the values to a range between 0 and 600.

Expression

To solve this, we create the following Python expression. 

0 if Signal.value<0 else (600 if Signal.value>600 else Signal.value)
Python Datatype Name Description
Expression Inputs SingleFloat Signal Represents the signal value.
Expression Result SingleFloat Not Applicable Represents the resulting expression value, with extreme outliers clipped.

Python Modules to Import for this Expression

None

(Only modules contained on the list of supported modules are available for this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON. For more information, refer to Constructing the JSON Configuration.
"script":"0 if Signal.value<0 else (600 if Signal.value>600 else Signal.value)", "parameters":[
{
"name":" Signal",
"source":{"address":"Simulation00001", "dataType":"SingleFloat"}
}
]
}

Note that the address parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags.

In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:
{"script":"0 if Signal.value < 0 else (600 if Signal.value > 600 else Signal.value)","parameters":
[{"name":"Signal","source":{"address":"Simulation00001",dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the File Collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:
[Tag]
Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionGEDSignalTag,SimulationCollector,PythonExpr,
"{""script"":""0 if Signal.value < 0 else (600 if Signal.value > 600 else Signal.value)"",""parameters"":
[{""name"":""Signal"",""source"":{""address"":""Simulation00001"",""dataType"":""SingleFloat""}}]}",
SingleFloat,Python Expression Tag example
Note the following:
  • The CalcType header is included and set to PythonExpr.
  • The Source Address is set to the minified JSON created in the previous step.
  • The CollectorName is set to SimulationCollector, which is a Simulation Collector. Your collector might be called by a different name.
  • The quotation marks within the JSON string are escaped with other quotation marks.

We then import the file, following the instructions specified in File Collector.

Example Using a Built-In Python Function

Problem Definition

In this example, we want to calculate the maximum of two temperature values to be collected.

Expression

To solve this, we create the following Python expression:
max(ThermocoupleA.value, ThermocoupleB.value)
Python Datatype Name Description
Expression Inputs SingleFloat ThermocoupleA Represents a temperature value.
  SingleFloat ThermocoupleB Represents a temperature value.
Expression Result SingleFloat Not Applicable Represents the maximum of the two given temperature values

Python Modules to Import for this Expression

None. A built-in Python module from the Python Standard Library is used.

(Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON:
{
 "script":"max(ThermocoupleA.value,ThermocoupleB.value)",
 "parameters":[
  {   
  "name":"ThermocoupleA",
  "source":{"address":"Simulation00001","dataType":"SingleFloat"}
  },
  {   
  "name":"ThermocoupleB",
  "source":{"address":"Simulation00002","dataType":"SingleFloat"}
  }
 ]
}

Note that the address parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:
{"script":"max(ThermocoupleA.value,ThermocoupleB.value)","parameters":[{"name":"ThermocoupleA","source":
{"address":"Simulation00001","dataType":"SingleFloat"}},{"name":"ThermocoupleB","source":
{"address":"Simulation00002","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the Historian Excel Add-In. (We could also have added the tag by using via the File Collector to import a CSV file. )

Ensure the following for your Python Expression Tag:
  • The CalcType is set to PythonExpr.

  • The SourceAddress contains the JSON configuration.

  • The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
  • The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.
Otherwise, the tag is added in exactly the same way as for a regular tag.

Example Using a Python Standard Library Module

Problem Definition

In this example we want to calculate a result based on a specific time range for an expression input. We set a supply voltage to zero within prescribed time ranges.

Expression

To solve this, we create the following Python expression:
0 if (SupplyVoltage.timestamp.astimezone().time() &gt;= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() &lt;= datetime.time(20, 30)) else 
SupplyVoltage.value
Python Datatype Name Description
Expression Inputs SingleFloat SupplyVoltage Represents the value we wish to transform.
Expression Result datetime Not Applicable Represents the resulting supply voltage, set to zero in the prescribed time ranges.

Python Modules to Import for this Expression

datetime module. This module is shipped with Historian.

Constructing the JSON

Using the created expression, we construct the following JSON:
{
 "imports":["datetime"],
 "script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[
  {   
  "name":" SupplyVoltage",
  "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
  }
 ]
}

Note that the address parameter is stipulated in the context of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:
{"imports":["datetime"],"script":"0 if (SupplyVoltage.timestamp.astimezone().time() >= datetime.time(18) and 
SupplyVoltage.timestamp.astimezone().time() <= datetime.time(20, 30)) else 
SupplyVoltage.value","parameters":[{"name":"SupplyVoltage","source":
{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the Historian Excel Add-In. (We could also have added the tag via the File Collector to import a CSV file.)

Ensure the following for your Python Expression Tag:
  • The CalcType is set to PythonExpr.

  • The SourceAddress contains the JSON configuration.

  • The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
Otherwise, the tag is added in exactly the same way as for a regular tag.

Example Using a Historian Python Module

Problem Definition

In this example we want to convert a temperature value from Fahrenheit to Celsius.

Expression

To solve this, we create the following Python expression:
uom.to_Celsius(Thermocouple.value, uom.Temperature.Fahrenheit)
Python Datatype Name Description
Expression Inputs SingleFloat Thermocouple Represents the temperature value in degrees Fahrenheit
Expression Result SingleFloat Not Applicable Represents the temperature value in degrees Celsius

Python Modules to Import for this Expression

uom module. This Python module is shipped with Historian.

(Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON:
{
 "imports":["uom"],
 "script":"uom.to_Celsius(Thermocouple.value, uom.Temperature.Fahrenheit)",
 "parameters":[
  {   
  "name":" Thermocouple",
  "source":{"address":"Simulation00001", "dataType":"SingleFloat"}
  }
 ]
}

Note that the address parameter is stipulated in the context of the chosen collector supporting Python Expression Tags. In this example, we have used the Simulation Collector. Your collector might use a different source address.

We then transform the above into a minified JSON string:
{"imports":["uom"],"script":"uom.to_Celsius(Thermocouple.value,uom.Temperature.Fahrenheit)","parameters":
[{"name":"Thermocouple","source":{"address":"Simulation00001","dataType":"SingleFloat"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the File Collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:
[Tag]
Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionConvertedTempTag,SimulationCollector,PythonExpr,
"{""imports"":[""uom""],""script"":""uom.to_Celsius(Thermocouple.value,uom.Temperature.Fahrenheit)"",
""parameters"":[{""name"":""Thermocouple"",""source"":{""address"":""Simulation00001"",""dataType"":
""SingleFloat""}}]}",
SingleFloat,Python Expression Tag example
Note the following: The CalcType header is included and set to PythonExpr.
Note the following:
  • The CalcType header is included and is set to PythonExpr.

  • The SourceAddress contains the JSON configuration.

  • The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
  • The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.

    For more information, see File Collector > CSV File Formats.

We then import the file, following the instructions in the File Collector ebook.

Example using Array/Table Lookup

Problem Definition

In this example we want to translate a string representing order of magnitude into a corresponding numerical value using array/table lookup.

This example will be explained by means of a hypothetical collector called PlantSensorCollector that is a Python Expression enabled collector. The collector collects a source tag with address TemperatureSetpoint of type VariableString, having values 'Low', 'Medium', and 'High'.

Expression

To solve this, we create the following Python expression:
{'Low':100, 'Medium':400, 'High':800}.get(Setpoint.value, 0)
Python Datatype Name Description
Expression Inputs VariableString Setpoint Represents the given ordinal string value we wish to transform.
Expression Result SingleFloat Not Applicable Represents the numerical value corresponding to the ordinal string input.

Python Modules to Import for this Expression

None

(Only modules contained on the list of supported modules are available to this expression.)

Constructing the JSON

Using the created expression, we construct the following JSON:
{
 "script":"{'Low':100,'Medium':400,'High':800}.get(Setpoint.value, 0)",
 "parameters":[
  {   
  "name":" Setpoint",
  "source":{"address":"TemperatureSetpoint", "dataType":"VariableString"}
  }
 ]
}
We then transform the above into a minified JSON string:
{"script":"{'Low':100,'Medium':400,'High':800}.get(Setpoint.value,0)","parameters":
[{"name":"Setpoint","source":{"address":"TemperatureSetpoint","dataType":"VariableString"}}]}

Adding the Expression Tag to Historian

For this example, we choose to add a Python Expression tag to the Historian using the File Collector to import a CSV file. (We could also have added the tag via the Historian Excel Add-In.)

The file contents look like this:
[Tag]Tagname,CollectorName,CalcType,SourceAddress,DataType,DescriptionNumericalTagDerivedFromOrdinalVal,PlantSensorCollector,PythonExpr,
"{""script"":""{'Low':100,'Medium':400,'High':800}.get(Setpoint.value,0)"",""parameters"":
[{""name"":""Setpoint"",""source"":{""address"":""TemperatureSetpoint"",""dataType"":""VariableString""}}]}",
VariableString, Python Expression Tag example
Note the following:
  • The CalcType is set to PythonExpr.

  • The SourceAddress contains the JSON configuration.

  • The CollectorName is set to the name of the chosen collector, which must be on the list of collectors supporting Python Expression Tags. Your collector might be called by a different name.
  • The quotation marks within the JSON string are escaped with other quotation marks in the CSV file.

    For more information, see File Collector > CSV File Formats.

We then import the file, following the instructions specified in File Collector.