Data Loaders
Download the Data Loader Workbooks
Procedure
Import Data Loader Workbooks
About This Task
You can import custom Data Loader workbooks or localized workbooks into APM, so that they can be accessed on the Data Loaders page.
Procedure
Results
The workbook that you imported can be download on the Data Loaders page.
Import an Additional Equipment and Functional Location Workbook
- Access the Export page,
- In the Available Items section, in the Select metadata type drop-down, select DataLoaders Template.
The Available Items section is populated with the Data Loader catalog entries.
- Select the Equipment and Functional Location box, and then select . The Equipment and Functional Location Data Loader appears in the Selected Items section.
- In the File Name box, enter Equipment and Functional Location Export, and then select the File Type XML(.xml).
- Select Start Export.
The Data Loader workbook files are exported, and a zip file is downloaded containing three files:
- Equipment and Functional Location Export Files: A folder containing the default Equipment and Functional Location Data Loader workbook.
- Equipment and Functional Location Export.xml: An .xml containing the metadata for the catalog item.
- ExportLog: A log file detailing the export from APM.
- Open the Equipment and Functional Location Export Files folder, and place a copy of the workbook to be imported into the folder. In this example the workbook name is Equipment and Functional Location_DE.
- Ensure the template to be imported has a unique name different from the naming structure of the default template, and then delete the default template.
- Open the Equipment and Functional Location Export.xml file.
- In the DATALOADERTEMPLATE_ID and DATALOADERTEMPLATE_DESC parameters, replace Equipment and Functional Location with Equipment and Functional Location_DE.
- In the parameter DATALOADERTEMPLATE_FILE_NM, replace the value with Equipment and Functional Location_DE.xml.
- Save the file.
- Create a zip file with the Equipment and Functional Location Export files.
- Access the Import page.
- In the Import File box, select Choose File, and then navigate to the zip file Equipment and Functional Location Export.
In the Available Items section The Equipment and Functional Location entry appears.
- Select the The Equipment and Functional Locationbox, and then select .
- Select Start Import Button.
The new template is imported into APM, and is available to download on the Data Loaders page.
Test Data Loader Connection
Procedure
Results
The following table describes the connection status for data import jobs.
Connection Test | Description | If you get an Error... |
---|---|---|
APM Connect Web API Test | Test the connection between the APM web API and the APM Connect server. | Ensure the ports are open between the APM Server and the APM Connect server. |
APM Connect Configuration Test | Test the configuration of the APM Connect connection parameters configured in APM. |
Ensure the parameters configured in APM are correct to establish connection from APM and APM Connect. |
APM Fileshare Access Test | Tests the file share system is connected to the APM Server, so that the server can read and write to the configured file shares. |
Ensure the parameters configured in APM are correct to establish connection from APM, and the user has the necessary permissions to access the file share. |
APM Staging Database Test | Tests the APM Server is connected tot the APM Connect IR database. | Ensure that PostgresSQL is accepting the connection from APM . These connections are managed in the file pg_hba.conf. |
APM Connect Fileshare Access Test | Tests the file share system is connected to the APM Connect server, so that the server can read and write to the configured file shares. |
Ensure the parameters configured in APM are correct to establish connection from APM, and the user has the necessary permissions to access the file share. |
APM Connect Staging Database Test | Tests the connection between the APM Connect server and the IR database to ensure that files can read and written to the IR. | Ensure the RunDataLoaderRoute.cfg is configured correctly. |
Run a Data Import Job
Before You Begin
- The format matching the Excel workbook templates.
- A conforming zip file containing .csv files.
Procedure
Access the Details of an Import Job
About Populating Site Reference Data
The APM Family Data Loader can be used to populate the Site Reference on Equipment and Functional Location records in APM.
About This Task
The APM Family Data Loader populates the ENTY_KEY system field and the MI_SITE_KEY system field associated with the Site Reference value to be populated. On asset records, the Site Reference is stored in the MI_SITE_KEY field, a system field in APM. The APM Data Loader uses the Site Name (MI_SITE_NAME) to translate the value to the corresponding Site Key and populate the MI_SITE_KEY field; therefore, you do not need to know the key to be able to populate the site reference. This functionality is important because this value can change from one database to another.
Procedure
Considerations for Loading with .CSV Files
Typically when you load data into APM you will use the provided Data Loader workbooks, however, if you have a program that creates reports that generates .csv files, APM can load that data as long as you follow these guidelines.
The Data Import job can accept a .zip file that contains .csv files as input. The format of the .zip file is specific to each Data Loader and must contain .csv files that conform match these specific rules.
- There must be a .csv file for each worksheet defined in the workbook template for the Data Loader.
- The filename must match the worksheet name with the file extension .csv. For example, the .csv for the Equipment worksheet must be named Equipment.csv.
- The .zip must contain all the .csv files that match the worksheets expected by the data loader.
A .zip that meets all of these requirements can be used as input to the Data Import job.
About the Rounds Pro Data Loaders
You can use the Rounds Pro Data Loaders to import content into the Rounds Pro module in bulk.
The Rounds Pro Data Loaders are used to create records in the following families and relevant relationships between them:
- Rounds Pro Picklist Values Data Loader
- Rounds Pro Step Template Data Loader
- Rounds Pro Route Masters Data Loader
- Create Route
- Create Step
- Human Resource
- Teams
About the Rounds Data Loaders Requirements
This documentation assumes that your organization has completed the deployment of the Rounds Pro module. The Rounds Pro Data Loaders should be used only after the Rounds Pro module has been implemented.
As part of the security settings, the Security User performing the data load operation must be associated with either the MI Data Loader User or MI Data Loader Admin Security Role, and must also be associated with the MI Operator Rounds Administrator Security Group or a Security Role that is associated with this Security Group.
- All imported date values are in UTC.
- Functional Locations and/or Equipment have already been loaded into APM.
- Blank values are not copied to the entities. If there are blank values, those fields will be skipped.
- No customization (that is, rules changes) is done on the related families that override the baseline families.
About the Rounds Pro Data Loaders General Loading Strategy
This section describes any prerequisites to loading the data and the order in which the data will be loaded.
Data Import
When importing data using the Rounds Pro Data Loaders:
- The batch records is limited .xls spreadsheet file.
- You must use the data loader workbook while considering the rules described in Workbook Layout and Use.
- When importing huge number of records using the Rounds Pro Data Loader, you can split the records in the spreadsheet into batches and assign a batch ID for each batch. Records within a spreadsheet of a batch should be coded with the same Batch ID. Each list of batch records should be loaded using an individual spreadsheet. Using the batch ID, you can import the records to the Data Loaders.
- Below are the recommendations of the number of records per batch that can be loaded into Data Loaders using the Batch IDs:
- Picklists: 1000 records per batch (Same for both Reading and Action types)
- Step Templates: 1000 records per batch
- Steps: 1000 records per batch
- Conditions: 1000 records per batch
- Routes: 10 Routes (Not more than 1500 steps per route, no more than 5 user or team assignments per route)
Note: Small batches are less efficient compared to larger batches. There is overhead with each batch that can be reduced by reducing the number of batches.
Schedule and Date Fields
- The Rounds Pro Data Loaders require that Schedule values are expressed using the JSON format used in the APM database. Valid schedule expressions may be easily generated via the schedule setting feature on a schedule field in a Route or Checkpoint Task record in Record Manager, and then copying the resulting text value to the worksheet.
-
The Route Data Loaders use a non-standard date format. Data should be formatted as yyyy-mm-dd hh-mm-ss (for example, 2015-05-23 17-45-34).
-
All date values are assumed to be in UTC. The time zone of the user loading the data is not relevant.
About the Rounds Pro Data Loader Workbooks Layout and Use
Overview of Rounds Data Loader Workbooks
This section provides a high-level overview and explanation of how the data loader workbooks are constructed.
To import data using the Rounds Data Loaders, GE Vernova provides separate data loader workbooks for:
- Rounds Pro Picklist Values
- Rounds Pro Steps
- Rounds Pro Step Template
- Rounds Pro Step Conditionals
- Rounds Pro Route Masters
The workbooks support the baseline Rounds Pro functionality in APM. The workbooks must be used to perform the data load.
Each data loader workbook contains one worksheet for each node that will be populated in the data model.
Each worksheet in these data loader workbooks contain field values that must be mapped to the appropriate APM family or field combination. The individual sections in this document specify the source and target fields and relationships required between predecessors and successor.
The list of fields tabulated is not exhaustive.
Data Loader Batch Fields
The Rounds Pro Data Loaders workbooks include columns of data that are used to identify batches of related records. These values are not imported to a field in the APM database. Generally, these will be the first few columns in each worksheet. They each use a prefix that identifies the relevant record type being imported, whereas columns that will be imported have IDs that match the field IDs in APM, which generally use the prefix MI_.
Rounds Pro Picklist Values Data Loader Workbook
When importing data using the Rounds Pro Picklist Values Data Loader, the data loader will create one Picklist Value record for each row of the worksheet. If the import worksheet specifies a Picklist Value that is an exact match for an existing record, the record is not imported. Since the unique identification of a Picklist Value record includes all its fields, it is not possible to update existing Picklist Value records using the data loader.
On the worksheet, you will specify the picklist values that will be used in Step Templates and Steps.
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Picklist Batch ID | RTE_PICK_PCKLST_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Picklist ID | MI_RTE_PICK_PCKLST_ID_C | Character (50) | This field is required and must be unique. |
Value | SYCD_ID |
Character (50) | This field is required. |
Description | SYCD_DESC_TX | Character (255) | Provide a brief description. |
Sequence | SYCD_SEQ_NBR | Numeric | This field is required and must be unique within Picklist Values. |
Rounds Pro Step Template Data Loader Workbook
When importing data using the Rounds Pro Step Template Data Loader, the data loader will create one Step Template record for each row in the data loader workbook. If the import worksheet specifies a Template ID value that matches the Template ID of an existing Step Template record, the existing record will be updated. If the worksheet specifies values for Category or Picklist Value that do not already exist in the database for the specified type (numeric or character) of Step Reading the record will not be imported.
The Step Templates will be loaded into APM and will be filtered based on the designated site name in the Site Reference Name field.
Step Template Worksheet
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Template Batch ID | RTE_PICK_PCKLST_ID | Character (50) | This field is required for identification during the data load process. It should match the ID of the Template Batch to which the Step Template will be linked. Use <Dummy ID> if you want to import standalone records not linked to a Template Batch. The value in this field is not imported to APM. |
Template ID | MI_RTE_PICK_PCKLST_ID_C | Character (50) | This field is required and must be unique. The value in this field is not imported to APM. |
Step Type | MI_STP_TMP_STEP_TYPE_C |
Character (50) | This field is required. |
Reading Type | MI_ML_STP_TMP_READING_TYPE_C | Character (50) | This field is required. The valid values are:
|
Reading Picklist | MI_ML_STP_TMP_RDNG_PCKLST_C | Numeric | This field is required if selected as the Reading Type. |
Reading Picklist Value | MI_ML_STP_TMP_RDNG_VALS_C | Character (50) | This field is required if Reading Picklist is selected. |
Action Picklist | MI_ML_STP_TMP_ACTN_PCKLST_C | Character (50) | This field is required if it is selected. |
Action Picklist value | MI_ML_STP_TMP_ACTN_VALS_C | Character (50) | This field is required if Action Picklist is selected. |
Site Reference Name | MI_SITE_NAME | Character (50) | This field is required and must match an existing Site name. Note: If you want to set the site reference to be a Global Site Reference, enter the constant ‘*Global*’ in the MI_SITE_NAME column on the spreadsheet. |
Description | MI_STP_TMP_DESC_C | Character (255) | Please provide a brief description. |
Locked | MI_STP_TMP_IS_LOCK_L | Boolean | The default value for this field is False. |
Status | MI_STP_TMP_STAT_C | Character (50) | The default value for this field is Active. |
Is Shutdown Required | MI_STP_TMP_REQ_EQUIP_SHUT_L | Logical | The default value for this field is False. |
Unit of Measure | MI_STP_TMP_UOM_C | Character (50) | The field must be a valid value in the global Unit of Measure list. This field is disabled if the Reading Type field is Character. |
Frequency | MI_STP_TMP_FREQ_C | <need information> | |
Lower Level 1 Value | MI_ML_STP_TMP_LO_LVL1_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Lower Level 1 Message | MI_ML_STP_TMP_LO_LVL1_MSG_C | Character (50) | None |
Lower Level 2 Value | MI_ML_STP_TMP_LO_LVL2_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Lower Level 2 Message | MI_ML_STP_TMP_LO_LVL2_MSG_C | Character (50) | None |
Lower Level 3 Value | MI_ML_STP_TMP_LO_LVL3_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Lower Level 3 Message | MI_ML_STP_TMP_LO_LVL3_MSG_C | Character (50) | None |
Upper Level 1 Value | MI_ML_STP_TMP_UP_LVL1_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Upper Level 1 Message | MI_ML_STP_TMP_UP_LVL1_MSG_C | Character (50) | None |
Upper Level 2 Value | MI_ML_STP_TMP_UP_LVL2_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Upper Level 2 Message | MI_ML_STP_TMP_UP_LVL2_MSG_C | Character (50) | None |
Upper Level 3 Value | MI_ML_STP_TMP_UP_LVL3_ALRT_VAL | Character (50) | Must use one of the Picklist Values specified for the Step Reading. This field is disabled if the Reading Type field is Numeric. |
Upper Level 3 Message | MI_ML_STP_TMP_UP_LVL3_MSG_C | Character (50) | None |
Rounds Pro Step Conditionals Data Loader Workbook
When importing data using the Rounds Pro Step Conditionals Data Loader, the data loader will create one Step Conditional record for each row in the data loader workbook. If the import worksheet specifies a Template ID value that matches the Template ID of an existing Step Conditional record, the existing record will be updated.
Step Conditionals Worksheet (MI_STP)
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Conditional Step Batch ID | ROUNDS_STP_COND_BATCH_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Step ID | MI_STP_ID_C | Character (50) | A unique ID to identify a step. |
Related Asset ID | ROUNDS_STP_REL_ASSET_ID | Character (50) | If this value matches an existing Asset ID, the Conditional Step will be linked to this asset. |
Related Asset Family ID | ROUNDS_STP_REL_ASSET_FAMILYID | Character (50) | If this value matches an existing Asset Family ID, the Conditional Step will be linked to this asset. |
Step Name | MI_STP_COND_STP_NAME | Character (50) | The name of the Primary step. This field is required and must match an existing Step name. |
Operator | MI_STP_COND_OPERATOR | Character (50) | This field specifies the operators that determine if the reading value entered for the primary Step meet the condition specified. |
Conditions Field Value | MI_STP_COND_FLD_VAL | Character (50) | None |
Conditional Step Name | MI_STP_COND_STP_NAME_COND | Character (50) | The name of the Conditional step. |
Conditional Step Index | MI_STP_COND_STP_INDEX | Numeric (50) | None |
Immediate Parent Step | MI_STP_COND_STP_PARENT | Character (50) | This field contains the name of the current step in the route to which you want to add conditional steps. |
Rounds Pro Routes Data Loader Workbook
- Create Route Master
- Assign or unassign a Security User to the route
- Route Master: The Route Master data loader does the following:
- Creates a Route Master record or a route template.
- Associates or links existing steps created in the step dataloader or in the step designer.
- Assigns the route to a user.
- Assigns the route to a team.
- Validates that the users, teams, and steps exist already when trying to link these items together.
Note: Update a Route Master or delete a Route Master functionality is not available.The Route Master is created, associated to all the steps included and is assigned to all individuals and/or teams included.
The Route Master will be loaded into APM and will be filtered based on the designated site name in the Site Reference Name field.
Note: If you want to set the site reference, to be a Global Site Reference, enter the constant ‘*Global*’ in the MI_SITE_NAME column on the spreadsheet. - Steps: The data loader generates one step record for the existing steps in each row in the Step worksheet. Using this Step data loader, the steps are associated to the required route. The data loader creates one step record for each row in the Step worksheet. Step must exist before including it in the Step data loader.
The Next Date, Non-compliance Date, and the Non-compliance Next Check Date fields on the Step record will be populated automatically by existing rules based on the value in the Schedule column. If the Schedule is invalid, the record will not be created.
- Human Resource:
A relationship is created between the Route and the Human Resource record. Therefore, the Human Resource records must already exist before the data loader is used.
- Teams:
A relationship is created between the Route and the teams records. Therefore, the Teams records must already exist before the data loader is used.
Route Master Worksheet (MI_RTE_MST)
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Route Batch ID | RTE_MST_BATCH_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Route ID | MI_RTE_MST_ID_C | Character (50) | This field is required and must be unique.
Note: This field is not required and may not be populated on this worksheet when you only assign or un-assign the security users to Routes. |
Route Description | MI_RTE_MST_DESC_C | Character (255) | A description of the route. |
Schedule | MI_RTE_MST_SCHED_C | Character (50) | This option generates the route instances from an associated route. |
EAM Job Key | MI_RTE_MST_EAM_JOB_KEY_C | Character (50) | This field is used to link a Route Master to a given EAM Job identifier. |
Status | MI_RTE_MST_STATUS_F | Character (50) | This field indicates if the route has been added to the My Routes list. |
Is Schedule Active | MI_RTE_MST_SCHEDULE_ISACTIVE | Logical |
This field indicates if the Route Schedule is active. A schedule can be created for a route in the Plan Routes section, after you select the schedule option. |
Next Due Date | MI_RTE_MST_NEXT_DUE_DT | Numeric (50) | The date when the next instance of the route will be generated if the route master corresponding to the instance has an active schedule. |
Frequency | MI_RTE_MST_FREQ_N | Numeric (50) | This field displays a number for the duration at which a route schedule recurs. |
Frequency Interval | MI_RTE_MST_FREQ_INTVAL_C | Character (50) | The time unit to measure the duration at which the route schedule recurs. |
Floc Equipment Reference Number | MI_RTE_MST_FLOC_EQUI_REF_NUM_C | Character (50) | This field contains a unique alphanumeric number. |
Floc Equipment Status | MI_RTE_MST_FLOC_EQUI_STS_C | Character (50) | This field displays the equipment status, Out of Service, Online, or Available. |
Route Master Compliance Tracking Flag | MI_RTE_MST_CMPLNC_ISACTIVE | Button | Enable the button to activate compliance tracking for a schedule on a route. |
Route Master Compliance Horizontal Interval | MI_RTE_MST_CMPLNC_HRZN_INTRVL | Character (50) | This field contains the time units to measure the frequency based on which the route instance must be completed. |
Route Master Compliance Horizon Unit | MI_RTE_MST_CMPLNC_HRZN_UNIT | Numeric(50) | A number to specify the frequency in time units, such as days, weeks, and so on, based on which the route instance must be completed. |
Basis For Occurrence | MI_RTE_MST_BASIS_FOR_OCCUR_C | Character (50) | You can select EAM or Schedule. |
Steps Worksheet (MI_STP)
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Route Batch ID | RTE_MST_BATCH_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Route ID | MI_RTE_MST_ID_C | Character (50) | This field is required and must be unique.
Note: This field is not required and may not be populated on this worksheet when you only assign or un-assign the security users to Routes. |
Related Asset ID | MI_STP_REL_ASSET_ID_C | Character (50) | If this value matches an existing Equipment ID (for Equipment) or Functional Location (for Functional Locations) the Checkpoint will be linked to this asset. |
Step ID | MI_STP_ID_C | Character (50) | A unique ID to identify a step. |
Step Type | MI_STP_STEP_TYPE_C | Character (50) | The type of the step, whether a strategy step, template step, or a normal step. This is a required field. |
Step Sequence Number | MI_STP_SEQ_NUM | Character (50) | A unique number to identify the step sequence. |
Human Resource Worksheet (MI Human Resource)
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Route Batch ID | RTE_MST_BATCH_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Route ID | MI_RTE_MST_ID_C | Character (50) | This field is required and must be unique.
Note: This field is not required and may not be populated on this worksheet when you only assign or un-assign the security users to Routes. |
First Name | MI_HR_FIRST_NAME_CHR | Character (50) | The first name of the human resource personnel. |
Last Name | MI_HR_LAST_NAME_CHR | Character (50) | The last name of the human resource personnel. |
Teams Worksheet (MI_TEAMS)
Field Caption |
Field ID |
Data Type (Length) |
Comments |
---|---|---|---|
Route Batch ID | RTE_MST_BATCH_ID | Character (50) | This field is required for identification during the data load process and must be unique. The value in this field is not imported to APM. |
Route ID | MI_RTE_MST_ID_C | Character (50) | This field is required and must be unique.
Note: This field is not required and may not be populated on this worksheet when you only assign or un-assign the security users to Routes. |
Team Name | MI_TEAMS_TEAM_NAME_CHR | Character (50) | A unique ID to identify a team. |
About the Rounds Pro Data Loaders Load Verification
- Review the data loader log to identify any errors or warnings
- Update the data loader workbook to correct any errors, and then re-import
- Repeat until the import of each record is error free
To perform the load verification of the loaded data loaders, log in to the Rounds Pro as a Rounds Pro Administrator user.
On the left navigation menu, select Health, and then select Rounds Pro Manager. The Rounds Pro Manager page appears. From the Rounds Pro Manager page, access the Rounds Designer and the Route details page to verify the data loaded using the data loaders.
Data Loader Load Verification for Rounds Designer
Access the Picklist Management page and verify that the loaded data using Data loader appears as expected with all the records imported and that the values can be edited, or new values added to both the Action and Reading Picklist type. Also verify that you can create new character (and numeric, if applicable) Picklists.
Access the Step Template page and verify that the loaded Step Templates list appear as expected and the Step template records displays all the details and can be edited as required.
Verify that the Health Indicator are created for the given levels as expected.
Access a Step and verify that the loaded Steps list appear as expected in the asset hierarchy.
Select a Step to view the step details and verify that the Step records displays all the imported values and can be edited as required.
Access a Step that was imported and verify that the Health Indicators record is linked to the Step as expected.
Access a Step Condition and verify that all the data is loaded from the data loader for that conditional step.
Data Loader Load Verification for Route Masters
Access the Route Masters page.
Verify that the loaded Route Masters page appears as expected with all the data for the Routes, associated steps and is assigned to all individuals and/or teams included in the Data Loader.
- The existing assigned assets and steps appear as expected.
- The existing steps created in the step data loader or in the step designer can be assigned to the route.
- The assigned Assets and Steps can be reordered.
- The asset details appearing as imported.
- The Route name and route description appear as imported.
- The route schedules have been populated.
- The route has been to a user or team.
- The Basic for Occurrence (EAM and Schedule) details are loaded as expected with all the required data.
- The Route Preview details are appearing as expected with the loaded data.
- It can be assigned to a new User or Team or remove the assigned team or user.
Verify that the updated changes in the routes are appearing as expected in the Route Masters page.