Processing raw (source) data through Data Pipeline (Data Transformers and Filters)
Data may need to be processed to support advanced use-cases with the data, such as:
-
Sort / Filter data based on the defined criteria
-
Calculate min/max/average values
-
Convert data format to a defined one
To do so, Data Pipeline must be applied to a Widget. Data Pipeline consists of a single or multiple pipes: Transformers and/or Filters. The output of one pipe is the input for the subsequent pipe, so the order of pipes matters to get the required pipeline output.
Transformers are used to adjust/change the data by applying the selected operation to the data.
Filters are used to select the required data according to the criteria specified.


Adding a Transformer
# | Step | Details | Screenshot |
1 |
Add a Data Source for a Widget |
Please find the instructions on setting up a Data Source at Graphics Builder - Adding a Data Source for a Widget | Adding-a-Data-Source-for-a-Widget |
|
2 |
Select the widget |
Left-click on a Widget put on the Canvas |
![]() |
3 |
Go to ‘Properties’ tab |
Left-click on a ‘Properties’ tab of the selected Widget |
|
4 |
Click on ‘Add Transformer’ button inside Data Pipeline section |
Left-click on ‘Add Transformer’ |
![]() ![]() |
5 |
Select the required Data Transformer and set up its parameters if availbale |
Please find the list of available transformers and their description at Graphics Builder - Adding a Data Source for a Widget | List-of-available-Transformers |
![]() ![]()
|
6 |
Click on ‘Save’ button |
Left-click on ‘Save’ |
List of Available Transformers
Data Transformer | Workflow | Parameters | Application Example | |
1 |
Converts value to string |
TBD |
None |
TBD |
2 |
Count |
Returns the number of data strings |
None |
Count will return 3 according to the number of data strings: ![]() ![]() |
Cumulative delta |
Returns the difference in target attribute of each data row versus the next one
|
|
Example 1: Calculating delta in absolute value Cumilative Delta with the following parameters
will return: row 109 = row 109 - row 108 = 51-55= -4 ![]() ![]() Before applying Cumulative Delta on Value
![]() ![]() After applying Cumulative Delta on Value
Example 2: Calculating delta in % Cumulative Delta with the following parameters
will return: row 360 = row 360 / row 359 *100% = 9 / 6.1 * 100%= 1,475 * 100% = 147,5% (+48%) ![]() ![]() Before applying Cumulative Delta on Value
![]() ![]() |
|
4 |
Extract attribute value |
Retrieve value of an attribute from data |
Attr - attribute name whose value needs to be retrieved from data. Possible values are:
|
Getting attributes from a Data String (see more details about Data String in Graphics Builder - Fetching Point Data | Data-String-Structure):
|
5 |
Format Relative data |
Formats relative date time strings in natural language ex: Last week, This month etc; to absolute date strings. |
None |
|
6 |
ISO formatter |
Formats ISO dates into user specified format |
|
|
7 |
Localize Data & Time |
Localizes the date and time to users's timezone |
|
Current User’s Timezone is UTC+3 Date&Time format before applying Localize Data & Time: ![]() ![]() Date&Time format after applying Localize Data & Time ![]() |
8 |
Parse URI |
Extracts attributes from Data String’s URI ("uri": "/client_id/site_name/properties/equip_name/ptype/point_name"). Example: ![]() ![]() See more details on attributes in Graphics Builder - Fetching Point Data | Data-String-Structure |
None |
Example 1: Table view Table widget applies Parse URI piplene by default to represent data from URI in multiple columns ![]() ![]() Example 2: String view
|
9 |
Reverse list |
Reverses the items in the list |
None |
![]() ![]() Original List
![]() ![]() Reversed List
|
10 |
Rollup data |
Rolls up history data based on the given rollup frequency and digest |
|
![]() ![]() Daily Mean Rollup
![]() Daily Variance Rollup
|
11 |
Set precision |
Sets the precision of floating point values |
|
![]() ![]() Original Value
![]() ![]() After set precision to 2 decimal places for Value
|
12 |
Sort data |
Sorts data by the specified attribute |
|
![]() ![]() Original List
![]() ![]() List sorted by Value (min to max)
![]() ![]() List sorted by Point Name (a-z)
|
13 |
Transform data to scatter |
|
Transforms data to support Scatter Chart widget. Will be always applied for that widget to see any data visualized. |
![]() ![]() Scatter Chart
|
Adding a Filter
# | Step | Details | Screenshot |
1 |
Add a Data Source for a Widget |
Please find the instructions on setting up a Data Source at Graphics Builder - Adding a Data Source for a Widget | Adding-a-Data-Source-for-a-Widget |
|
2 |
Select the widget |
Left-click on a Widget put on the Canvas |
![]() ![]()
|
3 |
Go to ‘Properties’ tab |
Left-click on a ‘Properties’ tab of the selected Widget |
|
4 |
Click on ‘Add Filter’ button inside Data Pipeline section |
Left-click on ‘Add Transformer’ |
![]() ![]()
|
5 |
Select the required Data Filter and set up its parameters if availbale |
Please find the list of available Filters and their description at Graphics Builder - Adding a Data Source for a Widget | List-of-available-Filters |
![]() ![]()
|
6 |
Click on ‘Save’ button |
Left-click on ‘Save’ |
List of Available Filters
Digital Transformer | Workflow | Attributes | Application Example | |
1 |
Contains |
Filters rows of data whose attr's value is contained in the specified list of of values |
|
![]() Original List
![]() ![]() Filtered List (Contains Attr='point_name', Values='Dewpoint', ‘Temperature’)
|
2 |
Equal to (==) |
Filters rows of data whose value is equal to the specified value |
|
![]() ![]() Original List
![]() ![]() Filtered List (Equal To Attr='value', Value='30.11')
|
3 |
First Item |
Retrieves the first item in the data |
None |
![]() ![]() Original Data Massive
![]() ![]() First Item
|
4 |
Greater than (>) |
Filters rows of data whose value is greater than the specified value |
|
![]() ![]() Original List
![]() ![]() Filtered List (Greater Than Attr='value', Value='31')
|
5 |
Greater than / equal to (>=) |
Filters rows of data whose value is greater than or equal to the specified value |
|
![]() ![]() Original List
![]() ![]() Filtered List (Greater Than or Equal TO Attr='value', Value='35')
|
6 |
Last Item |
Retrieves the last item in the data |
|
![]() ![]() Original Data Massive
![]() ![]() Last Item
|
7 |
Lesser than (<) |
Filters rows of data whose value is lesser than the specified value |
|
![]() ![]() Original List
![]() ![]() Filtered List (Lesser Than Attr='value', Value='7')
|
8 |
Lesser than/equal to (<=) |
Filters rows of data whose value is lesser than or equal to the specified value |
|
![]() Original List
![]() ![]() Filtered List (Lesser Than/Equal to Attr='value', Value='7')
|
9 |
Limit |
Limits/Truncates data to the no of items specified from the first item in the list |
|
![]() ![]() Original List
![]() ![]() Filtered List (Limit = ‘1', ‘From Bottom’ = 'No')
![]() ![]() Filtered List (Limit = ‘2', ‘From Bottom’ = 'Yes')
|
10 |
Not equal to (!=) |
Filters rows of data whose value is not equal to the specified value |
|
![]() ![]() Original List
![]() ![]() Filtered List (Not Equal To Attr='point_name', Value='Dewpoint')
|