Data Simulator
      Back to home
      1. Timebase Knowledge Base
      2. Data Collectors & Plugins
      3. Data Simulator

      The Data Simulator Plugin

      This plugin provides the ability to generate simulated historical data based on real world processes.

      Simulator Settings

      Specify Sequences to Repeat

      Mapping Instances using Fields

      Defining Process Steps

      Defining Tag Templates

      Tag Values


      Simulator Plugin

      Once configured and running, the Simulator will continuously generate data according to the processes modeled by its configuration.

      A useful feature of the Simulator is its ability to "backfill" generated data as needed (see the Backfill setting). The Simulator will generate data up to "now", and then periodically continue generating data so that you always have live data up to at least "now" (in fact, it generates "cycles" of data a little into the future, which you may see in Timebase Explorer and the Timebase Historian API).

      Example Configuration

      The following Simulator configuration generates data for a simple Juice Factory packaging facility. It generates data for 10 Fillers. See explanation below.

       {
        "Active": true,

        "Historians": [
          {
            "Host": "historian",
            "Port": 4511,
            "UseTls": false
          }
        ],

        "Type": "Simulator",
        "Dataset": "The Juice Factory",

        "Settings": {
          "Backfill": 7,
          "State": "/state",
          "Sequences": {
            "Fillers": {
              "map": [
                {
                  "name": "Area Code",
                  "data": "131"
                },
                {
                  "name": "Equipment",
                  "data": "Filler"
                },
                {
                  "name": "Equipment Number",
                  "data": "1-10"
                }
              ],
              "steps": [
                {
                  "value": "0",
                  "duration": "random(15, 45)"
                },
                {
                  "value": "10",
                  "duration": "random(35, 120)"
                },
                {
                  "value": "20",
                  "duration": "random(35, 155)"
                },
                {
                  "value": "random(21, 23)",
                  "duration": "random(0, 10)"
                },
                {
                  "value": "20",
                  "duration": "random(45, 155)"
                },
                {
                  "value": "random(21, 23)",
                  "duration": "random(0, 10)"
                },
                {
                  "value": "20",
                  "duration": "random(35, 155)"
                },
                {
                  "value": "random(21, 23)",
                  "duration": "random(0, 10)"
                },
                {
                  "value": "20",
                  "duration": "random(45, 55)"
                },
                {
                  "value": "random(21, 23)",
                  "duration": "random(0, 10)"
                },
                {
                  "value": "20",
                  "duration": "random(15, 45)"
                },
                {
                  "value": "30",
                  "duration": "random(35, 95)"
                }
              ],
              "tags": [
                {
                  "name": "[Equipment Number, FL000].[Measure]",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": {
                    "0": "Idle",
                    "10": "Startup",
                    "20": "Running",
                    "21": "Bottle starvation",
                    "22": "Bottle jam",
                    "23": "Downstream bottleneck",
                    "30": "Cleaning"
                  },
                  "fields": {
                    "Measure": "State",
                    "Product": "Juice"
                  },
                  "type": "System.Int32",
                  "values": [
                    "step"
                  ]
                },
                {
                  "name": "[Equipment Number, FL000].[Measure]",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": {
                    "0": "Water",
                    "1": "Apple",
                    "2": "Lemonade",
                    "3": "Raspberry",
                    "4": "Orange",
                    "5": "Peach"
                  },
                  "fields": {
                    "Measure": "Product"
                  },
                  "type": "System.Int32",
                  "values": [
                    "0",
                    "random(0, 5)",
                    "previous"
                  ]
                },
                {
                  "name": "[Equipment Number, FL000].[Measure]",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "fields": {
                    "Measure": "BatchId"
                  },
                  "type": "System.String",
                  "values": [
                    "Water",
                    "[Equipment Number, FL000][Step Date, yyMMddHHmm]",
                    "previous"
                  ]
                },
                {
                  "name": "[Area Code, 000]-[Instrument Code]-[Equipment Number, 000].PV",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": { "1": "°F" },
                  "fields": {
                    "Measure": "Temperature",
                    "Instrument Code": "TT"
                  },
                  "type": "System.Double",
                  "values": [
                    "flat(1.5, 18.0, 1.0)",
                    "ramp(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "flat(1.5, 64.8, 1.3)",
                    "ramp(1.5, 18.5, 1.0)"
                  ]
                },
                {
                  "name": "[Area Code, 000]-[Instrument Code]-[Equipment Number, 000].PV",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": { "1": "MPa" },
                  "fields": {
                    "Measure": "Pressure",
                    "Instrument Code": "PT"
                  },
                  "type": "System.Double",
                  "values": [
                    "flat(1.5, 1.5, 0.2)",
                    "ramp(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "flat(1.5, 6.48, 0.2)",
                    "ramp(1.5, 1.5, 0.2)"
                  ]
                },
                {
                  "name": "[Area Code, 000]-[Instrument Code]-[Equipment Number, 000].PV",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": { "1": "ft³/s" },
                  "fields": {
                    "Measure": "Flow Rate",
                    "Instrument Code": "FT"
                  },
                  "type": "System.Double",
                  "values": [
                    "flat(1.5, 0.0, 0.5)",
                    "ramp(1.5, 14, 0.5)",
                    "flat(1.5, 13.8, 0.5)",
                    "flat(1.5, 0.0, 0.5)",
                    "flat(1.5, 13.8, 0.5)",
                    "flat(1.5, 0.0, 0.5)",
                    "flat(1.5, 13.8, 0.5)",
                    "flat(1.5, 0.0, 0.5)",
                    "flat(1.5, 13.8, 0.5)",
                    "flat(1.5, 0.0, 0.5)",
                    "flat(1.5, 13.8, 0.5)",
                    "flat(1.5, 3.3, 0.5)"
                  ]
                },
                {
                  "name": "[Area Code, 000]-[Instrument Code]-[Equipment Number, 000].PV",
                  "description": "[Equipment] [Equipment Number] [Measure]",
                  "uom": { "1": "ft³" },
                  "fields": {
                    "Measure": "Flow Total",
                    "Instrument Code": "FQ"
                  },
                  "type": "System.Double",
                  "values": [
                    "0",
                    "integrate([Area Code, 000]-FT-[Equipment Number, 000].PV, second, true)",
                    "integrate([Area Code, 000]-FT-[Equipment Number, 000].PV, second, false)"
                  ]
                }
              ]
            }
          }
        }
      }

      Backfill and State

      • Backfill int: the number of days of historical data to generate. The Simulator will randomize a start date for each sequence. Default value is 30 days.
      • State string: the path where the Simulator will store its running state. This allows a seamless restart after tag modifications or additions have been made to the sequence definitions. Examples include "C:\\Timebase\\Collector\\Simulator\\State" for Windows and "/state" for Docker.

      Sequence Definition

      A Sequence represents a "cycle" of steps that repeat continuously, with some randomness built in so that each cycle is simulated slightly differently. Typically, a sequence would be used to drive a "state engine" for a class of equipment or a period of time, e.g. "Fillers", "Mixing Tanks", "Reactors", "Solar Days" etc. Multiple named sequences can be defined.

      A named sequence, e.g. Fillers, is defined by three sections:

      • Map
      • Steps
      • Tags

      Map Definition

      Map is used as a "template" to define the number and uniqueness of instances created for a sequence definition. To understand this, notice the Equipment Number field has a data value of 1-10. This syntax tells the Simulator to create 10 instances, namely 1, 2, 3, through 10. As we will see later, combining the Equipment and Equipment Number fields to generate tag metadata results in "Filler 1", "Filler 2", "Filler 3", through "Filler 10". Using this "template" mechanism allows for the generation of large models.

      "map": [
          {
              "name": "Area Code",
              "data": "131"
          },
          {
              "name": "Equipment",
              "data": "Filler"
          },
          {
              "name": "Equipment Number",
              "data": "1-10"
          }
      ]

      Map is a list of Field objects:

      • Name string required The name of the Field, e.g. "Area Code", "Equipment Type", "Equipment Number", "Product Type", etc.
      • Data string required The patterns of data used to generate sequence instances. There are a number of formats in which this pattern can be defined:
        • Strings Comma separated list of strings, e.g. "Mixing, Packaging, Distribution". This example would generate 3 instances.
        • Numbers Comma separated list of number, e.g. "1, 2, 3, 7, 8, 21, 28". This example would generate 7 instances.
        • String multiplier pairs Comma separated list of (string * multiplier), e.g. "ABC*10, DEF*3". This example would generate 13 instances.
        • Number multiplier pairs Comma separated list of (number * multiplier), e.g. "12.3*10, 13*3". This example would generate 13 instances.
        • Number range with optional step Comma separated list of number ranges, with an optional step, e.g. "1-10" or "1.5-2.5/0.1" or "1-10/2" or "1-10, 20-30" or "1-10, 20-30/2, -5.5-3.2". A step of 1 will be used if it is not provided.
      • Wrap bool Specifies whether to continuously wrap to the beginning of the Data sequence, or to repeat the last value in the sequence, e.g. If the Map calls for 5 instances, then "1, 2" with Wrap = true renders the equivalent of "1, 2, 1, 2, 1" whereas Wrap = false renders the equivalent of "1, 2, 2, 2, 2".

      Steps Definition

      Sequences are defined by a list of the Steps, in order of process, that define a single "cycle" of the equipment or process' "state engine", e.g. 0 = "Idle", 10 = "Startup", etc. In the example below, three steps are defined for a "cycle". The first step has a value of 0 and a random duration between 15 and 45 minutes. Each time the Simulator creates a new cycle for this sequence, step 0 will have a random duration. The second step has value of 10 and a fixed duration 35.5 minutes. Every cycle generated will have a step 10 exactly 35.5 minutes in duration. The third step has a value of 20 and a random duration between 35 and 155 minutes.

      "steps": [
          {
              "value": "0",
              "duration": "random(15, 45)"
          },
          {
              "value": "10",
              "duration": "35.5"
          },
          {
              "value": "20",
              "duration": "random(35, 155)"
          }
      ]
      Steps is a list of Step objects:
      • Value string required The value of the step in the context of a "step engine" cycle. The output must convert to an int. The following definitions are available:
        • Number As single int value, e.g. "0" or "10" or "30"
        • Random random(minimum, maximum) where minimum and maximum are int values, e.g. "random(20, 23)" will generate a random integer from 20 to 23, inclusive. This is useful for generating random downtime fault codes.
      • Duration string required The duration of the step in minutes double. The following definitions are available:
        • Number As single int or double value, e.g. "5" or "11.3" or "32.5"
        • Random random(minimum, maximum) where minimum and maximum are int or double values, e.g. "random(2.5, 23.8)" will generate a random duration from 2.5 to 23.8 minutes, inclusive.

      Tags Definition

      Tags will be generated for each sequence instance determined by the Map. For the example above, the map defines 10 Fillers (because of the Equipment Number being 1-10). The Simulator will generate the defined list of tags for each of these 10 Fillers. So, think of each tag definition as being a tag "template" or "class".

      The Tag definition is mostly metadata (e.g. name, description, fields, etc.), but an important part of its definition is the values list. The values list of each tag works in conjunction with the steps list of the sequence. A value object is a string defining the tag data to be generated for an aligned step, e.g. a number, a string, or a function (see below).

      For the tag definition to be useful as a "template", certain tag properties must be defined using [Placeholders]. Placeholders must reference fields defined either in the Map, or in the Tag itself. Placeholders are defined as:

      • [Field Name] The value of the Field will be used to replace the placeholder, e.g. [Area Code] where "Area Code" has a value of 131 will render as 131 in the tag instance. Or,
      • [Field Name, 000] The formatted value of the Field will be used to replace the placeholder, e.g. [Equipment Number, FL000] where "Equipment Number" has a value of 2 will render as FL002
      "tags": [
          {
              "name": "[Equipment Number, FL000].[Measure]",
              "description": "[Equipment] [Equipment Number] [Measure]",
              "uom": {
                  "0": "Water",
                  "1": "Apple",
                  "2": "Lemonade",
                  "3": "Raspberry",
                  "4": "Orange"
              },
              "fields": {
                  "Measure": "Product"
              },
              "type": "System.Int32",
              "values": [
                  "0",
                  "random(0, 4)",
                  "previous"
              ]
          }
      ]

      Tags is a list of Tag objects:

      • Name string required The name of a tag, which can include placeholders, e.g. "[Area Code, 000]-[Instrument Code]-[Device Number, 000].PV"
      • Description string The description of a tag, which can include placeholders, e.g. "[Equipment] [Equipment Number] [Measure] [Device Number]"
      • Format string The formatting of a tag's values, if they are numbers, e.g. "0.00"
      • UOM numbered dictionary of string The unit of measure of a tag. However, the numbered dictionary provides addition features of the UOM:
        • Single key Negative key places UOM in front of number, positive key places UOM after number, e.g. { "-1": "\(" } renders as "\) 14.95", { "1": "°F" } renders as "32.3 °F"
        • Boolean key For use with boolean tags, e.g. { 1: "Running", 0: "Stopped" }, { 1: "Open", 0: "Closed" }
        • Multiple keys For use with enumerated integer tags, e.g. { 1: "Apple", 2: "Orange", 4: "Lemon" }
      • Fields named dictionary of string Name Value pairs that represent a tag's metadata, e.g. { "Measure": "Temperature", "Measure Code": "TT", "Device Number": "1" }
      • Type string The datatype of a tag, as defined by .NET, e.g. "System.Double", "System.Int64", "System.String", "System.Boolean"
      • Values array of string required The data generation definition for each Step defined by the Sequence Steps. If there are less Value definitions than there are Steps, then the last Value definition will be repeated until the end of the cycle of Steps. A number of options are available for defining a step's data generation:
        • step Reference to the step number generated by the Sequence Step, e.g. "step"
        • null Generate a null value, e.g. "null"
        • previous Generate a value which is equal to the last value that was generated, e.g. "previous"
        • Number int or double, e.g. "5" or "3.14"
        • random(minimum, maximum) where minimum and maximum are int or double values, e.g. "random(0, 5)" will generate a random number between 0 and 5, inclusive.
        • flat(interval, value, noise) where interval is the number of seconds double between points generated, value is the value the flat function will generate, and noise (optional) is a random amplitude overlayed onto the ramp. noise defaults to 0. Note that the actual interval between points generated is randomized around the interval value. interval cannot be set lower than 1 second. E.g. "flat(1.5, 25.0)", "flat(1.5, 38.5, 2.5)"
        • ramp(interval, to, noise) where interval is the number of seconds double between points generated, to is the value the ramp function will be at the end of the step, and noise (optional) is a random amplitude overlayed onto the ramp. noise defaults to 0. Note that the actual interval between points generated is randomized around the interval value. interval cannot be set lower than 1 second. The ramp function will start at the last value of the previous step. E.g. "ramp(1.5, 65.0)", "ramp(1.5, 18.5, 2.5)"
        • integrate(tagname, timebase, reset) where tagname refers to a tag within the Sequence instance that will be integrated over time, timebase can be "second", "minute", or "hour", and reset is true or false and determines whether the integrator should reset back to 0 on this step or not. tagname can contain placeholders. As an example, if the tag being integrated represent a flow rate in ft³/s the timebase should be set to "second" so the result of the integration represent total ft³. reset defaults to false. Some examples, "integrate([Area Code, 000]-FT-[Equipment Number, 000].PV, second, true)", "integrate([Area Code, 000]-FT-[Equipment Number, 000].PV, second)"
        • For System.String tag types, standard Placeholder syntax can be used to generate string values, e.g. "[Equipment Number, FL000]-[Step Date, yyyyMMdd]" will generate "FL001-20241120". Note that Step Date is an internal field that is only available to make string value unique.