Log OPC Server Data

How Data Is Logged

The OPC Data Access Specification provides access to current values of data on an OPC server. Often, for analysis, troubleshooting, and prototyping purposes, you will want to know how OPC server data has changed over a period of time. For example, you can use time series data to perform control loop optimization or system identification on a portion of your plant. Industrial Communication Toolbox™ software provides a logging mechanism that stores a history of data that changed over a period of time. This section discusses how to configure and execute a logging task using the toolbox.

Note

The toolbox software logging mechanism is not designed to replace a data historian or database application that logs data for an extended period. Rather, the logging mechanism allows you to quickly configure a task to log data on an occasional basis, where modifications to the plant-wide data historian may be unfeasible.

Industrial Communication Toolbox software uses the data change event to log data. Each data change event that is logged is called a record. The record contains information about the time the client logged the record, and details about each item in the data change event. Data change events are discussed in detail in Data Change Events and Subscription.

The use of a data change event for logging means that you should consider the following points when planning a logging session:

• Logging takes place at the group level — When planning a logging task, configure the group with only the items you need to log. Including more items than you need to will only increase memory and/or disk usage, and using that data may be more difficult due to unnecessary items in the data set.

• Inactive items in a group will not be logged — You must ensure that the items you need to log are active when you start a logging session. You control the active state of a daitem object using the Active property of the daitem object.

• Data change events (records) may not include all items — A data change event contains only the items in the group that have changed their value and/or quality state since the last update. Hence, a record is not guaranteed to contain every data item. You need to consider this when planning your logging session.

• OPC logging tasks are not guaranteed to complete — Because data change events only happen when an item in the group changes state on the server, it is possible to start a logging task that will never finish. For example, if the items in a group never change, a data change event will never be generated for that group. Hence, no records will be logged.

• Logged data is not guaranteed to be regularly sampled — It is possible to force a data change event at any time (see Force a Data Change Event). If you do this during a logging task, the data change events may occur at irregular sample times. Also, a data change event may not contain information for every item in the group. Consequently, logged OPC server data may not occur at regular sample times.

An overview of the logging task, and a representation of how the above points impact the logging session, is provided in the following section.

Overview of a Logging Task

To illustrate a typical logging task, the following example logs to disk and memory six records of data from two items provided by the Matrikon™ OPC Simulation Server. During the logging task, data is retrieved from memory. When the task stops, the remaining records are retrieved.

Step 1: Create the OPC object hierarchy.  This example creates a hierarchy of OPC objects for two items provided by the Matrikon Simulation Server. To run this example on your system, you must have the Matrikon Simulation Server installed. Alternatively, you can replace the values used in the creation of the objects with values for a server you can access.

da = opcda('localhost','Matrikon.OPC.Simulation.1');
connect(da);
grp = addgroup(da,'CallbackTest');
itm1 = additem(grp,'Triangle Waves.Real8');
itm2 = additem(grp,'Saw-Toothed Waves.Boolean');

Step 2: Configure the logging duration.  This example sets the UpdateRate value to 1 second, and the RecordsToAcquire property to 6. See Control the Duration of a Logging Session for more information on this step.

grp.UpdateRate = 1;
grp.RecordsToAcquire = 6;

Step 3: Configure the logging destination.  In this example, data is logged to disk and memory. The disk filename is set to LoggingExample.olf. The LogToDiskMode property is set to 'overwrite', so that if the filename exists, the toolbox engine must overwrite the file. See Control the Logged Data Destination for more information on this step.

grp.LoggingMode = 'disk&memory';
grp.LogFileName = 'LoggingExample.olf';
grp.LogToDiskMode = 'overwrite';

Step 4: Start the logging task.  Start the dagroup object. The logging task is started, and the group summary updates to reflect the logging status. See Start a Logging Task for more information on this step.

start(grp)
grp

Step 5: Monitor the Logging Progress.  After about 3 seconds, retrieve and show the last acquired value. After another second, obtain the first two records during the logging task. Then wait for the logging task to complete. See Monitor the Progress of a Logging Task for more information on this step.

pause(3.5)
sPeek = peekdata(grp, 1);
% Display the local event time, item IDs and values
disp(sPeek.LocalEventTime)
disp({sPeek.Items.ItemID;sPeek.Items.Value})
pause(1)
sGet = getdata(grp, 2);
wait(grp)

Step 6: Retrieve the data.  This example retrieves the balance of the records into a structure array. See Retrieve Data from Memory for more information on this step.

sFinished = getdata(grp,grp.RecordsAvailable);

Step 7: Clean up.  When you no longer need them, always remove from memory any toolbox objects and the variables that reference them. Deleting the opcda client object also deletes the group and daitem objects.

disconnect(da)
delete(da)
clear da grp itm1 itm2

Configure a Logging Session

A logging session is associated with a dagroup object. Before you start a logging session, you will need to ensure that the logging session is correctly configured. This section explains how you can control

• The duration of a logging session (see Control the Duration of a Logging Session). By default, a group will log approximately one minute of data at half second intervals.

• The destination of logged data (see Control the Logged Data Destination). By default, a group will log data to memory.

• The response to events that take place during a logging session (see Configure Logging Callbacks). By default, a logging session takes no action in response to events that take place during a logging session.

Control the Duration of a Logging Session

While you cannot guarantee that a logging session will take a specific amount of time (see How Data Is Logged), you can control the rate at which the server will update the items and how many records the logging task should store before automatically stopping the logging task. You control these aspects of a logging task by using the following properties of the dagroup object:

• UpdateRate: The UpdateRate property defines how often the item values are inspected.

• RecordsToAcquire: The RecordsToAcquire property defines how many records the toolbox must log before automatically stopping a logging session. A logging task can also be stopped manually, using the stop function.

• DeadbandPercent: The DeadbandPercent property does not control the duration of a logging task directly, but has a significant influence over how often a data change event is generated for analog items (an item whose value is not confined to discrete values). By setting the DeadbandPercent property to 0, you can ensure that a data change event occurs each time a value changes. For more information on DeadbandPercent, consult the property reference page.

You can use the UpdateRate and RecordsToAcquire properties to define the minimum duration of a logging task. The duration of a logging task is at least

UpdateRate * RecordsToAcquire

For example, if the UpdateRate property is 10 (seconds) and the RecordsToAcquire property is 360, then provided that a data change event is generated each time the server queries the item values, the logging task will take 3600 seconds, or one hour, to complete.

Control the Logged Data Destination

Industrial Communication Toolbox software allows you to log data to memory, to a disk file, or both memory and a disk file. When logging data to memory, you can log only as much data as will fit into available memory. Also, if you delete the dagroup object that logged the data without extracting that data to the MATLAB^® workspace, the data will be lost. The advantage of logging data to memory is that logging to memory is faster than using a disk file.

Logging data to a disk file usually means that you can log more data, and the data is not lost if you quit MATLAB or delete the dagroup object that logged the data. However, reading data from a disk file is slower than reading data from memory.

The LoggingMode property of a dagroup object controls where logged data is stored. You can specify 'memory' (the default value), or 'disk', or 'disk&memory' as the value for LoggingMode.

The following properties control how the toolbox logs data to disk. You must set the LoggingMode property to 'disk' or 'disk&memory' for these properties to take effect:

• LogFileName: The LogFileName property is a character vector that specifies the name of the disk file that is used to store logged data. If the file does not exist, data will be logged to that filename. If the file does exist, the LogToDiskMode property defines how the toolbox behaves.

• LogToDiskMode: The LogToDiskMode property controls how the toolbox handles disk logging when the file specified by LogFileName already exists. Each time a logging task is started, if the LoggingMode is set to 'disk' or 'disk&memory', the toolbox checks to see if a file with the name specified by the LogFileName property exists. If the file exists, the toolbox will take the following action, based on the LogToDiskMode property:

  • 'append': When LogToDiskMode is set to 'append', logged data will be added to the existing data in the file.

  • 'overwrite': When LogToDiskMode is set to 'overwrite', all existing data in the file will be removed without warning, and new data will be logged to the file.

  • 'index': When LogToDiskMode is set to 'index', the toolbox automatically changes the log filename, according to the following algorithm:

    The first log filename attempted is specified by the initial value of LogFileName.

    If the attempted filename exists, LogFileName is modified by adding a numeric identifier. For example, if LogFileName is initially specified as 'groupRlog.olf', then groupRlog.olf is the first attempted filename, groupRlog01.olf is the second filename, and so on. If LogFileName already contains numeric characters, they are used to determine the next sequence in the modifier. For example, if the LogFileName is initially specified as 'groupRlog010.olf', and groupRlog010.olf exists, the next attempted file is groupRlog011.olf, and so on.

    The actual filename used is the first filename that does not exist. In this way, each consecutive logging operation is written to a different file, and no previous data is lost.

Configure Logging Callbacks

You can configure the dagroup object so that MATLAB will automatically execute a function when the logging task starts, when the logging task stops, and each time a specified number of records is acquired during a logging task. The dagroup object has three callback properties that are used during a logging session. Each callback property defines the action to take when a particular logging event occurs:

• Start event: A start event is generated when a logging task starts.

• Records acquired event: A records acquired event is generated each time a logging task acquires a set number of records.

• Stop event: A stop event is generated when a logging task stops, either automatically, or by the user calling the stop function.

For an example of using callbacks in a logging task, see View Recently Logged Data.

Execute a Logging Task

Once you have configured your logging task you can execute the task. Executing a logging task involves starting the logging task, monitoring the task progress, and stopping the logging task.

Start a Logging Task

You start a logging task by calling the start function, passing the dagroup object that you want to start logging. The following example starts a logging task for the dagroup object grp.

start(grp)

When you start a logging task, certain group and item properties become read-only, as modifying these properties during a logging task would corrupt the logging process. Also, the dagroup object performs the following operations:

1. Generates a start event and executes the StartFcn callback.

2. If Subscription is 'off', sets Subscription to 'on' and issues a warning.

3. Removes all records associated with the object from the toolbox engine.

4. Sets RecordsAcquired and RecordsAvailable to 0.

5. Sets the Logging property to 'on'.

Monitor the Progress of a Logging Task

During a logging task, you can monitor the progress of the task by examining the following properties of the dagroup object:

• Logging: The Logging property is set to 'on' at the start of a logging task, and set to 'off' when the logging task stops.

• RecordsAcquired: The RecordsAcquired property contains the number of records that have been logged to the destination specified by the LoggingMode property. When a start function is called, RecordsAcquired is set to 0. When RecordsAcquired reaches RecordsToAcquire, the logging task stops automatically.

• RecordsAvailable: The RecordsAvailable property contains the number of records that have been stored in the toolbox engine for this logging task. Data is only logged to memory if the LoggingMode is set to 'memory' or 'disk&memory'. You extract data from the toolbox engine using the getdata function. See Get Logged Data into the MATLAB Workspace for more information on using getdata.

You can monitor these properties in the summary display of a dagroup object, by typing the name of the dagroup object at the command line.

grp

grp =
Summary of OPC Data Access Group Object: group1
   Object Parameters
      Group Type   : private
      Item         : 1-by-1 daitem object
      Parent       : localhost/Matrikon.OPC.Simulation.1
      Update Rate  : 0.5
      Deadband     : 0%
   Object Status
      Active       : on
      Subscription : on
      Logging      : on
   Logging Parameters
      Records      : 120
      Duration     : at least 60 seconds
      Logging to   : disk
      Log File     : group1log.olf ('index' mode)
      Status       : 5 records acquired since starting.
                     0 records available for GETDATA/PEEKDATA

Stop a Logging Task

A logging task stops when one of the following conditions is met:

• The number of records logged reaches the value defined by the RecordsToAcquire property.

• You manually stop the logging task by using the stop function.

The following example manually stops the logging task for dagroup object grp.

stop(grp)

When a logging task stops, the Logging property is set to 'off', a stop event is generated, and the StopFcn callback is executed.

Get Logged Data into the MATLAB Workspace

Industrial Communication Toolbox software does not log data directly to the MATLAB workspace. When logging to memory, the data is buffered in the toolbox engine in a storage-efficient way. When logging to disk, the data is logged in ASCII format. To analyze your data, you need to extract the data from the toolbox engine or from a disk file into MATLAB for processing. This section describes how to get your logged data into the MATLAB workspace. The following sections describe this process:

• Retrieve Data from Memory, discusses how to retrieve data from the toolbox engine into MATLAB.

• Retrieve Data from Disk, discusses how to retrieve data from a disk file into MATLAB.

Whether you log data to memory or to disk, you can retrieve that logged data in one of two formats:

• Structure format: This format stores each data change event in a structure. Data from a logging task is simply an array of such structures.

• Array format: To visualize and analyze your data, you will need to work with the time series of each of the items in the group. The array format is the logged structure data, “unpacked” into separate arrays for the Value, Quality, and TimeStamp.

Retrieve Data from Memory

You retrieve data from memory using the getdata function, passing the dagroup object as the first argument, and the number of records you want to retrieve as the second argument. The data is returned as a structure containing data from each data change event in the logging task. For example, to retrieve 20 records for the dagroup object grp:

s = getdata(grp, 20);

If you do not supply a second argument, getdata will try to retrieve the number of records specified by the RecordsToAcquire property of the dagroup object. If the toolbox engine contains fewer records for the group than the number requested, a warning is generated and all of the available records will be retrieved.

To retrieve data in array format, you must indicate the data type of the returned values. You pass a character vector defining that data type as an additional argument to the getdata function. Valid data types are any MATLAB numeric data type (for example, 'double' or 'uint32') plus 'cell' to denote the MATLAB cell array data type.

When you specify a numeric data type or cell array as the data type for getdata, the logged data is returned in separate arrays for the item IDs logged, the value, quality, time stamp, and the local event time of each data change event logged. You must therefore specify up to five output arguments for the getdata function when retrieving data in array format.

For example, to retrieve 20 records of logged data in double array format from dagroup object grp.

[itmID,val,qual,tStamp,evtTime] = getdata(grp,20,'double');

Once you have retrieved data to the MATLAB workspace using getdata, the records are removed from the toolbox engine to free up memory for additional logged records. If you specify a smaller number of records than those available in memory, getdata will retrieve the oldest records. You can use the RecordsAvailable property of the dagroup object to determine how many records the toolbox engine has stored for that group.

During a logging task, you can examine the most recently acquired records using the peekdata function, passing the dagroup object as the first argument, and the number of records to retrieve as the second argument. Data is returned in a structure. You cannot return data into separate arrays using peekdata. You can convert the structure returned by peekdata into separate arrays using the opcstruct2array function. Data retrieved using peekdata is not removed from the toolbox engine.

For an example of using getdata and peekdata during a logging task, see Overview of a Logging Task.

When you delete a dagroup object, the data stored in the toolbox engine for that object is also deleted.

Retrieve Data from Disk

You can retrieve data from a disk file into the MATLAB workspace using the opcread function. You pass the name of the file containing the logged OPC data as the first argument. The data stored in the log file is returned as a structure array, in the same format as the structure returned by getdata. Records retrieved from a log file into the MATLAB workspace are not removed from the log file.

You can specify a number of additional arguments to the opcread function, that control the records that are retrieved from the file. The additional arguments must be specified by an option name and the option value. The following options are available.

Option Name	Option Value Description
'items'	Specify a cell array of item IDs that you want returned. Items not in this list will not be read.
'dates'	Specify a date range for the event times. The range must be [startDt endDt] where startDt and endDt are MATLAB date numbers.
'records'	Specify the index of records to retrieve as [startRec endRec]. Records outside these indices will not be read.
'datatype'	Specify the data type, as a character vector, that should be used for the returned values. Valid data type character vectors are the same as for getdata. If you specify a numeric data type or 'cell', the output will be returned in separate arrays. If you specify a numeric array data type such as 'double' or 'uint32', and the logged data contains arrays or character vectors, an error will be generated and no data will be returned.

The following example retrieves the data logged during the example on page Overview of a Logging Task, first into a structure array, and then records 3 to 6 are retrieved into separate arrays for Value, Quality, and TimeStamp.

sDisk = opcread('LoggingExample.olf')

sDisk = 
40x1 struct array with fields:
    LocalEventTime
    Items

[i,v,q,t,e] = opcread('LoggingExample.olf', ...
    'records',[3,6], 'datatype','double')
i = 
    'Random.Real8'    'Random.UInt2'    'Random.Real4'
v =
  1.0e+004 *
    0.7819    3.0712    1.4771
    1.5599    2.7792    2.2051
    1.4682    0.4055    0.5315
    0.0235    2.4473    1.5456
q = 
  'Good: Non-specific'  'Good: Non-specific'  'Good: Non-specific'
  'Good: Non-specific'  'Good: Non-specific'  'Good: Non-specific'
  'Good: Non-specific'  'Good: Non-specific'  'Good: Non-specific'
  'Good: Non-specific'  'Good: Non-specific'  'Good: Non-specific'
t =
  1.0e+005 *
    7.3202    7.3202    7.3202
    7.3202    7.3202    7.3202
    7.3202    7.3202    7.3202
    7.3202    7.3202    7.3202
e =
  1.0e+005 *
    7.3202
    7.3202
    7.3202
    7.3202

Note

For a record to be returned by opcread, it must satisfy all the options passed to opcread.