Create New System Objects for File Input and Output

Open Live Script

This example shows how to create and use two different System objects to facilitate the streaming of data in and out of MATLAB®: TextFileReader and TextFileWriter.

The objects discussed in this example address a number of realistic use cases, and they can be customized to achieve more advanced and specialized tasks.

Introduction

System objects are MATLAB classes that derive from matlab.System. As a result, System objects all inherit a common public interface, which includes standard methods:

setup — Initialize the object, typically at the beginning of a simulation
reset — Clear the internal state of the object, bringing it back to its default post-initialization status
release — Release any resources (memory, hardware, or OS-specific resources) used internally by the object

When you create new kinds of System objects, you provide specific implementations for all the preceding methods to determine its behavior.

In this example we discuss the internal structure and the use of the following two System objects:

TextFileReader
TextFileWriter

To create these System objects for streaming data in and out of MATLAB, this example uses standard low-level file I/O functions available in MATLAB (like fscanf, fread, fprintf, and fwrite). By abstracting away most usage details of those functions, they aim to make the task of reading and writing streamed data simpler and more efficient.

This example includes the use of a number of advanced constructs to author System objects. For a more basic introduction to authoring System objects, see Create System Objects.

Definition of the Class `TextFileReader`

The TextFileReader class includes a class definition, public and private properties, a constructor, protected methods overridden from the matlab.System base class, and private methods. The TextFileWriter class is similarly structured.

Class Definition

The class definition states that the TextFileReader class is derived from both matlab.System and matlab.system.mixin.FiniteSource.

   classdef (StrictDefaults)TextFileReader < matlab.System & matlab.system.mixin.FiniteSource

matlab.System is required and is the base class for all System objects
matlab.system.mixin.FiniteSource indicates this class is a signal source with a finite number of data samples. For this type of class, in addition to the usual interface, the System object™ will also expose the isDone function. When isDone returns true, the object reached the end of the available data.

Public Properties

Public properties can be changed by the user to adjust the behavior of the object to his or her particular application. TextFileReader has two nontunable public properties (they can only be changed before the first call to the object) and four tunable public properties. All the public properties have a default value. Default values are assigned to the corresponding properties when nothing else is specified by the user.

   properties (Nontunable)
       Filename   = 'tempfile.txt' 
       HeaderLines = 4
   end

   properties
       DataFormat = '%g' 
       Delimiter = ',' 
       SamplesPerFrame = 1024
       PlayCount = 1
   end

Private Properties

Private properties are not visible to the user and can serve a number of purposes, including

To hold values computed only occasionally, then used with subsequent calls to the algorithm. For example, values used at initialization time, when setup is called or the object is called for the first time. This can save recomputing them at runtime and improve the performance of the core functionality
To define the internal state of the object. For example, pNumEofReached stores the number of times that the end-of-file indicator was reached:

   properties(Access = private)
       pFID = -1 
       pNumChannels 
       pLineFormat 
       pNumEofReached = 0
   end

Constructor

The constructor is defined so that you can construct a TextFileReader object using name-value pairs. The constructor is called when a new instance of TextDataReader is created. The call to setProperties within the constructor allows setting properties with name-value pairs at construction. No other initialization tasks should be specified in the constructor. Instead, use the setupImpl method.

   methods
       function obj = TextFileReader(varargin)
           setProperties(obj, nargin, varargin{:});
       end
   end

Overriding `matlab.System` Base Class Protected Methods

The public methods common to all System objects each have corresponding protected methods that they call internally. The names of these protected methods all include an Impl postfix. They can be implemented when defining the class to program the behavior of your System object.

For more information on the correspondence between the standard public methods and their internal implementations, please refer to Summary of Call Sequence.

For example, TextFileReader overrides these Impl methods:

setupImpl
resetImpl
stepImpl
releaseImpl
isDoneImpl
processTunedPropertiesImpl
loadObjectImpl
saveObjectImpl

Private Methods

Private methods are only accessible from within other methods of the same class. They can be used to make the rest of the code more readable. They can also improve code reusability, by grouping under separate routines code that is used multiple times in different parts of the class. For TextFileReader, private methods are created for:

getWorkingFID
goToStartOfData
peekCurrentLine
lockNumberOfChannelsUsingCurrentLine
readNDataRows

Write and Read Data

This example shows how you can use TextFileReader and TextFileWriter by:

Creating a text file containing the samples of two different sinusoidal signals using TextFileWriter
Read from the text file using TextFileReader.

Create a Simple Text File

Create a new file to store two sinusoidal signals with frequencies of 50 Hz and 60 Hz. For each signal, the data stored is composed of 800 samples at a sampling rate of 8 kHz.

Create data samples:

fs = 8000;
tmax = 0.1;
t = (0:1/fs:tmax-1/fs)';
N = length(t);
f = [50,60];
data = sin(2*pi*t*f);

Form a header string to describe the data in a readable way for future use (optional step):

fileheader = sprintf(['The following contains %d samples of two ',...
    'sinusoids,\nwith frequencies %d Hz and %d Hz and a sample rate of',...
    ' %d kHz\n\n'], N, f(1),f(2),fs/1000);

To store the signal to a text file, create a TextFileWriter object. The constructor of TextFileWriter needs the name of the target file and some optional parameters, which can be passed in as name-value pairs.

TxtWriter = TextFileWriter('Filename','sinewaves.txt','Header',fileheader)

TxtWriter = 
  TextFileWriter with properties:

      Filename: 'sinewaves.txt'
        Header: 'The following contains 800 samples of two sinusoids,...'
    DataFormat: '%.18g'
     Delimiter: ','

TextFileWriter writes data to delimiter-separated ASCII files. Its public properties include:

Filename — Name of the file to be written. If a file with this name already exists, it is overwritten. When operations start, the object begins writing to the file immediately following the header. The object then appends new data at each subsequent call to the object, until it is released. Calling reset resumes writing from the beginning of the file.
Header — Character string, often composed of multiple lines and terminated by a newline character (\n). This is specified by the user and can be modified to embed human-readable information that describes the actual data.
DataFormat — Format used to store each data sample. This can take any value assignable as Conversion Specifier within the formatSpec string used by the built-in MATLAB function fprintf. DataFormat applies to all channels written to the file. The default value for this property is '%.18g', which allows saving double precision floating point data in full precision.
Delimiter — Character used to separate samples from different channels at the same time instant. Every line of the written file maps to a time instant, and it includes as many samples as the number of channels provided as input (in other words, the number of columns in the matrix input passed to the object).

To write all the available data to the file, a single call to can be used.

TxtWriter(data)

Release control of the file by calling the release function.

release(TxtWriter)

The data is now stored in the new file. To visually inspect the file, type:

edit('sinewaves.txt')

Because the header takes up three lines, the data starts on line 4.

In this simple case, the length of the whole signal is small, and it fits comfortably on system memory. Therefore, the data can be created all at once and written to a file in a single step.

There are cases when this approach is not possible or practical. For example, the data might be too large to fit into a single MATLAB variable (too large to fit on system memory). Alternatively, the data might be created cyclically in a loop or streamed into MATLAB from an external source. In all these cases, streaming the data into the file can be done with an approach similar to the following example.

Use a streamed sine wave generator to create a frame of data per loop. Run the desired number of iterations to create the data and store it into the file:

frameLength = 32;
tmax = 10; 
t = (0:1/fs:tmax-1/fs)';
N = length(t);
data = sin(2*pi*t*f);
numCycles = N/frameLength;

for k = 1:10 % Long running loop when you replace 10 with numCycles. 
    dataFrame = sin(2*pi*t*f);
    TxtWriter(dataFrame)
end

release(TxtWriter)

Read from Existing Text File

To read from the text file, create an instance of TextFileReader.

TxtReader = TextFileReader('Filename','sinewaves.txt','HeaderLines',3,'SamplesPerFrame',frameLength)

TxtReader = 
  TextFileReader with properties:

           Filename: 'sinewaves.txt'
        HeaderLines: 3
         DataFormat: '%g'
          Delimiter: ','
    SamplesPerFrame: 32
          PlayCount: 1

TextFileReader reads numeric data from delimiter-separated ASCII files. Its properties are similar to those of TextFileWriter. Some differences follow

HeaderLines — Number of lines used by the header within the file specified in Filename. The first call to the object starts reading from line number HeaderLines+1. Subsequent calls to the object keep reading from the line immediately following the previously read line. Calling reset will resume reading from line HeaderLines+1.
Delimiter — Character used to separate samples from different channels at the same time instant. In this case, the delimiter is also used to determine the number of data channels stored in the file. When the object is first run, the object counts the number of Delimiter characters at line HeaderLines+1, say numDel. Then for every time instant, the object reads numChan = numDel+1 numeric values with format DataFormat. The matrix returned by the algorithm has size SamplesPerFrame-by-numChan.
SamplesPerFrame — Number of lines read by each call to the object. This value is also the number of rows of the matrix returned as output. When the last available data rows are reached, there might be fewer than the required SamplesPerFrame. In that case, the available data are padded with zeros to obtain a matrix of size SamplesPerFrame-by-numChan. Once all the data are read, the algorithm simply returns zeros(SamplesPerFrame,numChan) until reset or release is called.
PlayCount — Number of times the data in the file is read cyclically. If the object reaches the end of the file, and the file has not yet been read a number of times equal to PlayCount, reading resumes from the beginning of the data (line HeaderLines+1). If the last lines of the file do not provide enough samples to form a complete output matrix of size SamplesPerFrame-by-numChan, then the frame is completed using the initial data. Once the file is read PlayCount times, the output matrix returned by the algorithm is filled with zeros, and all calls to isDone return true unless reset or release is called. To loop through the available data indefinitely, PlayCount can be set to Inf.

To read the data from the text file, the more general streamed approach is used. This method of reading data is also relevant to dealing with very large data files. Preallocate a data frame with frameLength rows and 2 columns.

dataFrame = zeros(frameLength,2,'single');

Read from the text file and write to the binary file while data is present in the source text file. Notice how the method isDone is used to control the execution of the while loop.

while(~isDone(TxtReader))
    dataFrame(:) = TxtReader();
end

release(TxtReader)

Summary

This example illustrated how to author and use System objects to read from and write to numeric data files. TextFileReader and TextFileWriter can be edited to perform special-purpose file reading and writing operations. You can also combine these custom System objects with built-in System objects such as dsp.BinaryFileWriter and dsp.BinaryFileReader.

For more information on authoring System objects for custom algorithms, see Create System Objects.