ImportOptions
This object requires Streaming Data Framework for MATLAB® Production Server™.
Description
An ImportOptions
object specifies how MATLAB imports tabular data from event streams. The object contains properties that
control the data import process, including handling of errors and missing data.
Creation
You can create an ImportOptions
object by using either the detectImportOptions
function or the eventStreamImportOptions
function. The preferred way is to use
detectImportOptions
.
Use
detectImportOptions
to detect and populate the import properties based on the contents of the event stream specified bystream
.opts = detectImportOptions(stream)
Use
eventStreamImportOptions
to create import properties by specifying import options as name-value arguments.opts = eventStreamImportOptions(Name1=Value1,...,NameN=ValueN)
Properties
SelectedVariableNames
— Subset of variables to import
character vector | string scalar | cell array of character vectors | string array
Subset of variables to import, specified as a character vector, string scalar, cell array of character vectors, or string array.
SelectedVariableNames
must be a subset of names contained in the VariableNames
property. By default, SelectedVariableNames
contains all the variable names from the VariableNames
property, which means that all variables are imported.
Use the SelectedVariableNames
property to import only the variables of
interest. Specify a subset of variables using the
SelectedVariableNames
property and use the
readtimetable
function to import only that subset.
Example:
opts.SelectedVariableNames = "x"
imports only the variable
x
from the event stream when you use
readtimetable
to import event stream data into a
timetable.
Data Types: char
| string
VariableNames
— Variable names
cell array of character vectors | string array
Variable names, specified as a cell array of character vectors or string array. The
VariableNames
property contains the names to use when importing
variables from the event stream into a timetable.
These variable names must exist in the stream. If the modified variable name is not in the stream, the import operation fails.
Example: io.VariableNames
returns the current variable names in the event
stream.
Example: io.VariableNames(3) = {'Mass'}
changes the name of the third variable to Mass
.
Data Types: cell
| string
VariableTypes
— Data type of variables
cell array of character vectors | string array
Data type of variables, specified as a cell array of character vectors or string array
containing a set of valid data type names. The VariableTypes
property
designates the data types to use when importing variables from the event stream into a
timetable.
The import operation attempts to convert the values in the stream to these types. This operation succeeds only when the conversion between primitive types is known and unambiguous, such as:
Integer to string conversions
Conversions where the constructor of the target type can accept a variable of that type in the stream
To update the VariableTypes
property, use the setvartype
function.
Example: io.VariableTypes
returns the data types that stream variables have
after they are imported into MATLAB timetables, which are the types of the
corresponding timetable columns.
Example: io = setvartype(io,"vMotor","int32")
changes the data type of the vMotor
variable to int32
.
Data Types: cell
| string
KeyVariable
— Key variable name
key
(default) | string scalar | character vector
Name of the key variable in the event stream, specified as a string scalar or
character vector. The default value is key
.
Data Types: string
| char
Object Functions
setvartype | Set data types used to import and export variables to stream |
Examples
Create Import Options from Event Stream Data
Assume that you have a Kafka® server running at the network address kafka.host.com:9092
that has a topic Triangles
with JSON-encoded event value
'{"triangle":{"x":"3","y":"4","z":"5"}}'
. Each event value contains a
variable "triangle"
, which is a structure of side lengths
"x"
, "y"
, and "z"
. The side
lengths are integers but are encoded as strings. readtimetable
creates a triangle
table column for this variable and sets the column
value to this
structure:
triangle = struct with fields: x: "3" y: "4" z: "5"
ImportOptions
enables you to change the types of the values in the imported structure and select which
fields to import.
Create a KafkaStream
object connected to the Triangles
topic.
ks = kafkaStream("kafka.host.com",9092,"Triangles");
Create an ImportOptions
object from the Kafka stream object. The data type of the length of each side is
string
.
opts = detectImportOptions(ks)
opts = ImportOptions with properties: VariableNames: ["triangle/x" "triangle/y" "triangle/z"] VariableTypes: ["string" "string" "string"] KeyVariable: "key" SelectedVariableNames: ["triangle/x" "triangle/y" "triangle/z"]
To perform mathematical operations on the imported data, update the data type of variables
to double
. Because the side length variables are nested within
"triangle"
, use a forward slash ("/"
) to specify the
path to these variables.
opts = setvartype(opts, ["triangle/x", "triangle/y", "triangle/z"], "double")
opts = ImportOptions with properties: VariableNames: ["triangle/x" "triangle/y" "triangle/z"] VariableTypes: ["double" "double" "double"] KeyVariable: "key" SelectedVariableNames: ["triangle/x" "triangle/y" "triangle/z"]
Update the ImportOptions
property of the KafkaStream
object.
ks.ImportOptions = opts
ks = KafkaStream with properties: Topic: "Triangles" Group: "85c42e39-695d-467a-86f0-f0095792e7de" Order: EventTime Host: "kafka.host.com" Port: 9092 ConnectionTimeout: 30 RequestTimeout: 61 ImportOptions: "Import to MATLAB types" ExportOptions: "Source: function eventSchema" PublishSchema: "true" WindowSize: 50 KeyVariable: "key" KeyEncoding: "utf16" KeyType: "text" KeyByteOrder: "BigEndian" BodyEncoding: "utf8" BodyFormat: "JSON" ReadLimit: "Size" TimestampResolution: "Milliseconds"
When importing the triangles, readtimetable
converts the side lengths
to double
values.
tt = readtimetable(ks); tt(1,:).triangle
ans = struct with fields: x: 3 y: 4 z: 5
Create Import Options by Specifying Import Variable Names and Types
Create a schema for importing data from an event stream into MATLAB by specifying variable names and their data types to use during the import.
names = ["x","symbol"]; types = ["double","string"];
Construct an ImportOptions
object using this data import schema.
opts = eventStreamImportOptions(VariableNames=names,VariableTypes=types)
opts = ImportOptions with properties: VariableNames: ["x" "symbol"] VariableTypes: ["double" "string"] KeyVariable: [0×0 string] SelectedVariableNames: ["x" "symbol"]
Apply the import options when creating a KafkaStream
object.
ks = kafkaStream("kafka.host.com",9092,"Your_Kafka_Topic",ImportOptions=opts);
Import the data. The "Your_Kafka_Topic"
topic must have events with
exactly two variables, x
and symbol
. In addition, the
types of these variables must be convertible to double and string, respectively. Otherwise,
readtimetable
throws an error.
tt = readtimetable(ks);
Version History
Introduced in R2022b
See Also
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)