Preprocess Data
The System Identification Toolbox™ app and command-line functions allow you to preprocess the estimation (and validation) data before using it for model estimation. Preprocessing helps refine the data and correct or remove inaccuracies. It ensures that the data is in a suitable form for model estimation.
After selecting the data for estimation, check your data for any undesired traits such as:
Missing or faulty values (also known as outliers). For example, you might see gaps that indicate missing data, values that do not fit with the rest of the data, or noninformative values.
Offsets and drifts in signal levels (low-frequency disturbances).
High-frequency disturbances above the frequency interval of interest for the system dynamics.
Depending on the data characteristics, you can reconstruct missing data, change the data sampling rate, remove the means, constant offsets, or linear trends from the data.
For a method to analyze time-domain or frequency-domain data, see How to Analyze Data Using the advice Command.
Functions
detrend | Subtract offset or trend from time-domain signals contained in
iddata objects |
retrend | Add offsets or trends to time-domain data signals stored in iddata
objects |
diff | Difference signals in iddata objects |
idfilt | Filter data using user-defined passbands, general filters, or Butterworth filters |
misdata | Reconstruct missing input and output data |
nkshift | Shift data sequences |
idresamp | Resample time-domain data by decimation or interpolation |
idresampOptions | Option set for idresamp (Since R2023a) |
resample | (Not recommended) Resample time-domain data that is stored in an
iddata object by decimation or interpolation (requires
Signal Processing Toolbox software) |
getTrend | Create trend information object to store offset, mean, and trend information for
time-domain signals stored in iddata object |
chgFreqUnit | Change frequency units of frequency-response data model |
fdel | Delete specified data from frequency response data (FRD) models |
TrendInfo | Offset and linear trend slope values for detrending data |
Topics
Handling, Resampling, and Filtering Data
- Handling Missing Data and Outliers
Handling missing or erroneous data values. - Handling Offsets and Trends in Data
Removing and restoring constant offsets and linear trends in data signals. - Resampling Data
Decimating and interpolating (resampling) data. - Filtering Data
Deciding whether to filter data before model estimation and how to prefilter data.
Preprocessing Data Using the App
- Preprocess Data Using Quick Start
Subtract mean values from data, and specify estimation and validation data. - How to Detrend Data Using the App
Before you can perform this task, you must have regularly-sampled, steady-state time-domain data imported into the System Identification app. - Resampling Data Using the App
Use the System Identification app to resample time-domain data. - How to Filter Data Using the App
The System Identification app lets you filter time-domain data using a fifth-order Butterworth filter by enhancing or selecting specific passbands.
Preprocessing Data Using the Command Line
- How to Detrend Data at the Command Line
Before you can perform this task, you must have time-domain data as aniddataobject. - Resampling Data at the Command Line
Decimate and interpolate time-domain data. - How to Filter Data at the Command Line
Useidfiltto apply passband and other custom filters to a time-domain or a frequency-domainiddataobject.
