Transform and Combine fileDatastores to Train CNN

11 ビュー (過去 30 日間)
Hi I would appreciatte your help,
I have a collection of 50x1x12 mat files, that I need to upload into some datastore to subsequently pass into a convolutional neural network, how can I Train my
but instead of having .mat file for the labels I am using this function to get the labels from the folders.
function label = readLabel(filename,classNames)
filepath = fileparts(filename);
[~,label] = fileparts(filepath);
label = categorical(string(label),classNames);
end
I am uploading the files into the fileDatastore
inputData=fileDatastore(fullfile('inputData'),'ReadFcn',@load,'FileExtensions','.mat');
%getting the labels from the folders
classNames = string(1:2)
targetData=fileDatastore(fullfile('targetData'),'ReadFcn',@load,'FileExtensions','.mat',readLabel(filename,classNames));
Should I transform before combine the filedatastores? if so, how should my transform function be?
I tried combining without transforming and I got this error when I tried to train my CNN: (trainNetwork)
%combininig
cdsTrain = combine(inputData,targetData);
%Training
net = trainNetwork(cdsTrain,layers,options);
"The training images are of size 1x1x1 but the input layer expects images of size 50x1x12"
Thanks for your help!
  2 件のコメント
Mauricio Piraquive
Mauricio Piraquive 2021 年 2 月 21 日
I am working with a "Image" like file when
50 would be the heigth of the "Image"
1 would be the width
12 would be the number of channels. (like 3 for rgb but in this case I have 12 channels).
These file correspond to the reading of a "material" using a lab instrument,
for example, one file could correspond to the results of the reading of material type-A
another file correspond to the results of the reading of material type-B
So I want to create this convolutional neural network, pass a reading and the network should identify the materia and tell me what type of Material that data correspond.
Right now I dont have any target data, I am getting the labels from the folders that contains each file. However, if needed I can create this output data files,
What Size this label files should be?
My current .mat files dont have the label in it.
Really appreciatte your help.
Regards,
Mauricio

サインインしてコメントする。

採用された回答

luisa di monaco
luisa di monaco 2021 年 2 月 22 日
編集済み: luisa di monaco 2021 年 2 月 22 日
Hi, this is my code. I know it is not efficient, but it should work! I hope it can help as a starting point to be optimized.
For simplicity, I added information about category inside each .mat file as string variable. However, I'm sure you can get labels from folder names too.
The key idea is to use fileDatastore for input data and tabularTextDatastore for categorical data. Then you can combine them succesfully.
Regards
clear
close all
clc
%% create dummy inputs and targets (.mat file)
% inputs are images of size 50x1x12 [heigth, width, channels]
% target are type of material: categorical array
mkdir trainingData
cd trainingData
n=10; % number of training cases
for i=1:n
material=ones(50,1,12)*i;
if i<n/2
matType='typeA';
else
matType='typeB';
end
mkdir(matType)
filename=sprintf('material_%d', i);
save(fullfile(matType,filename),'material', 'matType')
end
cd ..
%% load data
allData=fileDatastore(fullfile('trainingData'),'ReadFcn',@load,'FileExtensions','.mat', 'IncludeSubfolders', true);
%% create inputs
inputData = transform(allData,@(data) rearrange_input(data)); % extract and rearrange input
%% create targets (can be optimized...)
targetData = transform(allData,@(data) rearrange_target(data));
myLabels=targetData.readall;
writematrix(myLabels,'myLabels.txt');
labelStore = tabularTextDatastore('myLabels.txt','TextscanFormats','%C',"ReadVariableNames",false);
read_size=1; % this line is foundamental... I don't know why...
labelStore.ReadSize = read_size;
labelStoreCell = transform(labelStore,@setcat_and_table_to_cell);
%% combininig
cdsTrain = combine(inputData,labelStoreCell);
%% training
% dummy layers and options
numClasses=2;
layers=[
imageInputLayer([50 1 12],"Name","imageinput")
batchNormalizationLayer()
leakyReluLayer(0.1)
fullyConnectedLayer(numClasses,'Name','fc')
softmaxLayer('Name','soft')
classificationLayer('Name','classification')];
options = trainingOptions('adam');
net = trainNetwork(cdsTrain,layers,options);
%% functions
function inputData = rearrange_input(data)
inputData=data.material;
inputData= {inputData};
end
function targetData = rearrange_target(data)
targetData=data.matType;
targetData=categorical(cellstr(targetData));
end
function [dataout] = setcat_and_table_to_cell(datain)
validcats = ["typeA", "typeB"];
datain.(1) = setcats(datain.(1),validcats);
dataout = table2cell(datain);
end
  1 件のコメント
Mauricio Piraquive
Mauricio Piraquive 2021 年 3 月 26 日
@luisa di monaco Thank you very much this was very helpful. Really appreciate it.

サインインしてコメントする。

その他の回答 (1 件)

Mahesh Taparia
Mahesh Taparia 2021 年 2 月 24 日
Hi
One possible approach is already suggested by Luisa. In your approach, replace your read function with the following line of code
function label = readLabel(filename)
filepath = fileparts(filename);
[~,label] = fileparts(filepath);
label = categorical(string(label));
end
Also, the fileDatastore of label data is not created properly, replace it with following line of code
targetData=fileDatastore(fullfile('targetData'),'ReadFcn',@readLabel,'FileExtensions','.mat');
Try it, it may work. If it won't work, apply break points in readLabel function and try to create the function readLabel such that it should returns the categorical label. Hope it will help!

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by