MATLAB Answers

Deep Learning Memory issues for BiLstm mat files

1 ビュー (過去 30 日間)
Kiyan Afsari
Kiyan Afsari 2019 年 7 月 12 日
コメント済み: Kiyan Afsari 2019 年 7 月 19 日
Hi,
I have a huge dataset of over 20GB of numercial time-series data stored in files as shown below:
x1_1 = double(23*921600)
...
I have about 250 of these mat files. and the label or target of these are single row categorical array that shows 0 and 1.
y1_1= categorical(1*921600)
...
I loaded 5 of each file and was able to train and classiy them using a BiLSTM Netowrok of 3 hidden layers with over 300 neurorns. but now i want to run all of them. My example is very similar to this example, I also tried this which was no help. I know that i should use a datastore of some kind but i tried most[File,Tall] of them and couldn't solve the issue. Any ideas how to solve this issue? Thank you.

  0 件のコメント

サインインしてコメントする。

回答 (1 件)

Divya Gaddipati
Divya Gaddipati 2019 年 7 月 19 日
Hi,
You could use the Custom Mini-batch Datastore which uses the function sequenceDatastore that reads data from the specified folder and obtains labels from the subfolder names.
Refer to this link to learn more about how to train your networks.

  1 件のコメント

Kiyan Afsari
Kiyan Afsari 2019 年 7 月 19 日
Thank you for your answer. The issue that i am facing is that unlike the japanese vowel example, i do not have 1 label per sequence, i have a sequence label for each sequence for example:
x_train: 0 1 5 4 9 7 6 4 8 4 3 5 1 2
y_ label: 0 0 0 0 1 1 1 1 0 1 1 1 0
Something like that. I have sequence to sequence prediction rather than sequence to label.
1×2 cell array
{25×7680 double} {1×7680 categorical}

サインインしてコメントする。

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by