How to upload 800 csv files that only contains numbers in a cell keeping their names
1 回表示 (過去 30 日間)
古いコメントを表示
Hello, I would like to import my 800 csv files by keeping their name so I can identify them afterwards and perform my image processing. I try this script but it takes too much time. Thank you any help is welcome.
fichiersRecherches = '*.csv';
[FileName,PathName] = uigetfile(fichiersRecherches,'Sélectionnez les fichiers qui ont pour extention csv', 'MultiSelect', 'on');
FileName = cellstr(FileName);
m = cell(1,length(FileName));
for i_file = 1:size(m,2)
m{i_file} = xlsread(fullfile(PathName, FileName{i_file}));
end
6 件のコメント
Jan
2019 年 4 月 11 日
For "use the profiler":
doc profile
For the message
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 3) ==>
There seems to be an unexpected string in row 1 and column 3.
採用された回答
Jan
2019 年 4 月 12 日
編集済み: Jan
2019 年 4 月 13 日
As I said: The number contain commas as decimal separators. Before such a file can be imported, in much be converted. This costs a lot of time.
Maybe this is more efficient to fix the file contents:
function Comma2Dot(FileName)
file = memmapfile(FileName, 'writable', true);
comma = uint8(',');
point = uint8('.');
file.Data(transpose(file.Data == comma)) = point;
end
Afterwards a simple fscanf(fid, '%g;', [472, Inf]) will import the data efficiently.
By the way, all decimal places are "000000" only. This means that storing only the integer part would be ways better. Storing the data in binary format would even better again. So teh main problem is that a really inefficient file format has been chosen. and you cannot blame the import of MATLAB.
With the original data:
tic;
Comma2Dot('test2.csv');
% Emulatre DLMREAD:
fid = fopen('test2.csv');
C = textscan(fid, '', -1, 'Delimiter', ';', 'EndOfLine', '\r\n', ...
'CollectOutput', 1);
fclose(fid);
data = C{1};
toc
% Elapsed time is 0.120229 seconds.
Now try:
% Write data in binary format:
fid = fopen('TestData.bin', 'W');
% Number of dimensions and size
fwrite(fid, [ndims(data), size(data)], 'uint64');
fwrite(fid, data, 'uint16');
fclose(fid);
tic;
fid = fopen('TestData.bin', 'r');
nDimsData = fread(fid, 1, 'uint64');
sizeData = fread(fid, [1, nDimsData], 'uint64');
data = fread(fid, sizeData, 'uint16');
fclose(fid);
toc
% Elapsed time is 0.013736 seconds.
The timings might be unfair, because reading data, which have been written to disk directly before, will be taken from the disk cache. But the accelerateion is expected: Compare the file sizes of 2'632 kB for the text file and 442 kB for the binary file.
So the actual optimization is not to improve the Matlab code, but to use a smart format to store the file.
8 件のコメント
Jan
2019 年 4 月 16 日
編集済み: Jan
2019 年 4 月 16 日
"It does not work" is a lean explanation and does not allow to understand, what the problem is. Please post the details.
I've used the posted methods successfully and the timings show, that it will be much faster, if you use a proper file format instead of a text files with commas and meaingless zeros as decimal places. I've explained thois exhaustively already and do not know, how I can help you now.
その他の回答 (3 件)
参考
カテゴリ
Help Center および File Exchange で File Operations についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!