Why clearing memmapfile is taking too long?
3 ビュー (過去 30 日間)
古いコメントを表示
Hi team,
I am trying to use fileDataStore with custom read function. In this custom read function I am using memmapfile to read data from file. In this function I am clearing memmapfile object. It is taking approximately 3 seconds to get cleared. So, my questions are:
1) Why it is taking too long to clear memmapfile?
2) Is there any other way to clear memmapfile?
Following is attached snippets for the code.
[filename,filepath] = uigetfile('*.*', 'Select a file');
readTime = ReadDataStoreMapTime(fullfile(filepath, filename));
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
clear file;
disp(toc);
end
0 件のコメント
回答 (1 件)
Hassaan
2024 年 4 月 15 日
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
disp(toc); % The memmapfile object 'file' will be cleared automatically when it goes out of scope
end
By relying on MATLAB's automatic cleanup when file goes out of scope, this might slightly improve the performance or at least the predictability of memory management. If these steps don't help, consider profiling the code with MATLAB's profiler to identify any other bottlenecks or issues.
0 件のコメント
参考
カテゴリ
Help Center および File Exchange で Text Files についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!