Optimal Method For Saving Large Amounts of Data To Read Into MATLAB

5 ビュー (過去 30 日間)
Cole Smith
Cole Smith 2015 年 2 月 20 日
コメント済み: Tejash Patel 2018 年 6 月 25 日
Hi, I have somewhat of a unique problem in that I have a box with a constant stream of time-series network data being received and filtered, that I eventually want to read into MATLAB. This data can be several gigabytes per hour, and my MATLAB application only needs to query specific messages/timeframes. I know MATLAB uses HDF5 internally, so that looked like a good option but there's really no time-series capabilities. Other wrappers such as PyTables/TsTables exist to add time series to HDF5, but that requires an extra python wrapper. I've also looked into things like InfluxDB, but that doesn't save in the native HDF5 format, so I think there's a good chance large queries could be extremely slow if executed in MATLAB.
Has anyone done something similar or know of a good way to store continuous network data that I want to search/open in MATLAB at a later time?
  1 件のコメント
Tejash Patel
Tejash Patel 2018 年 6 月 25 日
Did you ever find a solution for this?


回答 (1 件)

Greig 2015 年 2 月 22 日
While I'm sorry to say that I don't have a direct solution for you, perhaps you will get some useful ideas from these blogs on handling big data in MATLAB...

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by