Unable to load, File may be corrupt

Hi

I have met with this(Unable to load, File may be corrupt) problem for several times lately. I also found a seems possible solution, which is at

http://www.mathworks.com/support/solutions/en/data/1-5U76BJ/index.html?product=ML&solution=1-5U76BJ

However this solution does not always work, this time when I run splitmat, I got the message : 'Found bad 0-byte size at variable #1'. I don't understand this, and I really want to know a better solution to this, the reason for problem, or any suggestion of easy and more trustworthy way of saving data in matlab instead of .mat?

Thanks very much. Lina

[per isakson]

• Matlab release?
• OS?
• Version of mat-file?
• How are the corrupt mat-files created?

I think mat-files are a "trustworthy way of saving data".

[lina]

Thanks for your reply, I'm using matlab R2011b (7.13.0.564) on a 64 bit linux cluster. In terms of the mat file version, since I did not specify when I saved it, so I think it should be version 7, by creating, I just use save(filename, variablename). I have been able to successfully load it several days ago, but since yesterday, it was told to be corrupted.

What could be the potential reasons for mat file to be corrupted?

Thanks

[gire]

Hi, I am having exactly the same issue, only difference is I am using Windows server 2008 (no cluster).

There is no obvious reason for the mat files to be corrupted (no concurrent writing, no HD failure).

Answer 1

[Jason Ross]

You mention that you are using a cluster. Is it possible that multiple processes are trying to write to the same file simultaneously? This will inevetabaly lead to corruption.

An example situation would be if you are saving a file to a shared location with the same filename, so N MATLAB processes are trying to save (for example) /home/lina/myoutputs.mat .

[Fritz Ratnasamy]

and how do we avoid that ?

[Jordan Tackett]

Hi, How can we write multiple thinks to the same file then?

[Walter Roberson]

If you have multiple processes on the same host, often the easiest approach is to call a system file lock function and when you are told that the lock is granted, make the file changes and release the lock. If you are using a network file system then it depends on the network file system as to whether you can use the same technique.

If you cannot use system locks, then take advantage of the fact that renaming a file is an "atomic" operation

• before you start any of this computation going on any host, make sure that the file exists. Do not do this per-worker, this is something outside of all of the workers
• Per worker, create a global unique file name. tempname() can help with that
• per worker, start a loop.
• test whether the mat file exists.
• if not, pause a random amount of time and then go back to the beginning of the loop
• having determined that the mat file exists, ask to movefile (rename file) to the unique name
• check to see if the rename was successful. If not then someone beat you to the race, pause a random time and go back to the beginning of the loop
• if the rename was successful, then you won control. Go ahead and add or modify the mat file using the unique name. When done, rename it back to the original name, and exit the loop
• provided that you have the permission to rename the file and the file existed before all this started, and the process doing the writing succeeds in renaming it back, statistically you will eventually win the race and get to write the file.