Matlab 2023 release of Datafeed Toolbox, with function "signals" breaks - well, so much

I have built a large database with records that use the name "signals" for an internal struct. This has worked fine for years. I recently updated Matlab to release 2023a. Now I cannot access my "signals" variables. 2023a has added a toolbox "Datafeed" that includes a function named "signals". Whenever I attempt to access the struct, e.g. as isfield(signals,'pads'), Matlab throws an error stating that the Datafeed toolbox is required.

I have rolled back to release 2022b, and everything works fine. But, I cannot now proceed to use 2023a, or presumably any subsequent release. I have thousands of records with the "signals" struct included, and many scripts and functions that access its data. I don't want to have to rewrite all of the records and the programs that use them.

How can I use newer releases without encountering this problem?

[Cris LaPierre]

How is your structure organized? Can you provide an simple example that reproduces the problem? I tried this but it works.

signals.pads = 1

signals = struct with fields:
    pads: 1

isfield(signals,'pads')

ans = logical
   1

[Walter Roberson]

Are you "poofing" variables into existence by using load without an output, or by using eval() or assignin() to create variables?

[jkr]

I am using 'load'. Why would this matter? The records in this database of files have internal structs named signals, info, channels, events and some other things. I have long used load to import the data for a particular case, whether from the command line or within a script or function. The particular instance I reported on involved 'load'ing such a record by its filename, within a script, then trying to access the 'signals' struct. I have no idea what you mean by "poofing". This seems a normal use of data to me. I have used it without issue for years.

[jkr]

Chris,

I don't see how this can be. However, I rolled back to 2022b and everything works fine, as it always has. I am busy getting some work done, running 2022b, and not ready to check in 2023a again. But, the problem was definitely repeatable in 2023a.

Jim

[Cris LaPierre]

Understood. It's just hard for this forum to help without more details.

[jkr]

As noted in response to Walter, 'signals' is a struct within each file in the database, along with other structs named info, events, channels, etc. Within signals are a bunch of binary recordings from a device, with a variety of names like 'pads', 'comps', 'vent'. There are a lot of alternative signal element names, not always present. That's why I test for their presence before further processing.

Actually, until recently, signals and its peers were kept under a master struct named 'rec'. Recently, we have moved them to the top level of the file so we can easly load the smaller elements (e.g. 'info') without needing to load the larger ones, such as signals, for some purposes. This worked fine under 2022b.

As Walter suggested, in his terminology, I am "poofing" the variable into existence by loading the file without specifying any internal variables, in instances where I want to access 'signals', the other elements being so relatively small that I don't bother avoiding them by specifying load(filename,'signals').

When I load such a file, 'signals' shows up in the workspace. But, under 2023a, I cannot programmatically access it because of the error Matlab throws. I tried simply changing its name:

Signals = signals;

(perhaps a poor choice of replacement name) but even this threw the error.

I can try to investigate further with 2023a after I get some other work done.

Have you tried, after:

signals.pads = 1;

save('anyfilename','signals');

clear signals;

load(anyfilename);

isfield(signals.pads)?

That would illuminate the possible problem of "poofing".

This is really a surprise. It is obvious from the context (e.g. isfield(signals,'pads')) that I am not invoking a function, but consulting a variable. I am surprised that Matlab has trouble with this.

Thanks for your interest and efforts.

Jim

[jkr]

Well, color me puzzled. After finishing the > 3 million measurements I needed to do using R2022b, I returned to R2023a and the script I had trouble with started to run just fine. It takes hours, so I interrupted it, but the error, previously, emerged on either 1) processing the first file or 2) possibly during a syntax check before processing any file.

In any case, it is no longer repeatable.

Answer 1

[Walter Roberson]

MATLAB now does more static analysis of functions, under the assumption that variables are not being "poofed" into existence.

At parse time, MATLAB looks to see which variables are assigned to, and puts those together in an expected-to-be-variables list.

Then it works through the code, examining each word.

• Each word that is on the expected-to-be-variables list is marked internally as expecting to be resolved as a variable.
• Each word that is not on the expected-to-be-variables list is looked look trying to find it as a function. If a function by the name is found, the word is added to the cache of expected-to-be-function, and the particular reference is marked as expected-to-be-function
• Any word that is not expected-to-be-variable and not resolved as a function is marked as expected-to-be-undefined

At execution time, each word is looked up only in the list according to what it is expected to be.

• if it was expected to be a variable but the variable was not assigned yet, then an error is raised about undefined variable (without mentioning "or function" in the message!) . This applies even if there is a function by that name. For example if you try to call the function sum() in code, but later you assign to a variable named sum then when the code encounters sum the first time, before it has been assigned to as a variable, then it will complain about sum being undefined, instead of using the function like it used to
• if it was expected to be a variable but the variable was cleared, then an error is raised about undefined (or else about refering to a cleared variable in some cases), even if there is a function by the same name. If you sum = 0; clear sum; sum(1:3) then you will get an error because the assignment somewhere in the code classified sum as a variable for the entire scope. MATLAB will not recheck whether the name now refers to a function in this situation.
• If it was expected to be a function (because there is no obvious assignment to the name in the code) but something assigns to the name as a variable, then MATLAB does not alter the references to say "Oh, turned out to be a variable after all" or anything similar. The expected-to-be-function references stay referring to the function, and code references to the name will be treated as function calls even if the name has somehow been assigned to. To make this more explicit: if you load MYMAT without any assignment, and MYMAT contains a variable with the same name as a known function (perhaps signal) then that name (such as signal) will be treated as a function call when it is encountered during execution. MATLAB will not recheck whether the name now refers to a variable in this situation.
• If it was expected to be undefined (because there is no obvious assignment to the name in the code and the name does not match any known function) but something assigns to it anyhow ("poofing" the variable into existence) then MATLAB will silently accept the name as now being a variable. MATLAB will recheck whether the name now refers to a variable in this situation.
• If it was expected to be undefined (because there is no obvious assignment to the name in the code and the name does not match any known function), and the name has not been assigned as a variable, but something uses it in potentially function form (remembering that name with no () is potentially a function call with no parameters), then I am not sure whether MATLAB will re-check to see whether a function of that name has somehow come into existence. If it continues to be unresolved after any hypothetical check done at this point, then the error message will talk about undefined function or variable.

So, if you get an error about undefined variable that does not mention "function or" in the message, then the reason is that MATLAB noticed that assign to the name somewhere in the code, and ruled out the possibility that the name might refer to a function, including in any portion executed before the expected assignment.

If you get an error about problems calling a function when you are expecting it to be talking about a variable of the same name, then the reason is that you do not have a clear assignment to the name and MATLAB looked up the function and marked the name as definitely a function, even if the name is later resolved to a variable.

What should you do?

You should convert all cases of

load FILENAME               %or
load('FILENAME', 'signal')

that loads signal without an assignment to signal , converting to something like

sigstruct = load('FILENAME'); signal = sigstruct.signal;

This would be an explicit assignment to signal and that will keep MATLAB from trying to resolve signal as a function.

This change to dealing with poofed variables was documented in the release notes a few releases ago (3-ish years)

The same problem occurs for the case where the code calls a script that assigns to variables that have the same name as functions. The work-around for that case is to assign something (anything) to the variable before the script is called. Scripts can replace the value of existing variables with no problems; the issue has to do with trying to figure out whether a name is going to be a function or a variable.

[jkr]

Walter,

That helps. I don't see how it explains that after restarting 2023a (having run 2022b to get the actual work I was interested in done), it did not complain when I ran the same scripts.

In this script, I load the files programmatically. However, often, I load a file simply by double-clicking on it, as I'm exploring various characteristics. Perhaps, in such scripts it would help to have a simple line at the start:

signals = struct;

?

Thanks.

[jkr]

Still, for whoever writes such procedures, it would seem that:

if isfield(signals,'anything')

should be sufficient to make it apparent that signals is a variable rather than a function.

[Walter Roberson]

if isfield(signals, 'anything')
    disp('yep, anything is a field')
else
    disp('nope, anything is not a field')
end

output = struct with fields:
       banana: 1
     anything: 2
    blueberry: 3

yep, anything is a field

function output = signals
    output = struct('banana', 1, 'anything', 2, 'blueberry', 3)
end

This establishes that using signals in isfield is not sufficient context to establish that signals is a variable. As in this example, signals could be a function that accepts no input parameters.

Answer 2

[Cris LaPierre]

You should report this to MathWorks. You can do so here: https://www.mathworks.com/support/contact_us.html

[jkr]

I have reported it.