Neural network inputs and Outputs with different time delay

It is possible to configure and train a NN with diffrent time delays in the MATLAB neural network tool box such as:

 Y(t)=F(x(t),x(t-4),x(t-7),y(t-1),y(t-3)) with F the model.

or

 Y(t)=F(x1(t-1),x2(t-2),x4(t-4))

Unfortunately NARX work only with same time delay for all inputs and outputs

Thanks

Answer 1

[Greg Heath]

Incorrect.

The number and values of ID and FD are independent.

Simple example:

 close all, clear all, clc, plt=0
 [X,T] = simplenarx_dataset;
     net = narxnet(1,1:2,10);
     view(net)
     [Xs,Xi,Ai,Ts] = preparets(net,X,{},T);
     whos
     rng(0)
     [net tr Ys Es Xf Yf] = train(net,Xs,Ts,Xi,Ai);
     view(net)
     whos
     ts = cell2mat(Ts);
     MSE00 = var(ts,1)           % 0.099154
     ys = cell2mat(Ys);
     es = cell2mat(Es);
     R2 = 1-mse(es)/MSE00         % 1
     plt=plt+1,figure(plt)
     hold on
     plot(ts,'o','LineWidth',2)
    plot(ys,'r--','LineWidth',2)

Hope this helps.

Thank you for formally accepting my answer

Greg

[Platon]

I think I did not explain well my question. When I said 'Unfortunately NARX work only with same time delay for all inputs and outputs' I meant the same ID for all inputs X1, X2 etc. and the same FD for all Outputs Y1, Y2, Y3 etc, but of course I agree with you concerning ID and FD are independent. This is an example of the model that I would to build in Matlab : Y(t)=F(x1(t-1),x2(t-2),x3(t-4),Y(t-2)) it contains one output Y but 3 delayed inputs with different TD x1(t-1),x2(t-2),x3(t-4) + delayed feedback output Y(t-2) please see picture below.

[Greg Heath]

I don't see a problem. Did you try the help example with this ID and FD?

help narxnet

[Platon]

Dear Greg, I tried the help narxnet and others but I did not succeed. It will be very helpful if you put a small code in your responses.

Let us take this code:

A=[1:50;1:50];% I used 1 2 3 4... in order to see how are the delayed variables
B=[1:50;1:50];
X = tonndata(A',false,false);
T = tonndata(B',false,false);
net = narxnet([0 1 2],[1 2],10);
[Xs,Xi,Ai,Ts] = preparets(net,X,{},T);
net.trainParam.showWindow=0;
net = train(net,Xs,Ts,Xi,Ai);
view(net)

In this example, each input (we have 2) will be delayed 2 times: x(t-1) et x(t-2) and the same thing for the feeed back delay. Now, let me suppose that I want to eliminate x(t-1) What expression can I use? I tried :

 net = narxnet([0 2],[1 2],10);

But unfortunately it did not work.

What do you propose to me to do I want to use a specific time delay for each Input?

[Greg Heath]

Maybe it didn't work because X(t-1) is heavily correlated with T(t) and X(t-2) is not.

Whenever you specify delays without first determining the significant lags via the autocorrelation of t and the cross-correlation of x and t, then it is a crap shoot.

search NEWSGROUP and ANSWERS using

greg nncorr

Hope this helps.

Greg

[Platon]

My question concerns Matlab functions and not the Input/Output auto/cross-correlation. Manually I can do only x(t-1) and x(t-45) in inputs but I have to code it...I thought Matlab propose an automatic function to do this action.

Using auto/cross-correlation functions to determine the lags is useful for linear systems but for nonlinear systems with high complexity it does not work. Other alternatives such as : approach based on Gram-Schmidt orthogonalization, Sensitivity analysis EFAST, approach based on Lipschitz quotient etc. can be very helpful to rank/select best inputs for a model. Sometimes when I present a full model with (just an example) x(t-1)...x(t-10) as inputs, methods cited above select only 4 relevent variables to be taken into account x(t-2), x(t-5), x(t-6), x(t-9).

[Greg Heath]

>Using auto/cross-correlation functions to determine the lags is useful for linear systems but for nonlinear systems with high complexity it does not work.

Bull.

Can you illustrate this with one of the many MATLAB timeseries datasets at

help nndatasets

Greg

[Greg Heath]

Very often using an underpowered model for input variable selection works surprisingly well.

Answer 2

[Greg Heath]

I think the above discussions are somewhat confusing. My main points are

 1. Use the target auto and target/input cross-correlation functions to determine the significant lags.
 2. In a MIMO system the significant lags may be different for different inputs and input/target 
combinations.
 3. However, 
   a. all inputs must have the same nonnegative lags (which may be nonconsecutive) 
   b. All output feedback must have the same positive lags (which may be nonconsecutive)

Hope this helps,

Greg

[Platon]

The line code : net = narxnet([1 3 6],[2 5],10) will create a net with 3 TD for each input such as In(t-1), In(t-3) and In(t-6) and 2 TD for the output such as Out(t-2) and Out(t-5). This is what I was asking for. But as you said perhaps it is more relevant to use the same lags (this is another question).

[Greg Heath]

No you misunderstood.

For a MIMO model all inputs must have the same set of input delays, e.g., ID = [ 1 3 6 ]. However, you might want to look at the significant lags for each crosscorrelation function of each input/output combination. Then choose a subset of those.

Correspondingly, for a MIMO model all targets must have the same set of feedback delays, e.g., FD = [ 2 6 ]. However, you might want to look at the significant lags for each target autocorrelation function of each target. Then choose a subset of those.

Answer 3

[Yu-hsuan Lo]

Still now,

I can't ensure that the Input delay set like [1 3 6] is equal to the result :

y(t) = f{ x(t-1) , x(t-2) , x(t-3) }.

Because the data treated by 'preparets' is the same when the delay seeting is [1 3 6] , [ 6 ] or [3 6], and the data feed into my NN model(NARX) are the same, like

   NARX = train(NARX,Xs,Ts,Xi).

So, will my NARX(below) work what I want : y(t) = f { y(t-1) , x(t-3) } ?