orthogonal matrix with different size of rows and columns

how to generate orthogonal matrix with different size of rows and columns?

Answer 1

[KALYAN ACHARJYA]

%Answer Credit: StackFlow site
 A=randn(m, n);  % random mxn rows x col
 R=orth( A.' ).'; % orthogonal rows
 O_Mat=bsxfun(@rdivide, R, sqrt( sum( R.^2, 2 ) ) )

[John D'Errico]

You can feel free to give credit to StackOverFlow, but you should always think about what you read and learn on the internet, BEFORE you accept it as truth.

For example, suppose n = 5, m = 4? Will this code work? Sorry, but not gonna happen in this universe. In fact, the answer is NOT possible, and provably so! Don't believe me? Try an example, then think about what we havehere.

n = 4;
m = 5;

A=randn(m, n);  % random mxn rows x col

So A has 5 rows, 4 columns. What happens when we call orth as you did?

R=orth( A.' ).'; % orthogonal rows

R
R =
     0.2678       0.8739     -0.34303      0.21661
      0.779      -0.2298      0.32758      0.48274
   -0.51161    -0.092235    -0.098559      0.84855
   -0.24433      0.41832      0.87482  -0.00023289

What size is R? R is 4x4. We can stop right here, because R has the wrong size. Your final result will also be square. FAIL.

So now, ask yourself why did that code fail? Why was the SOFLOW response flat out wrong?

It was wrong because orth produces a set of vectors that is orthogonal in 4 dimensions. You cannot have 5 vectors that are fully orthogonal to each other in 4 dimensions. That will be true whenever m>n, for ANY size of matrix.

Do I need to prove this is a problem when m<n too? Well, as long as all you want are 4 orthogonal rows, it works fine. You can't ALSO have 5 orthogonal columns.

m = 4;
n = 5;
A=randn(m, n);  % random mxn rows x col
size(A)
ans =
     4     5

R=orth( A.' ).'; % orthogonal rows
size(R)
ans =
     4     5

In fact, the rows of R are orthogonal.

R*R'
ans =
            1  -2.0817e-16  -1.1102e-16  -1.3878e-16
  -2.0817e-16            1  -2.2204e-16   1.6653e-16
  -1.1102e-16  -2.2204e-16            1  -6.9389e-17
  -1.3878e-16   1.6653e-16  -6.9389e-17            1

But not the columns of course, since you cannot have a matrix that is non-square and has both rows and columns orthogonal. I'll just show you now they are not.

R'*R
ans =
       0.8969      -0.2516     -0.15032    -0.040757    -0.070078
      -0.2516      0.38601     -0.36683    -0.099465     -0.17102
     -0.15032     -0.36683      0.78084    -0.059425     -0.10217
    -0.040757    -0.099465    -0.059425      0.98389    -0.027704
    -0.070078     -0.17102     -0.10217    -0.027704      0.95237

Did your computation of O_mat do anything at all? In fact, it was just blowing smoke.

O_Mat=bsxfun(@rdivide, R, sqrt( sum( R.^2, 2 ) ) )
O_Mat =
     -0.33468    -0.098195      0.25797     -0.62104      0.65277
       0.7042     -0.51496      0.45349     -0.17237    -0.059621
     -0.25458     -0.33314      0.35023      0.74259      0.38744
      0.47348     -0.01401     -0.62127      0.13058       0.6104

What was R?

R
R =
     -0.33468    -0.098195      0.25797     -0.62104      0.65277
       0.7042     -0.51496      0.45349     -0.17237    -0.059621
     -0.25458     -0.33314      0.35023      0.74259      0.38744
      0.47348     -0.01401     -0.62127      0.13058       0.6104

I'll be a horned toad! In fact, R and O_Mat are identical!

norm(O_Mat - R)
ans =
   7.9898e-16

Well, virtually so, except for some floating point trash in the least significant bits when you created O_Mat.

So that last line was totally worthless. Why? Because R already had rows that had unit norm. That is what ORTH does!

Remember where I computed

R*R'
ans =
            1  -2.0817e-16  -1.1102e-16  -1.3878e-16
  -2.0817e-16            1  -2.2204e-16   1.6653e-16
  -1.1102e-16  -2.2204e-16            1  -6.9389e-17
  -1.3878e-16   1.6653e-16  -6.9389e-17            1

The main diagonal of R*R' is all 1. So scaling the rows of R to have unit norm is a waste of time.

So, I cannot say that what you read was truly just blowing smoke, because you did not give a link. Maybe you misread/misunderstood it and misrepresented what they said. However, the computation of O_Mat is indeed blowing smoke, because it does nothing of any value. And depending on how many rows and columns are to be in the matrix, the result will sometimes be flat out not what was asked for.

I'm sorry. But if you post something that is mathematically wrong, I feel it is necessary to tell people why that is so.

Answer 2

[John D'Errico]

So can you do what you want? If your goal is to find a set of n vectors of length d, such that they are orthogonal to each other, then as long as n is NO larger than d, you can do so, and it is trivial to do that.

So if you want a nxd array, for example with 2 vectors of length 3. The answer is as simple as virtually one line of code.

n = 2;
d = 3;

R = orth(randn(d,n))
R =
     -0.70798    -0.026483
      0.64575      -0.4289
     -0.28596     -0.90296

To show that is so, we simply compute R'*R.

R'*R
ans =
            1   5.5511e-17
   5.5511e-17            1

The result is an identity matrix, to within floating point trash.

The columns of R are orthogonal, because that is what orth does, what it is designed to do. They have unit norm, which we see because the diagonal of R'*R is all 1.

But, what happens when n>d? Can we have 3 vectors in a 2 dimensional space, all of which are orthogonal?

The answer is no. If n>d, regardless of the size, as long as n>d, we can never find a set of n vectors in a d-dimensional space that are orthogonal. If we try the orth trick, it will produce an array of size d by d, thus a SQUARE matrix. This is true because d vectors will always be sufficient be needed to span a d-dimensional vector space. Any more vectors than that can always be represented as some linear combination of a set of d orthogonal vectors. So we cannot have 3 vectors that are orthogonal to each other in 2 dimensions. This comes from linear algebra 101.

The point is, if you wanted a 2x3 array, that has 3 orthogonal columns, orth will fail. ANY code that you ever write will fail as well. It is just linear algebra 101 thatyou have to blame here.

We can try it:

n = 3;
d = 2;

R = orth(randn(d,n))
R =
     -0.69907      0.71505
      0.71505      0.69907

But ORTH understands that it can always find an orthogonal d-dimensional spanning set for any set of n vectors (where n>d), that has size no larger than d.

So ORTH takes a set of n vectors in d dimensions where n>d, but they will always reduce to at MOST d vectors that will span that space. It cannot take more than d vectors to span that vector space of dimension d.

Let me make it clear. You CANNOT create a matrix of size d by n with n>d, so more columns than rows, that has orthogonal columns. It is not just that the trick I showed here using ORTH for the other case will not work. It is that ANYTHING you do, must fail for this case. But if n<=d, ORTH will do just fine.