Khatri-Rao Product

• This is a block Kronecker product.
• Given block matrices
  $$\mathbf{A} = \begin{pmatrix} \mathbf{A}_{11}& \mathbf{A}_{12}\\ \mathbf{A}_{21} & \mathbf{A}_{22} \end{pmatrix}$$

their product is

• Unlike the Kronecker product, the Khatri-Rao product is commutative at the outer level (though it is non-commutative overall because the Kronecker product is). Instead of constructing this 2x2 block matrix, we could have built the 4x4 block matrix that contains all pairwise Kronecker products, i.e., a 'double Kronecker product'. This is called the Tracy-Singh product; it is just the Kronecker product applied at the block level:

• By contrast we can think of the Khatri-Rao product as an 'elementwise' Kronecker product, generalizing the elementwise product of two non-block matrices:
  $$A * B = \begin{pmatrix}a_{11}b_{11} & a_{12}b_{12}\\a_{21}b_{21} & a_{22}b_{22}\end{pmatrix}$$
• Put differently, the Khatri-Rao product picks out the 'obvious' sub-block from each outer block of the Tracy-Sign product.