# Matrix Calculus Some useful rules that applies to matrix derivatives (in no particular order): ## Basic $\frac{\partial \mathbf{\mathbf{A x}}}{\partial \mathbf{x}}=\mathbf{A}$ $ \frac{\partial \mathbf{x}^T \mathbf{A}}{\partial \mathbf{x}} = \mathbf{A}^T $ $ \frac{\partial \mathbf{x}^T \mathbf{a}}{\partial \mathbf{x}}=\frac{\partial \mathbf{a}^T \mathbf{x}}{\partial \mathbf{x}}=\mathbf{a} $ $ \frac{\partial \mathbf{\mathbf{A x}}}{\partial \mathbf{z}}=\frac{\partial \mathbf{A x}}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{z}}=\mathbf{A} \frac{\partial \mathbf{x}}{\partial \mathbf{z}} $ ## Quadratic $ \frac{\partial \mathbf{a}^T \mathbf{X} \mathbf{b}}{\partial \mathbf{X}}=\mathbf{a b}^T $ $ \frac{\partial \mathbf{a}^T \mathbf{X}^{T} \mathbf{b}}{\partial \mathbf{X}}=\mathbf{b a}^T $ $ \frac{\partial \mathbf{y}^{T} \mathbf{A} \mathbf{x}}{\partial \mathbf{y}}= (\mathbf{A x})^T = \mathbf{x}^{T} \mathbf{A}^{T} $ $ \frac{\partial \mathbf{x}^{T} \mathbf{A} \mathbf{x}}{\partial \mathbf{x}}=\left(\mathbf{A}+\mathbf{A}^{T}\right)\mathbf{x} $ If $\mathbf{A}$ is symmetric, $ \frac{\partial \mathbf{x}^{T} \mathbf{A} \mathbf{x}}{\partial \mathbf{x}}=2 \mathbf{x}^{T} \mathbf{A} $ $ \frac{\partial \mathbf{x}^{T} \mathbf{x}}{\partial \mathbf{z}}=2 \mathbf{x}^{T} \frac{\partial \mathbf{x}}{\partial \mathbf{z}} $ --- ## References 1. Great summary of matrix derivatives from https://atmos.washington.edu/~dennis/MatrixCalculus.pdf The above rules are extracted from here.