Algebra Formulas

📌 Basic Algebra

Distributive Law:
\( a(b + c) = ab + ac \)
Commutative Addition:
\( a + b = b + a \)
Commutative Multiplication:
\( ab = ba \)
Associative Addition:
\( (a + b) + c = a + (b + c) \)
Associative Multiplication:
\( (ab)c = a(bc) \)
Additive Identity:
\( a + 0 = a \)
Multiplicative Identity:
\( a \cdot 1 = a \)
Additive Inverse:
\( a + (-a) = 0 \)
Multiplicative Inverse:
\( a \cdot \frac{1}{a} = 1 \)
Difference of Squares:
\( a^2 - b^2 = (a-b)(a+b) \)
Sum of Cubes:
\( a^3 + b^3 = (a+b)(a^2 - ab + b^2) \)
Difference of Cubes:
\( a^3 - b^3 = (a-b)(a^2 + ab + b^2) \)
Square of Sum:
\( (a + b)^2 = a^2 + 2ab + b^2 \)
Square of Difference:
\( (a - b)^2 = a^2 - 2ab + b^2 \)
Cube of Sum:
\( (a + b)^3 = a^3 + 3a^2b + 3ab^2 + b^3 \)
Cube of Difference:
\( (a - b)^3 = a^3 - 3a^2b + 3ab^2 - b^3 \)
Exponent Rules:
\( a^m \cdot a^n = a^{m+n} \), \( \frac{a^m}{a^n} = a^{m-n} \), \( (ab)^n = a^nb^n \), \( \left(\frac{a}{b}\right)^n = \frac{a^n}{b^n} \), \( (a^m)^n = a^{mn} \), \( a^{-n} = \frac{1}{a^n} \), \( a^{1/n} = \sqrt[n]{a} \), \( a^{m/n} = \sqrt[n]{a^m} \)
Absolute Value:
\( |a| = \begin{cases} a & \text{if } a \geq 0 \\ -a & \text{if } a < 0 \end{cases} \), \( |ab|=|a||b| \), \( \left|\frac{a}{b}\right|=\frac{|a|}{|b|} \)
Factorials:
\( n! = n \times (n-1)! \), \( 0! = 1 \), \( _nP_k = \frac{n!}{(n-k)!} \)
Arithmetic Sequence:
\( a_n = a_1 + (n-1)d \), \( S_n = \frac{n}{2}(a_1 + a_n) \)
Geometric Sequence:
\( a_n = a_1r^{n-1} \), \( S_n = a_1\frac{r^n - 1}{r - 1} \)

📌 Quadratic Equations

Standard Form:
\( ax^2 + bx + c = 0 \)
Quadratic Formula:
\( x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} \)
Vertex Form:
\( y = a(x-h)^2 + k \)
Vertex Coordinates:
\( \left( -\frac{b}{2a}, -\frac{D}{4a} \right) \)
Axis of Symmetry:
\( x = -\frac{b}{2a} \)
Discriminant:
\( D = b^2 - 4ac \)
Nature of Roots:
\( D > 0 \): Two distinct real roots, \( D = 0 \): One real root, \( D < 0 \): Two complex conjugate roots
Sum of Roots:
\( \alpha + \beta = -\frac{b}{a} \)
Product of Roots:
\( \alpha\beta = \frac{c}{a} \)
Parabola Opening:
\( a > 0 \): Opens upward, \( a < 0 \): Opens downward
Focus of Parabola:
\( \left( h, k + \frac{1}{4a} \right) \)
Directrix of Parabola:
\( y = k - \frac{1}{4a} \)
Roots as Factors:
\( ax^2 + bx + c = a(x - \alpha)(x - \beta) \)
Completing the Square:
\( x^2 + bx = \left(x + \frac{b}{2}\right)^2 - \frac{b^2}{4} \)
Quadratic-Linear System:
Solve \( y = ax^2 + bx + c \) and \( y = mx + k \)
Maximum/Minimum Value:
\( y = -\frac{D}{4a} \)
Graph Transformations:
\( y = a(x-h)^2 + k \): - Vertical stretch by \( |a| \), - Reflect over x-axis if \( a < 0 \), - Shift \( h \) units horizontally, - Shift \( k \) units vertically

📌 Polynomials

General Form:
\( P(x) = a_nx^n + a_{n-1}x^{n-1} + \cdots + a_1x + a_0 \)
Degree:
Highest power of \( x \) with non-zero coefficient
Remainder Theorem:
\( P(a) = \) remainder when \( P(x) \) ÷ \( (x-a) \)
Factor Theorem:
\( (x-a) \) is factor ⇨ \( P(a) = 0 \)
Synthetic Division:
Quick division by \( (x - c) \)
Rational Root Theorem:
Possible roots = \( \pm \frac{\text{factors of } a_0}{\text{factors of } a_n} \)
Descartes' Rule:
Number of positive roots = sign changes or less by even number
Fundamental Theorem:
Every non-constant polynomial has at least one complex root
Complex Roots:
Non-real roots come in conjugate pairs
Multiplicity:
If \( (x-a)^k \) is factor, \( a \) has multiplicity \( k \)
End Behavior:
- Leading term \( a_nx^n \): \( n \) even: Ends same direction, \( n \) odd: Ends opposite directions
Graph:
A smooth continuous curve with at most \( n-1 \) turning points
Binomial Expansion:
\( (x + y)^n = \sum_{k=0}^n \binom{n}{k}x^{n-k}y^k \)
Taylor Polynomial:
\( P(x) = \sum_{k=0}^n \frac{f^{(k)}(a)}{k!}(x-a)^k \)
Legendre Polynomials:
Solutions to Legendre's equation
Chebyshev Polynomials:
\( T_n(x) = \cos(n \arccos x) \)

📌 Logarithms

Definition:
\( \log_b a = c \iff b^c = a \)
Common Logarithm:
\( \log_{10} a = \log a \)
Natural Logarithm:
\( \log_e a = \ln a \)
Change of Base:
\( \log_b a = \frac{\log_k a}{\log_k b} \)
Product Rule:
\( \log_b (xy) = \log_b x + \log_b y \)
Quotient Rule:
\( \log_b \left( \frac{x}{y} \right) = \log_b x - \log_b y \)
Power Rule:
\( \log_b (x^n) = n \log_b x \)
Log of 1:
\( \log_b 1 = 0 \)
Log of Base:
\( \log_b b = 1 \)
Inverse Property:
\( b^{\log_b a} = a \)
Log of Reciprocal:
\( \log_b \left( \frac{1}{x} \right) = -\log_b x \)
Log of Root:
\( \log_b \sqrt[n]{x} = \frac{1}{n} \log_b x \)
Exponential-Log Relationship:
\( \log_b (b^x) = x \)
Logarithmic Differentiation:
\( \frac{d}{dx} \ln x = \frac{1}{x} \)
Logarithmic Integration:
\( \int \frac{1}{x} dx = \ln |x| + C \)
Logarithmic Inequalities:
\( \log_b x > \log_b y \iff x > y \) (if \( b > 1 \)), \( \log_b x > \log_b y \iff x < y \) (if \( 0 < b < 1 \))
Logarithmic Limits:
\( \lim_{x \to 0^+} \ln x = -\infty \), \( \lim_{x \to \infty} \ln x = \infty \)
Logarithmic Series Expansion:
\( \ln(1 + x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \cdots \) (for \( |x| < 1 \))
Logarithmic Identities:
\( \log_b (x^m \cdot y^n) = m \log_b x + n \log_b y \), \( \log_b \left( \frac{x^m}{y^n} \right) = m \log_b x - n \log_b y \)
Logarithmic Equations:
\( \log_b x = \log_b y \iff x = y \), \( \log_b x = k \iff x = b^k \)

📌 Matrices

Matrix Definition:
\( A = [a_{ij}]_{m \times n} \)
Matrix Addition:
\( [A + B]_{ij} = A_{ij} + B_{ij} \)
Matrix Subtraction:
\( [A - B]_{ij} = A_{ij} - B_{ij} \)
Scalar Multiplication:
\( [kA]_{ij} = k \cdot A_{ij} \)
Matrix Multiplication:
\( [AB]_{ij} = \sum_{k=1}^n A_{ik} B_{kj} \)
Identity Matrix:
\( I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \)
Zero Matrix:
\( 0 = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} \)
Transpose:
\( [A^T]_{ij} = A_{ji} \)
Determinant (2x2):
\( \det(A) = ad - bc \) for \( A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \)
Determinant (3x3):
\( \det(A) = a(ei - fh) - b(di - fg) + c(dh - eg) \) for \( A = \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix} \)
Inverse (2x2):
\( A^{-1} = \frac{1}{\det(A)} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix} \)
Inverse (3x3):
Use cofactor matrix and adjugate
Trace:
\( \text{tr}(A) = \sum_{i=1}^n A_{ii} \)
Diagonal Matrix:
\( D = \text{diag}(d_1, d_2, \dots, d_n) \)
Symmetric Matrix:
\( A = A^T \)
Skew-Symmetric Matrix:
\( A = -A^T \)
Orthogonal Matrix:
\( A^T = A^{-1} \)
Eigenvalues:
\( \det(A - \lambda I) = 0 \)
Eigenvectors:
\( (A - \lambda I)\mathbf{v} = 0 \)
Rank:
Number of linearly independent rows/columns
Nullity:
\( \text{nullity}(A) = n - \text{rank}(A) \)
Matrix Power:
\( A^k = A \cdot A \cdots A \) (\( k \) times)
Matrix Exponential:
\( e^A = \sum_{k=0}^\infty \frac{A^k}{k!} \)
Singular Matrix:
\( \det(A) = 0 \)
Non-Singular Matrix:
\( \det(A) \neq 0 \)
Cramer's Rule:
\( x_i = \frac{\det(A_i)}{\det(A)} \) for \( A\mathbf{x} = \mathbf{b} \)

📌 Vector Algebra

Vector Definition:
\( \mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} \)
Vector Addition:
\( \mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ u_3 + v_3 \end{bmatrix} \)
Vector Subtraction:
\( \mathbf{u} - \mathbf{v} = \begin{bmatrix} u_1 - v_1 \\ u_2 - v_2 \\ u_3 - v_3 \end{bmatrix} \)
Scalar Multiplication:
\( k\mathbf{v} = \begin{bmatrix} k v_1 \\ k v_2 \\ k v_3 \end{bmatrix} \)
Dot Product:
\( \mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + u_3 v_3 \)
Dot Product (Geometric):
\( \mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\| \|\mathbf{v}\| \cos \theta \)
Cross Product:
\( \mathbf{u} \times \mathbf{v} = \begin{bmatrix} u_2 v_3 - u_3 v_2 \\ u_3 v_1 - u_1 v_3 \\ u_1 v_2 - u_2 v_1 \end{bmatrix} \)
Cross Product (Geometric):
\( \|\mathbf{u} \times \mathbf{v}\| = \|\mathbf{u}\| \|\mathbf{v}\| \sin \theta \)
Magnitude (Norm):
\( \|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + v_3^2} \)
Unit Vector:
\( \hat{v} = \frac{\mathbf{v}}{\|\mathbf{v}\|} \)
Projection of \( \mathbf{u} \) onto \( \mathbf{v} \):
\( \text{proj}_{\mathbf{v}} \mathbf{u} = \left( \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{v}\|^2} \right) \mathbf{v} \)
Scalar Triple Product:
\( \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) \)
Vector Triple Product:
\( \mathbf{u} \times (\mathbf{v} \times \mathbf{w}) = (\mathbf{u} \cdot \mathbf{w}) \mathbf{v} - (\mathbf{u} \cdot \mathbf{v}) \mathbf{w} \)
Angle Between Vectors:
\( \theta = \cos^{-1} \left( \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|} \right) \)
Orthogonal Vectors:
\( \mathbf{u} \cdot \mathbf{v} = 0 \)
Parallel Vectors:
\( \mathbf{u} \times \mathbf{v} = \mathbf{0} \)
Line Equation (Vector Form):
\( \mathbf{r} = \mathbf{r}_0 + t\mathbf{d} \)
Plane Equation (Vector Form):
\( \mathbf{n} \cdot (\mathbf{r} - \mathbf{r}_0) = 0 \)
Distance Between Points:
\( d = \|\mathbf{v} - \mathbf{u}\| \)
Distance from Point to Line:
\( d = \frac{\|\mathbf{v} \times \mathbf{d}\|}{\|\mathbf{d}\|} \)
Distance from Point to Plane:
\( d = \frac{|\mathbf{n} \cdot (\mathbf{r}_0 - \mathbf{p})|}{\|\mathbf{n}\|} \)
Area of Parallelogram:
\( \|\mathbf{u} \times \mathbf{v}\| \)
Volume of Parallelepiped:
\( |\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w})| \)
Direction Cosines:
\( \cos \alpha = \frac{v_1}{\|\mathbf{v}\|}, \cos \beta = \frac{v_2}{\|\mathbf{v}\|}, \cos \gamma = \frac{v_3}{\|\mathbf{v}\|} \)
Vector Differentiation:
\( \frac{d\mathbf{v}}{dt} = \begin{bmatrix} \frac{dv_1}{dt} \\ \frac{dv_2}{dt} \\ \frac{dv_3}{dt} \end{bmatrix} \)

📌 Complex Numbers

Definition:
\( z = a + bi \), where \( i = \sqrt{-1} \)
Real Part:
\( \text{Re}(z) = a \)
Imaginary Part:
\( \text{Im}(z) = b \)
Complex Conjugate:
\( \overline{z} = a - bi \)
Modulus:
\( |z| = \sqrt{a^2 + b^2} \)
Argument:
\( \arg(z) = \theta = \tan^{-1}\left(\frac{b}{a}\right) \)
Polar Form:
\( z = r(\cos \theta + i \sin \theta) \), where \( r = |z| \)
Euler's Formula:
\( e^{i\theta} = \cos \theta + i \sin \theta \)
Exponential Form:
\( z = re^{i\theta} \)
Addition:
\( (a + bi) + (c + di) = (a + c) + (b + d)i \)
Subtraction:
\( (a + bi) - (c + di) = (a - c) + (b - d)i \)
Multiplication:
\( (a + bi)(c + di) = (ac - bd) + (ad + bc)i \)
Division:
\( \frac{a + bi}{c + di} = \frac{(a + bi)(c - di)}{c^2 + d^2} \)
Reciprocal:
\( \frac{1}{z} = \frac{\overline{z}}{|z|^2} \)
Power:
\( z^n = r^n (\cos(n\theta) + i \sin(n\theta)) \)
Roots:
\( z^{1/n} = r^{1/n} \left[ \cos\left(\frac{\theta + 2k\pi}{n}\right) + i \sin\left(\frac{\theta + 2k\pi}{n}\right) \right] \), \( k = 0, 1, \dots, n-1 \)
De Moivre's Theorem:
\( (\cos \theta + i \sin \theta)^n = \cos(n\theta) + i \sin(n\theta) \)
Logarithm:
\( \ln(z) = \ln(r) + i(\theta + 2k\pi) \), \( k \in \mathbb{Z} \)
Exponential Function:
\( e^{z} = e^{a}(\cos b + i \sin b) \)
Trigonometric Functions:
\( \sin z = \frac{e^{iz} - e^{-iz}}{2i} \), \( \cos z = \frac{e^{iz} + e^{-iz}}{2} \), \( \tan z = \frac{\sin z}{\cos z} \)
Hyperbolic Functions:
\( \sinh z = \frac{e^{z} - e^{-z}}{2} \), \( \cosh z = \frac{e^{z} + e^{-z}}{2} \), \( \tanh z = \frac{\sinh z}{\cosh z} \)
Complex Equations:
\( z^n = w \) has \( n \) distinct roots
Complex Plane:
\( z = (a, b) \) in \( \mathbb{C} \)
Vector Representation:
\( z = a\mathbf{i} + b\mathbf{j} \)
Fundamental Theorem of Algebra:
Every non-constant polynomial has at least one complex root

📌 Binomial Theorem

Binomial Theorem:
\( (a + b)^n = \sum_{k=0}^n \binom{n}{k} a^{n-k} b^k \)
Binomial Coefficient:
\( \binom{n}{k} = \frac{n!}{k!(n-k)!} \)
Pascal's Identity:
\( \binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k} \)
Symmetry Property:
\( \binom{n}{k} = \binom{n}{n-k} \)
Sum of Coefficients:
\( \sum_{k=0}^n \binom{n}{k} = 2^n \)
Alternating Sum:
\( \sum_{k=0}^n (-1)^k \binom{n}{k} = 0 \)
Vandermonde's Identity:
\( \sum_{k=0}^r \binom{m}{k} \binom{n}{r-k} = \binom{m+n}{r} \)
Hockey-Stick Identity:
\( \sum_{k=r}^n \binom{k}{r} = \binom{n+1}{r+1} \)
Binomial Expansion for Negative Exponents:
\( (1 + x)^{-n} = \sum_{k=0}^\infty \binom{-n}{k} x^k \)
Generalized Binomial Theorem:
\( (a + b)^\alpha = \sum_{k=0}^\infty \binom{\alpha}{k} a^{\alpha-k} b^k \), \( \alpha \in \mathbb{R} \)
Multinomial Theorem:
\( (a_1 + a_2 + \dots + a_m)^n = \sum_{k_1 + k_2 + \dots + k_m = n} \frac{n!}{k_1! k_2! \dots k_m!} a_1^{k_1} a_2^{k_2} \dots a_m^{k_m} \)
Binomial Recurrence:
\( \binom{n+1}{k} = \binom{n}{k} + \binom{n}{k-1} \)
Exponential Generating Function:
\( \sum_{n=0}^\infty \frac{x^n}{n!} \binom{n}{k} = \frac{x^k}{k!} e^x \)
Binomial Probability:
\( P(k) = \binom{n}{k} p^k (1-p)^{n-k} \)
Central Binomial Coefficient:
\( \binom{2n}{n} \)
Catalan Numbers:
\( C_n = \frac{1}{n+1} \binom{2n}{n} \)
Stirling's Approximation:
\( \binom{n}{k} \approx \frac{n^k}{k!} \) for large \( n \)
Binomial Integral Representation:
\( \binom{n}{k} = \frac{1}{2\pi i} \oint \frac{(1+z)^n}{z^{k+1}} dz \)
Binomial Matrix:
\( \begin{bmatrix} \binom{0}{0} & 0 & 0 \\ \binom{1}{0} & \binom{1}{1} & 0 \\ \binom{2}{0} & \binom{2}{1} & \binom{2}{2} \end{bmatrix} \)
Binomial Series:
\( (1 + x)^n = \sum_{k=0}^\infty \binom{n}{k} x^k \)

📌 Probability & Combinatorics

Probability of Event:
\( P(A) = \frac{\text{Number of favorable outcomes}}{\text{Total number of outcomes}} \)
Complementary Probability:
\( P(A^c) = 1 - P(A) \)
Union of Events:
\( P(A \cup B) = P(A) + P(B) - P(A \cap B) \)
Mutually Exclusive Events:
\( P(A \cup B) = P(A) + P(B) \)
Conditional Probability:
\( P(A | B) = \frac{P(A \cap B)}{P(B)} \)
Independent Events:
\( P(A \cap B) = P(A) \cdot P(B) \)
Bayes' Theorem:
\( P(A | B) = \frac{P(B | A) P(A)}{P(B)} \)
Law of Total Probability:
\( P(A) = \sum_{i=1}^n P(A | B_i) P(B_i) \)
Permutations:
\( P(n, k) = \frac{n!}{(n-k)!} \)
Combinations:
\( C(n, k) = \binom{n}{k} = \frac{n!}{k!(n-k)!} \)
Multinomial Coefficient:
\( \frac{n!}{k_1! k_2! \dots k_m!} \)
Binomial Probability:
\( P(k) = \binom{n}{k} p^k (1-p)^{n-k} \)
Geometric Probability:
\( P(k) = (1-p)^{k-1} p \)
Poisson Probability:
\( P(k) = \frac{\lambda^k e^{-\lambda}}{k!} \)
Hypergeometric Probability:
\( P(k) = \frac{\binom{K}{k} \binom{N-K}{n-k}}{\binom{N}{n}} \)
Expected Value:
\( E(X) = \sum_{i=1}^n x_i P(x_i) \)
Variance:
\( \text{Var}(X) = E(X^2) - [E(X)]^2 \)
Standard Deviation:
\( \sigma = \sqrt{\text{Var}(X)} \)
Covariance:
\( \text{Cov}(X, Y) = E(XY) - E(X)E(Y) \)
Correlation:
\( \rho(X, Y) = \frac{\text{Cov}(X, Y)}{\sigma_X \sigma_Y} \)
Markov's Inequality:
\( P(X \geq a) \leq \frac{E(X)}{a} \)
Chebyshev's Inequality:
\( P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2} \)
Central Limit Theorem:
\( \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim N(0, 1) \)
Normal Distribution:
\( f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \)
Exponential Distribution:
\( f(x) = \lambda e^{-\lambda x} \)
Uniform Distribution:
\( f(x) = \frac{1}{b-a} \) for \( x \in [a, b] \)
Gamma Distribution:
\( f(x) = \frac{\lambda^k x^{k-1} e^{-\lambda x}}{\Gamma(k)} \)
Beta Distribution:
\( f(x) = \frac{x^{\alpha-1} (1-x)^{\beta-1}}{B(\alpha, \beta)} \)
Chi-Square Distribution:
\( f(x) = \frac{x^{k/2-1} e^{-x/2}}{2^{k/2} \Gamma(k/2)} \)
Student's t-Distribution:
\( f(x) = \frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\sqrt{\nu\pi} \Gamma\left(\frac{\nu}{2}\right)} \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu+1}{2}} \)
F-Distribution:
\( f(x) = \frac{\sqrt{\frac{(d_1 x)^{d_1} d_2^{d_2}}{(d_1 x + d_2)^{d_1 + d_2}}}}{x B\left(\frac{d_1}{2}, \frac{d_2}{2}\right)} \)
Moment Generating Function:
\( M_X(t) = E(e^{tX}) \)
Characteristic Function:
\( \phi_X(t) = E(e^{itX}) \)
Joint Probability:
\( P(A \cap B) = P(A) P(B | A) \)
Marginal Probability:
\( P(A) = \sum_{i=1}^n P(A \cap B_i) \)
Conditional Expectation:
\( E(X | Y) = \sum_{x} x P(X = x | Y) \)
Law of Large Numbers:
\( \lim_{n \to \infty} \frac{1}{n} \sum_{i=1}^n X_i = \mu \)
Stirling's Formula:
\( n! \approx \sqrt{2\pi n} \left(\frac{n}{e}\right)^n \)
Birthday Problem:
\( P(\text{No match}) = \frac{365!}{(365-n)! 365^n} \)
Monty Hall Problem:
\( P(\text{Win}) = \frac{2}{3} \)
Gambler's Ruin:
\( P(\text{Ruin}) = \frac{1 - \left(\frac{q}{p}\right)^N}{1 - \left(\frac{q}{p}\right)^M} \)
Entropy:
\( H(X) = -\sum_{i=1}^n P(x_i) \log P(x_i) \)
Kullback-Leibler Divergence:
\( D_{KL}(P \parallel Q) = \sum_{i=1}^n P(x_i) \log \frac{P(x_i)}{Q(x_i)} \)
Mutual Information:
\( I(X; Y) = \sum_{x,y} P(x, y) \log \frac{P(x, y)}{P(x)P(y)} \)
Conditional Entropy:
\( H(X | Y) = -\sum_{x,y} P(x, y) \log P(x | y) \)
Joint Entropy:
\( H(X, Y) = -\sum_{x,y} P(x, y) \log P(x, y) \)
Information Gain:
\( IG(X, Y) = H(X) - H(X | Y) \)
Poisson Process:
\( P(N(t) = k) = \frac{(\lambda t)^k e^{-\lambda t}}{k!} \)
Markov Chains:
\( P(X_{n+1} = x | X_n = x_n, \dots, X_0 = x_0) = P(X_{n+1} = x | X_n = x_n) \)
Stationary Distribution:
\( \pi = \pi P \)
Ergodic Theorem:
\( \lim_{n \to \infty} \frac{1}{n} \sum_{k=1}^n f(X_k) = E(f(X)) \)