Bayes Theorem

Bayes’ theorem is used to describe the probability of an event. It is defined as:

$$p(y|x)=\frac{p(x|y)p(y)}{p(x)}$$

How do I use it?

In order to use bayes theorem to guess the class of each data point in a given dataset, we need to estimate

1. The class priors: $p(y)=\frac{N_y}{N}$ where $N$ is the size of our dataset.
2. Unconditional Class Probabilities: $p(\mathbf{x})={\sum }_{i=1}^{C}p(\mathbf{x}|y_i)p(y_i)$. This too is easy to calculate, since it does not require any assumptions.
3. The class conditional probability: $p(\mathbf{x}|y)$. In order to calculate this one, we need to make an assumption for the distribution of each class $y_i$.

Usually, for the class conditional, gaussian distribution is used. The dimensionality of the distribution depends on the dimensions of each data point $\mathbf{x}$.