Batch and Online learning are two different approaches in machine learning.
Batch learning algorithm takes all the available training data at once and trains a model on it. In other words, the algorithm processes a fixed set of training data before making any predictions. Batch algorithms are suited for training on a large amount of data, and generally result in more accurate models since the entire dataset is used for training. A well-known example of batch learning is Linear Regression, where the model is trained on a set of features and their corresponding labels in order to predict labels for new samples.
Mathematically, Batch learning algorithms attempt to minimize the sum of the error differences of the predicted outputs and the actual outputs for all the training data points. In Linear Regression, for example, the error is defined as the difference between the actual output and the predicted output, which leads to the objective function of minimizing the sum of squared errors, as shown below:
$J(\theta) = \frac{1}{2m} \sum_{i=1}^{m}(h_{\theta}(x^{(i)}) - y^{(i)})^{2}$
Online learning, on the other hand, involves training the model on the data as it comes in, one instance at a time. The algorithm updates the model parameters for every instance it observes, so the model can adapt to new data quickly. In contrast to batch learning, online learning is suitable for scenarios where training data is constantly changing or for data sets that are too large to fit into memory at once.
Mathematically, Online learning algorithms minimize the error difference for each training data point separately. The objective function for this type of algorithm would look like this:
J(θ) = (hθ(x(i)) − y(i))2
In summary, batch learning is typically slower since it uses all the available data to train the model, but it tends to result in more accurate models. Online learning is faster, but its accuracy can be affected if the incoming data contains a lot of noise or outliers that the algorithm has not seen before.