The mathematical term regularization has two main meanings, both associated with making a function more `regular' or smooth.
Regularization in physics
In physics, especially quantum field theory, regularization is a method of dealing with infinite, divergent, and non-sensical expressions by introducing an auxiliary concept of a regulator (for example, the minimal distance ε in space which is useful if the divergences arise from short-distance physical effects). The correct physical result is obtained in the limit in which the regulator goes away (in our example, 
), but the virtue of the regulator is that for its finite value, the result is finite.
However, the result usually includes terms proportional to expressions like 1 / ε which are not well-defined in the limit . Regularization is the first step towards obtaining a completely finite and meaningful result; in quantum field theory it must be usually followed by a related, but independent technique called renormalization. Renormalization is based on the requirement that some physical quantities - expressed by seemingly divergent expressions such as 1 / ε - are equal to the observed values. Such a constraint allows one to calculate a finite value for many other quantities that looked divergent.
Regularization of ill-posed problems
Inverse problems are often ill-posed. To solve these problems numerically one must introduce some additional information about the solution, such as an assumption on the smoothness or a bound on the norm.
The same idea arose in many fields of science. A simple form of regularization applied to integral equations, generally termed Tikhonov regularization after Andrey Nikolayevich Tychonoff, is essentially a trade-off between fitting the data and reducing a norm of the solution. In statistics a similar concept was introduced about the same time for finite-dimensional problems, where it is known as ridge regression.