Ray Solomonoff (born 1926) invented the concept of algorithmic probability around 1960.
Take a universal computer and randomly generate an input
program. The program will generate some possibly infinite
output.
The algorithmic probability of any given finite output
prefix q is the sum of the probabilities of the programs that
compute something starting with q.
Algorithmic probability is the main ingredient
of Solomonoff's theory of inductive inference,
the theory of prediction based on observations.
Given a sequence of symbols -- which will come next?
Solomonoff's theory provides an answer that is optimal
in a certain sense.
Unlike Karl Popper's informal theory, Solomonoff's
is mathematically sound.
Algorithmic probability is closely related to
the concept of Kolmogorov complexity. In fact,
Solomonoff was the first to prove the invariance theorem,
which shows that it is not really important which computer we
use.