The objective of this paper is to use some historical instances to explain in detail the meaning of mathematical physics theories in selected historical periods. The paper presents, in order, the first instances of applications of mathematics to physics, the first appearance of something resembling modern mathematical physics and a particular kind of mathematical physics theory, called rational mechanics. At the end of the paper, epistemological considerations clarify the difference between physical, mathematical, and mathematical physics theories.