Why is it called a cap?

At international level, when a player makes an appearance, it is called a cap. Because traditionally they would have received a cap every time they played.