For old-school CD keys, it was just a matter of making up an algorithm for which CD keys (which could be any string) are easy to generate and easy to verify, but the ratio of valid-CD-keys to invalid-CD-keys is so small that randomly guessing CD keys is unlikely to get you a valid one.
Starcraft and Half-life both used the same checksum, where the 13th digit verified the first 12. Thus, you could enter anything for the first 12 digits, and guess the 13th (there's only 10 possibilities), leading to the infamous 1234-56789-1234
The algorithm for verifying is public, and looks something like this:
x = 3;
for(int i = 0; i < 12; i++)
{
x += (2 * x) ^ digit[i];
}
lastDigit = x % 10;
Windows XP takes quite a bit of information, encrypts it, and puts the letter/number encoding on a sticker. This allowed MS to both verify your key and obtain the product-type (Home, Professional, etc.) at the same time. Additionally, it requires online activation.
The full algorithm is rather complex, but outlined nicely in this (completely legal!) paper, published in Germany.
Of course, no matter what you do, unless you are offering an online service (like World of Warcraft), any type of copy protection is just a stall: unfortunately, if it's any game worth value, someone will break (or at least circumvent) the CD-key algorithm, and all other copyright protections.
For online-services, life is a bit simpler, since even with the binary file you need to authenticate with their servers to make any use of it (eg. have a WoW account). The CD-key algorithm for World of Warcraft - used, for instance, when buying playtime cards - probably looks something like this:
- Generate a very large cryptographically-secure random number.
- Store it in our database and print it on the card.
Then, when someone enters a playtime-card number, check if it's in the database, and if it is, associate that number with the current user so it can never be used again.
For online services, there is no reason not to use the above scheme; using anything else can lead to problems.