Reverse engineering a shareware tool and writing a proper keygen for it

By: Azerton

This article deals with reverse engineering of a shareware tool and how to go about writing a proper keygen for it.

Contents:

0x1. Tools
0x2. Observing our victim
0x3. Jumping inside the code
a. Understanding how it works
b. Understanding the interesting code
c. Calculating the serial
0x4. Writing the keygen
0x5. Shouts0x1: Tools

The tools used in this tutorial are a brain, windows calc.exe (for hex-decimal
conversions for example), OllyDbg 1.10, Dev-C++ 4 for writing the keygen.

0x2: Observing our victim

When we start our victim without being registered before, a window pops up with a
reminder to register within our 30 days trial period. We can either click "ok" (to continue unregistered) or "register". When clicking the “register” button, a new window pops up asking us for a proper username and serial number. If the information we entered is wrong, a message box appears with the message “incorrect code”. If we enter a working username and password, the username and password we entered are written to the registry so we only have to enter the correct information once.

0x3: Jumping inside the code

a. Understanding how it works

I fire up OllyDbg and load the executable. The first thing is to check for all the imports
by pressing CTRL+N. A long list pops up but we are only interested in a few functions
(like functions who get user input, functions that pop up a message box telling us we
entered wrong information, functions that compare one string with another one, … We are not interested in functions that for example activate a button on the screen).
These are the interesting imported functions: (the complete list is much longer)

Address Section Type ( Name
004092D8 .idata Import ( USER32.GetDlgItemTextA
00409248 .idata Import ( KERNEL32.GetDriveTypeA
00409280 .idata Import ( KERNEL32.GetEnvironmentStrings
00409278 .idata Import ( KERNEL32.GetEnvironmentStringsW
00409220 .idata Import ( KERNEL32.lstrcmpA
00409210 .idata Import ( KERNEL32.lstrcmpiA
00409218 .idata Import ( KERNEL32.lstrcpyA
00409240 .idata Import ( KERNEL32.lstrlenA
00409334 .idata Import ( USER32.MessageBoxA
004092E0 .idata Import ( USER32.SetDlgItemTextA

I jump to the place in our code where USER32.GetDlgItemTextA is called by double
clicking on the first line in the names screen. GetDlgItemTextA is the function that will
probably be called when we click “ok” after entering a username and a password in the
“register” window.

This is where we arrive in the code:
00401280 /$ 81EC 00010000 SUB ESP,100

We set a breakpoint on 00401280 and start our application within OllyDbg. I click the
“register” button and the window pops up asking me for a username and a password. I enter “azerton”, password “112233” and press ok. Our program breaks (as aspected) on line 00401280 where GetDlgItemTextA is called. After this line the code gets interesting because this is where the code starts playing with the information we entered.

b. Understanding the interesting code

The interesting code is the code where the user input gets handled. This is what the
program does to verify our username and password:
-It checks if the username and password are long enough
-It calculates a password using the entered username
-It compares if the password you entered is equal to the calculated one
This is how a lot of username / password protection systems work and it’s important to
understand how they work.

The code after our breakpoint (00401280) looks like this :
00401280 /$ 81EC 00010000 SUB ESP,100
00401286 |. A0 64624000 MOV AL,BYTE PTR DS:[406264]
0040128B |. 884424 00 MOV BYTE PTR SS:[ESP],AL
0040128F |. 53 PUSH EBX
00401290 |. 56 PUSH ESI
00401291 |. 33C0 XOR EAX,EAX
00401293 |. 57 PUSH EDI

00401294 |. B9 3F000000 MOV ECX,3F
00401299 |. 8D7C24 0D LEA EDI,DWORD PTR SS:[ESP+D]
0040129D |. 55 PUSH EBP
0040129E |. F3:AB REP STOS DWORD PTR ES:[EDI]
004012A0 |. 66:AB STOS WORD PTR ES:[EDI]
004012A2 |. BD 6A000000 MOV EBP,6A
004012A7 |. 68 64624000 PUSH (censored).00406264
004012AC |. AA STOS BYTE PTR ES:[EDI]
004012AD |. 8BB424 1C01000> MOV ESI,DWORD PTR SS:[ESP+11C]
004012B4 |. 56 PUSH ESI
004012B5 |. FF15 D4924000 CALL DWORD PTR DS:[<&USER32.wsprintfA>
004012BB | MOV EBX,DWORD PTR SS:[ESP+11C]
004012C2 |. 83C4 08 ADD ESP,8
004012C5 |. 8BC3 MOV EAX,EBX
004012C7 |. 8B3D DC924000 MOV EDI,DWORD PTR DS:[<&USER32.CharNextA>
004012CD |. 803B 00 CMP BYTE PTR DS:[EBX],0
004012D0 |. 74 0F JE SHORT (censored).004012E1

I am not going to explain every line of code in detail but what happens of importance is that the username gets stored and checked (on line 004012CD the code checks if the username has at least one character. If not it takes the jump at 004012D0) and a
counter for a loop is prepared.

If everything goes well and your username passes the previous checks the code will
start calculating the valid password on the next lines. The serials used by the software cracked in this paper all look like this:
xxxx-xxxxx-xxxx-xxx

What I suspect is that the serial is calculated through 4 different loops where each of
these 4 loops creates a different part of the serial. Pew!

c. Calculating the serial

The code where the serial gets calculated follows immediately after 004012D0:
004012D2 |> 0FBE08 /MOVSX ECX,BYTE PTR DS:[EAX]
004012D5 |. 50 |PUSH EAX
004012D6 |. 8D6C4D 00 |LEA EBP,DWORD PTR SS:[EBP+ECX*2]
004012DA |. FFD7 |CALL EDI
004012DC |. 8038 00 |CMP BYTE PTR DS:[EAX],0
004012DF |.^75 F1 \JNZ SHORT (censored).004012D2

The above code generates the 4 first digits of our serial (part I) based on the username we entered. This is how it works:

The first time it loops, 004012D2 moves the first letter of our username (in my case “a”) onto ECX. The instruction on line 004012D5 contains the remaining characters to be processed (in the first loop “azerton”, the second one “zerton”, …).

The actual calculation is performed on line 004012D6: the operation between brackets
[EBP+ECX*2] means that the previous value of EBP is being read (in the first loop 6A,
106.) and ECX (the hexadecimal value of the character being processed in the loop)
times 2 is added to the existing value of EBP.

004012DA goes on to our next character (in my case “z”) and 004012DC compares if
there is still a character to process (to check if we are at the end of our username or
not). Depending on the test performed at this line, 004012DF will either jump (if there
are more characters) or “not jump” (continue to the next line) in case there are no more characters to process.

This is the calculation performed for username “azerton”:
- EBP = 6A + 61*2 (61h “a”) (->EBP is now 12C)
- EBP = 12C + 7A*2 (->EBP is now 220)
- EBP = 220 + 65*2 (->EBP is now 2EA)
- EBP = 2EA + 72 *2 (->EBP is now 3CE)
- EBP = 3CE + 74*2 (->EBP is now 4B6)
- EBP = 4B6 + 6F*2 (->EBP is now 594)
- EBP = 594 + 6E*2 (->EBP is now 670)
-
So the final value is 607. This is the first part of our serial. Be careful, 607 is the
hexadecimal value. The serial is decimal so we still have to convert it. Calc.exe tells us
that 670 is 1648. (the dot (.) after 1543 means we are working in the decimal system to avoid confusion with the hexadecimal system).

The next three parts of the serial get calculated using a similar loop as the one used for part I.

Part II:

00401308 |> 0FBE08 /MOVSX ECX,BYTE PTR DS:[EAX]
0040130B |. 03C9 |ADD ECX,ECX
0040130D |. 50 |PUSH EAX
0040130E |. 8D14C9 |LEA EDX,DWORD PTR DS:[ECX+ECX*8]
00401311 |. 03EA |ADD EBP,EDX
00401313 |. FFD7 |CALL EDI
00401315 |. 8038 00 |CMP BYTE PTR DS:[EAX],0
00401318 |.^75 EE \JNZ SHORT (censored).00401308

Try to understand what part II does. ADD ECX,ECX means ECX gets doubled.
For username “azerton”, part II is 3CA6 (15526.). Verify this.
Our serial looks like this after part II : 1648-15526-xxxx-xxx

Part III:

00401341 |> 0FBE08 /MOVSX ECX,BYTE PTR DS:[EAX]
00401344 |. 50 |PUSH EAX
00401345 |. 8D2C89 |LEA EBP,DWORD PTR DS:[ECX+ECX*4]
00401348 |. 8D0C69 |LEA ECX,DWORD PTR DS:[ECX+EBP*2]

0040134B |. 8D2C4D 0100000> |LEA EBP,DWORD PTR DS:[ECX*2+1]
00401352 |. FFD7 |CALL EDI
00401354 |. 8038 00 |CMP BYTE PTR DS:[EAX],0
00401357 |.^75 E8 \JNZ SHORT (censored).00401341

Three (very simple) calculations this time but notice that part III gets calculated
independently of part I and part II.
Verify for yourself that part III returns 975 (2421.).

Part IV (the final part):

Again, this one gets calculated without depending on the value returned by part I, II or

III. The code:
00401380 |> 0FBE08 /MOVSX ECX,BYTE PTR DS:[EAX]
00401383 |. 50 |PUSH EAX
00401384 |. 8D2C8D 1D00000> |LEA EBP,DWORD PTR DS:[ECX*4+1D]
0040138B |. FFD7 |CALL EDI
0040138D |. 8038 00 |CMP BYTE PTR DS:[EAX],0
00401390 |.^75 EE \JNZ SHORT (censored).00401380

Part IV returns 1D5 (469.). Verify this.
Our complete serial for username “azerton” looks like this now:
1648-15526-2421-469

0x4: Writing the keygen

Writing a keygen for this particular piece of software is pretty straight-forwarding. If you understand how the loops to calculate the serial work, you will have no problem writing a proper keygen for it.

This is what our keygen should do:
-Ask for a username
-Calculate the 4 parts of the serial and join them
-Return the serial to the user

I use Dev-C++ to write the keygen. This is the complete code:

0x5. Shouts

Shouts go out to l33tdawg, google and all the others
on the forums who try to keep knowledge free.

Cheers,
azerton

1.) Web Application Footprinting & Assessment with MSN Search - Shreeraj Shah
2.) Biometrics and You - Don Parker
3.) Review: Mac OS X x86 10.4.1 & 10.4.3 - L33tdawg
4.) eXploiting Local Stack on Windows - Nish Bhalla
5.) Reverse engineering a shareware tool and writing a proper keygen for it - azerton
6.) Story of a dumb patch - Cesar Cerrudo