These are buffer overrun protection methods, and have nothing to do with compiler optimisation. MSVC will (if you specify the [`/GS`][1] switch) push a security cookie onto the stack near the return address so that it can detect a common case of stack corruption.
Stack corruption can either be caused by bad code along the lines of:
char buff[5];
strcpy (buff, "Man, this string is waaay too long!!");
or by malicious users taking advantage of bad coding practices, like the use of `scanf ("%s", myBuff)` for user input. Carefully crafted attacks like that can suborn your program to do things you probably don't want it to.
By placing a cookie close to the return address, a large number of bugs (and attack vectors) can be prevented, simply due to the fact that the memory corruptions tend to be sequential in nature. In other words, if you've overwritten the return address, it's probably because you started writing on one side of the cookie and corrupted memory all the way up to the return address on the other side of the cookie (hence the cookie will be overwritten as well).
It doesn't catch _all_ bugs since you may have some code like:
char buff[5];
buff[87] = 'x';
which could potentially corrupt the return address _without_ touching the cookie. But it will catch all those malicious ones which rely on entering a longer string than expected, which corrupt up to the return address (including cookie).
The sequence you're probably seeing in the code is something like:
mov eax, dword ptr ds:___sec_cookie ; fixed value.
xor eax, ebp ; adjust based on base pointer.
mov [ebp+SOMETHING], eax ; store adjusted value.
which is customising the cookie, depending on the current base pointer.
This will change what is actually put on the stack at each stack level (and also depending on parameter count and sizes as well) and is probably an attempt to further secure the code from malicious intent, by ensuring a _variable_ signature is written to the stack rather than a fixed value (otherwise the attacker could enter characters _including_ a valid cookie).
And the sequence at the end will run something like this:
mov ecx, [ebp+SOMETHING] ; get the adjusted cookie.
xor ecx, ebp ; un-adjust it, since
; ((N xor X) xor X) == N.
call @__sec_check_cookie ; check the cookie.
It's basically just the reverse process of that described above. The `@__sec_check_cookie` call will only return if `ecx` is set to the correct cookie value. Otherwise it will raise a fault, as confirmed [here][2]:
> The `__security_check_cookie()` routine is straightforward: if the cookie was unchanged, it executes the `RET` instruction and ends the function call. If the cookie fails to match, the routine calls `report_failure()`.
>
> The `report_failure()` function then calls `__security_error_handler()`. Both functions are defined in the `seccook.c` file of the C run-time (CRT) source files.
>
> CRT support is needed to make these security checks work. When a security check failure occurs, control of the program is passed to `__security_error_handler()`, which is summarized here:
void __cdecl __security_error_handler(int code, void *data)
{
if (user_handler != NULL) {
__try {
user_handler(code, data);
} __except (EXCEPTION_EXECUTE_HANDLER) {}
} else {
//...prepare outmsg...
__crtMessageBoxA(
outmsg,
"Microsoft Visual C++ Runtime Library",
MB_OK|MB_ICONHAND|MB_SETFOREGROUND|MB_TASKMODAL);
}
_exit(3);
}
> By default, an application that fails a security check displays a dialog that states "Buffer overrun detected!". When the dialog is dismissed, the application terminates.
[1]:
[To see links please register here]
[2]:
[To see links please register here]
]
