Shellcoding¶
Common¶
Platform-neutral material (CPU architecture, general technique).
Register Layout¶
Each 64-bit register can be accessed at smaller widths. Writing to a 32-bit
sub-register (e.g. eax) zero-extends into the full 64-bit register; writing to
16-bit or 8-bit sub-registers leaves the upper bits unchanged.
| 64-bit | 32-bit | 16-bit | 8-bit high | 8-bit low |
|---|---|---|---|---|
| rax | eax | ax | ah | al |
| rbx | ebx | bx | bh | bl |
| rcx | ecx | cx | ch | cl |
| rdx | edx | dx | dh | dl |
| rsi | esi | si | - | sil |
| rdi | edi | di | - | dil |
| rbp | ebp | bp | - | bpl |
| rsp | esp | sp | - | spl |
| r8 | r8d | r8w | - | r8b |
| r9 | r9d | r9w | - | r9b |
| r10 | r10d | r10w | - | r10b |
| r11 | r11d | r11w | - | r11b |
| r12 | r12d | r12w | - | r12b |
| r13 | r13d | r13w | - | r13b |
| r14 | r14d | r14w | - | r14b |
| r15 | r15d | r15w | - | r15b |
al/ax/eaxare the low 8/16/32 bits ofrax;ahis bits 8-15.- Legacy high-byte regs (
ah,bh,ch,dh) only exist for the original four registers and can't be mixed with REX-prefixed operands. - Prefer the smallest width that holds your value. For example,
mov al, 59avoids null bytes thatmov rax, 59would introduce.
Linux¶
Syscalls¶
| Arch | Instruction | Number | Args | Return |
|---|---|---|---|---|
| x86-64 | syscall | rax | rdi, rsi, rdx, r10, r8, r9 | rax |
| x86 (32-bit) | int 0x80 | eax | ebx, ecx, edx, esi, edi, ebp | eax |
| ARM64 | svc #0 | x8 | x0-x5 | x0 |
| ARM32 | svc 0 | r7 | r0-r5 | r0 |
- Assemble .asm:
gcc -nostdlib -static shellcode.s -o shellcode - Extract shellcode:
objcopy --dump-section .text=shellcode-raw shellcode -
Analyze shellcode:
objdump -M intel -d shellcode -
Shellcode Loader for debugging
- Run:
cat shellcode-raw | ./tester
page = mmap(0x1337000, 0x100, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 0,0);
read(0, page, 0x1000);
((void(*)())page)();
Filter Bypass¶
- Write bytes and patch during execution to achieve syscall
# translates to code 0f 05 => syscall
inc BYTE PTR [rip+6]
inc BYTE PTR [rip+1]
.byte 0x0e
.byte 0x04
- Long NOP sleds
Windows¶
Same CPU (see Register Layout), but Windows differs from Linux in how functions and the kernel are called.
Calling convention (Microsoft x64)¶
| Purpose | Registers / location |
|---|---|
| Integer args | rcx, rdx, r8, r9, then stack (right-to-left) |
| Float args | xmm0–xmm3 |
| Return | rax |
| Shadow space | caller reserves 32 bytes on stack before every call |
| Alignment | rsp 16-byte aligned at the call site |
| Callee-saved | rbx, rbp, rdi, rsi, rsp, r12–r15 |
- The 32-byte shadow space (a.k.a. home space) must exist even if the callee
takes ≤4 args:
sub rsp, 0x28is the usual prologue (0x20 shadow + 8 to re-align after the return address).
Calling conventions (x86 / 32-bit)¶
Unlike x64 (one unified convention), 32-bit Windows uses several. The convention determines arg passing and who cleans the stack.
| Convention | Args | Stack cleanup | Used by |
|---|---|---|---|
| stdcall | all on stack, right-to-left | callee (ret N) |
most WinAPI functions |
| cdecl | all on stack, right-to-left | caller | C runtime, varargs (wsprintf) |
| fastcall | ecx, edx, then stack (right-to-left) | callee | some internal/perf functions |
| thiscall | ecx = this, rest on stack |
callee | C++ non-static methods |
- Return value in eax (edx:eax for 64-bit values).
- Callee-saved: ebx, esi, edi, ebp.
- No shadow space (that's an x64 thing).
- The caller/callee cleanup split matters for ROP/stack-pivot math: a
stdcallcallee pops its own args withret N, shifting esp for you.
Syscalls vs. WinAPI¶
- Linux syscall numbers are a stable ABI; Windows
Nt*/Zw*syscall numbers are undocumented and change between OS builds. Hardcoding them is fragile. - Real shellcode therefore calls documented WinAPI functions
(
WinExec,CreateProcessA,LoadLibraryA,VirtualAlloc, …) instead of issuing rawsyscallinstructions. - Direct/indirect syscalls (resolving the number at runtime from
ntdll) are an EDR-evasion technique, not the default calling mechanism.
Resolving APIs via the PEB¶
There is no syscall table to index; shellcode locates functions by walking loaded modules in memory. The TEB/PEB live at fixed segment-relative offsets:
| Arch | Segment | TEB | PEB |
|---|---|---|---|
| x64 | gs | gs:[0x30] |
gs:[0x60] |
| x86 | fs | fs:[0x18] |
fs:[0x30] |
-
fs/gsalready point at the TEB base, so[0]is the TEB start. The TEB slot above is the self-pointer (NT_TIB.Self), the TEB's own flat address, for when you need it as a normal pointer. The PEB pointer sits at+0x60(x64) /+0x30(x86) from the TEB base. -
Get the PEB:
gs:[0x60](x64) /fs:[0x30](x86). - Follow
PEB->Ldr->InMemoryOrderModuleListto findkernel32.dll's base. - Parse its PE export directory to resolve
GetProcAddress/LoadLibraryA. -
Use those to resolve and call everything else.
-
Functions are matched by hashing export names (avoids embedding ASCII strings and null bytes).