13337: The Final Boss

A multi-stage challenge involving self-modifying code, network handshakes, and cryptographic brute-forcing.

Referred to as the 'Final Boss' of the competition, this challenge merged reverse engineering with network protocols. Initial execution yielded nothing until a quick strace revealed the binary was silently attempting to connect to 127.0.0.1 on port 13337. The challenge expected a server to talk to.

Opening the stripped binary in Ghidra revealed a wall of obfuscation. Strategic mprotect calls and XOR routines signaled self-modifying code. The binary dynamically decrypted its own logic at runtime using a static key (0xDE) combined with a randomized stabilization key.

Using pwngdb, I bypassed the fog of war by placing a breakpoint immediately after the mprotect call to dump the freshly decrypted memory. Reversing the exposed logic revealed a complex network handshake requiring a magic DWORD (0x0DF7B6FA). Even after establishing execution flow, the binary heavily mangled the payload, XORing the incoming buffer with a session ID and routing it through MD5 hash validations.

Since the validator processed the payload in 4-byte chunks, I realized the resulting MD5 hashes could be trivially brute-forced on a modern CPU. By cracking the hashes, mapping the multi-round custom ciphers via GDB watchpoints, and executing a synchronized server script to handle the handshake and payload delivery, the final flag was exposed.


Key Takeaways

  • Hardware breakpoints and watchpoints in gdb are essential for dissecting self-modifying code dynamically after memory decryption.
  • Cryptographic checks operating on tiny structures (e.g., 32-bit integers) are susceptible to rapid brute-force attacks, regardless of the hashing algorithm used.

Beyond the challenge

Imagine a suspicious client that appears idle until a local service answers it. Process tracing can expose the missing dependency, while a memory capture can reveal code that was unreadable on disk. This is why an analyst combines file inspection with observations from a controlled run.

Keep the recovered protocol separate from assumptions about its strength. A hash check over a small input space can be searched because the candidates are limited, but the actual cost depends on the space and implementation. Measure that cost rather than assuming every 32-bit search is instant.

Keep learning

  • GDB manual: practice examining memory and setting watchpoints after code has been decoded.
  • Ghidra beginner guide: follow data through the recovered handshake logic.
Recovered flag Reveal spoilerC0ngratulations_U_3arn3d_1t_19432850&

Thanks for reading.

Back to articles