In this tutorial, I will go over an easy way to learn how assembly and binaries work on your own. I will show you a technique you can use to teach yourself on a practical level how assembly and binaries work. This is meant as an introduction, and will not go into detail on assembly itself (perhaps another tutorial!). However, by the end of this tutorial you will be able to create and answer some of your own questions regarding binary analysis and programming.
Environment
For this tutorial, you will need two things. First, a program called radare2. Second, a compiling environment and ability to compile a hello world program. While this can work on many environments, my environment of choice is Ubuntu, so I will be demonstrating on that.
Radare2
If you've looked into at least a few demos, you have likely heard of the tool radare2. Radare2 is touted as a "portable reversing framework". It is a shining achievement of the open source community coming together to provide a tool that is effective in a much needed area. In my opinion, it is a competitor (although not yet peer) of the venerable IDA Pro. However, where IDA Pro will tend to run you $1,000 or more, radare2 is completely free. To set it up, simply follow the instructions. It's so easy, I'm not going to even go into it here.
Compiling Environment
Most modern Operating Systems come with compilation support by default. However, on the off chance that it doesn't, you will need to install the GNU C Compiler (gcc). It is possible (and encouraged!) to perform these same tests with other compilers such as the LLVM compilers, however they will require slightly different command line flags. Again, likely this pre-requisite is already on your machine.
The Idea
Let's take a look at how binaries are compiled and look at the machine level. For most aspiring reverse engineers and hackers, this is a daunting step. As mentioned in the intro, I will not be going into a primer on assembly language. However, I believe this technique should allow even a beginner to get a better understanding for the inner workings of their binaries.
The concept is that we create a simple program. In our example it will be a "Hello World!" program. We will then compile it using the GNU C Compiler using a flag that tells the compiler to add in debugging information. From there, we will open it up in Radare2 and check out what it's doing.
Lets Go!
First off, we need to write our program. Here's one we'll use. It's a "Hello World!" program, but it has a few twists that are interesting.
#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
bool hello() {
printf("Hello ");
return true;
}
int main()
{
bool x;
x = hello();
if (x)
printf("World!\n");
else
printf("Why am I here??");
exit(0);
}
So this app will always print out "Hello World!\n". However, to make it more interesting, it's calling a function, as well as comparing a Boolean value. Now that we've written our masterpiece, we will compile it!
$ gcc -g hello_world.c
This will create an "a.out" file (default name for a compiled binary). Notice the "-g" switch which indicates to the compiler that we should add in debugging information. Also note that this will build to your default architecture. On a 64-bit Ubuntu, it will build a 64-bit binary. You can force it to create a 32-bit bit binary by adding the flag "-m32". You can also compile it for an entirely different architecture (called cross-compiling), but that's outside of the scope of this tutorial.
At this point, I must inform you that I am by no means a master of radare2. I know enough to get by and my default platform is usually IDA Pro. With that in mind, I will show you how I go about viewing the binary, and the radare2 curmudgeons will have to bite their tongues. :-)
Let's open up this binary in radare2:
$ r2 a.out -- Emulate the base address of a file with e file.baddr. [0x004004c0]>
Upon opening, it hasn't performed any analysis. You probably want it to figure a few things out, so let's tell it to go hog wild and analyze this binary.
[0x004004c0]> aaaa [x] Analyze all flags starting with sym. and entry0 (aa) [x] Analyze len bytes of instructions for references (aar) [x] Analyze function calls (aac) [x] Emulate code to find computed references (aae) [x] Analyze consecutive function (aat) [aav: using from to 0x400000 0x4026a0 Using vmin 0x400000 and vmax 0x601050 aav: using from to 0x400000 0x4026a0 Using vmin 0x400000 and vmax 0x601050 [x] Analyze value pointers (aav) [x] Constructing a function name for fcn.* and sym.func.* functions (aan)
No real need to worry about the details of what it's doing. I definitely recommend checking out their documentation if you're interested. The next series of commands will bring you to the graphical representation of the main function:
VV o main
The first command, "VV", will tell radare2 to go into the so-called "visual visual" mode. All this means is you get a pretty function picture very similar to what IDA Pro will give you. This view is my favorite as it allows me to see graphically what the application is doing. The second command, "o main", tells radare2 to go to "offset main". In this case, radare2 understands when we say "main" that we mean the main function for the program (i.e.: int main). Not to go into details, but this won't always work. In our case, however, it will.
You should now see a pretty picture like the following:
The top of this is the intro to main(). And follows down to the exit. Note how radare2 has incorporated the source code into this view for us. This is how you can begin to understand what's happening from a macro level. Let's walk through this example:
"sym.main" means that we have a known symbol called main. What's a symbol? Generally these are going to be your function names. In this case, "main" is the main function of the program. If you look further down, "call sym.hello" instruction is an instruction to call our function named "hello". Nice and simple, right?
push rbp move rbp, rsp sub rsp, 0x10
You will see this sequence a lot. Note that next to it, radare2 put "{". This is the function prologue. It's a way that assembly language performs book keeping and sets up room for your variables. For the purposes of this tutorial, it's enough to simple notice it, and move on.
mov eax, 0 call sym.hello mov byte [rbp - local_1h], al
Note that here, radare2 is telling us this is "x = hello()". When you see the c code referenced (such as at the top line of this example), this means that starting at that line is the c statement shown. This is important to distinguish since C is a higher level language and often takes a few assembly instructions for one C instruction. In this case, the compiler has chosen to set eax to 0 (compiler convention in this case, it doesn't really have anything to do with our code), then call the "hello()" function. Finally, it moves al (by convention, this is what is returned from the function), into rbp - local_1h.
I should note here, that when you see "local_1h", what this means is that it's a so-called local variable (i.e.: on the stack). Radare2 is helping us in understanding that we are saving this return value to a variable on the stack. Given how compilers work, you won't generally get a nice name to go with it. We can tell from inference that "local_1h" is our variable "x".
cmp byte [rbp - local_1h], 0
je 0x4005f7
This is "if (x)". Remember, we just saved our variable "x" into local_1h. In this case, it compares this value to the static value 0 (or False), and jumps to different code based on the result. This is the standard way of handling any if statement. Also notice that we coded it in the affirmative (if x is true then). The compiler doesn't have to follow this, and in our case didn't. The compiler reversed it and said "if x is false then". These two statements achieve the same result, so don't assume that the compiler will choose the same direction that you did.
Take a close look at the branches. They both start by moving the address of our respective string into edi (again, calling convention), however the left branch uses printf while the right branch uses puts. To be honest, I'm not sure why it chose to use two different print functions, but again the end result is the same behavior we're asking for, simply with different function calls. Finally, it moves 0 to edi and exits (exit(0)).
Let's quickly take a peek at our hello function. You can do this by typing:
o sym.hello
This is very similar to the previous, so i will not go over it. However, notice "mov eax, 1" at the end shows "return true". This lines up with the other part of the code using eax as the return value from this function.
Optimize 1
Just for fun, let's take a look at what happens when we ask gcc to optimize our code. This is done by passing the flag "-O1".
$ gcc -g -O1 hello_world.c
Repeat the same steps as above to get back to the visualization in radare2:
The first thing to note is that this is significantly different than the first compile. A fair amount of code has been removed. Also, the nice C code lines are gone too!! Well... unfortunately, when we told gcc it had the liberty to make things better, in doing so we gave up our right to be able to add debug symbols properly. There are various reasons, but bottom line is don't expect to have the nice debugging symbols when you use optimization. That said, we kept the name (main()/hello()) so it's not all bad.
See what optimizations you can find before moving on... So one nice optimization it did was with the return variable from hello(). Note in the un-optimized version we allocated space for a variable on the stack (memory), then copied the return value to that spot in memory and compared it. While not a big deal in small amounts, memory isn't where we really want to be for performance. GCC understood this, and in optimization mode 1 it decided to not bother writing this return value to memory at all. Why? Well it doesn't need to. It can keep the value in memory and use the "test" instruction to see if it's true or false. It also decided to cut down on the prologue/epilogue and other minor things in there. Again, this isn't a huge gain, but it is certainly an optimization.
It also decided to change the printf statement into printf_chk. This is an equivalent function to printf that performs a safety check. Again, the compiler made a choice but it does not have any adverse affects on the functionality of this application.
Optimize 2
GCC actually has many different optimizations it can attempt. However, it's possible some might adversely affect your application. While compilers do their best to be faithful to what you're asking them to do, they're not perfect. Default compiles are very stable in giving you exactly what you're asking for. However, optimizations can definitely cause inadvertent things to happen. If you're interested, there's a whole field of copmiler theory for you to get into! Also, to look at what optimizations are possible in gcc, try the following:
gcc --help=optimizers
For now, let's try the set of optimizations titled "O2". This should theoretically give us better performance than the set O1.
gcc -g -O2 hello_world.c
That's it. GCC is getting cheeky with us. It has discovered there is a code path that it doesn't believe can happen (in this case it's right), and decided to go it's own way and remove that if statement of ours completely! A quick look at our code indicates GCC has a valid point that that code path likely should never exist. It also entirely removed our "hello()" function and inlined it. The output it creates is significantly more optimized than where we started.
This is a good point to show where optimization can backfire. It's possible that we can add some sort of alarm/signal/thread situation in this case that is non-deterministic and will only handle properly with the original if statement intact. Those types of things are very difficult for compilers to take into account and can lead to cases. It's actually happened on more than a few occasions where the optimization has actually hurt security. For instance, timing attacks on crypto, and clearing out memory for security.
Conclusion
I hope by this point I have equipped the reader with a technique for investigating how binaries work, as well as a curiosity in the subject. If you have comments or if there's something you're interested in learning about, please drop a line in the feedback form at the top.