RC01: Some InconsistenC's
I'm attending the Fall 1 batch at Recurse Center! Posts in this series cover things I'm working on or find interesting during my time here.
Note: Commands preceeded with a
$are Linux commands, while commands preeceded with
%are macOS commands.
I came across an older CTF writeup of mine - There might be cake from Hacky Holidays 2021. The solution for it was this C program:
At the time, I tried running this on my machine (macOS) and received the wrong output.
% gcc -o pin pin.c % ./pin 444334282
One of my teammates happened to solve the challenge. Our programs were the same, but he solved it on a Linux machine.
$ gcc -o pin pin.c $ ./pin 1376299761
Same code, different output? I think we attributed the issue to some differences in implementations on different operating systems, I'm not sure. Regardless, I left it there and didn't look into it further.
However, upon seeing the writeup recently, I got curious and decided to figure out why this happened, and if there was a way to get the correct output on macOS.
gcc isn't the issue #
During the search process, I was shocked to find out that Apple actually executes
clang whenever you type
gcc in the terminal.
% gcc clang: error: no input files % gcc --version Apple clang version 13.1.6 (clang-13188.8.131.52.5) Target: arm64-apple-darwin21.5.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Thinking that this was the issue, I installed
gcc on my machine via Homebrew to have the same compiler on both operating systems.
% brew install gcc
The Homebrew version of
gcc contains the version number (
gcc-13 in this case), most likely to prevent conflict with the built-in
% gcc-13 --version gcc-13 (Homebrew GCC 13.1.0) 13.1.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I thought this would solve my issue, but nope, the output is the same.
% gcc-13 -o pin pin.c % ./pin 444334282
The libc looks different #
I was reading through a post from Julia Evans before starting RC - Behind Hello World on Linux - which briefly mentions about libc, the C standard library. These lines stood out:
But there are different libc implementations, and sometimes they behave differently. The two main ones are glibc (GNU libc) and musl libc.
rand() functions come from this library, so maybe they're implemented differently in macOS and Linux. This explains why the previous step to change my compiler didn't work.
My Linux VM uses glibc, which I can confirm by running
ldd on the binary.
$ ldd pin linux-vdso.so.1 (0x00007ffcb23d4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbc44b9f000) /lib64/ld-linux-x86-64.so.2 (0x00007fbc44d9b000)
This commands checks for the libraries that a binary needs to load at runtime.
libc.so.6 is the glibc file.
For macOS, I used the command
otool -L as suggested by Julia at the end of her post.
% otool -L pin pin: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
In macOS, libc is part of Apple's system library, called libSystem. It seems to be Apple's own implementation, as I found a link to the libc source code while searching.
So different implementations mean there is no way I could get the same output on both Linux and Mac? That doesn't seem right.
I came across a pair of similar functions in my searches -
random(). In their man page on macOS, they're termed as the "better random number generator"
NAME initstate, random, setstate, srandom, srandomdev – better random number generator; routines for changing generators
I replaced the functions in my code on macOS, and now I get the correct output!
% gcc-13 -o pin pin.c % ./pin 1376299761
So how is it better? According to the man page, while
random() work in a similar way, certain implementations of
rand() generate less random numbers:
The random() and srandom() functions have (almost) the same calling sequence and initialization properties as the rand(3) and srand(3) functions. The difference is that rand(3) produces a much less random sequence — infact, the low dozen bits generated by rand go through a cyclic pattern. All of the bits generated by random() are usable. For example, ‘random()&01’ will produce a random binary value.
random() are also not part of the C Standard. They come from the POSIX Standard, which contains additional functions for C.
Okay, so use
srandom() instead of
random() instead of
rand() and problem solved, right? Technically yes, but I still had one more question.
I went ahead and tested the newly found functions on Linux, and they return the correct output. While this is expected, why does
srand()/rand() return the incorrect output only on macOS?
I think this is where the standards matter. Here's a snippet of the man pages for
srand()/rand() from both operating systems:
Different standards likely mean different implementations, which explains the difference in output.
The standard mentioned in the macOS code, ISO C90, is an older standard, so the implementation might be outdated, and hence is declared obsolete in the man pages. This isn't the case for the Linux version though, which is interesting.
The man pages also seem to have quite a few differences in both OS's, the major one being that the macOS pages date back to the 1990s, while the Linux ones are from this year.