The function system() provided by the standard C library (libc) takes a string argument that is passed as command string to a shell command language interpreter such as sh (or bash). The use of this function is generally considered dangerous because the shell is a complex application that uses many implicit transformation rules. In addition, its behavior is controlled by several environment variables. In order to make a safe call to system(), the input has to be rigorously sanitized and the environment has to be sane. Even then, problems in the program(s) invoked by the shell through system() can be abused to compromise the calling application (e.g., you remember old days Amin talking about parameter injection stuff?).


After your last assignment, your new boss is not impressed yet. However, she thinks that you might have what it takes to be a good security engineer. Your next assignment takes you to Bostom University. Apparently, they have been hacked, once again and have contacted your company for help. You logon to the Linux server and see a number of applications that look like they may have vulnerabilities. These programs are written in C, so you first use grep to search for known insecure library functions such as gets(), strcpy(), or system(). After a brief search, you find three simple programs that use the system() function. You immediately bring them to the attention of your boss. Unfortunately, after a brief check, she cannot find anything wrong with these programs. She claims that the calls to system() and dlopen() were performed in a safe manner, and she orders you to continue working and stop bothering you without having something real. However, you know that these programs all have flaws. Demonstrate the vulnerabilities by exploiting each program and show your boss what your Florida International University secure programming education has taught you ;-).


Your first task is to exploit vulnerabilities in three programs that have their set-guid (i.e. set group identification) bit enabled. The programs are installed under /usr/local/bin/prog[1-3]. The source for the programs can be obtained here (not necessarily listed in order):


An enabled set-guid bit means that whenever you execute one of these programs, your process gets the effective group-id of the group that owns the file. Consider a file called “myProg” with the following access permissions shown with ls -la.

-rwxr-sr-x 1 boss panther 8192 Jan 1 2021 myProg

Whenever a user that belongs to the “other” group (i.e. not user boss and not belonging to group panther) executes this file, the process is executed with an effective group-id of panther and may access all resources according to the restrictions for group panther.

You have exploited a vulnerability in one of our three challenge programs successfully when you call /bin/grader with the effective group-id of the group that owns the vulnerable program (for our challenge, these are groups bsp[1-3]). In the example above, “myProg” would be considered to be exploited successfully when you are able to call (or force “myProg” to call) /bin/grader with an effective guid of panther. In that case, you receive a message stating that you have solved the assignment and get a code. This code has to be included in your submission to prove to us that your exploit was successful. Don’t try to fake, cheat or steal this code.


To submit your challenge solution to us, you need to follow these steps:

  1. Create a file called message.txt anywhere under your account (e.g., using vi).
  2. Write each code that you have received from /bin/grader for every program you exploited (i.e., prog[1-3]) on a single line in that file (make sure the ordering 1-3 is correct).
  3. The message.txt should be signed with your private GPG key, and encrypted using the class GPG public key. Your submission folder should contain the following file:
  4. You submit your project by running the turn-in script as follows:
    $ /course/cn5079sp22/bin/turnin project2 <project directory>

    where <project directory> is the name of the directory with your submission(the directory where your message.txt.asc file is located). The script will print out every file that you are submitting, so make sure that it prints out all of the files you wish to submit! The turn-in script will not accept submissions that are missing message file. You may submit as many times as you wish; only the last submission will be graded, and the time of the last submission will determine whether your assignment is late.

  5. Wait a couple of minutes and call the grading script to view the results of the automatic grading program. If you have managed to solve all the three challenges, you will get the full credit.


Each answer in your message.txt should be on its own line. For example, your message.txt file might look like the following:



This project is worth 6% of your final grade, broken down as follows (out of 100):

  • 33.3% points each per answer (three answers)

Points can be lost for turning in files in incorrect formats (e.g. not UNIX-line break ASCII), failing to follow specified formatting and naming conventions, or encrypting/signing your file using the wrong keys.


Good luck and happy hunting!