One of the Linux machines belonging to honeyp.edu was recently broken in to, and a software tool was installed on it by the attacker. Upon detailed analysis, the security team found that the installed software (referred to hereafter as the "Prot11 zombie" or generically as the "malware") is primarily a Distributed Denial-of-Service (DDoS) attack tool, capable of using well-known attack techniques like DNS flood, IP fragment attacks, TCP Syn flood, as well as providing a backdoor into the compromised system. The Prot11 zombie has the capability to be remotely controlled by the attacker using raw IP protocol 11 packets. Methods that can be used to detect the presence of the Prot11 zombie include using network scanning tools (e.g. Nmap), examining systems with trusted security software, and using an Intrusion Detection System (IDS) configured to identify abnormal IP protocols. Measures that can thwart or reduce the effectiveness of attacks launched by such tools include egress filtering at Internet gateways and filtering out unused protocols at the firewall. This document contains technical information about the Prot11 zombie, and specific instructions on how to detect its presence. In the appendix, we give a full technical documentation of how the zombie works, including the data encoding method and commands used to control the zombie. With this information, packets captured by an intrusion detection system can be analyzed to determine what commands were performed on the compromised machine, giving some insight into what systems might have been attacked and some clues on where the attack originated. We give an example of such an analysis, using actual captured data, in the appendix.

How does the tool work?

The binary is an ELF executable, statically linked and stripped. While discovered on a Linux system, it wouldn't be surprising to find this tool on other systems: it could run directly on FreeBSD using Linux compatibility, and as it uses only standard socket calls it could easily be recompiled to run on just about any Unix-like system. The Prot11 zombie must be run as root, as it opens a raw socket to receive commands from the attacker.

The commands are sent to the zombie in the payload of an IP packet with protocol number field set to 11. Protocol 11 is assigned to the Network Voice Protocol (NVP), and is very rarely used. This communication method allows the packets to go unnoticed in many settings. For example, we have noticed that the default RedHat ipchains firewall settings only block unwanted TCP and UDP packets, and let everything else through (including unknown or unused protocols like NVP). The payload of the control packet contains a command code, followed by the parameters, followed by a random amount of padding. Padding is required because the binary processes the IP packet only if its length (including the IP header) is greater than or equal to 200 bytes.

This payload is not sent as cleartext. It is encoded before sending. The binary decodes it upon receipt. See the section Payload Encoding and Decoding in the Appendix for further details.

Because the communication technique is connectionless, the source IPs on the command packets may be forged, so they are not a reliable indication of the true attacker's location. Furthermore, the few commands that require a response may send the response to 10 different IP addresses, obscuring the true recipient of the response (although if the attacker is careless it is still possible to determine his or her IP address -- see the appendix for a real example of this).

The Prot11 zombie accepts 12 different commands, numbered 1-12, which are summarized below. The list below is simply to familiarize you with the capabilities of the Prot11 zombie, and more detailed information about how the commands are invoked and the options available can be found in the Appendix to this advisory. All the commands are executed in a forked subprocess. Except for commands 3 and 7, the child process is allowed to run indefinitely, and command 8 has to be used in order to kill the child process. In case of commands 3 and 7, the child process is automatically killed by the parent after 10 seconds and 20 minutes respectively.

Command code 1: Attack status query

Returns whether any attack is currently in progress, and if so returns the command number of the attack in progress.

Command code 2: Set communication parameters

Sets the IP address for replies. There following three options are supported:
Command code 3: Execute shell command (10 sec. timeout)
Command code 7: Execute shell command (20 min. timeout)
Both of these commands send a shell command to the binary, and instruct it to execute it on the victim machine, in a different subprocess.
Command code 4: DNS reply flood (slow)
Command code 9: DNS reply flood (rate adjustable)
The binary has a hardcoded list of 8000 name servers. This command instructs the binary to send DNS request packets to these name servers one by one. The attacker provides the DoS target's IP address as a parameter. The source IP address of the DNS request packets is set to this address. The DNS replies will all go to the this IP address, flooding it with data. The rate at which to send the packets is configurable in command 9.

Command code 5: Fragment flood

This command provides a target host name (or IP address), and the source IP address to spoof the packets with. The binary floods the target machine with identical IP packets. Each of the IP packets is crafted as a single fragment with a high fragment offset in a 64k IP packet. The IP stack on the target machine will think that it's trying to reassemble a 64k fragmented IP packet, and allocate memory for the reassembly. The remaining fragments never arrive. Allocating 64k for each of the IP packets will soon exhaust the memory of the IP stack, thus making it non-responsive to regular network traffic for some time. The packets can be configured to appear as ICMP (IP protocol 1) or UDP (IP protocol 17). Note that there will be no replies to these packets.

NB: This attack seems to be inspired from a security advisory distributed by the Razor team (NTBugtraq, 19-May-00) (MS00-029) about the Jolt2 DoS tool. A SANS institute article describing the same can be read online here, and the analyzed binary seems to in fact come from exactly this source code (the packets are created identically).

Command code 6: Execute shell

This instructs the binary to bind a root shell to a port number (23281) hardcoded in the binary. The listening process expects for a password to be entered first, and the shell is started only if the correct password (SeNiF) is sent.

Command code 8: Stop current attack

This instructs the binary to kill any attack currently in progress.

Command code 10: SYN flood (slow)
Command code 11: SYN flood (rate adjustable)

This is the standard SYN flood attack. The target IP address and target TCP port number is provided by the attacker. Source IP address of the SYN packets can either of the following: The rate at which to send the packets is configurable.

Command code 12: DNS request flood (rate adjustable)

The binary sends DNS requests (UDP port 53) to the target IP address provided by the attacker. As in the SYN flood attack described above, the source IP address can be set to a particular IP address, or can be random, in order to avoid flooding two machines. For this attack, even the source port can be set so as to appear random.

How can you detect the presence of such tools?

The system administrator should be aware of what services are supposed to be offered by a machine. Only the corresponding protocols (e.g. TCP which is IP protocol no. 6, UDP which IP protocol no. 17) and ports should be open on the machine. This can be checked from a "safe" system (one that is known to not be compromised) using scanning software like Nmap. If any protocols or ports are found open other than the authorized services, you are most likely running some attack tool. In particular, a "protocol scan" can be performed with Nmap to see if the machine is accepting IP protocol 11 packets, which is an indication of the Prot11 zombie running.

In the following example, we ran Nmap on a network with 4 machines, with one of them running the malicious binary. Now that we know this binary uses IP protocol 11, we looked for the presence of that. All the protocols can be scanned using -p1-255 in place of -p11 in the following command, although this can take quite a while. The command shown below shows an undocumented "data_length" parameter to the nmap command, which is necessary in this case, and which should be kept small (say under 100) so that if a machine is running the Prot11 zombie it will not try to interpret the packet as a command (recall that it throws out all packets with length less than 200):

    # nmap -sO -p11 --data_length=50
    Starting nmap V. 2.54BETA22 ( www.insecure.org/nmap/ )
    The 1 scanned port on machine1 ( is: closed
    The 1 scanned port on machine2 ( is: closed
    The 1 scanned port on machine3 ( is: closed
    Interesting protocols on machine4 (
            Protocol   State       Name
            11         open        nvp-ii
    Nmap run completed -- 4 IP addresses (4 hosts up) scanned in 2 seconds
This shows that the machine at is accepting protocol 11 packets which is an almost certain sign that the Prot11 zombie is running on that machine.

The suspected machine can also be examined to look for signs of the Prot11 zombie. However the results of any test run on a potentially compromised machine can only be properly interpreted as showing evidence of the zombie, and never as evidence that the system is clean! The reason for this is that if the machine has been compromised, it is possible (and likely) that the attacker installed a "rootkit" to hide their tracks while they installed the Prot11 zombie. A rootkit can replace the "ps" command to hide any executing process, such as the Prot11 zombie, or can replace "netstat" so that it doesn't show the raw socket listener. We recommend that a machine be examined with both a trusted OS kernel running, as well as trusted tools. We have had success using bootable CDs for this purpose, rebooting the machine to examine the disk contents -- the disadvantages of this process are that you lose any information about currently running processes, and you must bring the system down temporarily.

With the above warning in mind, the following can give evidence that the Prot11 zombie is running. The Prot11 zombie may also show up in a process listing (the output from ps -ef) using the program name [mingetty]. Note that "mingetty" is actually a legitimate program, but the true mingetty program will show up in the process listing without the square brackets. The following example shows some of the output from "ps" on a machine running the Prot11 zombie (and no rootkit to hide it!). In this case, the ps output shows the following (among other entries)

    root       977  0.0  0.0  1356    4 tty1     S    May13   0:00 /sbin/mingetty tty1
    root       978  0.0  0.0  1356    4 tty2     S    May13   0:00 /sbin/mingetty tty2
    root       979  0.0  0.0  1356    4 tty3     S    May13   0:00 /sbin/mingetty tty3
    root       980  0.0  0.0  1356    4 tty4     S    May13   0:00 /sbin/mingetty tty4
    root       981  0.0  0.0  1356    4 tty5     S    May13   0:00 /sbin/mingetty tty5
    root       982  0.0  0.0  1356    4 tty6     S    May13   0:00 /sbin/mingetty tty6
    root      5151  0.0  1.1 10544  720 ?        S    May17   0:03 /etc/X11/X :0 -auth /var/gdm/:0.Xauth
    gdm       5157  0.0  2.1  6892 1336 ?        S    May17   0:00 /usr/bin/gdmlogin --disable-sound --d
    root     16209  0.0  0.0   240   44 ?        S    10:09   0:00 [mingetty]
Note the difference between the mingetty processes started during boot, and the [mingetty] name assumed by the attack tool. In addition, netstat can be used to see if there is a process listening for raw protocol 11 packets.

If an unknown binary is discovered on your Linux system, it can be checked to determine if it is the Prot11 zombie. First, execute the following command from a Linux shell (note that the middle character of "TfOjG" is a capital letter O):

    strings unknown-binary | grep -C3 TfOjG
If the "unknown-binary" is the Prot11 zombie, the following "signature" output will be seen:
    /bin/csh -f -c "%s" 1> %s 2>&1

Finally, an Intrusion Detection System (IDS) can be configured to detect the control packets for the Prot11 zombie. In particular, the IDS should flag any IP protocol 11 packets as being suspicious (in general, all protocols and ports other than the useful ones should be flagged). It is also possible to detect connections to the backdoor shell on TCP port 23281. However, since this is an unprivileged port number, there may be other applications using the port which would cause some false positives.

When attacking other sites, this tool will produce a very high volume of network data. You might be able to notice that using tools like sysstat, which summarize your system activity.

The network traffic generated during attacks has the following typical behavior:

IDS signatures can be developed or purchased to detect this kind of traffic, and flag it as suspicious.

How can you defend against such attacks?

Obviously the best defense against the Prot11 zombie is to not have your systems broken into in the first place. Systems should be kept up-to-date with the latest security patches, and should be properly administered to achieve a reasonable level of security. However, despite the best efforts toward these goals, system security will occassionally be breached. To protect against the Prot11 zombie, or other DDoS tools, the following general techniques are useful:
  1. All protocols and ports other than the useful ones should be blocked at the firewall, so that the attacker cannot communicate with the attack tool to launch any attacks (in particular, blocking protocol 11 packets would stop the controller from contacting the Prot11 zombie). One excellent way to do this is to make sure your firewall rules have a default policy of rejecting packets, and specifically allow packets through that seem to be legitimate. This way, the firewall blocks communication channels that haven't been considered before as being a threat.

  2. Egress filtering at the routers will give you limited protection from the attacks. It will be limited because if the attacker realizes that the attacks are not working, he/she might configure the (spoofed) source IP address used in the attacks to be one of the addresses inside your network. This will of course increase the chance of detection of the tool.

  3. The network traffic generated during attacks has a typical behavior, and the attacks are well-known (Jolt2, SYN flood, etc.). IDS signatures can be developed or purchased (if not already present) to detect the kind of traffic, and alert you to ongoing attacks.

Appendix: Technical Details

In this appendix we describe the capabilities of the Prot11 zombie in detail, including the exact format of control and reply packets. With the information given here, captured traffic can be analyzed to see what was requested of the zombie. We give an example of an actual analysis below, and show how the analysis gives useful information on the location of the attacker.

All communication to and from the zombie is done with IP packets that have the protocol field set to 11 decimal. While there is a protocol designated with this number (the Network Voice Protocol), the packets constructed don't have any relation to this protocol, and the use of this protocol number seems to be an arbitrary choice of a rarely-used protocol number. The data portion of each packet (immediately following the IP header) has the following format:

Byte offsetDescription
0Packet Type: 2 for control packet, or 3 for reply packet
1Doesn't seem to be used by the zombie!
2..end"Payload": encoded command or reply

One interesting thing to note is that the first byte, indicating the packet type, is a 2 or 3. What happened to packet type 1? One bit of wild speculation is that this is part of a larger DDoS system, where the attacker controls handlers (or master controllers), which in turn control the Prot11 zombies. This is a fairly common tree-like structure used in DDoS tools to allow more zombies to be controlled efficiently, and reduces the chances of the attacker being traced. The missing piece of this picture (the control of the handlers) might be what packet type 1 is used for.

Packets are always padded to be between 400 and 600 bytes long, and all data from offset 2 to the end of the packet is encoded to obscure the meaning. The following section describes the encoding process in detail.

Payload Encoding and Decoding

The following algorithm is used to encode the payload portion of the packets: the first byte simply has the constant 23 added to it. Second and subsequent bytes have the previous output byte added to it, as well as the constant 23. We'll call this encryption method "Caesar cipher with chaining," and it is terribly insecure (by way of comparison, a very similar tool, TFN2K, uses the CAST-256 strong encryption algorithm for the command stream). The encoding method used by Prot11 zombie is illustrated in the following diagram:

It is easy to reverse this process to get the decoding method.

The encoding and decoding routines are shown below in C code, as we implemented them. Note that the implementation below is much simpler than what is in the challenge binary, but is functionally equivalent.

Encoding Function:

    void encode(int len, u_char *in, u_char *out)
        int i;
        if (len <= 0)

        out[0] = in[0] + 23;
        for (i=1; i<len; i++)
            out[i] = in[i]+out[i-1]+23;

Decoding Function:

    void decode(int len, u_char *in, u_char *out)
        int i;
        if (len <= 0)
        for (i=len-1; i>0; i--)
            out[i] = in[i]-in[i-1]-23;
        out[0] = in[0]-23;

Command Packet Structure

Once the payload is decoded, all command packets have the command number (1-12) as the second byte (the first byte is ignored!). Parameters for controlling the command immediately follow the command byte. A brief overview of the commands was given in the main text of this advisory, and we will now document the precise parameters to each command. In the list below, the following short-hand notation is used to refer to the decoded payload of the command packet: P[i] is the byte at offset i in the payload portion, so for example P[1] is the second byte which is the command code. P[i..j] refers to a range of parameters, so for example we might use P[1..4] for a 4 byte IP address. Finally, P[i..z], where i is an integer and z is literal refers to a zero-terminated string that begins at offset i, so for example P[10..z] refers to a zero-terminated string that begins at offset 10.

One common characteristic for all attack commands is the way the target host is specified. The target host can be specified by either a 32-bit IP address or by a string hostname. Space for both of these formats is reserved in the parameters, and there is a third parameter which tells the Prot11 zombie which format to use. For example, in command 4, if P[8] is zero the 32-bit address stored in P[2..5] is used; if P[8] is non-zero, then the string hostname stored in P[9..z] is used.

Each individual command packet is documented below. A few parameters refer to notes below for further information.

Command 1: Attack status query

P[1]Command number (1)

Command 2: Set communication parameters

P[1]Command number (2)
P[2]0 for single reply; 1 to add 9 random IPs; 2 to reply to 10 supplied IPs
P[3..6]IP address (main IP address if P[1] is 0 or 1, first of 10 IPs if P[1] is 2)
P[3+4*k..6+4*k]For k=1..9, gives remaining IP addresses when P[1]=2

Command 3: Execute Shell Command

P[1]Command number (3)
P[2..z]The command to execute (string)

Command 4: DNS reply flood - slow

P[1]Command number (4)
P[2..5]IP address of target (if P[8]=0)
P[6..7]Source port number for DNS packets (0 means randomize)
P[8]Name lookup flag (0 means use P[2..5] for attack target, and non-zero means use string at P[9..z] for target (requires a call to "gethostbyname")
P[9..z]Hostname of target (used if P[8] is non-zero)

Command 5: IP Fragment Flood

P[1]Command number (5)
P[2]Attack type (0=ICMP packet, nonzero=UDP packets)
P[3]Byte for UDP data packet - see note below
P[4..7]IP address of target (if P[12]=0)
P[8..11]Source IP to use
P[12]Name lookup flag (0 means use P[4..7] for attack target, and non-zero means use string at P[13..z] for target (requires a call to "gethostbyname")
P[13..z]Hostname of target (used if P[12] is non-zero)

Command 6: Start backdoor shell

P[1]Command number (6)

Command 7: Silent Execute Shell Command

P[1]Command number (7)
P[2..z]The command to execute (string)

Command 8: Stop current attack

P[1]Command number (8)

Command 9: DNS reply flood - rate adjustable

P[1]Command number (9)
P[2..5]IP address of target (if P[9]=0)
P[6..7]Source port number for DNS packets (0 means randomize)
P[8]Attack rate (attacker sleeps every P[8] packets, so higher numbers give faster attacks) - see note below
P[9]Name lookup flag (0 means use P[2..5] for attack target, and non-zero means use string at P[10..z] for target (requires a call to "gethostbyname")
P[10..z]Hostname of target (used if P[9] is non-zero)

Command 10: SYN flood - slow

P[1]Command number (10)
P[2..5]IP address of target (if P[13]=0)
P[6..7]Target port
P[8]Random source IP flag (0=random addresses, nonzero=given address)
P[9..12]Source IP (used if P[8] is nonzero)
P[13]Name lookup flag (0 means use P[2..5] for attack target, and non-zero means use string at P[14..z] for target (requires a call to "gethostbyname")
P[14..z]Hostname of target (used if P[13] is non-zero)

Command 11: SYN flood - rate adjustable

P[1]Command number (11)
P[2..5]IP address of target (if P[14]=0)
P[6..7]Target port
P[8]Random source IP flag (0=random addresses, nonzero=given address)
P[9..12]Source IP (used if P[8] is nonzero)
P[13]Attack rate (attacker sleeps every P[10] packets, so higher numbers give faster attacks) - see note below
P[14]Name lookup flag (0 means use P[2..5] for attack target, and non-zero means use string at P[15..z] for target (requires a call to "gethostbyname")
P[15..z]Hostname of target (used if P[14] is non-zero)

Command 12: DNS request flood

P[1]Command number (12)
P[2..5]IP address of target if P[13]=0
P[6..9]IP address to use as source address (all zeros means randomize source address for each packet)
P[10]Attack rate (attacker sleeps every P[10] packets, so higher numbers give faster attacks) - see note below
P[11..12]Source port (0 to randomize)
P[13]Name lookup flag (0 means use P[2..5] for attack target, and non-zero means use string at P[14..z] for target (requires a call to "gethostbyname")
P[14..z]Hostname of target (used if P[13] is non-zero)


Reply Packet Structure

Only two commands send results back to the controller: command 1 (status query) and command 3 (execute command). The reply packets are of the same general format as the control packets, with the encoded payload beginning at the third byte (using the same encoding as the control packets). Replies are returned according to the parameters set using command 2, and can be sent back to either a single host or to a set of 10 hosts to hide the true destination of the controller.

Replies to command 1 are a single packet with the following structure:

P[1]Always 1 (possibly command number generating reply?)
P[2]Always 7 (no guess here!)
P[3]Attack running flag: 0=no attack running, non-zero=attack running
P[4]Command number of attack, if one is running

Replies to command 3 are zero-terminated strings containing the output of the executed command. The first packet of the output has P[1]=3, and the remaining packets all have P[1]=4. The end is signaled by a packet containing an empty string (so P[2]=0).

Example from a captured session

An exchange with 22 packets was captured between an installed Prot11 zombie and a controller. We analyze that session here to see what we can learn.

The first packet is going into our compromised system, and the data part of the IP packet begins as follows (bytes given in hex):

    02 00 17 30 48 2A EE 95 CF E6 FD 14 2B 42 59 70
Notice that the packet starts with a 2, indicating that it is indeed a control packet. The bytes starting at "17 30..." can be decoded using the procedure described above to reveal the payload, which starts as follows:
          00 02 01 cb ad 90 23 00 00 00 00 00 00 00
Decoding the payload we see from the second byte that this is command 2, so the attacker is setting the communications parameters. Using the packet structure outlined above, we see that the attacker is requesting that replies be sent to 10 addresses, with 9 generated randomly by the Prot11 zombie, and the 10th supplied in the packet with IP address (obtained from converting the hex values cb, ad, 90, and 23 in the above output).

We have learned something very, very important from decoding this packet! While replies will be sent to 10 random-looking addresses, we now know precisely which address is the correct one for the attacker! The attacker is at (or at least has a listening process there), and we do in fact see that as one of the IP addresses in the reply packets that follow.

While decoding a captured packet by hand is an interesting example, it is tedious for multiple packets. With that in mind we started writing a very basic decoder program that uses the PCAP library to read packets from a tcpdump-style packet dump, and prints out a description of what is happening. This program is very incomplete now, as it only prints out a few basic packet types (just the ones we needed!), but could easily be expanded to a complete packet analyzer for this control protocol. The program (decode.c) is included in the files.tar which accompanies this advisory. A section of the output from our decoder is shown below:

    Command packet:
        Command 2 (set comm parameters): 
          Reply type 1 (reply to IP address + 9 random)

    Command packet:
        Command 3 (execute command): rpcinfo -p
    Reply packet:
        Reply to command 3 (execute command first reply packet)
       program vers proto   port
        100000    2   tcp    111  portmapper
        100000    2   udp    111  portmapper
        100021    1   udp   1024  nlockmgr
        100021    3   udp   1024  nlockmgr
        100021    1   tcp   1024  nlockmgr
        100021    3   tcp   1024  nlockmgr
        100024    1   udp    924  status
        100024    1   tcp    926  status

From this packet dump you can see the command 2 packet (which we hand-decoded above), followed by a command 3 packet where the attacker is checking to see what RPC services are running, followed by a reply packet showing the output of the rpcinfo command. Fortunately, this was all our attacker did -- no DoS attacks were launched.