Uses for Nlog and Nmap

nlog

Uses for Nlog and Nmap

So now you can port scan with Nmap and sort and analyze the results with Nlog. what can you do with these programs? There are, indeed, some interesting applications for port scanners. Here are some examples for you to try on your network:

      1. Scan for the least common services: if you have a service or port number that is only showing up on one or two machines, chances are that it is not something that is standard for your network. It could be a Trojan horse or a banned service (e.g. a file-sharing application). It could also be a misconfigured machine running an FTP server or other type of public server. You can set Nlog to show the number of occurrences of each and sort them by the least often occurring. This will generate a list for you to check. You probably won’t want to include your company’s servers in this scan as the will have lots of one-of-a-kind services running. However, it would not hurt to scan these servers separately either to fine-tune or eliminate extraneous services.
      2. Hunt for illicit/unknown web servers: Chances are that if you run one or more web servers for your company, you will see the HTTP services showing up a few times on your network. However, it is also likely that you will see it on machines where you don’t expect it. Some manufacturers of desktop computers are now loading small web servers by default on their systems for use by their technical support personnel. Unfortunately, these web servers are often barebones programs with security holes in them. You will also find web servers running on printers, routers, firewalls, and even switches and other dedicated hardware. You may need these servers to configure the hardware, but if you aren’t using these servers, you should shut them off. These mini-servers are often configured with no password protection by default and can offer a hacker a foothold onto that machine. They can also offer access to the files on the machines if an intruder knows how to manipulate them. Scan for these hidden web servers, and either turn them off or properly protect them. you should also search for ports other than 80 that are commonly used for HTTP. At the end of this article, there is a table listing some of those ports.
      3. Scan for servers running on desktops: Going a step further with the last exercise, restrict the IP range to only those that are nonserver machines and set a port range from 1 to 1024. This will find desktop machines running services that are normally done by servers, such as mail, web and FTP. Unless there is a good reason for this (e.g. PCAnywhere), your desktop machines should not be running these types of services.
      4. Hunt for Trojan horses: To hunt for Trojan horses on your network, run a scan of your network and translate it into the Nlog database format. Open the Nlog search page, select the ports, and set the range from 30,000 and 65,400. This is the favored range for Trojan horses because it is out of the range of normal services and so they usually will go unnoticed – that is, unless you are port scanning your network. However, just because there are some services running on high-level ports doesn’t always mean you have Trojan horses, but it is worth paying attention to services running on these high port numbers. Once you’ve narrowed it down to the machine and port numbers, you can rule them out by checking the services running on those machines or by SSHing to those port numbers and seeing if you get a service banner.
      5. Check your external network exposure: Put your Nmap box outside your network, either on a dial-up or home broadband connection, and try scanning your company’s public IP addresses. By doing this you will see what services are accessible from the Internet (and thereby to any port scanner-wielding person). This is the most vulnerable part of your network, and you should take extra care to secure any services that are public-facing by using a vulnerability scanner, such as the one described in the next chapter. It will also show if your firewall is properly filtering ports that it is forwarding to internal LAN addresses.
        So you’ve seen all the cool things you can do with a port scanner like Nmap. These programs are useful for finding out what you have running and where your exposures might be. But how do you know if those exposed points might be vulnerable? Or if services that are supposed to be open are safe and secure? That goes beyond the function of a port scanner and into the realm of a vulnerability scanner.


Web Ports

Common Port Number Protocol
81 Alternate web
88 Web
443 HTTPS, secure web
8000-8002 Web
8080 Web
8888 Web

External Links:

Download Nlog at packetstormsecurity.com

2003 archive of secureaustin.com (the former official site of H.D. Moore, creator of Nlog)

Nlog Add-Ons and Extensions

NlogIn the previous article, we discussed installing and using Nlog. In this article, we will discuss using add-ons and writing your own Nlog extensions.

Nlog Add-Ons

As mentioned earlier, Nlog is easily extensible and you can write add-ons to do other tests or functions on any protocols or ports found. In fact, there are several included with the program. If there is an add-on available, there will be a hypertext line next to the port and you can click on it to run the subprogram.

Nlog Built-in Extensions

Extensions Descriptions
Nlog-rpc.pl This add-on takes any RPC services that are found and attemps to find out if there are any current RPC attachments and exports for that service
Nlog-smb.pl For any nodes running NetBIOS, this script tries to retrieve shares, user lists, and any other domain information it can get. It uses the user name and login specified in the nlog-config.ph file.
Nlog-dns.pl This script runs a standard nslookup command on the IP address.
Nlog-finger.pl This runs a query against any finger service found running to see what information is sent.

If you examine these add-on scripts, you will observe that they are all just basic Perl programs. If you are experienced with Perl, you can write your own extensions to execute just about any function against your scanned hosts. For example, you can retrieve and display the HTTP header for any web servers found so you can more easily idenfiy it. You don’t need to go overboard with this, because programs like Nessus can do much more comprehensive testing, but if you just need a banner or some small bit of information, then using Nlog is a good solution.


Nlog comes with a sample custom add-on called nlog-bind.pl. This scrupt is designed to poll a DNS server and tell you what version of BIND (the Berkeley Internet Naming Domain) it is running. However, this script is not finished; it is provided as an exercise to create your own add-ons. The sample script is in /nlog*/extras/bind/. The following procedure guides you through finishing the script. You can use that format to create any custom script of your own.

  1. Compile the script using the Gcc compiler with the following command from that directory:
    gcc -o bindinfo binfo-wdp.c

    This creates a binary file called bindinfo in that directory.

  2. Copy this binary file to the directory where you are keeping your nlog scripts.
  3. Change the permissions on it to make it executable (remember that you have to be root to issue this command):
    chmod 700 bindinfo
  4. Open your nlog-config.ph file in a text editor.
  5. Add this line:
    $bindinfo = "/path/to/bindinfo";

    Replace path/to/bindinfo with the location where you put the binary file.

  6. Save this file.
  7. Now edit nlog-search.pl. This is te Perl script that creates your search results page.
  8. Find the section that looks like this:
    1: # here we place each cgi-handler into a temp var for readability.
    2: 
    3: $cgiSunRPC = "sunrpc+$cgidir/nlog-rpc.pl+SunRPc";
    4: $cgiSMB = "netbios-ssn+$cgidir/nlog-smb.pl+NetBIOS";
    5: $cgiFinger = "finger+$cgidir/nlog-finger.pl+Finger";
    6:
    7: $qcgilinks = "$cgiSunRPc $cgiSMB $cgifinger";
  9. Between lines 5 and 6, add a line that looks like:
    $cgiBIND = "domain+cgidir/nlog-bind.pl+BIND";
  10. Edit line 7 to look like this:
    $qcgilinks = "$cgiSunRPC $cgiSMB $cgiFinger $cgiBIND";

    Line 7 is also where you would add, in a similar fashion, links to any other scripts you had created.

  11. Copy the nlog-bind.pl file from this directory into your cgi-bin directory (/var/www/cgi on Mandriva), and change the permissions (chmod0 so the application can read it.

Now when your Nmap scans find port 53 open (which is generally a DNS server), you can click on the link that Nlog creates and find out what version of BIND is running. You can write additional scripts to extend Nlog by following the logic in this example.

External Links:

Download Nlog at packetstormsecurity.com

2003 archive of secureaustin.com (the former official site of H.D. Moore, creator of Nlog)

Nlog: A Utility for Analyzing Nmap Logs

nlogIn a previous article, we covered the Nmap utility. You can save Nmap logs in a number of formats, including plain text or machine-readable, and import them into another program. However, if these options aren’t enough for you, Nlog can help you make sense of your Nmap output. Running it on very large networks can be a lifesaver, because perusing hundreds of pages of Nmap output looking for nefarious activity can be tedious.

The Nlog program helps you organize and analyze your Nmap output. It presents them in a customizable web interface using CGI scripts. Nlog makes it easy to sort your Nmap data in a single searchable database. On larger networks, this kind of capability is vital to making Nmap useful. H.D. Moore put together these programs and made them available. You can find more information about Nlog at securiteam.com. You can download Nlog at packetstormsecurity.com.

Nlog is also extensible. You can add other scripts to provide more information and run additional tests on the open ports it finds. The author provides several of these add-ons and instructions on how to create your own. Nlog requires Perl and works on log files generated by Nmap 2.0 and higher.

Installing Nlog

Follow these steps to install and prepare Nlog:

  1. Download the files from the Nlog web site.
  2. Unpack the Nlog files using the tar-zxvf command. It will unzip and neatly organize all the files for Nlog in a directory called nlog-1.6.0 (or other numbers, depending on the version number).
  3. You can use the installer script provided to automatically install and prepare the program. Note that you need to edit the program before you run it. Go to the Nlog directory and, using a text editor program such as vi or emacs, open the file installer.sh and enter the variables where indicated for you system. Edit the following parameters with the correct values for your installation.
    CGIDIR=/var/www/cgi/
    HTMLDIR=/var/www/
    

    Put the path to your CGI directory. The above represents the correct values on a default Mandrake installation. Make sure you enter the correct ones for your system. For other Linux systems, find the path to this directory by using the locate command. This useful command will find any files with the text you insert after it.

  4. Save the file, then run it by typing:
    ./install.sh

    The installation script automatically copies the CGI files to your CGI directory and the main HTML file to your HTML directory. It also changes the permissions on those files so they can be executed by your web browser.

  5. For the final step, go into the /html directory and edit the nlog.html file. In the POST statement, change the reference to the cgi files to your cgi files, which should be the same one used above (/var/www/cgi/). Save the file and you are ready to go.


Running Nlog

Nlog can be used as follows:

  1. The first thing you must do is create a Nlog database file to view. You do this by converting an existing Nmap log file. Make sure you save your Nmap logs with the machine-readable option (-m on the command line) to be able to use them in Nlog. You can then use a script provided with Nlog to convert the Nmap log into the database format that Nlog uses. To convert a Nmap machine readable log, run the log2db.pl script using this command:
    Ip2db.pl logfile 
    

    Replace logfile with your log file name and location.

  2. To combine multiple log files into a single database, use the following commands:
    cat * > /PATH/temp.db
    cat * > /PATH/temp.db | sort -u > /PATH/final.db
    
  3. Replace /PATH with the path to your Nmap files and final.db with the name you want to use for the combined Nmap database. This sorts the files into alphabetical order and eliminates any duplicates.
  4. Start your web browser and go to the web directory (/var/www/ from the previous section).
  5. Select the Nmap database file you want to view and click Search.
  6. You can now open your Nmap database and sort it based on the following criteria:
    • Hosts by IP address
    • Ports by number
    • Protocols by name
    • State (open, closed, filtered)
    • OS match

    You can also use any combination of these criteria. For example, you could search for any web servers (http protocol) on Windows systems with a state of open.

In the next article, we will look at Nlog add-ons and creating Nlog extensions.

External Links:

Download Nlog at packetstormsecurity.com

2003 archive of secureaustin.com (the former official site of H.D. Moore, creator of Nlog)

Open Source Tools: Part Three (Even more nmap options)

nmap optionsWhen you specify your targets for scanning, nmap will accept specific IP addresses, address ranges in CIDR format, and octet format (i.e. x.x.x.x). If you have a host file, which may have been generated from your ping sweep earlier, you can specify it as well using the -iL flag. There are other, more formal nmap parsing programs out there, but awk can be used to create a quick and dirty hosts file from an nmap ping sweep. Scripting can be a very powerful addition to any tool, but remember to check all the available output options to avoid doing too much work.

nmap allows the user to specify the speed of the scan, or the amount of time from probe sent to replay received, and therefore how fast packets are sent. On a fast LAN, you can optimize your scanning by setting the -T option to 4, or Aggressive, usually without dropping any packets during send. If you find that a normal scan is taking very long due to ingress filtering or a firewall device, you may want to enable Aggressive scanning. If you know that an IDS sits between you and the target, and you want to be as stealthy as possible, the using -T0 or Paranoid should do what you want; however, it will take a long time to finish a scan, perhaps several hours, depending on your scan parameters.


By default, nmap 6.40 with Auditor scans 1000 ports for common services, which will catch most open TCP ports out there. However, sneaky sysadmins may run ports on uncommon ports, practicing security through obscurity. Without scanning those uncommon ports, you may be missing these services. If you have time, or suspect that a system may be running other services, run nmap with the -p1-65535 parameter, which will scan all 65k TCP ports. Even on a LAN with responsive systems, this will take anywhere from 30 minutes to a few hours. Performing a test like this over the Internet may take even longer, which will allow more time for the system owners, or watchers, to note the excessive traffic and shut you down.

Ping Sweeping with netenum

Finally, if you have a need for a very simple ICMP ping sweep program that you can use for scriptable applications, netenum might be useful. It performs a basic ICMP ping and then replies with only the reachable targets. One quirt about netenum is that it requires a timeout to be specified for the entire test. If no timeout is specified, it outputs a CR-delimited dump of the inputted addresses. If you have tools that will not accept a CIDR formatted range of addresses, you might use netenum to simply expand that into a listing of individual IP addresses. netenum is part of the Internetwork Routing Protocol Attack Suite, which also includes such utilities as cdp (for sending Cisco router Discovery protocol messages), and ass (Automated System Scanner).


External Links:

The official nmap site

Official site for the Internetwork Routing Protocol Attack Suite (IRPAS) – netenum is part of IRPAS

Open Source Tools: Part Two (More nmap options)

nmap optionsIn the previous article, we began our look at open source tools, beginning with nmap. In this article, we continue our look at some nmap options.

nmap Options: Stealth Scanning

For any scanning you perform, it is not a good idea to use a connect scan (-sT), which fully establishes a connection to a port. Excessive port connections can cause a DoS to older machines, and will definitely raise alarms on any IDS system. Therefore, you should use a stealthy port testing method with nmap, such as a SYN scan. To launch a SYN scan from nmap, you use the -sS flag, which produces a listing of the open ports on the target, and possibly open/filtered ports if the target is behind a firewall. The ports returned as open are listed with what service that port corresponds to, based on IANA port registrations, as well as any commonly used ports.


In addition to lowering your profile with half-open scans, an nmap option you may also consider is the FTP or “bounce” scan and idle scan options that can mask your IP from the target. The FTP scan (which was discussed in a previous article) takes advantage of a feature of some FTP servers, which allow anonymous users to proxy connections to other systems. If you find during your enumeration that an anonymous FTP server exists or one to which you have login credentials, try using the -b option with user:pass@server:ftpport. If the server does not require authentication, you can skip the username and password, and unless FTP is running on a nonstandard port, you can leave out the FTP port option as well. The idle scan, using -sI zombiehost:port, has a similar result, but a different method of scanning. If you can identify a target with low traffic and predictable IPID values, you can send spoofed packets to your target, with the source set to the idle target. The result is that an IDS sees the idle scan target as the system performing the scanning, keeping your system hidden. If the idle target is a trusted IP address and can bypass host-based access control lists (ACLs), then you’ll get even better results. Do not expect to be able to use a bounce or idle scan on every penetration test, but keep looking around for potential targets. Older systems, which do not offer useful services, may be the best targets for some of these scan options.

nmap Options: Fingerprinting

You should be able to create a general idea of the remote target’s operating system from the services running and the ports open. For example, ports 135, 137, 139 or 445 often indicate a Windows-based target. [135 is used by the End Point Manager (EPMAP) to remotely manage services (and is also used by DCOM); 137 and 139 are used by NetBIOS; 445 is used by Active Directory.] However, if you want to get more specific, you can use nmap’s -O flag, which invokes nmap’s fingerprinting mode. Care needs to be taken here as well, as some older operating systems such as AIX prior to 4.1 and older SunOS versions have been known to die when presented with a malformed packet. Keep this in mind before using -O across a Class B subnet. Note also that the fingerprint option without any scan types will invoke a SYN scan, the equivalent of -sS.


In the next article, we will look at some more nmap options.

External Links:

nmap.org – the nmap site

Open Source Tools: Part One (nmap)

open source toolsNow that we’ve described the concepts of port scanning, enumeration and fingerprinting, it is time to discuss implementing them with open source tools. This article will cover two categories of tools: scanning tools and enumeration tools.

Port scanners accept a target or a range as input, send a query to specified ports, and then create a list of the responses for each port. The most popular scanner is nmap, written by Fyodor, and which is available from www.insecure.org. There are several open source tools for scanning, but Fyodor’s multipurpose tool has become a standard item among penetration testers and network auditors.


Open Source Tools: Using nmap

Before scanning active targets, consider using the ping sweep functionality of nmap with the -sP option. This option will not port scan a target, but will simply report which targets are up. When invoked as root with nmap -sP ip_address, nmap will send both ICMP echo packets and TCP SYN packets to determine if a host is up. However, if you know that ICMP is blocked, and don’t want to send those unnecessary ICMP packets, you can simply modify nmap’s ping type with the -P option. For example, -P0 -PS enables a TCP ping sweet, with -P0 indicating “no ICMP ping” and -PS indicating “use TCP SYN method.” By isolating the scanning method to just one variant, you increase the speed as well, which may not be a major issue when scanning only a handful of systems, but when scanning multiple Class C networks, or even a Class B network, you may need the extra time for other testing.

If nmap can’t see the target, it won’t scan unless the -P0 (do not ping) option is used. Using the -P0 option can create problems, since nmap will scan each of the target’s ports, even if the target isn’t up, which can waste time. To strike a good balance, consider using the -P option to select another type of ping behavior. For example, the -PP option will use ICMP timestamp requests, and the -PM option will use ICMP netmask requests. Before you perform a full sweep of a network range, it might be useful to do a few limited tests on known IP addresses, such as web servers, DNS, and so on, so you can streamline your ping sweeps and reduce the number of total packets sent and the time taken for the scan.

Capturing the results of the scan is extremely important, as you will be referring to the this information later in the testing process. The easiest way to capture all the needed information is to use the -oA flag, which outputs scan results in three different formats simultaneously: plain text (file extension .nmap), greppable test (.gnmap), and XML (.xml). The gnmap format is especially important to note, because if you need to stop a scan and resume it later, nmap will require this file to continue by using the –resume switch.


In the next article, we will continue our look at open source tools by covering some of nmap’s various options.

External Links:

nmap official site

insecure.org – features nmap news and several open source tools, including security tools

Port Scanning with nmap

port scanningThe list of potential targets from the footprinting phase of penetration testing can be expansive. To streamline the port scanning process, it makes sense to first determine if the systems are up and responsive. Several methods can be used to test a TCP/IP-connected system’s availability, but the most common technique uses Internet Control Message Protocol (ICMP) packets.

Of course, if you’ve done any type of network troubleshooting and/or are a reader of this blog, you probably recognize this as the protocol that ping uses. The ICMP echo request packet is a basic one that, according to RFC 1122, every host needs to implement and respond to. In reality, many networks, internally and externally, block ICMP echo requests to defend against one of the earliest DoS attacks, the ping flood. They may also block it to prevent scanning from the outside.

If ICMP packets are blocked, TCP ACK packets can also be used for port scanning. This is often referred to as a TCP ping. RFC 1122 states that unsolicited ACK packets should return a TCP RST. Therefore, sending this type of packet to a port that is allowed through a firewall (e.g. port 80), the target should respond with an RST indicating that the target is active. When you combine either ICMP or TCP ping methods to check for active targets in a range, you are performing a “ping sweep”. Such a sweep should be done and captured to a log file that specifies active machines that you can later input into a scanner. Most scanner tools will accept a cariage return delimited file of IP addresses.


Although there are many different port scanners, they all operate in pretty much the same way. Port scanning software, in its most basic state, simply sends out a request to connect to the target computer on each port sequentially and makes a note of which ports responded or seem open to more in-depth probing. There are a few basic types of TCP port scans, the most common of which is a SYN scan (also called a SYN stealth scan), named for the TCP SYN flag, which appears in the TCP connection sequence (the handshake). This type of scan begins by sending a SYN packet, responding with a SYN/ACK response if the port is open, or an RST if the port is closed. This is what happens with most scans: a packet is sent, the return is analyzed, and a determination is made about the state of the system or port. SYN scans are relatively fast, and relatively stealthy, since a full handshake does not occur. Since the TCP handshake did not complete, the service on the target does not see a connection, and does not get a chance to log.

Port Scanning Methods

Other types of port scans that may be used for specific situations include port scanning with various TCP flags set, such as FIN, PUSH, and URG. Different systems respond differently to these packets, so there is an element of OS detection when using these flags, but the primary purpose is to bypass access controls that specifically key on connections initiated with specific TCP flags set.

One of the more interesting port scanning options for nmap is the FTP bounce scan. RFC 959 specifies that FTP servers should support “proxy” FTP connections. In other words, you should be able to connect to an FTP server’s protocol interpreter (PI) to establish the control communication connection. Then you should be able to request that the server-PI initiate an active server data transfer process (DTP) to send a file anywhere on the Internet. This protocol flaw can be used to post virtually untraceable mail and news, hammer on servers at various sites, fill up disks, and try to hop firewalls. The FTP bounce scan can be done with nmap using the -b flag.


Here is a summary of a few nmap options:

nmap Switch Type of Packet Sent Response if Open Response if Open Response if Closed
-sT OS-based connect() Connection Made Connection Refused or Timeout Basic nonprivileged scan type
-sS TCP SYN packet SYN/ACK RST Default scan type with root privileges
-sN Bare TCP packet (no flags) Connection Timeout RST Designed to bypass non-stateful firewalls
-sW TCP packet with ACK flag RST RST Uses value of TCP Window (positive or zero) in header to determine if filtered port is open or close
-b OS-based connect() Connection Made Connection Refused or Timeout FTP bounce scan used to hide originating scan source

External Links:

RFC 1122 at tools.ietf.org

The Art of Port Scanning at nmap.org

nmap documentation (in 16 different languages) at nmap.org

Penetration Testing: Enumeration

penetration testingOnce you have hardened your system and network, it is always a good idea to scan, or penetration test, your own systems for weaknesses that may already exist or may develop. Changes are constantly made to production systems. In addition, malicious users are constantly discovering and exploiting new weaknesses. Penetration testing your own network will help you see potential weaknesses through the eyes of an attacker and will help you to close the holes.

During the scanning phase of penetration testing, you will begin to gather information about your network’s purpose: specifically, what ports, and possibly what services, it offers. Information gathered during this phase is traditionally also used to determine the operating system (or firmware version) of the target devices. The list of active targets gathered from the footprinting phase is used as the target list for this phase. You can specify any host within your approved ranges, but you may lose time trying to scan a system that perhaps does not exist, or may not be reachable from your network location.

Penetration Testing: The Enumeration Process

In penetration testing, enumeration is the process of listing and identifying the specific services and resources that are offered by a network. You perform enumeration by starting with a set of parameters, like an IP address range, or a specific Domain Name Service (DNS) entry, and the open ports on the system. You goal for enumeration is a list of services that are known and reachable from your source. From these services, you move further into deeper scanning, including security scanning and testing. Terms such as banner grabbing and fingerprinting fall under the category of enumeration. The most common tools associated with enumeration include nmap and amap.


An example of successful enumeration would be to start with host 10.0.0.10, and TCP port 22 open. After enumeration, you should be able to state that OpenSSH is running, and what version of OpenSSH is running along with the protocol versions. Moving into fingerprinting, ideal results would tell what version of Linux/Unix is running, and what version of the kernel is running. Often your enumeration will not get to this level of detail, but you should set that as your goal.

Keeping good notes is also important during penetration testing, and is important during this phase as well. If the tool you are using cannot output a log follow, make sure you use tools like tee, which allow you to direct the output of a command to a log file. Sometimes you may also want to know the exact flags or switches you used when you ran a tool, or what the verbose output was.

You can perform enumeration using either active or passive methods. Proxy methods may also be considered passive, as the information you gather will be from a third source, rather than intercepted from the target itself. But a truly passive scan should not involve any data being sent from the host system. Active methods are the more familiar ones in which you send certain types of packets and then receive packets in return.


Once enumeration is complete, you will have a list of targets that you will use for the next stage: scanning. You need to have specific services that are running, versions of these services, and any host or system fingerprinting that you could determine. Moving forward without this information could hamper your further efforts in exploitation.

External Links:

Penetration Testing at Wikipedia

© 2013 David Zientara. All rights reserved. Privacy Policy