Penetration Testing ( pentesting) [tutorials]

Penetration Testing is used to find flaws in the system in order to take appropriate security measures to protect the data and maintain functionality. This tutorial provides a quick glimpse of the core concepts of Penetration Testing.

Audience

This tutorial has been prepared for beginners to help them understand the basics of Penetration Testing and how to use it in practice.

Prerequisites

Before proceeding with this tutorial, you should have a basic understanding of software testing and its related concepts.

What is Penetration Testing?

Penetration testing is a type of security testing that is used to test the insecurity of an application. It is conducted to find the security risk which might be present in the system.
If a system is not secured, then any attacker can disrupt or take authorized access to that system. Security risk is normally an accidental error that occurs while developing and implementing the software. For example, configuration errors, design errors, and software bugs, etc.

Why is Penetration Testing Required?

Penetration testing normally evaluates a system’s ability to protect its networks, applications, endpoints and users from external or internal threats. It also attempts to protect the security controls and ensures only authorized access.
Penetration testing is essential because −

• It identifies a simulation environment i.e., how an intruder may attack the system through white hat attack.
• It helps to find weak areas where an intruder can attack to gain access to the computer’s features and data.
• It supports to avoid black hat attack and protects the original data.
• It estimates the magnitude of the attack on potential business.
• It provides evidence to suggest, why it is important to increase investments in security aspect of technology

When to Perform Penetration Testing?

Penetration testing is an essential feature that needs to be performed regularly for securing the functioning of a system. In addition to this, it should be performed whenever −

• Security system discovers new threats by attackers.
• You add a new network infrastructure.
• You update your system or install new software.
• You relocate your office.
• You set up a new end-user program/policy.

How is Penetration Testing Beneficial?

Penetration testing offers the following benefits −

• Enhancement of the Management System − It provides detailed information about the security threats. In addition to this, it also categorizes the degree of vulnerabilities and suggests you, which one is more vulnerable and which one is less. So, you can easily and accurately manage your security system by allocating the security resources accordingly.
  
• Avoid Fines − Penetration testing keeps your organization’s major activities updated and complies with the auditing system. So, penetration testing protects you from giving fines.
  
• Protection from Financial Damage − A simple breach of security system may cause millions of dollars of damage. Penetration testing can protect your organization from such damages.
  
• Customer Protection − Breach of even a single customer’s data may cause big financial damage as well as reputation damage. It protects the organizations who deal with the customers and keep their data intact.

Penetration testing is a combination of techniques that considers various issues of the systems and tests, analyzes, and gives solutions. It is based on a structured procedure that performs penetration testing step-by-step.
This chapter describes various steps or phases of penetration testing method.

Steps of Penetration Testing Method

The following are the seven steps of penetration testing −

Planning & Preparation

Planning and preparation starts with defining the goals and objectives of the penetration testing.
The client and the tester jointly define the goals so that both the parties have the same objectives and understanding. The common objectives of penetration testing are −

• To identify the vulnerability and improve the security of the technical systems.
• Have IT security confirmed by an external third party.
• Increase the security of the organizational/personnel infrastructure.

Reconnaissance

Reconnaissance includes an analysis of the preliminary information. Many times, a tester doesn’t have much information other than the preliminary information, i.e., an IP address or IP address block. The tester starts by analyzing the available information and, if required, requests for more information such as system descriptions, network plans, etc. from the client. This step is the passive penetration test, a sort of. The sole objective is to obtain a complete and detailed information of the systems.

Discovery

In this step, a penetration tester will most likely use the automated tools to scan target assets for discovering vulnerabilities. These tools normally have their own databases giving the details of the latest vulnerabilities. However, tester discover

• Network Discovery − Such as discovery of additional systems, servers, and other devices.
• Host Discovery − It determines open ports on these devices.
• Service Interrogation − It interrogates ports to discover actual services which are running on them.

Analyzing Information and Risks

In this step, tester analyzes and assesses the information gathered before the test steps for dynamically penetrating the system. Because of larger number of systems and size of infrastructure, it is extremely time consuming. While analyzing, the tester considers the following elements −

• The defined goals of the penetration test.
• The potential risks to the system.
• The estimated time required for evaluating potential security flaws for the subsequent active penetration testing.

However, from the list of identified systems, the tester may choose to test only those which contain potential vulnerabilities.

Active Intrusion Attempts

This is the most important step that has to be performed with due care. This step entails the extent to which the potential vulnerabilities that was identified in the discovery step which possess the actual risks. This step must be performed when a verification of potential vulnerabilities is needed. For those systems having very high integrity requirements, the potential vulnerability and risk needs to be carefully considered before conducting critical clean up procedures.

Final Analysis

This step primarily considers all the steps conducted (discussed above) till that time and an evaluation of the vulnerabilities present in the form of potential risks. Further, the tester recommends to eliminate the vulnerabilities and risks. Above all, the tester must assure the transparency of the tests and the vulnerabilities that it disclosed.

Report Preparation

Report preparation must start with overall testing procedures, followed by an analysis of vulnerabilities and risks. The high risks and critical vulnerabilities must have priorities and then followed by the lower order.
However, while documenting the final report, the following points needs to be considered −

• Overall summary of penetration testing.
• Details of each step and the information gathered during the pen testing.
• Details of all the vulnerabilities and risks discovered.
• Details of cleaning and fixing the systems.
• Suggestions for future security. 

Generally, these two terms, i.e., Penetration Testing and Vulnerability assessment are used interchangeably by many people, either because of misunderstanding or marketing hype. But, both the terms are different from each other in terms of their objectives and other means. However, before describing the differences, let us first understand both the terms one-by one.

• Penetration Testing

  Penetration testing replicates the actions of an external or/and internal cyber attacker/s that is intended to break the information security and hack the valuable data or disrupt the normal functioning of the organization. So, with the help of advanced tools and techniques, a penetration tester (also known as ethical hacker) makes an effort to control critical systems and acquire access to sensitive data.
  

  Vulnerability Assessment

  On the other hand, a vulnerability assessment is the technique of identifying (discovery) and measuring security vulnerabilities (scanning) in a given environment. It is a comprehensive assessment of the information security position (result analysis). Further, it identifies the potential weaknesses and provides the proper mitigation measures (remediation) to either remove those weaknesses or reduce below the risk level.
  The following diagram summarizes the vulnerability assessment −
  The following table illustrates the fundamental differences between penetration testing and vulnerability assessments −
  

  Penetration Testing	Vulnerability Assessments
  Determines the scope of an attack.	Makes a directory of assets and resources in a given system.
  Tests sensitive data collection.	Discovers the potential threats to each resource.
  Gathers targeted information and/or inspect the system.	Allocates quantifiable value and significance to the available resources.
  Cleans up the system and gives final report.	Attempts to mitigate or eliminate the potential vulnerabilities of valuable resources.
  It is non-intrusive, documentation and environmental review and analysis.	Comprehensive analysis and through review of the target system and its environment.
  It is ideal for physical environments and network architecture.	It is ideal for lab environments.
  It is meant for critical real-time systems.	It is meant for non-critical systems.

  Which Option is Ideal to Practice?

  Both the methods have different functionality and approach, so it depends upon the security position of the respective system. However, because of the basic difference between penetration testing and vulnerability assessment, the second technique is more beneficial over the first one.
  Vulnerability assessment identifies the weaknesses and gives solution to fix them. On the other hand, penetration testing only answers the question that "can anyone break-in the system security and if so, then what harm he can do?"
  Further, a vulnerability assessment attempts to improve security system and develops a more mature, integrated security program. On the other hand, a penetration testing only gives a picture of your security program’s effectiveness.
  As we have seen here, the vulnerability assessment is more beneficial and gives better result in comparison to penetration testing. But, experts suggest that, as a part of security management system, both techniques should be performed routinely to ensure a perfect secured environment.
  The type of penetration testing normally depends on the scope and the organizational wants and requirements. This chapter discusses about different types of Penetration testing. It is also known as Pen Testing.
  

  Types of Pen Testing

  Following are the important types of pen testing −
  
• Black Box Penetration Testing
• White Box Penetration Testing
• Grey Box Penetration Testing

For better understanding, let us discuss each of them in detail −

Black Box Penetration Testing

In black box penetration testing, tester has no idea about the systems that he is going to test. He is interested to gather information about the target network or system. For example, in this testing, a tester only knows what should be the expected outcome and he does not know how the outcomes arrives. He does not examine any programming codes.

Advantages of Black Box Penetration Testing

It has the following advantages −

• Tester need not necessarily be an expert, as it does not demand specific language knowledge
• Tester verifies contradictions in the actual system and the specifications
• Test is generally conducted with the perspective of a user, not the designer

Disadvantages of Black Box Penetration Testing

Its disadvantages are −

• Particularly, these kinds of test cases are difficult to design.
• Possibly, it is not worth, incase designer has already conducted a test case.
• It does not conduct everything.

White Box Penetration Testing

This is a comprehensive testing, as tester has been provided with whole range of information about the systems and/or network such as Schema, Source code, OS details, IP address, etc. It is normally considered as a simulation of an attack by an internal source. It is also known as structural, glass box, clear box, and open box testing.
White box penetration testing examines the code coverage and does data flow testing, path testing, loop testing, etc.

Advantages of White Box Penetration Testing

It carries the following advantages −

• It ensures that all independent paths of a module have been exercised.
• It ensures that all logical decisions have been verified along with their true and false value.
• It discovers the typographical errors and does syntax checking.
• It finds the design errors that may have occurred because of the difference between logical flow of the program and the actual execution.

Grey Box Penetration Testing

In this type of testing, a tester usually provides partial or limited information about the internal details of the program of a system. It can be considered as an attack by an external hacker who had gained illegitimate access to an organization's network infrastructure documents.

Advantages of Grey Box Penetration Testing

It has the following advantages −

• As the tester does not require the access of source code, it is non-intrusive and unbiased
• As there is clear difference between a developer and a tester, so there is least risk of personal conflict
• You don’t need to provide the internal information about the program functions and other operations

Areas of Penetration Testing

Penetration testing is normally done in the following three areas −

• Network Penetration Testing − In this testing, the physical structure of a system needs to be tested to identify the vulnerability and risk which ensures the security in a network. In the networking environment, a tester identities security flaws in design, implementation, or operation of the respective company/organization’s network. The devices, which are tested by a tester can be computers, modems, or even remote access devices, etc
• Application Penetration Testing − In this testing, the logical structure of the system needs to be tested. It is an attack simulation designed to expose the efficiency of an application’s security controls by identifying vulnerability and risk. The firewall and other monitoring systems are used to protect the security system, but sometime, it needs focused testing especially when traffic is allowed to pass through the firewall.
• The response or workflow of the system − This is the third area that needs to be tested. Social engineering gathers information on human interaction to obtain information about an organization and its computers. It is beneficial to test the ability of the respective organization to prevent unauthorized access to its information systems. Likewise, this test is exclusively designed for the workflow of the organization/company.

• 
  Both manual penetration testing and automated penetration testing are conducted for the same purpose. The only difference between them is the way they are conducted. As the name suggests, manual penetration testing is done by human beings (experts of this field) and automated penetration testing is done by machine itself.
  This chapter will help you learn the concept, differences, and applicability of both the terms.
  

  What is Manual Penetration Testing?

  Manual penetration testing is the testing that is done by human beings. In such type of testing, vulnerability and risk of a machine is tested by an expert engineer.
  Generally, testing engineers perform the following methods −
  
• Data Collection − Data collection plays a key role for testing. One can either collect data manually or can use tool services (such as webpage source code analysis technique, etc.) freely available online. These tools help to collect information like table names, DB versions, database, software, hardware, or even about different third party plugins, etc
• Vulnerability Assessment − Once the data is collected, it helps the testers to identify the security weakness and take preventive steps accordingly.
• Actual Exploit − This is a typical method that an expert tester uses to launch an attack on a target system and likewise, reduces the risk of attack.
• Report Preparation − Once the penetration is done, the tester prepares a final report that describes everything about the system. Finally the report is analyzed to take corrective steps to protect the target system.

Types of Manual Penetration Testing

Manual penetration testing is normally categorized in two following ways −

• Focused Manual Penetration Testing − It is a much focused method that tests specific vulnerabilities and risks. Automated penetration testing cannot perform this testing; it is done only by human experts who examine specific application vulnerabilities within the given domains.
• Comprehensive Manual Penetration Testing − It is through testing of whole systems connected with each other to identify all sorts of risk and vulnerability. However, the function of this testing is more situational, such as investigating whether multiple lower-risk faults can bring more vulnerable attack scenario, etc

What is Automated Penetration Testing?

Automated penetration testing is much faster, efficient, easy, and reliable that tests the vulnerability and risk of a machine automatically. This technology does not require any expert engineer, rather it can be run by any person having least knowledge of this field.
Tools for automated penetration testing are Nessus, Metasploit, OpenVAs, backtract (series 5), etc. These are very efficient tools that changed the efficiency and meaning of penetration testing.
However, the following table illustrates the fundamental difference between the manual and automated penetration testing −

Manual Penetration Testing	Automated Penetration Testing
It requires expert engineer to perform the test.	It is automated so even a learner can run the test.
It requires different tools for the testing.	It has integrated tools does required anything from outside.
In this type of testing, results can vary from test to test.	It has fixed result.
This test requires to remember cleaning up memory by the tester.	It does not.
It is exhaustive and time taking.	It is more efficient and fast.
It has additional advantages i.e. if an expert does pen test, then he can analyze better, he can think what a hacker can think and where he can attack. Hence, he can put security accordingly.	It cannot analyze the situation.
As per the requirement, an expert can run multiple testing.	It cannot.
For critical condition, it is more reliable.	It is not.

• 
  

  What are Penetration Testing Tools?

  The following table collects some of the most significant penetration tools and illustrates their features −
  

  Tool Name	Purpose	Portability	Expected Cost
  Hping	Port Scanning Remote OC fingerprinting	Linux, NetBSD, FreeBSD, OpenBSD,	Free
  Nmap	Network Scanning Port Scanning OS Detection	Linux, Windows, FreeBSD, OS X, HP-UX, NetBSD, Sun, OpenBSD, Solaris, IRIX, Mac, etc.	Free
  SuperScan	Runs queries including ping, whois, hostname lookups, etc. Detects open UDP/TCP ports and determines which services are running on those ports.	Windows 2000/XP/Vista/7	Free
  p0f	Os fingerprinting Firewall detection	Linux, FreeBSD, NetBSD, OpenBSD, Mac OS X, Solaris, Windows, and AIX	Free
  Xprobe	Remote active OS fingerprinting Port Scanning TCP fingerprinting	Linux	Free
  Httprint	Web server fingerprinting SSL detection Detect web enabled devices (e.g., wireless access points, switches, modems, routers)	Linux, Mac OS X, FreeBSD, Win32 (command line & GUI	Free
  Nessus	Detect vulnerabilities that allow remote cracker to control/access sensitive data	Mac OS X, Linux, FreeBSD, Apple, Oracle Solaris, Windows	Free to limited edition
  GFI LANguard	Detect network vulnerabilities	Windows Server 2003/2008, Windows 7 Ultimate/ Vista, Windows 2000 Professional, Business/XP, Sever 2000/2003/2008	Only Trial Version Free
  Iss Scanner	Detect network vulnerabilities	Windows 2000 Professional with SP4, Windows Server 2003 Standard with SO1, Windows XP Professional with SP1a	Only Trial Version Free
  Shadow Security Scanner	Detect network vulnerabilities, audit proxy and LDAP servers	Windows but scan servers built on any platform	Only Trial Version Free
  Metasploit Framework	Develop and execute exploit code against a remote target Test vulnerability of computer systems	All versions of Unix and Windows	Free
  Brutus	Telnet, ftp, and http password cracker	Windows 9x/NT/2000	Free