The Importance of Penetration Testing for PCI DSS Compliance

Back in 2020, Secora Consulting released a blog post titled "The Importance of Penetration Testing for PCI DSS Compliance". We decided recently that given the release of the new PCI DSS v4.0 that there was a good opportunity to give the guidance a refresh and discuss what has changed (and provide guidance on some areas that we often get queries from our customers).

A summary of what is different in PCI DSS v4.0 regarding Penetration Testing

Defining, documenting and implementing a penetration methodology

Starting off with Penetration Testing Methodologies, there are a couple of things to note here which are more clarifications than outright root and branch changes.

• The PCI Council has added a new action item to ensure that there is a documented approach to assessing and addressing the risk posed by exploitable vulnerabilities and security weaknesses found during penetration testing

• The guidance has been updated to include references to what the PCI Council considers to be industry accepted penetration approaches, namely:

  1. The Open Source Security Testing Methodology and Manual (OSSTMM)
  2. Open Web Application Security Project (OWASP) penetration testing program
• Retention requirements for penetration tests have been updated to explicitly state that penetration test results must be kept for a minimum of 12 months
• The guidance has been updated to include that testing should also include “The testing of security monitoring and detection methods—for example, to confirm the effectiveness of logging and file integrity monitoring mechanisms, should also be considered.” This is a welcome addition as it is suggesting that the assessed entities should use the test to identify how well their logging, monitoring and File Integrity Monitoring (FIM) solutions are performing and continually improve these detection methods.

Performing internal and external penetration testing

The requirements for Internal and External penetration testing haven’t had any ground-breaking changes, so we thought it would be a good opportunity to discuss some of the core concepts that are required as we have found over the years that some of our customers are unclear on what is actually meant by the requirement.

Internal / External penetration is performed after any significant infrastructure or application change

The big question here is “what constitutes a significant change”. When the PCI Council addressed this back in 2017, they specifically identified that what is deemed significant is highly dependent on the assessed entities risk assessment processes and environment and stated that because of the variability between environments a specific definition was not appropriate.

Generally speaking, any changes affecting the access to the Cardholder Data Environment (CDE) or the security of Cardholder Data (CHD) would, in our view, constitute a significant change.

Some examples of changes may be considered to be significant might include:

• Network upgrades (e.g. adding / replacing firewalls)
• Server upgrades (e.g. upgrading operating systems to new versions)
• Application upgrades (e.g. changing the payment mechanism on your payment page)

Some examples of changes that may not necessarily be considered significant might include:

• Infrastructure updates (e.g. applying security patches to systems)
• Standard access to systems (e.g. providing a new System Admin with approved access to systems)
• Standard applications updates (e.g. updating the look and feel of an application)

Internal / External penetration is performed by a qualified internal resource or qualified external third-party

Understanding the qualification requirements for internal resources or external third parties can will often come down to two criteria:

• The qualifications of the tester
• The past experience of the tester

Looking into what these mean in more detail, the qualifications of the tester should be recognised in the industry and the PCI Council has provided some examples of these, namely:

• Offensive Security Certified Professional(OSCP)
• Certified Ethical Hacker (CEH)
• GIAC Certified Penetration Tester (GPEN)
• GIAC Web Application Penetration Tester (GWAPT)
• GIAC Exploit Researcher Advanced Penetration Tester (GXPN)
• CREST Penetration Testing Certifications
• Communication Electronic Security Group (CESG)
• IT Health Check Service (CHECK) certification

Arguably, a more important factor here for ensuring that the test is carried out in a professional and effective manner will the the experience of the tester and some of the questions that the assessed entity need to be asking the penetration testing provider are:

• How many years’ experience does the penetration tester have?
• How many years has the organisation that employs the penetration tester been performing penetration tests?
• Has the penetration tester performed assessments against organisations of similar size and scope?

Another interesting statement in the guidance is that assessed entities should be suspicious of penetration tests that have no findings as this will be “typically indicative of shortcomings of the penetration tester, rather than being a positive reflection of the security posture of the entity.”

Organisational independence of the tester exists (not required to be a QSA or ASV).

From experience, this question usually only comes up where the assessed entity is looking to have a suitably qualified employee conduct their penetration tests. Organisational independence as defined by the PCI Council means that the tester must be “organisationally separate from the management of the target systems”. In most cases, this would require the assessed entity to have a dedicated resource who has no access or responsibility for target systems under assessment.

This requirement also applies to third party companies in that they cannot provide penetration testing services for the purposes of PCI DSS if they have been involved in the installation, maintenance or support of the target systems under assessment.

Exploitable vulnerabilities are in accordance with the entity’s assessment of the risk

Again, no groundbreaking changes here on this requirement, but a good opportunity to discuss an issue that crops up from time to time. Part of the requirement here states that “Penetration Testing is repeated to verify the corrections”. From experience, many assessed entities often conduct the penetration testing of their PCI environment shortly before, or during their QSA led assessment. This then leaves them in a difficult position of not having evidence that the vulnerabilities identified by the tester have been effectively remediated. To avoid this, our advice would be for organisations to ensure that their testing is organised well in advance of their PCI assessment so that evidence of both the initial penetration test and retest can be provided to your QSA.

Using segmentation within your environment

Segmentation of an environment can be a highly effective way of reducing the scope of your PCI assessment, but it also comes with extra requirements when it comes to testing the effectiveness of the segmentation.

There has been a new action that has been added to this requirement which is “Confirming effectiveness of any use of isolation to separate systems with differing security levels (see Requirement 2.2.3).”

Interestingly, if you look at Requirement 2.2.3 in more detail, this is now identifying functions of differing security levels that may be isolated by either physical or logical controls and specifically calls out virtualization technologies to isolate and contain multiple functions into different subsystems. That guidance states that where virtualization technologies are used, the security levels should be identified and managed for each virtual component, for example:

• The function of each application, container, or virtual server instance.
• How virtual machines (VMs) or containers are stored and secured.

If you are unsure of how this would affect you going forward, it would be worth discussing the requirement with your QSA to determine what they expect in terms of testing.

If you are a Service Provider, the guidance has been updated to say that while the requirement specifies that this scope validation is carried out at least once every six months and after significant change, this exercise should be performed as frequently as possible to ensure it remains effective at isolating the CDE from other networks.

Multi-tenant Service Providers

Multi-tenant service providers are addressed as a new requirement in PCI DSS v4.0. The requirement here is defined as “best practice” until 31 March 2025 after which it will become a mandatory requirement. The main elements of the new requirement will be that the Multi-tenant Service Provider provides either:

• Evidence to its customers to show that penetration testing has been performed according to Requirements 11.4.3 and 11.4.4 on the customers’ subscribed infrastructure, or Provide prompt access to each of its customers, so customers can perform their own penetration testing.
• Evidence provided to customers can include redacted penetration testing results but needs to include sufficient information to prove that all elements of Requirements 11.4.3 and 11.4.4 have been met on the customer’s behalf.

Get our expert help

If you are currently preparing for an upcoming PCI DSS assessment or looking to implement a vulnerability assessment routine, get in touch.

Our team is available on email at info@secoraconsulting.com or over the phone on +353 (0) 74 970 7876, and we are happy to assist you with your PCI DSS compliance or any other compliance projects you may have.

Author, Sean Crowley, Director at Secora Consulting.