20 Questions for Cyber Security Assessment

by Michael Werneburg
on 2017.04.22

You are here:
Risk topics » Third party risk » 20 Questions for Cyber Security Assessment
be the first to comment

This is a guide to evaluating a service organization's cyber security capabilities. It is based on several years I spent on the vendor side of the equation. Rather than start asking about backups and firewalls, I take a more holistic approach, understanding the business purpose of the proposed arrangement for the purpose of understanding the full scope of the risk involved.

Part I – Understand the data and the risk

These are questions that must be understood internally before asking the service organization anything.

1. What are the potential liabilities?

Criteria: By understanding the sensitivity of the data and the legal liabilities of a breach, the impact of a data breach or other cybercrime may be understood.

Questions: Will the service provider’s systems or personnel have access to or outright possession of investor data? What are the record counts? What is the sensitivity of the fields being shared (for instance, personally identifying information)?

Interpreting evidence: The crucial aspect is whether the service provider has access to the data.

Sensitivity ratings provided by the service provider can be confirmed with sources such as PIPEDA. A service provider unable to respond in a competent manner may not understand the risk inherent.

For sensitive data, such as personally identifying information, the risk to a member dealer is two-fold:

Member dealers have to weigh the value of the service they are obtaining in light of the possibility of a data breach caused by the differential in service provider competence and motivation when it comes to protecting your client’s data. It is vital that the potential liabilities are discussed with legal counsel.

2. What is the service being provided?

Criteria: By understanding the nature of the service being acquired, the impact of a data breach or other cybercrime may be understood.

Questions: Who owns the software being provided? Is it the service organization or a subservice organization? Who owns the data once it is in possession of the service organization? Who bears liability in case of a breach?

Interpreting evidence:

This must be spelled out in the contract. The terms must be clear and in line with your objectives. Again, legal counsel should be consulted on the realities.

3. What are the regulatory requirements?

Criteria: Financial regulator(s) issue guidance on outsourcing and on cyber security due diligence. If this service is deemed material according to that guidance, the service must conform to the guidance.

Questions: What are the requirements for physical location of the storage of the data being shared with the service organization? What are the requirements around duration of that storage? What are the security standards?

Interpreting evidence: Guidance has been issued by OSFI (e.g. B-10 on outsourcing) and IIROC (on security) to name but two. All federal, provincial, and SRO guidance must be consulted.

Part II – Identify the service

4. Who is the service provider?

Criteria: There are several counterparty issues in dealing with service providers.

Additionally, the nature of the counterparty determines the extent of further questioning – smaller, privately held providers may have different competencies and less transparency than larger or public organizations.

Questions: Please provide your latest audited financial statements and insurance certificates. Is your firm Canadian owned? Are you publicly traded or privately held? If privately held, please name the entities that own your firm, and provide the percentage ownership. Is your firm subject to direct regulatory oversight? Are the data centers that house your service located in Canada?

Interpreting evidence:

These questions are relevant to cyber security because of the underlying problem of counterparty risk. Ideally, a service organization will:

Service organizations that are not directly regulated may not operate at the same level of risk competence as those that are regulated.

Foreign-owned service organizations may not operate under the same legal jurisdiction as Canadian providers. Cross-border relationships may introduce complexities in dealing with a data breach, recovering data, pursuing financial compensation.

5. What is the history of this service?

Criteria: An organization with no history of providing the service to be contracted cannot demonstrate adequate control over security in the production environment. Also, a service provider cannot provide (and provision) an adequately secure service if that service is not a core strategic offering from at least one line of business. An adjunct service is unlikely to receive sufficient internal support or resources, or remain current.

Questions: Please describe your current ability, capabilities, and past experience performing the service activities.

Interpreting evidence: A service provider must be able to point to existing customers of a similar size and nature to your own firm, for the same service, in production. Those clients must be able to confirm the security of the production environment.

6. What is the degree of cyber capability?

Criteria: A service organization in today’s financial industry must be aware of the escalating risk of cyber crime, and must be actively engaged on the problem.

Questions: Who is responsible for setting and implementing information security policies and practices at your organization? How are these policies overseen?

Interpreting evidence: There should be a named “Chief Information Security Officer” (CISO) who is responsible for overseeing cyber security or implementing controls, but not both. It’s possible that the CISO is someone within the IT organization, but in such cases there should be another party responsible for risk and/or audit. The CISO should have career history in technology and/or risk and should have accreditations such as a certification from ISACA (CRISC, CISA, CISM) or ISC2 (CISSP).

There should be ample evidence of oversight such as regular meetings of an oversight committee. There should be some form of oversight committee such as an IT steering committee or operational risk committee or both, which is attended by senior management.

Failing all of the above, a contract with a third party security consultancy may suffice. However, the CEO, CFO, and/or COO of the service provider should still be conversant with the information risks and the controls put in place.

7. What is your breach response plan?

Criteria: The use of a detailed cyber security incident response plan is the only way that a service provider can respond to security incidents ranging from small violations of internal process to a full-scale data breach.

Questions: Do you have an internally-published security incident response plan? Does this extend to the scenario of a major data breach?

Interpreting evidence: An annually-reviewed plan should exist, with roles assigned to internal specialists such as the CEO, legal counsel, IT, and customer support. Moreover, it should include plans to involve the external service organization’s support organizations, including insurance providers, incident response coordination (e.g. media, legal, regulatory), forensic investigators, and providers of services such as credit bureau monitoring.

The plan must be a plain-language document usable in times of great stress and uncertainty.

Part III – Service Provider Entity Controls

8. What are your hiring and retention practices?

Criteria: An organization mindful of its security stance specifies skills requirements in its security personnel job descriptions, and routinely reviews the performance of staff. It also conducts background checks for all new hires, firm-wide. Failure to do so invites incompetence and inattention to security.

Questions: Do security personnel job descriptions specify the skills required of their jobs? Are personnel skills evaluated on a periodic basis? Are background checks conducted for all new hires?

Interpreting evidence: Any service provider should be easily able to provide evidence that background checks are in place for new hires. The best way of assuring this while preserving service provider personnel privacy is to find the control in a SOC-2 service auditor’s report and find that no exceptions are found for this control.

9. What are your internal policies?

Criteria: Insider security issues continue to plague organizations of all kinds. A service organization must be able to demonstrate total compliance by employees with internally published statements of policy and process.

Questions: Do you have internally published statements of policy and processes surrounding acceptable use of information technologies? Do these policies cover the secure handling of client data? Do they contain clear penalties in case of mis-use?

Do you have policies regarding non-competition, non-disclosure, intellectual property, and client contractual obligations? Are there policies around media relations and use of social media?

Do termination practices include the immediate revocation of access rights to the service organization LAN and client data?

Interpreting evidence: The service organization should be able to promptly share these policies. A SOC-2 report should cover this section in detail. Exceptions to these controls should not be noted in the report.

10. What is your cyber governance strategy?

Criteria: Service auditors love to say, “A process manual is not a control.” In the information risk space, written policies &procedures, employee knowledge, and employee habits should all be tested on a regular basis.

Questions: Does a quarterly risk-control self-assessment (RCSA) program exist? How are employee habits governed? How are policies benchmarked against industry standards?

Interpreting evidence: An RCSA process should produce work products such as a body of test results that cover all internal controls. Control exceptions should be identified and accompanied with remedial action.

Strong tests of internal policies against external baselines must exist. For example, a SOC-2 audit report (see next question).

Evidence must exist of annual training on security for all personnel. Evidence of annual testing on employee responses to emergent risks (such as spearphishing) should exist.

11. Who provides assurance of the service?

Criteria: Assurance of cyber security capabilities from a third party greatly increases the likelihood of consistent controls on actual day-to-day practice.

Questions: Do you undertake an annual audit of service organization controls? Can you provide a SOC-2 report provided by an accredited third party?

Interpreting evidence: If the service organization has substantial access to (or possession of) your data, a “SOC-2” audit report should be furnished. This should be a Type-2 report (covering an entire year) not a point-in-time Type-1 report unless the service is brand new.

The report should be free of exceptions, and should cover the trust services principles of security, confidentiality, processing integrity, and availability.

All subservice organizations used by the service provider must also be able to provide a SOC-2 report, albeit perhaps with fewer trust services principles in-scope. For instance, a provider of cloud server hosting should provide a report covering the principles of security and availability.

Part IV – Service Provider Technical Security Controls

The following list is aligned with the “Critical Security Controls” published by the Center for Internet Security (version 6.0).

12. Do you have an inventory of authorized and unauthorized Devices?

Criteria: A secure network requires that all assets on the network are in control of the organization, are configured to meet security standards, are free of malware, and permit central monitoring.

Questions: Do you have an automated inventory of IT assets that includes servers, workstations, databases, and network devices? Do network devices refuse access to devices not on the inventory?

Interpreting evidence: The service provider should be able to provide evidence of:

  1. A central database with automated discovery of new items, and some mechanism of assessing the configuration and software present on endpoint devices (laptops, desktops).
  2. Configuration of network devices that forbids undocumented systems from utilizing the network. A simply whitelist of MAC addresses might suffice.


NIST Cybersecurity Framework—ID.AM-1, ID.AM-2

SOC-2 Trust Services Principles, Criteria, and Illustrative Controls – CC3.1, CC5.1

13. Do you have an inventory of authorized and unauthorized Software?

Criteria: A secure system requires that only known-good software is installed and used, and that installed software is not modified.

Questions: Do you have a process by which only whitelisted software is permitted to operate on servers and workstations? Does this process detect and prevent unauthorized changes to installed software?

Interpreting evidence: The service provider should be able to provide evidence of:

  1. A whitelist-based system of preventing unauthorized software from running on workstations and servers.
  2. A system that maintains a record of known-good software states, and ensures that only software matching a known-good state can run.


NIST Cybersecurity Framework—ID.AM-2, ID.AM-5, PR-DS-6, DE.CM-7

SOC-2 Trust Services Principles, Criteria, and Illustrative Controls – CC3.1, CC5.1, CC5.8, CC6.1, CC7.1, CC7.3

Public Safety Canada’s “Top 4 Strategies to Mitigate Targeted Cyber Intrusions”[1] – Number 4.

[1] https://www.publicsafety.gc.ca/cnt/ntnl-scrt/cbr-scrt/tp-strtgs-eng.aspx

14. Do you deploy secure configurations for hardware and software?

Criteria: Servers and workstations not built to a standard are difficult to configure, maintain, and evaluate. An enterprise in which systems configuration is allowed to deviate from an established norm introduces an uncertain security stance, suffers security management complexity, and allows standards to be challenged.

Questions: Do you have a golden source for server and workstation configuration? Are automated tools in place that prevent user configuration changes, detect configuration deviations and enforce server configuration standards? Are automated tools overseen by humans?

Interpreting evidence: The vendor should be able to provide evidence of:

  1. A golden source library for system configurations.
  2. System tools (e.g. such as “Ansible” or “Puppet”) are used to deploy configuration files to servers.
  3. Administrative privileges on operating systems and applications are restricted based on user duties.


NIST Cybersecurity Framework – PR.IP-1, PR.AC-1, PR.AC-4

SOC-2 Trust Services Principles, Criteria, and Illustrative Controls – CC5.1, CC5.2, CC5.3, CC5.4

Public Safety Canada's "Top 4 Strategies to Mitigate Targeted Cyber Intrusions" – Number 1.

15. Do you deploy continuous vulnerability assessment and remediation?

Criteria: Servers and workstations contain operating system code, third party code, and bespoke vendor code. All may include configuration files. Unpatched code and poorly configured systems introduce security vulnerabilities.

Questions: Do you perform continuous automated vulnerability scanning of system code and configuration? Do you have a system for prioritizing and patching vulnerabilities discovered?

Interpreting evidence: The vendor should be able to provide evidence of:

  1. An automated system of scanning servers for vulnerabilities.
  2. A process for prioritizing and patching systems in a timely fashion that incorporates the results of the continuous scanning regime.
  3. Automated patch deployment tools on all systems where such tools are available and safe.


NIST Cybersecurity Framework – ID.RA-1, ID.RA-2, IDRA-5, PR-IP.12, DE.CM-8, RS.MI-5

SOC-2 Trust Services Principles, Criteria, and Illustrative Controls – CC3.1, CC3.2, CC3.3, CC4.1, CC6.1

Public Safety Canada’s “Top 4 Strategies to Mitigate Targeted Cyber Intrusions” – Numbers 2 &3.

16. Do you deploy controlled use of administrative privileges

Criteria: Servers and workstations not configured around the principles of separation of duties and least privileges allow for unauthorized changes to security stance and exposure of data.

Questions: When a new system is deployed, are all system account default passwords changed? Are server systems configured to require initial remote access by non-privileged accounts named for a human user? Is access to administrative accounts restricted by role and logged?

Interpreting evidence: The vendor should be able to provide evidence of:

  1. Administrative privileges on operating systems and applications are restricted based on user duties.
  2. Date-stamped output of automated scripts demonstrate that default passwords do not work on the system.
  3. Access to privileged accounts on servers is restricted to two phase: login, then escalation by systems configuration. Snapshots of systems configuration are provided.


NIST Cybersecurity Framework – PR.AC-1, PR.AC-4, PR.PT-1

SOC-2 Trust Services Principles, Criteria, and Illustrative Controls – CC5.1, CC5.2, CC5.3, CC5.4, CC6.1

Public Safety Canada’s “Top 4 Strategies to Mitigate Targeted Cyber Intrusions” – Number 4.

17. Do you maintain, monitor, and analyze audit logs?

Criteria: Quoting the CIS Critical Security Controls, “Deficiencies in security logging and analysis allow attackers to hide their location, malicious software, and activities on victim machines. Even if the victims know that their systems have been compromised, without protected and complete logging records they are blind to the details of the attack and to subsequent actions taken by the attackers. Without solid audit logs, an attack may go unnoticed indefinitely and the particular damages done may be irreversible.”

Questions: Are systems logs configured in a consistent fashion that includes datestamp, source system of the log entry? Are they forwarded to a log aggregation server? Are automated routines in place to detect anomalies?

Interpreting evidence: These should all be firmly in place for all production systems used to provision the service being acquired. Log monitoring standards and practices should be covered in a SOC-2 report.

18. Are email and web browser protections in place?

Criteria: Quoting the CIS Critical Security Controls, “Web browsers and email clients are very common points of entry and attack because of their high technical complexity and flexibility, and their direct interaction with users and with the other systems and websites. Content can be crafted to entice or spoof users into taking actions that greatly increase risk and allow introduction of malicious code, loss of valuable data, and other attacks.”

Questions: Is a system in place to ensure that only approved email and web browser clients are installed on the service organization’s laptops/PC’s? Is logging conducted on the LAN exit points for forensic purposes? Is a system deployed to filter traffic to known-bad destinations? Are emails scanned for malicious content and links to known-bad destinations?

Interpreting evidence: The service organization’s software inventory (touched upon in question 13) should include approved email and web clients.

Documentation should exist for gateway configuration of web traffic logging and for browser and email scanning and filtering. Cloud-based filtering services and the scanning/filtering capacities of cloud-based email services such as Microsoft’s Office 360 do an excellent job in this respect.

19. Have you deployed malware defenses?

Criteria: Quoting the CIS Critical Security Controls, “Malicious software is an integral and dangerous aspect of Internet threats, and can be designed to attack your systems, devices, or your data. It can be fast-moving, fast-changing, and enter through any number of points like end-user devices, email attachments, web pages, cloud services, user actions, and removable media. Modern malware can be designed to avoid defenses, or to attack or disable them.”

Questions: Is an automated system in place to enforce the malware monitoring of end-point systems (laptops, PC’s) from a central location? Does this system receive and escalate positive scan results coming from the end-points? Are updates applied to all end-point systems in an automated fashion?

Interpreting evidence: The service organization should have an enterprise license of a reputable anti-malware package. SO's with Windows 10 will have this built into the operating system. Documentation should exist for the solution regardless.

20. Are you limiting and controlling use of network ports, protocols, and services?

Criteria: Quoting the CIS Critical Security Controls, “Attackers search for remotely accessible network services that are vulnerable to exploitation. Common examples include poorly configured web servers, mail servers, file and print services, and domain name system (DNS) servers installed by default on a variety of different device types, often without a business need for the given service. Many software packages automatically install services and turn them on as part of the installation of the main software package without informing a user or administrator that the services have been enabled. Attackers scan for such issues and attempt to exploit these services, often attempting default user IDs and passwords or widely available exploitation code.”

Questions: Are active server ports, protocols, and services limited to those with current valid business needs? How is this achieved? How is this verified?

Interpreting evidence: The service organization’s application servers should typically have only a bare number of ports accessible (e.g. web or email, depending on function, and administrative ports such as Secure Shell (SSH)). Ports like FTP and Telnet, database services, and other insecure or high-value services should not be available on Internet-facing systems. Restriction of ports &services should be done with network firewalls and server-side firewalls. Verification should be being done on a regular basis through port-scanning, penetration testing, and review of the server’s configuration.