Which of the following is the best way to determine the effectiveness of a security awareness and training program?

There is little point in doing anything in life unless you know it is worthwhile and what you are doing is going to work. This thinking applies to many aspects of a business including a security awareness program.

When a company sets in place a security awareness program, they should always have in mind what it is they are trying to achieve. With a 2021 Verizon report stressing that the “human element” is behind 85% of data breaches, a focus on creating good security behaviour should be paramount.

But how exactly can you measure a seemingly nebulous element such as behaviour?

Establish a Baseline for a Great Security Awareness Program?

To measure something, you need a starting point, a baseline. A security awareness program is designed to change the behaviour and attitude of employees to improve overall security in a company.

A set of clearly defined objectives is this essential starting point that you can use to build upon when testing the success of your security awareness program. These objectives will form the foundation of your security awareness program metrics using a ‘train and test’ methodology.

When developing a security awareness program, it’s best practice to keep in mind how the success of each area of the program can be measured. In this way, you can build testable elements into the program. Typical parts of a holistic security awareness program should include:

1. Security hygiene: covers various elements such as password choices and web browsing habits
2. Social engineering: what is social engineering and the types of scams affecting an organisation and its employees
3. Phishing awareness: training employees in how to spot phishing tactics
4. Computer security habits: including how to spot if a computer may be infected and the use of security-enhancing measures such as a VPN for remote working

Each of these aspects of a security awareness program can be measured, and the results are then used to provide feedback to optimise the success of the program.

The Measure of Success (or not) of a Security Awareness Program?

Measuring the results of a security awareness program is not a pass/fail exercise. Instead, it offers an insight into the effectiveness of the different strands of Security Awareness Training.

The results can be used to feedback into the various parts of the security program to optimise the training; if something isn’t working, metrics and feedback will give an indicator of this, some metrics can even identify weaknesses in specific training events. This information can then be used to more closely tailor awareness education.

Areas that can be used to capture data on the metrics of security training include:

Awareness Surveys and Employee Feedback

Awareness surveys are questionnaires that employees complete to give an insight into the effectiveness of awareness training. Whilst this method is manual, it can be a useful part of a portfolio of measurement exercises to establish the success of your security awareness program. Questionnaires are often handled by HR or a security consultant, and typically include questions or quizzes that test an employee’s ability to spot a threat.

Phishing Simulations and Metrics

Phishing simulations are performed using automated platforms that send out test phishing emails to employees. The simulation platform will record if the employee successfully spots that the test message is a phishing message, or not. Phishing simulation platforms, and advice on their use, along with the metrics they provide, are available through specialist third parties such as MetaCompliance.

Social Engineering and Metrics

How employees respond to scams is an important aspect of security awareness. Scams such as Business Email Compromise (BEC) are often sophisticated and use social engineering as the basis of the scam.

An employee’s reaction to a simulated social engineering event can be measured both quantitatively and qualitatively, depending on how the simulations are carried out. Third-party companies can advise in creating simulated social engineering tests that are amenable to measurement.

Incident Capture and Reporting

The proof is in the pudding, and that pudding takes the form of security incident reporting by employees. A security incident reporting system that allows employees to easily input security events can offer a way to measure the effectiveness of a security awareness program.

Incident reporting has dual benefits, acting as a triage and response system for security issues as they arise, and logging and auditing the awareness of staff. Employees should be trained to use the incident reporting system to record security events, such as:

• Receipt of a phishing message
• Accidental exposure of a password
• Suspected malware infection
• Accidental exposure of information via email misdelivery
• Lost or stolen devices
• Social engineering attempts, e.g., a phone scam

Measure, Listen, Optimise

Security awareness programs are renowned for being difficult to measure. The measurement of behaviour and understanding is often qualitative rather than just quantitative. However, by using a combination of factors that capture indicators of behavioural change and awareness, a business can ensure that its program is effective.

Ultimately, an organisation needs an effective security awareness program to result in reduced cyber attacks and better overall company cyber-safety. As employees continue to understand how cyber security threats work, optimising the effectiveness of a security awareness program will result in a culture of security that pays off with better security.

Which of the following is the best way for an auditor to determine the effectiveness of a security awareness and training program?

Which of the following is most indicative of the effectiveness of an information security awareness program?

How can an organization maximize the effectiveness of security awareness training program?

How effective is security awareness training?