This section contains information on the following topics:
Before embarking on any analysis of your data you need to check it for completeness and accuracy. Where data are incomplete, you need to decide how you will deal with this. For example, you could decide simply to exclude cases where ethnicity is unknown from an ethnicity analysis, or you could (preferably) include a separate category for analysis of them.
You also need to check that for numerical variables, a missing value is shown as such, rather than as 0.
Checks for accuracy will include looking for any values that look strange, for example a full time equivalent salary of £1,000 per year, or a salary of £100,000 per year for a cleaner. Common job titles need to be expressed in the same way; otherwise they will be treated differently in the computerised sort process - for example, Admin Manager, Admin. Manager, Administrative Manager.
Where you are ‘merging’ data from two databases, you may find that there are differences in the information each database provides about the same person. For example, someone might show as grade 1 on one database and grade 3 on another. Is this the result of an inputting error? Where possible such disparities need to be reconciled.
In addition to cleansing the data, depending on the software you are using, you may also wish to create new variables - for example, if you anticipate analysing pay data by age, you may need to create age group variables from the date of birth field. Similarly, if you anticipate analysing pay by service, you may need to create a new variable from date of joining the organisation (or possibly, the date of joining the grade or the job, if you have these dates in your system).
Typical information would include:
Note: These analyses may be further sub-divided by department/division (or whatever is appropriate to your organisation) and by ethnic categories.
Example tables are available to download below. Although the data can be presented in other formats, for example as pie charts or histograms, it is usually better to have the figures available for your pay review.
It is usually useful to have the percentage of men and women in each grade as this illustrates the comparative distributions and can be used for comparison with distributions of additional payments.
Download example tables.
Note: Given the available data on the ethnic composition in this organisation, it has decided to group staff into broad ‘BME’ and ’white’ categories. Depending on the quality of your data on ethnicity, the number of employees from minority ethnic groups and their range of ethnicities, you may decide to split your pay analysis by narrower ethnic groupings. That could involve numerous analyses of pay between ethnic groups. For an example of analysing pay data by ethnicity see Analysis by ethnicity.
Notes on disability table:
The toolkit asks you to compare average basic pay and average total earnings for men and women (or other protected groups) doing equal work. There are three different ways of measuring ‘central tendency’: the mean; the median and the mode.
The mean is what is generally called an ‘average’. It is the total of the earnings for everyone in a group, divided by the number of people. The mean forms the basis of much statistical testing for differences between groups, but has some disadvantages. The mean can be heavily influenced by some extreme values (outliers) particularly when employee numbers are small.
The median is the number that falls in the middle of the range of data, with half of the cases having values higher than the median, half having values lower. The median is less subject to the effects of extreme values.
The mode is the figure (or range of figures) that applies to more people in the group than any other figure or range of figures. For example, salaries might be divided into six bands ranging from 'under £10,000' to '50,000 to £59,999'. If 30% of salaries fall within the third category, with the remaining cases evenly distributed among the other 5 bands, then the ‘modal’ salary range is the third category – 'between £20,000 and £29,999'. This need not necessarily be the same as either the median or the mode.
Where the mean is different from the median or the mode, there is some ‘skew’ in the distribution.
Some organisations adopt the median in preference to the mean or calculate it as a cross-check against the mean to see whether it will produce different indicators of significant pay gaps.
There is little consensus amongst statisticians about which is the more useful measure in this context – the mean or median - but many organisations use the mean because it is easily understood and, generally, useful.
In practice, most equal pay audits adopt a practical approach by calculating pay gaps where, say, the average salaries of women are always expressed as a percentage of men’s doing equal work, or the salaries of minority ethnic staff are always calculated as a percentage of white staff’s. That enables patterns of pay gaps to be easily determined and shown.
From a purely statistical standpoint, it is arguable that gaps should be calculated by taking the higher average salary and dividing it by the lower, irrespective of gender, ethnicity…
For relatively large pay gaps these differences of approach will not matter because the gaps identified will be identified as ‘significant’ in either case.
The different approaches to calculating pay gaps are unlikely to be of practical importance in the conduct of an overall equal pay audit.
The advice given in the kit concerning ‘significance’ in the difference between the average salaries of various groups can be seen as a useful rule of thumb. It is intended to be easy to understand and easy to apply.
However, the term 'significant' has a precise statistical meaning, which is about the probability of an observed difference of whatever size arising by chance when there is no real difference. The convention is that where the probability of the difference having arisen by chance is 5% or less (i.e. where the test shows p ≤ 0.05), this is described as 'statistically significant', and where the probability of a chance result is 1% or less (p≤ 0.01) as 'highly statistically significant'.
Statistical significance is a function of the size of the difference and the size of the samples. In practice, this means that where there is a genuine difference, however small, the difference will be 'statistically significant' if the sample sizes are large. However, the difference may not be what is meant by ‘significant’ in normal parlance.
It also means that where a very large difference is observed – a difference that would be regarded as important or substantial - this could fall short of 'statistical significance' if the numbers are very small.
Thus, if you used statistical significance alone as a guide to where further investigation should be carried out, you could end up investigating insubstantial differences between large groups of men and women. Conversely, if you use 'effect size' - the amount of the difference as your sole guide, you could end up looking into differences that resulted from chance variations.
The equal pay kit suggests an 'effect size' of a 5% difference in the pay of men and women doing equal work, or where there is a pattern of differences favouring one sex or another, a 3% difference, as 'significant' and therefore justifying further investigation. (The way to calculate this is to see whether the difference between the salaries for the two groups is more than 5% - or 3% - of the lower of the two salaries.) This is a sensible rule of thumb, but needs also to be applied sensibly. If you have the resources to do so, you should also calculate statistical significance.
Where numbers are very small, a difference of 3%, or even 5%, may not merit further investigation, although you might want to keep this particular type of work under careful review. And where a statistically significant difference is found, it may be worth looking into this even if the gap is less than 3%. You would certainly want to examine the size of the difference in any cases where it was found to be statistically significant.
In general it is preferable to investigate a non-significant difference rather than to fail to investigate a significant difference, so, unless there is a substantial resource cost, you should 'if in doubt, check it out'.
The most common test of statistical significance used to compare two means is the t-test. The t-test requires that certain assumptions about the data are satisfied – that the data are normally distributed (the classic 'bell curve'), and that there are equal 'variances' for the two groups. However, in practice it is considered sufficiently robust to give valid results even if these two assumptions are not fully satisfied, and there is also a version of the test that does not assume equal variances.
There are also 'non-parametric' tests that can be used if it is likely that the assumptions are violated. These use the median rather than the mean to make comparisons between groups. They can be a useful check on the validity of the t-test.
The t-test can only be used to compare the means of two different groups, so can be used to compare men and women, white and non-white, disabled and non-disabled, but not for example to compare the mean pay of three different main ethnic groups. A more complex analysis of variance (ANOVA) can be used for this purpose, but you do need to be able to understand and interpret the results properly. There are also non-parametric techniques for comparing more than two different groups.
There may be occasions when you are not interested in differences between mean basic pay or pay elements, but in whether one group or another is more likely to receive a particular pay element. For example, are men more likely than women doing equal work to attract a particular allowance, such as a car allowance? In this case the comparison is between the proportions of the two groups who do and do not receive the allowance. The main statistical test used for this purpose is the 'chi-squared' (X2) test, which again provides a probability figure to indicate statistical significance. The test does not assume normality of distribution but it is compromised if the ‘expected’ frequencies in any cell are very small. An alternative test can be used in these circumstances. As with the t-test, a finding of statistical significance is more likely when numbers of people are large.
A rule of thumb that has been used, particularly in the context of selection procedures, is the ‘four-fifths rule’. This is that where the ‘success rate’ or ‘pass rate’ of one group is less than 4/5 of the same rate for the other group, this difference needs to be looked into. Translated into the field of equal pay, you might say, for example, that if 25% of men are receiving a particular benefit, but less than 20% of women, then this should be looked into. Again this is a rule of thumb intended to be easier to understand and operate than a formal statistical test, based more on what looks like possible ‘material significance’.
The analysis required varies slightly depending how you want to review the results - by grade; by department; by division, location and so on. Below is a typical sequence. How far you go depends on the pay gaps that emerge at each stage.
Gender distribution by grade/equal work groups
Note: These analyses may be further sub-divided by department/division (for larger organisations)
In addition, you may find it helpful to include average length of service by grade by protected ground in this overview of your workforce.
Note: Within each section there may be some further ‘drilling down’ depending on the pay gaps that emerge. A common sequence would be to analyse pay by grade and then, where there are significant gaps, to analyse pay by jobs within grade (like work), particularly jobs where large numbers are employed. This helps to determine the root of the pay gap.
If you have large numbers of male part-time workers, you might want to compare their pay against women working part-time - as well as comparing the pay of part-time women workers against men working full-time.
Note: as before, it is likely there will be some further drilling down, depending on the pay gaps that emerge.
Note: Depending on the ethnic composition of your workforce and the quality of your data, you may wish to analyse pay by more detailed ethnic groupings. For more on this see Analysis by ethnicity
Note: Depending on the quality of your data on disability and the numbers of employees with disabilities, you may wish to analyse pay by types of impairment.
To access a full report of an equal pay audit, together with four previous audits go to:
University of Sunderland Equal Pay Audits
University of Sunderland, Gender
Download Basic pay comparison – by gender and grade – full-time staff (Word 192kb)
Relative length of service of men and women can be plotted on a scattergraph to provide a visual check. An example from another organisation is shown on the graph available below.
Download scatter graph and description
The graph suggests a stronger relationship between length of service for men than for women, and shows a greater ‘scatter’ (spread) of salaries for men, who are also more likely to receive high salaries at most levels of length of service. This may require further investigation.
Example 1: Average basic pay by grade by ethnicity in XYZ Dept and Example 2: Average basic pay by grade by ethnicity in an organisation.
This organisation then explored the pay gaps separately for different ethnic groupings most appropriate for their workforce composition - Black, Asian, Chinese and mixed heritage. These groupings were based on their ethnic monitoring categories:
BME ASIAN
Asian or Asian British – includes Bangladeshi, Indian, Kashmiri, Pakistani, Asian Other
BLACK
Black or Black British – includes African, Caribbean, Black Other
CHINESE
Chinese or other Ethnic Group – includes Arab, Chinese, Afghan, Kurdish, Vietnamese, Chinese Other
MIXED
Mixed – includes white and Asian, white and Black African, white and Black Caribbean, Asian and Black, Mixed Other
WHITE
White – includes Albanian, British, Irish, Romany, white Other.
Commentary: Within this workforce there are 634 people across the seven grades who are recorded as having a disability. The organisation has included all the remaining staff as the comparison group in each grade (those who have declared they have no disabilities, plus those for whom they have no information). Some would argue that it is more appropriate to exclude the staff for whom you have no information.
Please send any feedback or enquiries to equalpayfeedback@equalityhumanrights.com.