Interpretating rate ratios and funnel plots

Rate ratios

Analysis are conducted to assess variation in incidence rates between years, maternal age groups, body mass index categories and nulliparous and multiparous women.  This analysis involved using Poisson regression which calculates a rate ratio (for example, the rate in one year divided by the rate in the previous year). Rate ratios have the advantage of being easy to interpret. They are interpreted against the rate to which they are being compared (the reference group/reference rate). A rate ratio is greater than one if a rate is greater than the rate to which it is being compared. For example, a rate ratio of 1.25 indicates the rate being examined is 25% higher than (or 1.25 times) the rate to which it is being compared. Conversely, a rate ratio will be less than one if a rate is less than the rate to which it is being compared. For example, a rate ratio of 0.80 indicates that the rate being examined is equivalent to 80% of the rate to which it is being compared, i.e. it is 20% lower. The Poisson regression analysis provides a 95% confidence interval for the rate ratio and the associated p-value, both of which indicate whether the rate difference is in line with what might be expected due to chance. A rate difference is considered to be beyond what might be expected by chance, i.e. statistically significant, if the 95% confidence interval for the rate ratio does not include the value one. This is equivalent to the p-value derived from the analysis being less than 0.05. If the p-value is less than 0.001 then the rate difference may be considered highly statistically significant.

Funnel plots

Variations in outcomes rates between maternity units could potentially be due to random chance or reflect differences in baseline characteristics of the characteristics of the population served. For this reason, funnel plots are used to assess performance outcomes for individual units in comparison to the overall average. In brief, the plot is a scatter diagram of individual maternity unit outcomes rates against the number of births or cases within that unit. The national rate is indicated by the solid straight line. The 95% confidence interval is indicated by the curved dashed line. The dashed lines represent the limits within which 95% of units are expected to lie (i.e. within two exact binomial standard errors). The 99.8% confidence interval for the national rate is plotted using solid lines. These solid lines represent the limits within which 99.8% of units are expected to lie (i.e. within three exact binomial standard errors).

The width of the confidence interval is adjusted to allow for meaningful comparison between unit-specific rates and the national rate. The confidence interval is wider for smaller units reflecting the lack of precision in rates calculated based on small numbers. The confidence interval narrows for larger maternity units, giving the diagram a ‘funnel’ shape. Maternity unit rates outside the 95% and 99.8% confidence interval are statistically significantly different from the national rate. In general, one in 20 units would be expected to lie outside the 95% confidence limits by chance alone whereas an observation outside the 99.8% confidence limits is especially rare, i.e. there is a 0.2% chance of this happening.

In the funnel plots, unit-specific rates has been identified by individual letters, with each letter corresponding to each unit listed in the adjacent legend. It is important to note that the funnel plots are based on a hierarchy of total births or relevant cases per unit. As such, depending on the specific outcome being evaluated the letter identifying each unit may vary between funnel plots and across different reporting years. Red markers are used to indicate changes associated with adjustments for relevant factors, such as transfers between units.