Coming Soon: A New Tool to Support Robust Evaluation with Matched Comparator Groups

Whilst the delivery of this development forms part of HEAT’s commitment in the Beyond the Barracks project, this tool will be available for all members to use as part of our collaborative shared system.

This new functionality is designed to help you evaluate outreach activities more robustly, supporting the commitments you have made in your Access and Participation Plan.

Why This Tool Matters

Evaluation is critical for demonstrating the impact of your work. The Matched Comparator Group Report will make it easier to create matched comparator groups, enabling you to assess outcomes for your participants against similar students.

Using a quasi-experimental method such as matched comparator groups can strengthen your evaluation evidence base, and writing up your results may help you to meet Office for Students regulatory expectations for publishing robust evaluation findings (for instance by publishing on the HEEL). For more information on the benefit of using of matched comparator groups see our Comparators Working Group Resources in the File Store (members only).

How It Works

The tool will be embedded in the database as a Report and it will use the familiar HEAT tab structure.

When using the tool, you will be able to:

1. Select your participant group – e.g. by ‘Programme Title’, ‘Descriptor’, ‘Student HEAT ID’, or ‘Activity ID’
2. Choose a pool of potential comparator Students
3. Define the matching variables
4. The system will then perform matching based on your selections
5. You will receive an export file showing participants and their matched comparators

Matching Variable Options

When selecting matching variables, you can choose from a range of fields found under the General, Education, Personal, and Geography tabs on the Student record. Because matching relies on the data you have entered for each student, the variables available to you will depend on the information you have collected.

Streamlined and Integrated Comparator Matching

Our current and final specification for this tool represents a significant advancement on our preliminary scoping. Initially, we anticipated the tool would only identify potential comparator Students, leaving users to complete the actual matching to participants in external statistical software.

However, HEATs Technical team and Analysts have identified a way for the HEAT system to not only identify potential comparators based on user input as originally planned, but to also deliver actual matches in an export file, streamlining the process and making matched comparator work more accessible for members.