Research and evaluation is at the heart of the HEAT Service and we regularly publish reports, for both our members and the public, based on our members’ dataset of outreach participants. We work with external agencies to match this dataset with educational outcome data. This matching allows us to gain insight into the student lifecycle, from school exam attainment through to Higher Education progression and subsequent graduation and employment. HEAT members can view the Service Schedule in the member’s area.
Member-level Tracking and Evaluation
HEAT members enjoy access to data and reporting from the HEAT Track, an ongoing longitudinal tracking study whereby outreach participants are tracked through a range of administrative datasets to provide data showing their educational outcomes. The tracking ‘jigsaw’ illustrates the educational datasets through which HEAT tracks participants, including the range of agencies through which the data are accessed.
Members receive three HEAT Track reports annually. These relate to the three main outcomes available for tracked participants: Key Stage 4 (GCSE) exam attainment, Key Stage 5 (A-level and equivalent) exam attainment and Higher Education progression.
Report 1 (Level 2) Attainment at Key Stage 4
Research suggests that attainment at Key Stage 4 has a profound impact on progression to HE and our members work with many schools to provide pre-16 attainment-raising outreach activities. We therefore produce Report 1 to ascertain the impact of outreach delivered by our members on Key Stage 4 attainment. It compares the Key Stage 4 attainment of HEAT outreach participants with their school average by matching HEAT data with the Department for Education (DfE) Key Stage 4 data.
To read more, download the latest Key Stage 4 Performance report (Report 1), which uses an aggregate dataset from all matched HEAT member data. Individual HEAT members can access their institution specific Report 1 in the HEAT File Store (subject to sufficient sample size).
We have worked with Make Happen, one of the Uni Connect members, to create a practical example demonstrating the potential of Report 1 when used optimally.
Report 2 (Level 3) Attainment at Key Stage 5
The HEAT dataset is matched with the Department for Education National Pupil Database (NPD) to compare the Key Stage 5 attainment of HEAT outreach participants with their school or college average.
The Key Stage 5 Performance report (Report 2), based on aggregate data from all HEAT members, will be available to download from the File Store. Individual members can access their institution specific Report 2 in the HEAT File Store (subject to sufficient sample size).
Report 3 (Level 4+) Progression to Higher Education
Report 3 looks at the effect of outreach delivered by our members on the Higher Education student lifecycle, from access to HE through to employment.
For this report the Higher Education Statistics Agency (HESA) match our participant database to their data on enrolment, continuation and attainment, and progression into postgraduate study or employment. This allows us to look at the outcomes of our outreach and inreach participants against the full student lifecycle, from access to employment.
To read more, download the latest HESA Track Infographic (Report 3), which uses an aggregate dataset from all matched HEAT member data. Individual members can access their organisation’s Report 3 in the HEAT File Store (subject to sufficient sample size). Members are also able to interrogate their data via our Power BI Report file and using the flat file of their tracked students.
Members receive datasets for each outcome, which can be linked to a Power BI template to view quick, interactive reporting. Please note that student-level data are not available for Key Stage 4 and 5 exam attainment, and this data must be aggregated by HEAT before being shared with members.
HEAT also works with member organisations to help improve evaluation design. Part of this is to support the collection of data for comparator and control groups. In the context of outreach, the challenge is often to find or create a comparison group that is as similar as possible in terms of characteristics to carefully targeted participant groups. The most robust evaluation designs draw on comparator groups that have been carefully selected prospectively, in advance of delivery. Data collected for comparator groups can be tracked by HEAT alongside that of participants, enabling comparison of outcomes when data are made available. HEAT have a range of resources providing guidance on selecting and collecting data for comparator groups.
Aggregate Research and Evidence
HEAT also draws on the longitudinal tracking data compiled by all HEAT members to produce evidence and insight into the impact of outreach at an aggregate level. We publish annual reports for members and the public which examine the impact of outreach on participants’ educational outcomes from KS4 through to employment. Please see our latest analysis examining the progression of outreach participants into Higher Education.
HEAT works on a number of projects, finding new ways to work with sector partners and to combine HEAT’s data with other data sources to add value by investigating themes such as the geography of outreach delivery, links between outreach delivery and educational outcomes, and the impact of nationally-funded outreach programmes. Please see our latest projects for more information.
The HEAT Groups
Since widening participation students are not one homogenous group, HEAT classifies its members’ outreach participants into groups based on both their socioeconomic background and their prior attainment. This classification system allows us to compare similar students in terms of their likelihood of progressing to Higher Education. We use the HEAT Groups alongside other individual deprivation measures such as POLAR4 or the Indices of Multiple Deprivation. HEAT has published a number of Reports using the HEAT methodology:
The HEAT Groups Thematic Paper (updated March 2019)
Using the HEAT Groups, this paper examines attainment and progression outcomes for HEAT’s student groups and identifies critical points in the educational pathways of certain groups.
The report provides insight into when outreach interventions might be needed most. This paper has been updated with Universities and Colleges Admissions Service (UCAS) data to provide insight into application and acceptance in Higher Education (HE) for widening participation (WP) outreach participants.
To read more, download The HEAT Groups Thematic Paper.
The Predictive Power of the HEAT Groups (in partnership with the Behavioural Insights Team (BIT)
For this report HEAT asked the Behavioural Insights Team to deconstruct the HEAT Groups methodology to ascertain how well the HEAT Groups predict progression to Higher Education.
To read more, download the Predictive Power of the HEAT Groups report.
Exploring the relationship between on-campus outreach and access to Higher Education
HEAT commissioned the Behavioural Insights Team to interrogate the HEAT outreach database to try to ascertain whether taking part in an on-campus outreach activity increases propensity towards HE entry.
To read more, download the Exploring the relationship between on-campus outreach and HE entry summary report.
Access to Higher Education for Rural and Coastal Communities
With a focus on young people living in isolated rural and coastal areas, this thematic explores whether remoteness may be a barrier to participation in outreach. This report aims to improve our understanding of the influence of place on access to Higher Education.
To read more, download the Progression to Higher Education in Rural and Coastal Communities report.
Disadvantaged White Males: Outreach Participation and Progression Outcomes
This infographic draws on HEAT’s tracking data to increase our understanding of why disadvantaged white males are underrepresented in Higher Education (HE). In this report, the educational outcomes of disadvantaged white males, from Key Stage 2 to Key Stage 5, are compared with outcomes for other disadvantaged groups.
To read more, download the Outreach Participation and Progression Outcomes of Disadvantaged White Males Report and Outreach Participation and Progression Outcomes of Disadvantaged White Males Infographic.
Exploring the Relationship between Outreach Engagement, KS4 Performance and HE Progression
This report characterises our HEAT collective outreach cohort in terms of both socio-economic disadvantage and prior attainment at age 16. The report details quasi-experimental research to provide evidence that universities are working with schools to raise attainment at KS4. It uses a quasi-experimental research design in response to the Office for Fair Access (now the Office for Students) Standards of Evaluation report and meets the highest level (Level 3) of evidence quality.
To read more, download the Exploring the relationship between Outreach engagement, Key Stage 4 performance and HE progression report.
Exploring regional differences in the delivery and impact of pre-16 outreach
This infographic shows regional differences in the Key Stage 4 (KS4) performance of pupils who engaged in pre-16 outreach delivered by HEAT’s member organisations. Participants are allocated to a region based on the location of their school. The analysis shows the number of participants engaged in pre-16 outreach in each region alongside a profile of their prior attainment at Key Stage 2 (KS2). The report then looks at the absolute and relative KS4 performance of these participants in comparison with their school average.
To read more, download the Key Stage 4 Performance Tracking Regional Breakdown report.
Case studies and Bespoke Analysis
HEAT also works with members to evaluate their activities using data supplied through the HEAT Track. HEAT can add value by bringing expertise and knowledge of evaluation methods, as well as familiarity with HEAT’s data, to help produce high quality robust evaluation.
We are building a library of evaluation case studies to be shared within the membership. These case studies are activity-level and therefore provide some evidence of what has worked in what context but also show how HEAT’s tracking data and reporting can be used in an optimal way.