Using HEAT to record and evaluate Post-Entry Interventions

Whilst we envisage that in the longer-term, post-entry will become a seamless part of members’ interaction with HEAT, in the short term we know that there will need to be system and tool enhancements to accommodate post-entry working, as well as the integration of post-entry into HEAT’s research agenda.

With this in mind, and with the support of its Research Group, we will be forming a new Post-Entry Working Group with members drawn from across the membership. This group will provide steer for the broader integration of Post-Entry into HEAT’s tools and strategy, as well as providing a forum for the exchange of member expertise and experience. The group will meet twice a year.

Central to HEAT’s capacity to categorise and evaluate Access activities in a consistent way has been the Database Activity Typology, which helps members to indicate consistently what type/s of activity they have delivered. For post-entry, the sector will, of course, need new activity type categorisations to encompass the wide range of interventions relating to a higher education (HE)-enrolled student and their experiences. These may be as diverse as financial interventions, dissertation writing workshops, mental health support or activities designed to foster a sense of belonging at a higher education institution.

Fortunately, working with the sector including HEAT, TASO have published a Post-Entry MOAT. This framework maps the wide range of possible post-entry activities, delivery modes and potential outcomes. The HEAT team have worked with TASO to ensure we can support our members in using this MOAT to record their post-entry Activity and Student records on the database.

We recommend members use Custom Data fields to integrate TASO’s post-entry MOAT into the database. This is an interim measure whilst we consider how best to integrate permanent fields into the database for post-entry data recording. We have published a Post-entry User Guide for members to support them in setting up their custom fields. The guide explains how to code your post-entry Activity records, ensuring they are separated from your outreach data in all reporting. It also explains the benefits of adding Student records to post-entry Activities and gives recommendations of minimum data capture to avoid duplication with your institutional student data system. Setting your environment up using this guide will mean that the HEAT team will be able to support you to migrate your data over to the permanent fields, once they are available, making that process as streamlined as possible.

We would encourage members to look at the User Guide and consider how they could use custom fields to record post-entry activities on the database in support of their new APP, and to contribute to a consistently recorded HEAT aggregate dataset, supporting sector-wide future research. In the future, as post-entry becomes a more integrated part of HEAT, we will be considering a post-entry monitoring dashboard.

Members will know that the ability to record Activities is only one part of the capabilities of the system. HEAT and its members can also record information about participants which can later be tracked into other datasets, either internally by users linking to outcomes stored within their own student data systems, and externally HEAT linking into HESA. To research the outcomes for post-entry participants, members will need to make sure that the correct information about post-entry participants has been included on the database (Student Number, Name, Date of Birth and Home Postcode) and that permissions are in place to track those participants (on the database ‘Permission to Track =Yes’).

Post-entry participants will need to have either seen the HEAT Privacy Notice when taking part in Activity, or members will need to have HEAT included in their institution’s student enrolment privacy notice. The latter is an ideal scenario since, if agreed by your institution, you can be sure that students have been presented, on enrolment, with a privacy notice which includes HEAT research. However, we know that different institutions may be more or less ready to allow this.

To learn more about HEAT member experiences in preparing privacy notices to support post-entry work, see the section below on this post.

Starting in 2024, we will be including Student Ambassador records in the annual HESA Track to extend our research into the post-entry space by aggregating data across institutions and undertaking national-level research on Student Ambassador outcomes. Many members record information about students at their institution who assist them in delivering outreach activities but currently, across the sector, there is little quantitative research into the higher education outcomes of these student ambassadors. We would like to address this gap, using our collective data and resources to lobby for the sector in post-entry, the same way as we do for access.

In 2024, we will undertake this Student Ambassador tracking for the first time with data from a subset of members who have agreed this with HEAT, but in the future all student ambassadors will be included in the track where they are ‘Permission to Track=Yes’ on the database. Including them in the track will give us information about their continuation, degree attainment, post graduate progression and their Graduate Outcomes Survey responses. We will be working with academics at the University of Suffolk to publish research into the outcomes of student ambassadors once we begin to track student ambassadors from a larger number of members.

Extending this research into post-entry will only be possible if members record three key pieces of personal information to allow inclusion of Student Ambassador records in the HEAT HESA Track – Name, Date of Birth and Home Postcode – whilst ensuring students have seen HEAT’s Privacy Notice and are marked as ‘Permission to Track = Yes’. We are investigating automating this process for members who use the Student Ambassador Programme by requiring students to have sight of the HEAT Privacy Notice when Ambassadors log in to the Ambassador Portal.

We launched a new Assessment of Performance (AoP) Dashboard on the 9^th of May 2024. The Assessment of Performance Dashboard is a PowerBI dashboard which allows members to visualise their Institution’s OfS Portal data, assisting them in planning and monitoring for their APP. OfS Portal data is provided by the OfS to every institution with an APP.

The AoP Dashboard is essentially a shell template into which members can upload their own student level OfS Portal Data. The AoP Dashboard visualises OfS Portal data using a visual tab system whose taxonomy is based on the OfS EORR. Each tab on the dashboard is designed to show visualisations for the Access, Continuation, Completion and Success of your entrants as contained in the student level OfS Portal data. Members can download the dashboard template and accompanying notes from the HEAT File Store here. The notes include guidance on which OfS Portal file is needed to populate the dashboard.

The AoP Dashboard adds value and functionality to the online OfS dashboard by allowing members to interrogate student characteristics not currently shown on the OfS dashboard, but which are available in the OfS Portal data files. For those members with limited institutional data resource to call on, the AoP dashboard will give you an opportunity to see your data in a visual form and to build on what the OfS makes available online. Members with larger data departments or who may have already considered building a visualisation of their data, are welcome to adapt the dashboard to suit their institutional needs.

We have a number of enhancements to the current AoP Dashboard which we will be working towards in the coming months.

• Progression

We will be working towards including a Progression tab in the AoP Dashboard.

• Academic Years vs Cohorts

HEAT has been asked by some members about the possibility of visualising ‘cohorts’ of entrants through the student lifecycle in the AoP Dashboard. We will be investigating this in the coming months.

Currently, mirroring the OfS methodology, the data shows results for each academic year in each tab. For instance, the Attainment tab shows the Attainment for an institution in a given academic year, regardless of when students entered the institution. It does not show the Attainment of a specific cohort of entrants as individuals within that cohort may have retaken years, or permanently withdrawn along the way. Visualising data by entrant year (or ‘cohort’) would allow members to follow the trajectory and outcomes for a cohort of students through the lifecycle at their institution, rather than an academic year rate or result for and institution at each stage of the student lifecycle.

• 3 and/or 5 year averages

At the request of members, we will investigate visualising 3 and/or 5 year averages in the AoP Dashboard.

• Guidance on adding in HESA fields

Currently there are some student characteristics included in the OfS EORR that are not found in the OfS Portal data and which therefore cannot be visualised in the AoP Dashboard. Those characteristics would need to be sourced from HESA data. HEAT will be looking to investigate their inclusion in the AoP dashboard. In the first instance we will be approaching the OfS to see whether they can be included in the OfS Portal files (which utilises HESA data as source data anyway). If this is not possible, we will look to write some guidance for members about how to upload some HESA fields into the AoP Dashboard alongside the OfS Portal data.

• Include sector comparators

Members have requested the inclusion of comparators or benchmarks in the AoP Dashboard, and we will be exploring this with the OfS.

Why we chose to consider Post-entry

NTU have recorded pre-entry outreach activity and related participant data since 2011/12. This data is stored on the HEAT database. In August 2023, HEAT released a call out for members who may be interested in recording student success/post-entry interventions. At around the same time, NTU were one of four higher education institution (HEI) partners selected to work with TASO on their Institutional Data Use (IDU) project which involved the use of institutional data to support better evaluation of the student experience.

Next Steps

At this time, NTU had no systematic way of recording participation in post-entry interventions. This was done in an ad-hoc manner with individual teams/departments across the institution keeping their own records. The initial stage in the process of recording post-entry data was to have discussions with colleagues from HEAT and from our own Information Governance team. The discussions with NTU’s Information Governance Team helped us to identify what was feasible within the realms of data protection considerations. The focus was primarily on participant data collection, as this involves the processing of personal data and is therefore subject to GDPR principles. Accordingly, a participant data privacy notice is needed to explain to students how and why we will be processing their data.

Limitations on our next steps

Ideally, we would have liked this privacy notice to have been included in the student enrolment documentation, so that all students enrolling at NTU would have been automatically informed of the process. Once participation had taken place, the student could then be entered onto HEAT without the need for any further communication.

However, our Information Governance Manager did not feel that this was the most appropriate way to inform students. It was determined that it couldn’t be a reasonable expectation that a student’s details may be put on an external database without a more specific notification.

For example, it was suggested that at the time of enrolment, a student may not fully understand the implications if they then participated in a post-entry intervention later in their studies. It was therefore deemed that a more specific and timely communication was needed.

What we chose to do

The process for 2023/24 has therefore been as follows:

1) NTU’s HEAT Coordinator contacts the lead of specific intervention and establishes the best way to communicate the privacy notice with participating students. Previously this was done as a one-off communication to students. Going forwards, the plan is to embed the privacy notice into the intervention recruitment or sign-up process where this is appropriate. Where it isn’t, it will be sent as a one-off communication.

2) Once participating students have been informed of the data collection process the student numbers can be sent securely to NTU’s HEAT Coordinator to be stored securely on the server.

3) The participants are added to HEAT using the following fields:

• First name
• Last name
• DOB
• Postcode
• Permission to track and lawful basis of data collection
• NTU student number
• Expected HE entry year (this will be the actual year the student commenced HE)

5) Files containing the student numbers are deleted from the server at the end of the academic year.

The legal basis for processing the data is for the performance of a task in the public interest and therefore the student does not need to provide consent. They can, however, request that their data is removed from HEAT.

Why we chose to consider Post-entry

UCL is already using HEAT to manage student ambassador work opportunities, including promoting work and assigning work to the student ambassador pool. More recently we have decided to extend our use of the student ambassador function on HEAT to better manage our student ambassador records on a longer-term basis, in line with our work on local data retention at UCL. We hope to extend our work in this area to cover student ambassador tracking as part of our post entry work in the future. The HEAT Privacy Notice is currently incorporated into our own Privacy Notice for Access Programmes as well as the Data Protection Statements for all student worker forms.

Next Steps

Following a recent exercise in data retention, we realised that HEAT could help support our work in this area by securely managing and storing Student Ambassador records on a long-term basis, in line with the HEAT Retention Schedule. Moving this data to HEAT will, we anticipate, mean that it will be easier to correlate Ambassador information with those who have been on access programmes and to student success. It will also enable ambassadors to move more easily between jobs.

Limitations on our next steps

One drawback that become apparent was that not all the staff who would need to make this work had access to HEAT, and those that did often did not have sufficient understanding of HEAT to either make reports or correlate the data on the system. Additionally, records were being regularly deleted as it wasn’t understood how to make them inactive.

What we chose to do

The next steps will be to identify HEAT representatives within the team and ensure that all relevant staff have access to the Student Ambassador function, and that those staff are sufficiently trained.

Student Ambassador records dating back to our first use of the HEAT Privacy Notice will be recreated retrospectively to allow us to better manage this data going forward. Ambassador records will be used to enable work on post entry tracking, helping to evaluate student success interventions.

Why we chose to consider Post-entry

The University of Salford was involved in the first wave of the APP submission for 2024-2028 and identified their headline areas as predominantly in the student success areas of continuation, attainment, and completion, rather than access. In particular we are interested in gap analysis of:

• Minoritised ethnic backgrounds
• IMD Quintile 1
• Highest qualification on entry (HQE) BTEC

Salford identified that they could use the HEAT database as a way of recording and evaluating their continuation, attainment and completion (Post-Entry) interventions, and the Salford students who take part in them.

Next Steps

We identified that we would like to aim to record all our undergraduate entrants on the HEAT database in their first year of entry by the bulk upload of information from our Student record/enrolment data onto the HEAT database. This would require all new students who are enrolling at the University of Salford to acknowledge the HEAT Privacy Notice as part of their enrolment. We asked the Director of Student Admissions and the Data Protection Manager if this would be possible, and they requested the Evaluation and Impact Manager carry out a short Data Protection Impact Assessment to cover what the student data might be used for. They requested a copy of HEAT’s Retention Policy and Data Protection and Privacy Notices and once they were satisfied, they ensured that the Salford Student Policy makes direct reference to the HEAT Privacy Notice.

What we chose to do

Now, moving forward, all students who enrol at University of Salford are eligible to be registered for post-entry tracking on the HEAT database. The University of Salford can now upload data from its own student data system into the HEAT database. We ensure that, in order to make sure we upload ‘Complete’ records to HEAT, the following fields are imported:

• First Name
• Last Name
• Date of Birth
• Home Postcode

To ensure that we can make best use of the uploaded records to support our APP we also upload the following fields:

• University of Salford Student ID
• Disability
• First Gen HE
• Ethnicity
• IMD Quintile 1
• Highest Qualification on Entry (HQE)
• If students entered via Clearing