RIP Performance Linked Funding

On 7 November TEC announced that Performance Linked Funding (PLF) is to be discontinued. Instead TEC will continue “to use a range of levers to encourage better educational performance and address poor performance.”

So, rest in peace PLF. It has been a long and complex exercise which appears to have achieved very little except increase compliance costs for providers.

One recalls the Ministry of Education’s Statement of Service Objectives (SSO)/Statement of Service Performance (SSP) regime that was in place before TEC’s dead hand reached out for new levers. That regime involved setting two types of objectives: those for equal educational opportunity and those for educational performance. This seems to have covered a wide range of factors in a simple manner and without all the expenditure of ineffective initiatives such as MyQ.

But SSO/SSP morphed into KPIs (Key Performance Indictors) which went through various revisions before finally being replaced by EPIs (Education Performance Indicators).

So, what went wrong?

There seem to have been a number of factors:

  • Too many CEOs at TEC over the years, and a high turnover of those charged with policy making.
  • An inability to clearly define performance indicators: remember it took eight versions published from 2010 to 2014 to get to a definitive set of definitions and methodology for EPIs. No sooner had version 8 been released than the cohort-based indicators were being developed and promulgated – again with several revisions.
  • There has been an ongoing tinkering with the relationship between performance and funding, most recently exhibited by an unannounced trial of points-based EPI modelling.
  • There has been an increasing obfuscation of the calculation and publication of EPIs on the part of TEC. What were useful tools in Workspace2 became a nightmare in Nga Kete.
  • Nga Kete’s interface reflects an overfondness for technology that seems more aimed at satisfying TEC’s internal requirements (and IT predilections) than meeting the needs of providers.

This said, it must also be pointed out that The Ministry of Education’s SSO/SSP regime relied on attestations which were not rigorously monitored. Over the years some providers found how to game the system. As they were found out increased compliance measures have had to be introduced. PEI’s and Intueri’s actions were the most notable instances.

As Roy Sharp (TEC’s CEO back around 2010) observed, there is a balance between trust and compliance costs. His inclination being to place faith in trust, a position that did not last long after his brief tenure  ended.

Experienced staff – not clanking robots

There is also a balance between a wholly mechanistic method of allocating funding, and a method which involves mature discretion on the part of experienced staff at the funding agency. Such staff do exist within TEC so hopefully it will be their hands on the levers in future and not some clanking robots.

[RH1]

Dazed and Confused

I think I’m beginning to feel my age because I simply don’t understand the information being supplied by TEC about performance measurement.

Here is what I have found and don’t understand. Perhaps one of you can explain it to me?

  1. 2017 SAC3+ Funding Condition 8.1 (c) Criteria to be a “qualifying TEO” refers to an average 2016 course completion rate of 70%.
  2. The same 2018 SAC3+ Funding Condition refers to an average course 2017 completion rate of 70%.
  3. Historically 70% has been the basic threshold for ongoing funding. If you didn’t hit 70% in a return year (e.g. 2015) you wouldn’t receive any ongoing funding – i.e. a “zero initial allocation” would apply two years hence (that being in 2017 when 2015 was the return year). This method has been in place since 2012, albeit initially with a lower threshold.
  4. This threshold was different from the Performance Linked Funding (PLF) thresholds which could see you losing up to 5% of your funding.
  5. A TEC document circulated in October 2017 entitled “Changes to Educational Performance Indicators (EPIs) and performance-linked funding (PLF) for 2018 onwards” tells us several things:

5.1.    “There will be no PLF funding adjustments in 2018 based on 2017 (educational delivery performance data). This is because no thresholds were set for educational delivery in 2017 as the new EPI methodologies were under development and consultation. However 2017 EPI data will still be published.”

5.2.    “We are also making a small change to the definition of course completion rates following this year’s consultation process. The data used to calculate the course completion rate will now include the same students as used for the cohort-based qualification completion rate. This change will apply to 2017 performance onwards.”

  1. I have several questions. I’ll ask you first because maybe you have answers. Otherwise I guess that I will have to ask TEC and then wait, and wait, and wait ….. for answers.

6.1.    While the 2017 performance is not going to be taken into account for PLF purposes, is the modified 2017 SCC rate going to be taken into account for future initial allocations?

6.2.    If so commencing with which year?

6.3.    What are the details of the small change to the definition of the successful course completion rate from 2017?

6.4.    How many angels can dance on the head of a pin?

2017 Successful Course Completion Rate

There are a couple of things you may like to think about in relation to how TEC will measure your 2017 SCC rate for SAC3+ and SAC3&4 Competitive funding.

• TEC has yet to announce the parameters for performance linked funding for 2017.
• If ultimately such parameters are announced, and you are adversely affected by any decision that TEC made, you could have ample grounds to challenge that decision.

The first point is that TEC’s funding condition 7.3 which relates to performance linked funding still reads “There will be a performance-linked funding element that is applied to SAC provision for qualifications at Level 3 and above. We will inform you of how that will be applied separately.” To my knowledge this hasn’t happened. So, well into the 2017 year you have no knowledge of how TEC will make funding decisions for 2019 on the basis of your 2017 performance.

You will be aware that the 2016 funding condition stated “The minimum threshold for educational performance is an average course completion rate of 70% or higher” as had been the case for a number of years.

The question then is why has TEC not published any guidance and, when chased up about the matter, TEC simply passes the buck to the Ministry of Education? I quote from a TEC email: “The Ministry of Education is about to consult on PLF options.

One answer to this question is that the inept and muddled introduction of the reporting of confirmed enrolments, and the related announcement that these would have an adverse effect on your 2017 Successful Course Completion and Student Retention rates, has left TEC unsure of its ground. Having already been hit by smart lawyers TEC may have lost its taste for confrontation.

How About No Goal Posts?

As an organisation TEC has plenty of fine words. Anybody remember the “High trust, low compliance cost” boast?

Or this bold statement? “Change in the tertiary education system cannot be delivered without improving the way that we operate. As an organisation, we also need to be more responsive, relevant and flexible to ensure that we are an enabler of system change.”

But when it comes down to doing the business TEC seems determined to fail, especially when it comes to letting the hapless TEOs know how their performance is actually monitored.

Previous practice has been for TEC to move the goal posts. Now we have gone one step further down the rabbit hole, there are no goal posts. Witness this recent question to TEC and TEC’s reply.

QPlease provide details of the performance-based funding methodology and minimum threshold being applied against the 2017 SAC and YG education performance indicators?

AThe Ministry of Education is currently consulting with the sector on performance-based funding methodology to be used with 2016 and 2017 education performance indicators. We will be able to provide further details once this consultation process is finished. If you want information about the consultation process please contact the Ministry of Education.

So, folks, you don’t know how your 2016 or 2017 performance is measured and TEC won’t tell you. Go and ask the Ministry. Maybe that’s being “more responsive, relevant and flexible”.