RIP Performance Linked Funding

On 7 November TEC announced that Performance Linked Funding (PLF) is to be discontinued. Instead TEC will continue “to use a range of levers to encourage better educational performance and address poor performance.”

So, rest in peace PLF. It has been a long and complex exercise which appears to have achieved very little except increase compliance costs for providers.

One recalls the Ministry of Education’s Statement of Service Objectives (SSO)/Statement of Service Performance (SSP) regime that was in place before TEC’s dead hand reached out for new levers. That regime involved setting two types of objectives: those for equal educational opportunity and those for educational performance. This seems to have covered a wide range of factors in a simple manner and without all the expenditure of ineffective initiatives such as MyQ.

But SSO/SSP morphed into KPIs (Key Performance Indictors) which went through various revisions before finally being replaced by EPIs (Education Performance Indicators).

So, what went wrong?

There seem to have been a number of factors:

  • Too many CEOs at TEC over the years, and a high turnover of those charged with policy making.
  • An inability to clearly define performance indicators: remember it took eight versions published from 2010 to 2014 to get to a definitive set of definitions and methodology for EPIs. No sooner had version 8 been released than the cohort-based indicators were being developed and promulgated – again with several revisions.
  • There has been an ongoing tinkering with the relationship between performance and funding, most recently exhibited by an unannounced trial of points-based EPI modelling.
  • There has been an increasing obfuscation of the calculation and publication of EPIs on the part of TEC. What were useful tools in Workspace2 became a nightmare in Nga Kete.
  • Nga Kete’s interface reflects an overfondness for technology that seems more aimed at satisfying TEC’s internal requirements (and IT predilections) than meeting the needs of providers.

This said, it must also be pointed out that The Ministry of Education’s SSO/SSP regime relied on attestations which were not rigorously monitored. Over the years some providers found how to game the system. As they were found out increased compliance measures have had to be introduced. PEI’s and Intueri’s actions were the most notable instances.

As Roy Sharp (TEC’s CEO back around 2010) observed, there is a balance between trust and compliance costs. His inclination being to place faith in trust, a position that did not last long after his brief tenure  ended.

Experienced staff – not clanking robots

There is also a balance between a wholly mechanistic method of allocating funding, and a method which involves mature discretion on the part of experienced staff at the funding agency. Such staff do exist within TEC so hopefully it will be their hands on the levers in future and not some clanking robots.

[RH1]

Single Data Return – NOT

Those of us with elephantine memories will remember the advent of the Single Data Return which was glibly sold on the basis that it was going to be the one return that rules us all. The grand idea was that there would be one, and only one, reporting mechanism and that would satisfy all data collection and monitoring requirements on the part of all agencies with an interest in the Tertiary Sector.

Even now in 2018 the SDR Manual misleadingly states “The SDR provides one central point for the collection, processing and delivery of information from TEOs to education agencies.”

Over the years other reporting mechanism have been implemented. How do you report to NZQA, how do you report to StudyLink, did you use the Electronic Receipting System (ERS) developed and discarded by TEC? And now we all suffer from dealing with TEC’s Workspace 2 – the kind of reporting mechanism that was presumably designed by a spreadsheet enthusiast and is destined to be replaced.

In fairness to TEC in 2009 work was started on a replacement to the SDR. This was the Tertiary Learner Event Collection (“TLEC”) which, as the project faltered, then became the Tertiary Learner Event Project (“TLEP”), only to be abandoned in 2013 after $2.7 million had been expended and it had to be written off.

The ill-fated UIP (Unfunded International Provider) return was cobbled together from 2016 onwards by the Ministry of Education for a single purpose and without – by the Ministry’s admission – any input from TEC; although NZQA airily told us it would become a “Universal Record of Achievement”. That project must have cost well over $1 million to develop and has ongoing costs.

In addition to the millions of dollars being expended by the tertiary agencies there is of course also the compliance cost to the TEOs.

Not to be left out in the cold TEC at the end of 2016 started to hint at a replacement for the SDR to be developed from 2017 and ready for use in 2020. Several individuals from the sector were asked if they would be open to consultation in 2017. They weren’t asked and have heard nothing since.

But anyway, it did seem like it might be time to check with TEC what is occurring. So, I asked “Please can you provide me with information on any plans to replace the Single Data Return?” This very polite request – using the Official Information Act – has so far elicited the following response.

“Thank you for your email of 19 June 2018 requesting the following information:

please can you provide me with information on any plans to replace the Single Data Return?

Your request necessitates a search through a large quantity of information. In addition, the consultations necessary to make a decision on the request are such that a proper response cannot reasonably be made within the original time limit. Therefore, the TEC is extending the time limit for responding to your request  by 15 working days. The TEC will respond to your request as soon as possible but no later than 7 August 2018.”

I almost felt guilty that I could be the cause of such consternation, but I guess I’ll get over that. The scary bit about the response are these words: “the consultations necessary to make a decision on the request”.

The cynic in me, and past experience, suggests that they just mean that TEC needs another 15 days to construct a response along the lines that the information must be withheld because of commercial confidentiality; or to notify me of the vast cost to TEC of compiling a response. A cost I could not cover.

Somehow, I doubt though that they can use the cost argument since my original request elicited an invitation from TEC to meet for a briefing. The meeting would take an hour I was informed. You will understand that I had to turn down this offer because a meeting of this type with a verbal exchange of information has, in the past, proved to be far from satisfactory.

It is one thing to say something, and another to write it.

Compliance costs, what compliance costs?

You may remember my post of 15 December.

Well it seems now that the universities are getting peeved about compliance costs, and if the universities are upset they sometimes get their way. You may recall the SDR postcode debacle. TEC said you must report two postcodes for each student. It was to be mandatory. The universities said don’t be silly, we use email. TEC buckled. It is not mandatory.

However this time the Minister has become involved; not perhaps the smartest move in these early months of his tenure. One wonders who is advising the Minister.

What is undeniable is that there is a very significant compliance cost and not only for TEC. Certainly TEC is bearing the brunt right now and even the most hard-hearted amongst feel sorry for the staff that have been given a massive tasks at very short notice. One doesn’t feel quite so much sympathy for those who are responsible for communicating TEC’s processes for handling fees free. The bold statement on 5 December that “TEC’s guidance and support systems for the sector are underway as we assist TEO’s navigate the ins and outs of fees-free” has not be born out, unless you count the sneaky emails with unreasonable demands on Friday afternoons.

The underlying problem is that TEC’s analysts and policy makers have little understanding of the reality of processing enrolments and fees. Their focus is – to some extent understandably – on meeting their own needs, even if that means shunting the compliance costs further down the line. That’s to you on an ongoing basis.

One interesting side-effect of TEC’s haste seems to be the possibility of legal challenges, so at least the lawyers will be happy.

Fishing Trip

Did you receive an email from TEC recently which stated “We have been working with StudyLink to monitor whether TEOs are reporting confirmed student enrolments correctly. Our analysis indicates that your organisation may have some enrolments that met the requirement for being reported under SoF 31 but were not reported in the SDR.

The email went on to say “Please review your enrolment data and if necessary, report any confirmed student enrolments that may not have been reported in August.”

Now, if TEC actually thinks you have confirmed enrolments which you have not reported, why don’t they give you the details instead of sending you off on a hunt without any guidance as to where to look?

Two of my clients have had TEC’s email and, on investigation, could not identify any unreported confirmed enrolments.

This would appear to be a case of TEC wantonly increasing your stress levels and compliance costs as part of an unnecessary fishing trip.