Fees-Free – An Unfortunate Series of Events

Sadly, the implementation of Fees-Free is not an engaging set of novels for the younger reader however, like the books it has “a dark, mysterious feeling”. One is never sure what will happen next.

And it was whilst pondering the latest debacle (being a fail by TEC with the preparation and distribution of the “Fees Free All Enrolments and Costs Data Validation – March 2019” file), that I decided that I would try to document the set of processes used by TEC to manage Fees-Free and to enumerate the causes of its failure.

A consequence of that failure is that you cannot know how much your Fees-Free payment from TEC will be, you are likely to be non-compliant with TEC’s funding conditions, and your compliance costs will have increased significantly.

If you would like to read this sorry story, please let me know. You may even find it useful when explaining to your boss why you can’t provide an accurate report of expected income, or when dealing with auditors.

Oh, and the most recent fail with the Validation Report? Well if the problem was restricted to the two new (and undocumented) fields in this report it will cause you confusion but no hurt. Still, if TEC gets something this simple wrong you have to wonder what other unannounced problems there are with TEC’s reporting processes.

SDR Validation Error 143

If you are being a good provider and following TEC’s advice to carry out a trial SDR well in advance of the April SDR you are likely to fall foul of Validation Error 143. This error is explained as “IWI is not valid”. The problem you will find is that even if you have reported a valid Iwi code – i.e. one listed in the 2019 SDR Manual Appendix – you still get error 143.

The TEC Sector Helpdesk explains that this is a known bug and will be rectified by 5 April. When I suggested to the Helpdesk that it could lighten its load by publicising the problem, and thus reducing the number of calls it receives. This idea was not warmly welcomed, and it was suggested that the Ministry of Education was responsible for the SDR: buck passed.

Again, one is prompted to suggest that the title “Helpdesk” is something of a misnomer. TEC really doesn’t care if providers waste time proving to themselves that they are not going crazy, and that the problem lies instead with the validation software. Meaning that the providers then have to call the Helpdesk and, in all likelihood their calls won’t go through because the Helpdesk is overloaded.

In short TEC’s basic principle of “less trust, higher compliance cost” is still in effect.

How’s this for a plan

You may recall that in my last post I pointed out that the latest Youth Guarantee funding conditions are not workable.

The 2019 funding conditions 4.1 and 5.4 contradict each other.

  • 4.1 defines 1 EFTS as 100 credits.
  • 5.4 defines 1 EFTS as 120 credits.

This bizarre situation is as a result of a well-meaning but flawed policy change designed to deliver 20% more funding to YG providers. However, in practice it will significantly reduce funding for certain YG providers – especially the more successful. I know of one case where it could result in a cut of nearly 60%.

TEC has been promising to resolve the anomaly since December 2018 but, despite acknowledging that a solution is urgent, there has been no action.

This is where my good idea comes in. YG has been around since 2012. TEC took until 2018 to recognize that funding was insufficient. So, pending finding a way of delivering 20% more funding without disadvantaging some providers, TEC should make a one-time payment equal to 20% of the funding delivered to providers from 2012 to 2018.

After all TEC is very quick to claim back overfunding. It would be only fair for TEC to make up underfunding.

Can TEC Survive Much Longer?

I am not noted for my warm feelings towards TEC, but the opinion piece by Tina Nixon in the Wairarapa Times Age was a doozy and in a league of its own. You can read it yourself, but here are just a couple of samples.

  • TEC is without a doubt one of the most bureaucratic organisations I have ever interacted with, and I have worked with a few.”
  • If the TEC and its current administration survive the next year, then this government will have failed the sector.”

I can’t compete with that, but I can supply a few thoughts on TEC’s current performance – or lack thereof. In all cases the problem seems to relate to a lack of consistency on the part of policy makers. One condition/policy/requirement contradicts another.

Fees Free

Admittedly TEC had very short notice to implement the Fees Free policy but over a year has gone by and they are still struggling. The wash-up return for 2018 was initially scheduled for 11 January 2019 but after receiving a lot of complaints the return date was extended to 16 January. This is where it gets tricky because the Fees Free return will not necessarily match data to be reported through the SDR which is not due until 31 January 2019. For example, a student who started in 2018 with a programme running into 2019 notifies the provider on 17 January that they will not be returning – i.e. they are withdrawing. The provider can record that for the SDR but it will have missed the Fees Free return on 16 January.

Youth Guarantee

The 2019 funding conditions 4.1 and 5.4 contradict each other.

  • 4.1 defines 1 EFTS as 100 credits.
  • 5.4 defines 1 EFTS as 120 credits.

 Go figure!

Until this issue is resolved YG providers would be ill-advised to start enrolling students because they may inadvertently contravene other YG funding conditions.

Funding Source 03 – Domestic Full Fee Paying Students

The Single Data Return Manual states that all students must be reported through the SDR, including non-funded students. However, from the 2018 version 2 edition of the SDR manual the use of Funding Source 03 – Domestic Full Fee Paying Students was changed to exclude students validly repeating courses already completed in a TEC funded programme. This is a practice that was applied to a student who withdrew from a programme several years ago but who returns to complete it and – at their own wish and the wish of the training provider – wants to repeat the few courses already successfully completed, paying a fee to do so. There is now no funding source code that can be applied to these enrolments and so how can they be reported?

RIP Performance Linked Funding

On 7 November TEC announced that Performance Linked Funding (PLF) is to be discontinued. Instead TEC will continue “to use a range of levers to encourage better educational performance and address poor performance.”

So, rest in peace PLF. It has been a long and complex exercise which appears to have achieved very little except increase compliance costs for providers.

One recalls the Ministry of Education’s Statement of Service Objectives (SSO)/Statement of Service Performance (SSP) regime that was in place before TEC’s dead hand reached out for new levers. That regime involved setting two types of objectives: those for equal educational opportunity and those for educational performance. This seems to have covered a wide range of factors in a simple manner and without all the expenditure of ineffective initiatives such as MyQ.

But SSO/SSP morphed into KPIs (Key Performance Indictors) which went through various revisions before finally being replaced by EPIs (Education Performance Indicators).

So, what went wrong?

There seem to have been a number of factors:

  • Too many CEOs at TEC over the years, and a high turnover of those charged with policy making.
  • An inability to clearly define performance indicators: remember it took eight versions published from 2010 to 2014 to get to a definitive set of definitions and methodology for EPIs. No sooner had version 8 been released than the cohort-based indicators were being developed and promulgated – again with several revisions.
  • There has been an ongoing tinkering with the relationship between performance and funding, most recently exhibited by an unannounced trial of points-based EPI modelling.
  • There has been an increasing obfuscation of the calculation and publication of EPIs on the part of TEC. What were useful tools in Workspace2 became a nightmare in Nga Kete.
  • Nga Kete’s interface reflects an overfondness for technology that seems more aimed at satisfying TEC’s internal requirements (and IT predilections) than meeting the needs of providers.

This said, it must also be pointed out that The Ministry of Education’s SSO/SSP regime relied on attestations which were not rigorously monitored. Over the years some providers found how to game the system. As they were found out increased compliance measures have had to be introduced. PEI’s and Intueri’s actions were the most notable instances.

As Roy Sharp (TEC’s CEO back around 2010) observed, there is a balance between trust and compliance costs. His inclination being to place faith in trust, a position that did not last long after his brief tenure  ended.

Experienced staff – not clanking robots

There is also a balance between a wholly mechanistic method of allocating funding, and a method which involves mature discretion on the part of experienced staff at the funding agency. Such staff do exist within TEC so hopefully it will be their hands on the levers in future and not some clanking robots.

[RH1]

Changing Student Management System?

It’s the time of year when providers that are thinking about changing student management system start to do something about it. If you are in this situation you may find the notes below useful.

As you will be aware I have been involved with student management systems for quite a while, and I was responsible for the development and support of one particular system from 1998 to early 2017. At the start of 2016 my business partner and I sold our interest in that product and I was employed by the new owner until February 2017. At that point I left to return to work as an independent consultant and, I must say, life without management meetings has been a glorious relief.

I have since had the opportunity of working with clients with other student management systems, or who are moving to other systems. This leads me to make some observations which may be relevant to you if you are thinking of changing.

  •  The people who market and sell systems don’t necessarily know very much about either the technology of the system they are selling nor, for that matter, the nuts and bolts of student management. They probably also have little insight into the compliance requirements of government agencies.
  • You should ask to talk to the people who will provide support. After all a system is only as good as the support that is provided.
  • Check that the software used by the system is not out of date or possibly even redundant; for example, “Flash”.
  • Where your organisation has multiple organisational units (campuses, departments, etc.) and/or EDUMIS numbers you will want what is called “multitenancy” software. A tenant is a group of users with specific privileges. For example, users at campus “A” can’t see student records at Campus “B”, however a head office user can see all students and produce reports across the whole organisation as well as for the individual campuses. Not all systems which claim multitenancy do in fact work in this way.
  • Nobody likes reading user manuals or online help (and system developers don’t necessarily like writing this material) but you should make a point of asking to see the material and check that it is up-to-date.
  • Make sure that you own your data and can access it independently of the student management system itself and the supplier of the system. This is particularly important if your data is being stored in the cloud.

EPIC: non-Māori, non-Pasifika

TEC’s EPIC template for 2019 is quite different from previous years. It omits an age focused goal (under 25 and 25+) but puts a lot more emphasis on ethnicity by introducing the concept of non-Māori, non-Pasifika.

Government and TEC’s policy is now focused on improving participation and achievement by Māori and Pasifika and therefore a means of accurately determining ethic rates is needed. With Statistics New Zealand’s move from “prioritised” to “total response” counting it is no longer possible to reliably express ethnic rates that sum to 100%.

Consider the tables below. There are 10 students.


The old “prioritised” method always resulted in being able to calculate a percentage for each ethnic group, and those percentages added to 100%.

The new “total response” method will always result in a percentage total of over 100% if any student identified with more than one ethnic group.

However when the count is binary. i.e. a student is either Maori or Pasifika if any one of their ethnicities is Maori or Pasifika, or they are non-Māori, non-Pasifika is none of their declared ethnicities is Maori or Pasifika, we once again have a total of 100%. Hence the table below for the same 10 students.

 

Single Data Return – NOT

Those of us with elephantine memories will remember the advent of the Single Data Return which was glibly sold on the basis that it was going to be the one return that rules us all. The grand idea was that there would be one, and only one, reporting mechanism and that would satisfy all data collection and monitoring requirements on the part of all agencies with an interest in the Tertiary Sector.

Even now in 2018 the SDR Manual misleadingly states “The SDR provides one central point for the collection, processing and delivery of information from TEOs to education agencies.”

Over the years other reporting mechanism have been implemented. How do you report to NZQA, how do you report to StudyLink, did you use the Electronic Receipting System (ERS) developed and discarded by TEC? And now we all suffer from dealing with TEC’s Workspace 2 – the kind of reporting mechanism that was presumably designed by a spreadsheet enthusiast and is destined to be replaced.

In fairness to TEC in 2009 work was started on a replacement to the SDR. This was the Tertiary Learner Event Collection (“TLEC”) which, as the project faltered, then became the Tertiary Learner Event Project (“TLEP”), only to be abandoned in 2013 after $2.7 million had been expended and it had to be written off.

The ill-fated UIP (Unfunded International Provider) return was cobbled together from 2016 onwards by the Ministry of Education for a single purpose and without – by the Ministry’s admission – any input from TEC; although NZQA airily told us it would become a “Universal Record of Achievement”. That project must have cost well over $1 million to develop and has ongoing costs.

In addition to the millions of dollars being expended by the tertiary agencies there is of course also the compliance cost to the TEOs.

Not to be left out in the cold TEC at the end of 2016 started to hint at a replacement for the SDR to be developed from 2017 and ready for use in 2020. Several individuals from the sector were asked if they would be open to consultation in 2017. They weren’t asked and have heard nothing since.

But anyway, it did seem like it might be time to check with TEC what is occurring. So, I asked “Please can you provide me with information on any plans to replace the Single Data Return?” This very polite request – using the Official Information Act – has so far elicited the following response.

“Thank you for your email of 19 June 2018 requesting the following information:

please can you provide me with information on any plans to replace the Single Data Return?

Your request necessitates a search through a large quantity of information. In addition, the consultations necessary to make a decision on the request are such that a proper response cannot reasonably be made within the original time limit. Therefore, the TEC is extending the time limit for responding to your request  by 15 working days. The TEC will respond to your request as soon as possible but no later than 7 August 2018.”

I almost felt guilty that I could be the cause of such consternation, but I guess I’ll get over that. The scary bit about the response are these words: “the consultations necessary to make a decision on the request”.

The cynic in me, and past experience, suggests that they just mean that TEC needs another 15 days to construct a response along the lines that the information must be withheld because of commercial confidentiality; or to notify me of the vast cost to TEC of compiling a response. A cost I could not cover.

Somehow, I doubt though that they can use the cost argument since my original request elicited an invitation from TEC to meet for a briefing. The meeting would take an hour I was informed. You will understand that I had to turn down this offer because a meeting of this type with a verbal exchange of information has, in the past, proved to be far from satisfactory.

It is one thing to say something, and another to write it.

Fees Free – and so it continues

I don’t suppose that the Minster of Education reads my blog. He’s a busy person. One might say “a man with a mission”.

Unfortunately, sometimes it is better to hasten slowly, and this is particularly true of Fees Free. I suspect that the Minster is unaware of the problems affecting Fees Free and the rather messy train crash which awaits him. Regrettably you as a provider may get caught up in the crash.

TEC, which was given the task of implementing a highly complex and ill-planned scheme, was put in a wretched position, but we are nearly in June now and TEC is not helping itself or the Minister by pretending it is all under control.

How about this for example? A smallish provider reported the April SDR on 19 April. It contained enrolment data for 194 SAC3+ funded students, all of whom are consuming more than 0.5 EFTS. On 25 May a file containing 67,409 records was downloaded from Workspace2. This file purports to “contain a list of all NSNs that have been assessed as eligible for Fees Free tertiary study”. It doesn’t, because for those 67,409 records 9,519 show a status of “U – Unknown”. This type of misleading wording happens a lot with Fees Fee. It doesn’t help.

But here’s the real problem, 48 records for the 194 reported by the provider in the April SDR can be matched to records in the Workspace2 file: 47 of which show as a status of “Y- eligible” and one of which shows a status of “U – Unknown”.  Evidently then TEC is not successfully matching SDR data when determining Fees Free status. Five of the 48 students have withdrawn. They could happily wander off and enrol elsewhere thinking that they are Fees Free entitled.

48 out of 194 is certainly a high proportion but the same type of problem exists for other providers albeit to a lesser degree depending on the percentage of Fees Free students enrolling.

Another problem identified with TEC’s determination of Fees Free status is that what is shown in the Workspace2 file does not match what is on the FeesFree.govt.nz web site. Again a hapless student can be misled.

These problems and others have been drawn to TEC’s attention – some as far back as December 2017 – but have still not been addressed.

So, what does it mean?

  • A lot of extra compliance work for the providers.
  • A lot of frustration for individual students.
  • An inability by TEC to reimburse providers the correct value of fees covered by the Fees Free scheme.
  • Trouble for someone when the auditors get to work.
  • An unhappy Minister.

Now, TEC has never paid much heed to the compliance load on providers and students, as people, probably don’t figure much in TEC’s consideration. The misallocation of tax payers’ funds, on the other hand could lead to reviews and tears before bedtime.

Compliance costs, what compliance costs?

You may remember my post of 15 December.

Well it seems now that the universities are getting peeved about compliance costs, and if the universities are upset they sometimes get their way. You may recall the SDR postcode debacle. TEC said you must report two postcodes for each student. It was to be mandatory. The universities said don’t be silly, we use email. TEC buckled. It is not mandatory.

However this time the Minister has become involved; not perhaps the smartest move in these early months of his tenure. One wonders who is advising the Minister.

What is undeniable is that there is a very significant compliance cost and not only for TEC. Certainly TEC is bearing the brunt right now and even the most hard-hearted amongst feel sorry for the staff that have been given a massive tasks at very short notice. One doesn’t feel quite so much sympathy for those who are responsible for communicating TEC’s processes for handling fees free. The bold statement on 5 December that “TEC’s guidance and support systems for the sector are underway as we assist TEO’s navigate the ins and outs of fees-free” has not be born out, unless you count the sneaky emails with unreasonable demands on Friday afternoons.

The underlying problem is that TEC’s analysts and policy makers have little understanding of the reality of processing enrolments and fees. Their focus is – to some extent understandably – on meeting their own needs, even if that means shunting the compliance costs further down the line. That’s to you on an ongoing basis.

One interesting side-effect of TEC’s haste seems to be the possibility of legal challenges, so at least the lawyers will be happy.