TSB probe pins IT crisis on data centre test debacle

18 November 2019, 11:48 | Updated: 18 November 2019, 16:13

The crisis that left 2m TSB customers without current account access last year will be partly blamed this week on a failure to test one of its new data centres even as the company prepared for one of the most complex IT upgrades in banking history.

Sky News has learnt that the inquiry, by the law firm Slaughter and May, has concluded that the decision to proceed with a massive IT migration when only one of the data centres had been tested "made it impossible to identify the problems" with a new system.

That triggered a further slew of problems with the technology that left the bank reeling for weeks - and facing a bill that could eventually total more than £400m.

The report, which will be published on Tuesday, will lay bare the governance, technical and operational failings that led to one of the biggest-ever meltdowns of a major UK bank's infrastructure.

Some of the central criticisms in it will be levelled at those at the top of the company.

An insider said it would charge the TSB board with having failed to address "common sense challenges" about aspects of the IT migration process.

Another source said it had concluded that the decision not to test the performance of both data centres was taken by Sabis, the IT services arm of Sabadell, TSB's Spanish owner.

The omission of one of the two sites from the performance testing process was concealed from board members ahead of the new system going live in April 2018, although Carlos Abarca, the bank's chief information officer at the time, was among the TSB employees who were aware of it, according to the report.

The principal attribution of blame for the failure of the upgraded system will reinforce disclosures made in TSB's annual report this year.

It said that "the underlying issues related to some aspects of three key interconnected issues - the initial configuration, the capacity of the infrastructure and also some aspects of coding".

One insider said that Sabis had recommended testing only one of the data centres in order to avoid interrupting ATM services for TSB customers.

Banking technology analysts said on Monday, however, that it should have been possible to test both data centres with minimal impact on live services by scheduling the tests during quieter periods for customer activity.

TSB has already had to pay close to £370m in "post-migration charges" as a result of the meltdown, including the £25m cost of the investigation - which equates to roughly £83,000 for each of its roughly 300 pages.

The final bill could be higher if any regulatory fines are imposed.

Sky News revealed on Sunday that Mr Abarca failed to inform board members about "shortcomings" with the testing of the new system, and that the report would argue that he made an "ill-judged" assessment of TSB's readiness to proceed with the migration.

The ensuing crisis triggered hundreds of thousands of customer complaints, cost TSB chief executive Paul Pester his job, and propelled the bank to a £105m pre-tax loss last year.

The majority of the £370m cost to TSB related to compensation for customers who were left out of pocket by the IT problems.

TSB received a £450m dowry from Lloyds Banking Group to pay for the building of, and migration to, the new IT system.

Lloyds was forced to divest under a state aid deal in the aftermath of its £20bn taxpayer bailout in 2008.

The IT migration was designed to transfer millions of customers from a system created by Lloyds to one designed by Sabadell.

Slaughter and May's report will be followed by a joint investigation run by the Financial Conduct Authority and Prudential Regulation Authority, which could lead to formal enforcement action against the bank and individuals.

To date, Mr Pester is the only senior figure to have lost his job over the fiasco, although a number of other managers have also collectively had millions of pounds in deferred bonuses cancelled by TSB's remuneration committee.

He faced criticism from Andrew Bailey, the FCA chief executive, for failing to be "open and transparent" about the IT issues.

Mr Pester has already surrendered one £1.6m instalment of a deferred share-based Sabadell Integration Award, with no TSB executives receiving bonuses for 2018.

The report's publication will come less than a week before a strategy update from Debbie Crosbie, Mr Pester's replacement as chief executive, that will involve substantial job cuts and branch closures.

A TSB spokesman declined to comment.