O2 data services have been fully restored following yesterday's near-24 hour outage caused by a software fault in one of its supplier's IT systems.
Up to 32.1 million UK users experienced difficulties as 3G and 4G suffered a complete blackout, with additional reports of voice and messaging issues. Organisations that rely on the O2 network, such as Transport for London (TfL), also found themselves facing problems.
Customers only had online access via Wi-Fi until engineers restored 3G services at approximately 9.30pm last night, with 4G services fully restored earlier this morning after 3am.
Several hours after O2 confirmed it was aware of the issue, the networking giant pointed the finger at Swedish-based telecoms firm Ericsson, one of its main suppliers.
Ericsson said its engineers recorded a number of "network disturbances" in the early hours of 6 December, and subsequently identified an issue in nodes in the core network. These led to network outages for a "limited" number of customers, through two faulty versions of the software SGSN-MME (Serving GPRS Support Node - Mobility Management Entity).
"The faulty software that has caused these issues is being decommissioned," said Ericsson's CEO in the UK and Ireland Marielle Lindgren. "Ericsson sincerely apologises to customers for the inconvenience caused."
Customers in the UK were reporting network difficulties as early as around 5am yesterday, with complaints reaching a crescendo during the morning commute.
Meanwhile, Ericsson's software glitch not only affected O2 in the UK, however, but hit a host of other companies such as Softbank and users in several other countries including Japan.
Telefnica UK's CEO Mark Evans apologised for the outage and said the company was monitoring its networks before embarking on a full review to determine what happened.
"We can now report that our 4G network has been restored," the company posted on its network status page. "Our technical teams will continue to monitor service performance closely over the next few days to ensure we remain stable.
"A review will be carried out with Ericsson to understand fully what happened. We'd like to thank our customers for their patience during the loss of service on Thursday 6 December and we're sorry for any impact the issue may have caused."
Tim Callan, who is a member of the industry body that started the Extended Validation (EV) certificate, suggested that yesterday's outage may have been caused by unexpected certificate expiration.
"This episode illustrates the essential role certificates play in keeping IT infrastructure safe and running and also the risk that enterprises face if they don't have a firm handle on the certificates installed in business-critical systems.
"The proliferation of certificates and ever-increasing complexity of IT infrastructure has made it more and more challenging for IT professionals to stay on top of this component of their networks."
Opsview CEO Mike Walton, meanwhile, said the O2 outage is just the latest in a line of many and stressed it was important that IT departments across all industries have full visibility across the entire estate.
"By implementing a robust monitoring system mobile networks such as O2 will be able to identify potential outages not only across the infrastructure, but also across all applications," he said.
"By doing so, companies can ensure that their time to detect problems is vastly reduced and any potential issues can be rectified before they cause disruption to consumers, and reputational harm to their brand."
06/12/18: O2 hit by severe 4G network outage across UK
O2 is experiencing a severe outage to data services with customers across the UK unable to access 4G and network features since early this morning.
Users have been reporting issues since 04.45 today, according to system information site DownDetector, which shows a spike of complaints and severe outages across the entirety of the UK.
O2 confirmed it was aware of difficulties with its data services in a tweet sent this morning at approximately 07:30, and subsequently confirmed the source of outage lies with a software issue in a third-party supplier's IT system.
While data services such as 4G are out of action, the network operator confirmed voice calls and text messaging was working as normal, but several users have complained they do not have phone signal either. IT Pro approached O2 with further questions but did not get a response at the time of publication.
While it's unclear exactly how many users are affected, the outage has also hit customers signed-up to mobile operators that use the O2 network, such as GiffGaff and Tesco Mobile.
According to Telefnica, O2's parent company, approximately 32.1 million people use the network, with the parent firm claiming to be the UK's "largest mobile network carrier".
But the issue isn't just isolated to consumers, with organisations and public services also affected, including Transport for London (TfL), which use the O2 network for its signalling and communications.
The organisation confirmed on Twitter countdown tickers used by buses across London is currently experiencing difficulties with limited information being shown to commuters on screens at bus stops and on mobile phone apps.
The networking giant is asking its customers to rely on Wi-Fi for now, while asking them to review its status checker for updates.
IT Pro was only able to access the site intermittently, however, presumably due to a vastly-heightened volume of traffic the website is coping with.
