Every office comes with its own unique quirks — we all know this, of course. They might include dodgy windows that won’t close properly, meeting rooms with broken webcams, or an unnecessarily loud colleague. But spare a thought for Arturs Banks and Rajeev Bhandari, head of IT and third-line infrastructure analyst, respectively, at Fulham FC, whose workplace environment is highly challenging.
Managing the IT and cyber security needs of a professional soccer club is challenging for a number of reasons, chief among which is the ‘office’ layout. There aren’t many headquarters in the world, after all, that’s cleaved out its core, forcing staff to take the longest route possible when traveling from A to B.
The geography of Craven Cottage, home of Premier League stars such as Alexander Mitrovic and Andreas Perreira, is vastly different from that of, say, Apple’s Cupertino campus. The layout of Banks and Bhandari’s office — a 25,000-seat stadium in West London — they say, proves a major headache.
The soccer stadium struggle
“It's not a data center environment, so often our equipment is in various places around the stadium, because of the size of it,” says Banks. “We have to have a lot of local equipment dotted around to supply the whole stadium so we can't just have a centralized point where everything is air-tight and up to the standards that most larger organizations or more office-like organizations would have.
“You can have a van driving over your fiber cables, or a beer keg being dropped on your data cable. That is the reality of our day-to-day.”
Banks has been Fulham FC's head of IT since September 2019, and was previously the soccer club's third line support analyst - having been at the organization for more than six years. He was previously IT system engineer at Metro Bank.
Keeping the stadium connected and ensuring systems remain online is absolutely essential to the smooth running of match days for Fulham FC. Everything from site-wide security to keeping vendors’ payment systems online relies on Banks and Bhandari’s expertise. And, to keep these systems online, the team relies on an arsenal of equipment.
“We have network locations dotted around everywhere,” says Banks. “So it's not like one rack with three switches, and that runs everything. No, it's 20 racks with three switches each and various other pieces of network equipment.”
All equipment is spread between the four stands of the sprawling Thames-adjacent stadium, which makes patching each switch the biggest of their headaches, they say. Adding to the obvious geographic complexity, the pair also need to instigate upgrades in stages to minimize downtime.
Top-flight cyber security
Windows and Linux servers are patched monthly, they say, but often networking equipment is only patched when the team receives “a serious enough bulletin” from vendors such as Cisco. The strategy falls in line with most other organizations, realistically speaking. Of course, in a perfect world, everything would be patched seamlessly and lightning-fast, but with issues arising from applying fixes, it’s common to see patching strategies take a similar form to Fulham’s.
Banks says the ‘best’ phishing attacks haven't been email-based, but instead delivered via WhatsApp.
A first-team medical team member, for example, one day received a message purporting to be from the club’s CEO, requesting information on a specific player. Fortunately, the staff member recognized the true nature of the message and reported it to the IT team.
Being a medical professional is high-pressured enough. Stuck in a cramped dugout on match days, in a frantic environment – perhaps after just sprinting hundreds of meters to treat an injured player – it’s easy to see how one could fall for such attempts.
Like any Premier League soccer club, the need for robust cyber security is essential. Fulham were promoted to the top flight for the 2022/23 season, having spent the previous three seasons in the second division. The team had to be even more scrupulous than they were used to. Phishing, for example, is a major concern, given the Premier League’s requirement for club officials’ contact information to be posted online. Attacks, therefore, targeting the CEO and finance director, among other senior staff, are common.
Fulham FC has never actually succumbed to a serious attack, and it’s perhaps due to the team’s round-the-calendar work to achieve a state of cyber readiness that’s contributed to the achievement. When the weekend stars spend time over the summer relaxing in various sun-soaked locations across the world, Banks and Bhandari are commissioning annual penetration tests to strengthen the club’s cyber resolve. The most recent one, Bhandari says, shone a light on the club’s password practices.
“I think the biggest [issue] we had to take quite serious action on was password complexity,” Bhandari says. “Basically purchasing a tool that will restrict password reuse and enforce more complexity onto passwords.
Bhandari has been third-line infrastructure anlyast at Fulham FC since January 2020, and his responsibilities include managing a Cisco network of switches, routers and typical network devices like printers and CCTV units.
“There was also a vulnerability that allowed the pen testers to gain access to our domain controllers. But, that was on an authenticated network with credentials, so they were kind of halfway there.
“But they managed to find one very old unpatched Linux box, which we actually didn't even realize was still running. Then once they actually decrypted our AD passwords, that's where we saw the biggest report. So it was a 100-page report and 99 pages were just bad passwords.”
The club already enforces a three-month password reset, but the pen test’s findings indicated it wasn’t enough. They invested in Specops’ password policy tool to enforce greater complexity beyond the weak passwords that were too often used. Bhandari says the names of the manager and key players, for example, featured heavily in passwords and weren’t the most secure.
A backup strategy for the modern age
Bolstering its cyber security posture even further, Fulham FC recently signed a three-year partnership with Acronis and its technology partner EveryCloud to implement Microsoft 365 backups across the club and the club’s charitable foundation, and disaster recovery for the foundation only. The backup solution for the club’s Microsoft 365 files is already up and running, with the remaining upgrades expected to complete shortly. It’s the key component of Fulham FC’s partnership with Acronis and EveryCloud.
Trend Micro security predictions for 2023
Prioritise cyber security strategies on capabilities rather than costs
It’s also already proven its worth; in just one month, for example, the team restored a staffer’s lost email files. Fulham’s IT experts declined to detail the nature of the deleted emails but did say such incidents “could have quite significant impacts”, including, say, carrying implications for player contracts.
Agreements made over email are contractual and binding — so if an agreement between two clubs is disputed, and one couldn’t provide the evidence to support its position, it would carry huge ramifications.
“We can't quantify these impacts because we just provide the service to the business — a lot of it is outside what we get told,” says Banks. “So it could have been a dispute over a £5 refund for a ticket, it could have been a dispute of a £70 million player.”
Banks also highlights the importance of having such measures in place at Fulham FC’s foundation, which runs soccer outreach schemes with children and disabled players, among other initiatives. Banks alluded to a child sexual abuse scandal that rocked English soccer in 2021, using it to underline the need to have records of emails and other files when safeguarding vulnerable people.
Backups, in addition to preventing the loss of sensitive files, also act as the club’s ransomware defense. The club stores 30 days’ worth of daily backups, after which time the only restore point is the first day of every month. Beyond that, the club keeps restoration points on a once-yearly basis.
Asked whether the club would ever consider paying a ransom in the event of an attack, Banks replied with an emphatic “no”.
He adds Fulham FC has built its backup plan so it can accept some degree of loss and rebuild within three hours. When it comes to finance systems, file shares, domain controllers, and similar systems, that’s “data we want to protect”.
In the absolute worst-case scenario, if a ransomware attack took hold an hour before kick-off on match day, Fulham FC has the power to push a figurative “big red button” to initiate a near-total IT shutdown to begin a system restore.
As long as any attack doesn’t impact systems that protect fans’ safety, such as CCTV cameras, radios, and stadium floodlights, a system shutdown could commence. In this scenario, stadium turnstiles would open fully to anyone who approached them and Fulham would simply “assume that everyone who comes in has the tickets”.
If fan-safety systems were to fall, that’s when the Premier League itself would step in, either delaying or postponing the game. Luckily, Fulham FC has never been targeted with ransomware to the extent it’s prevented a match from taking place.
Naturally, Fulham FC stores as much data as any large organization. From fan and member data, to commercial contracts, player performance analytics — all the way to match highlights from the 1900s. As such, the club must be smart about what data is backed up and what isn’t.
“Anything that's nice to keep, but isn't business critical, we potentially don't back up and just keep it on redundant storage,” says Banks.
“Because the other thing we have to take into account with cloud backups, and the more files we send to Acronis, the more impact it has on our bandwidth. So, obviously, we have to run these backups overnight, and be selective because we haven't got unlimited bandwidth. We have to strike a fine balance between cost, performance, and our requirements.”
