What have system administrators ever done for us?

In his book The Atrocity Archives, author Charles Stross wrote: “Idiots emit bogons, causing machinery to malfunction in their presence. System administrators absorb bogons, letting machinery work again.”  Although pure fiction, there’s a grain of truth that will strike a chord with those familiar with system administration.

The role of the system administrator, or sysadmin for short, can be a thankless one.  When everything is working, sysadmins are barely noticed; when things go wrong, however, they are often the first people to be called, or blamed.

Compared with other IT roles, sysadmins may feel overworked and under-appreciated.  Although developers and coders are responsible for products or services, it’s sysadmins that are accountable for the continued business operations.  They ensure the upkeep, configuration and reliable operation of the computer systems and servers.  If the systems are the foundations of the organisation, then sysadmins are the mortar that holds it together.

“Sysadmin is one of those jobs where, if you do it right, no one even knows you’re there,” says former sysadmin Dave Lear, a lead security architect at an end-user organisation. “But when things go wrong, all of a sudden you’re the number one bad guy, because you didn’t fix it.”

Despite the necessity of their role, sysadmins may struggle to meet the many demands placed upon them, often holding together an organisation’s network infrastructure, with limited resources, while causing minimum disruption to other teams.

In theory, the simplest solution to the workload problem would be to recruit more sysadmins.  However, the ongoing skills gap within the technology sector means that finding a sysadmin with the appropriate skillset can be difficult and expensive.  Also, in the current economic environment, budgets may not be able to stretch to recruiting more staff.

Therefore, policies and processes are required to enable sysadmins to operate more efficiently and with the minimum of hassle and interruptions.

Opening channels of communication

Communication is key for any proposed efficiency improvement programs.  Unfortunately, the highly technical world of sysadmins can seem impenetrable to the uninitiated, leading to confusion and misunderstandings.  It can be helpful, where possible, to have staff with sysadmin experience assigned to liaise between the sysadmin team and the rest of an organisation.  This could be as part of an existing role.

“When I made the sideways move into the security world, I kept all my technical knowledge and it stood me in good stead for understanding the way forward,” recalls Lear.  “I was always the one to monitor IT and the desktops. It helped walking in knowing that [being a] sysadmin is a thankless task and how to talk to them and explain what my role is.”

Automating mundane tasks can ease the demands placed on sysadmin teams.  There is an overwhelming choice of tools available and what may work for one sysadmin team may not be suitable for another.  There needs to be a curation of the available tools in order to find the most appropriate tool(s) for the sysadmin team.  Although there will be an initial expense, the benefit of investing in sysadmin tools is that they allow teams to automate simple tasks and focus their attention on critical duties.

Peter Gatehouse, a network administrator and former sysadmin, jokes: “The key step is eliminating those pesky users.”  Although intended in jest, there is an element of truth in this.  The more people using a system, the greater the sysadmin’s workload will be.  Changing any system, such as releasing a new product or upgrading an existing service, can lead to unintended consequences.   This, in turn, can increase the demands expected of the sysadmins, exacerbating stress and potentially leading to expensive overtime.

Disruptions can be minimised by consulting sysadmins during the development process.  Ongoing consultations between development teams and sysadmins enables the sysadmin teams to be forewarned of any future upgrades or new products, allowing them to have the required infrastructure in place, ready for when the development is released.

“What you ultimately need is somebody with that knowledge on the development team, who understands the hardware and software requirements, and if they can't answer those questions, they need the help of the sysadmin team,” says Lear. “It need not be a dedicated to part of the team, but someone needs to be there as part of the knowledge share.”

Having a set of mutually-agreed priorities can allow sysadmins to effectively schedule their tasks with the minimum of disruption to the wider organisation.  These established priorities can also prevent sysadmins from being dragged into inter-departmental ‘tugs of war’ competing for their time.  These are not only disruptive to the organisation as a whole, but can lead to resentment, distract sysadmins from their duties, and waste time.

“That’s where the project management team steps in. They don’t necessarily need to have intricate knowledge of IT, but they have an oversight of what’s happening and when they’re planning to do it,” explains Lear. “They can deconflict at that level and, in an ideal world, they would have the ear of somebody at the top of the chain, who would set those priorities.”

An overlooked hero?

Of course, being a sysadmin is not all bad.  The simple task of restoring an accidentally deleted file will engender huge appreciation.  “That's one of the simplest fixes, because the sysadmin can just go to last night's backup and there you go,” laughs Lear.  “They might not have the work they did this morning, but they've got it back to where it was last night and then, all of a sudden, the sysadmin is a god among men.”

Sysadmins are often overlooked until things go wrong, hence they may feel under-valued.  However, they form a critical part of ensuring an organisation’s continued operations, regardless of their business sector.  As such, it’s vital to ensure that sysadmins have all the necessary tools and resources to perform their duties with the minimum of disruption from conflicting priorities and that system administration is incorporated into business plans.