The Scottish government’s Agriculture and Rural Economy (ARE) Directorate has quietly become one of the most agile, dynamic government agencies as part of its longstanding relationship with Pure Storage.
Responsible for coordinating subsidies for businesses and farmers operating in agriculture, the ARE Directorate plays a crucial role in ensuring the industry in Scotland remains productive – and, crucially, profitable.
Serving upwards of 20,000 farmers, the ARE Directorate pays out more than £700 million in subsidies annually. These payments are vital to ensure farmers can operate and the country remains fed. But to ensure it can continue to serve the industry, the division was forced to adopt an agile approach to innovation.
This began with an ambitious overhaul of its storage infrastructure eight years ago, according to Neill Smith, head of IT infrastructure at the ARE Directorate. Speaking to ITPro, Smith says the decision to embark on this transformation process was due to a changes to its subsidy application practices and the fact teams were wasting valuable time and resources maintaining existing infrastructure.
Ditching the shackles of legacy hardware
Reforms and changes to subsidy applications meant development teams at the ARE Directorate were forced to build a raft of new applications to accommodate for new schemes and an increased volume in user traffic.
“During our last big agricultural reform piece in 2014, we were revamping our subsidy based applications,” he says. “All of Europe did it.
“We were developing using new technologies both for applications and hardware, and we had traditional storage with Dell EMC. This was a large project, but what we noticed was that we had a large number of developers – circa 100 or thereabouts – and they were doing around four builds a day.”
Neill Smith is a Cloud, Virtualization and Linux specialist. He is currently the Head of Infrastructure at The Scottish Government (ARE Directorate). Neill specializes in leading and managing multidisciplinary technical teams.
He also has strong hands-on technical knowledge, whilst being highly proficient in Cloud, Virtualization and DevOps technologies. Neill is an accomplished public speaker, having previously spoken at numerous major technical conferences including AWS re:Invent, VMworld and Gartner Infrastructure, Operations & Cloud Strategies.
We’re all taxpayers here; we’re trying to drive good value.
Conservative estimates show 30-min build times were reduced by around half, saving roughly an hour a day for developers and relieving immense pressure on the team. “It was making those developers far more efficient,” he says.
“They were never kicking off a build and twiddling their thumbs, but it was instantly recognisable that we were pushing out releases and achieving a quicker path to live [for applications].”
Each of these builds, Smith explains, took an average of 30 minutes per developer, consuming a significant amount of time from each day.
The reality for Smith was its existing storage infrastructure wasn’t able to contend with the scale of demand, and the pace of development. This was placing significant strain on the division.
Farmers have a lengthy window to submit subsidies through the ARE site. However, in the run up to deadlines, there’s often a traffic surge, meaning performance is key. Combine this with the rapid pace of app development, and it was obvious change was required.
With this in mind, the ARE Directorate began exploring alternative storage options, such as solid state drives (SSD), Smith says. After an extensive procurement process – that included a proof of value (PoV) that saw teams compare the performance of different solutions – the department selected Pure Storage’s FlashArray. This option won both in terms of performance and cost. “We’re all taxpayers here; we’re trying to drive good value,” he adds.
Organizational buy-in on hard mode
Smith notes while the storage transformation process was relatively streamlined, a key challenge the team encountered was emphasizing the benefits of the overhaul itself. In this way, working within a public sector environment, with taxpayer value in mind, presents different challenges compared to similar projects in the private sector.
Some government departments can be inherently resistant to change, Smith suggests. “Sometimes it’s easy enough just to keep the lights on and not take risks,” he adds. However, in this instance, Smith was keen to drill home the long-term benefits that this overhaul could deliver for the developers, broader department, and, ultimately, the taxpayer.
“The government is always resistant to change,” he says. “I think that’s the nature of the beast in a political world – and that’s a prevailing trend in any government. There’s a fear factor but I’m always on the opposite side of that.”
Smith recalls that framing the project as a value driver was critical to ensuring it had the necessary approval and backing. “It’s how you frame it,” he continues. “It’s about us being able to provide a better solution and use cool new tech.
“It’s about creating the right culture and allowing people to fail and get things wrong. But in doing that, you’re allowing people to relax and deliver something that’s actually going to be fit for purpose rather than rushing through a solution.
“You might deliver it on time, but then it ultimately becomes a higher operational overhead because you’ve implemented something that’s not sound.”
Going against the grain on containerization
Smith tells ITPro the ARE Directorate’s current focus is on embracing containerization for core applications as part of what he describes as an “agnostic multi cloud approach”.
The flexibility of containerization and the portability benefits it affords was something that the team had been considering for some time. Smith adds that while many organizations have traditionally went “all-in” with a specific hyperscaler, the ARE Directorate was keen to explore alternative options.
Download this eBook today and find out how Dell is helping solve some of the world’s most pressing and significant breakthroughs
“We were looking at containerization, or the orchestration of containers, so Kubernetes was very much on our agenda,” he explains. “But we were seeing the bigger picture in terms of portability with containers. Having the ability to run that container when you want, where you want, was exciting.
“We were already on our AWS journey by then, but I wanted to explore if I could possibly look to do an agnostic approach to multi-cloud - so move a containerized workload on-premise on a virtual machine (VM) estate, running in OpenShift, then move that into AWS.”
For this, Smith once again engaged with Pure Storage, namely Portworx. The storage giant acquired the startup in 2020 in a bid to support and accelerate customers’ hybrid and multi-cloud strategies. The team conducted another PoV with Portworx and quickly realized there was scope to deliver marked improvements to flexibility and developer productivity.
This proof of value involved taking a core application running on-prem, migrating it to AWS, and then Azure, before bringing it back on-prem. The entire process was completely in “around one and a half minutes”, he says.
The Portworx roll-out took roughly two weeks , Smith adds, and has since unlocked significant value for the broader team in terms productivity, efficiency, cost, and, crucially, security.
If a cloud provider encounters a security incident or breach, then workloads can be taken back on-prem to bolster safety. Similarly, in an era of rising cloud prices, the ability to take workloads back on-prem and avoid costly overheads is another key benefit, he says.
Portwork now forms a core part of the ARE Directorate’s multi-cloud approach, Smith says. And, moving forward, the department sees the flexibility of multi-cloud as a means to drive further innovation, deliver long-term value for the division, and continue supporting the agriculture sector across Scotland.
