Genome project reduces storage requirements
The Durham-based research centre has reduced its VMware-based genomic information storage requirements by 83 per cent with data deduplication technology.


The Duke Institute for Genome Sciences and Policy (IGSP) has revealed it has achieved a dramatic reduction in the virtualised storage capacity of its ever-growing genomic research information.
The Durham-based institute is dedicated to the study of life through scientific advancement in genetics and genomics. It requires substantial amounts of data storage for the creation and analysis of genomic information.
Over a two-year period, this saw its data storage infrastructure requirements rocket from 4TB to 300TB, made up of large Oracle databases on dedicated servers and locally attached storage.
Rob Wagner, Duke IGSP database administrator said: "We just bought more disks and kept giving them to Oracle. Over time this proved to be very difficult to manage. Server sprawl and inefficient data storage use became a constant headache."
In order to solve these issues, the IGSP IT team implemented a server virtualisation project to consolidate 40 physical servers down to just three running the Oracle applications. At the same time, it also migrated and consolidated its locally attached storage to a networked fabric attached storage (FAS) system from NetApp.
The project enabled the institute to use data deduplication techniques to identify and eliminate redundant data across applications, as the technology is a key component of the NetApp core operating architecture, Data ONTAP.
Alan Cowles, Duke IGSP system administrator explained: "Previously, I spent more time managing 60TB to 70TB of direct-attached storage than I do managing the 225TB of NetApp storage we have now."
Get the ITPro daily newsletter
Sign up today and you will receive a free copy of our Future Focus 2025 report - the leading guidance on AI, cybersecurity and other IT challenges as per 700+ senior executives
The current IGSP environment consists of a 40-node high-compute server farm, three VMware ESX servers and 50 mixed-use application servers, where all data is provisioned from a NetApp FAS3070 clustered storage system.
Cowles said deduplication now plays a major role in managing data growth. "When we originally set up VMware, I allocated about 2.4TB for it. With NetApp deduplication, I've been able to shorten that down to less than 700GB. We now see an average of 83 per cent reduction in redundant data on our VMware systems."
A 25-year veteran enterprise technology expert, Miya Knights applies her deep understanding of technology gained through her journalism career to both her role as a consultant and as director at Retail Technology Magazine, which she helped shape over the past 17 years. Miya was educated at Oxford University, earning a master’s degree in English.
Her role as a journalist has seen her write for many of the leading technology publishers in the UK such as ITPro, TechWeekEurope, CIO UK, Computer Weekly, and also a number of national newspapers including The Times, Independent, and Financial Times.
-
Asus ZenScreen Fold OLED MQ17QH review
Reviews A stunning foldable 17.3in OLED display – but it's too expensive to be anything more than a thrilling tech demo
By Sasha Muller
-
How the UK MoJ achieved secure networks for prisons and offices with Palo Alto Networks
Case study Adopting zero trust is a necessity when your own users are trying to launch cyber attacks
By Rory Bathgate
-
Lenovo and VMware expand partnership to offer turnkey Nvidia-powered AI solutions
News The collaboration aims to help customers leverage purpose-built Nvidia AI solutions for more predictable outcomes
By Daniel Todd
-
VMware brings XDR capabilities to Carbon Black in a push for lateral security
News The cloud giant aims to provide customers with the means to identify and rectify weaknesses across their environments
By Rory Bathgate
-
NetApp teams up with VMware to help businesses migrate enterprise workloads
News Amazon FSx for NetApp ONTAP is the first native AWS cloud storage to be certified as a supplemental datastore for VMware cloud on AWS
By Daniel Todd
-
Dell EMC PowerEdge R650xs review: An ‘xtra special’ rack server
Reviews A powerful and expandable 1U rack server that’s ideal for businesses on a strict budget
By Dave Mitchell
-
Dell EMC PowerEdge R250 review: A powerful package at a promising price
Reviews A compact and powerful Xeon E-2300 rack server with room to grow and great remote management
By Dave Mitchell
-
Dell EMC PowerEdge R350 review: A compact and powerful server
Reviews Dell EMC’s entry-level rack server packs a great specification into a space saving chassis
By Dave Mitchell
-
Virgin Media O2 partners with VMware to speed up 5G rollout
News The UK operator has committed to deploying 5G services to 50% of the UK's population in 2023
By Sabina Weston
-
Dell EMC PowerEdge R450 review: Rack-dense server power for SMBs
Reviews A versatile all-round rack workhorse with plenty of hardware choices at a good price
By Dave Mitchell