This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Uprooting legacy tech'. We apologise for any errors.
Rory Bathgate
Hi, I'm Rory Bathgate.
Jane McCallion
And I'm Jane McCallion.
Rory
And you're listening to the IT Pro Podcast, where today we're discussing legacy tech.
Jane
Most companies that have been in business for more than a few years will have old technology stacks. While most firms are following some kind of digital transformation strategy, shedding legacy tech is no easy job and can lead people to take the path of least resistance, using a mix of old and new tech across their IT estate.
Rory
This can lead to big issues down the line, however. A very public recent example is the outage that led to thousands of flights being cancelled and delayed across the United States, which the FAA has blamed on accidental file deletion while updating a database. The FAA is heavily-reliant on old systems, and has said it will take until 2029 to phase out old methods such as paper flight strips.
Jane
Today, we're speaking to Ravi Mayuram, SVP of engineering and CTO at cloud database platform Couchbase, to discuss the need for businesses to upgrade their infrastructure. Ravi, thanks so much for joining us.
Ravi Mayuram
Thank you for having me. It's a pleasure to be here.
Jane
So can we start off, could you define what we mean by legacy infrastructure? Is it something that's old, is it obsolete, is it difficult to manage? What is this thing?
Ravi
Yeah, I think you know, we can maybe define legacy to be something that has been in production, in use for at least 15-odd years and more. And many of the systems that we are generally talking here have been in use, in production for at least 40-odd years. And so that definitely is legacy. It's, you know, to use an analogy here, it's like using landlines when the modern technology is sort of mobile and cellular phones. It's almost like that, many of these technologies that we're talking about in the stack were designed and invented before the internet came into be. And so you can see how that impacts your thinking and the access to those systems. And that's what leads to sort of some of these issues that we're seeing now.
Jane
Yeah, so more old than necessarily obsolete? Would you say?
Ravi
Well, obsolescence in technology happens very quickly. So it is obsolete on a certain level, but they are still useful in many ways. So in that regard, they're not useless. Definitely, there's a lot of use in them, but they are not up to the modern challenges and demands. And in that process, you find that they're difficult to maintain, difficult to use, and they have, you know, the cost of obsolescence is also high. That's where these things come into play, where you're sort of sitting on systems which are very old, and very few people know what's happening inside. These are very complex systems and enterprise software. And like what you mentioned recently happened, these are all very sophisticated systems built over many years, and layers and layers, various years of technology, various generations of technology, if you will have sort of layered on each other and ossified.
Jane
Yeah.
Ravi
Now, these are difficult systems. And so, modernising them is one of the sort of initiatives that many of them take.
Rory
So on this issue of modernising layers, what are some of the immediate challenges that it comes to when you're handling legacy infrastructure?
Ravi
Yeah, I think the the biggest thing is that what has really happened in the last, you know, decade or so perhaps a little bit more than that, is that it's the movement of direct to consumer, if you will. Which means that you and I get to directly interact with many of these systems which were actually built for, we used to call them backends, which were built for specialised people to work and operate and interact with them. Classic example, not to beat this, but it is basically very easy for all of us to relate to is travel reservation. You know, there was a time only travel agents would interact with these systems. Now you and I have straight access to that thing, including the pricing, the inventory, the availability, the flight schedules, the routes, everything's available to you. It'll tell you, "the middle seat is you know, $50 or £50 cheaper than the aisle, right? So you have, that means the pricing engine, the inventory, the availability, everything is coming directly to your thumbs. Whereas earlier, you had to have specialised people. So it's disintermediation of the intermediary, that's the sort of the generic theme. Direct to the consumer is the movement. So all these systems were built for enterprises where specialised people could interact with them. But now, you know, the common, millions of folks are interacting with them directly. And that basically poses two issues. One is scale, instead of tens of thousands, or hundreds of thousands of people sort of accessing them now, immediately millions and even billions of people are accessing them with, you know hundreds of millions concurrently using these systems. So, straightaway those systems which were built for, not for this scale start to show. And then, you know, then the ease of use, usability, maintenance, all this comes as a consequence of this. And those systems were not built for this, they could take a downtime of a night. Nowadays it's five nines availability, you cannot have the systems go down for more than five seconds in a year or something. So yeah, those systems were not built for it. So now how do you modernise it? It's like a ‘fixing the plane while flying it’ situation. So these are complex ones. But that's what the challenge is, that's what people are undergoing. So that I hope that gives you a flavour of the kinds of problems that that is existing in the system and why it came to be
Jane
Sure. Is there an element of risk by which I mean, you've kind of touched on sort of business continuity risk there if there's any downtime. But is there an element of potential cybersecurity risks if you're running older systems, and they are more vulnerable to attacks? And if you don't have the skill set to understand them and patch them, then you're going to have further problems?
Ravi
True. It’s at two levels, it's an absolute binary answer here. There are two sides of this equation. First, is that yes, because of these systems being so old there are systems here, which were written in COBOL and other sort of programming languages where there are physically they don't, they cannot find the resources, talent to actually work on this system. So part of it is people are a little bit scared in touching these systems, because people don't know what's happening inside at a certain level. Right? So they have to be careful in how they have to modernise it. And so it's, people are a little bit gun shy, correctly so. I wouldn't want to do my banking or fly an airline which is taking this lightly. So they are careful, and that's required. So on the one hand, to your security question yes, there could be loopholes which nobody would even know how to identify and fix. That's possible. The flip side is also a security thing that people are acutely aware of, which is that when you now modernise the system there are a whole bunch of modern challenges, because now you're exposing that system to everybody. That brings a different level of security and threat. And so that has got to be secured. So both sides, you know, it's not a clear case of one side fixes the problem, but it is a case of you have to modernise from the standpoint of at least you know, where the vulnerabilities are. With the older ones you don't even know, because then that's when these kinds of surprises happen. When you didn't even know what is going on, somebody was trying to do something correct and yet, it causes because these are complex systems and a little, like they would say a little butterfly flapping its wing can lead to a huge storm on the other end. That's what happens in many of these situations, without knowing what the systemic impact is, if you make a minor change it shows up somewhere else as a huge issue. So that goes away when you go from a legacy to a modern, but it does bring some new challenges of exposing your system to millions of users, and the way you have to secure that. But the good news out there is it's a well known problem, there are many companies working on it. And there's a lot of talent which can go and address that and fix it, versus the other one where you were hard pressed to sort of go fix those. Even if you find that talent, the cost of maintaining older systems always gets more and more expensive. And you might not have the right kind of returns for it.
Jane
Yeah, absolutely.
Rory
I'm interested in this point you raised of skills, of people being actually able to interface with the systems. Is there a sense of urgency here, that we need to modernise these while these people are still on the payroll or maybe that we need to train people up to be able to handle some of these systems?
Ravi
Absolutely, there is that sense of urgency. And sort of, we have ourselves seen this, when we work with many in the airline industry in the sort of operations of airports and reservation systems, which are all huge systems. Not just here but in retail and many other sectors, in every sector, the same thing is happening. Just the very simple act of curbside pickup completely changes the backend systems in retail industry, because the entire system was set up for everything to be brought to the back side of the warehouse and then stocked in shelves. Now you're saying well, direct to the curb where somebody can pick it up. So the whole mechanics, the logistics of this completely changes like this, every industry has its own. You can see the whole streaming, how that is changing the entire television and set top box to going away to cloud, DVR is how the whole thing changes. That gaming changes, instead of it being like console based, nowadays it’s mixed where half the stuff is happening in the cloud where inter-player games could be played yet, for speed and stuff, you can use your local consoles. It's this kind of transition is going everywhere. And the scale is the defining factor in the middle of all of this. And so your modern systems have to basically address that scale, and performance at scale. These are the two main things. Yes I can scale, but it's going to be slower and slower and as more people walk into the system it doesn't work anymore. The first user, as a customer put it, the first user and the billion user both should have the same experience. Saying to the billionth “So take a number and stand in line while I service the first one” doesn't work. So that's a sort of the change that is sort of going through, and people are sort of modernising. And there is urgency, simply because of this just to give you statistics. You know, when you're trying to book anything, a hotel room or or flight, they used to say the ratio was 100:1. 100 looks before one book happens. Nowadays it’s 10,000:1. 10,000 looks are happening before one booking happens, because a lot more of us are exposed to these systems obviously. The same thing is happening in banking. Anytime you introduce your mobile banking, the amount of times people are checking their bank balances seems to go up 100 times. Otherwise you have to go to the bank and ask somebody to tell you what the balance is, now you fire up your thing, “Oh my God, what was the last time” so like that, and everybody does it two times, it's already, you know, 100 times more than what you would see in a day. So it's a scale, by which I mean it's two to three orders of magnitude and in some cases even more of accessing these systems. That's what it has basically come down to. And so there is a huge, there's already a bunch of stuff work already done in the last 10-15 years, but as you can imagine, the devices are exploding, the axis is exploding, everything's becoming untethered. And so correspondingly, there is a lot more traffic. And so there is urgency in many of these systems to be modernised, and some are undertaking it, some are a little bit slower. But the game’s on.
Jane
You kind of mentioned banking, aviation, and retail there. Are there any industries in particular, whether it's those or others, that are particularly prone to the problem of legacy infrastructure?
Ravi
You know by definition, legacy infrastructure means these businesses have to be in business and thriving in the last 40-50 years, or more. It might sound negative, but it is actually not because they made some of the smart decisions for the time when they made those decisions. And the times have changed. Classic examples are the financial industry. You can imagine insurance, banking, healthcare, these are all places where people put the best money on technologies of old, just mainframe systems. They were expensive, super expensive, but they knew they understood the value of data. But the same mainframes are now what's holding you back, because that means physically your request, let's say you're sitting in London, and you're making a request and to let's say, let's say if Couchbase, were to use a mainframe, it would mean basically mean that you would have to go physically hit that mainframe sitting in some data centre in Utah, or wherever else we keep it as a cheap. And that takes a lot of time, the back and forth of it. In these days, where you're just one click away from your competition, you will be gone. “This site is too slow, I don't want to be here”. So what is happening in the modernisation is basically distribution, it’s called geo distribution of data. That same data, which was sitting in one box is actually in the cloud and distributed whereas actually, the data is sitting very close to you in London someplace. So it does not have to make the pond hop to come and fetch the data. So this is how the modernisation actually has to happen, where the data comes to meet you at your point of engagement as opposed to your request having to travel halfway across the globe to come to wherever the data is housed. So that's the legacy story already as in, you know, your modern technology of old now is sort of becoming legacy. And you just have to modernise that to get to a point where it's relevant to the way people are accessing it now.
Rory
I mean, you've touched on a key solution just now. But what options do businesses have when it comes to handling legacy systems beyond just leaving them in situ?
Ravi
Yeah I think, like I said, modernising or going away from legacy systems is not an easy thing. So what you… many times whatever happens in technology, it sounds like you're disrupting something in the past, this past is obsolete, throw that away, come to something new. That doesn't work in these kinds of situations, because there is value that they can still provide, and they are complex. And these are multi-year projects sometimes to modernise them. So the way you have to go about this, and it's difficult to do in many cases, and only sometimes certain types of software could be used to do this thing is basically instead of going this route of “let me rewrite the whole thing in a new stack”. What you have to basically do is think like, they use an analogy, things like remodelling a house as opposed to rebuilding a house, where you basically have to think in terms of “I'm going to extend”, and then maybe redesign the entire old stuff, but put something new adjacent. And the way you typically, and the problem with many of this infrastructure is that they are the weight bearing sort of systems. If you touch them, it shakes the whole foundation so to say. So how do you make foundational fixes typically, in a remodel is you actually put a beam on top which takes all the weight. And then you can move walls and stuff like that. That's how we should think about this, what can you add into your mix of systems, which will take most of your sort of where the access is happening, and then that gives you… and here there are newer technologies that help you including sort of patterns that people use in using microservices, and those sort of technologies, because you want to take small chunks of it instead of rewriting the whole thing because that's a never-ending project. Say you just want to take one microservice away from your legacy monolithic system to modernise it, and use other systems which will give you this, I'm using this analogy of a weight bearing wall. Which you would know in your system, how you have to sort of take the weight off of the one that you want to modernise. And if you do that, that's how sort of we have seen many companies approach this, and we have sort of partnered with them. And this comes from experience, not not some theory.
Jane
So it's a case of looking, I guess, when you're thinking about you want to modernise your infrastructure, you're looking for reasonable use cases where you can do that and not necessarily looking to do the whole thing if it doesn't need to be. If there is this app that's running on this old server, and it's doing its thing, and it's not harming anybody we'll just leave it where it is.
Ravi
Yes I think you just need to, but that's the toughest thing. Because what would happen is that you say “this is the part I need to modernise”, but that is the part that is getting most accessed. So how do I do that without, but you're absolutely right. Which is basically you just have to study your system to say, “which are the parts I want to modernise in which order?” So I don't have to do this whole boil the ocean thing, because at the end of it, you're not going to find anything at the bottom of it useful. But if you basically do this in a sort of organised, thought through manner where you take appropriate levels of risk, you succeed. I have heard customers say that “look, my entire career has been trying to get off of mainframes, I’ve not been able to do it so far but now I'm able to do it, because there are sort of modern tools and technology available to do it”. Because it's, it's not easy. Number one, and number two, the problem basically is one of when you have to modernise you're, you're already behind the eight ball, so to say. That's the thing that basically slows people down from modernising, because they are there risk averse. So how to mitigate your risk is the first like you mentioned, that would be the best way to analyse the system and modernise the ones, that way you can sort of manage the risk. And when you take these small chunks away, the risk automatically becomes lesser and lesser as you proceed.
Jane
When we're talking about legacy apps, rather than legacy infrastructure, one of the things that's often put forward is containers. That you take your old apps, you shove them in a container, you put it in the cloud and that is the answer to your problem. Is that the answer to the problem? Or is it a bit more complex than that?
Ravi
It is more complex than that. And that is a mistake many people have made, and they don't realise that they made the mistake directly because it's not exactly one for one. But where it ends is that, oh now my cloud costs are through the roof, I’ve got to contain the cloud costs. This is where it ends without realising. Because what they're actually in one sense doing is what I call a cut and paste. They're cutting and pasting the enterprise architecture onto the cloud, that's what they basically did. They said “I'm going to wrap this up in a container and put it there”. But then what you're losing is, on the enterprise you had bought, whereas in the cloud you're renting. So obviously, this is going to cost you way more than what you did. Whatever worked for you in the enterprise will not work, just economics doesn't work there. That's what you need to understand, and this is where your earlier point comes into play. When you're modernising don't do a cut and paste, understand which part of your systems need to change. First move when you're doing that modernisation, consider cloud as the first place where you want to start there. Because it's now changed, you don't have to go about buying hardware, installing software, maintaining patching, upgrading, all that is taken care of by the service provider so already it’s relieving you of a lot of stuff. So use that to your advantage and sort of now write those pieces of software, which you want to modernise one microservice at a time in the cloud. And you know, bridge the gap, keep some in the on-prem some on on the cloud, until such point as you've decided to move everything onto the cloud. Or whichever hybrid is the architecture that works best for you, stay there. You don't have to necessarily move everything to the cloud, because one day you may not bring it back. Because regulations, there are so many things that are unknown out here. So people who are in that domain would know what makes sense to move and what doesn't, and that's they need to take the time to do.
Jane
Is it a question of maybe finding a partner, whether that's an MSP or a consultancy or something like that, to kind of guide you through this process?
Ravi
Yes, that would help. But you also need to have a lot of in house resources, who understand because that domain knowledge is your own, your own company’s. And that's what you're protecting in one sense, right? Your IP, so to say. And yes, people from the outside can help guide you, navigate because the landscape is confusing. But you also need to have your talent be trained for this new reality, because it's not…what got you here won't get you there. You have to learn your newest set of skills, and modernise. And your domain expertise is what you bring to the table, you can use external consultants and sort of ISVs and others who can help and help you build that stuff as well who have expertise in, in the modern sort of systems in the cloud and containers and orchestrators and all that stuff, and both are required. And then you have to build a talent pool in house as well. So because these are, if anything more and more your business is, critically it's not like business is running and IT is helping No, it has become one, your IT is your business at a certain level. You know, earlier we used to say you're a software company or a service company, all that is good but now you're a data company, every company has to become a data company. Because the next phase is more that every company will become an information company. In one sense, that's when sort of AI gets in the middle of all of this stuff and, you know, separates the noise from signal. So that's the evolution because that doesn't mean others go away. They are necessary, but not sufficient. You just need to have more and where is that going? You've got to be providing sort of, look at it as what data, what value is coming out of this because everything else is sort of already spoken for.
Rory
You've mentioned resources, the need to have resources in house. We're entering into a recession right now. You know there's layoffs happening, companies are kind of reassessing their budgets. How can they do this and at the same time be handling legacy architecture?
Ravi
Yeah, I think that's a very good question. It's about training your existing people, in my opinion mainly. Of course, you have to balance your resource allocation too from your business standpoint, there's a newer business reality for that, whatever adjustments you need to make. But why you are making that you have to also invest in your talent pool, your your employees, and find ways for them to be more knowledgeable. And it's not a training, which is basically, you know, “we just go for a training”, that's not the point. How do you apply that back to what you're doing, it's in that loop where managers and leaders in the company have to sort of sit and invest their time with the engineers and sort of get them to par with the technology stack that they are planning to adopt, in the modern architecture, and they be trained. And there's also other movements happening, the DevOps movement. Agility has been there, around for a while, but there are a lot of things changing even in how you can be agile. So training and all those, you know, it's not just a skill thing, it's a philosophy as well that is changing, which is where some investment is a good time to invest in one sense in those things, so that you can, when the economy comes roaring back, you're ahead of the curve.
Jane
And it related to a lot of what we've said, when it comes to upgrading these systems how can you do that without causing downtime, which will alienate your customers, and losing money? Nobody wants to be losing it, the last time we want to be losing it is now. What's the strategy for this?
Ravi
You know, each company will basically have to zoom in this specific sort of strategy, but when you are sort of going from your older stack to the newer one, one thing you basically may always have to keep in mind is that the older one is the known, the trusted one, and the newer one is a yet inexperienced system. So you may well have to consider building more redundancy, and more sort of resources into that. Sometimes you constrain the resources that you're going to put in the new, because you're sort of cost-conscious and stuff like that, correctly so. But you’ve gotta be prudent about where you're trying to be sort of cost-conscious in a newer system, give it a little bit more. Maybe just 10%, maybe 20%, that's what, it's not gonna be like double it, triple it. It's not like that, but you just have to have a little bit more resources. You know, just to use a classic example our resting heart rate is 70 beats per second, if if our heart was at best, if you said, “let's double it, 140 beats per second ought to be good enough for a heart, because most of the time, you're not even using half of it”. You know, if a dog chased you, and you ran up a flight of stairs, you will keel over because it'll go to 200, 300. So redundancy is important in certain systems, where you just sort of give it until you understand the system better, then you can tune it back. And so this is not going to be forever investment. And this is where cloud is very useful. Because it's about elasticity, you can provision more, and then you can put it back. It's not like in the good old days, where you have to say “oh, I need to go buy one more system, what is the CapEx of this thing? Can we really afford it?” and that goes away. So this is the advantage, take advantage of what cloud gives you elasticity. And you can actually build this redundancy into the system. So you know always have, these days it’s very easy as storage is also cheap, always have replicas of stuff. So in case one thing fails, another one is easily available. The good old ways of taking backup and then recovering is gone. Because by the time, let's say these days people are talking about hundreds of terabytes and petabytes, even if you take a backup just for storing that, and getting the system back into the health where it used to be, is a week long or a couple of weeks long in many cases. You can't afford that. So you just have to have a Hot Standby as people would call it, a replica of your running system. So that you can just switch over to that, this is how five nines availability is handled in many of the cases, using redundancy and replication as a strategy.
Jane
So if you have any sort of final advice to give somebody who is looking to modernise their infrastructure, modernise their business, what would it be? Where do you start?
Ravi
You know, coming from the buyers, where they come from, and what we have done so far, most of the sort of the problems that exist when you're trying to modernise is that you've modernised various elements of your stack already. Your UI is more modern than where it was before. Your application logic tier is sort of more modern than it was before. The place you had delayed the most to make changes in your data tier, because that's the toughest and that's the that's your foundation, right? That's the one you didn't want to move all this while because it delayed it as much as possible, because you could get away with it. Now you can't, you have to realise that, that the more you sort of stick with legacy systems, the more obsolescence and the cost of it. And it's not just the bottom line, it's a top line, also, you're not sort of now able to attract more newer customers, because you're not accessible to so many of them, or you're not easily available to them, those kinds of issues show up. So you have to look deeply into your data infrastructure, data tier, and see how we want to sort of modernise that and that's where these are all the weight-bearing walls situation, whenever you attach data infrastructure. And that's where we saw, even in the recent airline situation, it was a data infrastructure tier issue which somehow bubbled up to be this bigger issue. So you just have to modernise that tier, and that's not for the faint of heart, so to say. And that's why it has come this far. Now, if you don't take that challenge and start the journey now, you will find yourself on the wrong side of the digital divide. You may not cross the chasm and things will pass by, or you will be completely upended. All these things are happening, and many are aware of this, but yet there's a time for urgency to modernise that portion of your stack. And there are enough examples now of people having done it, so you can do this safely, and there are enough guardrails and resources available so people can undertake this one safely and succeed at this.
Rory
Well, Ravi, thank you so much for being on the show.
Ravi
Fantastic, thanks for having me. It was a pleasure to talk to both of you, you guys seem to know a lot of what's going on and I hope the listeners benefited from this lively chat.
Jane
Thank you.As always, you can find links to all of the topics we've spoken about today in the show notes, and even more on our website at itpro.co.uk.
Rory
You can also follow us on social media, as well as subscribe to our daily newsletter. Don't forget to subscribe to the IT Pro Podcast wherever you find podcasts. And if you're enjoying the show, why not tell a friend or colleague about us?
Jane
We'll be back next week with more from the world of IT, but until then, goodbye.
