JustGiving has been running since 2001, helping people to raise money for charity. Since its foundation, around 22 million users have raised £3 billion for their favourite causes.
What JustGiving wanted to do was find a way of connecting people with not only more worthy causes but also ones that would have greater meaning for them.
Mike Bugembe, chief analytics officer at JustGiving, says that fundraising has evolved and now goes beyond people running marathons or bathing in a tub of baked beans.
“What we've seen is the giving space itself hasn't moved as much as the other industries. It hasn't really transformed much. It is still quite transactional,” he says.
“Our objective is that giving has to become far more engaging. Firstly, it has to become social. The second one is that it has to be relevant. So this is where our challenge comes in."
He continues: “In order for us to be relevant in the giving space, we've got to show you stuff you care about. Some people care about animals, some people care about the elderly. There are different causes from cancer through to childhood illness and so forth. So the challenge is how do we find out using the data that we've got, what you care about?”
Bugembe says simply using traditional methods wouldn't work. This would normally involve looking at previous transaction history. Groups of people would be segmented and then the organisation would then look at other people that look like you and what they have typically given money to.
“The thing is giving doesn't really work that way. You might be in the same segment as someone, but you care about children's charities. I know people that completely dislike animal charities because they think we need to look after human beings,” he says.
“If we look at people as a system of connections and associations then therefore we start to build up a graph." Bugembe acknowledges that this approach bears similarities to Facebook’s Open Graph.
As a proof of concept of JustGiving’s graph, dubbed GiveGraph, was set in motion about two years ago and then put into production in 2013.
The organisation had around 14 years’ worth of data with 70 million nodes and 280 million relationships. Crunching this data required support from the cloud.
Bugembe says his team uses a range of languages for the graph, such as Linux and Python in some areas and Java in others.
“When looking through all these solutions that were available we found quite a few things that dissuaded us from. So for example we were looking for an enterprise ready solution, i.e. we could write testable code and we could get close to a guarantee of it being up most of the time,” he says.
“As an enterprise we have to be able take donations anytime someone has the urge to give to a cause and so those were the factors that we're looking at. Security became a really big concern too.”
Bugembe says he went with a Microsoft-based solution as it had everything his team was looking for, adding that Azure was selected after a lengthy process of evaluation.
Flexibility in particular was something major that Microsoft going for it. As the team can code using whatever language they need. “It allowed us to pick the right technology for the problem and it didn't have to be a Windows solution.”
“We are also a .Net shop so that contributed to that decision. Where a lot of our developers are .Net developers. I don't think there is anything wrong with AWS, we also use AWS for other things on our platform. It is just for us to get quick to market solution we went straight with Microsoft,” he says.
The set up
Bugembe says Azure’s HDInsight Hadoop distribution handled the heavy lifting needed to build the GiveGraph with Azure Cloud Services to move all the output from the analysis onto the table storage.
Service Bus, Microsoft Azure’s messaging system, listens out for events so that the graph can be kept updated. “That is quite key for us because we you care about changes based on the connections that have and the new connections that you make,” he says.
F Sharp calls some of the algorithms while via Azure Websites, APIs link everything back to the cloud and the intelligence built into the system.
Challenges
When developing the data system, Bugembe says when the engineers faced a problem with getting Giraffe, a graph processing engine working on HDInsight, one particular component wouldn’t work and needed the engineers to delve into the Azure code to get it to work.
“Graph processing is going to become essential for us and I don't know if we can do all of it in HD Insight. Some of those calculations are monstrous,” he says.
He adds that in reflection, his team should have reached out to Microsoft’s engineers on Azure. He says when his team his has contacted them, they have been very helpful and responsive.
“That came across because of the degree to which they were responding. I actually wished we had reached out to them sooner. One of the things we did was we might have spent a bit longer than necessary trying to solve the problems ourselves,” he says.
Benefits
Bugembe says working with Azure has enabled its developers to focus on improving the graph and helping to match donors to good causes. “We have got 14 engineers that can focus on actually building this app.,” he says. With the system in place, people are 91 per cent more likely to give to causes than before, JustGiving believes.
Next steps
Version one of the graph is running and Bugembe says the next task is to make the algorithm smarter with additional data sources. By integrating these into the infrastructure he hopes it can become purely self-learning.
“I'm hoping we can get there significantly before 18 months, maybe in the next five months. That’s my aim.”
