The Comprehensive Knowledge Archive Network – CKAN – is the world’s leading open source data portal software. While it is free and has native out of the box functionality, setting up and running larger and more complex CKAN instances can be difficult.
This is where managed CKAN hosting comes into the picture. Managed CKAN hosting is a service where a third-party vendor handles all the technical heavy lifting – server setup, maintenance, security, updates, and scaling – allowing your organisation to focus purely on managing and sharing data.
A practical illustration of the benefits of managed hosting can be found in the Columbia Basin Water Hub, a centralised online repository for water data collected from across the Columbia Basin, North America’s fourth largest freshwater basin. Run by Living Lakes Canada, the Hub is a charitable, non-governmental organisation focused on water stewardship, with over 20 years of experience working with community groups to protect freshwater.
Developed with the assistance of Link Digital using CKAN and launched in 2021, the Hub is designed to store past, current and future water data, ensure adequate quality control and make the data more accessible and shareable. Link Digital recently transitioned the Hub to a fully managed CKAN hosting solution.
Maggie Finkle-Aucoin, GIS and Database Manager at Living Lakes and manager of the Hub, talked to us about what this involved in the move and what the benefits have been.
Why the switch to managed hosting?
Several internal resource issues and pain points led the Hub team to switch to CKAN-managed hosting.
When the Hub was first established, Living Lakes Canada partnered with two other organisations that were also working towards water data monitoring goals, and the Hub was hosted on one of their servers.
“So, we did have some additional support there, but it was largely still our responsibility to oversee that hosting environment and deal with the occasional bugs or issues that were propping up,” says Finkle-Aucoin. “And within the Colombia Basin Water Hub team and within Living Lights Canada, we don’t have a ton of capacity when it does come to IT and digital services.”
While Living Lakes had been mulling over the decision to switch to a managed hosting service for a couple of years, the turning point was an incident where the Hub became overrun with bot activity. “Speaking with the Link Digital staff, we discovered that it was some sort of model training itself on the water Hub, and it slowed the service down to the point where the Hub was no longer functioning.”
“And despite our best efforts, doing what we could with our skill set to manage that kind of traffic of bots, we just couldn’t get around it. At that point, it was really clear we needed to make that change from a security standpoint as well.” Living Lakes staff worked closely with Link Digital’s Canadian subsidiary, Datashades Ltd, and we were able to put extra security in place, which eliminated the bot problem.
“To be honest, I think something that I had internalised a little bit was feeling this was going to be a really big job,” admits Finkle-Aucoin. “But your team made it so straightforward. For us it was just a matter of kicking off that process with Link Digital and moving that service over, giving Link Digital the proper permissions and then having it over there.”
“On our end there wasn’t much change that had to be done, and primarily now it’s just a matter of being more confident and comfortable in the kind of hosting environment that we have and rewriting some of our protocols which result in a bit less work on our end.”
Also to read: Why CKAN Managed Hosting Matters (and how to get it right)
Benefits of managed hosting
According to Finkle-Aucoin, managed CKAN hosting has had several other benefits.
The Hub’s future plans include talking to groups in watershed areas outside the Columbia Basin about setting up their own data portals and helping to facilitate those kinds of builds. Until now, storage has been a prohibiting factor in making this happen. “There was a limit, a ceiling that we had with our last [hosting] provider. And now being with Link Digital and being on this new managed service, the storage component is no longer a concern.”
“It also gives us a lot more confidence and trust in the infrastructure that is running the Hub. Obtaining more data is also one of the things that typically slow the repository down… But within the new platform, things are not slowing down in the way that we would have seen on our previous service.”
Also to read: How CKAN supports community-based water monitoring in Canada’s Columbia Basin Watershed
Living Lakes currently works with more than fifty groups actively collecting data on water quality or quantity, these include First Nations communities. This takes place within the framework of the OCAP Principles – ownership, control, access, and possession – which are designed to protect Indigenous ownership, control, and stewardship of their data. Managed hosting has helped with this work by providing more sophisticated access controls and security protocols necessary to respect indigenous data ownership.
“Being with Link Digital and being on this managed server gives us a little bit more control over the data storage and allows us to enact some of the protocols when it comes to deletion protocols and secure data storage. I definitely see how this move has enabled us to improve our ability to offer indigenous communities, First Nation governments, and then also just our other communities who are concerned about their data security, a better space to host their data.”
Finkle-Aucoin lists several other indicators that Living Lakes staff will be looking at to measure the success of the move to managed hosting.
“One of the first ones that come to mind is the reduction in bot traffic on the Hub. We’re not seeing an interruption to service. We’re not having users experience an issue where the hub is slow or lagging. In fact, it’s the snappiest it’s ever been.”
“We’re constantly in chat with our data contributors and our data users. So, we’ll be looking to them to kind of hear their feedback about any issues that they’ve experienced with the transition. So far, it’s been nothing but positive.”
“And then in terms of cost savings, we will continue to kind of monitor that, but we’re already seeing a reduction in time that we need on our end internally. So, this again looks like another success point for us.”
Want to talk about transitioning your CKAN portal to a fully managed service?
Link Digital’s CKAN experts are ready to help you.
You can watch the full interview with Maggie Finkle-Aucoin about the Columbia Basin Water Hub’s transition to a fully managed CKAN hosting service on our YouTube channel here.
