This blog post was co-authored by Adrià Mercader, Senior Solutions Architect at Link Digital

Go to a specific topic:

The data management system known as CKAN (the Comprehensive Knowledge Archive Network) was originally developed as an open source open data catalogue for publishing data online. While governments at all levels where its early adopters and continue to be a major component of its user base, CKAN’s uptake has gradually extended to domains, including the not-for-profit and private sectors.

During a panel presentation at the recent annual conference of CLARIN (Common Language Resources and Technology Infrastructure), a network of mostly European research centres specialising in language research to support humanities and social science, Link Digital’s Spain-based Senior Solutions Architect, Adrià Mercader, discussed the challenges and lessons of building bridges between CKAN and the private sector.

The following post recaps Adrià’s presentation and explores some of the issues arising from it.

CKAN’s growing user base

CKAN was originally designed to be very customisable and adaptable, so as to be easy to integrate it with other systems. It is open source, not just in the sense that the code is public and available online, but in that it is maintained by a large and growing community of stakeholders. These organizations and individuals include a mix of representatives and contributors from the not-for-profit, academic, and government sectors, as well as commercial vendors and industry members.

The growing role of CKAN in the digital ecosystem is evidenced by the fact that it serves as the key component of some of the largest and most successful open data portals around the world. Research in 2023 identified nearly 400 regularly updated CKAN data portals of different sizes in 59 countries. These are spread across six major continents, with Brazil and the United States having the largest prevalence, followed by Argentina, Australia, Canada, Germany, the UK, Spain, Italy, and Japan. They include globally significant portals such as the EU’s open data portal, the United States open data portal, and Open Africa, the largest repository of data on the African continent, but also many smaller scale users with limited resources or technological expertise at their disposal.

And while governments remain a major part of CKAN’s reach, the software’s user base has grown and diversified. CKAN is currently used by multilateral organisations like the UN Refugee Agency and the United Nations Office for the Coordination of Humanitarian Affairs, as well as not-for-profit organizations and grassroots civil society movements. And it has been steadily adopted in academic and research circles to organise and publish data in a wide range of subject areas.

While it is hard to quantify exactly how much of CKAN’s total user base comes from the private sector, all the indications are that the software is being adopted more and more by the private sector, in areas ranging from pharmaceuticals and real estate to energy management. While it is harder to cite public URLs for this activity because these projects tend to be internal catalogues for private data, we know that many of the developers involved in the CKAN community are affiliated with the private sector, including commercial vendors such as Link Digital. Link Digital has worked on CKAN projects for Suncorp, Wesfarmers and Lego Global, and has helped many other private sector organisations validate and select CKAN based applications for their work.

CKAN’s governance structure

CKAN was developed by the Open Knowledge Foundation (OKF) in 2006, with the guiding principle that open data is a public good that should be non-right restricted and easily transferable, and that creates more value the more it is shared. And, initially, all the project development and management were done at OKF.

There were some early high-profile sites that adopted CKAN, such as the United Kingdom open data portal, but a significant surge in sites adopting CKAN did not occur until the second decade of the 2000s. It was at this point that OKF decided to set up the CKAN project in a way that would allow other stakeholders to contribute and, more importantly, to have a say in its direction. This decision was taken out of a desire to open the project’s governance to other parties and the more practical need to ensure its sustainability by welcoming contributions from outside OKF. CKAN’s governance structure has evolved over the years, and currently comprises three levels:

Benefits and challenges of private sector involvement in CKAN

With this background in mind, it is instructive to have a closer look at the role that the private sector plays in the CKAN ecosystem today. 

First and foremost, if we are talking about the actual number and volume of contributions to CKAN, a big part of them come from private sector-funded individuals. That’s anything from occasional patches and features sent by developers working on commercial projects to people funded to work full-time on the maintenance of CKAN. A major proportion of the CKAN tech team are in some way linked to private sector vendors, and both CKAN co-stewards are from the private sector.

And while much of the private sector’s growing adoption of CKAN doesn’t directly translate into tangible project contributions, it brings wider benefits. It increases the private sector’s exposure to CKAN and open source technology more generally and injects novel points of view, scenarios, technologies, and development practices into the project that help it grow and explore new areas.

Seen from this perspective, the private sector plays a major role in the CKAN ecosystem. While this is in most cases positive, it does come with challenges and risks that need to be monitored so that they don’t become issues. Private sector participants sometimes have needs and goals that may not be 100% aligned with those of the CKAN project, so it’s important to make sure that CKAN’s original goals in terms of intended audience and use cases, and the required level of skill to use the software, are maintained. Directly related to this, it is important that the novel points of view or practices mentioned previously don’t leave a section of the CKAN community behind, i.e., by increasing the onboarding period due to more complexity or increasing the out of the box costs of using software.

Then there’s the more philosophical question relating to asymmetry in contributions. An issue faced by all open source projects, it is an especially important consideration for CKAN, a project which is based on the premise of democratising access to knowledge or data and, more broadly, to provide a benefit to society. Seen in this context, it is vital to always ask the question – what value is the private sector getting from the project?

Conclusion: the importance of diversity

There are three key take-aways in terms of building bridges between the private sector and open source software projects such as CKAN, and ensuring their sustainability:

  • If done properly, involving the private sector can bring a net benefit to the project.
  • It’s important to set up the project in ways that facilitate it to grow, including allowing external players to be involved, without creating friction.
  • We always need to ensure that the original goals and philosophy of the CKAN project are maintained

The best way to ensure the success of linkages between CKAN and the private sector is to ensure a diverse community around CKAN, to ensure all sectors are represented. The more community, not-for-profit, research, and government interests are represented, the less likely it is that decisions that don’t benefit them are going to be taken.

Want to build a CKAN-based portal?

Contact us and one of our data experts will be in touch.