The open source data management software known as CKAN – the Comprehensive Knowledge Archive Network – and the community around are always evolving. As an Australian private company that has built a global reputation as a CKAN expert and one of the software’s ‘co-stewards’, Link Digital has always strived to move in tandem with these changes. And in response to recent CKAN-related developments we have made some exciting announcements about our future direction. This post will recap these and explore the important context within which they are happening.

CKAN origins and development

CKAN was developed by the Open Knowledge Foundation in 2006, with the guiding principle that open data is a public good that should be non-right restricted and easily transferable, and that creates more value the more it is shared. It was enabled largely by the then new package capabilities of Debian Linux; tools that assisted with the automation and installing and upgrading computer programs, and which made it easier for software developers to re-use and build upon each other’s work.

In the broader context, CKAN was also part of what is known as the Third Wave of Open Data. The Third Wave was based on the proposition that the open data ecosystem has moved beyond simplifying data and its economic and social impacts; to how governments, private corporations and civil society organisations can better manage and understand the data they have and share it to provide improved products and services, and a more informed citizenry.

Supporting CKAN’s user base

CKAN became the basis of the United Kingdom’s open data portal, data.gov.au in 2010, and it has since become central to the global expansion of open data initiatives. Research last year identified nearly 400 regularly updated CKAN data portals in 59 countries. This includes portals with national and transnational significance, such as the European Union’s open data portal and the United States open data portal, data.gov. But CKAN’s use by large actors can obscure a more nuanced picture. CKAN is what is known as out of the box software, i.e., it can work immediately when placed into service. CKAN can be set up and configured to function as an open or internal data platform once deployed and hosted on a web server. And because it is open source, the cost of accessing a CKAN instance is small compared to commercial software. This means that much of CKAN’s usage is by individuals or small groups with only limited developer experience and resources.

While the CKAN software has a core development team that contributes to keep it in good shape, it is also supported by this much larger global user base. How to nurture and grow this base is the aim of an initiative called Pathways to Enable Open-Source Ecosystems (POSE), supported by the United State National Science Foundation and based out of the University of Pittsburgh. POSE’s second stage, which was recently funded, seeks to strengthen the community around the software, including creating onboarding pathways for new people to contribute, improved documentation and governance, better tracking of CKAN adoption and help to get users up to speed with the latest version of the software.

The rise of ‘civic tech’ and the importance of digital public goods

CKAN now plays a vital role in what is known as civic tech. According to a 2016 paper by CKAN founder Rufus Pollock, civic tech is ‘digital technology (websites, apps and platforms) related to “civic” life. As defined, it includes most notably all government IT (whether in house or procured) but also includes technology created by others directly for citizens – from apps that let you register to vote to those that let you email your elected representative.’ Governments, for example, are increasingly using CKAN to meet two interconnected agendas: open data portals to make data open to promote transparency; and internal data management platforms to assist data custodians to more efficiently and securely store, catalogue and share their organisations data assets internally.

In recognition of the real world benefits of civic tech as a form of digital public infrastructure, CKAN has been awarded the status of being a Digital Public Good (DPG) by the Digital Public Goods Alliance. This was a recognition of CKAN’s value as a driver of positive change in helping to attain many of the Sustainable Development Goals put forward by United Nations (UN), and its use by governments and multilateral organisations like the UN Refugee Agency and the United Nations Office for the Coordination of Humanitarian Affairs, as well as philanthropic organisations and by grassroots civil society movements.

Improving CKAN’s core functions

All the developments mentioned so far, including the growth of CKAN and its community, and the software’s increasingly important role in civic tech, have informed two recent announcements by Link Digital

First, in May Link Digital announced a substantial investment to fund several new roles targeted at enhancing CKAN’s core capabilities. These new roles, in addition to our current support for the software, focus on:

  • Support for CKAN to improve interoperability with Data Catalog Vocabulary (DCAT) 3, the global standard for making data catalogs interoperable.
  • Providing a better user experience.
  • Enhanced file management.

These capabilities also align with improvements as part of the release of the next version of CKAN, known as v3.0, and are very much aimed at helping smaller and less resourced users to get more out of CKAN’s out of the box experience. They include:

  • Better user interface.
  • Modernising CKAN’s tech stack with HTMX technology.
  • The addition of Elastic Search, an additional technology that can be used to improve search ability.

Funding digital public goods into the future

Second, are ambitious changes to Link Digital’s operations going forward. Announced by founder and Chairman, Steven De Costa, at our Monthly Forum in June, these include the transition to a different operating model and the establishment of a beneficiary fund that would sit above Link Digital’s commercial entity. This fund aims to supplement government funding for civic tech initiatives involving digital public goods, collaborative governance, and public code, globally.

As to why now is the right time to announce these changes, De Costa stated: “Trust is always a barrier to partnerships and a company that operates singularly within a competitive market is, for better or worse, trusted only as far as a for-profit company should be. For-profit companies are very important within a commercial marketplace, especially one based on knowledge operations, yet they will remain constrained by their defensible value chain and unique offerings.”

“Link Digital can stay competitive and profitable as a bespoke service provider servicing a small number of high value clients. We can also deliver entirely free and open value inputs for other organisations and individuals without any disadvantage to our commercial operations.  But we need to compartmentalise, and we need the trust of the ecosystem we hope to collaborate with and support.”

De Costa is under no illusions as to the future challenges that lie in store for this new direction. “Data is no small thing and no new thing, it is a large part of how we observe, model, plan and manage the things we collectively care about. Many voices will have an opinion on how to bridle this machine readable and machine trainable resource, for the betterment or worsening of every aspect of our lives.”

“Link Digital is moving to a model where we will become an entirely positive influence on the entire open source community, and the very special open data and civic tech community that already has surrounded and drawn great benefits from the CKAN project. No doubt, others will pick a path too. I’m sure, and we will look forward to collaborating with those who choose to architect a future that builds for trust through openness and honesty.”

We invite you to be a part of this journey—whether you’re a developer, policy maker, or passionate advocate for open data. Let’s collaborate to create a world where data is a tool for transparency, empowerment, and positive change.

Reach out today and discover how you can contribute to building a better digital future.