Secure Data Exchange: a critical conversation for data portals

This blog post was co-authored by Lex Du Plessis, Link Digital’s Technical Delivery Manager

Secure Data Exchange (SDE) is a new capability by Link Digital to help data custodians in large organisations, primarily government agencies, exchange data in a completely secure manner across their domain.

The SDE approach, outlined by Link Digital’s Technical Delivery Manager, Lex Du Plessis, in a session at the recent 2024 Canadian Open Data Sumit, comes from our understanding of the challenges involving collaboration, regulatory compliance, accountability and the safeguarding sensitive data, in the evolving open data ecosystem. It is also part of a broader trend in which the public governance achieved through open data and transparency fosters greater trust among individuals and institutions.

Why Secure Data Exchange?

But before we get to what SDE is, it is useful to briefly touch on why it is needed.

During the COVID 19 pandemic, organisations and government’s recognised that their existing workflows were ineffective to face the challenges of the new environment. In the government context, the pandemic response triggered a frantic effort to create ad hoc data-sharing arrangements across various departments to address emerging gaps.

This spanned not only healthcare and emergency response, but every area of the public sector. The urgency for data exceeded the existing processes designed for its responsible sharing, emphasising the necessity for a platform that can facilitate the secure exchange of data, complete with implementable and storable sharing agreements.

Another driver concerns the timing of data sharing. Many of Link Digital clients have stringent workflows that govern when and how data can be published. This caution is entirely justified, as improper use of data could lead to harmful consequences. In such cases, diligence is essential—not because the data shouldn’t be published, but because it is the responsible course of action.

The SDE is also a response to a broader trend in which large organisations and governments at all levels are collecting and using data more than ever to increase transparency and inform policy development and service provision. But as the amount of data increases, so do the challenges of effectively managing and sharing data across its life cycle. In addition to sharing data securely, these challenges include:

Deriving real insights and benefits from data.
Enabling staff to find the right data for their research or project, quickly and efficiently, ascertain its quality, and identify its custodian.

The upshot of all of this is that large organisations and governments are now not only seeking high trust and near real-time support for their governance practices. They are looking to enhance their data governance and management; they are aiming for holistic improvements in governance practices.

What is Secure Data Exchange?

The secure data exchange is one outcome of new patterns of data sharing modelled on the open sharing of public domain data, ensuring interoperability, discoverability, and governance for reuse by institutional partners and key third parties.

SDE is a powerful, flexible, and easy to use approach for data custodians to manage the complete lifecycle of their data across multiple departments, organisational branches, and internal and external users, in a completely controlled access environment.

It offers three things:

Structured data sharing

SDE enables structured and secure data sharing that allows government departments and other large organizations to retain control of their important data while reaping the benefits of collaboration from its sharing.

Access management

SDE provides a streamlined and easy-to-use and interpret dashboard where they can view all data requests, their status (pending, final approval, ended), and all data sharing agreements. This includes the ability to revoke approval to use data.

Customised metadata scheme

A customized metadata scheme is incorporated into the workflow of the SDE system. This complies with the criteria set out in the latest Data Catalog Vocabulary standards, designed to facilitate interoperability between data catalogs.

SDE won’t solve all the problems facing data custodians in government and other large organisations. But utilising open source software such as CKAN, the Comprehensive Knowledge Archive Network, and Drupal, a content management system used to build and manage websites, it can assist large organisations to develop the innovative solutions they require to simplify complex data sharing and delivery across their operations. And because SDE is open source, there are no licensing fees, making it not only an accessible but a cost-effective solution.

SDE can make it easier and more efficient to reuse, evaluate and securely share internal data in one core data platform. It provides an intuitive and simple workflow to manage data requests, negotiate and generate data sharing agreements, grant and revoke data access, and discern patterns of data usage over time and across different levels of operations and functionality.

Conclusion

SDE allows data custodians to see all their data in one place. This ensures that there is one source of truth for the custodian’s data sets and prevents aged or incorrect data from being shared. SDE can assist large organisations to:

Build trust and accountability.
Improve the visibility of who is using data.
Improves data management.

Link Digital can integrate SDE into a data lake, data stores, etc, so that the data sharing agreement is always pointing to the most up-to-date data in core systems. These operations all sit behind corporate Identity Access and Management systems, allowing confidence when users are updated on the core security platform.