Many public agencies, research bodies, and environmental organisations manage far more than just maps. The data they hold frequently combines geospatial data – information tied to specific real-world locations, such as boundaries, satellite imagery, and survey points – with massive collections of non-spatial data, such as spreadsheets, research papers, document archives, and tabular databases that have no geographic coordinates.
A lot of organisations already use Esri ArcGIS Hub, a cloud-based platform that enables organisations to share spatial data, publish maps, and build interactive map applications with location-based content. However, when an organisation needs a single front door for all its assets, pairing ArcGIS Hub with the Comprehensive Knowledge Archive Network (CKAN) creates a much more complete and unified catalogue across every data format.
The following article explains the functionality that CKAN can add alongside ArcGIS Hub, how the platforms can connect, and how Link Digital has engineered CKAN solutions that integrate with Esri tools for geoscience agencies in Australia and New Zealand.
Why connect CKAN to ArcGIS Hub?
While ArcGIS Hub excels at spatial storytelling, CKAN provides a broader catalogue foundation capable of handling general corporate files, restricted workflows, and data governance.
1. Unified cataloguing for mixed data assets
ArcGIS Hub is tailored for location files, but CKAN can catalogue virtually any digital resource, including tabular spreadsheets, APIs, unstructured PDF archives, satellite feeds, and specialist industry files. Rather than forcing non-spatial files into map-centric tools or fragmenting records across different websites, a combined setup lets users search both spatial datasets and regular documents in one place.
2. Managing internal records, workflows, and embargoes
ArcGIS Hub’s public feed only distributes items already approved for external viewing; it is not designed to serve as a space for organisations to internally draft or review data or as a secure internal data register. CKAN solves this by supporting split, internal-to-public publishing pipelines. Agencies can publish data and also maintain a private, secure CKAN registry where staff catalogue, check, and curate internal records. This setup is essential for managing data that may be restricted for legal or other reasons, or to keep sensitive research or commercial data secure and confidential until an agreed upon release date. Once approved, records can be automatically pushed from the internal CKAN system out to the public-facing catalogue.
Simultaneously, the public CKAN system automatically harvests metadata from ArcGIS Hub’s public feed, creating a unified discovery search across all assets.
3. Shared metadata and interoperable standards
For catalogues to connect smoothly, they rely on high quality metadata, such as titles, author names, publication dates, summaries, and license details. CKAN uses globally recognised metadata standards so systems can exchange records without requiring every department to use identical software:
- DCAT (Data Catalog Vocabulary): A standardised format created by the W3C that allows different data portals to read and understand each other’s catalogues. Extensions like ckanext-dcat enable CKAN to consume and publish records in multiple DCAT versions.
- ISO 19139 / ISO 19115: Global technical schemas designed specifically for describing geographic metadata. CKAN’s ckanext-spatial extension uses these standards to process geographic search boundaries and catalogue feeds.
- Schema.org: Structured web markup that helps mainstream search engines like Google discover datasets and index them directly into tools like Google Dataset Search.
4. An open-source, adaptable foundation
CKAN is open-source software. Because its source code is publicly accessible and not locked to a single vendor, agencies can introduce an open catalogue layer while protecting their existing investments in Esri tools. CKAN connects easily into wider IT infrastructure, including cloud storage (such as AWS), search engines (Solr), project tracking (Jira), and data transformation engines (FME).
5. Optional AI-assisted data discovery
Once both spatial and non-spatial records are organised in a clean, metadata-rich catalogue, organisations can add conversational search tools, such as Link Digital’s Ask AI capability. This is a conversational layer that provides a grounded and secure way for users to ask questions and get verified answers from an organisation’s own data, which means the answers are not only accurate but also accountable and traceable.
How the connection works: federation and harvesting
Connecting an ArcGIS Hub site to CKAN relies on two core mechanisms: data federation and harvesting.
Data federation means bringing together records from multiple separate repositories so users can search them from a single interface, even though the original files still reside in their source systems.
Data harvesting is the automated process CKAN uses to achieve federation: CKAN regularly visits an external system, reads its public catalogue feed, and synchronises the latest metadata.
In practice, the integration follows four steps:
- Enable the feed: The public ArcGIS Hub site generates an open catalogue feed URL formatted using standards like DCAT.
- Configure CKAN extensions: CKAN is configured with connector tools, such as ckanext-harvest and ckanext-dcat, targeting the ArcGIS Hub feed.
- Automate the harvest: CKAN runs regular, scheduled harvests to pull in dataset titles, summaries, tags, and download links from ArcGIS Hub.
- Enable map previews: Optional extensions such as ckanext-geoview display map boundary boxes and let visitors preview geospatial resources directly inside CKAN without leaving the catalogue. Because CKAN is highly extendable, it can also continuously adapt to new spatial formats, for instance, a developed extension allows users to natively preview complex GeoPackage files as well.
Important deployment note: Esri’s standard catalogue feed documentation applies to cloud-hosted ArcGIS Hub sites. Organisations running on-premises ArcGIS Enterprise must check the specific federation, API, and harvesting options supported by their exact version and architecture.
Real-world examples of CKAN solutions that integrate Esri tools for geoscience agencies
- Geological Survey of Queensland (Australia): Link Digital integrated Esri mapping tools with a CKAN-powered public portal, introducing automated data harvesters, customised metadata schemas, custom access rules, and advanced geographic search. You can read a case study of the project our site.
- Ministry of Business, Innovation and Employment (New Zealand): MBIE replaced an ageing proprietary platform with an interconnected two-tier CKAN architecture. The system uses an internal asset register to hold petabytes of geoscience files, PDFs, and commercially embargoed records, alongside a public facing geodata catalogue holding over 60,000 public datasets—all integrated directly with ArcGIS, Azure, Solr, Jira, and FME. A detailed case study of the project is here.
When does this combined architecture make sense?
Connecting CKAN and ArcGIS Hub is ideal when an organisation:
- Already relies on ArcGIS Hub for public map engagement, but needs a centralised home for non-spatial files, reports, and spreadsheets.
- Needs a secure staging area to review, govern, or embargo records before making them publicly searchable.
- Wants a single discovery portal across disparate departments, file formats, and storage environments.
- Prioritises open-source governance and open metadata standards while preserving existing GIS infrastructure.
Interested in talking about harnessing the functionality that CKAN can deliver alongside ArcGIS Hub? Get in touch and one of our data engineers will be in contact.
