Skip to main content

Federal Government Open Data Portal


CKAN Development
www.data.gov.au

The client's needs

Department of Finance was sharing open data with the public via a Wordpress system. To provide a better experience to data publishers and public users, they recognised the need to migrate to a specialised data publishing application that would provide a better experience for both data publishers and public users. Link Digital was chosen to make this possible via CKAN.

The challenges

Cloud based hosting of CKAN was not widely understood at the time. Link developed a reference architecture with Amazon Web Services.

Additionally the migration of data from legacy systems is always challenging. Link developed a data migration tool specifically to achieve migration of data from the custom legacy system which did not have any export functionality.

The solution

Link’s proposal covered Cloud based CKAN hosting and Professional services.

The open data platform CKAN is a tool Finance is able to use for:

Publishing data files – primarily tabular and spatial.
Searching for data – using the metadata and descriptive information stored on each dataset.
Organising data files – by organisation, category or specially created groups.
Accessing data programmatically – CKAN generates API access to uploaded CSV and some spatial files.
Linking to externally hosted data services (APIs) – we will link to external data services, such as ABS or Geoscience Australia data APIs.
Basic data visualisation – CKAN supports a basic spreadsheet view, graphing and mapping of datasets. We will be supporting additional data visualisation tools down the track and can assist agencies in finding appropriate tools in the meantime.

How the solution evolved

Data.gov.au has gone on to become one of the largest open data catalogues in Australia. The CKAN platform has allowed new functionality to be developed such as data harvesting and spatial data hosting.

Finance has developed a governance guide to help agency data publishers in specifically publishing to and using the data.gov.au platform. A particularly encouraging element of the Finance decision to use CKAN is that it is open source. In recent years government agencies were largely reluctant to adopt this kind of solution, however, the governance guide is supportive of the strong open data community in Australia and recommends agency data publishers join the Open Knowledge Foundation Network (OKFN) Australia community mailing lists to tap into the broader national community and where appropriate, international communities of interest.

The results

Data.gov.au has grown from 500 datasets to over 5000. The ability of the CKAN system to allow automated publishing has allowed several datasets to be updated on a daily basis.

Promising longevity, the public can request datasets they would like to see added to the site. Adding desired datasets is easier with this more targeted platform.

Finance are confident CKAN will better support future upgrades and tools that they will be addressing down the track, after first focussing on quality data publishing.

UPDATE

The data.gov.au platform has been running for over a year and in that time Link Digital has worked with the Department to install a number of plugins and develop a number of enhancements.

Plugins and customisations of particular note are:

KML JavaScript preview (spatial), AGLS metadata, Google Analytics dashboard, API metrics, statistics, and GeoServer integration.

With interest growing in the particular setup that has been established for data.gov.au, all of these additions have now been made available from a single GitHub account.

www.github.com/datagovau

We have also included the spatial ingestor script.

 

Follow us on