This blog post was co-authored by CKAN expert Sergey Motornyuk.
Upgrading to CKAN 2.11 is not just about staying up-to-date—it’s an opportunity to discover new capabilities that can greatly enhance your data management processes.
For organisations already using CKAN, this migration offers improvements in file management, resource handling, and overall performance.
If you haven’t yet integrated ckanext-files, CKAN 2.11 provides a starting point to adopt it for better file storage, discoverability, and workflow flexibility.
Here are the reasons why CKAN 2.11, alongside ckanext-files, is a critical upgrade and how different projects have successfully implemented this transition.
Why upgrade to CKAN 2.11?
CKAN 2.11 offers a range of enhancements that make managing data more efficient and effective, especially for organisations with complex data workflows. The upgrade allows for better file handling, more secure data exchanges, and smoother integration with existing systems. Migrating ensures compatibility with the latest extensions and improves the performance of large datasets. In particular, ckanext-files is a critical extension that enhances file management, enabling users to upload, organise, and manage files with greater flexibility and control.
It’s important to note that СKAN 2.11 introduces compatibility with Python versions 3.9 to 3.12. With Python v3.11users could see a potential 10%–60% increase in code execution speed (as detailed in the release notes). Although this doesn’t guarantee that your application will run faster, it’s a possibility worth considering.
The key benefits of adopting ckanext-files
With the integration of ckanext-files, CKAN 2.11 allows for file management. This extension enables you to:
- Maintain comprehensive metadata and access control, ensuring files remain secure and traceable.
- Improve the discoverability of raw and processed data through structured file indexing.
- Improve workflows for managing files across various stages of data publishing.
- Handle multiple file types and large datasets with ease.
Migrating from your legacy data management system?
Use cases
Managing complex publishing workflows
In some projects, datasets go through multiple stages of publishing, with temporal files produced on different publishing stages. With the potential upgrade to CKAN 2.11, the newer version of the file extension would maintain this core functionality while introducing opportunities for smoother workflows and better integration.
Enhanced data quality requests
Another implementation involves handling multiple types of requests, including those aimed at improving data quality. Users submit categorized research data that is reviewed and used to enhance or create new datasets. The flexibility of the file extension supports these diverse workflows, and with CKAN 2.11, there’s the potential to introduce further improvements to file management and data quality workflows. This includes providing access to the source data used for assembling the resource or exposing multiple resource’s attachments in different formats, provided in advance.
File thumbnails and previews
In some use cases, thumbnails are generated for uploaded files, providing a visual preview similar to book covers. By utilizing the file extension, these thumbnails can either be automatically generated or provided manually. With CKAN 2.11, API improvements would allow for easier integration, enabling the creation of thumbnails at the time of resource creation with minimal configuration changes.
Streamlining data sharing across portals
Certain projects involve managing large datasets across multiple sub-portals. In the past, duplicating data across portals was slow and costly. The files extension now allows for the use of shared cloud storage, where the main portal generates download links based on the appropriate storage configurations. Migrating to CKAN 2.11 could further streamline this process, offering better integration with cloud storage solutions and reducing data duplication costs.
Ingesting CSV/XLSX/ZIP Files
For portals that handle file ingestion in formats like CSV, XLSX, and ZIP, the files extension retains source files, allowing for re-ingestion and archival of older data. This ensures traceability and long-term accessibility of datasets. With CKAN 2.11, the possibility to optimize storage solutions further, such as moving older files to lower-cost storage while maintaining accessibility, presents a significant efficiency gain for these portals.
Future-proofing with CKAN 2.11 and ckanext-files
Upgrading to CKAN 2.11 with the file extension prepares your organisation for future workflow changes. Upcoming features will include various UI widgets for different scenarios of file uploads and high-level features. This includes: the ability to pause the upload; to upload the file while the user is still filling in the rest of the form fields, making the UI more responsive; and previewing the file upon selection.
In summary
The CKAN 2.11 migration is a step toward improved, scalable, and efficient data management. Whether your organisation is already using CKAN or considering implementing it, integrating ckanext-files will enable you to manage documents more effectively, enhance user workflows, and ensure your data infrastructure is ready for the future.
Need help to migrate to CKAN 2.11?
Your upgrade shouldn’t be a hassle. But if you need help, our CKAN experts have guided numerous organisations through the migration process.
