Summary

In the final episode of Link Digital’s Tuesday Talks, Sergey Motornyuk, a core developer of CKAN (Comprehensive Knowledge Archive Network), discusses the latest developments and future plans aimed at improving the software’s user experience, developer convenience, and system performance.

He talks about the involvement of two of Link Digital’s Senior Solutions Architects, Ian Ward and Adrià  Mercader. The key developments include: Ian’s work on the new Table Designer feature to enable users to push partial updates for some of the large datasets rather than reloading the entire file, which reduces page latency. Adrià’s enhancements to  search functionality through semantic search and a common search interface for multiple search engines and Sergey’s own work on a file management system. This new system treats files as first-class entities with version control, flexible storage options, resumable uploads, and better access control, promising significant benefits for developers, maintainers, and end-users.

He also mentioned the efforts to enable dynamic metadata validation for easier extension integration and a modern, highly customizable UI framework that moves beyond the current Bootstrap dependency. 

Overall, the focus is on increasing CKAN’s flexibility, extensibility, and usability for a broad spectrum of users and developers, ensuring backward compatibility while pushing innovation.

CKAN functionalities update from Link Digital, the CKAN experts.
CKAN Functionalities Update as shown on Link Digital’s Tuesday Talks episode 7

Timestamp and Key Insights

Performance Optimization with Partial Page Updates

03:39 – By integrating HTMX for dynamic content refreshing, CKAN drastically reduces page reload times during search filtering, providing a smoother and faster user experience critical for large-scale portals. This approach minimizes unnecessary data transfer and rendering, a best practice for web applications dealing with large datasets.

Partial page updates on CKAN - Link Digital

Semantic Search Using Machine Learning

06:55 – Incorporating word embeddings transforms traditional keyword-based search into a context-aware system, allowing users to find relevant data even when search queries use synonyms or related terms. This enhances discoverability and aligns with modern AI-driven search trends.

New file management flow and resumable uploads

14:21 – Treating files as first-class entities with metadata and version history enables better tracking, audit trails, and sharing flexibility. Support for resumable uploads and multi-cloud storage addresses real-world challenges like unstable connections and heterogeneous infrastructure, key for scalability and enterprise adoption.

CKAN file management then and now - Link Digital
CKAN’s File Management (Then and Now)

File metadata, versioning, and access control

16:19 – Automating schema updates when enabling/disabling extensions reduces human error and maintenance overhead. This modular approach fosters a more sustainable ecosystem where plugins can evolve independently without breaking compatibility.

Benefits of new file system for users and developers

21:18 – Sergey’s work is focused on developer-centric improvements. This is to streamline interfaces for integrating new storage backends and file handling logic, cut down customization complexity and technical debt, and encourage contributions and wider ecosystem growth.

For portal maintainers, they will benefit from flexibility in storage and file management, while for end users there’s convenience in the following features:

  • Drag & drop uploads.
  • Resumable uploads (pause/resume on network failure).
  • Parallel uploads from multiple sources for faster large file uploads.
  • Incremental uploads (upload only changed parts of a file).
  • Automatic metadata extraction from files to create datasets/resources.

Dynamic data validation and metadata schemas

26:41 – Discussing metadata schemas, Sergey highlights the challenges developers face when enabling extensions that require schema changes. He is currently developing dynamic schema validation, which enables extensions to dynamically register fields. This plug-and-play approach simplifies extension management, reduces manual schema updates, and could enable runtime extension toggling from the admin interface.

UI restyling and flexible design frameworks

29:43 – The current CKAN UI is being restyled to look modern and support easier customization. Sergey envisions a two-layer abstraction allowing portals to switch design frameworks (e.g., Bootstrap, Tailwind) while preserving custom styles and structures. This flexibility will enable organizations to tailor CKAN’s appearance and layout extensively, breaking free from Bootstrap constraints and improving adoption.

The Community and Openness as Core Values

Sergey mentions CKAN’s commitment to open source principles, welcoming contributions and ideas from users, which fuels continuous improvement and ensures the software remains relevant across various industries.

Throughout the discussion, Sergey emphasizes CKAN’s core philosophy of openness – both in terms of the code and open community contributions.

This episode of Tuesday Talks shows Link Digital’s continuous investment and direct involvement in the community, making CKAN an ideal choice for organizations seeking a flexible, extensible, and powerful data management platform.

Watch the full recording: Developer Talks about CKAN Functionalities and Its Future

Want to build a CKAN-powered open data portal or internal data catalogue? Contact us and one of our experts will be in touch.

Read more articles