What is data democratisation, and why do you need a No-Code data connector?

In times of digital transformation, there is a concept that is rising in importance across innovation-led businesses. This concept is Data Democratisation. Companies of all sizes are searching for ways to make data quickly and safely available. 

But what does “Data Democratisation” actually mean? Forbes made it easy to understand when they stated that “Data Democratisation means that everybody has access to data and there are no gatekeepers… for people to understand the data so that they can use it to expedite decision-making and uncover opportunities for an organisation.” In the real world, however, other aspects come into play. Forbes’ statement is more of a vision than an actual plan. 

Companies often find the need to meet the vision of Data Democratisation with different levels of openness, and on such a dynamic journey, there are 3 requirements;

1. Make data quickly available

It’s all about data. Most software solutions have one overarching goal: to provide data to the business. Companies get insight from raw data by applying modern analytics techniques, enabling them to identify opportunities and innovate. This allows businesses to respond rapidly to market changes to thrive. 

APIs are powering the next industrial revolution, and for a very simple reason, they enable Data Democratisation and allow hyperconnectivity to flourish. This increases the reliance on live data integrations as software solutions interact with each other more.

2. Standardise to simplify and reduce friction

There has been an explosion of API tools and platforms in recent years. They aim to speed up the process of building and deploying APIs. There has been significant effort worldwide to create patterns and protocols to standardise how systems communicate.

The OData protocol is a serious example of a data interaction and handling protocol. Microsoft created it over 15 years ago, and it has emerged as an ISO open standard. It has been helping businesses shorten their learning curve when manipulating data available over the web. With comprehensive documentation publicly available, and an exhaustingly tested set of patterns, the OData protocol allows IT teams to deal with new APIs in an enterprise portfolio without the need for much training.

3. Adopt a powerful governance mechanism

Monitoring what data concepts are available and who should, and effectively does, have access to them is a complex task. The more you decentralise the governance, the more control you lose, leading to potential cyber threats. Conversely, the more control you have, the more bottlenecks you create, slowing down the business’s access to data insights.

Nevertheless, data governance needs to be made easy and less technical so that it can leave the realm of IT and be owned by non-technical people in your organisation. Only then can speed, governance, and security converge with Democratisation.

The OData Connector for OutSystems applications

Aiming to solve the puzzle of democratising data in the OutSystems ecosystem, Hubway Connect was created. As a connector that exposes data out of OutSystems databases through Restful OData APIs in a no-code approach, Hubway Connect compresses the time-to-openness when it comes to making data available for business needs. Highly respected integration platforms in the market have been widely adopting the Odata protocol, such as MuleSoft, SoftwareAG, IBM Websphere, and SAP.

OData Connector
Traditional end-to-end dashboarding x compressed data openness journey with Hubway Connect

Hubway Connect use cases and the problems it solves

The main use case involves sourcing data to staging pipelines used in reporting and dashboarding processes for BI and analytics functions. Interoperability use cases can also leverage the power of Hubway Connect APIs, making it possible for platforms and solutions to integrate easily with OutSystems Apps.

Hubway Connect brings data governance to another level in the OutSystems ecosystem, giving technical and business teams the same view of how data is being exposed. Besides, its capabilities help IT teams provide business or development teams with a standardised and powerful way to source data using OData, an established data exchange protocol. Any well-known BI platform on the market, such as PowerBI, Tableau or even Excel, can easily tap into OData feeds.

Hubway Connect’s high-level architecture and common scenarios of use
Hubway Connect’s high-level architecture and common scenarios of use

Hubway Connect can easily be used with Enterprise data integration scenarios, playing the role of a data source and providing data readily to ETL tools. In these use cases, high volumes of data need to be handled by a robust data staging mechanism, with platforms like Snowflake or Azure data storage, while incremental load approaches are adopted.

Real use case of a data pipeline using Hubway Connect as a data source
Real use case of a data pipeline using Hubway Connect as a data source

An example of a real use case in a large real estate enterprise, Hubway Connect, was added to their portfolio of data sources providing data in real-time to their data pipeline. AWS Glue, an ETL tool, automated the data retrieval process implementing incremental load using the Odata query options and introspecting the Odata metadata to auto-detect new tables and fields added to the APIs.

The data lands in AWS S3 buckets to be processed in a corporate standard along with multiple other data sources. Snowflake then coordinates the data staging process onto its data lake engine, where all the calculations, data mashup, aggregation and merging happen. This data is then made available for consumption at the presentation layer.

The data visualisation layer is built with Tableau. BI analysts use it to build rich visual dashboards that support data-driven decision-making.

For interoperability scenarios, consumer apps can easily retrieve OutSystems data through data-driven APIs or even create and update records. As the Hubway Connect APIs are RestFull, one can interact with them, reading and writing data by using different request methods (Get, Post, Update, Delete). To guarantee high-security standards, this option is available through an optional configuration functionality used by administrators and properly secured through OutSystems out-of-the-box authentication mechanisms.

By offering OData connectivity capabilities, Hubway Connect is elevating the OutSystems platform to another level and better enabling development teams to quickly and easily add OutSystems Apps into Enterprise grade applications landscapes.

João Melo has worked in the software industry for over 20 years, including roles as a Software Developer, Product Owner, Solution Architect and Tech Lead. He has helped companies achieve results with a pragmatic approach to software development. An OutSystems MVP for over 7 years, Joao is the Head of Product Delivery at PhoenixDX, the builder of Hubway Connect.

About PhoenixDX & Hubway Connect

PhoenixDX is a specialised services provider in the software industry based in Australia. It has thrived by providing its customers with high-quality and mission-critical solutions with great user experience (UX) across several industries, including Real Estate, Finance, Field Services, Mining & Energy, and Retail.

Specialising in OutSystems, with over 120 certifications and 5 MVPs, PhoenixDX has a track history of successful projects delivered over 5 years. We deliver high-quality enterprise-grade solutions and are considered strategic advisory partners in our customers’ digital transformation journeys.

With exposure to many technical challenges, our team has expertise in engagements requiring interoperability with other enterprise-grade platforms, such as SAP, SalesForce and Microsoft. We have explored the capabilities of these market-leading platforms, and create solutions that extend the OutSystems platform’s capabilities to solve high-complexity problems by applying reusable solutions that offer great user experience and that fit nicely into our customer’s enterprise portfolio.

Hubway Connect is a spin-off of the PhoenixDX incubator, with the goal of producing solutions that solve recurring problems. Hubway Connect is a data connector product for OutSystems built with the purpose of making it easy to expose data from OutSystems apps through restful APIs using the OData protocol. By standardising the way BI analysts or integration specialists handle and interact with data, this product brings the OutSystems platform to another level of interoperability, further improving its value proposition. With Hubway Connect, APIs can be created in just a few seconds without having to write a line of code.

Talk to us about Data Democratisation.

A selection from our recent work