How to use SAP Lumira as the Swiss Army Knife of accelerated data exploration and governed discovery
Like the Swiss Army Knife which is known to pack a powerful toolset into pocket-sized companion, SAP Lumira packs a powerful punch when it comes to helping BI users accelerate data exploration and governed discovery. Data exploration, governed data discovery and intuitive visualization are the key enterprise data topics today. User-driven data blending, analytic models, predefined measures, bins, groupings and hierarchies are a must for any modern analytical tool. Semantic discovery, smart joins and data profiling are not exceptions. The data is simply far from being perfect. Yes, ideally we’d love to have all data sources, tables and columns be of high quality, with a glossary, meaningful names and relevant data types. In reality, data exploration is a heck of a dirty job.
Tool #1: Data Preparation and Improvement
In the following example, we are dealing with the customer data coming from an enterprise data source. After running the query, we do some basic profiling and exploration using the SAP Lumira tool. It reveals an issue with the customer type. Some of the customer records got incorrectly coded in the data warehouse as “Smal Retailer.”
In Lumira’s data preparation room, this is clearly visible, as highlighted below. The facets view also shows the revenue projected for that type of customer, a significant $24M in this case.
A simple typo like that, usually an error somewhere in the upstream process, can make our life more miserable when aggregating and visualizing data. Most of the tools are not smart enough to fix it on the fly. The source is not from our local file either, where we could control it. Usually we’d ask IT and/or the owner to fix the process or data, reload and query again. Depending on the organization, this can take days, weeks or even longer, effectively bringing our analytical progress to a grinding halt.
Lumira’s functionality accelerates just-in-time data improvements and also helps close the loop with data governance and management. The end user has a variety of data improvement tools at his or her disposal (circled). In this case, we can leverage the replace function to correct the customer type, changing it to “Small Retailer.”
And here it is. What actually gets fixed are the metadata and business rules that govern the set in Lumira. The back-end DW data remains as it was, of course. Because this is metadata-driven, the next time data is refreshed, the fix will still be applied even if the number of incorrectly coded records changes.
Tool #2: Goverened Data Discovery
With this improvement, we can now further explore the data or move to visualizing and discovering more insights. But at the same time, the tool allows us to engage with the data owners and IT on what needs to be done permanently.
The enriched data export functionality produces a data set with both the original bad data and the corrected version, with the records filtered.
Here’s how it looks with the extract opened in Excel:
We can now work with the IT and data owners on the governed data improvement process while continuing our data discovery, visualization and analysis.
In the next blog, we will cover intelligent data blending and smart join discovery.