The answer to this question is straightforward within the context of quantitative disciplines like mathematics in which “linear” and “non-linear” are well defined and differentiated. The answer is less obvious in reference to data management and analysis. The industry acknowledges that a traditional, strictly linear IT-centric approach is ineffective in view of today’s evolving data landscape. Many organizations advocate bypassing IT altogether with non-linear solutions emphasizing self-service data discovery, preparation and analysis in order to accelerate the transition from raw data to insights.
The customary view of a linear approach is one in which data travels a step-by-step path from acquisition to insights with a focus on streamlining the process for efficiency and predictability. This approach is considered appropriate for data that is predominantly structured, well-understood, and originates from familiar data sources. The disadvantages of following the linear approach exclusively are lack of agility and the potential for not realizing the full value of the data.
In contrast, the non-linear approach is viewed as more flexible and iterative in support of discovering and facilitating understanding of the data, and making data accessible quickly for exploration and ad hoc analysis. It often reveals the criticality and quality of data assets and extent of governance required downstream, and might result in addition of new data assets while discarding others determined to be irrelevant to the use case. This approach is considered appropriate for data that is multi-structured and originates from a number of disparate data sources. The disadvantage of following the non-linear approach exclusively is lack of visibility, and loss of opportunity to share and reuse the knowledge gained and the process applied during discovery and analysis.
The hybrid approach embraces both linear and non-linear solutions by promoting agility, fostering collaboration, evolving best practices, accommodating new requirements, and providing a data foundation that scales with the needs of the business to realize the full value of the data.
The following considerations are imperative to adopting a hybrid paradigm:
IT must be an enabler, encouraging agility by empowering business users to discover, analyze and glean insights from data. At the same time, the business should be a sponsor of IT in establishing much-needed governance and stewardship initiatives.
Implement a top-down approach for pre-existing requirements and a bottoms-up, iterative approach when defining and refining requirements through discovery.
Discover relevant data sources and make all pertinent data accessible to appropriate stakeholders very quickly. Choose the appropriate stack to enable efficient and effective and discovery.
Incorporate a decentralized approach to discovering data products and gleaning insights, and a centralized approach for sharing and reusing these data products in order to enable consistency.
Avoid applying a single model or technology to every situation. The use case should drive the stack, not vice versa.
To remain competitive, track emerging trends and technologies so as to adapt tactically and strategically to their potential impact on business.
Neither a strictly linear, nor a strictly non-linear approach to data management and analytics is sufficient to accommodate the changing data landscape. Emerging architectures and stacks reflecting a flexible, hybrid approach should be considered for quickly transitioning raw data into insightful data and subsequently operationalizing this transition for predictability and reuse. In promoting a hybrid approach, organizations should incorporate the following guidelines in their information management strategy:
Category: Data Catalog Data Discovery Smart Data