Information Processing and Data Transmutation

Preparing The Ingredients

The framework of information to knowledge management is often the basis of Context Setting, Data Collection, Data Cleaning, Analysis, Visualization, and Insights and Recommendations.

Context Setting --> Data Collection --> Data Cleaning --> Analysis --> Visualization --> Insights and Recommendations
""" Steps to Extract Insights
[Context Setting]
Understand the context (cruxes), especially with user and stakeholder needs. Define Objectives (criteria) and Key Metrics (crucibles). 
Refer: Think Clearly: The Crux, The Criteria And The Crucible
e.g. Scenario: Service or utility wishes to understand user behavior to improve user experience and increase conversions, identify key areas for improvement in their products or services.
[Data Collection]: e.g. 
Identify Data Sources: Determine where data originates from (e.g., databases, APIs, applications, IoT devices).
Transaction history data, including date, time, user ID, product or item ID, quantity, and amount. This can also be campaign event sessions, including type of campaign (e.g. A/B testing experiments to compare different versions of trials), channels used, and campaign duration, activity intensity, frequency and recency rates, etc.
User data can include demographic data such as age, gender, location, behavioral such as interactions, such as clicks, scroll depth, navigation patterns, satisfaction outlook, reviews, etc.
Service or facilities analytics data, e.g. traffic, including views, session duration, bounce rate, and conversion rate.
[Data Cleaning]: e.g.
Eliminate duplicate records, interpolate and/or extrapolate missing data, etc. 
Remove Noise: Filter out and exclude irrelevant (e.g., spam comments, unrelated social media posts), false or counterfeit data, e.g. traffic from bots and other non-human sources.
Standardize Formats: Ensure data fields (e.g., dates, deanonymizing IDs) are in a consistent format.
Normalize Data: Ensure consistency in data metrics (e.g., time zones for session durations). Ensure consistency in text data (e.g., handle different spellings, terminologies, acronyms and abbreviations).

Data that varies in range can be mapped into a normalized range to relate to each other, e.g. cost and frequencies, weight and speed, etc.

[Data Analysis]: e.g.
Descriptive Statistics, thresholding…