Wednesday, October 29, 2008

Eight Levels Of Analytics

The number of companies advertising Analytics as one of their service offerings leaves me astounded. I also come across an endless number of people who say they are into Analytics.

Maybe all this has to do with the very broad definition of Analytics. Or the way some people/companies interpret Analytics. A little more research, and a few more minutes of conversation reveals that the "Analytics" these companies or people actually do is just reporting. Personally speaking, I feel "Analytics" has become a widely misinterpreted and misused term.

According to Gartner, “Analytics leverage data in a particular functional process (or application) to enable context-specific insight that is actionable.”

The lastest
sascom online magazine describes the eight level of Analytics, from the simplest to the most advanced.

1. STANDARD REPORTS

What happened? When did it happen?
E.g. Monthly or quarterly financial reports.

Generated on a regular basis, they just describe “what happened” in a particular area. They’re useful to some extent, but not for making long-term decisions.

2. AD HOC REPORTS

How many? How often? Where?
E.g. Custom reports that describe the number of hospital patients for every diagnosis code for each day of the week.

At their best, ad hoc reports let you ask the questions and request a couple of custom reports to find the answers.

3. QUERY DRILLDOWN (OR OLAP)

Where exactly is the problem? How do I find the answers?
E.g. Sort and explore data about different types of cell phone users and their calling behaviors.

Query drilldown allows for a little bit of discovery. OLAP lets you manipulate the data yourself to find out how many, what color and where.

4. ALERTS

When should I react? What actions are needed now?
E.g. Sales executives receive alerts when sales targets are falling behind.

With alerts, you can learn when you have a problem and be notified when something similar happens again in the future. Alerts can appear via e-mail, RSS feeds or as red dials on a scorecard or dashboard.

5. STATISTICAL ANALYSIS

Why is this happening? What opportunities am I missing?
E.g. Banks can discover why an increasing number of customers are refinancing their homes.

Here we can begin to run some complex analytics, like frequency models and regression analysis. We can begin to look at why things are happening using the stored data and then begin to answer questions based on the data.

6. FORECASTING

What if these trends continue? How much is needed? When will it be needed?
E.g. Retailers can predict how demand for individual products will vary from store to store.

Forecasting is one of the hottest markets – and hottest analytical applications – right now. It applies everywhere. In particular, forecasting demand helps supply just enough inventory, so you don’t run out or have too much.

7. PREDICTIVE MODELING

What will happen next? How will it affect my business?
E.g. Hotels and casinos can predict which VIP customers will be more interested in particular vacation packages.

If you have 10 million customers and want to do a marketing campaign, who’s most likely to respond? How do you segment that group? And how do you determine who’s most likely to leave your organization? Predictive modeling provides the answers.

8. OPTIMIZATION

How do we do things better? What is the best decision for a complex problem?
E.g. Given business priorities, resource constraints and available technology, determine the best way to optimize your IT platform to satisfy the needs of every user.

Optimization supports innovation. It takes your resources and needs into consideration and helps you find the best possible way to accomplish your goals.

Thursday, October 16, 2008

Market Basket Analysis

Market Basket Analysis (MBA) is the process of analyzing transactional level data to determine the likelihood that a set of items/products will be bought together.

Retailers use the results/observations from an MBA to understand the purchase behaviour of customers for cross-selling, store design, discount plans and promotions. MBA can, and should be done across different branches/stores as the customer demographics/profiles and their purchase behavior usually varies across regions.

The most common technique used in MBA is Association Rules. The three measures of Association Rules are - Support, Confidence, and Lift

A --> B = if a customer buys A, then B is also purchased
LHS --> RHS
Condition --> Result
Antecedent --> Consequent

Support: Ratio of the # of transactions that includes both A & B to the total number of all transactions

Confidence: Ratio of the # of transactions with all items in the rule (A + B) to the # of transactions with items in the condition (A )

Lift: Indicates how much better the rule is at predicting the “result” or “consequent” as compared to having no rule at all, or how much better the rule does rather than just guessing

Lift = Confidence/P(result) = [P (A+B)/P(A)]/P(B)

EXAMPLE
If a customer buys milk, what is the likelihood of orange juice being purchased?

Milk --> Orange Juice
Customer Base: 1000
600 customers buy milk
400 customers buy orange juice
300 customers buy milk & orange juice

Support = P(milk & orange juice)/1000 = 300/1000 = 0.3

Confidence = P(milk & orange juice)/P(milk) = (300/1000)/(600/1000) = 0.5

Lift = Confidence/P(result) = 0.5/(400/1000) = 1.25

Interpretation: A customer who purchases milk is 1.25 times likely to purchase orange juice, than a randomly chosen customer.

THREE TYPES OF RULES PRODUCED BY ASSOCIATION RULES
Actionable: rules that can be justified and lead to actionable information
Trivial: rules that are obvious or already known (because of past/existing promotions, mandatory/required purchase of a stabilizer with an air conditioner…)
Inexplicable: rules that have no explanation and no course of action

DATA TYPE
Transactional data characterized by multiple rows per customer or order is the norm for MBA.

BASIC PROCESS FOR BUILDING ASSOCIATION RULES
1. Choose the right set of items/level of detail – items, product category, brands…?
2. Generate rules - one-way rules (2 items, A-->B), 2-way rules (3 items, A & B --> C)…?
3. Limit the orders/items in the analysis by
- considering only orders having at least as many items as are in the rule
- requiring a minimum support for the rule
- removing the largest orders having multiple items/products

MBA doesn’t refer to a single technique but a set of business problems related to understanding of POS transaction data. The most popular of these techniques happens to be Association Rules.