Bài 4: EDA & Visualization for Decision-Making

Series: Business Analytics Cheatsheet Series

Topics: eda visualization correlation-causation simpsons-paradox chart-design

Source: BA4 — EDA & Visualization for Decision-Making. Running case: telco churn, 1,200 customers (plans, usage, charges, churn label). CRISP-DM position: Data Understanding, feeding Data Preparation and Modeling choices.

Core claim: EDA sharpens questions; visuals must inform decisions. EDA suggests, experiments confirm.

1. EDA Fundamentals

Term	Type	Definition / Usage
EDA — Exploratory Data Analysis	Method	Systematic exploration of data to form and test hypotheses: summarize distributions, relationships, anomalies. Bridges business questions and modeling choices.
EDA mindset	Principle	Start with questions and context; avoid aimless fishing. Iterate: profile → explore → hypothesize → validate. Triangulate with multiple views to reduce bias.
Data types & scales	Taxonomy	Numeric (continuous / discrete) · Categorical (nominal / ordinal) · Date-time and periodic encodings · derived features. The type dictates both the summary statistic and the chart.
Reproducible EDA	Practice	Notebook scripts, versioned data, saved figures and tables. Share EDA artifacts for review — an unreviewed EDA is an untested assumption.

2. Univariate Analysis

Goal: understand shape, center, spread; spot outliers and missingness; inform transformation and binning choices.

Term	Type	Definition / Usage
Central tendency	Summary statistic	Mean (sensitive to outliers) · median (robust) · mode (categorical). Report mean and median; a gap between them signals skew.
Dispersion	Summary statistic	Variance, standard deviation, quantiles, IQR. IQR is the robust choice under heavy tails.
Skewness	Summary statistic	Asymmetry of the distribution. Positive skew (long right tail) is the norm for revenue, usage, tenure. Drives the choice of log/Box-Cox transform.
Kurtosis	Summary statistic	Tail heaviness. High kurtosis ⇒ prefer robust summaries and robust scaling.
Histogram	Chart	Distribution of a numeric variable. Bin width selection matters: Freedman-Diaconis rule or sqrt rule. Two histograms of the same data with different bins can tell opposite stories.
Binning strategy	Technique	Equal-width vs equal-frequency (quantile). Keep bins consistent across compared groups.
Log / Box-Cox / Yeo-Johnson	Transform	Compress skew before visualizing or modeling. Box-Cox requires x > 0; Yeo-Johnson handles x ∈ ℝ.
Boxplot	Chart	Shows median, spread, and outliers compactly. Best for comparing many groups side by side.
Violin plot	Chart	Adds density shape to the boxplot’s summary. Use when the distribution is multimodal — a boxplot hides bimodality entirely.
Bar chart (counts)	Chart	Category frequencies (e.g. counts by plan type). Zero baseline mandatory.
Chart hazards	Anti-pattern	Misleading axis scales and overplotting. Both are correctable; neither is forgivable in a stakeholder deck.

3. Bivariate Analysis

Goal: quantify pairwise relationships; compare distributions across categories; guide transformations and interaction terms.

Pair type	Method / Chart	Definition / Usage
Numeric–Numeric	Scatter plot	Reveals shape, clusters, and outliers. Look explicitly for nonlinearity and heteroscedasticity (fanning spread).
Numeric–Numeric	Pearson correlation	Measures linear association, −1 to +1. Sensitive to outliers and blind to curvature.
Numeric–Numeric	Spearman correlation	Rank-based; captures monotonic non-linear association. Robust to outliers. Report alongside Pearson when the scatter is curved.
Numeric–Categorical	Grouped boxplots	Compare distributions across levels. Add strip/dot plots to show raw observations when n is small.
Numeric–Categorical	Aggregate + CI	Group means/medians with confidence intervals. A bar of means without an interval is a claim without evidence.
Categorical–Categorical	Contingency table	Cross-tabulation of counts. Report both row and column percentages — they answer different questions.
Categorical–Categorical	Stacked / grouped bars	Grouped for comparing levels; stacked for composition. Avoid stacking more than ~4 categories.
Categorical–Categorical	Chi-square test	Compares observed vs expected counts under independence. Significance ≠ practical importance; report effect size (Cramér’s V) too.
Multi-numeric	Correlation heatmap	Pearson matrix over numeric features. Fast multicollinearity scan before regression.

4. Multivariate Analysis

Goal: reveal joint structures and interactions beyond pairs; identify segments and conditional patterns.

Term	Type	Definition / Usage
Segmented views / faceting	Technique	Split by a key dimension (e.g. `plan_type`, `complaints_30d`). Use small multiples with shared axes to compare segments honestly.
Scatter matrix (pairs plot)	Chart	Grid of pairwise scatters. Use alpha and jitter to reduce overplotting. Sanity check before modeling.
Dimensionality reduction (EDA lens)	Technique	PCA for linear structure; t-SNE / UMAP for visualization only. Do not read cluster distances in t-SNE/UMAP as real distances.
Feature interactions (EDA cues)	Technique	Ratios, differences, domain-informed composites. Record the hypothesis when you spot an interaction — test it later, don’t hard-code it now.

5. Correlation vs Causation

This section is the deck’s centre of gravity. Decisions require causal reasoning; EDA alone cannot supply it.

Term	Type	Definition / Usage
Spurious correlation	Concept	Association with no causal link, arising from chance, confounding, or selection. Common in wide datasets — with enough columns, some pairs correlate by construction.
Confounder	Causal structure	A variable affecting both X and Y. Leaving it uncontrolled biases the X→Y estimate. Handle by stratification, matching, or explicit adjustment.
Collider	Causal structure	A variable affected by both X and Y. Conditioning on a collider creates spurious association where none existed. This is why “controlling for everything” is wrong.
DAG thinking	Technique	Draw the assumed causal graph before choosing controls. It makes confounder-vs-collider explicit and is cheap to do.
Simpson’s Paradox	Concept	The pooled trend disagrees with every within-group trend. Cause: unequal group sizes plus a confounder. Always check stratified views before reporting an aggregate direction.
Observational vs experimental	Method class	Randomized experiments identify causal effects. Observational data does not, absent strong assumptions.
DID — Difference-in-Differences	Quasi-experiment	Compares before/after change between treated and control groups. Assumes parallel trends — test it on pre-period data.
IV — Instrumental Variables	Quasi-experiment	Uses a variable affecting treatment but not the outcome directly. Valid instruments are rare and the assumption is untestable.
RDD — Regression Discontinuity	Quasi-experiment	Exploits a threshold rule assigning treatment. Strong local validity; results generalize only near the cutoff.
From EDA to decisions	Practice	Move from patterns → testable hypotheses. Quantify uncertainty; document assumptions. Label every statement as observation, hypothesis, or causal claim.

A/B testing, practically

Term	Type	Definition / Usage
Metric set	Design	One primary metric plus explicit guardrails, declared before launch.
Sample size & duration	Design	Derived from baseline rate and MDE; cover at least one full weekly cycle.
Randomization & bucketing	Design	Stable hash-based assignment on the correct unit; verify balance (sample ratio mismatch) before reading results.
Peeking	Anti-pattern	Repeated significance checks inflate false positives. Fix the horizon or use a sequential method designed for it.

6. Visualization Principles

Term	Type	Definition / Usage
Marks & channels	Principle	Position and length are the most precisely decoded channels. Area, volume, and colour saturation are decoded poorly — avoid them for primary comparisons.
Context	Principle	Clear titles, labels, units, and a reference point (target, baseline, prior period). A chart without a reference is a chart without a message.
Chart choice by question	Decision rule	Distribution → histogram / boxplot. Relationship → scatter / line. Composition → stacked bar. Comparison → grouped bar / dot plot. Avoid 3D pies — and 2D pies beyond three slices.
Zero baseline	Rule	Mandatory for bar charts (length encodes value). Optional for line charts of trends, where truncation is acceptable if labelled.
Log scale	Rule	Use for heavy-tailed data or multiplicative growth. Always label the scale — readers assume linear by default.
Dual y-axes	Anti-pattern	Avoid unless unavoidable; the apparent crossing point is an artifact of arbitrary scaling. Prefer indexed series or small multiples.
Binning & smoothing	Trade-off	Noise vs detail. Keep bins consistent across compared panels; show raw points with jitter/alpha when feasible.
Small multiples / faceting	Technique	Repeated layout across groups or time. The most reliable way to show conditional patterns without overloading one chart.
Chartjunk	Anti-pattern	Non-data ink: gradients, shadows, decorative icons, redundant gridlines. Remove. Limit encodings per chart; use annotation sparingly to guide the story.
Decision dashboards	Practice	Tie every visual to a question and an action; highlight targets and thresholds explicitly. A dashboard nobody acts on is a maintenance liability.

7. Checklists

EDA checklist

[ ] Types, ranges, missingness, duplicates
[ ] Univariate: shape, center, spread, outliers
[ ] Bivariate: correlations (Pearson + Spearman), group comparisons
[ ] Multivariate: faceting, interactions, joint structure
[ ] Stratify by key segments (channel, device, plan, cohort)
[ ] Log every insight as observation / hypothesis / causal claim
[ ] Save figures, tables, and the notebook; version the data

Common anti-patterns

Anti-pattern	Why it fails	Fix
Cherry-picking	Selecting the segment/window that supports a preferred conclusion.	Pre-register the cuts; report all of them.
Overplotting	Dense scatter hides the true density.	Alpha, jitter, hexbin, sampling, or 2D density.
Causal claims from correlation	Confounding and selection are unaddressed.	Label as hypothesis; name the experiment that would test it.
Aimless fishing	Multiple comparisons guarantee false discoveries.	Start from questions; adjust or replicate.
Reporting only pooled results	Simpson’s Paradox.	Always show a stratified view.

8. Self-check questions

Name two ways a histogram can mislead. (bin width choice; truncated or non-zero axis)
Explain Simpson’s Paradox in one sentence. (the pooled trend can reverse every within-group trend when group sizes and a confounder interact)
Suggest a visualization for a numeric–categorical comparison. (grouped boxplot, or dot plot of group means with CIs)
Why can correlation heatmaps be misinterpreted? (Pearson captures only linear association; nonlinearity, outliers, and confounding are invisible)
When should you use a log scale? (heavy-tailed distributions or multiplicative growth — and label it)

Definition of Done

Every EDA finding is labelled observation / hypothesis / causal claim.
Every aggregate direction has been checked against a stratified view.
Both Pearson and Spearman reported wherever a scatter shows curvature.
Every chart has a title, units, and a reference point; no 3D, no dual axes, no chartjunk.
Notebook, data version, and figures saved and shared for review.
A one-page executive EDA summary exists, written as decisions-supported rather than charts-produced.

References: Anscombe (1973) · Tufte · Cleveland · scikit-learn and matplotlib documentation.

🔗 Cùng series

Bài 3: Data Quality & Feature Engineering
Bài 5: Linear Regression (Simple & Multiple)

Nếu bài này hữu ích, hãy chia sẻ cho người đang học Data Science / Business Analytics. Mọi góp ý về lỗi kỹ thuật rất được hoan nghênh.