Bài 7: Segmentation & Dimensionality Reduction

Series: Business Analytics Cheatsheet Series

Topics: clustering k-means gmm pca rfm segmentation

Source: BA8 — Segmentation & Dimensionality Reduction. CRISP-DM position: Modeling (unsupervised) + Data Preparation (PCA as feature extraction).

Core claim: clustering produces groups; only profiling, naming, and attached actions produce segments. A cluster without a KPI and an owner is a chart, not a strategy.

1. Why Segmentation & DR

Term	Type	Definition / Usage
Segmentation	Method class	Partition customers/products into groups for personalization, targeted marketing, and pricing; prioritize sales/service resources; discover structure in high-dimensional data.
Dimensionality reduction	Method class	Compress features while retaining signal — for noise removal, storage/compute reduction, visualization, and mitigating the curse of dimensionality.
Business framing	Principle	A segmentation is only useful if the segments differ in a way that justifies different actions. Statistical separation without actionable difference is a false positive.

2. Data Preparation for Clustering

Term	Type	Definition / Usage
Feature engineering	Technique	Domain-informed metrics (usage, tenure, spend, support tickets) beat raw columns. The feature set defines what “similar” means — this is the highest-leverage choice in the whole pipeline.
Scaling	Requirement	Critical, not optional. k-means uses Euclidean distance; large-scale features (Income) dominate small-scale ones (Age). Use `StandardScaler` (mean 0, std 1) so all features contribute equally; `MinMaxScaler` for bounded features.
Outlier handling	Requirement	Centroids are means — one outlier drags the centroid and skews the whole cluster. Options: trimming (remove extremes before clustering) or `RobustScaler` (median/IQR based, outlier-insensitive).
Missing values	Requirement	Impute before clustering; distance is undefined with NaNs. See file 03 for mechanism-appropriate strategies.

3. Distance Metrics

Metric	Type	Definition / Usage
Euclidean	Distance	Straight-line “ruler” distance; measures pure magnitude. Default for k-means. Highly sensitive to feature scale ⇒ standardization is almost always required.
Cosine	Distance	Angle between two vectors, ignoring magnitude. Use for high-dimensional sparse data — text (TF-IDF), recommender systems. Choose when orientation matters more than absolute values (e.g. topic content over document length).
Mahalanobis	Distance	Statistical distance: how many standard deviations a point is from the distribution centre. Automatically accounts for the covariance matrix, so it is scale-invariant. Best for outlier detection and clustering correlated features.
Hamming	Distance	Mismatch count for categorical features. Used inside k-prototypes.

4. k-means

Term	Type	Definition / Usage
Goal	Definition	Partition n data points into k distinct, non-overlapping clusters. Each cluster is represented by its centroid (mean of its points); each point is assigned to the nearest centroid.
Algorithm	Algorithm	1) Place k centroids (random or k-means++). 2) Assign: each point to its closest centroid. 3) Update: recompute each centroid as the mean of its cluster. 4) Repeat 2–3 until centroids stop moving (convergence).
Objective — WCSS	Formula	Minimize Within-Cluster Sum of Squares (also Inertia / SSE): `WCSS = Σᵢ₌₁ᵏ Σ_{x∈Cᵢ} ‖x − μᵢ‖²`. Total squared Euclidean distance from each point to its assigned centroid.
Interpretation of WCSS	Concept	Makes clusters as tight/compact as possible. Low inertia ⇒ points close to their centres. Statistical equivalence: minimizing WCSS = minimizing within-cluster variance; the Update step (move centroid to the mean) is exactly the operation that minimizes it.
Random initialization	Technique	Pick k random points as initial centroids. Pro: very fast. Con: can be unlucky — multiple centroids in one dense region ⇒ poor clusters, stuck in a bad local minimum.
k-means++	Technique	Probabilistic seeding that spreads initial centroids apart. Slower to initialize, far more reliable convergence. Use by default.
Mini-batch k-means	Algorithm	Updates centroids from small random batches instead of the full dataset each iteration. Speed: 2×–100× faster; Accuracy: approximate, inertia slightly higher — usually negligible for business insight. Use when the dataset does not fit comfortably in memory.

k-means failure modes

Failure mode	Cause	Result
Non-convex shapes	Algorithm assumes spherical clusters.	Fails on crescents, rings, and other complex geometries.
Varying densities	Euclidean distance ignores variance/spread.	Sparse clusters get split or absorbed into dense noise.
Unequal cluster sizes	Centroids are pulled toward the larger group.	Small, high-value segments get misclassified — often the exact segments you cared about.

Choosing k

Method	Type	Definition / Usage
Elbow method	Diagnostic	Plot Inertia (WCSS) against k. Look for the elbow — where the curve bends and the rate of decrease slows sharply. Business reading: diminishing returns — beyond the elbow, added complexity buys little. Weakness: often ambiguous.
Silhouette score	Metric	Measures how similar a point is to its own cluster (cohesion) vs other clusters (separation). Range −1 to +1: ≈ +1 well-clustered (dense, clearly separated); ≈ 0 overlapping/boundary; < 0 likely assigned to the wrong cluster. Use when the elbow is ambiguous. Business value: confirms segments are distinct enough to justify different strategies.
Business constraint	Practical	The number of segments the organization can actually run distinct campaigns for. Frequently a tighter constraint than any statistic — state it explicitly.

Practical diagnostics — is the model usable?

Check	Type	Definition / Usage
Stability across seeds	Diagnostic	Run k-means multiple times with different random initializations. Centroid drift between runs ⇒ unstable solution (noise or wrong k). Do not ship an unstable segmentation.
Cluster size distribution	Diagnostic	Are sizes reasonably balanced? Watch for micro-clusters (<1% of data) — these usually capture outliers/noise, not a valid market segment.
Business profiling	Diagnostic	Compute mean/median of key features per cluster; translate into labels (“Cluster 1 = High Income, Low Frequency”). If you cannot describe a cluster in one sentence, it is not a segment.

5. From Clusters to Segments

Term	Type	Definition / Usage
Feature set for customer clustering	Example	RFM-like + engagement: Usage (logins/month) · Tenure (months since signup) · Spend (avg monthly revenue) · Support Tickets (proxy for friction or engagement).
Interpreted segments	Example	Power Users: high usage, high spend, moderate tenure, few tickets. Loyal Low-Spend: high tenure, low spend, moderate usage, few tickets. At-Risk: low usage, low tenure, high tickets, low spend (struggling new users or churning old ones).
Naming segments	Practice	Replace “Cluster 3” with a persona derived from descriptive statistics: “High-Value Loyalists,” “Newbie Explorers,” “Churn Risks.” Stakeholders act on names, not indices.
Attach KPIs & actions	Practice	Per segment: identify relevant KPIs (churn rate, AOV, engagement score) and tailored interventions. Example — Churn Risks: KPI = reduced churn rate; Action = proactive support + win-back offers. This step is what converts a model into a strategy.

6. Gaussian Mixture Models (GMM)

Term	Type	Definition / Usage
GMM concept	Model	Assumes the dataset is generated from a mixture of several underlying Gaussian distributions. Analogy: the customer base is not one group but a blend of segments, each with its own bell-curve behavior.
Advantage over k-means — shape	Property	k-means assumes spherical clusters and fails on elongated/stretched patterns. GMM models clusters as ellipses, explicitly handling variance and covariance so boundaries can stretch and rotate to fit the data.
Advantage over k-means — assignment	Property	k-means forces hard assignment (100% to one cluster). GMM gives soft membership — probabilistic. Business value: identifies borderline customers (60% “Loyal” / 40% “At-Risk”) enabling more sophisticated targeting.
Component parameters	Parameters	Mean μ — cluster centre; the typical feature values of that segment. Covariance Σ — shape and orientation; the variance and correlation among features within the segment (e.g. how Spend and Usage co-vary). Weight π — the proportion of data in this cluster; how dominant the segment is.

Covariance types

Type	Shape	Constraint	Note
Spherical	Round spheres	Equal variance in all directions	Effectively reduces GMM to k-means. Least flexible, fastest.
Diagonal	Axis-aligned ellipses	Can stretch, cannot rotate	Assumes features are uncorrelated.
Full	Any ellipse	Free stretch and rotation	Most flexible, most expensive; prone to overfitting when data is scarce.
Tied	All clusters share one shape/orientation	Segments differ only by location (mean)	Good when you believe spread is common across segments.

The EM algorithm

Step	Type	Definition / Usage
The chicken-and-egg problem	Motivation	Cannot know parameters (μ, Σ, π) without memberships; cannot know memberships without parameters. Solution: iterate.
E-step (Expectation)	Algorithm step	Compute responsibilities — the probability that each point belongs to each cluster, given current parameters. “Point A is 80% Cluster 1, 20% Cluster 2.”
M-step (Maximization)	Algorithm step	Update parameters to fit the new soft assignments: move μ toward the weighted average of points; stretch/rotate Σ to fit the spread; adjust π by total probability mass.
Convergence	Criterion	Repeat E and M until the log-likelihood stops increasing.

Selecting the number of components

Term	Type	Definition / Usage
The overfitting problem	Concept	Unlike inertia, log-likelihood keeps increasing with more components — unpenalized, the model would create one cluster per data point (perfect fit, zero utility).
AIC — Akaike Information Criterion	Metric	Balances fit (likelihood) against complexity (number of parameters). Lower is better; take the minimum of the curve.
BIC — Bayesian Information Criterion	Metric	Same idea, stricter complexity penalty. Prefers simpler models ⇒ generally the safer default for business analytics, where over-segmentation is costly.
Cross-validation	Alternative	Check whether test-set log-likelihood stays high. Training up while test drops ⇒ overfitting.

Soft assignments in practice

Band	Interpretation	Action
> 0.8 — Core members	Firmly in the segment	Standard retention/loyalty campaign
≈ 0.5 / 0.5 — Borderline	On the fence, transitioning between behaviors	Personalized nudges; specific incentives to push them into the high-value segment

Example policy: P(VIP) > 0.9 → auto-upgrade to Gold. 0.5 < P(VIP) < 0.9 → send a challenge (“spend $50 more to unlock Gold”).

k-means vs GMM

Feature	k-means	GMM
Parameters	Means (centroids) only	Means (μ), Covariances (Σ), Weights (π)
Cluster shape	Spherical	Elliptical; can rotate
Assignment	Hard labels (100% one cluster)	Soft labels (probabilities)
Algorithm	Iterative centroid updates	EM (E-step & M-step)
Choosing k / components	Elbow (inertia)	AIC / BIC
Varying densities	Struggles	Handles effectively
Computational cost	Faster, scales to large N	Slower, especially with full covariance
Key advantage	Simplicity, speed	Flexibility, nuance, complex structure

7. PCA

Term	Type	Definition / Usage
Why reduce dimensionality	Motivation	Noise reduction (filter irrelevant variance so the model focuses on signal) · Compression (less storage and compute) · Visualization (project to 2D/3D for human interpretation) · Curse of dimensionality (mitigate performance degradation from sparsity in high dimensions).
Core idea	Concept	Identify orthogonal directions along which the data varies most. Perpendicularity ensures the components are uncorrelated. PCA effectively diagonalizes the covariance matrix to isolate signal from noise.
PCA via SVD	Algorithm	1) Center: `X ← X − μ`. 2) Decompose: `X = UΣVᵀ`. 3) Components: columns of `V` (right-singular vectors) are the principal directions. 4) Scores: projections `= UΣ`.
PCA steps	Process	Prepare: standardize (optional but usually required) and centre → `X_centered = X − μ`. Compute: SVD of centred data, or eigendecomposition of the covariance matrix. Select: choose k components via explained variance (e.g. 90% cumulative). Transform: `Z = X_centered · W_k`.
Explained Variance Ratio	Metric (formula)	`λᵢ / Σⱼ₌₁ᵈ λⱼ` — share of total variance carried by component i.
Scree plot	Chart	Eigenvalues (variance explained) vs component number. Elbow rule: keep components before the bend (signal), discard those after (noise/scree).
Cumulative explained variance	Technique	Instead of an elbow, choose the smallest m whose cumulative `Σᵢ₌₁ᵐ VarianceRatioᵢ` meets a target (90% or 95%). Trade-off: higher threshold ⇒ better reconstruction, less compression.
Reconstruction error	Metric (formula)	Information lost when projecting to a lower-dimensional subspace: `(1/m)·Σᵢ₌₁ᵐ ‖xᵢ − x_approx‖²` (MSE). Decreases as k rises; zero when k = d.
Whitening (optional)	Technique	`Z_white = PCᵢ/√λᵢ` — rescale components to unit variance. Turns an oriented ellipse into an isotropic sphere. Essential preprocessing for algorithms assuming isotropic covariance, e.g. ICA.
PCA projection (2D)	Usage	Project onto PC1 & PC2. Clusters and separability often become visible here even when hidden in the original space. Critical for spotting outliers and intrinsic structure before modeling.
Loadings (φ)	Interpretation	Coefficients defining the linear combination: `PC₁ = φ₁X₁ + φ₂X₂ + … + φ_pX_p`. Large absolute loading ⇒ that feature strongly drives the component. This is how you name a component.
Biplot	Chart	Plots Scores (samples as dots) and Loadings (features as vectors) together. Reading angles between loading vectors: ≈0° positive correlation · ≈180° negative correlation · ≈90° uncorrelated (orthogonal).

PCA caveats

Caveat	Consequence
Linear limitation	Assumes data lies on a linear subspace. Fails to unfold non-linear manifolds — it squashes the “Swiss Roll” and destroys its structure.
Scale sensitivity	Large-magnitude variables dominate the variance. Standardization is mandatory, not optional.
Outlier sensitivity	PCA minimizes least-squares error, so extreme outliers pull and distort the principal axes.

Beyond PCA

Term	Type	Definition / Usage
t-SNE	Technique	Non-linear; excels at preserving local cluster structure. Unfolds manifolds PCA flattens.
UMAP	Technique	Non-linear; faster than t-SNE and better balances local vs global structure.
Use with care	Warning	Both are stochastic (results vary per run) and highly sensitive to hyperparameters (perplexity, `n_neighbors`). Axes have no interpretable meaning, and inter-cluster distances are not reliable. Visualization only — do not feed into forecasting.

8. RFM Analysis

Term	Type	Definition / Usage
RFM	Framework	Behavioral segmentation quantifying customer value from past purchase behavior. Recency (R) = days since last transaction → engagement. Frequency (F) = count of total transactions → loyalty. Monetary (M) = total revenue generated → value.
Quantile binning	Technique	Divide customers into 5 equal groups (quintiles) per metric, normalizing to a 1–5 scale.
Scoring logic	Rule	F & M: higher value ⇒ higher score (5 best). R: lower value (more recent) ⇒ higher score (5 best) — the sign flip is the most common implementation bug.
Concatenation	Technique	Combine the three digits into a segment code (e.g. `555`). Simple, transparent, explainable to non-technical stakeholders — a real advantage over black-box clustering.
Distribution shape	Diagnostic	Real customer data is rarely bell-curved. Expect heavy right skew (long tail). F and M typically follow the Pareto principle (80/20): most customers transact rarely and small; a few “whales” drive value.
Implication for binning	Rule	Equal-width bins fail on skewed data. Quantile binning is essential to keep segments balanced and meaningful.
RFM scatter	Chart	Recency (X) vs Monetary (Y) separates “Active Spenders” from “Lost Cheap.” Use bubble size for Frequency. Champions = large bubbles in the recent + high-spend quadrant.

RFM segment taxonomy

Segment	Group	Definition
Champions	Growth & High Value	Bought recently, buy often, spend the most.
Loyal Customers	Growth & High Value	Buy on a regular basis; responsive to promotions.
Potential Loyalist	Growth & High Value	Recent customers with average frequency.
At Risk	Risk & Churn	Big spenders who haven’t purchased lately. Highest-priority intervention target.
Hibernating	Risk & Churn	Last purchase long ago, low spenders.
Lost	Risk & Churn	Lowest recency, frequency, and monetary scores.

RFM dashboard

Element	Purpose
KPIs per segment	AOV, churn rate, profitability by cluster (Champions vs At Risk).
Trendlines & migration	How customers move between segments over time — are Potential Loyalists graduating to Champions? Migration is the real signal; a static snapshot is not.
Conversion tracking	Response rates and ROI for campaigns targeted at specific segments.

9. Hybrid Segmentation Pipeline

Combines RFM’s interpretability with the structure-finding power of PCA and clustering.

Standardize  →  PCA  →  k-means / GMM  →  Profile & Name
   │             │           │                  │
 z-score      compress    partition       attach KPIs
 scaling      + denoise   or soft-assign  + actions + owner

Stage	What to do	Watch for
Standardize	Z-score all features; RobustScaler if outliers are legitimate.	Skipping this silently makes the highest-variance feature the only one that matters.
PCA	Retain components to a variance threshold (90–95%) or the scree elbow.	PCA components are not interpretable until you read the loadings.
Cluster	k-means (fast, hard) or GMM (flexible, soft). Select k via elbow + silhouette, or components via BIC.	Check stability across seeds; reject micro-clusters.
Profile & Name	Descriptive stats per cluster → persona → KPIs → actions → owner.	A segment without an owner and an action will not be used.

Definition of Done

Features standardized; outlier strategy chosen and documented.
Distance metric matches the data type (Euclidean for dense numeric, Cosine for sparse/text, Mahalanobis for correlated, k-prototypes for mixed).
k selected using elbow and silhouette, cross-checked against the number of segments the business can actually operate.
Stability verified across random seeds; no micro-clusters below 1% shipped as segments.
For GMM: covariance type justified; components chosen by BIC (preferred) or AIC.
For PCA: standardization applied; component count justified by cumulative explained variance; loadings read and components named.
t-SNE/UMAP used for visualization only — never as model input.
Every segment has: a human-readable name, a one-sentence profile, a KPI, a recommended action, and an owner.
Segment migration tracked over time, not just a one-off snapshot.

🔗 Cùng series

Bài 6: Classification Evaluation: ROC/PR, Thresholding, Cost-Sensitive, Calibration

Nếu bài này hữu ích, hãy chia sẻ cho người đang học Data Science / Business Analytics. Mọi góp ý về lỗi kỹ thuật rất được hoan nghênh.