Can unsupervised learning identify penguin species on its own?

This weekend I set myself a challenge: could a machine learning model identify penguin species without ever being told what they are? No labels. No supervision. Just raw physical measurements and a question.

The Palmer Penguins dataset is a classic starting point for anyone exploring classification and clustering. It covers three species — Chinstrap, Gentoo, and Adélie — recorded across several Antarctic islands. The features I worked with were:

Beak (culmen) length & depth
Flipper length
Body mass
Sex

The approach

I used two techniques together. K-Means to partition the data into clusters, and t-SNE (t-distributed Stochastic Neighbour Embedding) to project the high-dimensional feature space down to two dimensions — making the structure of the data visible.

The first pass gave promising results. Three distinct groupings emerged, broadly aligning with the three species. But something was off: the clusters were messy in places, and one species in particular was being split across two groups.

t-SNE plot showing penguin species clusters with K-Means centroids marked as blue X — Initial t-SNE projection coloured by true species label. Blue Xs mark K-Means centroids. Notice the Adelie cluster splitting, and a centroid falling between two Chinstrap sub-clusters.

The key discovery: sexual dimorphism

The culprit was sexual dimorphism — the systematic physical differences between male and female penguins. Males tend to be heavier and have longer flippers; females are smaller across most measurements. These differences were large enough that the model was effectively treating male Adélies and female Adélies as separate groups.

Once I split the data by sex and visualised each group separately, the picture became much cleaner.

Side-by-side t-SNE plots for male and female penguins, each showing three clean species clusters — t-SNE projections split by sex. Each panel shows three tight, well-separated species clusters — a significant improvement over the combined view.

With sex accounted for, the clusters became tight and well-separated. From there, assigning species labels post-hoc was straightforward — the model had genuinely learned the structure in the data.

What I took away from this

A few things stood out:

Data preprocessing unlocks the magic of ML. The model wasn't wrong on the first pass — it was faithfully reflecting a real signal in the data. Understanding that signal (sexual dimorphism) was what made the difference.
Domain knowledge can matter as much as the model itself. No amount of hyperparameter tuning would have fixed the root issue. The insight was conceptual, not technical.
“Unsupervised” doesn't mean “hands-off”. The algorithm found real structure, but interpreting and validating that structure required human judgement throughout.

For anyone just getting into data science, I'd strongly recommend the Palmer Penguins dataset as a playground. It's small enough to iterate on quickly, but rich enough to surface genuinely interesting problems around clustering, visualisation, and feature engineering.

Curious — what's been your most surprising insight from a dataset?