Predicting Titanic survival: my first Kaggle ML project

Context

If you have spent any time in the data science world, you know the Titanic dataset. It is the classic entry point for supervised machine learning on Kaggle, and for good reason: the problem is intuitive, the data is approachable, and there is just enough messiness in it to teach you real skills. Missing values, mixed feature types, noisy columns -- it covers the fundamentals.

This was my first proper ML project. I wanted a challenge that would force me to build a complete pipeline from scratch: loading raw data, cleaning it, engineering features, comparing models, tuning hyperparameters, and submitting predictions. The Titanic competition gave me all of that in one place.

Objective

The task is a binary classification problem: given a set of features about each passenger, predict whether they survived (1) or did not survive (0). The dataset contains 891 labelled training records and 418 unlabelled test records. Submissions are scored on prediction accuracy against the held-out ground truth.

My final Gradient Boosting model achieved a Kaggle score of 0.78, which is a strong result for a first attempt and sits comfortably above many baseline submissions. The model reached 83.4% accuracy in cross-validation and 92.7% on the training set.

Method

The first challenge was preprocessing. The dataset has missing values in Age (about 20% missing), Cabin (77% missing), Fare, and Embarked. I built a ColumnTransformer pipeline to handle each feature type separately:

• Numeric features (Age, Fare, SibSp, Parch): median imputation for missing values
• Categorical features (Sex, Pclass, Embarked, Cabin): mode imputation then one-hot encoding

The Cabin column required extra attention. Most values were missing, and those that existed contained a deck letter followed by a cabin number (e.g. C123). I stripped the numbers and kept only the deck letter, turning a mostly-useless free-text column into a meaningful categorical feature.

I then ran five classifiers through 5-fold cross-validation to identify the best performers:

Random Forest and Gradient Boosting were the top two. I ran GridSearchCV on both, tuning depth, number of estimators, and learning rate. Gradient Boosting came out ahead at 0.834 and was selected for the final submission.

Findings

Before committing to the final model, I fitted a Logistic Regression and inspected the absolute values of its coefficients to understand which features were driving predictions. The results were striking.

Sex was the dominant predictor by a wide margin. The coefficient for being male (2.59) was more than twice the next largest feature. This reflects the historical reality of the disaster: women and children were prioritised for lifeboats, and survival rates for women were dramatically higher than for men.

Cabin deck came next. Decks C, G, E, F, and D all appeared in the top features. Cabin deck is a rough proxy for passenger class and location on the ship. Cabins on higher decks tended to belong to wealthier passengers with faster access to lifeboats. The fact that this feature held predictive power despite the high proportion of missing cabin values shows it was worth engineering.

Port of embarkation had a smaller but noticeable effect. Passengers who boarded at Southampton (S) or Queenstown (Q) showed a different survival profile compared to those from Cherbourg (C), which is the reference category. Cherbourg had a higher share of first-class passengers, which partly explains the difference.

Family size played a role. SibSp (number of siblings and spouses aboard) had a meaningful coefficient, suggesting that travelling with a small family was associated with slightly different survival odds compared to travelling alone or in a large group.

Taken together, the picture that emerges is consistent with historical accounts: survival on the Titanic was heavily influenced by gender, wealth (proxied by cabin and embarkation port), and to a lesser extent, family structure. The model learned these patterns from the data without any prior knowledge of the event.

The full notebook is on GitHub.