CatBoost layered P6 target and object-weighting logic

Overview#

This article documents the current target-construction and object-weighting logic used in the layered CatBoost notebook for the P6 task.

The reference implementation currently lives in NN/NN_5/CatBoost_5_layered/CatBoost_5_EAM_on_P6_bump2_Layered.ipynb.

The important distinction is that the notebook keeps a binary target for training and threshold selection, but it also preserves the raw P6 signal strength so Pool(weight=...) can amplify stronger positive examples during fit.

Why this logic exists#

• P6aims defines whether a row belongs to the positive class.
• P6dev adds a business constraint: some rows that look positive by P6aims still become negative if deviation is too weak.
• raw P6aims and P6dev still contain useful gradation above that binary split, so the notebook stores them in service columns and uses them for weighting.
• the model remains a classifier, not a regressor; weighting only changes how strongly each training row influences the loss.

Primary runtime flow#

1. Load raw rows and convert textual P6aims/P6dev values into fractions.
2. Store raw fractions in service columns before binary target conversion.
3. Build the binary target from P6aims.
4. Apply the business reclassification rule for P6dev < -6%.
5. Optionally drop rows with P6dev > 0%.
6. Build object weights from raw P6aims and raw P6dev.
7. Pass these weights into CatBoost.Pool(weight=...) for train, validation, and test splits.

Target construction#

t_frac <  P6AIMS_POS_THRESHOLD_FRAC -> class 0
t_frac >= P6AIMS_POS_THRESHOLD_FRAC -> class 1

if P6dev < -0.06:
    target = 0

Service columns used for weighting#

The notebook stores raw fractions in service columns before target conversion:

• _P6aims_raw_frac
• _P6dev_raw_frac

This is necessary because after binary conversion the target no longer distinguishes cases such as 0.15, 0.20, 0.80, and 1.20, while weighting still needs that strength information.

Object-weighting formula#

Each row starts with a base weight of 1.0.

Additional weight is applied only to rows that remain in class 1 after the full target pipeline.

weight = 1.0
       + OBJECT_WEIGHT_POSITIVE_P6AIMS_GAIN * p6aims_strength
       + OBJECT_WEIGHT_POSITIVE_P6DEV_GAIN  * p6dev_strength

In the current notebook configuration:

• OBJECT_WEIGHT_POSITIVE_P6AIMS_GAIN = 0.90
• OBJECT_WEIGHT_POSITIVE_P6DEV_GAIN = 0.25

How P6aims strength is computed#

P6aims is the main weighting driver.

The notebook maps raw P6aims from the positive threshold to a configurable cap:

p6aims_strength = clip(
    (P6aims_raw_frac - P6AIMS_POS_THRESHOLD_FRAC)
    / (OBJECT_WEIGHT_P6AIMS_CAP_FRAC - P6AIMS_POS_THRESHOLD_FRAC),
    0,
    1
)

With the current defaults:

• P6AIMS_POS_THRESHOLD_FRAC = 0.15
• OBJECT_WEIGHT_P6AIMS_CAP_FRAC = 1.20

Operational meaning:

• P6aims = 0.15 -> p6aims_strength = 0
• P6aims = 0.20 -> small positive bonus
• P6aims = 0.80 -> strong positive bonus
• P6aims >= 1.20 -> capped maximum bonus

How P6dev strength is computed#

P6dev is a secondary, softer weighting signal.

p6dev_strength = clip(
    (P6dev_raw_frac - P6DEV_NEG_THRESHOLD_FRAC)
    / (P6DEV_GT_ZERO_THRESHOLD_FRAC - P6DEV_NEG_THRESHOLD_FRAC),
    0,
    1
)

With the current defaults:

• P6DEV_NEG_THRESHOLD_FRAC = -0.06
• P6DEV_GT_ZERO_THRESHOLD_FRAC = 0.00
• OBJECT_WEIGHT_P6DEV_MISSING_STRENGTH = 0.0

Operational meaning:

• P6dev = -0.06 -> p6dev_strength = 0
• P6dev = -0.03 -> medium bonus
• P6dev = 0.00 -> maximum bonus
• missing P6dev -> neutral bonus 0

Important edge case: disabling P6dev > 0% drop#

If APPLY_P6DEV_GT_ZERO_DROP is changed from True to False, rows with P6dev > 0% are no longer removed.

Under the current weighting formula such rows do not just remain in the dataset. If they still belong to class 1, they receive the maximum P6dev bonus because the formula clips all values above 0% to p6dev_strength = 1.

That means values such as P6dev = 1%, 2%, ..., 6% would be treated by weighting as the strongest possible P6dev cases unless the formula is changed as well.

What happens to the negative class#

The notebook intentionally amplifies only the positive class.

Rows that end up in class 0 do not receive a custom negative bonus. They keep the base weight, subject only to final clipping and normalization.

This is important for rows reclassified by P6dev < -6%: once they become negative, they stop receiving any positive weighting bonus even if raw P6aims looked strong.

Clip and normalization#

After the raw bonuses are applied, the notebook stabilizes the scale in two steps:

1. clip weights into [OBJECT_WEIGHT_CLIP_MIN, OBJECT_WEIGHT_CLIP_MAX]
2. optionally divide all weights by the split mean so the average weight becomes 1.0

Current defaults:

• OBJECT_WEIGHT_CLIP_MIN = 0.75
• OBJECT_WEIGHT_CLIP_MAX = 2.25
• OBJECT_WEIGHT_NORMALIZE_TO_MEAN_1 = True

This keeps the overall loss scale close to the baseline experiment and prevents a small subset of extreme rows from dominating fit.

Where the weights are used#

The notebook builds weights separately for the train, validation, and test splits and passes them directly into CatBoost pools:

train_pool = Pool(..., weight=train_object_weights)
val_pool   = Pool(..., weight=val_object_weights)
test_pool  = Pool(..., weight=test_object_weights)

Weight summaries are printed before fit so changes in mean, median, max, and positive/negative split behavior can be checked before a long training run starts.

Practical diagnosis checklist#

1. If the positive share suddenly changes, first inspect the P6aims >= threshold rule and the P6dev < -6% reclassification path.
2. If unexpectedly strong rows dominate fit, inspect OBJECT_WEIGHT_P6AIMS_CAP_FRAC, gains, clip bounds, and the final weight summaries.
3. If P6dev > 0% rows start appearing, confirm whether APPLY_P6DEV_GT_ZERO_DROP was disabled intentionally.
4. If that drop was disabled, verify whether the weighting formula was also adapted so positive P6dev values do not receive an unintended maximum bonus.