System Information
- OS & Version: Linux x64
- ML.NET Version:
Microsoft.ML.LightGbm 5.0.0
- Native LightGBM Version:
4.6.0
- .NET Version: reproduced with .NET 8.0 and .NET 10.0.12 (SDK 10.0.401)
Describe the bug
LightGbmRankingTrainer.Fit(trainData, validationData) does not preserve the validation query groups.
The reproduction below contains 5,000 contiguous groups with 4 rows each. Training without validation succeeds. When validation data is supplied, LightGBM treats all 20,000 validation rows as one query and fails with:
Number of rows 20000 exceeds upper limit of 10000 for a query
For validation datasets with 10,000 total rows or fewer, this appears to fail silently: training succeeds, but validation metrics may be calculated as if the entire validation set were one query.
To Reproduce
Create a .NET 8 console application and install ML.NET LightGBM 5.0.0:
dotnet new console -n LightGbmValidationGroupRepro -f net8.0
cd LightGbmValidationGroupRepro
dotnet add package Microsoft.ML.LightGbm --version 5.0.0
Replace Program.cs with:
using Microsoft.ML;
using Microsoft.ML.Data;
var ml = new MLContext(seed: 1);
var train = ml.Data.LoadFromEnumerable(Rows(groupIdOffset: 0));
var validation = ml.Data.LoadFromEnumerable(Rows(groupIdOffset: 10_000));
var trainer = ml.Ranking.Trainers.LightGbm(
labelColumnName: nameof(Row.Label),
featureColumnName: nameof(Row.Features),
rowGroupColumnName: nameof(Row.GroupId),
minimumExampleCountPerLeaf: 1,
numberOfIterations: 5);
// trainer.Fit(train) succeeds.
trainer.Fit(train, validation);
static IEnumerable<Row> Rows(int groupIdOffset)
{
for (var group = 0; group < 5_000; group++)
for (var position = 0; position < 4; position++)
yield return new Row
{
GroupId = (uint)(groupIdOffset + group + 1),
Label = 3 - position,
Features = new[] { (float)position, (float)(group % 17) },
};
}
public sealed class Row
{
public uint GroupId { get; set; }
public float Label { get; set; }
[VectorType(2)]
public float[] Features { get; set; } = [];
}Run:
Actual behavior
[LightGBM] [Fatal] Number of rows 20000 exceeds upper limit of 10000 for a query
System.InvalidOperationException:
LightGBM Error, code is -1, error message is
'Number of rows 20000 exceeds upper limit of 10000 for a query'.
Expected behavior
The validation dataset should contain 5,000 queries of 4 rows each. No query exceeds LightGBM's 10,000-row limit, so training with validation should succeed.
Screenshots, Code, Sample Projects
The complete minimal reproduction is included above. It reproduces in a standard net8.0 console project.
Additional context
This reproduction uses a non-zero uint GroupId directly. It does not involve MapValueToKey, the Hash transformer, missing keys, or hash collisions.
Source inspection suggests that the validation dataset is created by reference to the training dataset. The explicit group sizes are set, but FinishLoad() later reconstructs the validation query boundaries from a zero-initialized query-ID buffer and overwrites them with one group containing every validation row.
A possible fix may be to set the validation groups after loading the validation rows, or to clear the pending query-ID buffer when explicit group counts are supplied.
System Information
Microsoft.ML.LightGbm 5.0.04.6.0Describe the bug
LightGbmRankingTrainer.Fit(trainData, validationData)does not preserve the validation query groups.The reproduction below contains 5,000 contiguous groups with 4 rows each. Training without validation succeeds. When validation data is supplied, LightGBM treats all 20,000 validation rows as one query and fails with:
For validation datasets with 10,000 total rows or fewer, this appears to fail silently: training succeeds, but validation metrics may be calculated as if the entire validation set were one query.
To Reproduce
Create a .NET 8 console application and install ML.NET LightGBM 5.0.0:
dotnet new console -n LightGbmValidationGroupRepro -f net8.0 cd LightGbmValidationGroupRepro dotnet add package Microsoft.ML.LightGbm --version 5.0.0Replace
Program.cswith:Run:
Actual behavior
Expected behavior
The validation dataset should contain 5,000 queries of 4 rows each. No query exceeds LightGBM's 10,000-row limit, so training with validation should succeed.
Screenshots, Code, Sample Projects
The complete minimal reproduction is included above. It reproduces in a standard
net8.0console project.Additional context
This reproduction uses a non-zero
uint GroupIddirectly. It does not involveMapValueToKey, the Hash transformer, missing keys, or hash collisions.Source inspection suggests that the validation dataset is created by reference to the training dataset. The explicit group sizes are set, but
FinishLoad()later reconstructs the validation query boundaries from a zero-initialized query-ID buffer and overwrites them with one group containing every validation row.A possible fix may be to set the validation groups after loading the validation rows, or to clear the pending query-ID buffer when explicit group counts are supplied.