[client-v2] Out-of-range numeric values are silently wrapped when inserted into Int32/Int64/UInt64 columns (Int8/Int16 correctly throw)

Describe the bug

SerializerUtils.serializePrimitiveData narrows the user-supplied value to a Java primitive without a range check for some integer column types. When the value does not fit, the extra bits are dropped silently and a wrong number is stored — no exception is raised and nothing is logged.

The behaviour is inconsistent across widths:

Column type	Conversion used	Out-of-range behaviour
`Int8`, `Int16`, `UInt8`, `UInt16`	`convertToInteger` → `BinaryStreamUtils.writeInt8/16/...`	Correct — `ClickHouseChecker.between` throws `IllegalArgumentException`
`Int32`	`convertToInteger(value)` → `((Number) value).intValue()`	Silently wraps
`Int64`, `UInt32`	`convertToLong(value)` → `((Number) value).longValue()`	Silently wraps (`Int64`)
`UInt64`, `Int128`, `UInt128`, `Int256`, `UInt256`	`NumberConverter.toBigInteger(value)` → `BigInteger.valueOf(((Number) value).longValue())`	Silently wraps

The module even has a correct helper already — NumberConverter.toInt / toLong compare intValue() against longValue() and throw ArithmeticException("integer overflow: ...") — but serializePrimitiveData does not call it for these branches.

The server rejects such a narrowing (SELECT toInt32(toDecimal64(4294967296, 0)) → DECIMAL_OVERFLOW), so the client is silently more permissive than the server and corrupts data.

ClickHouse server version

26.9.7.9 (local server at http://localhost:8123). The test below was run against it.

Reproduction

TestNG test in client-v2:

package com.clickhouse.client;

import com.clickhouse.client.api.Client;
import com.clickhouse.client.api.data_formats.RowBinaryFormatWriter;
import com.clickhouse.client.api.insert.InsertResponse;
import com.clickhouse.client.api.insert.InsertSettings;
import com.clickhouse.client.api.metadata.TableSchema;
import com.clickhouse.client.api.query.GenericRecord;
import com.clickhouse.data.ClickHouseFormat;
import org.testng.annotations.Test;

import java.math.BigDecimal;

public class NarrowingTest {

    private static String root(Throwable t) {
        while (t.getCause() != null) t = t.getCause();
        return t.getClass().getSimpleName() + ": " + t.getMessage();
    }

    @Test
    public void narrowing() throws Exception {
        Client client = new Client.Builder()
                .addEndpoint("http://localhost:8123")
                .setUsername("default").setPassword("")
                .setDefaultDatabase("default")
                .compressServerResponse(false).compressClientRequest(false)
                .build();

        client.execute("DROP TABLE IF EXISTS narrow32").get();
        client.execute("CREATE TABLE narrow32 (id UInt8, v Int32) ENGINE = Memory").get();

        Object[][] rows = {
                {(short) 1, new BigDecimal("100000")},      // in range
                {(short) 2, new BigDecimal("4294967296")},  // 2^32, out of Int32 range
                {(short) 3, 4294967296L},                   // same, as a Long
                {(short) 4, new BigDecimal("2147483648")},  // Int32 max + 1
        };

        TableSchema schema = client.getTableSchema("narrow32");
        ClickHouseFormat format = ClickHouseFormat.RowBinary;
        for (Object[] row : rows) {
            try (InsertResponse r = client.insert("narrow32", out -> {
                RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);
                for (int i = 0; i < row.length; i++) w.setValue(i + 1, row[i]);
                w.commitRow();
            }, format, new InsertSettings()).get()) {
                System.out.println("accepted id=" + row[0] + " value=" + row[1]);
            } catch (Exception e) {
                System.out.println("rejected id=" + row[0] + " value=" + row[1] + " -> " + root(e));
            }
        }
        for (GenericRecord rec : client.queryAll("SELECT id, v FROM narrow32 ORDER BY id")) {
            System.out.println("STORED id=" + rec.getInteger(1) + " v=" + rec.getLong(2));
        }

        // Contrast: Int16 is range-checked and rejects the same kind of value
        client.execute("DROP TABLE IF EXISTS narrow16").get();
        client.execute("CREATE TABLE narrow16 (id UInt8, v Int16) ENGINE = Memory").get();
        TableSchema s16 = client.getTableSchema("narrow16");
        try (InsertResponse r = client.insert("narrow16", out -> {
            RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, s16, format);
            w.setValue(1, (short) 1);
            w.setValue(2, new BigDecimal("40000"));
            w.commitRow();
        }, format, new InsertSettings()).get()) {
            System.out.println("Int16 40000 accepted");
        } catch (Exception e) {
            System.out.println("Int16 40000 -> " + root(e));
        }

        // 64-bit columns have the same defect
        client.execute("DROP TABLE IF EXISTS narrow64").get();
        client.execute("CREATE TABLE narrow64 (id UInt8, a Int64, b UInt64) ENGINE = Memory").get();
        TableSchema s64 = client.getTableSchema("narrow64");
        try (InsertResponse r = client.insert("narrow64", out -> {
            RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, s64, format);
            w.setValue(1, (short) 1);
            w.setValue(2, new BigDecimal("18446744073709551616")); // 2^64
            w.setValue(3, new BigDecimal("18446744073709551616")); // 2^64
            w.commitRow();
        }, format, new InsertSettings()).get()) {
            System.out.println("Int64/UInt64 2^64 accepted");
        } catch (Exception e) {
            System.out.println("Int64/UInt64 2^64 -> " + root(e));
        }
        for (GenericRecord rec : client.queryAll("SELECT id, toString(a), toString(b) FROM narrow64")) {
            System.out.println("STORED64 id=" + rec.getInteger(1) + " a=" + rec.getString(2) + " b=" + rec.getString(3));
        }
        client.close();
    }
}

Actual output (observed, `main` at `74b0a62`-era checkout, `0.11.0-rc1-SNAPSHOT`)

accepted id=1 value=100000
accepted id=2 value=4294967296
accepted id=3 value=4294967296
accepted id=4 value=2147483648
STORED id=1 v=100000
STORED id=2 v=0
STORED id=3 v=0
STORED id=4 v=-2147483648
Int16 40000 -> IllegalArgumentException: int(40000) should be between -32768 and 32767 inclusive of both values
Int64/UInt64 2^64 accepted
STORED64 id=1 a=0 b=0

Expected output

Rows 2, 3 and 4 should be rejected the same way the Int16 row is (an ArithmeticException/IllegalArgumentException about integer overflow), and the Int64/UInt64 row with 2^64 should be rejected too. The in-range row (100000 → 100000) is already correct and must stay correct.

Instead 4294967296 becomes 0, 2147483648 becomes -2147483648, and 2^64 becomes 0 in both the Int64 and the UInt64 column — silently.

Suggested fix

In client-v2/src/main/java/com/clickhouse/client/api/data_formats/internal/SerializerUtils.java:

serializePrimitiveData (around lines 649-685): use the already-existing range-checking helpers NumberConverter.toInt(value) for Int32, NumberConverter.toLong(value) for Int64, and a checked conversion for UInt32/UInt64 instead of convertToInteger / convertToLong.
convertToInteger (around line 1050) and convertToLong (around line 1064): the bare ((Number) value).intValue() / longValue() is where the bits are dropped. Either add the overflow check here or stop using these for the fixed-width integer branches.
NumberConverter.toBigInteger (around line 117 of NumberConverter.java): BigInteger.valueOf(((Number) value).longValue()) loses the high bits of a BigDecimal/BigInteger argument; it should use ((BigDecimal) value).toBigIntegerExact() for BigDecimal and pass a BigInteger through unchanged, then let the writeUnsignedInt64/writeInt128/... range checks apply.

Link

Found while checking whether ClickHouse/clickhouse-cs#646 (unchecked narrowing in ClickHouseDecimal's IConvertible members) also affects this client. The two C#-specific defects in that report (the (short) typo in ToInt32, and the Convert.ChangeType stack overflow) have no counterpart here — clickhouse-java uses java.math.BigDecimal and has no IConvertible/Convert.ChangeType equivalent. The third defect — out-of-range narrowing wrapping instead of raising an overflow error — does reproduce here, at Int32/Int64/UInt64 rather than at Int8/Int16.

Tracking: ClickHouse/integrations-ai-playground#532

@chernser

@chernser I have a fix ready for this. The root cause is one thing rather than the per-type table in the issue: the value is narrowed to a Java primitive before any range check runs, so the check is handed an already-wrapped number.

That also makes the narrow types wrong, which the issue lists as correct — 2^32 written to an Int8 column becomes 0 through intValue(), which then passes the Byte range check in BinaryStreamUtils and is stored. Same for Int16/UInt8/UInt16. So it is the narrowing, not a missing check.

The fix keeps the full magnitude during conversion so the range checks that already exist see the value the caller passed: NumberConverter.toBigInteger goes through toBigDecimal (already exact for every Number) instead of BigInteger.valueOf(longValue()), and convertToInteger/convertToLong range-check anything that is not already a Byte/Short/Integer/Long, so the common path stays allocation-free. Only the magnitude is checked — a fractional value is still truncated toward zero, and every value that fits its column serializes exactly as before.

Before I open it, one question, since this is a behaviour change on the insert path:

Is IllegalArgumentException on out-of-range the behaviour you want? It matches what Int8/Int16 already do today, so the write path would report one exception type instead of two behaviours — but it is breaking for anyone currently relying on the wrap. I am happy to put it behind a setting instead, or to default to lenient with a warning, if you would rather not change this by default.

Two smaller points for the same decision:

NaN and the infinities are rejected by the same change, where they previously stored 0 and Long.MAX_VALUE.
UInt64 is narrower than the issue suggests: a BigInteger argument already passes through untouched and is rejected correctly. Only a non-BigInteger Number (such as a BigDecimal) loses its value.

Locally: 84 unit tests pass with the fix, and 15 of them fail with the production change reverted, which are exactly the newly-fixed cases.

@piyush15102003

Good day, @piyush15102003 !

Thank you!

This is the point where we may be need two options - in most production setup we should not check each value range and just use wider type or accept it is not safe (because we know it is safe). So we might need two versions of converters.

The checked path is two long comparisons for Byte/Short/Integer/Long with no allocation — only BigDecimal/BigInteger/floating values take the slow path. I'll benchmark the insert path and post numbers.

If it's noise, I'd keep one converter. If you still want two, I'll select the implementation when the serializer is built rather than per value, checked by default — say if you want the opposite default.

One note: an Int32 column with 2^33 has no wider type to pick, so unchecked means storing a different number.

Benchmarked. My "two long comparisons, no allocation" was wrong — that only held for Integer.

JMH, convertToInteger over 1024 values per op, ns per value:

              old      new
Integer      1.4      1.6     no difference
Long         8.8     11.5     within noise here
BigInteger  11.3     22.0     ~2x
BigDecimal  21.2     71.1     ~3.4x

The first run was worse (Long +57%, BigInteger 2.9x, BigDecimal 8x). Two of those were my code rather than the check itself: Long fell through a generic instanceof chain instead of comparing directly, and the BigInteger path boxed both bounds with BigInteger.valueOf and ran two compareTo per value. Both fixed — bitLength() is 63 for Long.MIN_VALUE and MAX_VALUE, so one scan rules out anything too wide and the bounds compare as longs.

What's left is real. BigDecimal has to be converted exactly before its magnitude can be checked at all, so that cost doesn't go away. Integer and Long — what a POJO int/long field produces — are free or close to it.

So your call stands up: worth an opt-out for BigDecimal/BigInteger-heavy inserts, not worth it for primitives.

Two caveats. This measures convertToInteger only (the Int32 path), not Int64/UInt64 or a full row through RowBinaryFormatWriter. And it ran on a laptop with wide error bars — the Long row's intervals overlap, so that one needs a quiet host before anyone trusts it.

Navigation Menu

Repository navigation

[client-v2] Out-of-range numeric values are silently wrapped when inserted into Int32/Int64/UInt64 columns (Int8/Int16 correctly throw) #3176

Describe the bug

ClickHouse server version

Reproduction

Actual output (observed, `main` at `74b0a62`-era checkout, `0.11.0-rc1-SNAPSHOT`)

Expected output

Suggested fix

Link

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[client-v2] Out-of-range numeric values are silently wrapped when inserted into Int32/Int64/UInt64 columns (Int8/Int16 correctly throw) #3176

Description

Describe the bug

ClickHouse server version

Reproduction

Actual output (observed, main at 74b0a62-era checkout, 0.11.0-rc1-SNAPSHOT)

Expected output

Suggested fix

Link

Activity

piyush15102003 commented on Oct 5, 2026

chernser commented on Oct 5, 2026

piyush15102003 commented on Oct 8, 2026

piyush15102003 commented on Oct 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Actual output (observed, `main` at `74b0a62`-era checkout, `0.11.0-rc1-SNAPSHOT`)