[Bug](stream-load) Stream Load crashes BE with strict_mode and all-expression `columns`

[re20052]

Search before asking

• I had searched in the issues and found no similar issues.

Version

Apache Doris 4.0

What's Wrong?

BE crashes (SIGSEGV) when a Stream Load uses strict_mode=true together with a columns header in which every target column is assigned via an expression (no direct-mapped column), and at least one row produces a NULL on a derived column.

Crash location

FileScanner::_convert_to_output_block() in be/src/vec/exec/scan/file_scanner.cpp.

The strict-mode branch indexes _src_slot_descs_order_by_dest[dest_index] and _dest_slot_to_src_slot_index[dest_index] without checking their size.

Root cause

These two members are populated only when FE sends dest_sid_to_src_sid_without_trans, which only happens if at least one target column is direct-mapped (no = in columns). When every target column uses an expression, both containers stay empty and the strict-mode branch reads them out of bounds.

What You Expected?

Stream Load should not crash BE. The strict-mode branch should be guarded by a size / existence check on _src_slot_descs_order_by_dest and _dest_slot_to_src_slot_index, and fall through to the regular nullable check when no source-column mapping exists for a derived column.

How to Reproduce?

Table

CREATE TABLE sl_min (
    k BIGINT NOT NULL,
    v VARCHAR(64) NULL
) ENGINE=OLAP
DUPLICATE KEY(k)
DISTRIBUTED BY HASH(k) BUCKETS 1
PROPERTIES ("replication_num" = "1");

Source file sl_min.csv (one row, two fields):

1,\N

Stream Load

curl --location-trusted -u root: \
    -H "label:sl_min_$(date +%s)" \
    -H "format:csv" \
    -H "column_separator:," \
    -H "columns:c1,c2,k=c1,v=c2" \
    -H "strict_mode:true" \
    -H "max_filter_ratio:0" \
    -T ./sl_min.csv \
    "http://<fe_host>:<fe_http_port>/api/<db>/sl_min/_stream_load"

The BE process crashes with SIGSEGV. curl reports Warning: Binary output can mess up your terminal because the connection is dropped mid-response.

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[zclllyybb]

Breakwater-GitHub-Analysis-Slot: slot_bd75fca6c132
This content is generated by AI for reference only.

Initial code-level triage: this report looks valid and points to a real BE crash bug, not a user-side load configuration problem.

What I checked:

• On the refreshed upstream/branch-4.0 ref (e9057613d08), be/src/vec/exec/scan/file_scanner.cpp initializes _src_slot_descs_order_by_dest only inside if (_params->__isset.dest_sid_to_src_sid_without_trans), but _convert_to_output_block() later reads _src_slot_descs_order_by_dest[dest_index] in the strict-mode NULL branch without first proving that the vector was populated.
• On current upstream/master (556586ce729), the scanner file has moved to be/src/exec/scan/file_scanner.cpp, but the same access pattern is still present.
• FE load planning in fe/fe-core/src/main/java/org/apache/doris/nereids/load/NereidsLoadPlanInfoCollector.java only fills destSlotIdToSrcSlotIdWithoutTrans for target slots that are direct table-column mappings. With columns:c1,c2,k=c1,v=c2, the target table columns are produced through expressions, so that map can legitimately be empty and the optional Thrift field is not set.

Failure chain:

1. The source tuple has file slots such as c1 and c2.
2. The destination tuple has table slots k and v, both evaluated through expressions k=c1 and v=c2.
3. Since there is no direct destination-column to source-column mapping, FE can send no dest_sid_to_src_sid_without_trans.
4. BE still builds _dest_vexpr_ctx for the destination columns, but leaves _src_slot_descs_order_by_dest empty.
5. When an expression result is NULL, FileScanner::_convert_to_output_block() enters the strict-mode branch and indexes _src_slot_descs_order_by_dest[dest_index]. For an empty vector this is out-of-bounds and can SIGSEGV before the regular nullable-column handling is reached.

Recommended fix direction:

• Treat "no direct source slot for this destination slot" as a valid state for expression-derived columns.
• In BE, either size _src_slot_descs_order_by_dest to the destination-slot count and fill missing entries with nullptr, or guard the strict source-value rejection path with an explicit "direct source mapping exists for this dest slot" check.
• Avoid using _dest_slot_to_src_slot_index[dest_index] unless that direct mapping has been proven to exist. A find()/checked lookup or a shared helper would avoid accidental insertion through operator[].
• For expression-derived columns without a direct source mapping, fall through to the existing nullable check. That preserves the intended distinction: strict mode can reject "source value was non-NULL but converted to NULL" only when there is a direct source value to report; ordinary destination nullability still decides whether a NULL derived result is allowed.

Suggested regression coverage:

• Stream Load CSV case with strict_mode=true, columns:c1,c2,k=c1,v=c2, and one row 1,\N into a table where v is nullable. Expected: no BE crash; the load should complete according to normal nullable semantics.
• A companion case where the derived destination column is NOT NULL. Expected: no BE crash; the row should be rejected by the normal not-null load validation path.
• A mixed direct plus derived-column case should also be kept to ensure the strict-mode "bad source value" error path still reports the original source value when a direct mapping exists.

The only extra information that would help confirm deployment impact is the exact 4.0.x build or commit and the BE stack trace from be.out, but the code path above is already sufficient to justify a fix.