Skip to content

chore: restore nightly protobuf compatibility check#13884

Draft
whowes wants to merge 1 commit into
mainfrom
whowes/restore-proto-compat-check
Draft

chore: restore nightly protobuf compatibility check#13884
whowes wants to merge 1 commit into
mainfrom
whowes/restore-proto-compat-check

Conversation

@whowes

@whowes whowes commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

WIP

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the nightly downstream Protobuf compatibility scripts to test local monorepo submodules directly instead of cloning them from GitHub, removing the special-casing for the google-cloud-java repository. Feedback on these changes recommends using standard CLI commands to retrieve tool versions rather than manually parsing versions.txt, redirecting error messages to standard error (>&2), and appending || exit 1 to pushd commands to ensure the scripts terminate safely if directory changes fail.

# The grep query tries to match `{artifact_id}:{released_version}:{current_version}`.
# The artifact_id must be exact otherwise multiple entries may match
version=$(cat "versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3)
version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Prefer using standard CLI commands to retrieve tool versions rather than manually parsing configuration files, unless performance or latency issues are actively observed.

Suggested change
version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3)
version=$(librarian config get version "${artifact_id}")
References
  1. Prefer using standard CLI commands (such as librarian config get version) to retrieve tool versions rather than manually parsing configuration files, unless performance or latency issues are actively observed.

mvn clean install -B -V -ntp -T 1C -DskipTests -Dclirr.skip -Denforcer.skip -Dmaven.javadoc.skip
repo_dir="${monorepoRoot}/${repo}"
if [ ! -d "${repo_dir}" ]; then
echo "Directory ${repo_dir} does not exist. Skipping or failed."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Error messages should be redirected to standard error (>&2) instead of standard output.

Suggested change
echo "Directory ${repo_dir} does not exist. Skipping or failed."
echo "Directory ${repo_dir} does not exist. Skipping or failed." >&2

exit 1
fi

pushd "${repo_dir}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is safer to append || exit 1 to pushd to ensure the script terminates if the directory change fails, preventing subsequent commands (like mvn clean install) from running in the wrong directory.

Suggested change
pushd "${repo_dir}"
pushd "${repo_dir}" || exit 1

pushd "$repo"
repo_dir="${monorepoRoot}/${repo}"
if [ ! -d "${repo_dir}" ]; then
echo "Directory ${repo_dir} does not exist. Skipping or failed."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Error messages should be redirected to standard error (>&2) instead of standard output.

Suggested change
echo "Directory ${repo_dir} does not exist. Skipping or failed."
echo "Directory ${repo_dir} does not exist. Skipping or failed." >&2

exit 1
fi

pushd "${repo_dir}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is safer to append || exit 1 to pushd to ensure the script terminates if the directory change fails, preventing subsequent commands from running in the wrong directory.

Suggested change
pushd "${repo_dir}"
pushd "${repo_dir}" || exit 1

@whowes
whowes force-pushed the whowes/restore-proto-compat-check branch 21 times, most recently from cd8ef4f to d8b7a8e Compare July 25, 2026 01:44
@whowes
whowes force-pushed the whowes/restore-proto-compat-check branch from d8b7a8e to 24ef975 Compare July 25, 2026 01:54
@sonarqubecloud

Copy link
Copy Markdown

@sonarqubecloud

Copy link
Copy Markdown

@whowes

whowes commented Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Kokoro nightly downstream compatibility scripts to test local monorepo submodules instead of cloning external repositories. It also adds KMS dependencies to the installation list, excludes StorageITRunnerTest from storage tests, and configures test options for Java 8 compatibility. Feedback on the changes suggests improving Bash script efficiency and robustness, specifically by using native parameter expansion instead of cut, avoiding useless use of cat with grep, and using Bash arrays instead of unquoted string variables to prevent word splitting issues.

artifact_list="${module_linkage_checker_arguments[$1]}"

for artifact in ${artifact_list//,/ }; do # Split on comma
artifact_id=$(echo "${artifact}" | cut -d ':' -f2)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a subshell with echo and cut to extract the artifact ID is inefficient. We can use native Bash parameter expansion ${artifact#*:} instead, which is faster and cleaner.

Suggested change
artifact_id=$(echo "${artifact}" | cut -d ':' -f2)
artifact_id="${artifact#*:}"

# The artifact_id must be exact otherwise multiple entries may match
version=$(cat "versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3)
repo_gav_coordinate="${artifact}:${version}"
version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3 || true)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid useless use of cat (UUOC) by passing the file directly to grep.

Suggested change
version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3 || true)
version=$(grep -E "^${artifact_id}:.*:.*$" "${monorepoRoot}/versions.txt" | cut -d ':' -f3 || true)

Comment on lines 41 to +60
if [ -n "${JAVA8_HOME}" ]; then
surefire_opt="-Djvm=${JAVA8_HOME}/bin/java"
jvm_opt="-Djvm=${JAVA8_HOME}/bin/java"
# Required for JDK 8 to bypass Java 11 LogbackServiceProvider and exclude JDK 11 Jqwik property tests
test_opts="-Dslf4j.provider=org.slf4j.helpers.NOP_FallbackServiceProvider -P!jqwik-tests -Pexclude-jqwik-on-java8"
else
# Provide a default value for local executions that don't configure JAVA8_HOME
surefire_opt="-Djvm=${JAVA_HOME}/bin/java"
jvm_opt="-Djvm=${JAVA_HOME}/bin/java"
test_opts=""
fi

# Compile the Handwritten Library with the Protobuf-Java version to test source compatibility
# Compile the library with the Protobuf-Java version to test source compatibility
# Run unit tests to help check for any behavior differences (dependant on coverage)
if [ "${repo}" == "google-cloud-java" ]; then
# The `-am` command also builds anything these libraries depend on (i.e. proto-* and grpc-* sub modules)
mvn test -B -V -ntp \
-Dclirr.skip \
-Denforcer.skip \
-Dmaven.javadoc.skip \
-Denforcer.skip \
-Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \
-pl "${google_cloud_java_handwritten_maven_args}" -am \
"${surefire_opt}" \
-T 1C
else
mvn test -B -V -ntp \
-Dclirr.skip \
-Denforcer.skip \
-Dmaven.javadoc.skip \
-Denforcer.skip \
-Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \
"${surefire_opt}" \
-T 1C
fi
mvn test -B -V -ntp \
-Dclirr.skip \
-Denforcer.skip \
-Dmaven.javadoc.skip \
-Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \
"${jvm_opt}" \
${test_opts} \
-T 1C

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using unquoted variables like ${test_opts} can lead to unexpected word splitting or globbing issues if they contain special characters. A more robust and idiomatic way to handle multiple command-line arguments in Bash is using arrays.

  declare -a test_opts=()
  if [ -n "${JAVA8_HOME}" ]; then
    jvm_opt="-Djvm=${JAVA8_HOME}/bin/java"
    # Required for JDK 8 to bypass Java 11 LogbackServiceProvider and exclude JDK 11 Jqwik property tests
    test_opts=(
      "-Dslf4j.provider=org.slf4j.helpers.NOP_FallbackServiceProvider"
      "-P!jqwik-tests"
      "-Pexclude-jqwik-on-java8"
    )
  else
    # Provide a default value for local executions that don't configure JAVA8_HOME
    jvm_opt="-Djvm=${JAVA_HOME}/bin/java"
  fi

  # Compile the library with the Protobuf-Java version to test source compatibility
  # Run unit tests to help check for any behavior differences (dependant on coverage)
  mvn test -B -V -ntp \
    -Dclirr.skip \
    -Denforcer.skip \
    -Dmaven.javadoc.skip \
    -Dprotobuf.version="${PROTOBUF_RUNTIME_VERSION}" \
    "${jvm_opt}" \
    "${test_opts[@]}" \
    -T 1C

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant