chore: restore nightly protobuf compatibility check#13884
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the nightly downstream Protobuf compatibility scripts to test local monorepo submodules directly instead of cloning them from GitHub, removing the special-casing for the google-cloud-java repository. Feedback on these changes recommends using standard CLI commands to retrieve tool versions rather than manually parsing versions.txt, redirecting error messages to standard error (>&2), and appending || exit 1 to pushd commands to ensure the scripts terminate safely if directory changes fail.
| # The grep query tries to match `{artifact_id}:{released_version}:{current_version}`. | ||
| # The artifact_id must be exact otherwise multiple entries may match | ||
| version=$(cat "versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3) | ||
| version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3) |
There was a problem hiding this comment.
Prefer using standard CLI commands to retrieve tool versions rather than manually parsing configuration files, unless performance or latency issues are actively observed.
| version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3) | |
| version=$(librarian config get version "${artifact_id}") |
References
- Prefer using standard CLI commands (such as
librarian config get version) to retrieve tool versions rather than manually parsing configuration files, unless performance or latency issues are actively observed.
| mvn clean install -B -V -ntp -T 1C -DskipTests -Dclirr.skip -Denforcer.skip -Dmaven.javadoc.skip | ||
| repo_dir="${monorepoRoot}/${repo}" | ||
| if [ ! -d "${repo_dir}" ]; then | ||
| echo "Directory ${repo_dir} does not exist. Skipping or failed." |
| exit 1 | ||
| fi | ||
|
|
||
| pushd "${repo_dir}" |
There was a problem hiding this comment.
| pushd "$repo" | ||
| repo_dir="${monorepoRoot}/${repo}" | ||
| if [ ! -d "${repo_dir}" ]; then | ||
| echo "Directory ${repo_dir} does not exist. Skipping or failed." |
| exit 1 | ||
| fi | ||
|
|
||
| pushd "${repo_dir}" |
cd8ef4f to
d8b7a8e
Compare
d8b7a8e to
24ef975
Compare
|
|
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates the Kokoro nightly downstream compatibility scripts to test local monorepo submodules instead of cloning external repositories. It also adds KMS dependencies to the installation list, excludes StorageITRunnerTest from storage tests, and configures test options for Java 8 compatibility. Feedback on the changes suggests improving Bash script efficiency and robustness, specifically by using native parameter expansion instead of cut, avoiding useless use of cat with grep, and using Bash arrays instead of unquoted string variables to prevent word splitting issues.
| artifact_list="${module_linkage_checker_arguments[$1]}" | ||
|
|
||
| for artifact in ${artifact_list//,/ }; do # Split on comma | ||
| artifact_id=$(echo "${artifact}" | cut -d ':' -f2) |
There was a problem hiding this comment.
| # The artifact_id must be exact otherwise multiple entries may match | ||
| version=$(cat "versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3) | ||
| repo_gav_coordinate="${artifact}:${version}" | ||
| version=$(cat "${monorepoRoot}/versions.txt" | grep -E "^${artifact_id}:.*:.*$" | cut -d ':' -f3 || true) |
There was a problem hiding this comment.
| if [ -n "${JAVA8_HOME}" ]; then | ||
| surefire_opt="-Djvm=${JAVA8_HOME}/bin/java" | ||
| jvm_opt="-Djvm=${JAVA8_HOME}/bin/java" | ||
| # Required for JDK 8 to bypass Java 11 LogbackServiceProvider and exclude JDK 11 Jqwik property tests | ||
| test_opts="-Dslf4j.provider=org.slf4j.helpers.NOP_FallbackServiceProvider -P!jqwik-tests -Pexclude-jqwik-on-java8" | ||
| else | ||
| # Provide a default value for local executions that don't configure JAVA8_HOME | ||
| surefire_opt="-Djvm=${JAVA_HOME}/bin/java" | ||
| jvm_opt="-Djvm=${JAVA_HOME}/bin/java" | ||
| test_opts="" | ||
| fi | ||
|
|
||
| # Compile the Handwritten Library with the Protobuf-Java version to test source compatibility | ||
| # Compile the library with the Protobuf-Java version to test source compatibility | ||
| # Run unit tests to help check for any behavior differences (dependant on coverage) | ||
| if [ "${repo}" == "google-cloud-java" ]; then | ||
| # The `-am` command also builds anything these libraries depend on (i.e. proto-* and grpc-* sub modules) | ||
| mvn test -B -V -ntp \ | ||
| -Dclirr.skip \ | ||
| -Denforcer.skip \ | ||
| -Dmaven.javadoc.skip \ | ||
| -Denforcer.skip \ | ||
| -Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \ | ||
| -pl "${google_cloud_java_handwritten_maven_args}" -am \ | ||
| "${surefire_opt}" \ | ||
| -T 1C | ||
| else | ||
| mvn test -B -V -ntp \ | ||
| -Dclirr.skip \ | ||
| -Denforcer.skip \ | ||
| -Dmaven.javadoc.skip \ | ||
| -Denforcer.skip \ | ||
| -Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \ | ||
| "${surefire_opt}" \ | ||
| -T 1C | ||
| fi | ||
| mvn test -B -V -ntp \ | ||
| -Dclirr.skip \ | ||
| -Denforcer.skip \ | ||
| -Dmaven.javadoc.skip \ | ||
| -Dprotobuf.version=${PROTOBUF_RUNTIME_VERSION} \ | ||
| "${jvm_opt}" \ | ||
| ${test_opts} \ | ||
| -T 1C |
There was a problem hiding this comment.
Using unquoted variables like ${test_opts} can lead to unexpected word splitting or globbing issues if they contain special characters. A more robust and idiomatic way to handle multiple command-line arguments in Bash is using arrays.
declare -a test_opts=()
if [ -n "${JAVA8_HOME}" ]; then
jvm_opt="-Djvm=${JAVA8_HOME}/bin/java"
# Required for JDK 8 to bypass Java 11 LogbackServiceProvider and exclude JDK 11 Jqwik property tests
test_opts=(
"-Dslf4j.provider=org.slf4j.helpers.NOP_FallbackServiceProvider"
"-P!jqwik-tests"
"-Pexclude-jqwik-on-java8"
)
else
# Provide a default value for local executions that don't configure JAVA8_HOME
jvm_opt="-Djvm=${JAVA_HOME}/bin/java"
fi
# Compile the library with the Protobuf-Java version to test source compatibility
# Run unit tests to help check for any behavior differences (dependant on coverage)
mvn test -B -V -ntp \
-Dclirr.skip \
-Denforcer.skip \
-Dmaven.javadoc.skip \
-Dprotobuf.version="${PROTOBUF_RUNTIME_VERSION}" \
"${jvm_opt}" \
"${test_opts[@]}" \
-T 1C


WIP