You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This change addresses feedback from onboarding sessions.
- Add verification step after pushing to CI in getting started.
- Clarify that `discover` and `run` are still required for TIA.
- Fix the locally generated impact data file name.
- Clarify how to validate TIA locally.
- Add verification step after pushing to CI in TIA.
- Move "Next Steps" above setup examples to better show those examples
are optional and TIA is now setup.
Copy file name to clipboardExpand all lines: docs/guides/modules/test/pages/getting-started-with-smarter-testing.adoc
+8Lines changed: 8 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -422,6 +422,14 @@ jobs:
422
422
423
423
Commit both `.circleci/test-suites.yml` and `.circleci/config.yml` to your feature branch and push to your VCS.
424
424
425
+
=== Verify in CI
426
+
427
+
After your pipeline runs, verify the test job is behaving as expected before merging.
428
+
429
+
. In the CircleCI web app, navigate to your pipeline and open the test job.
430
+
. Check the *Tests* tab and confirm the number of test results matches what you see when running tests without the `testsuite` command. A difference in test results could be a misconfiguration in your `discover` command.
431
+
. Compare the job duration to a recent run without the `testsuite` command. The runtime should be similar - a significant difference may indicate a misconfiguration in your `test-suites.yml`.
432
+
425
433
== Next steps
426
434
427
435
At this stage, all of your tests should successfully run locally and in CI using the `testsuite` command.
Copy file name to clipboardExpand all lines: docs/guides/modules/test/pages/set-up-test-impact-analysis.adoc
+43-8Lines changed: 43 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,7 +64,7 @@ Before enabling test impact analysis, ensure you have completed the xref:getting
64
64
65
65
== 1. Enable test impact analysis option in your `.circleci/test-suites.yml` file
66
66
67
-
Update the testsuite from the getting started to include an `analysis` command and `test-impact-analysis` option.
67
+
Update the testsuite from the getting started to include an `analysis` command and `test-impact-analysis` option, keep the `discover` and `run` commands in the config.
68
68
69
69
Some starter configs are shown below. Choose the one for your test runner and copy/paste the `analysis: ...` line into `.circleci/test-suites.yml`.
70
70
@@ -220,7 +220,7 @@ Remove the `--doctor` flag and add the following flags: `--local --select-tests=
220
220
$ circleci run testsuite "ci tests" --verbose --local --select-tests=none --analyze-tests=impacted
221
221
----
222
222
223
-
Then inspect the locally stored impact data in `.circleci/impact-ci\ tests.json` to see the mapping between tests and files. This is a JSON file with a form like:
223
+
Then inspect the locally stored impact data in `.circleci/ci tests-impact.json` to see the mapping between tests and files. This is a JSON file with a form like:
224
224
[source,json]
225
225
----
226
226
{
@@ -230,6 +230,10 @@ Then inspect the locally stored impact data in `.circleci/impact-ci\ tests.json`
230
230
"path": ".circleci/test-suites.yml",
231
231
"hash": "c9684be83632a628"
232
232
},
233
+
"2": {
234
+
"path": "src/foo.ts",
235
+
"hash": "c9684be83632a628"
236
+
},
233
237
...
234
238
},
235
239
"edges": {
@@ -238,13 +242,33 @@ Then inspect the locally stored impact data in `.circleci/impact-ci\ tests.json`
238
242
}
239
243
----
240
244
241
-
Try modifying a source file whose ID appears in the list for a test atom and then run test selection to verify only impacted test atoms are selected.
245
+
* `files` are a map of IDs to files with the `path` relative to the working directory and a `hash` of the contents.
246
+
* `edges` are a map of test atoms with an array of file IDs that impact this test atom.
247
+
248
+
Try modifying a source file whose ID appears in the `edges`, then run test selection to verify only impacted test atoms are selected.
242
249
243
250
[source,console]
244
251
----
245
252
$ circleci run testsuite "ci tests" --verbose --local --select-tests=impacted
246
253
----
247
254
255
+
Look for the "Selecting tests..." section in output to see why a test was selected.
256
+
257
+
[source,console]
258
+
----
259
+
==> Selecting tests...
260
+
--> Selecting 'test-atom-one' due to modified file: 'src/foo.ts'
261
+
==> - 0 new test atoms
262
+
==> - 0 test atoms impacted by new files
263
+
==> - 1 test atoms impacted by modified files
264
+
==> - 0 test atoms impacted by removed files
265
+
==> - 0 test atoms failed previously
266
+
==> - 0 test atoms with no source file mappings in impact data
267
+
==> - 0 test atoms impacted by include rule
268
+
==> - 0 test atoms impacted by full test run paths
269
+
==> Selected 1 test atoms, Skipped 123 test atoms in 168ms
270
+
----
271
+
248
272
=== Local flags
249
273
250
274
[#local-cli-flags]
@@ -352,12 +376,27 @@ jobs:
352
376
path: test-reports
353
377
----
354
378
355
-
Commit both `.circleci/test-suites.yml` and `.circleci/config.yml` to your feature branch and push to your VCS. Follow your usual process to merge to your default branch.
379
+
Commit the restored `.circleci/config.yml` to your feature branch and push to your VCS. Follow your usual process to merge to your default branch.
380
+
381
+
NOTE: The local impact JSON files generated from running the local CLI should not be checked in to VCS.
382
+
383
+
=== Verify in CI
384
+
385
+
After analysis completes on your feature branch, verify test impact analysis is working correctly.
386
+
387
+
. In the CircleCI web app, navigate to your pipeline and open the test job.
388
+
. Check the job was successful and analyzed all tests. A successful analysis run will output "Found n files impacting tests" for each test atom.
389
+
. Modify a source file that appears in the impact data, then push. Only the tests that cover the modified file should be selected to run. Look for the "Selecting tests..." output in the job to confirm the correct tests were selected and the reason for selection.
356
390
357
391
*Test impact analysis is now set up for your test suite*. Feature branches will run the tests impacted by code changes, and your default branch will first run all tests and then analyze the tests impacted by code changes.
358
392
359
393
There are xref:#set-up-examples[other options] available if these defaults don't suit your project.
360
394
395
+
== Next steps
396
+
397
+
* xref:use-dynamic-test-splitting.adoc[Use Dynamic Test Splitting] to evenly split tests across parallel nodes.
398
+
* xref:auto-rerun-failed-tests.adoc[Auto Rerun Failed Tests] to automatically retry flaky tests.
399
+
361
400
[#set-up-examples]
362
401
== Common setup examples
363
402
@@ -487,7 +526,3 @@ jobs:
487
526
path: test-reports
488
527
----
489
528
490
-
== Next steps
491
-
492
-
* xref:use-dynamic-test-splitting.adoc[Use Dynamic Test Splitting] to evenly split tests across parallel nodes.
493
-
* xref:auto-rerun-failed-tests.adoc[Auto Rerun Failed Tests] to automatically retry flaky tests.
0 commit comments