Skip to content

DefaultTestSet fields are const in 1.13, don't try to change 'em - #137

Open
kleinschmidt wants to merge 7 commits into
JuliaTesting:masterfrom
kleinschmidt:dfk/dont-change-default-testset
Open

kleinschmidt wants to merge 7 commits into
JuliaTesting:masterfrom
kleinschmidt:dfk/dont-change-default-testset

Conversation

@kleinschmidt

@kleinschmidt kleinschmidt commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

This is probably the simplest possible fix to #136 , which just does not attempt to set the elapsed time on the default test set since those fields are now const. I've updated the reference tests for a few of the tests; the only difference is that there's no trailing new line in how some of the outputs are printed.

Finally, we were not testing on versions of julia that we support and have different reference outputs for; I've added CI jobs for every supported minor version of Julia on ubuntu, keeping windows only on min/1/nightly.

fixes #136

@codecov

codecov Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.88%. Comparing base (be0d517) to head (6ca3275).
⚠️ Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
src/testsets.jl 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #137      +/-   ##
==========================================
+ Coverage   92.73%   92.88%   +0.14%     
==========================================
  Files           7        7              
  Lines         482      492      +10     
==========================================
+ Hits          447      457      +10     
  Misses         35       35              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kleinschmidt

Copy link
Copy Markdown
Contributor Author

Only failures appear to be that the test failures don't pretty print in XML with a trailing line for some reason. If it were me I'd just update the reference tests since this doesn't seem like a meaningful change...

@oxinabox

Copy link
Copy Markdown
Member

If it were me I'd just update the reference tests since this doesn't seem like a meaningful change...

Go ahead

@oxinabox oxinabox left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge once you fix CI

@omus omus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there is a minor test flaw but othewise looking good

Comment thread test/reportgeneration.jl Outdated
@tz-lom

tz-lom commented Oct 2, 2026

Copy link
Copy Markdown

@kleinschmidt are you planning to follow up on this?
do you need any help?

kleinschmidt and others added 3 commits October 2, 2026 10:05
Co-authored-by: Curtis Vogt <curtis.vogt@gmail.com>
@kleinschmidt

kleinschmidt commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

I've gone ahead and added matrix entries for all supported minor versions of Julia on ubuntu, and test min/1/nightly on windows.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broken on 1.3 as DefaultTestSet fields are now const

4 participants