Create PRs Suggesting Updates to Conformance Docs
Several of our PRs reviewers gave feedback indicating that particular tests were not good candidates for Conformance.
The reviewer's judgement was often not directly based on the documented conformance criteria. This is a challenge, as different reviewers may be keeping personal and distinct lists of criteria by which they judge a test promotion candidate.
Sometimes PRs are rejected due to tests not following best practices.
We also have run into some of the constraints of the conformance tooling which we have not seen documented.
We are opening PRs against the documentation with our suggestions. The hope is to generate a productive discussion, with the end goal of getting reviewers and test writers/promoters to use the same authoritative list of criteria for evaluating a test.