- Tests must work without access to the internet. That is, tests cannot rely on external services like `1.1.1.1`, `8.8.8.8`, `a.root-servers.net.`, etc. To this effect, each test runs into its own ephemeral network isolated from the internet and from the networks of other tests running concurrently.
- Test code must be decoupled from the API of any DNS implementation. That is, DNS implementation specific details (library/FFI calls, configuration files) must not appear in test code. To this end, interaction with DNS implementations is done at the network level using tools like `dig`, `delv` and `tshark`.
- It must be possible to switch the 'implementation under test' at runtime. In other words, one should not need to recompile the tests to switch the DNS implementation being tested. To this end, the `DNS_TEST_SUBJECT` environment variable is used to switch the DNS implementation that'll be tested.
-`DNS_TEST_SUBJECT`. This variable controls what the `dns_test::subject` function returns. The variable can contain one of these values:
-`unbound`
-`hickory $REPOSITORY`. where `$REPOSITORY` is a placeholder for git repository. Examples values for `$REPOSITORY`: `https://github.com/hickory-dns/hickory-dns`; `/home/user/git-repos/hickory-dns`. NOTE: when using a local repository, changes that have not been committed, regardless of whether they are staged or not, will **not** be included in the `hickory-dns` build.
-`DNS_TEST_VERBOSE_DOCKER_BUILD`. Setting this variable prints the output of the `docker build` invocations that the framework does to the console. This is useful to verify that image caching is working; for example if you set `DNS_TEST_SUBJECT` to a local `hickory-dns` repository then consecutively running the `explore` example and/or `conformance-tests` test suite **must** not rebuild `hickory-dns` provided that you have not *committed* any new change to the local repository.
At the RFC module level there's a special module called `scenarios`. This module contains tests that map to representative use cases of the parent functionality. Each use case can be tested in successful and failure scenarios, hence the name. The organization within this module will be ad hoc.
When adding a new test to the test suite, it must pass with the `unbound` implementation, which is treated as the *reference* implementation. The CI workflow will check that *all* tests, including the ones that have the `#[ignore]` attribute, pass with the `unbound` implementation.
New tests that don't pass with the `hickory-dns` implementation must be marked as `#[ignore]`-d. The CI workflow will check that non-`#[ignore]`-d tests pass with the `hickory-dns` implementation. Additionally, the CI workflow will check that all `#[ignore]`-d tests *fail* with the `hickory-dns` implementation; this is to ensure that fixed tests get un-`#[ignore]`-d.