package octez-libs
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=ddfb5076eeb0b32ac21c1eed44e8fc86a6743ef18ab23fff02d36e365bb73d61
sha512=d22a827df5146e0aa274df48bc2150b098177ff7e5eab52c6109e867eb0a1f0ec63e6bfbb0e3645a6c2112de3877c91a17df32ccbff301891ce4ba630c997a65
doc/octez-libs.tezt-wrapper/Tezt_wrapper/index.html
Module Tezt_wrapper
Source
Wrapper around Tezt to declare test dependencies.
include module type of Tezt
Tezt (pronounced "tezty", as in "tasty" with a "z") is a test framework for OCaml. It is well suited for writing and executing unit, integration and regression tests. It integrates well with continuous integration (CI).
Tezt provides a function (Test.register
) to register tests. Tests have a title, some tags, and an implementation (a function). Titles and tags can be used on the command-line to select which tests to run. The name of the file in which the test was registered can also be used to do so. The implementation of the test is ran if the test is selected. If this implementation does not raise an exception (for instance by calling Test.fail
), the test is considered to be successful. Along with modules Base
, which is supposed to be open
ed a la Pervasives
and which provides a few useful generic functions, and Check
, which provides ways to perform assertions with nice error messages, the result is a framework that is suitable for unit tests.
Tests can be ran in a CI, optionally producing JUnit reports. Tezt can automatically compute a partition of the set of tests where subsets are balanced to take roughly the same amount of time to run. The intent is that each of those subset can be one CI job, resulting in automatic balanced parallelisation.
Specific features supporting integration tests include:
- running external processes;
- invoking distant runners through SSH;
- decoding JSON values (e.g. to test REST APIs);
- cleaning up temporary files automatically.
Specific features supporting regression tests include:
- capturing output from local or external processes;
- applying regular-expression-based substitutions to those outputs;
- comparing with previous captured outputs.
Tezt provides a flexible user interface:
- using colors in logs (e.g. to distinguish external processes or to make error messages easy to see);
- adjusting automatically the verbosity of logs around errors;
- making it easy to copy-paste the invoked commands to reproduce errors;
- giving extensive information to precisely locate errors;
- supporting flaky (randomly failing) tests, by running them repeatedly.
To get started, register tests with Test.register
, then call the main function Test.run
. Tests are run from the command-line, for instance with dune runtest
or dune exec
. The latter gives access to a large list of command-line options to select which tests to run and how. Execute your program with --help
to get the list. See also the Tezt mini-tutorial.
Backends
Tezt executables can be compiled to bytecode, native code, or JavaScript. This is reflected by three Dune libraries:
- code which is common to all backends is provided by
tezt.core
; - the bytecode and native code backends are provided by
tezt
; - the JavaScript backend is provided by
tezt.js
.
Function Test.run
is only available in tezt
and tezt.js
. In other words, to actually run the tests you have to decide whether to use the JavaScript backend or not.
The difference between tezt
and tezt.js
is that tezt.js
does not provide:
- the
Process
module; - the
Temp
module; - the
Runner
module. So the JavaScript backend is not well suited for integration tests and regression tests. But it does support unit tests.
If you want to run your tests on multiple backends you have to write two executables: one linked with tezt
and one linked with tezt.js
. To share the code of your tests, you can write your calls to Test.register
in a library that depends on tezt.core
. Here is an example of dune
files:
; Dune file for the library that calls [Test.register].
(library
(name tezts)
(libraries tezt.core)
(library_flags (:standard -linkall))
(flags (:standard) -open Tezt_core -open Tezt_core.Base))
; Dune file for the executable used to run tests in native or bytecode mode.
(executable (name main) (libraries tezts tezt))
; Dune file for the executable used to run tests using nodejs.
(executable (name main_js) (modes js) (libraries tezts tezt.js))
Modules
Support for running promises in the background.
Base primitives useful in writing tests.
Support for expressing assertions.
Command-line interface options.
Compute the difference between two sequences of items.
module JSON = JSON
JSON handling (encoding/decoding, accessors, error management, etc).
Functions for logging messages.
Managing external processes.
Process hooks, in particular for use with regression tests.
Runner specifications for processes spawned remotely using SSH.
Error modes.
Ignore
: do not warn, do not fail.Warn
: warn, but do not fail.Fail
: output an error and fail the test.
What to do if a test uses something without having it declared in its ~uses
.
Recommended setting for tests that are intended to run in the CI is Fail
, which is the default.
What to do if a test declares something in its ~uses
and does not use it.
Recommended setting for tests that are intended to run in the CI is Warn
, which is the default.
Using Fail
is possible but:
- if a test is non-deterministic and only sometimes uses something, one needs to make sure
Uses.path
is always called anyway; - if a test actually uses something but not through
Uses.path
, the wrapper will not be able to detect it, so one will have to callUses.path
just to suppress the error.
What to do if a test declares a ~uses
with a path that does not exist.
Recommended setting is Fail
, which is the default.