package tezt
Install
Dune Dependency
Authors
Maintainers
Sources
md5=643f48378265ae18b7aab4675a1e836e
sha512=f399d4d8b935fc3dcb4354596c2b7fa26a019f99c83cb4c1d48b0e26c5bc6da13b2cad9792dd50420a88b777d3b8a1d67c7ddd0457322e65d311b859be03879f
CHANGES.md.html
Changelog
Version 4.0.0
Breaking Changes
The
Cli
module has changed a lot: instead of being all together in a record with mutable fields, command-line argument values are now separate values and are grouped in thematically-related submodules. Types such aslog_level
have also moved to those submodules. Those values are usually not used in user-code however, so this is only a breaking change for very niche cases. FunctionsCli.get_*
are unchanged, although there is now a better way to define custom command-line arguments.true
andfalse
are now invalid tags.The
JSON
module is now in its own librarytezt.json
. This allows one to use theJSON
module of Tezt without linking with Tezt, and in particular without Tezt parsing the command-line.tezt.core
depends ontezt.json
but does not include theJSON
module itself. If your code accessedTezt_core.JSON
, you must replaceTezt_core.JSON
withJSON
and you should addtezt.json
as a dependency (especially if yourdune-project
contains(implicit_transitive_deps false)
).tezt
andtezt.js
, however, still include theJSON
module in their interface. If your code only accessedTezt.JSON
orTezt_js.JSON
, you have nothing to do.Using both
--loop-count N
and--loop
now always limits the number of iterations to N. Before,--loop-count N --loop
would result in an unbounded number of iterations, while--loop --loop-count N
limited the number of iterations to N.
New Features
--help
is now much prettier.Tezt now uses Clap instead of
Stdlib.Arg
. This means that you can now define custom command-line arguments directly usingClap
instead of usingCli.get_*
functions. Such custom command-line arguments do not need to be prefixed with-a
and will appear in--help
.Tests can now be selected using generic predicates from the command-line. See SELECTING TESTS in
--help
(search for "TSL").Added new module
Version
with valueVersion.full
which contains the version number of Tezt.Added
--version
to print the version number of Tezt on standard output and exit.Added
--cleanup-timeout
to be able to specify the delay between SIGTERM and SIGKILL when cleaning up external processes after a test.Added
Test.current_test_file
.Added
Test.current_test_title
.Added
Test.current_test_tags
andTest.current_test_has_tag
.Added
Test.declare_clean_up_function
.
Bug Fixes
Fixed a bug which caused tests to be started after the global timeout was reached if
--keep-going
was specified.Fixed a bug in
Process
where terminating a process running on a remote runner would not behave the same as terminating a local process. More precisely, the remote process used to only receive SIGKILL. Now it receives SIGTERM then SIGKILL after the timeout delay, which is the behavior for local processes.Fixed a bug which caused "bad file descriptor" (
EBADF
) errors when logging inat_exit
handlers if thoseat_exit
were declared at the toplevel of modules linked before theLog
module. A rare occurrence since one does not have access to theLog
module in such handlers without some reference hacks.
Version 3.1.1
Bug Fixes
Fixed a bug causing records to not be taken into account when balancing jobs for
--job
.
Version 3.1.0
Breaking Changes
On OCaml 4.12.1 or earlier, the new
?seed
argument can break calls toTest.register
which are written using the formTest.register ~__FILE__ ~title ~tags @@ fun () -> ...
. To fix this, either upgrade to OCaml 4.13.0 or later, add?seed: None
, or replace@@
with parentheses.Internal type
Test.SCHEDULER.response
changed. This change is very unlikely to impact you unless you wrote your own backend.--match
now applies to both test title and test filename.Removed the
--starting-port
command-line argument. You can use-a starting-port=
instead.Selecting a
--job
that is empty now results in an error. This means that if you have more jobs than tests in your CI, your CI will fail.Skipping all tests with
--skip
or--only
now results in an error. This makes those command-line arguments behave like other filters.Tests are now registered with the full value passed as
~__FILE__
toTest.register
, instead of just the basename.
New Features
Added
?seed
toTest.register
and the--seed
command-line parameter to help control the determinism of randomness.Added
--log-worker-ids
which adds worker ids to logs when--job-count
is more than 1.Added
--not-match
to deselect tests with a title that matches a regular expression.Added
--not-file
to deselect tests with a given filename.--time
can now be passed along with--from-record <record.json> --list
to pretty-print timings from previous executions.--file FILE
now selects all tests registered from a source file that ends withFILE
.Added
--not-file FILE
to deselect tests registered from a source file that ends withFILE
.In
Cli
: added_opt
variants for functions that retrieve custom arguments.
Bug Fixes
Fixed a bug where the log file does not contain logs from tests in the presence of
--job-count
. In the presence of--job-count N
and--log-file FILE
, test results are now written toFILE
, where as the logs from tests are written in a separate file per worker namedBASENAME-WORKER_ID.EXT
, as detailed in--help
.Fixed a bug where a warning is not emitted when the argument to
--not-title
does not correspond to any known files.Fixed a bug where the list of selectors printed with an empty test selection would not contain
--not-title
.
Version 3.0.0
Breaking Changes
Argument
~output_file
ofRegression.register
is now optional and has been renamed into?file
. Previously,~output_file
was automatically prefixed by the value given to--regression-dir
. Now, files are put in a directory namedexpected
next to the test itself, and the default filename is a sanitized version of the test title.Removed the
--regression-dir
command-line parameter.
New Features
The
tezt
Dune library was split intotezt.core
andtezt
. Librarytezt.core
contains the parts that can run both on UNIX and using Node.js. It does not containTest.run
though so it cannot be used on its own. Librarytezt
is the UNIX backend. It also includestezt.core
, so this is not a breaking change.New library
tezt.js
is a partial backend for Node.js. Compared totezt
, it does not contain modulesProcess
,Temp
andRunner
.Added
--resume
(short-hand:-r
) and--resume-file
, which allow to resume a previous run. For instance, if the previous run stopped after a failed test, and if that previous run was itself ran with--resume
or--resume-file
, resume from this run to avoid running the tests that already succeeded again.Added
--match
(short-hand:-m
) to select tests with a title that match a regular expression.Added a
?timeout
argument toProcess.terminate
andProcess.clean_up
. These functions sendSIGTERM
to processes. If this timeout is reached without the process actually terminating,Process.terminate
also sendsSIGKILL
.Added module
Diff
which allows to compare two sequences. It is used internally by theRegression
module to compare test outputs between runs.Added module
Main
with functionMain.run
, which is the same asTest.run
. In practice you can still useTest.run
, which now delegates toMain.run
. ButTest.run
is not available intezt.core
, only intezt
andtezt.js
. This is not a breaking change since existing applications would usetezt
, nottezt.core
directly.Added
Base.project_root
which is the path to the root of the current dune project according to environment variableDUNE_SOURCEROOT
, falling back to the current directory if unavailable.Added
Base.span
to split a list in two sublists given a predicate.Added
Base.rexf
, a short-hand forrex @@ sf
.Added
Check.file_exists
,Check.file_not_exists
,Check.directory_exists
andCheck.directory_not_exists
.Added
Check.is_true
andCheck.is_false
.Added
Check.json
andCheck.json_u
, type definitions forJSON.t
andJSON.u
respectively.Added support for
int32
in theJSON
module.Added
JSON.merge_objects
,JSON.filter_map_object
andJSON.filter_object
.Added
JSON.equal
andJSON.equal_u
, equality predicates onJSON.t
andJSON.u
respectively.Added
Temp.set_pid
. TheTemp
module no longer callsUnix.getpid
directly, this is done byTest.run
instead, which callsTemp.set_pid
. This allows backends whereUnix.getpid
cannot be used, like JS.Added
Test.run_with_scheduler
, and theTest.SCHEDULER
signature. AddedTest.get_test_by_title
andTest.run_one
. Those functions are used internally to provide different backends (UNIX / JS), but users usually would not have any use for them.Some fields in records obtained with
--record
became optional, with default values. New records may not be compatible with old versions of Tezt.Added module
Process_hooks
which contains the type definition forProcess.hooks
.
Bug Fixes
Fixed some cases where regression tests would crash instead of failing gracefully for some system errors.
Temp.clean_up
no longer follows symbolic links, it just deletes the links themselves.Fixed
--junit
which no longer worked when using-j
.Fixed
--loop-count 0
which did not actually run zero loops.Fixed a bug where Tezt would exit prematurely without running tests when using
--on-unknown-regression-files delete
or--on-unknown-regression-files fail
.Fixed a redundancy in the warning message when removing a non-empty temporary directories registered with the
Temp
module.Fixed some whitespace issues in
--help
.
Older Versions
Tezt 2.0.0 was the first announced release and as such it had no changelog. Tezt 1.0.0 was released in opam but was not announced.