package alcotest
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=b1aaccfb2d651c902592c04953e2619169c91f797cf4f04a7dda2cab09b93ec1
sha512=8a13d5d4c8c77f115903e6b8e58160c6e6ec27870440bd38a674e9406f57f1eff299e65f006fd77728015d1a8f0ae30a714fe47e035824950a71ebfdff2cf3c9
Description
Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.
Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.
Published: 16 Apr 2021
README
Alcotest is a lightweight and colourful test framework.
Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE
module type, a check
function to assert test predicates and a run
function to perform a list of unit -> unit
test callbacks.
Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.
For information on contributing to Alcotest, see CONTRIBUTING.md.
Examples
A simple example (taken from examples/simple.ml
):
Generated by the following test suite specification:
(* Build with `ocamlbuild -pkg alcotest simple.byte` *)
(* A module with functions to test *)
module To_test = struct
let lowercase = String.lowercase_ascii
let capitalize = String.capitalize_ascii
let str_concat = String.concat ""
let list_concat = List.append
end
(* The tests *)
let test_lowercase () =
Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")
let test_capitalize () =
Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")
let test_str_concat () =
Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])
let test_list_concat () =
Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])
(* Run it *)
let () =
let open Alcotest in
run "Utils" [
"string-case", [
test_case "Lower case" `Quick test_lowercase;
test_case "Capitalization" `Quick test_capitalize;
];
"string-concat", [ test_case "String mashing" `Quick test_str_concat ];
"list-concat", [ test_case "List mashing" `Slow test_list_concat ];
]
The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help
to see the runtime options.
Here's an example of a of failing test suite:
By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors
to print all error messages.
Selecting tests to execute
You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9
):
$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP] string-case 0 Lower case.
[SKIP] string-case 1 Capitalization.
[OK] string-concat 0 String mashing.
[OK] list-concat 0 List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.
$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP] string-case 0 Lower case.
[OK] string-case 1 Capitalization.
[SKIP] string-concat 0 String mashing.
[SKIP] list-concat 0 List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.
Note that you cannot filter by test case name (i.e. Lower case
or Capitalization
), you must filter by test name & number instead.
See the examples folder for more examples.
Quick and Slow tests
In general you should use `Quick
tests: tests that are ran on any invocations of the test suite. You should only use `Slow
tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q
flag on the command line, e.g.:
$ ./test.exe -q # run only the quick tests
$ ./test.exe # run quick and slow tests
Passing custom options to the tests
In most cases, the base tests are unit -> unit
functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit
, where 'a
is the type of the extra parameter.
In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a
(note: do not use positional arguments, only optional arguments are supported).
For instance:
let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42
let int =
let doc = "What is your prefered number?" in
Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")
let () =
Alcotest.run_with_args "foo" int [
"all", ["nice", `Quick, test_nice]
]
Will generate test.exe
such that:
$ test.exe test
test.exe: required option -n is missing
$ test.exe test -n 42
Testing foo.
[OK] all 0 int.
Lwt
Alcotest provides an Alcotest_lwt
module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit
, you provide one with the type unit -> unit Lwt.t
and alcotest-lwt calls Lwt_main.run
for you.
However, there are a couple of extra features:
If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).
You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.
For instance:
let free () = print_endline "freeing all resources"; Lwt.return ()
let test_lwt switch () =
Lwt_switch.add_hook (Some switch) free;
Lwt.async (fun () -> failwith "All is broken");
Lwt_unix.sleep 10.
let () =
Lwt_main.run @@ Alcotest_lwt.run "foo" [
"all", [
Alcotest_lwt.test_case "one" `Quick test_lwt
]
]
Will generate:
$ test.exe
Testing foo.
[ERROR] all 0 one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken
Comparison with other testing frameworks
The README is pretty clear about that:
Alcotest is the only testing framework using colors!
More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:
Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;
Alcotest uses combinators to define pretty-printers and comparators between the things to test.
Other nice tools doing different kind of testing also exist:
qcheck qcheck does random generation and property testing (e.g. Quick Check)
crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.
ppx_inline_tests
allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.
Dependencies (9)
Dev Dependencies (1)
-
cmdliner
with-test & < "1.1.0"
- ahrocksdb
-
albatross
>= "1.5.0"
-
alcotest-async
< "1.0.0" | = "1.4.0"
-
alcotest-lwt
< "1.0.0" | = "1.4.0"
-
alcotest-mirage
= "1.4.0"
- alg_structs_qcheck
- ambient-context
- ambient-context-eio
- ambient-context-lwt
-
angstrom
>= "0.7.0"
-
ansi
>= "0.6.0"
-
anycache
>= "0.7.4"
- anycache-async
- anycache-lwt
-
archetype
>= "1.4.2"
- archi
-
arp
!= "2.3.1"
-
arp-mirage
< "2.0.0"
- arrakis
- art
-
asak
>= "0.2"
-
asli
>= "0.2.0"
-
asn1-combinators
>= "0.2.2"
-
atd
>= "2.3.3"
-
atdgen
>= "2.10.0"
- atdpy
- atdts
- base32
-
base64
>= "2.1.2" & < "3.2.0" | >= "3.4.0"
- bastet
- bastet_async
- bastet_lwt
- bech32
-
bechamel
>= "0.5.0"
- bigarray-overlap
- bigstringaf
- bitlib
- blake2
- bloomf
-
bls12-381
< "0.4.1" | >= "3.0.0" & < "18.0"
- bls12-381-hash
-
bls12-381-js
>= "0.4.2"
-
bls12-381-js-gen
>= "0.4.2"
- bls12-381-legacy
- bls12-381-signature
- bls12-381-unix
- blurhash
- builder-web
- bulletml
- bytebuffer
- ca-certs
- ca-certs-nss
- cactus
- caldav
-
calendar
>= "3.0.0"
- callipyge
- camlix
- camlkit
- camlkit-base
-
capnp-rpc
< "1.2.3"
-
capnp-rpc-lwt
< "0.3"
-
capnp-rpc-mirage
>= "0.9.0"
-
capnp-rpc-unix
>= "0.9.0" & < "1.2.3"
- carray
- carton
- carton-git
-
carton-lwt
>= "0.4.1"
- cborl
-
ccss
>= "1.6"
- cf-lwt
- chacha
- channel
- charrua-client
- charrua-client-lwt
-
charrua-client-mirage
< "0.11.0"
-
charrua-server
>= "1.4.1"
-
checkseum
>= "0.0.3"
- cid
- clarity-lang
- class_group_vdf
-
cohttp
>= "0.17.0"
- cohttp-curl-async
- cohttp-curl-lwt
-
cohttp-eio
>= "6.0.0~beta2"
-
colombe
>= "0.2.0"
- color
- conan
- conan-cli
- conan-database
- conan-lwt
- conan-unix
-
conduit
= "3.0.0"
-
conex
< "0.10.0"
- conex-mirage-crypto
- conex-nocrypto
- conformist
- cookie
-
cow
>= "2.2.0"
- css
- css-parser
-
cstruct
>= "3.3.0"
- cstruct-sexp
- ctypes-zarith
- cuid
- curly
-
current
>= "0.4"
- current-albatross-deployer
-
current_git
>= "0.6.4"
- current_incr
- cwe_checker
- data-encoding
-
datakit
>= "0.12.0"
-
datakit-bridge-github
>= "0.12.0"
- datakit-ci
-
datakit-client-git
>= "0.12.0"
-
decompress
>= "0.8" & < "1.5.3"
- depyt
-
digestif
>= "0.8.1"
- dirsp-exchange-kbb2017
- dirsp-proscript-mirage
- dirsp-ps2ocaml
-
dispatch
>= "0.4.1"
- dkim
- dkim-bin
- dkim-mirage
- dkml-install
- dkml-install-installer
- dkml-install-runner
- dkml-package-console
-
dns
>= "4.0.0"
- dns-cli
-
dns-client
>= "4.6.0"
-
dns-forward
< "0.9.0"
- dns-forward-lwt-unix
- dns-resolver
- dns-server
- dns-tsig
- dnssd
- dnssec
-
docfd
>= "2.2.0"
-
dog
< "0.2.1"
- domain-name
-
dot-merlin-reader
>= "5.3~5.3preview"
- dream
- dream-pure
- duff
-
dune-release
>= "1.0.0"
-
duration
>= "0.1.1"
-
eio
< "0.12"
-
eio_linux
< "0.12"
-
eio_windows
< "0.12"
- emile
- encore
-
eqaf
>= "0.5"
- equinoxe
- equinoxe-cohttp
- equinoxe-hlc
- eris
- eris-lwt
- ezgzip
-
ezjsonm
>= "0.4.2" & < "1.3.0"
- ezjsonm-lwt
- FPauth
- FPauth-core
- FPauth-responses
- FPauth-strategies
-
faraday
!= "0.2.0"
- farfadet
-
fat-filesystem
>= "0.12.0"
- ff
- ff-pbt
- flex-array
- fsevents-lwt
-
functoria
>= "2.2.0"
-
functoria-runtime
>= "2.2.0" & < "3.0.1" | = "3.1.2"
- geojson
-
geoml
>= "0.1.1"
-
git
= "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0"
- git-cohttp
- git-cohttp-mirage
- git-cohttp-unix
- git-mirage
- git-split
-
git-unix
>= "1.10.0" & != "2.1.0"
- git_split
- gitlab-unix
- glicko2
-
gmap
>= "0.3.0"
- gobba
- gpt
- graphql
- graphql-async
-
graphql-cohttp
>= "0.13.0"
- graphql-lwt
-
graphql_parser
!= "0.11.0"
-
graphql_ppx
>= "0.7.1"
- h1
- h1_parser
- h2
- hacl
-
hacl-star
>= "0.6.0" & < "0.7.2"
- hacl_func
-
hacl_x25519
>= "0.2.0"
- highlexer
- hkdf
- hockmd
- html_of_jsx
- http
-
http-multipart-formdata
< "2.0.0"
-
httpaf
>= "0.2.0"
- httpun
- httpun-ws
- hvsock
-
icalendar
>= "0.1.4"
-
imagelib
>= "20200929"
- index
-
inferno
>= "20220603"
- influxdb-async
- influxdb-lwt
-
inquire
< "0.2.0"
- interval-map
- iomux
-
irmin
< "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.0.0" | >= "2.0.0" & != "2.3.0"
-
irmin-bench
>= "2.7.0"
-
irmin-chunk
< "1.3.0" | >= "2.3.0"
- irmin-cli
- irmin-containers
-
irmin-fs
< "1.3.0" | >= "2.3.0"
-
irmin-git
< "2.0.0" | >= "2.3.0"
-
irmin-graphql
>= "2.3.0"
-
irmin-http
< "2.0.0"
-
irmin-mem
< "1.3.0" | >= "2.3.0"
-
irmin-pack
>= "2.4.0" & != "2.6.1"
- irmin-pack-tools
-
irmin-test
>= "2.2.0" & < "3.0.0"
- irmin-tezos
- irmin-tezos-utils
-
irmin-unix
>= "1.0.0" & < "1.3.3" | >= "2.4.0" & != "2.6.1"
-
irmin-watcher
!= "0.3.0"
- jekyll-format
- jerboa
- jitsu
- jose
-
json-data-encoding
>= "0.9"
- json_decoder
- jsonxt
- junit_alcotest
- jwto
- kdf
-
ke
>= "0.2"
- kkmarkdown
- lambda-runtime
- lambda_streams
- lambda_streams_async
-
lambdapi
>= "2.0.0"
-
lambdoc
>= "1.0-beta4"
-
ledgerwallet-tezos
>= "0.2.1" & < "0.4.0"
- letters
-
lmdb
>= "1.0"
- logical
-
logtk
>= "1.6"
- lp
- lp-glpk
- lp-glpk-js
- lp-gurobi
- lru
- lt-code
- luv
-
mbr-format
>= "1.0.0"
-
mdx
>= "1.6.0"
- mec
-
mechaml
= "1.0.0" | >= "1.2.1"
-
merge-queues
>= "0.2.0"
-
merge-ropes
>= "0.2.0"
-
merlin
>= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
-
merlin-lib
>= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
- metrics
- middleware
- mimic
-
minicaml
= "0.3.1" | >= "0.4"
-
mirage
>= "4.0.0~beta1"
- mirage-block-partition
-
mirage-block-ramdisk
= "0.3"
-
mirage-channel
>= "4.0.0"
-
mirage-channel-lwt
< "3.1.0"
-
mirage-crypto-ec
!= "0.9.2"
-
mirage-flow
>= "1.0.2" & < "1.2.0"
-
mirage-flow-unix
!= "1.3.0" & < "1.5.0" | = "2.0.0" | >= "3.0.0"
- mirage-fs-mem
-
mirage-fs-unix
>= "1.2.0" & < "1.4.1"
-
mirage-kv
>= "2.0.0"
- mirage-kv-mem
-
mirage-kv-unix
>= "3.0.0"
-
mirage-logs
>= "0.3.0"
- mirage-nat
-
mirage-net-unix
>= "2.3.0"
-
mirage-runtime
>= "4.0.0~beta1" & < "4.5.0"
- mirage-tc
- mjson
-
mmdb
< "0.3.0"
- mnd
- monocypher
-
mrmime
>= "0.2.0"
- mrt-format
-
msgpck
>= "1.6"
-
mssql
>= "2.0.3"
- multibase
- multihash
- multihash-digestif
- multipart-form-data
- multipart_form
- multipart_form-eio
- multipart_form-lwt
- named-pipe
- nanoid
-
nbd
>= "4.0.3"
- nbd-tool
- nloge
- nocoiner
- non_empty_list
-
OCADml
>= "0.6.0"
- obatcher
-
ocaml-index
>= "1.1"
-
ocaml-r
>= "0.4.0"
-
ocaml-version
>= "3.1.0"
-
ocamlformat
>= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
- ocamlformat-lib
-
ocamlformat-rpc
< "removed"
- ocamline
-
ocluster
< "0.3.0"
-
odoc
>= "1.4.0" & < "2.1.0"
- ohex
- oidc
- opam-0install
-
opam-0install-cudf
>= "0.5.0"
- opam-compiler
-
opam-file-format
>= "2.1.1"
-
opentelemetry
>= "0.6"
-
opentelemetry-client-cohttp-lwt
>= "0.6"
-
opentelemetry-client-ocurl
>= "0.6"
-
opentelemetry-cohttp-lwt
>= "0.6"
-
opentelemetry-lwt
>= "0.6"
-
opium
>= "0.15.0"
- opium-graphql
- opium-testing
- opium_kernel
- orewa
- orgeat
- ortac-core
-
osnap
< "0.3.0"
- osx-acl
- osx-attr
- osx-cf
- osx-fsevents
- osx-membership
- osx-mount
- osx-xattr
- otoggl
-
owl
>= "0.6.0" & != "0.9.0" & != "1.0.0"
-
owl-base
< "0.5.0"
-
owl-ode
>= "0.1.0" & != "0.2.0"
- owl-symbolic
- passmaker
- patch
- pbkdf
-
pecu
>= "0.2"
- pf-qubes
-
pg_query
>= "0.9.6"
-
pgx
>= "1.0"
-
pgx_unix
>= "1.0"
- pgx_value_core
- pgx_value_ptime
- phylogenetics
- piaf
- polyglot
- polynomial
-
ppx_blob
>= "0.3.0"
- ppx_deriving_cmdliner
- ppx_deriving_qcheck
- ppx_deriving_rpc
- ppx_deriving_yaml
-
ppx_graphql
>= "0.2.0"
- ppx_inline_alcotest
- ppx_parser
-
ppx_protocol_conv
>= "5.0.0"
-
ppx_protocol_conv_json
>= "5.0.0"
-
ppx_protocol_conv_jsonm
>= "5.0.0"
-
ppx_protocol_conv_msgpack
>= "5.0.0"
-
ppx_protocol_conv_xml_light
>= "5.0.0"
- ppx_protocol_conv_xmlm
-
ppx_protocol_conv_yaml
>= "5.0.0"
- ppx_repr
- ppx_subliner
- ppx_units
-
ppx_yojson
>= "1.1.0"
- pratter
-
prbnmcn-ucb1
>= "0.0.2"
- prc
- preface
- pretty_expressive
- prettym
- proc-smaps
-
producer
< "0.2.0"
- progress
- prom
-
prometheus
< "1.2"
- prometheus-app
- protocell
-
protocol-9p
>= "0.3" & < "0.11.0" | >= "0.11.2"
- protocol-9p-unix
- psq
- pyast
-
qcheck
>= "0.18"
- qcheck-alcotest
-
qcheck-core
>= "0.18"
- quickjs
- radis
- randii
- reason-standard
- red-black-tree
-
reparse
>= "2.0.0" & < "3.0.0"
-
reparse-unix
< "2.1.0"
- resp
-
resp-unix
>= "0.10.0"
-
rfc1951
< "1.0.0"
-
routes
< "2.0.0"
-
rpc
>= "7.1.0"
-
rpclib
>= "7.1.0"
- rpclib-async
-
rpclib-lwt
>= "7.1.0"
-
rpmfile
< "0.3.0"
- rpmfile-eio
- rpmfile-unix
- rubytt
-
SZXX
>= "4.0.0"
- salsa20
- salsa20-core
-
sanddb
>= "0.2"
-
scaml
>= "1.5.0"
- scrypt-kdf
-
secp256k1
>= "0.4.1"
- secp256k1-internal
-
semver
>= "0.2.1"
- sendmail
- sendmail-lwt
- sendmail-miou-unix
- sendmail-mirage
- sendmsg
- server-reason-react
- session-cookie
- session-cookie-async
- session-cookie-lwt
- sherlodoc
-
sihl
< "0.2.0"
- sihl-type
- slug
- smol
- smol-helpers
- sodium-fmt
- solidity-alcotest
- spdx_licenses
- spectrum
-
spin
>= "0.7.0"
- squirrel
- ssh-agent
-
ssl
>= "0.6.0"
- stramon-lib
- styled-ppx
- syslog-rfc5424
-
tcpip
>= "2.4.2" & < "3.4.2" | >= "6.2.0" & < "7.0.0"
-
tdigest
< "2.1.0"
- term-indexing
- term-tools
- terminal
-
terminal_size
>= "0.1.1"
- terminus
- terminus-cohttp
- terminus-hlc
- terml
-
textmate-language
>= "0.3.0"
- textrazor
-
tezos-base-test-helpers
< "13.0"
- tezos-bls12-381-polynomial
-
tezos-client-base
< "12.0"
-
tezos-crypto
>= "8.0" & < "9.0"
- tezos-lmdb
-
tezos-plompiler
= "0.1.3"
-
tezos-plonk
= "0.1.3"
-
tezos-signer-backends
>= "8.0" & < "13.0"
-
tezos-stdlib
>= "8.0" & < "12.0"
-
tezos-test-helpers
< "12.0"
- tftp
- timedesc
- timere
- timmy
- timmy-jsoo
- timmy-lwt
- timmy-unix
-
tls
>= "0.12.0"
- toc
- topojson
- topojsone
- transept
- twostep
- type_eq
- type_id
- typebeat
-
typeid
>= "1.0.1"
-
tyre
>= "0.4"
-
tyxml
>= "4.0.0"
- tyxml-jsx
-
tyxml-ppx
>= "4.3.0"
- tyxml-syntax
- uecc
- ulid
- universal-portal
- unix-dirent
-
unix-errno
>= "0.3.0"
-
unix-fcntl
>= "0.3.0"
- unix-sys-resource
- unix-sys-stat
- unix-time
- unstrctrd
-
uring
< "0.4"
- user-agent-parser
- uspf
- uspf-lwt
- uspf-mirage
- uspf-unix
-
utop
>= "2.13.0"
- validate
- validator
- vercel
- vpnkit
-
wayland
>= "2.0"
- wcwidth
- websocketaf
-
x509
>= "0.7.0"
-
xapi-rrd
>= "1.8.2"
- xapi-stdext-date
- xapi-stdext-encodings
-
xapi-stdext-std
>= "4.16.0"
-
yaml
< "3.2.0"
- yaml-sexp
- yocaml
-
yocaml_syndication
>= "2.0.0"
-
yocaml_yaml
< "2.0.0"
-
yojson
>= "1.6.0"
- yojson-five
-
yuscii
>= "0.3.0"
-
yuujinchou
= "1.0.0"
- zar
-
zed
>= "3.2.2"
-
zlist
< "0.4.0"
Conflicts
None