package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (41)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-mirage-1.2.1.tbz
sha256=4cc037abb03d9685003d7313c9209f0481dca9fbcf3f7f0bc4802b1d75e3cc6c
sha512=31b2b460042e0cebb21a143dbd3bda81858ff58ddd1b8a1b91dcbb6547284eaab5d33c3d03da6cbf11c99eb046cd40438097ac37cf8cd1bacfc0eed54111c1b4

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 16 Jul 2020

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

For information on contributing to Alcotest, see CONTRIBUTING.md.

Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (9)

  1. uutf >= "1.0.0"
  2. stdlib-shims
  3. re >= "1.7.2"
  4. uuidm
  5. cmdliner >= "1.0.3" & < "1.1.0"
  6. astring
  7. fmt >= "0.8.7"
  8. ocaml >= "4.03.0"
  9. dune >= "2.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0" | = "1.2.1"
  4. alcotest-lwt < "1.0.0" | = "1.2.1"
  5. alcotest-mirage = "1.2.1"
  6. alg_structs_qcheck
  7. ambient-context
  8. ambient-context-eio
  9. ambient-context-lwt
  10. angstrom >= "0.7.0"
  11. ansi >= "0.6.0"
  12. anycache >= "0.7.4"
  13. anycache-async
  14. anycache-lwt
  15. archetype >= "1.4.2"
  16. archi
  17. arp
  18. arp-mirage
  19. arrakis
  20. art
  21. asak >= "0.2"
  22. asli >= "0.2.0"
  23. asn1-combinators >= "0.2.2"
  24. atd >= "2.3.3"
  25. atdgen >= "2.10.0"
  26. atdpy
  27. atdts
  28. base32
  29. base64 >= "2.1.2" & < "3.2.0" | >= "3.4.0"
  30. bastet
  31. bastet_async
  32. bastet_lwt
  33. bech32
  34. bechamel >= "0.5.0"
  35. bigarray-overlap
  36. bigstringaf
  37. bitlib
  38. blake2
  39. bloomf
  40. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  41. bls12-381-hash
  42. bls12-381-js >= "0.4.2"
  43. bls12-381-js-gen >= "0.4.2"
  44. bls12-381-legacy
  45. bls12-381-signature
  46. bls12-381-unix
  47. blurhash
  48. builder-web
  49. bulletml
  50. bytebuffer
  51. ca-certs
  52. ca-certs-nss
  53. cactus
  54. caldav
  55. calendar >= "3.0.0"
  56. callipyge
  57. camlix
  58. camlkit
  59. camlkit-base
  60. capnp-rpc < "1.2.3"
  61. capnp-rpc-lwt < "0.3"
  62. capnp-rpc-mirage >= "0.9.0"
  63. capnp-rpc-unix >= "0.9.0" & < "1.2.3"
  64. carray
  65. carton
  66. cborl
  67. ccss >= "1.6"
  68. cf-lwt
  69. chacha
  70. channel
  71. charrua-client
  72. charrua-client-lwt
  73. charrua-client-mirage < "0.11.0"
  74. checked_oint < "0.1.1"
  75. checkseum >= "0.0.3"
  76. cid
  77. clarity-lang
  78. class_group_vdf
  79. cohttp >= "0.17.0"
  80. cohttp-curl-async
  81. cohttp-curl-lwt
  82. cohttp-eio >= "6.0.0~beta2"
  83. colombe >= "0.2.0"
  84. color
  85. conan
  86. conan-cli
  87. conan-database
  88. conan-lwt
  89. conan-unix
  90. conduit = "3.0.0"
  91. conex < "0.10.0"
  92. conex-mirage-crypto
  93. conex-nocrypto
  94. cookie
  95. cow >= "2.2.0"
  96. css
  97. css-parser
  98. cstruct >= "3.3.0"
  99. cstruct-sexp
  100. ctypes-zarith
  101. cuid
  102. curly
  103. current >= "0.4"
  104. current_git >= "0.6.4"
  105. current_incr
  106. cwe_checker
  107. data-encoding
  108. datakit >= "0.12.0"
  109. datakit-bridge-github >= "0.12.0"
  110. datakit-ci
  111. datakit-client-git >= "0.12.0"
  112. decompress >= "0.8" & < "1.5.3"
  113. depyt
  114. digestif >= "0.8.1"
  115. dispatch >= "0.4.1"
  116. dkim
  117. dkim-bin
  118. dkim-mirage
  119. dns >= "4.0.0"
  120. dns-cli
  121. dns-client >= "4.6.0"
  122. dns-forward < "0.9.0"
  123. dns-forward-lwt-unix
  124. dns-resolver
  125. dns-server
  126. dns-tsig
  127. dnssd
  128. dnssec
  129. docfd >= "2.2.0"
  130. dog < "0.2.1"
  131. domain-name
  132. dream
  133. dream-pure
  134. duff
  135. dune-release >= "1.0.0"
  136. duration >= "0.1.1"
  137. emile
  138. encore
  139. eqaf >= "0.5"
  140. equinoxe
  141. equinoxe-cohttp
  142. equinoxe-hlc
  143. eris
  144. eris-lwt
  145. ezgzip
  146. ezjsonm >= "0.4.2" & < "1.3.0"
  147. ezjsonm-lwt
  148. FPauth
  149. FPauth-core
  150. FPauth-responses
  151. FPauth-strategies
  152. faraday != "0.2.0"
  153. farfadet
  154. fat-filesystem >= "0.12.0"
  155. ff
  156. ff-pbt
  157. fiat-p256
  158. flex-array
  159. fsevents-lwt
  160. functoria >= "2.2.0"
  161. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  162. geojson
  163. geoml >= "0.1.1"
  164. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0"
  165. git-mirage < "3.0.0"
  166. git-split
  167. git-unix >= "1.10.0" & != "2.1.0"
  168. git_split
  169. gitlab-unix
  170. glicko2
  171. gmap >= "0.3.0"
  172. gobba
  173. gpt
  174. graphql
  175. graphql-async
  176. graphql-cohttp >= "0.13.0"
  177. graphql-lwt
  178. graphql_parser != "0.11.0"
  179. graphql_ppx >= "0.7.1"
  180. h1
  181. h1_parser
  182. h2
  183. hacl
  184. hacl-star >= "0.6.0" & < "0.7.2"
  185. hacl_func
  186. hacl_x25519 >= "0.2.0"
  187. highlexer
  188. hkdf
  189. hockmd
  190. html_of_jsx
  191. http
  192. http-multipart-formdata < "2.0.0"
  193. httpaf >= "0.2.0"
  194. httpun
  195. httpun-ws
  196. hvsock
  197. icalendar >= "0.1.4"
  198. imagelib >= "20200929"
  199. index
  200. inferno >= "20220603"
  201. influxdb-async
  202. influxdb-lwt
  203. inquire < "0.2.0"
  204. interval-map
  205. iomux
  206. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.0.0" | >= "2.0.0" & != "2.3.0"
  207. irmin-bench >= "2.7.0"
  208. irmin-chunk < "1.3.0" | >= "2.3.0"
  209. irmin-cli
  210. irmin-containers
  211. irmin-fs < "1.3.0" | >= "2.3.0"
  212. irmin-git < "2.0.0" | >= "2.3.0"
  213. irmin-http < "2.0.0"
  214. irmin-mem < "1.3.0"
  215. irmin-pack >= "2.4.0" & != "2.6.1"
  216. irmin-pack-tools
  217. irmin-test >= "2.2.0" & < "3.0.0"
  218. irmin-tezos
  219. irmin-tezos-utils
  220. irmin-unix >= "1.0.0" & < "1.3.3" | >= "2.4.0" & != "2.6.1"
  221. irmin-watcher
  222. jekyll-format
  223. jerboa
  224. jitsu
  225. jose
  226. json-data-encoding >= "0.9"
  227. json_decoder
  228. jsonxt
  229. junit_alcotest
  230. jwto
  231. kdf
  232. ke >= "0.2"
  233. kkmarkdown
  234. lambda-runtime
  235. lambda_streams
  236. lambda_streams_async
  237. lambdapi >= "2.0.0"
  238. lambdoc >= "1.0-beta4"
  239. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  240. letters
  241. lmdb >= "1.0"
  242. logical
  243. logtk >= "1.6"
  244. lp
  245. lp-glpk
  246. lp-glpk-js
  247. lp-gurobi
  248. lru
  249. lt-code
  250. luv
  251. mbr-format >= "1.0.0"
  252. mdx >= "1.6.0"
  253. mec
  254. mechaml >= "1.0.0"
  255. merge-queues >= "0.2.0"
  256. merge-ropes >= "0.2.0"
  257. merlin >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  258. merlin-lib >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  259. metrics
  260. middleware
  261. minicaml = "0.3.1" | >= "0.4"
  262. mirage >= "4.0.0~beta1"
  263. mirage-block-partition
  264. mirage-block-ramdisk >= "0.3"
  265. mirage-channel >= "4.0.0"
  266. mirage-channel-lwt
  267. mirage-crypto-ec
  268. mirage-flow >= "1.0.2" & < "1.2.0"
  269. mirage-flow-unix
  270. mirage-fs-mem
  271. mirage-fs-unix >= "1.2.0"
  272. mirage-kv >= "2.0.0"
  273. mirage-kv-mem
  274. mirage-kv-unix
  275. mirage-logs >= "0.3.0"
  276. mirage-nat
  277. mirage-net-unix >= "2.3.0"
  278. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  279. mirage-tc
  280. mjson
  281. mmdb
  282. mnd
  283. monocypher
  284. mrmime >= "0.2.0"
  285. mrt-format
  286. msgpck >= "1.6"
  287. mssql >= "2.0.3"
  288. multibase
  289. multihash
  290. multihash-digestif
  291. multipart-form-data
  292. multipart_form
  293. multipart_form-eio
  294. multipart_form-lwt
  295. named-pipe
  296. nanoid
  297. nbd >= "4.0.3"
  298. nbd-tool
  299. nloge
  300. nocoiner
  301. non_empty_list
  302. OCADml >= "0.6.0"
  303. obatcher
  304. ocaml-index >= "1.1"
  305. ocaml-r >= "0.4.0"
  306. ocaml-version >= "3.1.0"
  307. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  308. ocamlformat-rpc < "removed"
  309. ocamline
  310. ocluster < "0.3.0"
  311. odoc >= "1.4.0" & < "2.1.0"
  312. ohex
  313. oidc
  314. opam-0install
  315. opam-0install-cudf >= "0.5.0"
  316. opam-compiler
  317. opam-file-format >= "2.1.1"
  318. opentelemetry >= "0.6"
  319. opentelemetry-client-cohttp-lwt >= "0.6"
  320. opentelemetry-client-ocurl >= "0.6"
  321. opentelemetry-cohttp-lwt >= "0.6"
  322. opentelemetry-lwt >= "0.6"
  323. opium >= "0.15.0"
  324. opium-graphql
  325. opium-testing
  326. opium_kernel
  327. orewa
  328. ortac-core
  329. osx-acl
  330. osx-attr
  331. osx-cf
  332. osx-fsevents
  333. osx-membership
  334. osx-mount
  335. osx-xattr
  336. otoggl
  337. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  338. owl-base < "0.5.0"
  339. owl-ode >= "0.1.0" & != "0.2.0"
  340. owl-symbolic
  341. passmaker
  342. patch
  343. pbkdf
  344. pecu >= "0.2"
  345. pf-qubes
  346. pg_query >= "0.9.6"
  347. pgx >= "1.0"
  348. pgx_unix >= "1.0"
  349. pgx_value_core
  350. pgx_value_ptime < "2.2"
  351. phylogenetics
  352. piaf
  353. polyglot
  354. polynomial
  355. ppx_blob >= "0.3.0"
  356. ppx_deriving_cmdliner
  357. ppx_deriving_rpc
  358. ppx_deriving_yaml
  359. ppx_graphql >= "0.2.0"
  360. ppx_inline_alcotest
  361. ppx_parser
  362. ppx_protocol_conv >= "5.0.0"
  363. ppx_protocol_conv_json >= "5.0.0"
  364. ppx_protocol_conv_jsonm >= "5.0.0"
  365. ppx_protocol_conv_msgpack >= "5.0.0"
  366. ppx_protocol_conv_xml_light >= "5.0.0"
  367. ppx_protocol_conv_xmlm
  368. ppx_protocol_conv_yaml >= "5.0.0"
  369. ppx_repr < "0.4.0"
  370. ppx_subliner
  371. ppx_units
  372. ppx_yojson >= "1.1.0"
  373. pratter
  374. prc
  375. preface
  376. pretty_expressive
  377. prettym
  378. proc-smaps
  379. producer < "0.2.0"
  380. progress < "0.2.0"
  381. prom
  382. prometheus < "1.2"
  383. prometheus-app
  384. protocell
  385. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  386. protocol-9p-unix
  387. psq
  388. qcheck >= "0.18"
  389. qcheck-alcotest
  390. qcheck-core >= "0.18"
  391. quickjs
  392. radis
  393. randii
  394. reason-standard
  395. red-black-tree
  396. reparse >= "2.0.0" & < "3.0.0"
  397. reparse-unix < "2.1.0"
  398. resp
  399. resp-unix >= "0.10.0"
  400. rfc1951 < "1.0.0"
  401. routes < "2.0.0"
  402. rpc >= "7.1.0"
  403. rpclib >= "7.1.0"
  404. rpclib-async
  405. rpclib-lwt >= "7.1.0"
  406. rpmfile < "0.3.0"
  407. rpmfile-eio
  408. rpmfile-unix
  409. rubytt
  410. SZXX >= "4.0.0"
  411. salsa20
  412. salsa20-core
  413. sanddb >= "0.2"
  414. scaml >= "1.5.0"
  415. scrypt-kdf
  416. secp256k1 >= "0.4.1"
  417. secp256k1-internal
  418. semver >= "0.2.1"
  419. sendmail
  420. sendmail-lwt
  421. sendmsg
  422. server-reason-react
  423. session-cookie
  424. session-cookie-async
  425. session-cookie-lwt
  426. sherlodoc
  427. sihl < "0.2.0"
  428. sihl-type
  429. slug
  430. sodium-fmt
  431. solidity-alcotest
  432. spin >= "0.7.0"
  433. squirrel
  434. ssh-agent
  435. ssl >= "0.6.0"
  436. stramon-lib
  437. styled-ppx
  438. syslog-rfc5424
  439. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  440. tdigest < "2.1.0"
  441. term-indexing
  442. terminal_size >= "0.1.1"
  443. terminus
  444. terminus-cohttp
  445. terminus-hlc
  446. terml
  447. textrazor
  448. tezos-base-test-helpers < "13.0"
  449. tezos-bls12-381-polynomial
  450. tezos-client-base < "12.0"
  451. tezos-crypto >= "8.0" & < "9.0"
  452. tezos-lmdb
  453. tezos-plompiler = "0.1.3"
  454. tezos-plonk = "0.1.3"
  455. tezos-signer-backends >= "8.0" & < "13.0"
  456. tezos-stdlib >= "8.0" & < "12.0"
  457. tezos-test-helpers < "12.0"
  458. tftp
  459. timedesc
  460. timere
  461. tls >= "0.12.0"
  462. toc
  463. topojson
  464. topojsone
  465. transept
  466. twostep
  467. type_eq
  468. type_id
  469. typebeat
  470. typeid >= "1.0.1"
  471. tyre >= "0.4"
  472. tyxml >= "4.0.0"
  473. tyxml-jsx
  474. tyxml-ppx >= "4.3.0"
  475. tyxml-syntax
  476. uecc
  477. ulid
  478. universal-portal
  479. unix-dirent
  480. unix-errno >= "0.3.0"
  481. unix-fcntl >= "0.3.0"
  482. unix-sys-resource
  483. unix-sys-stat
  484. unix-time
  485. unstrctrd
  486. user-agent-parser
  487. uspf
  488. uspf-lwt
  489. uspf-unix
  490. utop >= "2.13.0"
  491. validate
  492. validator
  493. vercel
  494. vpnkit
  495. wcwidth
  496. websocketaf
  497. x509 >= "0.7.0"
  498. xapi-rrd >= "1.8.2"
  499. xapi-stdext-date
  500. xapi-stdext-encodings
  501. xapi-stdext-std >= "4.16.0"
  502. yaml < "3.2.0"
  503. yaml-sexp
  504. yocaml
  505. yocaml_yaml
  506. yojson >= "1.6.0"
  507. yojson-five
  508. yuscii >= "0.3.0"
  509. yuujinchou = "1.0.0"
  510. zar
  511. zed >= "3.2.2"
  512. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct