package alcotest-lwt

  1. Overview
  2. Docs
Lwt-based helpers for Alcotest

Install

Dune Dependency

Authors

Maintainers

Sources

alcotest-mirage-1.3.0.tbz
sha256=79f9debdbca895374d6fdd73af8a470dcbe068b410483d35c04bb6ccc33e89ac
sha512=c41b17354d391d72f5f7bbbf520d7d227ec3df1bb25183e4a6761bb6d76e787ab89302bf58695cfe5a05b7d00cd77fe9d18d1eee396ecc724dfe942ecd1144aa

Description

Lwt-based helpers for Alcotest

Published: 17 Feb 2021

README

Alcotest logo"

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

For information on contributing to Alcotest, see CONTRIBUTING.md.

OCaml-CI Build Status" TravisCI Build Status" docs"

Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).
  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;
  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)
  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.
  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (6)

  1. logs
  2. lwt
  3. alcotest = version
  4. ocaml >= "4.03.0"
  5. fmt
  6. dune >= "2.2"

Dev Dependencies (2)

  1. cmdliner with-test
  2. re with-test

Used by (83)

  1. ambient-context-eio
  2. ambient-context-lwt
  3. ask
  4. ask-integrator
  5. azure-cosmos-db >= "0.1.3"
  6. capnp-rpc-mirage >= "0.6.0"
  7. capnp-rpc-unix >= "0.6.0" & < "1.2.3"
  8. carton < "1.0.0"
  9. carton-git
  10. carton-lwt < "1.0.0"
  11. current >= "0.4"
  12. current_git >= "0.6.4"
  13. equinoxe
  14. equinoxe-cohttp
  15. equinoxe-hlc
  16. git >= "3.0.0"
  17. git-cohttp
  18. git-cohttp-mirage
  19. git-cohttp-unix
  20. git-mirage >= "3.0.0"
  21. git-unix >= "3.0.0"
  22. gitlab-unix < "0.1.1"
  23. guardian
  24. http-mirage-client
  25. irmin >= "2.4.0"
  26. irmin-containers
  27. irmin-graphql >= "2.2.0"
  28. irmin-pack != "2.3.0" & != "2.6.1"
  29. ledgerwallet-tezos >= "0.4.0"
  30. letters
  31. mimic
  32. mirage-block-partition
  33. mirage-vnetif-stack < "0.6.1"
  34. multipart_form-lwt
  35. nbd >= "4.0.3"
  36. nbd-tool
  37. nbd-unix < "6.0.1"
  38. obuilder < "0.6.0"
  39. ocluster < "0.3.0"
  40. opium >= "0.19.0"
  41. opium-graphql
  42. opium-testing
  43. otoggl
  44. paf
  45. paf-cohttp
  46. pgx_lwt_unix
  47. piaf < "0.2.0"
  48. prometheus-app >= "1.2"
  49. rpclib-lwt >= "7.1.0"
  50. SZXX < "4.0.0"
  51. server-reason-react
  52. sihl != "0.3.0~rc1" & < "1.0.0~rc1"
  53. sihl-cache < "1.0.0~rc1"
  54. sihl-core
  55. sihl-email >= "0.3.0~rc1" & < "1.0.0~rc1"
  56. sihl-facade
  57. sihl-persistence >= "0.3.0~rc1"
  58. sihl-queue >= "0.3.0~rc1" & < "1.0.0~rc1"
  59. sihl-session
  60. sihl-storage >= "0.3.0~rc1" & < "1.0.0~rc1"
  61. sihl-token < "1.0.0~rc1"
  62. sihl-type
  63. sihl-user >= "0.3.0~rc1" & < "1.0.0~rc1"
  64. sihl-web >= "0.3.0~rc1"
  65. swapfs
  66. terminus
  67. terminus-cohttp
  68. terminus-hlc
  69. tezos-clic < "16.0"
  70. tezos-context < "12.3"
  71. tezos-crypto < "12.3"
  72. tezos-error-monad < "12.3"
  73. tezos-lwt-result-stdlib < "12.3"
  74. tezos-micheline < "16.0"
  75. tezos-p2p < "16.0"
  76. tezos-requester < "16.0"
  77. tezos-rpc-http-server < "16.0"
  78. tezos-shell-services = "12.3"
  79. tezos-stdlib < "12.3"
  80. tezos-test-helpers < "12.3"
  81. universal-portal
  82. vhd-format-lwt >= "0.13.0"
  83. wayland < "2.0"

Conflicts

None

OCaml

Innovation. Community. Security.