package core_kernel

  1. Overview
  2. Docs
package core_kernel
Legend:
Library
Module
Module type
Parameter
Class
Class type

Implements a token-bucket-based throttling rate limiter. This module is useful for limiting network clients to a sensible query rate, or in any case where you have jobs that consume a scarce but replenishable resource.

In a standard token bucket there is an infinite incoming supply of tokens that fill a single bucket.

This version implements a closed system where tokens move through three possible states:

  • in hopper
  • in bucket
  • in flight

Tokens "drop" from the hopper into the bucket at a set rate, and can be taken from the bucket by clients and put into flight. Once the client is finished with whatever tokens are required for its task, it is responsible for moving them from "in flight" back into the hopper.

Most use cases are covered by the Token_bucket, Throttle, and Throttled_rate_limiter modules, but the Expert module provides full access to the module internals.

This interface is the simple, non-concurrent interface, and requires machinery on top to implement a specific strategy. See Async_extra for an async-friendly implementation on top of this module.

Most functions in this interface take an explicit time as an argument. now is expected to be monotonically increasing. now's that are set in the past are effectively moved up to the current time of the bucket.

type t
include sig ... end
val sexp_of_t : t -> Sexplib.Sexp.t
type limiter = t
include sig ... end
val sexp_of_limiter : limiter -> Sexplib.Sexp.t
module Infinite_or_finite : sig ... end
module Try_take_result : sig ... end
module Try_return_to_bucket_result : sig ... end
module Tokens_may_be_available_result : sig ... end
module Token_bucket : sig ... end

Implements a basic token-bucket-based rate limiter. Users of the throttle must successfully call try_take before doing work.

module Throttle : sig ... end

Implements a basic throttle. Users of the throttle must successfully call start_job before beginning work and must call finish_job once, and only once, when a job is completed.

module Throttled_rate_limiter : sig ... end

A Throttled_rate_limiter combines a Token_bucket and a Throttle. Unlike a Token_bucket, jobs cannot consume variable numbers of tokens, but the number of outstanding jobs is also limited to max_concurrent_jobs. Like a Throttle, finish_job must be called once, and only once, when a job is completed.

Common read-only operations

val bucket_limit : t -> Base.Int.t
val in_bucket : t -> now:Time_ns.t -> Base.Int.t

Tokens available to immediately take.

val in_hopper : t -> now:Time_ns.t -> Base.Int.t Infinite_or_finite.t

Tokens waiting to drop at the hopper_to_bucket_rate_per_sec.

val in_flight : t -> now:Time_ns.t -> Base.Int.t

Tokens that have been taken, but not yet returned.

val in_limiter : t -> now:Time_ns.t -> Base.Int.t Infinite_or_finite.t

Total number of tokens in the limiter in_hopper + in_bucket.

val in_system : t -> now:Time_ns.t -> Base.Int.t Infinite_or_finite.t

Total number of tokens in the entire system in_hopper + in_bucket + in_flight.

val hopper_to_bucket_rate_per_sec : t -> Base.Float.t Infinite_or_finite.t

Note that this isn't guaranteed to be equal to the rate_per_sec that was passed in to the constructor, due to floating point error.

module Expert : sig ... end

Expert operations.

include Base.Invariant.S with type t := t
val invariant : t -> unit
OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct