package core_kernel
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=e37370bad978cfb71fdaf2b1a25ab1506b98ef0b91e0dbd189ffd9d853245ce2
doc/core_kernel.limiter/Limiter/index.html
Module LimiterSource
Implements a token-bucket-based throttling rate limiter. This module is useful for limiting network clients to a sensible query rate, or in any case where you have jobs that consume a scarce but replenishable resource.
In a standard token bucket there is an infinite incoming supply of tokens that fill a single bucket.
This version implements a closed system where tokens move through three possible states:
- in hopper
- in bucket
- in flight
Tokens "drop" from the hopper into the bucket at a set rate, and can be taken from the bucket by clients and put into flight. Once the client is finished with whatever tokens are required for its task, it is responsible for moving them from "in flight" back into the hopper.
Most use cases are covered by the Token_bucket, Throttle, and Throttled_rate_limiter modules, but the Expert module provides full access to the module internals.
This interface is the simple, non-concurrent interface, and requires machinery on top to implement a specific strategy. See Limiter_async for an async-friendly implementation on top of this module.
Most functions in this interface take an explicit time as an argument. now is expected to be monotonically increasing. now's that are set in the past are effectively moved up to the current time of the bucket.
Implements a basic token-bucket-based rate limiter. Users of the throttle must successfully call try_take before doing work.
Implements a basic throttle. Users of the throttle must successfully call start_job before beginning work and must call finish_job once, and only once, when a job is completed.
A Throttled_rate_limiter combines a Token_bucket and a Throttle. Unlike a Token_bucket, jobs cannot consume variable numbers of tokens, but the number of outstanding jobs is also limited to max_concurrent_jobs. Like a Throttle, finish_job must be called once, and only once, when a job is completed.
Common read-only operations
Tokens available to immediately take.
Tokens waiting to drop at the hopper_to_bucket_rate_per_sec.
Tokens that have been taken, but not yet returned.
Total number of tokens in the limiter in_hopper + in_bucket.
Total number of tokens in the entire system in_hopper + in_bucket + in_flight.
Note that this isn't guaranteed to be equal to the rate_per_sec that was passed in to the constructor, due to floating point error.