Module `B00.File_cache`

File caches.

A file cache maps a key to a metadata hunk and an ordered list of file contents (filenames are irrelevant).

B0 uses file caches to capture the contents of files written by build operations, it also stores their exit code and standard outputs in the metadata. This allows to recreate the effect of a build operation without having to rerun it.

FIXME. The following notions use the file system information and can be inaccurate. It's a bit unclear whether it's a good idea to rely on them. They are only used for cache trimming.

The access time of a key is the greatest access time to the file contents or metadata hunk it maps to.
A key is unused if all its file contents are not referenced outside the cache. Key usage cannot be determined if the key file contents and their references do not live on the same file system device or if the file system does not support hard links, see need_copy. FIXME. Now that we allow keys to map to empty list of file contents, unused might not make much sense for trimming. Maybe refine the notion to unused content.

File caches

type feedback = [

| `File_cache_need_copy of B0_std.Fpath.t

]

The type for file cache feedback. See create.

val pp_feedback : feedback B0_std.Fmt.t

pp_feedback formats file caches feedback.

type key = string

The type for keys. A key maps to a metadata hunk and an ordered list of file contents. The module treats keys as sequence of bytes however since they are used as file names they should satisfy the B0_std.Fpath.is_seg predicate; this is not checked by the module.

type t

The type for file caches.

val create : 
  ?feedback:(feedback -> unit) ->
  B0_std.Fpath.t ->
  (t, string) result

create ~feedback dir is a file cache using directory dir for data storage. The full path to dir is created by the call if it doesn't exist. feedback is invoked once if the cache switches to copying mode, see need_copy for details (defaults is a nop).

val dir : t -> B0_std.Fpath.t

dir c is c's storage directory.

val need_copy : t -> B0_std.Fpath.t option

need_copy c is Some file iff the cache switched to copying mode because of an operation that involved the file path file, external to the cache. This can happen due to one of the following conditions:

The cache directory dir and file live on different file system devices.
The underlying file system does not support hard links.
Too many hard links exist on a file.

file is also given to the notify_need_copy callback provided on create as soon as the condition is detected (and before the actual copy occurs). Note that once a file did copy, all the remaining transfers from or to the cache do copy aswell, which may be slow.

Cache operations

Note. In general, whenever a cache operation modifies the file system and errors with Error _ the resulting file system state is undefined.

val mem : t -> key -> bool

mem c k is true iff k is bound in c.

val add : t -> key -> string -> B0_std.Fpath.t list -> (bool, string) result

add c k m fs, binds the metadata m and the contents of the ordered list of files fs to k in c. The function returns:

Ok true if the operation succeeds.
Ok false if a file of fs could not be accessed. In this case k is guaranteed to be unbound in c.
Error _ if an unexpected error occurs. In that case the resulting state of the cache for key k is undefined.

val rem : t -> key -> (bool, string) result

rem c k removes the binding of k in c. Ok true is returned if k was bound in c and Ok false otherwise.

val find : 
  t ->
  key ->
  ((B0_std.Fpath.t * B0_std.Fpath.t list) option, string) result

find c k is Some (mf, fs) if k is bound in c with mf the file that holds the key metadata and fs the files that hold the file contents of the key in the order given on add. The result is None if k is unbound in c.

val revive : 
  t ->
  key ->
  B0_std.Fpath.t list ->
  ((string * B0_std.Fpath.t list) option, string) result

revive c k fs binds the file contents of key k to the inexistent file paths (directories are created) of fs. The function returns:

Ok (Some (m, existed) in case of success, with m the metadata of the key and existed the files that already existed and were left untouched by the operation. Assuming no other process fiddles with fs all these file paths now exist (but those of existed might differ from the corresponding one in the cache).
Ok None if the length of fs does not match the sequence of files of k or if k is unbound in c. In this case the file paths fs are left untouched.
Error _ if an unexpected error occurs. In that case the resulting state of the file system for paths fs is undefined.

val fold : 
  t ->
  (key -> B0_std.Fpath.t -> B0_std.Fpath.t list -> 'a -> 'a) ->
  'a ->
  ('a, string) result

fold c f acc folds f over the contents of each key of c starting with acc.

val keys : t -> (key list, string) result

keys c are the keys of cache c.

val is_unused : t -> key -> (bool, string) result

is_unused c k is true iff k is bound in c and its content is being unused. Warning For caches which are referenced across file system device or on file systems that do not support hard links this may return true even though the key is actually in use.

val delete_unused : t -> (unit, string) result

delete_unused c deletes unused keys of c.

val trim_size : t -> max_byte_size:int -> pct:int -> (unit, string) result

trim_size c max_byte_size ~pct delete keys of c until they either weight at most max_byte_size or are pct of their current size; whichever is the smaller. The function deletes by order of increasing access time but unused keys are deleted first.

Cache statistics

module Stats : sig ... end

Cache statistics.

Install

Dune Dependency

Authors

Maintainers

Sources

doc/b0.b00/B00/File_cache/index.html

Module `B00.File_cache`

File caches

Cache operations

Cache statistics

package b0

Install

Dune Dependency

Authors

Maintainers

Sources

doc/b0.b00/B00/File_cache/index.html

Module B00.File_cache

File caches

Cache operations

Cache statistics

Module `B00.File_cache`