package irmin-pack
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=9e72efdc13324602d8da46e0492b1f4e09ad524149845e1a9106882f60ad6e1f
sha512=ae9cb4b086c7b93962aafc2ea339ac85c6d66bdf769f70a16f62fa5c74d9b194e12ac00b510be5ffac5adff3aa20fd69c4ca4067dcc966767fefcf5c02bc9969
doc/irmin-pack.unix/Irmin_pack_unix/Append_only_file/Make/index.html
Module Append_only_file.MakeSource
Abstraction for irmin-pack's append only files (i.e. suffix and dict).
It is parameterized with Io, a file system abstraction (e.g. unix, mirage, eio_linux).
It comprises a persistent file, an append buffer and take care of automatically shifting offsets to deal with legacy headers.
Parameters
Signature
val create_rw :
path:string ->
overwrite:bool ->
auto_flush_threshold:int ->
auto_flush_callback:(t -> unit) ->
(t, [> Io.create_error ]) resultCreate a rw instance of t by creating the file.
val open_rw :
path:string ->
end_offset:Optint.Int63.t ->
dead_header_size:int ->
auto_flush_threshold:int ->
auto_flush_callback:(t -> unit) ->
(t,
[> Io.open_error
| `Closed
| `Invalid_argument
| `Read_out_of_bounds
| `Inconsistent_store ])
resultCreate a rw instance of t by opening an existing file at path.
End Offset
The file has an end offset at which new data will be saved. While this information could be computed by looking at the size of the file, we prefer storing that information elsewhere (i.e. in the control file). This is why open_rw and open_ro take an end_offset parameter, and also why refresh_end_offset exists. The abstractions above Append_only_file are responsible for reading/writing the offsets from/to the control file.
dead_header_size
Designates a small area at the beginning of the file that should be ignored. The offsets start after that area.
The actual persisted size of a file is end_offset + dead_header_size.
This concept exists in order to keep supporting `V1 and `V2 pack stores with `V3.
Auto Flushes
One of the goal of the Append_only_file abstraction is to provide buffered appends. auto_flush_threshold is the soft cap after which the buffer should be flushed. If a call to append_exn fills the buffer, auto_flush_callback will be called so that the parent abstraction takes care of the flush procedure.
val open_ro :
path:string ->
end_offset:Optint.Int63.t ->
dead_header_size:int ->
(t,
[> Io.open_error
| `Closed
| `Inconsistent_store
| `Invalid_argument
| `Read_out_of_bounds ])
resultCreate a ro instance of t by opening an existing file at path
Close the underlying file.
The internal buffer is expected to be in a flushed state when close is called. Otherwise, an error is returned.
end_offset t is the number of bytes of the file. That function doesn't perform IO.
RW mode
It also counts the bytes not flushed yet.
RO mode
This information originates from the latest reload of the control file. Calling refresh_end_offset t updates end_offset.
val read_to_string :
t ->
off:Optint.Int63.t ->
len:int ->
(string, [> Io.read_error ]) resultread_exn t ~off ~len b puts the len bytes of t at off to b.
Raises Io.Read_error
RW mode
Attempting to read from the append buffer results in an `Read_out_of_bounds error. This feature could easily be implemented in the future if ever needed. It was not needed with io_legacy.
RO mode
It is not possible to read from an offset further than end_offset t.
append_exn t ~off b writes b to the end of t. Might trigger an auto flush.
Post-condition: end_offset t - end_offset (old t) = String.length b.
Raises Io.Write_error
RW mode
Always raises Io.Write_error `Ro_not_allowed
Flush the append buffer. Does not call fsync.
RO mode
Always returns Error `Ro_not_allowed.
Tell the os to fush its internal buffers. Does not call flush.
RO mode
Always returns Error `Ro_not_allowed.
Ingest the new end offset of the file.
RW mode
Always returns Error `Rw_not_allowed.