Spool directory structure:
Async_smtp uses a spool directory structure heavily inspired by that of Exim (see 1 and 2 for details on that). On startup, async_smtp takes out a lock on the spool directory (using the Lock_file module with the file $spool_dir/lock) and assumes that no other process will be manipulating the files or directories below it without using the async_smtp RPC interface.
The lifetime of a message looks like this:
When async_smtp accepts a message it immediately processes it, possibly expanding it to multiple additional messages (at least one per recipient). Each of the expanded messages are added to the spool (by calling Spool.add), writing it first to $spool_dir/tmp/$msgid (with a timestamp, as well as information about whether the message is frozen or not, the last relay attempt, and the parent message ID if any) and then renaming it to $spool_dir/active/$msgid (to minimize the chance of a message being sent multiple times in case of a crash).
Newly spooled messages are also immediately written to a queue. A background loop iterates over this queue, processing and relaying messages in accordance with the max_concurrent_send_jobs configuration option. Async_smtp attempts to send each message in turn.
On success, the message is removed from the active directory.
On failure, the last_relay_attempt_date is immediately updated in the on disk spool file (again using tmp to make the change as atomic as possible). If the message has any remaining retry intervals in envelope_routed.retry_intervals then async_smtp schedules a retry for after the interval has elapsed (rewriting the spooled message with the interval popped off the list of remaining retry intervals only after the interval has elapsed). If there are no remaining retry intervals then the message is marked as frozen and moved into $spool_dir/frozen/$msgid (and no further attempts to send it are made).
If async_smtp crashes (or is shutdown) and the spool has contents then it is reloaded as follows:
If there are any contents of $spool_dir/tmp then async_smtp will refuse to start. Such messages indicate that mailcore died while changing a message on disk, which is a serious problem.
The contents of $spool_dir/active are read in and re-queued based on the last attempted relay time and the remaining retry intervals as above.
1 http://www.exim.org/exim-html-current/doc/html/spec_html/ch-how_exim_receives_and_delivers_mail.html 2 http://www.exim.org/exim-html-current/doc/html/spec_html/ch-format_of_spool_files.html
Lock the spool directory and load all the files that are already present there. Note that for the purposes of locking, the spool directory assumed to NOT be on an NFS file system.
Immediately write the message to disk and queue it for sending. The Smtp_envelope.Routed.Batch.t list represents the different "sections" of one message. We make no guarantees about the order of delivery of messages.
kill_and_flush t makes sure no new delivery sessions are being started and waits until all the currently running sessions have finished. It will not affect frozen messages or those waiting for retry intervals to elapse.
This is not necessarily a snapshot of the spool at any given point in time. The only way to obtain such a snapshot would be to pause the server and we don't want to do that. However, this status will include emails that are stuck on the spool, and those are the ones we care about.
You should not try to work out the total number of unsent messages by counting the messages in the status. You should use the count_from_disk function instead.
val client_cache : t -> Async_smtp__.Client_cache.tmodule Event : sig ... end