package datakit-bridge-local-git

  1. Overview
  2. Docs
DataKit Local-Git bridge

Install

Dune Dependency

Authors

Maintainers

Sources

datakit-v0.12.3.tbz
sha256=2634f6e04ca28caebed020484c945b8fa68c21f2215ed5dcda9923898de9368d
md5=5ff3d81b093cbe11bd6b29ae0c18aa62

Description

This service is a drop-in replacement for the DataKit-GitHub bridge that instead just monitors a local Git repository. It is useful for testing a new DataKitCI configuration without having to configure GitHub integration first.

The local bridge monitors the state of one or more local Git repositories, writing the current head of each branch to DataKit. DataKitCI can be configured to run the CI tests against the project each time a commit is made.

Once you are happy with the way the CI is working, you can replace this service with the GitHub bridge service to have the CI test a project hosted on GitHub instead.

Unlike the GitHub bridge, this service:

  • only reports on branches, not tags or pull requests;
  • does not report build statuses from other CI systems; and
  • does not push the statuses set by the CI anywhere.

Published: 11 Mar 2019

README

DataKit -- Orchestrate applications using a Git-like dataflow

DataKit is a tool to orchestrate applications using a Git-like dataflow. It revisits the UNIX pipeline concept, with a modern twist: streams of tree-structured data instead of raw text. DataKit allows you to define complex build pipelines over version-controlled data.

DataKit is currently used as the coordination layer for HyperKit, the hypervisor component of Docker for Mac and Windows, and for the DataKitCI continuous integration system.

There are several components in this repository:

  • src contains the main DataKit service. This is a Git-like database to which other services can connect.

  • ci contains DataKitCI, a continuous integration system that uses DataKit to monitor repositories and store build results.

  • ci/self-ci is the CI configuration for DataKitCI that tests DataKit itself.

  • bridge/github is a service that monitors repositories on GitHub and syncs their metadata with a DataKit database. e.g. when a pull request is opened or updated, it will commit that information to DataKit. If you commit a status message to DataKit, the bridge will push it to GitHub.

  • bridge/local is a drop-in replacement for bridge/github that just monitors a local Git repository. This is useful for local testing.

Quick Start

The easiest way to use DataKit is to start both the server and the client in containers.

To expose a Git repository as a 9p endpoint on port 5640 on a private network, run:

$ docker network create datakit-net # create a private network
$ docker run -it --net datakit-net --name datakit -v <path/to/git/repo>:/data datakit/db

Note: The --name datakit option is mandatory. It will allow the client to connect to a known name on the private network.

You can then start a DataKit client, which will mount the 9p endpoint and expose the database as a filesystem API:

# In an other terminal
$ docker run -it --privileged --net datakit-net datakit/client
$ ls /db
branch     remotes    snapshots  trees

Note: the --privileged option is needed because the container will have to mount the 9p endpoint into its local filesystem.

Now you can explore, edit and script /db. See the Filesystem API for more details.

Building

The easiest way to build the DataKit project is to use docker, (which is what the start-datakit.sh script does under the hood):

docker build -t datakit/db -f Dockerfile .
docker run -p 5640:5640 -it --rm datakit/db --listen-9p=tcp://0.0.0.0:5640

These commands will expose the database's 9p endpoint on port 5640.

If you want to build the project from source without Docker, you will need to install ocaml and opam. Then write:

$ make depends
$ make && make test

For information about command-line options:

$ datakit --help

Prometheus metric reporting

Run with --listen-prometheus 9090 to expose metrics at http://*:9090/metrics.

Note: there is no encryption and no access control. You are expected to run the database in a container and to not export this port to the outside world. You can either collect the metrics by running a Prometheus service in a container on the same Docker network, or front the service with nginx or similar if you want to collect metrics remotely.

Language bindings

  • Go bindings are in the api/go directory.

  • OCaml bindings are in the api/ocaml directory. See examples/ocaml-client for an example.

Licensing

DataKit is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Contributions are welcome under the terms of this license. You may wish to browse the weekly reports to read about overall activity in the repository.

Dependencies (13)

  1. datakit-github >= "0.12.0"
  2. datakit-client-9p >= "0.12.0"
  3. datakit-client >= "0.12.0"
  4. protocol-9p-unix >= "0.11.0"
  5. fmt
  6. logs
  7. lwt >= "3.0.0"
  8. irmin-unix >= "1.2.0" & < "2.0.0"
  9. irmin >= "1.2.0" & < "2.0.0"
  10. irmin-watcher
  11. cmdliner
  12. dune
  13. ocaml

Dev Dependencies

None

Used by

None

Conflicts

None

OCaml

Innovation. Community. Security.