IML Distributed Architecture

Software Contributor Documentation Table of Contents

architecture

The current setup

IML is composed of a series of modules and services. Many modules are hosted as separate repositories, and many are hosted in a single repo.

For brevity, we will refer to the separate repos as just repos, and the integrated-manager-for-lustre repo as a monorepo

Repos

Each repo is a separate entry under the whamCloud organization.

Many of the repos utilize travis to run their automated tests. Travis is a general hosted CI platform where unit / integration tests can be run. Here is an example. They also are published individually by travis on a public facing package registry; here is an example.

The larger repos may depend on smaller repos for reusable code. In this way, they build complex apps out of small pieces. Here is an example of a module depending on other versioned modules.

These modules can be thought of as a tree. The leaves and lower nodes consist of useful functionality that is consumed by the root of the tree.

In addition to publishing in a language specific package registry, the root nodes are usually published using a tool called Copr. This allows OS level aggregation and installation.

Root Repos publish workflow

┌─────────────────────────────────────┐
│                                     │
│          Push code to repo          │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│         Pass Travis testing         │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│             Code Review             │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│          Gatekeeper lands           │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│ Bump version + publish using Travis │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│            Add / update             │
│manager-for-lustre-dependencies repo │
│            with package.            │
└─────────────────────────────────────┘
                   │
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│             Pass build.             │
│                                     │
└─────────────────────────────────────┘
                   │
                   │
                   ▼
┌─────────────────────────────────────┐
│                                     │
│           Publish on Copr           │
│                                     │
└─────────────────────────────────────┘

Top of page