Skip to content

Project Layout

The blackline_project_layout directory is organized into three main parts: the blackline_project.yaml file, the adapters folder, and the catalogue folder.

  1. blackline_project.yaml: This is the configuration file for the Blackline project. It contains project-level settings such as project name, version, and dependencies. This file is located in the root directory of the project.

  2. adapters folder: This folder contains all the adapters for the Blackline project. Each adapter is located in a sub-directory that is named after the organisation → system → resource that it interfaces with. Each resource directory contains a YAML file that defines the dataset for that resource.

  3. catalogue folder: This folder contains the metadata for the organisations, systems, resources and datasets used by the Blackline project. Within each resource directory, there are subdirectories for each dataset, and each dataset directory contains YAML files that define the collections and properties of the dataset.

The directory layout is designed to make it easy to organise and maintain the project's adapters and datasets. The blackline_project.yaml file provides project-level settings, while the adapters folder and catalogue folder contain the code and metadata, respectively, for interfacing with external systems and working with internal datasets.

Please reference the Concepts section for the specifics related to organisations, systems, resources, and datasets.

Folder structure

blackline_project_layout/
├── adapters/
│   └── <organization_name>/
│       ├── <system_name>/
│       │   ├── <resource_name>/
│       │   │   ├── dataset_name.yaml
│       │   │   └── ...
│       │   └── ...
│       └── ...
├── blackline_project.yml
└── catalogue/
    └── <organization_name>/
        ├── <system_name>/
        │   ├── <resource_name>/
        │   │   ├── <dataset_name>/
        │   │   │   ├── <collection_name>.yaml
        │   │   │   ├── ...
        │   │   │   └── dataset.yaml
        │   │   ├── ...
        │   │   └── resource.yml
        │   ├── ...
        │   └── system.yaml
        ├── ...
        └── organization.yaml

Example project

blackline_project_sample/
├── adapters/
│   └── blackline <- Organization/
│       ├── gcp <- Service/
│       │   ├── storage <- Resource/
│       │   │   └── logs.yml <- Dataset
│       │   └── bigquery/
│       │       ├── marketing.yml
│       │       └── user.yml
│       └── backend/
│           ├── user_microservice/
│           │   └── postgres.yml
│           ├── order_microservice/
│           │   └── sqlite.yml
│           └── logs/
│               └── logs.yml
├── blackline_project.yml
└── catalogue/
    └── blackline <- Organization/
        ├── backend <- Service/
        │   ├── order_microservice <- Resource/
        │   │   ├── order_db <- Dataset/
        │   │   │   ├── dataset.yml <- Collection/Field
        │   │   │   ├── order.yml
        │   │   │   └── shipment.yml
        │   │   └── resource.yml
        │   ├── service.yml
        │   └── user_micro/
        │       ├── user_db/
        │       │   ├── dataset.yml
        │       │   ├── user_contact.yml
        │       │   └── user_address.yml
        │       └── resource.yml
        ├── gcp/
        │   ├── big_query/
        │   │   ├── marketing/
        │   │   │   ├── marketing_email.yml
        │   │   │   ├── marketing_performance.yml
        │   │   │   └── dataset.yml
        │   │   ├── resource.yml
        │   │   └── user/
        │   │       ├── user_info.yml
        │   │       ├── user_pageviews.yml
        │   │       └── dataset.yml
        │   ├── storage/
        │   │   ├── logs/
        │   │   │   ├── client_logs.yml
        │   │   │   ├── server_logs.yml
        │   │   │   └── dataset.yml
        │   │   └── resource.yml
        │   └── service.yml
        └── organization.yml