module Farmstead
Database subcommands
Test subcommands
Extract
data from the source
Runs a Consumer and it will automatically pick up messages from the Field Topic and do it's job and then send a message as a Producer to the Forest Topic
Every micro-service inherits the Service
class
Load
data into database
Runs a Consumer and it will automatically pick up messages from the Road Topic and load the message into the database
Every micro-service inherits the Service
class
It works off of the DB
when 1) A new source is added 2) A scheduled source pull is configured to happen
Field –> Forest –> Road
It then takes the config from the DB
and passed it to the Field topic
The Extract
class pulls the source config from the Field Topic and then pushes source data to the Forest topic.
The Transform
class pulls the source data from teh Forest topic and then pushes to the Road topic.
The Load
class pulls data from the Road topic and loads it into the database.
Topics are created when Kafka comes up HINT: See .env Every micro-service inherits the Service
class
Transform
data
Arrange data into usable blocks
Running as a Consumer and it will automatically pick up messages from the Forest topic and do it's job and then send a message as a Producer to the Road topic
Every micro-service inherits the Service
class
Base Class for all micro-services
Constants
- VERSION