|
1 | 1 | # FyrMesh |
2 | 2 |  |
3 | | -## A Python package for mesh orchestration and control with a built-in CLI and RPC-based mesh handling server. |
| 3 | +## A Go package for sensor mesh orchestration powered by gRPC & Protocol Buffers with a built-in FyrCLI. |
4 | 4 |
|
5 | | -**Version: 0.1.0** |
6 | | -**Platform: Raspberry Pi (Linux)** |
7 | | -**Language: Python 3.7** |
| 5 | +**Version: 0.2.0** |
| 6 | +**Platform: Raspbian OS & Windows** |
| 7 | +**Language: Go 1.16 & Python 3.8** |
8 | 8 |
|
9 | 9 | ### **Contributors** |
10 | 10 | - **Manish Meganathan** |
11 | 11 | - **Mariyam A. Ghani** |
12 | 12 |
|
13 | | -This repository is a python package that includes: |
14 | | -- **An RPC Server Program** |
15 | | - An RPC Service that runs the orchestartion interface between the mesh and the Raspberry Pi while exposing the methods of the service as an RPC object that is hosted on Raspberry Pi and can be connected through the localnet. |
16 | | -- **A Command Line Interface library** |
17 | | - An interface library that exposes commands as a CLI built using the Click framework. It connects to the RPC server and can be spun on up on machine on the local network and configured. |
18 | | -- **A Firbase Cloud Interface library** |
19 | | - An interface library that communicates with the Firebase backend and other cloud services. |
20 | | -- **A Mesh Orchestration library** |
21 | | - A library of custom thread runtimes, data classes, mesh messaging protocol runtimes, etc. that play a role in the orchestation of the mesh. |
| 13 | +### **Contents** |
| 14 | + - [**Overview**](#overview) |
| 15 | + - [**FyrLINK**](#fyrlink) |
| 16 | + - [**FyrORCH**](#fyrorch) |
| 17 | + - [**FyrCLI**](#fyrcli) |
| 18 | + - [``gopkg`` **fyrorch/orch**](#gopkg-fyrorchorch) |
| 19 | + - [``gopkg`` **tools**](#gopkg-tools) |
| 20 | + - [**gRPC and Protocol Buffers**](#grpc-and-protocol-buffers) |
| 21 | + - [**Installation**](#installation) |
| 22 | + - [**1. Prerequisites**](#1-prerequisites) |
| 23 | + - [**2. Install FyrMesh**](#2-install-fyrmesh) |
| 24 | + - [**3. Setup Environment Variables**](#3-setup-environment-variables) |
| 25 | + - [*Linux*](#linux) |
| 26 | + - [*Windows*](#windows) |
| 27 | + - [**Using the FyrCLI**](#using-the-fyrcli) |
| 28 | + - [**Starting the FyrMesh**](#starting-the-fyrmesh) |
22 | 29 |
|
| 30 | +### **Overview** |
| 31 | +This package is a collection of microservices and a CLI tool that collectively form the FyrMesh platform. The communication between microservices is done with protocol buffers over gRPC. |
| 32 | +The various components of the package are described below. |
23 | 33 |
|
24 | | -### **Installation** |
25 | | -The ``fyrmesh`` package and CLI can be installed by first ``cd`` into the repository and running the command: |
26 | | -``` |
27 | | -pip3 install -e . |
28 | | -``` |
29 | | -The ``-e`` flag sets the installation to *editable*. This is reserved just for development use. Similarly the use between ``pip3`` and ``pip`` is dependant on the configuration of the machine. |
| 34 | +#### **FyrLINK** |
| 35 | +A Python server that handles the communication between the Raspberry Pi and the Control Node |
| 36 | +over the serial port. It exposes an interface with the ``LINK`` gRPC server which implements methods that allow clients to read from and write to the serial port of the Raspberry Pi while ensuring the integrity of the message structures expected by the control node and semi-parsing the messages recieved. |
30 | 37 |
|
31 | | -The fyrmesh CLI is now installed. Run ``fyrmesh`` on the terminal to see the help document. If an 'entry point not found' error is thrown, do the following: |
32 | | -- Run the following to open the ``.bash_profile`` file. |
33 | | - ``` |
34 | | - nano ~/.bash_profile |
| 38 | +#### **FyrORCH** |
| 39 | +A Go server that handles the orchestration of the mesh and the communication to the cloud backend services and the database through Firebase. It exposes an interface with the ``ORCH`` gRPC server |
| 40 | +which implements methods for various mesh functionality that manipulate the state of the mesh or send messages on it. |
| 41 | + |
| 42 | +#### **FyrCLI** |
| 43 | +A command-line interface application written in Go with the [**Cobra**](https://github.com/spf13/cobra) framework. It contains commands that allow a user to interact with the mesh through the orchestrator over a gRPC connection. The CLI tool can be used on the mesh controller itself or even on a remote system within the local network that is configured as a mesh observer device. |
| 44 | + |
| 45 | +#### ``gopkg`` **fyrorch/orch** |
| 46 | +A Go package that contains the implementation of the ``LINK`` gRPC client, the ``ORCH`` gRPC server and the ``ORCH`` gRPC client. The client implementations contain helper functions to call its service methods appropriately. |
| 47 | + |
| 48 | +#### ``gopkg`` **tools** |
| 49 | +A Go package that contains tools used by various **FyrMesh** services such interacting with the config file, handling logging I/O, interfacing with cloud services and manipulating internal data structures. |
| 50 | + |
| 51 | +#### gRPC and Protocol Buffers |
| 52 | +[**gRPC**](https://grpc.io/) is a modern open source high performance Remote Procedure Call (RPC) framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication. It is also applicable in last mile of distributed computing to connect devices, mobile applications and browsers to backend services. While, gRPC as a protocol supports communication between services with JSON but it was designed to work well with the Protocol Buffers encoding format. |
| 53 | + |
| 54 | +[**Protocol Buffers**](https://developers.google.com/protocol-buffers) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. |
| 55 | + |
| 56 | +The protocol buffers for the FyrMesh are defined in a file ``fyrmesh/protos/fyrmesh.proto``. |
| 57 | + |
| 58 | +**How does it work?** |
| 59 | + |
| 60 | +In gRPC, a client application can directly call a method on a server application on a different machine as if it were a local object, making it easier for you to create distributed applications and services. As in many RPC systems, gRPC is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. On the server side, the server implements this interface and runs a gRPC server to handle client calls. On the client side, the client has a stub (referred to as just a client in some languages) that provides the same methods as the server. |
| 61 | + |
| 62 | +gRPC clients and servers can run and talk to each other in a variety of environments - from servers inside Google to your own desktop - and can be written in any of gRPC’s supported languages. So, for example, you can easily create a gRPC server in Java with clients in Go, Python, or Ruby. In addition, the latest Google APIs will have gRPC versions of their interfaces, letting you easily build Google functionality into your applications. |
| 63 | + |
| 64 | +gRPC Source files: |
| 65 | +- [gRPC GitHub Repository](https://github.com/grpc/grpc) |
| 66 | +- [gRPC-Go GitHub Repository](https://github.com/grpc/grpc-go) |
| 67 | + |
| 68 | +For more information: |
| 69 | +- [gRPC Introduction](https://grpc.io/docs/what-is-grpc/introduction/) |
| 70 | +- [gRPC Core Concepts](https://grpc.io/docs/what-is-grpc/core-concepts/) |
| 71 | +- [Protocol Buffers Overview](https://developers.google.com/protocol-buffers/docs/overview) |
| 72 | +- [Working with Protocol Buffers](https://grpc.io/docs/what-is-grpc/introduction/#working-with-protocol-buffers) |
| 73 | + |
| 74 | +### **Installation** |
| 75 | +The FyrMesh library can be installed onto a Linux or a Windows system. Linux support is only intended for the Raspbian OS that is designed for the Raspberry Pi. |
| 76 | + |
| 77 | +#### 1. Prerequisites |
| 78 | +Install Go 1.16 and Python 3.8 |
| 79 | + - [Install Go on the Raspberry Pi](https://www.jeremymorgan.com/tutorials/raspberry-pi/install-go-raspberry-pi/) |
| 80 | + - [Install Go on Windows](https://golang.org/doc/install) |
| 81 | + - [Install Python on the Raspberry Pi](https://projects.raspberrypi.org/en/projects/generic-python-install-python3#:~:text=Open%20your%20web%20browser%20and,a%20download%20will%20start%20automatically.) |
| 82 | + - [Install Python on Windows](https://realpython.com/installing-python/) |
| 83 | + |
| 84 | + |
| 85 | +Download this repository to some location on your system with one of the following methods. This location will be used to configure ``FYRMESHSRC`` in the next steps. |
| 86 | + - Download the latest release from the [releases](https://github.com/fyrwatch/fyrmesh/releases) (v0.2 or above). |
| 87 | + - Use the Git bash to clone the repository. |
35 | 88 | ``` |
36 | | -- Add the following line to the file. |
| 89 | + git clone https://github.com/fyrwatch/fyrmesh.git |
37 | 90 | ``` |
38 | | - export PATH=~/.local/bin:$PATH |
| 91 | + - Use ``go get`` to download the package. |
39 | 92 | ``` |
40 | | -- Run the following on a terminal |
| 93 | + go get -u github.com/fyrwatch/fyrmesh |
41 | 94 | ``` |
42 | | - source ~/.bash_profile |
| 95 | + |
| 96 | +#### 2. Install FyrMesh |
| 97 | +- Navigate into the ``/fyrmesh`` directory of the repository after downloading it. |
| 98 | +- Open a terminal window in this directory and run the following command |
| 99 | +``` |
| 100 | +python3 install.py |
| 101 | +``` |
| 102 | +- The FyrMesh components will now be installed. |
| 103 | + |
| 104 | +#### 3. Setup Environment Variables |
| 105 | +The FyrMesh services and applications require the environment variables ``FYRMESHCONFIG`` and ``FYRMESHSRC``. These environment variables have to be persistent and define the path to the the config file and the path to fyrmesh source folder respectively. These variables while not required at installation are required for the application to work. |
| 106 | + |
| 107 | +The steps to add environment variables are detailed below for each platform. |
| 108 | + |
| 109 | +##### Linux |
| 110 | +- Run the following to cd into the ``/etc/profile.d`` directory and create a new file ``fyrmesh.sh``. This directory contains the shell scripts that are run at startup. |
| 111 | + ``` |
| 112 | + $ cd /etc/profile.d |
| 113 | + $ nano envvars.sh |
| 114 | + ``` |
| 115 | +- Add the following lines to ``fyrmesh.sh`` file. |
43 | 116 | ``` |
44 | | -The last command might have to be run for each new terminal window depending on the machine configuration. |
| 117 | + export FYRMESHCONFIG=<<path to the config file>> |
| 118 | + export FYRMESHSRC=<<path to the repository source files directory>> |
| 119 | + export PATH=$PATH:<<path to Go bin install directory>> |
| 120 | + ``` |
| 121 | + The recommended value for ``FYRMESHCONFIG`` is ``/home/pi/.fyrmesh/config.json`` |
| 122 | + The recommended value for go bin path extension is ``/home/pi/go/bin`` |
| 123 | +- Reboot the system to apply changes. |
| 124 | + |
| 125 | +##### Windows |
| 126 | +- Press the Start Menu and type *'env*'. |
| 127 | +- An option *'Edit the environment variables for your account'* will appear. Select it. |
| 128 | +- A list of environment variables will be visible. Select the *'New'* option. |
| 129 | +- Fill in the value for ``FYRMESHCONFIG``. The recommended value is ``C:\Users\<user>\.fyrmesh\config.json``. |
| 130 | +- Repeat and fill in the value for ``FYRMESHSRC`` with the path to the install directory. |
| 131 | +- PATH extension wouldn't be required because the Go installer for windows handles it. |
| 132 | +- Reboot the systems to apply changes. |
| 133 | + |
| 134 | +The installation of FyrMesh is now complete. |
| 135 | +Run the ``fyrcli`` command on the console to try it. |
| 136 | + |
| 137 | +### Using the FyrCLI |
| 138 | +Run the command ``fyrcli help`` to view the usage of the FyrCLI Application. |
| 139 | +The output will be something along the lines of the following. |
| 140 | + |
| 141 | +``` |
| 142 | +A CLI Application to interact with the FyrMesh API. Powered by Cobra CLI, Golang and gRPC. |
| 143 | +
|
| 144 | +Usage: |
| 145 | + FyrCLI [command] |
| 146 | +
|
| 147 | +Available Commands: |
| 148 | + boot Boots a FyrMesh gRPC server. |
| 149 | + config View/Manipulate configuration values of the FyrCLI. |
| 150 | + connect Set the connection state of the control node. |
| 151 | + help Help about any command |
| 152 | + observe Observes the logstream of the ORCH server. |
| 153 | + ping Pings the mesh for sensor readings. |
| 154 | + status Displays the current status of the mesh. |
| 155 | +
|
| 156 | +Flags: |
| 157 | + --config string config file (default is $HOME/.cli.yaml) |
| 158 | + -h, --help help for FyrCLI |
| 159 | +
|
| 160 | +Use "FyrCLI [command] --help" for more information about a command. |
| 161 | +``` |
| 162 | + |
| 163 | +You can also run ``fyrcli help <command>`` or ``fyrcli <command> --help`` to recieve the |
| 164 | +usage instruction for any particular command. |
| 165 | + |
| 166 | +### Starting the FyrMesh |
| 167 | +Starting the FyrMesh involves booting ``LINK`` and ``ORCH`` services. After which commands can be called and they will perform as expected. The FyrMesh service can however only be run a device that is configured as mesh-controller, which is usually a ARM based controller like the Raspberry Pi. |
| 168 | + |
| 169 | +Running the FyrCLI on Windows machines configured as a mesh-observer is simply useful to send commands remotely and observing the behaviour of the mesh-controller. |
| 170 | + |
| 171 | +**How to boot the services?** |
| 172 | + |
| 173 | +To boot the ``ORCH`` and ``LINK`` services, run the following commands. |
| 174 | +``` |
| 175 | +fyrcli boot --server LINK |
| 176 | +fyrcli boot --server ORCH |
| 177 | +``` |
| 178 | + |
| 179 | +The ``LINK`` server must **always** be booted before the ``ORCH`` server to avoid gRPC errors. |
0 commit comments