Creating services
One of the main features of the Octue SDK is to allow you to easily create services that can accept questions and return answers. They can run locally on any machine or be deployed to the cloud. Currently:
The backend communication between twins uses Google Pub/Sub whether they’re local or deployed
The deployment options are Google Cloud Run or Google Dataflow
The language of the entrypoint must by
python3
(you can call processes using other languages within this though)
Anatomy of an Octue service
An Octue service is defined by the following files (located in the repository root by default).
app.py
This is the entrypoint into your code - read more...
This is where you write your app. The
app.py
file can contain any valid python, including import and use of any number of external packages or your own subpackages. It has only two requirements:
It must contain exactly one of the
octue
python app interfaces that serve as an entrypoint to your code. These take a singleAnalysis
instance:
Option 1: A function named
run
with the following signature:def run(analysis): """A function that uses input and configuration from an ``Analysis`` instance and stores any output values and output manifests on it. :param octue.resources.Analysis analysis: :return None: """ ...Option 2: A class named
App
with the following signature:class App: """A class that takes an ``Analysis`` instance and anything else you like. It can contain any methods you like but it must also have a ``run`` method. :param octue.resources.Analysis analysis: :return None: """ def __init__(self, analysis, *args, **kwargs): self.analysis = analysis ... def run(self): """A method that that uses input and configuration from an ``Analysis`` instance and stores any output values and output manifests on it. :return None: """ ... ...It must access configuration/input data from and store output data on the
analysis
parameter/attribute:
Configuration values:
analysis.configuration_values
Configuration manifest:
analysis.configuration_manifest
Input values:
analysis.input_values
Input manifest:
analysis.input_manifest
Output values:
analysis.output_values
Output manifest:
analysis.output_manifest
This allows standardised configuration/input/output for services while allowing you to do anything you like with the input data to produce the output data.
twine.json
Dependencies file
octue.yaml
This describes the service configuration - read more...
This file defines the basic structure of your service. It must contain at least:
services: - name: my-appIt may also need the following key-value pairs:
app_source_path: <path>
- if yourapp.py
file is not in the repository root
app_configuration_path: <path>
- if your app needs an app configuration file that isn’t in the repository root
dockerfile_path: <path>
- if your app needs aDockerfile
that isn’t in the repository rootAll paths should be relative to the repository root. Other valid entries can be found in the
ServiceConfiguration
constructor.Warning
Currently, only one service can be defined per repository, but it must still appear as a list item of the “services” key. At some point, it will be possible to define multiple services in one repository.
App configuration file (optional)
Dockerfile (optional)
Provide this if your needs exceed the default Octue Dockerfile - read more...
Octue services run in a Docker container if they are deployed. They can also run this way locally. The SDK provides a default
Dockerfile
for these purposes that will work for most cases:
For deploying to Google Cloud Run
For deploying to Google Dataflow
However, you may need to write and provide your own
Dockerfile
if your app requires:
Non-python or system dependencies (e.g.
openfast
,wget
)Python dependencies that aren’t installable via
pip
Private python packages
Here are two examples of a custom
Dockerfile
that use different base images:If you do provide one, you must specify its path in
octue.yaml
under thedockerfile_path
key.As always, if you need help with this, feel free to drop us a message or raise an issue!
Template apps
We’ve created some template apps for you to look at and play around with. We recommend going through them in this order:
The fractal app template - introduces a basic Octue service that returns output values to its parent.
The using-manifests app template - introduces using a manifest of output datasets to return output files to its parent.
The child-services app template - introduces asking questions to child services and using their answers to form an output to return to its parent.
Deploying services automatically
Automated deployment with Octue means:
Your service runs in Google Cloud, ready to accept questions from and return answers to other services.
You don’t need to do anything to update your deployed service with new code changes - the service simply gets rebuilt and re-deployed each time you push a commit to your
main
branch, or merge a pull request into it (other branches and deployment strategies are available, but this is the default).Serverless is the default - your service only runs when questions from other services are sent to it, meaning there is no cost to having it deployed but not in use.
To enable automated deployments, contact us so we can create a Google Cloud Build trigger linked to your git repository. This requires no work from you apart from authorising the connection to GitHub (or another git provider).
If you want to deploy services yourself, see here.