One of the main features of the Octue SDK is to allow you to easily create services that can accept questions and return answers. They can run locally on any machine or be deployed to the cloud. Currently:
The backend communication between twins uses Google Pub/Sub whether they’re local or deployed
The deployment options are Google Cloud Run or Google Dataflow
The language of the entrypoint must by
python3(you can call processes using other languages within this though)
Anatomy of an Octue service
An Octue service is defined by the following files (located in the repository root by default).
This is the entrypoint into your code - read more here.
A file specifying your app’s dependencies. This is a setup.py file, a requirements.txt file, or a pyproject.toml file listing all the python packages your app depends on and the version ranges that are supported.
This describes the service configuration - read more...
This file defines the basic structure of your service. It must contain at least:services: - namespace: my-organisation name: my-app
It may also need the following key-value pairs:
app_source_path: <path>- if your
app.pyfile is not in the repository root
app_configuration_path: <path>- if your app needs an app configuration file that isn’t in the repository root
dockerfile_path: <path>- if your app needs a
Dockerfilethat isn’t in the repository root
All paths should be relative to the repository root. Other valid entries can be found in the
Currently, only one service can be defined per repository, but it must still appear as a list item of the “services” key. At some point, it will be possible to define multiple services in one repository.
App configuration file (optional)
Provide this if your needs exceed the default Octue Dockerfile - read more...
Octue services run in a Docker container if they are deployed. They can also run this way locally. The SDK provides a default
Dockerfilefor these purposes that will work for most cases:
However, you may need to write and provide your own
Dockerfileif your app requires:
Non-python or system dependencies (e.g.
Python dependencies that aren’t installable via
Private python packages
Here are two examples of a custom
Dockerfilethat use different base images:
If you do provide one, you must specify its path in
As always, if you need help with this, feel free to drop us a message or raise an issue!
- Service revision
A specific instance of an Octue service that can be individually addressed. The revision could correspond to a version of the service, a dynamic development branch for it, or a deliberate duplication or variation of it.
- Service revision unique identifier (SRUID)
The combination of a service revisions’s namespace, name, and revision tag that uniquely identifies it. For example,
octue/my-service:1.3.0where the namespace is
octue, the name is
my-service, and the revision tag is
- Service namespace
The group to which the service belongs e.g. your name or your organisation’s name. If in doubt, use the GitHub handle of the user or organisation publishing the services.
Namespaces must be lower kebab case (i.e. they may contain the letters [a-z], numbers [0-9], and hyphens [-]). They may not begin or end with hyphens.
- Service name
A name to uniquely identify the service within its namespace. This usually corresponds to the name of the GitHub repository for the service. Names must be lower kebab case (i.e. they may contain the letters [a-z], numbers [0-9] and hyphens [-]). They may not begin or end with hyphens.
- Service revision tag
A tag that uniquely identifies a particular revision of a service. The revision tag could correspond to a commit hash like
a3eb45, a release number like
0.12.4, a branch name (e.g.
development), a particular environment the service is deployed in (e.g.
production), or a combination like
0.12.4-production. Tags may contain lowercase and uppercase letters, numbers, underscores, periods, and hyphens, but can’t start with a period or a dash. They can contain a maximum of 128 characters. These requirements are the same as the Docker tag format.
- Service ID
The SRUID is a special case of the service ID. A service ID can be an SRUID or just the service namespace and name. It can be used to ask a question to a service without specifying a specific revision of it. This enables asking questions to, for example, the service
octue/my-serviceand automatically having them routed to its latest revision. Note that this will be a future feature; currently, you will still be required to provide a revision tag (i.e. a full SRUID).
Where to specify the namespace, name, and revision tag
OCTUE_SERVICE_NAMESPACEenvironment variable (takes priority)
OCTUE_SERVICE_NAMEenvironment variable (takes priority)
Default: a random “coolname” (e.g.
octue startcommand, the
--revision-tagoption (takes priority)
We’ve created some template apps for you to look at and play around with. We recommend going through them in this order:
The fractal app template - introduces a basic Octue service that returns output values to its parent.
The using-manifests app template - introduces using a manifest of output datasets to return output files to its parent.
The child-services app template - introduces asking questions to child services and using their answers to form an output to return to its parent.
Deploying services automatically
Automated deployment with Octue means:
Your service runs in Google Cloud, ready to accept questions from and return answers to other services.
You don’t need to do anything to update your deployed service with new code changes - the service simply gets rebuilt and re-deployed each time you push a commit to your
mainbranch, or merge a pull request into it (other branches and deployment strategies are available, but this is the default).
Serverless is the default - your service only runs when questions from other services are sent to it, meaning there is no cost to having it deployed but not in use.
To enable automated deployments, contact us so we can create a Google Cloud Build trigger linked to your git repository. This requires no work from you apart from authorising the connection to GitHub (or another git provider).
If you want to deploy services yourself, see here.