Octue SDK (Python)

The python SDK for Octue Twined scientific data services and digital twins - get faster data groundwork so you have more time for the science!

Definition

Octue Twined service

A data service or digital twin built on the octue SDK that can be asked questions, process them, and return answers. Twined services can communicate with each other with minimal extra setup.

Key features

Unified cloud/local file, dataset, and manifest operations

  • Create and build datasets easily

  • Organise them with timestamps, labels, and tags

  • Filter and combine them using this metadata

  • Store them locally or in the cloud (or both for low-latency reading/writing with cloud-guaranteed data availability)

  • Use internet/cloud-based datasets as if they were local e.g.

    • https://example.com/important_dataset.dat

    • gs://example-bucket/important_dataset.dat

  • Create manifests (a set of datasets needed for a particular analysis) to modularise your dataset input/output

Ask existing services questions from anywhere

  • Send them data to process from anywhere

  • Automatically have their logs, monitor messages, and any errors forwarded to you and displayed as if they were local

  • Receive their output data as JSON

  • Receive a manifest of any output datasets they produce for you to download or access as you wish

Create, run, and deploy your apps as services

  • No need to change your app - just wrap it

  • Use the octue CLI to run your service locally or deploy it to Google Kubernetes Engine (GKE)

  • Create JSON-schema interfaces to explicitly define the form of configuration, input, and output data

  • Ask other services questions as part of your app (i.e. build trees of services)

  • Automatically display readable, colourised logs, or use your own log handler

  • Avoid time-consuming and confusing devops, cloud configuration, and backend maintenance

High standards, quick responses, and good intentions

  • Open-source and transparent on GitHub - anyone can see the code and raise an issue

  • Automated testing, standards, releases, and deployment

  • High test coverage

  • Works on MacOS, Linux, and Windows

  • Developed not-for-profit for the renewable energy industry

Need help, found a bug, or want to request a new feature?

We use GitHub Issues [1] to manage:

  • Bug reports

  • Feature requests

  • Support requests

Footnotes