Code

Introduction

The ClearBlade IoT Platform utilizes a microservices model powered by a Code Engine. This Code Engine is JavaScript, and has several built-in integrations to interact with your IoT assets. Adhering to microservice practices, a new execution context is sandboxed, created upon invocation, and once complete, is destroyed.

Within the context of the ClearBlade IoT Platform, a Service is synonymous with a microservice, and will be used extensively across the documentation.

There are two types of Code assets:

For a tutorial on how to set up code services, click here

Purpose

Microservices are a software development technique that allow for a high degree of modularity, scale, and paralleization - all of which are essential for effectively building and running full-scale IoT Applications.

Libraries

Libraries are reusable sets of code that can be imported by one or more code services.

Native Libraries

Native Libraries are available in every System, and have an underlying implementation in Golang. Each library is optimized for computational performance.

These libraries can be imported using Requires

Library Desc Docs
clearblade Interacts with Collections, Messaging, Users, Edges, Devices View docs
log Appends messages to a log View docs
http Makes outbound HTTP/S requests View docs
geo Geospatial Computation Library View docs
crypto Creates cryptographical hashes View docs
file_writer Write files to host filesystem View docs
jsutil JS Utility Library View docs
mailer Send email directly from host machine View docs
net_cb Net library View docs
oauth2_lib OAuth Utility Library View docs

Custom Libraries

A developer can create his or her own custom library with custom logic.

Services

A Code Service, or Service, is a short-lived microservices that is expected to complete in under 30 seconds.

Lifecycle

Invocation

A Service can be invoked in any of the three ways:

  1. A REST request to the API Endpoint
  2. Trigger Event
  3. Timer Event

Our SDKs can be used as a REST Client to accomplish the first REST option

Context

Each Code Service service has a name, and that name corresponds to an autogenerated function name.

For example, a code service named analyze will have an autogenerated function like so:

function analyze(req, resp) {}

This function will have two parameters, similar to Node’s Express library:

req

req is an object containing the following contextual values:

key desc type example
userEmail Email of user who invoked the service string “user@clearblade.com”
params JSON with parameters, structure is based upon type of Invocation Object {“key”:“value”}
systemKey System Key for system in which code service is running string e095a19d0b94e48dee9bff87e5fd01”
systemSecret System Secret for system in which code service is running string “E095A19D0B9ACD9EFA86AEEDFCC001”
userToken Invoking user’s OAuth token string “U1hNa7o4bEXpiE4aGhb-79MQv4…”
isLogging Whether logging is enabled boolean true
userid internal uid associated with invoking user string “fc8cfcea0a96ebebc7f5d4edd414”

resp

resp is an object that is used to return values and exit the Code Service with two methods:

resp.success = function (object | string){} resp.error = function (object | string ){}

function analyze(req, resp) {
  var message = "Everything went well!";
  resp.success(output); // Service exits successfully with "Everything went well!"
}

function analyze(req, resp) {
  var message = "It broke!";
  resp.error(error); // Service exits with failure with "It broke!"
}

Exit

The Code Service will exit executing either resp.success or resp.error.

Note: By default, if no exit method is called and execution reaches the end, resp.success will be called to exit

Requires

A Code Service can import a Library using the Requires feature. In the Service settings, you can select one or more libraries to import, and that code is accessible from within the Code Service.


The same library can be used for multiple code services

Debugging

There is no native support for debugging line-by-line. Recommended to use log method extensively, like so:

function ServiceA(req, resp) {
  log("Something happened");
  resp.success("All done");
}

Events

Events are actions or occurrences that happen in the ClearBlade System, response to them through execution of the associated Code Service. ClearBlade supports two types of events:

Timers

Timers provide a mechanism where a ClearBlade code service can be scheduled to execute at certain time intervals. Timers are similar to Unix cron jobs, with a similar scheduling mechanism.

Some example Timer configurations:

  • Run forever every five minutes starting now
  • Run 26 times every two weeks starting Friday, December 25th, 2018
  • Run once at midnight

Triggers

Triggers are a mechanism that tie an action, called a Trigger Source, to the invocation of a code service. The trigger source and payload information is passed into a code-service as req.params.When creating a trigger, add log(req) to the Code Editor to log the payload information.

"params": {
    "topic": "ClearBladeTopic",
    "body": "214.312",
    "userId": "8e89e2af0fc9cbeadfd6afb501",
    "trigger": "Messaging::Publish"
}

Trigger Source

When you define a trigger, you provide the following information:

Asset Category Action
Messaging Publish Publish
Messaging Subscriptions Upon Subscribe
Messaging Subscriptions Upon Unsubscribe
Messaging Connection User Connected
Messaging Connection User Disconnected
Messaging Connection Device Connected
Messaging Connection Device Disconnected
Data Table Created
Data Table Updated
Data Table Deleted
Data Item Created
Data Item Updated
Data Item Deleted
User n/a Created
User n/a Updated
User n/a Deleted
Device n/a Created
Device n/a Updated
Device n/a Deleted
Edge/Platform Platform Platform Started
Edge/Platform Platform Platform Connected on Edge
Edge/Platform Platform Platform Disconnected on Edge
Edge/Platform Edge Edge Started
Edge/Platform Edge Edge Connected on Platform
Edge/Platform Edge Edge Disconnected on Platform

Example Object Response

Ensure the relevant permissions for code service are set before using triggers.

Following is a list of categories grouped by action types,

Messaging (Publish/Subscribe/Unsubscribe):

When there is a Publish/Subscribe/Unsubscribe made to the ClearBlade’s MQTT broker.

// Template
{
  "params": {
      "topic": "<message_topic>",
      "body": "<message_body>",
      "userId": "<user_id>",
      "trigger": "Messaging::<action_type>"
  }
}

// Example
{
  "params": {
      "topic": "ClearBladeTopic",
      "body": "example message",
      "userId": "8e89e2af0fc9cbeadfd6afb501",
      "trigger": "Messaging::Publish"
  }
}

Messaging (UserConnected/UserDisconnected):

When an user Connects/Disconnects to the ClearBlade’s MQTT broker.

// Template
{
  "params": {
      "email": "<user_email>",
      "trigger": "Messaging::<action_type>"
  }
}

// Example

{
  "params": {
    "email": "example@clearblade.com",
    "trigger": "Messaging::MQTTUserDisconnected"
  }
}

Messaging (DeviceConnected/DeviceDisconnected):

When an device Connects/Disconnects to the ClearBlade’s MQTT broker.

// Template

{
  "params": {
    "deviceName": "<device_name>",
    "trigger": "Messaging::<action_type>"
  }
}

// Example

{
  "params": {
    "deviceName": "exampleDevice",
    "trigger": "Messaging::MQTTDeviceDisconnected"
  }
}

Data-Table (CollectionCreated/CollectionUpdated/CollectionDeleted):

// Template 

{
  "params": {
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::<action_type>"
  }
}

// Example

{
  "params": {
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::CollectionCreated"
  }
}

Data (ItemCreated):

// Template

{
  "params": {
    "items": [
      {
        "item_id": "<item_id>"
      }
    ],
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::ItemCreated"
  }
}

//Example

{
  "params": {
    "items": [
      {
        "item_id": "083196bf-8059-46c0-8280-38234aa73fc1"
      }
    ],
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::ItemCreated"
  }
}

Data (ItemUpdated/ItemDeleted):

// Template

{
  "params": {
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::<action_type>",
    "items": [
    // First row
    // This includes all of the data from the row
      {
        "<column1_name>": <column1_data>,
        "item_id": "<item_id1>",
        "<column2_name>": <column2_data>
      }
    // A second row
      {
        "<column1_name>": <column1_data>,
        "item_id": "<item_id1>",
        "<column2_name>": <column2_data>
      }
    ]
  }
}

// Example

{
  "params": {
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::ItemUpdated",
    "items": [
      {
        "hi": 6,
        "item_id": "14b12adf-4c32-4740-8461-546daa660393",
        "low": 3
      },
      {
        "hi": 7,
        "item_id": "3700fa91-ef49-41e8-9fe7-64e9ee6aa3b2",
        "low": 5
      }
    ]
  }
}

User (UserCreated)

// Template

{
  "params": {
    "user": {
      "creation_date": <date_time>,
      "email": "<user_email>",
      "user_id": "<user_id>"
    },
    "trigger": "User::UserCreated"
  }
}

//Example

{
  "params": {
    "user": {
      "creation_date": 2019-06-20T19:43:10Z,
      "email": "example@clearblade.com",
      "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
    },
    "trigger": "User::UserCreated"
  }
}

User (UserUpdated/UserDeleted)

// Template

{
  "params": {
    "query": {
      "FILTERS": [
        [
          {
            "EQ": {
              "user_id": "<user_id>"
            }
          }
        ]
      ],
      "PAGENUM": <page_number>,
      "PAGESIZE": <page_size>,
      "SELECTCOLUMNS": <columns_selected>,
      "SORT": []
    },
    // Includes all of the data from the row
    "user": {
      "email": "<user_email>"
      <column1_name>: <row_data>
      <column2_name>: <row_data>
      "user_id": "<user_id>"
    },
    "trigger": "User::<action_type>"
  }
}

// Example

{
  "params": {
    "query": {
      "FILTERS": [
        [
          {
            "EQ": {
              "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
            }
          }
        ]
      ],
      "PAGENUM": 0,
      "PAGESIZE": 0,
      "SELECTCOLUMNS": null,
      "SORT": []
    },
    "user": {
      "email": "example@clearblade.com"
      "name": "Example"
      "age": 22
      "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
    },
    "trigger": "User::UserUpdated"
  }
}

Device (DeviceCreated/DeviceDeleted)

// Template

{
  "params": {
    "trigger": "Device::<action_type>",
    "device": {
      "allow_certificate_auth": <boolean>,
      "allow_key_auth": <boolean>,
      "certificate": "<certificate>",
      "created_date":  <date_time>,
      "description": "<device_description>",
      "device_key": "<device_key> :: <device_name>",
      "enabled": <boolean>,
      "last_active_date":  <date_time>,
      "<columnX_name>": "<columnX_data>",
      "<columnY_name>": "<columnY_data>",
      "system_key": "<system_key>",
      "type": "<device_type>"
    },
    "deviceName": "<device_name>"
  }
}

//Example 

{
  "params": {
    "trigger": "Device::DeviceCreated",
    "device": {
      "allow_certificate_auth": true,
      "allow_key_auth": true,
      "certificate": "",
      "created_date":  2019-06-20T19:43:10Z,
      "description": "",
      "device_key": "ccafb4cf0bd0dcbcadaccaf9ebba01 :: ExampleDevice",
      "enabled": true,
      "last_active_date":  2019-06-20T19:43:10Z,
      "name": "ExampleDevice",
      "state": "active",
      "system_key": "ccafb4cf0bd0dcbcadaccaf9ebba01",
      "type": "Sensor"
    },
    "deviceName": "ExampleDevice"
  }
}

Device (DeviceUpdated)

  // Template

  {
    "params": {
      "changes": {
        "description": "<device_description>"
      },
      "deviceName": "<device_name>",
      "trigger": "Device::DeviceUpdated"
    }
  }

  // Example
  
{
  "params": {
    "changes": {
      "description": "device description"
    },
    "deviceName": "example_device",
    "trigger": "Device::DeviceUpdated"
  }
}

Edge/Platform

  // Template

  {
    "params": {
      "edgeName": "edge_name",
      "trigger": "StartConnectDisconnect::<action_type>"
    }
  }

  // Example

  {
    "params": {
      "edgeName": "example",
      "trigger": "StartConnectDisconnect::<Edge Started>"
    }
  }

Webhooks

A webhook is a mechanism that allows you to execute a code service by targeting the public endpoint above. Any HTTP method, such as GET, POST, PUT, DELETE, can be used and will result in the same action, executing the service. The URL we provide can be used by third parties to push real-time data to ClearBlade’s server. A code service can be invoked by multiple webhooks and they are syncable. See tutorial.

Authorization

These requests are to be added when executing a webhook.

Auth Method Request
clearblade_auth ClearBlade-Usertoken: <USER_TOKEN> See Example
http_basic_auth Authorization: Basic <USER_TOKEN> See Example
payload_auth (GET) Request Body: “token”: <USER_TOKEN> See Example
payload_auth (POST) Query String: token=<USER_TOKEN> See Example

Authorization Examples

ClearBlade Auth

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c80f79d0a/ExampleWebhook",
        headers:{
            "ClearBlade-UserToken":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErDNRpSNJsD5w=="
        },
        body:{
            "data":"hello"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    })
}

HTTP Basic Auth

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10bbae683c7ace4dfd701/Webhook_2_HTTP_Auth",
        headers:{
            "Authorization":"Basic Id19kSWr2vkB4CUc5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w=="
        },
        body:{
            "data":"Authenticated using Http Auth"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Payload Auth GET

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c19ff79d0a/Webhook_3_Payload_Auth",
        qs:{
            "token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67ELmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w==",
            "data":"Authenticated using Get method"
        }
    }
    //Using get method
    http.get(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Payload Auth POST

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10b8ebae3c7ace4dfd701/Webhook_3_Payload_Auth",
        body:{
            "token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUDnYu52XGErPEjDNRpSNJsD5w==",
            "data":"Authenticated using Http Payload"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Advanced

Execution Timeout

Code Services are usually used as microservices, and expected to have a lifecycle of under 30 seconds. However, this execution timeout can be extended to make the services run longer. When execution timeout is set to infinity(“Never” in the console) it’s called a Stream Service, meaning the code service will run indefinitely. This allows for the following high performance use cases:

  • Exponentially faster rate for any operation which is otherwise performed in an event triggered micro-service.
  • Users can also design stream services to perform bulk operations on collections.
  • Data generation

Concurrency

This property is valid when the service is a stream service. It allows the developer to cap the maximum number of instances of code services that can run on each of the ClearBlade node. Any more than this number will kill the oldest code service instance.

Run as User

A code service can be invoked on the behalf of a certain user. If Code Services needs escalated privileges, the developer can use this ‘Run as User’ property to set an user with escalated permissions to invoke code services. This is usually done when the code service accesses/updates certain system assets to which only an user with a certain role has access, but the code service is desiged to be invoked by an user with any other role.

Auto Restart

This property is valid when the service is a Stream Service. When the box is checked, it restarts service automatically if it stops for any reason, including successes.

Best Practice : Do not check the box until fully testing the service. There could be undesired deadlocks if there are errors in the code.

Auto Balance

This property is valid when the service is a Stream Service. The platform automatically starts a service on each available node with the number of concurrency instances. It kills and restarts services when saved with a new code.

If auto_balance = true :

  • req.params are not allowed
  • A normal code service must be developed first to confirm that it is functioning properly before developing it as a stream service. Do not check the auto balance box until the code service works.

Failed Runs

If a Code Service exits with resp.error, or throws an uncaught error, it will be stored in Failed Runs. A developer can then fix any issues, and retry that failed run with the same input parameters.

Standard Javascript Functions (es5.1)

Method not supported Alternative
setTimeout(),setInterval() Not supported at all
atob(), btoa() buffer-node-js library
promises q-promise-library