Code

Introduction

The ClearBlade IoT Platform utilizes a microservices model powered by a Code Engine. This Code Engine is JavaScript, and has several built-in integrations to interact with your IoT assets. Adhering to microservice practices, a new execution context is sandboxed, created upon invocation, and once complete, is destroyed.

There are four types of Code assets:

• Micro Service: A short-lived service which is expected to complete within a fixed period of time, default 30s
• Stream Service: A service configured to continuously to handle requests with an infinite execution timeout.
• Configuration: A library that contains a key-value object to be used as constants.
• Library: A custom javascript library that can be imported into one or more code services

For a tutorial on how to set up code services, click here

Purpose

Microservices are a software development technique that allow for a high degree of modularity, scale, and parallelization - all of which are essential for effectively building and running full-scale IoT Applications.

Libraries

Libraries are reusable sets of code that can be imported by one or more code services.

Native Libraries

Native Libraries are available in every System, and have an underlying implementation in Golang. Each library is optimized for computational performance.

These libraries can be imported using Requires

Library	Desc	Docs
clearblade	Interacts with Collections, Messaging, Users, Edges, Devices	View docs
log	Appends messages to a log	View docs
http	Makes outbound HTTP/S requests	View docs
geo	Geospatial Computation Library	View docs
crypto	Creates cryptographical hashes	View docs
file_writer	Write files to host filesystem	View docs
jsutil	JS Utility Library	View docs
mailer	Send email directly from host machine	View docs
net_cb	Net library	View docs
oauth2_lib	OAuth Utility Library	View docs

Custom Libraries

A developer can create his or her own custom library with custom logic.

Services

A Code Service, or Service, is a short-lived microservices that is expected to complete in under 30 seconds.

Lifecycle

Invocation

A Service can be invoked in any of the three ways:

1. A REST request to the API Endpoint
2. Trigger Event
3. Timer Event

Our SDKs can be used as a REST Client to accomplish the first REST option

Context

Each Code Service service has a name, and that name corresponds to an autogenerated function name.

For example, a code service named analyze will have an autogenerated function like so:

function analyze(req, resp) {}

This function will have two parameters, similar to Node’s Express library:

req

req is an object containing the following contextual values:

key	desc	type	example
userEmail	Email of user who invoked the service	string	“user@clearblade.com”
params	JSON with parameters, structure is based upon type of Invocation	Object	{“key”:“value”}
systemKey	System Key for system in which code service is running	string	e095a19d0b94e48dee9bff87e5fd01”
systemSecret	System Secret for system in which code service is running	string	“E095A19D0B9ACD9EFA86AEEDFCC001”
userToken	Invoking user’s OAuth token	string	“U1hNa7o4bEXpiE4aGhb-79MQv4…”
isLogging	Whether logging is enabled	boolean	true
userid	internal uid associated with invoking user	string	“fc8cfcea0a96ebebc7f5d4edd414”

resp

resp is an object that is used to return values and exit the Code Service. Following methods are available in the resp object:

• resp.success
• resp.error
• resp.set
• resp.status
• resp.send

resp.success

resp.success = function (object | string){}

function analyze(req, resp) {
  var message = "Everything went well!";
  resp.success(output); // Service exits successfully with "Everything went well!"
}

resp.error

resp.error = function (object | string ){}

function analyze(req, resp) {
  var message = "It broke!";
  resp.error(error); // Service exits with failure with "It broke!"
}

resp.set This method has multiple signatures:

• resp.set = function(string, string){}

resp.set(keyString, valueString): adds or updates a key in the header to the HTTP response from a service. Example: resp.set("Content-Type", "text/html")

• resp.set = function(object){}
• resp.set({}): will set the response headers to those in the map. It replaces the header object so it removes any headers added with resp.set(k,v). Example:

resp.set({
  "Content-Type": "application/json",
  "ClearBlade-UserToken: req.userToken"
})

resp.status

resp.status = function(int){}

resp.status(int): changes the HTTP response code from a service. Example: resp.status(400)

resp.send

resp.send = function(object|string){} resp.send(value): exit the service and send the value back as the body of the response. value will be stringified if not already a string. value will be sent as-is and not wrapped in our typical json response object. Example: resp.send({valueA: "ABC-123-DEF", valueB: 123})

Exit

The Code Service will exit executing either resp.success or resp.error.

Note: By default, if no exit method is called and execution reaches the end, resp.success will be called to exit

Logs

Logs record and store the history of service responses. Each service can store up to 25 of the latest logs with the max size of 100KB each.

Requires

A Code Service can import a Library using the Requires feature. In the Service settings, you can select one or more libraries to import, and that code is accessible from within the Code Service.

The same library can be used for multiple code services

Debugging

There is no native support for debugging line-by-line. Recommended to use log method extensively, like so:

function ServiceA(req, resp) {
  log("Something happened");
  resp.success("All done");
}

Events

Events are actions or occurrences that happen in the ClearBlade System, response to them through execution of the associated Code Service. ClearBlade supports two types of events:

• Timers
• Triggers

Timers

Timers provide a mechanism where a ClearBlade code service can be scheduled to execute at certain time intervals. Timers are similar to Unix cron jobs, with a similar scheduling mechanism.

Some example Timer configurations:

• Run forever every five minutes starting now
• Run 26 times every two weeks starting Friday, December 25th, 2018
• Run once at midnight

Triggers

Triggers are a mechanism that tie an action, called a Trigger Source, to the invocation of a code service. The trigger source and payload information is passed into a code-service as req.params.When creating a trigger, add log(req) to the Code Editor to log the payload information.

"params": {
    "topic": "ClearBladeTopic",
    "body": "214.312",
    "userId": "8e89e2af0fc9cbeadfd6afb501",
    "trigger": "Messaging::Publish"
}

Event Source

The platform publishes for every event regardless of whether the event is created programmatically in a service or created externally in the console.

$trigger and $timer are custom ClearBlade topics.

For more information on publishing event topics, click here

Asset	Category	Action	Topic
Messaging	Publish	Publish	$trigger/messaging/publish
Messaging	Subscriptions	Upon Subscribe	$trigger/messaging/subscribe
Messaging	Subscriptions	Upon Unsubscribe	$trigger/messaging/unsubscribe
Messaging	Connection	User Connected	$trigger/messaging/user/connected
Messaging	Connection	User Disconnected	$trigger/messaging/user/disconnected
Messaging	Connection	Device Connected	$trigger/messaging/device/connected
Messaging	Connection	Device Disconnected	$trigger/messaging/device/disconnected
Data	Table	Created	$trigger/collection/created
Data	Table	Updated	$trigger/collection/updated
Data	Table	Deleted	$trigger/collection/deleted
Data	Item	Created	$trigger/data/created
Data	Item	Updated	$trigger/data/updated
Data	Item	Deleted	$trigger/data/deleted
User	n/a	Created	$trigger/user/created
User	n/a	Updated	$trigger/user/updated
User	n/a	Deleted	$trigger/user/deleted
Device	n/a	Created	$trigger/device/created
Device	n/a	Updated	$trigger/device/updated
Device	n/a	Deleted	$trigger/device/deleted
Edge/Platform	Platform	Platform Started	$trigger/startstopdisconnect/platformstarted
Edge/Platform	Platform	Platform Connected on Edge	$trigger/startstopdisconnect/platformconnectedonedge
Edge/Platform	Platform	Platform Disconnected on Edge	$trigger/startstopdisconnect/platformdisconnectedonedge
Edge/Platform	Edge	Edge Started	$trigger/startstopdisconnect/edgestarted
Edge/Platform	Edge	Edge Connected on Platform	$trigger/startstopdisconnect/edgeconnectedonplatform
Edge/Platform	Edge	Edge Disconnected on Platform	$trigger/startstopdisconnect/edgedisconnectedonplatform
Timer	n/a	Create Timer	$timer/{timerName}

Examples of Incoming Object

Ensure the relevant permissions for code service are set before using triggers and timers.

Following is a list of categories grouped by action types, showing structures for incoming object:

Messaging (Publish/Subscribe/Unsubscribe):

When there is a Publish/Subscribe/Unsubscribe made to the ClearBlade’s MQTT broker.

// Template
{
  "params": {
      "topic": "<message_topic>",
      "body": "<message_body>",
      "userId": "<user_id>",
      "trigger": "Messaging::<action_type>"
  }
}

// Example
{
  "params": {
      "topic": "ClearBladeTopic",
      "body": "example message",
      "userId": "8e89e2af0fc9cbeadfd6afb501",
      "trigger": "Messaging::Publish"
  }
}

Messaging (UserConnected/UserDisconnected):

When a user Connects/Disconnects to the ClearBlade’s MQTT broker.

// Template
{
  "params": {
      "email": "<user_email>",
      "trigger": "Messaging::<action_type>"
  }
}

// Example

{
  "params": {
    "email": "example@clearblade.com",
    "trigger": "Messaging::MQTTUserDisconnected"
  }
}

Messaging (DeviceConnected/DeviceDisconnected):

When a device Connects/Disconnects to the ClearBlade’s MQTT broker.

// Template

{
  "params": {
    "deviceName": "<device_name>",
    "trigger": "Messaging::<action_type>"
  }
}

// Example

{
  "params": {
    "deviceName": "exampleDevice",
    "trigger": "Messaging::MQTTDeviceDisconnected"
  }
}

Data-Table (CollectionCreated/CollectionUpdated/CollectionDeleted):

// Template 

{
  "params": {
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::<action_type>"
  }
}

// Example

{
  "params": {
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::CollectionCreated"
  }
}

Data (ItemCreated):

// Template

{
  "params": {
    "items": [
      {
        "item_id": "<item_id>"
      }
    ],
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::ItemCreated"
  }
}

//Example

{
  "params": {
    "items": [
      {
        "item_id": "083196bf-8059-46c0-8280-38234aa73fc1"
      }
    ],
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::ItemCreated"
  }
}

Data (ItemUpdated/ItemDeleted):

// Template

{
  "params": {
    "collectionId": "<collection_id>",
    "collectionName": "<collection_name>",
    "trigger": "Data::<action_type>",
    "items": [
    // First row
    // This includes all of the data from the row
      {
        "<column1_name>": <column1_data>,
        "item_id": "<item_id1>",
        "<column2_name>": <column2_data>
      }
    // A second row
      {
        "<column1_name>": <column1_data>,
        "item_id": "<item_id1>",
        "<column2_name>": <column2_data>
      }
    ]
  }
}

// Example

{
  "params": {
    "collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
    "collectionName": "ExampleCollection",
    "trigger": "Data::ItemUpdated",
    "items": [
      {
        "hi": 6,
        "item_id": "14b12adf-4c32-4740-8461-546daa660393",
        "low": 3
      },
      {
        "hi": 7,
        "item_id": "3700fa91-ef49-41e8-9fe7-64e9ee6aa3b2",
        "low": 5
      }
    ]
  }
}

User (UserCreated)

// Template

{
  "params": {
    "user": {
      "creation_date": <date_time>,
      "email": "<user_email>",
      "user_id": "<user_id>"
    },
    "trigger": "User::UserCreated"
  }
}

//Example

{
  "params": {
    "user": {
      "creation_date": 2019-06-20T19:43:10Z,
      "email": "example@clearblade.com",
      "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
    },
    "trigger": "User::UserCreated"
  }
}

User (UserUpdated/UserDeleted)

// Template

{
  "params": {
    "query": {
      "FILTERS": [
        [
          {
            "EQ": {
              "user_id": "<user_id>"
            }
          }
        ]
      ],
      "PAGENUM": <page_number>,
      "PAGESIZE": <page_size>,
      "SELECTCOLUMNS": <columns_selected>,
      "SORT": []
    },
    // Includes all of the data from the row
    "user": {
      "email": "<user_email>"
      <column1_name>: <row_data>
      <column2_name>: <row_data>
      "user_id": "<user_id>"
    },
    "trigger": "User::<action_type>"
  }
}

// Example

{
  "params": {
    "query": {
      "FILTERS": [
        [
          {
            "EQ": {
              "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
            }
          }
        ]
      ],
      "PAGENUM": 0,
      "PAGESIZE": 0,
      "SELECTCOLUMNS": null,
      "SORT": []
    },
    "user": {
      "email": "example@clearblade.com"
      "name": "Example"
      "age": 22
      "user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
    },
    "trigger": "User::UserUpdated"
  }
}

Device (DeviceCreated/DeviceDeleted)

// Template

{
  "params": {
    "trigger": "Device::<action_type>",
    "device": {
      "allow_certificate_auth": "<boolean>",
      "allow_key_auth": "<boolean>",
      "certificate": "<certificate>",
      "created_date":  "<date_time>",
      "description": "<device_description>",
      "device_key": "<device_key> :: <device_name>",
      "enabled": "<boolean>",
      "last_active_date":  "<date_time>",
      "<columnX_name>": "<columnX_data>",
      "<columnY_name>": "<columnY_data>",
      "system_key": "<system_key>",
      "type": "<device_type>"
    },
    "deviceName": "<device_name>"
  }
}

//Example 

{
  "params": {
    "trigger": "Device::DeviceCreated",
    "device": {
      "allow_certificate_auth": true,
      "allow_key_auth": true,
      "certificate": "",
      "created_date":  2019-06-20T19:43:10Z,
      "description": "",
      "device_key": "ccafb4cf0bd0dcbcadaccaf9ebba01 :: ExampleDevice",
      "enabled": true,
      "last_active_date":  2019-06-20T19:43:10Z,
      "name": "ExampleDevice",
      "state": "active",
      "system_key": "ccafb4cf0bd0dcbcadaccaf9ebba01",
      "type": "Sensor"
    },
    "deviceName": "ExampleDevice"
  }
}

Device (DeviceUpdated)

  // Template

  {
    "params": {
      "changes": {
        "description": "<device_description>"
      },
      "deviceName": "<device_name>",
      "trigger": "Device::DeviceUpdated"
    }
  }

  // Example
  
{
  "params": {
    "changes": {
      "description": "device description"
    },
    "deviceName": "example_device",
    "trigger": "Device::DeviceUpdated"
  }
}

Edge/Platform

  // Template

  {
    "params": {
      "edgeName": "edge_name",
      "trigger": "StartConnectDisconnect::<action_type>"
    }
  }

  // Example

  {
    "params": {
      "edgeName": "example",
      "trigger": "StartConnectDisconnect::<Edge Started>"
    }
  }

Webhooks

A webhook is a mechanism that allows you to execute a code service by targeting a public endpoint. Any HTTP method, such as GET, POST, PUT, DELETE, can be used and will result in the same action, executing the service. The URL we provide can be used by third parties to push real-time data to ClearBlade’s server. A code service can be invoked by multiple webhooks and they are syncable. See tutorial.

Authorization

These requests are to be added when executing a webhook.

Auth Method	Request
clearblade_auth	ClearBlade-Usertoken: <USER_TOKEN>	See Example
http_basic_auth	Authorization: Basic <USER_TOKEN>	See Example
payload_auth (GET)	Query String: token=<USER_TOKEN>	See Example
payload_auth (POST)	Request Body: “token”: <USER_TOKEN>	See Example
no_auth	No Authorization Required. The “Anonymous” role should have Run permission checked for the service. This allows for the no auth webhook to work.	See Example

Authorization Examples

ClearBlade Auth

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c80f79d0a/ExampleWebhook",
        headers:{
            "ClearBlade-UserToken":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErDNRpSNJsD5w=="
        },
        body:{
            "data":"hello"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    })
}

HTTP Basic Auth

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10bbae683c7ace4dfd701/Webhook_2_HTTP_Auth",
        headers:{
            "Authorization":"Basic Id19kSWr2vkB4CUc5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w=="
        },
        body:{
            "data":"Authenticated using Http Auth"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Payload Auth GET

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c19ff79d0a/Webhook_3_Payload_Auth",
        qs:{
            "token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67ELmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w==",
            "data":"Authenticated using Get method"
        }
    }
    //Using get method
    http.get(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Payload Auth POST

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10b8ebae3c7ace4dfd701/Webhook_3_Payload_Auth",
        body:{
            "token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUDnYu52XGErPEjDNRpSNJsD5w==",
            "data":"Authenticated using Http Payload"
        }
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

No Auth

function ExampleWebhookInvoker(req,resp){
    // These are parameters passed into the code service
    var params = req.params
    var http = Requests();
    var options = {
        uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10b8ebae3c7ace4dfd701/Webhook_4_No_Auth"
    }
    http.post(options, function(err, response){
        if(err){
            resp.error("Failed to call the webhook");
        }
        resp.success("Success"+ response);
    });
}

Advanced

Execution Timeout

Code Services are usually used as microservices, and expected to have a lifecycle of under 30 seconds. However, this execution timeout can be extended to make the services run longer. When execution timeout is set to infinity(“Never” in the console) it’s called a Stream Service, meaning the code service will run indefinitely. This allows for the following high performance use cases:

• Exponentially faster rate for any operation which is otherwise performed in an event triggered micro-service.
• Users can also design stream services to perform bulk operations on collections.
• Data generation

Best Practices:

• For Stream Services, set Execution Timeout to Never

• For Micro Service, set Execution Timeout to its default of Infinite

Concurrency

Concurrency allows the developer to cap the maximum number of instances of code services that can run on each of the ClearBlade node. Any more than this number will kill the oldest code service instance.

Best Practices:

• For Stream Services, set Concurrency to Other, and then set the number of instances you need. Recommended is 4.

  • For Micro Service, set Concurrency to its default of Infinite, to optimize performance

Run as

A service can have permissions to invoke as a specific user role. If a run as parameter is set, it runs as user that is stated in the parameter, regardless of the user that invokes it. If the user set in the run as parameter doesn’t have permissions to execute the service (and the permissions to execute all the operations performed within the code service), then the service will fail. If the user invoking the service does not have the permission to execute the code service, then the service will fail as well. For example: An anonymous user can invoke and run a service as a different user.

Run as User

A code service can be invoked on the behalf of a certain user. It can also have permissions to invoke as a specific user role. If Code Services needs escalated privileges, the developer can use this ‘Run as User’ property to set an user with escalated permissions to invoke code services. This is usually done when the code service accesses/updates certain system assets to which only an user with a certain role has access, but the code service is desiged to be invoked by an user with any other role.

Auto Restart

This property is valid when the service is a Stream Service. When the box is checked, it restarts service automatically if it stops for any reason, including successes.

Best Practice : Do not check the box until fully testing the service. There could be undesired deadlocks if there are errors in the code.

Auto Balance

This property is valid when the service is a Stream Service. The platform automatically starts a service on each available node with the number of concurrency instances. It kills and restarts services when saved with a new code.

If auto_balance = true :

• req.params are not allowed
• A normal code service must be developed first to confirm that it is functioning properly before developing it as a stream service. Do not check the auto balance box until the code service works.

Failed Runs

If a Code Service exits with resp.error, or throws an uncaught error, it will be stored in Failed Runs. A developer can then fix any issues, and retry that failed run with the same input parameters.

Standard Javascript Functions (es5.1)

Method	Usage
setTimeout(),setInterval()	Available natively in version 9.3.0 or later
atob(), btoa()	buffer-node-js library
promises	q-promise-library/Available natively in version 9.3.0 or later