Code
Introduction
The ClearBlade IoT Platform utilizes a microservices model powered by a Code Engine
. This Code Engine
is JavaScript, and has several built-in integrations to interact with your IoT assets. Adhering to microservice practices, a new execution context is sandboxed, created upon invocation, and once complete, is destroyed.
There are four types of Code assets:
- Micro Service: A short-lived service which is expected to complete within a fixed period of time, default 30s
- Stream Service: A service configured to continuously to handle requests with an infinite execution timeout.
- Configuration: A library that contains a key-value object to be used as constants.
- Library: A custom javascript library that can be imported into one or more code services
For a tutorial on how to set up code services, click here
Purpose
Microservices are a software development technique that allow for a high degree of modularity, scale, and parallelization - all of which are essential for effectively building and running full-scale IoT Applications.
Libraries
Libraries are reusable sets of code that can be imported by one or more code services.
Native Libraries
Native Libraries are available in every System, and have an underlying implementation in Golang. Each library is optimized for computational performance.
These libraries can be imported using Requires
Library | Desc | Docs |
---|---|---|
clearblade | Interacts with Collections, Messaging, Users, Edges, Devices | View docs |
log | Appends messages to a log | View docs |
http | Makes outbound HTTP/S requests | View docs |
geo | Geospatial Computation Library | View docs |
crypto | Creates cryptographical hashes | View docs |
file_writer | Write files to host filesystem | View docs |
jsutil | JS Utility Library | View docs |
mailer | Send email directly from host machine | View docs |
net_cb | Net library | View docs |
oauth2_lib | OAuth Utility Library | View docs |
Custom Libraries
A developer can create his or her own custom library with custom logic.
Services
A Code Service
, or Service
, is a short-lived microservices that is expected to complete in under 30 seconds.
Lifecycle
Invocation
A Service
can be invoked in any of the three ways:
- A REST request to the API Endpoint
- Trigger Event
- Timer Event
Our SDKs can be used as a REST Client to accomplish the first REST option
Context
Each Code Service service has a name, and that name corresponds to an autogenerated function name.
For example, a code service named analyze
will have an autogenerated function like so:
function analyze(req, resp) {}
This function will have two parameters, similar to Node’s Express library:
req
req
is an object containing the following contextual values:
key | desc | type | example |
---|---|---|---|
userEmail | Email of user who invoked the service | string | “user@clearblade.com” |
params | JSON with parameters, structure is based upon type of Invocation | Object | {“key”:“value”} |
systemKey | System Key for system in which code service is running | string | e095a19d0b94e48dee9bff87e5fd01” |
systemSecret | System Secret for system in which code service is running | string | “E095A19D0B9ACD9EFA86AEEDFCC001” |
userToken | Invoking user’s OAuth token | string | “U1hNa7o4bEXpiE4aGhb-79MQv4…” |
isLogging | Whether logging is enabled | boolean | true |
userid | internal uid associated with invoking user | string | “fc8cfcea0a96ebebc7f5d4edd414” |
resp
resp
is an object that is used to return values and exit the Code Service
. Following methods are available in the resp
object:
- resp.success
- resp.error
- resp.set
- resp.status
- resp.send
resp.success
resp.success = function (object | string){}
function analyze(req, resp) {
var message = "Everything went well!";
resp.success(output); // Service exits successfully with "Everything went well!"
}
resp.error
resp.error = function (object | string ){}
function analyze(req, resp) {
var message = "It broke!";
resp.error(error); // Service exits with failure with "It broke!"
}
resp.set This method has multiple signatures:
- resp.set = function(string, string){}
resp.set(keyString, valueString)
: adds or updates a key in the header to the HTTP response from a service. Example: resp.set("Content-Type", "text/html")
- resp.set = function(object){}
-
resp.set({})
: will set the response headers to those in the map. It replaces the header object so it removes any headers added withresp.set(k,v)
. Example:
resp.set({
"Content-Type": "application/json",
"ClearBlade-UserToken: req.userToken"
})
resp.status
resp.status = function(int){}
resp.status(int)
: changes the HTTP response code from a service. Example: resp.status(400)
resp.send
resp.send = function(object|string){}
resp.send(value)
: exit the service and send the value back as the body of the response. value
will be stringified if not already a string. value
will be sent as-is and not wrapped in our typical json response object.
Example: resp.send({valueA: "ABC-123-DEF", valueB: 123})
Exit
The Code Service
will exit executing either resp.success
or resp.error
.
Note: By default, if no exit method is called and execution reaches the end, resp.success
will be called to exit
Logs
Logs record and store the history of service responses. Each service can store up to 25 of the latest logs with the max size of 100KB each.
Requires
A Code Service can import a Library using the Requires
feature. In the Service settings, you can select one or more libraries to import, and that code is accessible from within the Code Service.
The same library can be used for multiple code services
Debugging
There is no native support for debugging line-by-line. Recommended to use log
method extensively, like so:
function ServiceA(req, resp) {
log("Something happened");
resp.success("All done");
}
Events
Events are actions or occurrences that happen in the ClearBlade System, response to them through execution of the associated Code Service. ClearBlade supports two types of events:
Timers
Timers provide a mechanism where a ClearBlade code service can be scheduled to execute at certain time intervals. Timers are similar to Unix cron jobs, with a similar scheduling mechanism.
Some example Timer configurations:
- Run forever every five minutes starting now
- Run 26 times every two weeks starting Friday, December 25th, 2018
- Run once at midnight
Triggers
Triggers are a mechanism that tie an action, called a Trigger Source
, to the invocation of a code service. The trigger source and payload information is passed into a code-service as req.params
.When creating a trigger, add log(req)
to the Code Editor
to log the payload information.
"params": {
"topic": "ClearBladeTopic",
"body": "214.312",
"userId": "8e89e2af0fc9cbeadfd6afb501",
"trigger": "Messaging::Publish"
}
Event Source
The platform publishes for every event regardless of whether the event is created programmatically in a service or created externally in the console.
$trigger
and $timer
are custom ClearBlade topics.
For more information on publishing event topics, click here
Asset | Category | Action | Topic |
---|---|---|---|
Messaging | Publish | Publish | $trigger/messaging/publish |
Messaging | Subscriptions | Upon Subscribe | $trigger/messaging/subscribe |
Messaging | Subscriptions | Upon Unsubscribe | $trigger/messaging/unsubscribe |
Messaging | Connection | User Connected | $trigger/messaging/user/connected |
Messaging | Connection | User Disconnected | $trigger/messaging/user/disconnected |
Messaging | Connection | Device Connected | $trigger/messaging/device/connected |
Messaging | Connection | Device Disconnected | $trigger/messaging/device/disconnected |
Data | Table | Created | $trigger/collection/created |
Data | Table | Updated | $trigger/collection/updated |
Data | Table | Deleted | $trigger/collection/deleted |
Data | Item | Created | $trigger/data/created |
Data | Item | Updated | $trigger/data/updated |
Data | Item | Deleted | $trigger/data/deleted |
User | n/a | Created | $trigger/user/created |
User | n/a | Updated | $trigger/user/updated |
User | n/a | Deleted | $trigger/user/deleted |
Device | n/a | Created | $trigger/device/created |
Device | n/a | Updated | $trigger/device/updated |
Device | n/a | Deleted | $trigger/device/deleted |
Edge/Platform | Platform | Platform Started | $trigger/startstopdisconnect/platformstarted |
Edge/Platform | Platform | Platform Connected on Edge | $trigger/startstopdisconnect/platformconnectedonedge |
Edge/Platform | Platform | Platform Disconnected on Edge | $trigger/startstopdisconnect/platformdisconnectedonedge |
Edge/Platform | Edge | Edge Started | $trigger/startstopdisconnect/edgestarted |
Edge/Platform | Edge | Edge Connected on Platform | $trigger/startstopdisconnect/edgeconnectedonplatform |
Edge/Platform | Edge | Edge Disconnected on Platform | $trigger/startstopdisconnect/edgedisconnectedonplatform |
Timer | n/a | Create Timer | $timer/{timerName} |
Examples of Incoming Object
Ensure the relevant permissions for code service are set before using triggers and timers.
Following is a list of categories grouped by action types, showing structures for incoming object:
Messaging (Publish/Subscribe/Unsubscribe)
:
When there is a Publish/Subscribe/Unsubscribe made to the ClearBlade’s MQTT broker.
// Template
{
"params": {
"topic": "<message_topic>",
"body": "<message_body>",
"userId": "<user_id>",
"trigger": "Messaging::<action_type>"
}
}
// Example
{
"params": {
"topic": "ClearBladeTopic",
"body": "example message",
"userId": "8e89e2af0fc9cbeadfd6afb501",
"trigger": "Messaging::Publish"
}
}
Messaging (UserConnected/UserDisconnected)
:
When a user Connects/Disconnects to the ClearBlade’s MQTT broker.
// Template
{
"params": {
"email": "<user_email>",
"trigger": "Messaging::<action_type>"
}
}
// Example
{
"params": {
"email": "example@clearblade.com",
"trigger": "Messaging::MQTTUserDisconnected"
}
}
Messaging (DeviceConnected/DeviceDisconnected)
:
When a device Connects/Disconnects to the ClearBlade’s MQTT broker.
// Template
{
"params": {
"deviceName": "<device_name>",
"trigger": "Messaging::<action_type>"
}
}
// Example
{
"params": {
"deviceName": "exampleDevice",
"trigger": "Messaging::MQTTDeviceDisconnected"
}
}
Data-Table (CollectionCreated/CollectionUpdated/CollectionDeleted)
:
// Template
{
"params": {
"collectionId": "<collection_id>",
"collectionName": "<collection_name>",
"trigger": "Data::<action_type>"
}
}
// Example
{
"params": {
"collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
"collectionName": "ExampleCollection",
"trigger": "Data::CollectionCreated"
}
}
Data (ItemCreated)
:
// Template
{
"params": {
"items": [
{
"item_id": "<item_id>"
}
],
"collectionId": "<collection_id>",
"collectionName": "<collection_name>",
"trigger": "Data::ItemCreated"
}
}
//Example
{
"params": {
"items": [
{
"item_id": "083196bf-8059-46c0-8280-38234aa73fc1"
}
],
"collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
"collectionName": "ExampleCollection",
"trigger": "Data::ItemCreated"
}
}
Data (ItemUpdated/ItemDeleted)
:
// Template
{
"params": {
"collectionId": "<collection_id>",
"collectionName": "<collection_name>",
"trigger": "Data::<action_type>",
"items": [
// First row
// This includes all of the data from the row
{
"<column1_name>": <column1_data>,
"item_id": "<item_id1>",
"<column2_name>": <column2_data>
}
// A second row
{
"<column1_name>": <column1_data>,
"item_id": "<item_id1>",
"<column2_name>": <column2_data>
}
]
}
}
// Example
{
"params": {
"collectionId": "ceafb4cf0bbaf5f4ddfbf1b48730",
"collectionName": "ExampleCollection",
"trigger": "Data::ItemUpdated",
"items": [
{
"hi": 6,
"item_id": "14b12adf-4c32-4740-8461-546daa660393",
"low": 3
},
{
"hi": 7,
"item_id": "3700fa91-ef49-41e8-9fe7-64e9ee6aa3b2",
"low": 5
}
]
}
}
User (UserCreated)
// Template
{
"params": {
"user": {
"creation_date": <date_time>,
"email": "<user_email>",
"user_id": "<user_id>"
},
"trigger": "User::UserCreated"
}
}
//Example
{
"params": {
"user": {
"creation_date": 2019-06-20T19:43:10Z,
"email": "example@clearblade.com",
"user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
},
"trigger": "User::UserCreated"
}
}
User (UserUpdated/UserDeleted)
// Template
{
"params": {
"query": {
"FILTERS": [
[
{
"EQ": {
"user_id": "<user_id>"
}
}
]
],
"PAGENUM": <page_number>,
"PAGESIZE": <page_size>,
"SELECTCOLUMNS": <columns_selected>,
"SORT": []
},
// Includes all of the data from the row
"user": {
"email": "<user_email>"
<column1_name>: <row_data>
<column2_name>: <row_data>
"user_id": "<user_id>"
},
"trigger": "User::<action_type>"
}
}
// Example
{
"params": {
"query": {
"FILTERS": [
[
{
"EQ": {
"user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
}
}
]
],
"PAGENUM": 0,
"PAGESIZE": 0,
"SELECTCOLUMNS": null,
"SORT": []
},
"user": {
"email": "example@clearblade.com"
"name": "Example"
"age": 22
"user_id": "9c87dfd00beefbd8a0f3cc82f0c001"
},
"trigger": "User::UserUpdated"
}
}
Device (DeviceCreated/DeviceDeleted)
// Template
{
"params": {
"trigger": "Device::<action_type>",
"device": {
"allow_certificate_auth": "<boolean>",
"allow_key_auth": "<boolean>",
"certificate": "<certificate>",
"created_date": "<date_time>",
"description": "<device_description>",
"device_key": "<device_key> :: <device_name>",
"enabled": "<boolean>",
"last_active_date": "<date_time>",
"<columnX_name>": "<columnX_data>",
"<columnY_name>": "<columnY_data>",
"system_key": "<system_key>",
"type": "<device_type>"
},
"deviceName": "<device_name>"
}
}
//Example
{
"params": {
"trigger": "Device::DeviceCreated",
"device": {
"allow_certificate_auth": true,
"allow_key_auth": true,
"certificate": "",
"created_date": 2019-06-20T19:43:10Z,
"description": "",
"device_key": "ccafb4cf0bd0dcbcadaccaf9ebba01 :: ExampleDevice",
"enabled": true,
"last_active_date": 2019-06-20T19:43:10Z,
"name": "ExampleDevice",
"state": "active",
"system_key": "ccafb4cf0bd0dcbcadaccaf9ebba01",
"type": "Sensor"
},
"deviceName": "ExampleDevice"
}
}
Device (DeviceUpdated)
// Template
{
"params": {
"changes": {
"description": "<device_description>"
},
"deviceName": "<device_name>",
"trigger": "Device::DeviceUpdated"
}
}
// Example
{
"params": {
"changes": {
"description": "device description"
},
"deviceName": "example_device",
"trigger": "Device::DeviceUpdated"
}
}
Edge/Platform
// Template
{
"params": {
"edgeName": "edge_name",
"trigger": "StartConnectDisconnect::<action_type>"
}
}
// Example
{
"params": {
"edgeName": "example",
"trigger": "StartConnectDisconnect::<Edge Started>"
}
}
Webhooks
A webhook is a mechanism that allows you to execute a code service by targeting a public endpoint. Any HTTP method, such as GET, POST, PUT, DELETE, can be used and will result in the same action, executing the service. The URL we provide can be used by third parties to push real-time data to ClearBlade’s server. A code service can be invoked by multiple webhooks and they are syncable. See tutorial.
Authorization
These requests are to be added when executing a webhook.
Auth Method | Request | |
---|---|---|
clearblade_auth |
ClearBlade-Usertoken: <USER_TOKEN> |
See Example |
http_basic_auth |
Authorization: Basic <USER_TOKEN> |
See Example |
payload_auth (GET) |
Query String: token=<USER_TOKEN> |
See Example |
payload_auth (POST) |
Request Body: “token”: <USER_TOKEN> |
See Example |
no_auth |
No Authorization Required. The “Anonymous” role should have Run permission checked for the service. This allows for the no auth webhook to work. |
See Example |
Authorization Examples
ClearBlade Auth
function ExampleWebhookInvoker(req,resp){
// These are parameters passed into the code service
var params = req.params
var http = Requests();
var options = {
uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c80f79d0a/ExampleWebhook",
headers:{
"ClearBlade-UserToken":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErDNRpSNJsD5w=="
},
body:{
"data":"hello"
}
}
http.post(options, function(err, response){
if(err){
resp.error("Failed to call the webhook");
}
resp.success("Success"+ response);
})
}
HTTP Basic Auth
function ExampleWebhookInvoker(req,resp){
// These are parameters passed into the code service
var params = req.params
var http = Requests();
var options = {
uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10bbae683c7ace4dfd701/Webhook_2_HTTP_Auth",
headers:{
"Authorization":"Basic Id19kSWr2vkB4CUc5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w=="
},
body:{
"data":"Authenticated using Http Auth"
}
}
http.post(options, function(err, response){
if(err){
resp.error("Failed to call the webhook");
}
resp.success("Success"+ response);
});
}
Payload Auth GET
function ExampleWebhookInvoker(req,resp){
// These are parameters passed into the code service
var params = req.params
var http = Requests();
var options = {
uri:"https://platform.clearblade.com/api/v/4/webhook/execute/f6edffd60bdc95a5c19ff79d0a/Webhook_3_Payload_Auth",
qs:{
"token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67ELmrHvkqUD8vnYu52XGErPEjDNRpSNJsD5w==",
"data":"Authenticated using Get method"
}
}
//Using get method
http.get(options, function(err, response){
if(err){
resp.error("Failed to call the webhook");
}
resp.success("Success"+ response);
});
}
Payload Auth POST
function ExampleWebhookInvoker(req,resp){
// These are parameters passed into the code service
var params = req.params
var http = Requests();
var options = {
uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10b8ebae3c7ace4dfd701/Webhook_3_Payload_Auth",
body:{
"token":"Id19kSWr2vkB4CUcxa5lm6Akzh1jXr6Fw3sxW-ynaaV67EJRLmrHvkqUDnYu52XGErPEjDNRpSNJsD5w==",
"data":"Authenticated using Http Payload"
}
}
http.post(options, function(err, response){
if(err){
resp.error("Failed to call the webhook");
}
resp.success("Success"+ response);
});
}
No Auth
function ExampleWebhookInvoker(req,resp){
// These are parameters passed into the code service
var params = req.params
var http = Requests();
var options = {
uri:"https://platform.clearblade.com/api/v/4/webhook/execute/84c3e6d10b8ebae3c7ace4dfd701/Webhook_4_No_Auth"
}
http.post(options, function(err, response){
if(err){
resp.error("Failed to call the webhook");
}
resp.success("Success"+ response);
});
}
Advanced
Execution Timeout
Code Services are usually used as microservices, and expected to have a lifecycle of under 30 seconds. However, this execution timeout can be extended to make the services run longer. When execution timeout is set to infinity(“Never” in the console) it’s called a Stream Service, meaning the code service will run indefinitely. This allows for the following high performance use cases:
- Exponentially faster rate for any operation which is otherwise performed in an event triggered micro-service.
- Users can also design stream services to perform bulk operations on collections.
- Data generation
Best Practices:
For Stream Services, set Execution Timeout to
Never
For Micro Service, set Execution Timeout to its default of
Infinite
Concurrency
Concurrency allows the developer to cap the maximum number of instances of code services that can run on each of the ClearBlade node. Any more than this number will kill the oldest code service instance.
Best Practices:
For Stream Services, set Concurrency to
Other
, and then set the number of instances you need. Recommended is 4.- For Micro Service, set Concurrency to its default of
Infinite
, to optimize performance
- For Micro Service, set Concurrency to its default of
Run as
A service can have permissions to invoke as a specific user role. If a run as
parameter is set, it runs as user that is stated in the parameter, regardless of the user that invokes it. If the user set in the run as
parameter doesn’t have permissions to execute
the service (and the permissions to execute all the operations performed within the code service), then the service will fail. If the user invoking the service does not have the permission to execute
the code service, then the service will fail as well.
For example: An anonymous user can invoke and run a service as a different user.
Run as User
A code service can be invoked on the behalf of a certain user. It can also have permissions to invoke as a specific user role. If Code Services needs escalated privileges, the developer can use this ‘Run as User’ property to set an user with escalated permissions to invoke code services. This is usually done when the code service accesses/updates certain system assets to which only an user with a certain role has access, but the code service is desiged to be invoked by an user with any other role.
Auto Restart
This property is valid when the service is a Stream Service. When the box is checked, it restarts service automatically if it stops for any reason, including successes.
Best Practice : Do not check the box until fully testing the service. There could be undesired deadlocks if there are errors in the code.
Auto Balance
This property is valid when the service is a Stream Service. The platform automatically starts a service on each available node with the number of concurrency instances. It kills and restarts services when saved with a new code.
If auto_balance = true
:
req.params
are not allowed- A normal code service must be developed first to confirm that it is functioning properly before developing it as a stream service. Do not check the auto balance box until the code service works.
Failed Runs
If a Code Service exits with resp.error, or throws an uncaught error, it will be stored in Failed Runs. A developer can then fix any issues, and retry that failed run with the same input parameters.
Standard Javascript Functions (es5.1)
Method | Usage |
---|---|
setTimeout(),setInterval() | Available natively in version 9.3.0 or later |
atob(), btoa() | buffer-node-js library |
promises | q-promise-library/Available natively in version 9.3.0 or later |