AWS IoT: Service Limits and their Impact on Solution Architecture

Thmb_47

AWS IoT provides a managed, multitenant service that enables communication between IoT devices, applications, and the backend. By virtue of being a managed service, IoT platform eases out several deployment and infrastructure monitoring issues. However, it imposes some limits and restrictions on how data can be modeled and the size of device information that can be stored in a “shadow” on AWS IoT 1. Primarily, the restrictions ensure the safety and security of the platform and deployments, and avoid misuse of the platform by a tenant or rogue entity. It is critical to be aware of and understand these restrictions before finalizing the solution architecture as some of them may impact the feasibility of the solution architecture and operating costs.

Modelling an IoT deployment in AWS IoT

AWS IoT maintains the state representation of all devices and accessories that are part of an IoT deployment in the form of “things” and “shadows.” 2 The state of IoT devices in AWS IoT can be modelled in two ways:

1. One thing per device - In this model, a thing is created for each device that is part of the IoT deployment and is associated with its own shadow. (See Figure 1)

wp11

Figure 1: One thing per device

2. One thing per gateway - In this model, a thing corresponds to the aggregator node and represents the state of all devices that connect to AWS IoT via this aggregator node. (See Figure 2)

wp2

Figure 2: One thing per gateway

Key AWS IoT Limitations

AWS IoT imposes a number of limitations that can impact or limit the choice of modeling the IoT network in AWS IoT.

1. Shadow Size Limit

Maximum size of a JSON state document 8 KB

 
AWS IoT caches data corresponding to an IoT accessory (a thing) in the form of a JSON document known as a shadow. The maximum allowed size for this JSON document is 8 Kilobytes (KB).

Of the 8 KB, 4 KB is for the actual state and 4 KB is for the desired state. This effectively leaves only 4KB of space in the shadow JSON to store “thing-state” data.

The following is a sample depiction of a “light” object containing a minimal set of attributes that needs to be stored in a shadow:

{
"id": "16 characters",
"name": "Assume 50 characters for name",
"createdAt": "10 characters long string",
"reachabilityState": 1,
"batteryLevel": 98,
"onoff": 1,
"dimLevel": 50,
"colorX": 50,
"colorY": 50
}

Assuming one character per byte, this depiction of the thing state will consume around 150 bytes of shadow size, which fits very well into the 4KB limit if the one-thing-per-accessory model is followed.

However, if the one-thing-per-gateway model is used, where one thing represents multiple accessories, the current JSON document size will allow approximately 30 devices to be accommodated in the JSON document. This puts a limitation on the number of accessories that can be supported per gateway and is hence not a future-proof solution.

2. Maximum subscriptions per subscribe call

Maximum subscriptions per subscribe call A single SUBSCRIBE call is limited to request a maximum of eight subscriptions.

 
AWS IoT allows a maximum of eight topics in one subscribe call. This service limit of AWS IoT does not pose a problem if the total number of topics to be subscribed to are not more than eight, or if it is possible to use a wildcard subscription. If the number of subscribe-to topics exceed eight, multiple subscribe calls will have to be made. But, as shown below, it may not always be possible to limit the number of topics to eight, and a wildcard subscription may not be the preferred option either.

The format of AWS IoT topics is ‘$aws/things/thingName/shadow/update’ and the following is the list of topics supported by AWS IoT for a thing’s shadow 3 :

1. /update
2. /update/accepted
3. /update/documents
4. /update/rejected
5. /update/delta
6. /get
7. /get/accepted
8. /get/rejected
9. /delete
10. /delete/accepted
11. /delete/rejected

1. A thing name-based wildcard subscription (‘$aws/things/thingName/#’) may not be preferred in many cases because a given entity—an app, gateway or any other service that needs these messages—may not need to subscribe to all the topics.

For example, a device that is acting as the source of data, may not need to subscribe to “get”, “get/accept”, or “get/reject” because the device is the source of data; device will only update data on AWS IoT shadow and will never fetch data from AWS IoT. Using a wildcard subscription will result in a lot of unnecessary messages being sent to the entity. This will increase the running cost of the solution because charging in AWS is done on a per-message basis.

2. AWS IoT documentation suggests avoiding subscribing to topic filters such as “$aws/things/thingName/shadow/#” because the number of topics that match this topic filter might increase as AWS IoT introduces new shadow topics.

For example, “update” till some time ago had three subtopics, and “…/update/document” was recently added. If an implementation had subscribed to “…/update/#”, it would also get published on “…/update/document” topic. Therefore, even though the application may not be using the information provided in this topic, the application will be charged for the delivery of messages related to this new topic.

A given IoT device would typically be interested in subscribing to the following topics:

1. /update/accepted
2. /update/rejected
3. /update/delta
4. /delete
5. /delete/accepted
6. /delete/rejected

In addition, a device may want to subscribe for the “lifecycle event."4

”So, on an average, a device will need to subscribe to between six and eight topics. This fits well if the one-thing-per-accessory model is used and all the subscription topics can be accommodated in one single subscribe call. But if the one-thing-per-gateway model is used, gateway will have to perform six to eight subscriptions per connected device to the gateway. If there are 100 accessories connected to Gateway, there will be roughly 100 calls to AWS IoT, each having eight topics to subscribe to. This many network calls adds considerable overhead onto the gateway and increases the cost of the solution.

3. Subscription per connection limit

Subscriptions per session The message broker limits each client session to subscribe to up to 50 subscriptions. A SUBSCRIBE request that pushes the total number of subscriptions past 50 will result in the connection being disconnected.

 
As discussed in the above section, if the one-thing-per-gateway model is being used, the gateway will have to subscribe on behalf of all the accessories connected to it. Since one connection supports only 50 subscriptions, and if 10 subscriptions are to be made for each accessory, a gateway can at the most support five accessories per connection. If there are 100 accessories, the gateway will have to make 20 connections with AWS IoT.

To keep the gateway implementation simple, it may require opening a separate connection to AWS IoT for each accessory. Supporting so many socket connections adds considerable overhead to the gateway.

4. Policy Size limit

Maximum number of policies that can be attached to a certificate or Amazon Cognito identity 10
Maximum policy document size 2048 characters (excluding white space)

 
The maximum-policy-document-size limitation puts a restriction on how many policy statements can be added to a given policy document. Together with a maximum number of policies that can be attached to a certificate or Amazon Cognito identity, this limits:

1. The number of Client IDs that can be added in the “connect” section
2. The number of resources that can be specified in publish-and-subscribe sections

A bare-bones policy such as the one described below requires about 600 characters, which leaves space for 14 or 15 resource statements. Adding “condition” statements to the policy will reduce this number significantly.

"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["iot:Connect"],
"Resource": ["*"],
"Condition": {
"Bool": {"iot:Connection.Thing.IsAttached": ["true"]
}}},
{
"Effect": "Allow",
"Action": ["iot:Subscribe"],
"Resource":["arn:aws:iot:us-east-1:111111111111:topicfilter/${iot:Connection.Thing.ThingName}/*"]
},
{
"Effect": "Allow",
"Action": ["iot:Receive"],
"Resource":["arn:aws:iot:us-east-1: 111111111111:topic/${iot:Connection.Thing.ThingName}/*"]
},
{
"Effect": "Allow",
"Action": [
"iot:Publish"
],
"Resource":["arn:aws:iot:us-east-1: 111111111111:topic/${iot:Connection.Thing.ThingName}/*",]
}]
}

Effort needs to be made to define generic policies using variable names, etc. However, if that is not possible, it can create a roadblock for defining appropriate authentication and authorization policies.

Conclusion

As of now, it appears that the AWS IoT architecture is aligned more towards the one-thing-per-accessory model where each accessory is represented as an individual thing.

Using the one-thing-per-gateway model with the current implementation of AWS IoT puts a lot of processing requirements on the gateways, which typically are resource-constrained devices. Opening multiple sockets and making multiple network calls puts a strain on the memory and CPU of such constrained devices and increases the boot-up latency.

The topic format inherently assumes that topics are corresponding to a standalone thing. The topic format does not provide flexibility to accommodate hierarchical arrangement of accessories in a gateway. This kind of flexibility enables hierarchical data modelling by allowing wildcard subscription and can ease some of the issues related to maximum number of subscriptions or maximum policy size.

For IoT deployments, where IoT devices are IP enabled and communicate with backend infrastructure directly, supporting the one-thing-per-accessory model is relatively easy and so AWS IoT is a good option in those cases.

In typical home or enterprise deployments, IoT devices communicate to an aggregator node-the home gateway or access point-over low-power/short-range communication protocols. In turn, the aggregator node communicates with the backend infrastructure on behalf of all these devices. Modelling the devices using the one-thing-per-gateway model is a preferred option in such cases. However, supporting this model with AWS IoT is not straight forward. It needs additional development on top of AWS IoT, and results in higher development and operational costs.

As discussed at the start of this post, these are service limits that are imposed to take care of multitenancy and security of platform. There are a lot of other service limits related to transaction rates and throughput as well. It is important to review and validate the architecture against those service limits.

Equally important is the fact that AWS IoT is an evolving platform and some of these issues and limitations may be resolved in the future. It is important to define the solution architecture, keeping in mind the long-term alignment of the solution with the platform so the solution management effort and overall operating cost can be reduced when the issues and limitations cease to exist.

Aricent is a leading provider of IoT Engineering services with expertise in design and development of products to drive growth for our clients. Click here to talk to our experts.

References:

[1] See AWS Service Limits document http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html

[2] See Device Shadows for AWS IoT http://docs.aws.amazon.com/iot/latest/developerguide/iot-thing-shadows.html

[3] See AWS Service Limits document http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html

[4] See Device Shadows for AWS IoT http://docs.aws.amazon.com/iot/latest/developerguide/iot-thing-shadows.html

[5] See AWS Lifecycle Event http://docs.aws.amazon.com/iot/latest/developerguide/life-cycle-events.html

 

Leave a Reply

Your email address will not be published. Required fields are marked *