Measures reported by AWSAmazonIoTTest

AWS IoT provides secure, bi-directional communication between Internet-connected devices such as sensors, actuators, embedded micro-controllers, or smart appliances and the AWS Cloud. This enables you to collect telemetry data from multiple devices, and store and analyze the data. You can also create applications that enable your users to control these devices from their phones or tablets.

When using AWS IoT, devices are managed as things. A thing is a representation of a specific device or logical entity. It can be a physical device or sensor (for example, a light bulb or a switch on a wall). It can also be a logical entity like an instance of an application or physical entity that does not connect to AWS IoT but is related to other devices that do (for example, a car that has engine sensors or a control panel). Information about a thing is stored in the registry as JSON data.

These devices/things report their state by publishing messages, in JSON format, on MQTT topics. Each MQTT topic has a hierarchical name that identifies the device whose state is being updated. When a message is published on an MQTT topic, the message is sent to the AWS IoT MQTT message broker, which is responsible for sending all messages published on an MQTT topic to all clients subscribed to that topic. The message broker supports the use of the MQTT protocol to publish and subscribe and the HTTPS protocol to publish.

You can create rules that define one or more actions to perform based on the data in a message. For example, you can insert, update, or query a DynamoDB table or invoke a Lambda function. Rules use expressions to filter messages. When a rule matches a message, the rules engine invokes the action using the selected properties. Rules also contain an IAM role that grants AWS IoT permission to the AWS resources used to perform the action.

Each device has a shadow that stores and retrieves state information. Each item in the state information has two entries: the state last reported by the device and the desired state requested by an application. An application can request the current state information for a device. The shadow responds to the request by providing a JSON document with the state information (both reported and desired), metadata, and a version number. An application can control a device by requesting a change in its state. The shadow accepts the state change request, updates its state information, and sends a message to indicate the state information has been updated. The device receives the message, changes its state, and then reports its new state.

Typically, AWS IoT policies are used to enable devices to connect to the message broker and get authorized, so they can perform AWS IoT operations, such as subscribing or publishing to MQTT topics via the broker, or get, update, or delete a device's shadow. If any of these policy actions fail or experience errors, then devices may no longer be able to communicate with the AWS cloud. Also, where messages match configured rules, if the defined rule actions fail, the failure may disrupt communication between the devices and critical AWS services such as AWS Lambda, Amazon S3, Amazon DynamoDB, etc. Moreover, AWS also imposes limits on the AWS IOT service; if these limits are violated, it might adversely impact the overall performance of AWS IOT . To ensure the peak performance of AWS IOT therefore, administrators should continuously track the different types of requests to the message broker, capture errors, failures, and violations promptly, and rapidly initiate measures to fix the faults. This is where the AWSAmazonIoTTest test helps!

By default, this test tracks the different types of requests (connection, publish, subscribe, unsubscribe, get/delete/update device shadow, etc.) to the message broker, and automatically discovers the protocols in use. For each protocol, the test then reports the count of requests that were successful, that violated the AWS IOT limits, and that which experienced errors. In the process, administrators can determine whether/not specific protocols are failing or are returning error responses frequently; such protocols could be suspect and may have to be investigated. Optionally, you can configure this test to report metrics for each rule or each action type configured for rules. This provides insights into errors (if any) in messages that match each rule and failed invocations of every action type.

Outputs of the test : One set of results for each protocol / rule / action type.

First-level descriptor: AWS Region

Second-level descriptor: Protocol / rule / action type, depending upon the option chosen against the IOT FILTER NAME parameter

The measures made by this test are as follows: