Category Monitoring ingested data

Monitoring IoT networks for security purposes is critical to ensure that the devices and data are protected from cyber threats. AWS offers a range of tools and services to help organizations monitor the security of their IoT networks.

One key tool for security monitoring on AWS is Amazon GuardDuty, a threat detection service that continuously monitors for malicious activity and unauthorized behavior across AWS accounts and workloads. GuardDuty can be used to detect and respond to potential security threats in real time and can alert relevant personnel when suspicious activity is detected.

Another important tool for security monitoring on AWS is AWS IoT Device Defender, a managed service that audits and monitors the security of IoT devices and applications. Device Defender can be used to detect and respond to security threats by monitoring device behavior, identifying anomalies, and triggering alerts when suspicious activity is detected.

AWS also offers a range of identity and access management (IAM) tools, including AWS IAM and AWS Single Sign-On (AWS SSO), which can be used to control and manage user access to IoT devices and data. IAM and SSO can be used to set permissions, enable multi-factor authentication, and enforce policies to ensure that only authorized personnel can access sensitive data.

To implement an effective security monitoring solution on AWS for IoT networks, organizations should first identify their security requirements and establish key security metrics to measure system performance. They should also establish security policies and procedures to govern access control, vulnerability management, incident response, and other security-related activities.

Organizations should also leverage security monitoring tools and services, such as GuardDuty and Device Defender, to continuously monitor for security threats and vulnerabilities. They should regularly review and analyze security logs and audit trails to identify potential security issues and opportunities for improvement.

Outside of AWS, there are many solutions that help with monitoring IoT deployments as well. It starts with doing an analysis of network traffic. Tools such as Wireshark or tcpdump can be employed for packet sniffing and network traffic analysis, providing insights into potential malicious activities or unauthorized data transmission. In terms of mitigating risks, endpoint security solutions such as antivirus software, intrusion prevention systems (IPSs), or advanced threat protection (ATP) services can be used to safeguard IoT devices. These solutions help in identifying and mitigating malware, ransomware, and other forms of cyberattacks targeting IoT endpoints.

Finally, organizations should establish processes for continuous improvement, regularly reviewing and updating their security monitoring strategies to ensure that they remain effective and up-to-date. By continually refining their security monitoring strategies, organizations can ensure that their IoT networks remain secure and protected from cyber threats.

These KPIs measure the overall performance and health of the IoT network and include network latency, packet loss, and throughput. By monitoring these KPIs, organizations can identify potential network bottlenecks and take corrective actions to ensure smooth network operation.

User-level KPIs

These KPIs measure the user experience of IoT applications and services and include response time and availability. By monitoring these KPIs, organizations can ensure that end users are satisfied with the performance of IoT applications and services.

Security KPIs

These KPIs measure the effectiveness of security controls in place to protect IoT devices and data and include the number of security incidents, the severity of incidents, and the time to resolution. By monitoring these KPIs, organizations can identify potential security threats and take corrective actions to prevent them.

Business-level KPIs

These KPIs measure the business impact of the IoT network and include revenue generated, cost savings, and customer satisfaction. By monitoring these KPIs, organizations can understand the overall value of their IoT networks and identify opportunities for improvement.

Selecting, analyzing, and monitoring KPIs

Selecting, analyzing, and monitoring KPIs is an essential step in implementing an effective continuous monitoring strategy for IoT networks. Here are some steps to follow to ensure that KPIs are selected, analyzed, and monitored effectively:

Identify your objectives: Before selecting KPIs, it is important to identify your monitoring objectives. This involves understanding what you want to achieve with your monitoring program and what metrics will help you track progress toward those objectives.

Choose relevant KPIs: Once you have identified your objectives, choose KPIs that are relevant to your objectives. Ensure that the KPIs are specific, measurable, achievable, relevant, and time-bound (SMART).

Analyze the KPIs: Analyze the selected KPIs to ensure that they provide the necessary insights into the performance and health of the IoT network. Use data analysis tools to identify trends and patterns in the KPI data and gain insights into areas that may require improvement.

Monitor the KPIs: Implement a system for monitoring the KPIs continuously. This can be achieved using tools such as AWS CloudWatch or other monitoring tools. Set up alerts and notifications to inform relevant personnel when certain KPIs or metrics fall outside predefined thresholds.

Regularly review and adjust KPIs: Regularly review and adjust your KPIs to ensure that they remain relevant and aligned with your monitoring objectives. This involves regularly analyzing KPI data and using the insights gained to refine KPIs as necessary.

Now that we’ve learned how to use KPIs, we can look to understand the different monitoring capabilities that are present both on-premises and on the cloud.

Monitoring capabilities on-premises and on the cloud

Monitoring capabilities are certainly an aspect that is crucial to understanding the limits of how far we can go within what we invest in our solutions. In this section, we will look at how some monitoring capabilities can be taken into consideration in designing our IoT networks.

It is important to understand why you are conducting the monitoring that you are doing, and the appropriate milestones for managing its progress. In this section, we will look into how we can set clear objectives and appropriately define KPIs to measure how well we are progressing.

Setting clear objectives and goals for monitoring

Setting clear objectives and goals is an important step in implementing a successful continuous monitoring strategy for IoT networks. Organizations should identify the specific metrics and KPIs they want to track and establish thresholds for acceptable performance levels. This will allow them to quickly identify any issues that may arise and take corrective action before they cause significant disruptions to their networks.

Some common objectives and goals for continuous monitoring in IoT networks include the following:

Improving network reliability: Organizations may set objectives to reduce downtime and improve overall network uptime. This could include monitoring key network components and identifying potential issues before they cause disruptions.

Enhancing security: Security is a critical concern for IoT networks, and organizations may set goals to ensure that their networks are protected from potential cyber threats. This could include monitoring network traffic and identifying anomalous behavior that may indicate a security breach.

Optimizing network performance: Organizations may set objectives to improve the overall performance of their IoT networks, such as reducing latency or improving throughput. This could involve monitoring network traffic and identifying areas where improvements could be made.

Minimizing operational costs: Organizations may set goals to reduce operational costs associated with managing their IoT networks. This could involve identifying inefficiencies in their networks and automating processes to reduce the need for manual intervention.

Once objectives and goals have been established, organizations should identify the specific metrics and KPIs that will be used to measure performance. For example, if the goal is to improve network reliability, organizations may track metrics such as network uptime, response time, and error rates. These metrics should be tracked continuously and compared against predefined thresholds to identify any potential issues.

In addition to defining metrics and KPIs, organizations should also establish processes for reviewing and analyzing monitoring data to identify opportunities for optimization and improvement. This may involve using visualization tools such as dashboards and reports to gain insights into network performance and identify areas where improvements can be made.

Different types of KPIs

There are different types of KPIs that can be used to monitor IoT networks. There are five categories of KPIs that they can fall under: device-level, network-level, user-level, security, and business-level.

Device-level KPIs

These KPIs measure the performance and health of individual IoT devices and include their availability, response time, and error rates. By monitoring these KPIs, organizations can identify devices that are not functioning properly and take corrective actions to prevent downtime.

Automation and machine learning are important aspects of keeping IoT networks running smoothly and securely. With the help of AWS tools and services, organizations can implement these capabilities to identify and predict issues before they happen and take necessary actions automatically to prevent downtime and performance issues.

One useful tool for automation and machine learning on AWS is Amazon SageMaker. This is a service that allows developers and data scientists to build, train, and deploy machine learning models quickly and easily. By analyzing and predicting IoT devices and network behavior, SageMaker can automatically identify potential issues and trigger necessary actions.

AWS IoT Events is another helpful tool for automation and machine learning on AWS. It is a service that allows organizations to detect and respond to events from multiple IoT devices and applications in real time. This service can automate the detection and resolution of common IoT devices and network issues, improving the overall reliability of the system and reducing the need for manual intervention.

AWS also provides a range of data analytics and processing tools, such as AWS Glue, AWS Lambda, and AWS Data Pipeline. These tools can be used to automate the collection, processing, and analysis of IoT data. By identifying patterns and trends in IoT data, these tools can trigger automated responses when specific conditions are met. To implement automation and machine learning capabilities on AWS for IoT networks, organizations should first define their monitoring requirements and establish KPIs to measure system performance. They should also develop machine learning models and algorithms to analyze and predict IoT devices and network behavior and automate the detection and resolution of common issues.

Organizations can use dashboards and visualization tools, such as AWS QuickSight, to provide real-time visibility into system performance and health. These dashboards can be customized to show relevant metrics and KPIs and can be shared with relevant stakeholders to ensure everyone has a comprehensive view of system performance.

By continually reviewing and analyzing monitoring data, organizations can identify opportunities for optimization and enhancement. This process of continuous improvement ensures that their automation and machine learning strategies remain effective over time, keeping their IoT networks reliable and secure.

Exercise on simulating monitoring networks

In this exercise, we will be looking at simulating an IoT network with AWS IoT Core and monitoring it through the tools provided by the service. Here are the steps to follow along:

Log in to the AWS Management Console and navigate to the AWS IoT Core dashboard.

Click on the Test menu and select Simulator to access the AWS IoT Simulator.

Click on Create a new simulation to create a new simulation model.

Enter a name for the simulation model and click on Create to create the model.

Click on Add a device to add a new virtual device to the simulation model.

Enter a name for the device and select a device type from the drop-down list.

Enter the device’s metadata, including the device ID, device attributes, and device shadow state.

Click on Add a behavior to add a behavior to the device. A behavior is a script that simulates the device’s behavior and generates messages that are sent to AWS IoT Core.

Enter the behavior’s name, type, and script code. The script can be written in JavaScript or Python.

Click on Add a topic to add a topic that the device will publish messages to.

Enter the topic name and click on Add to add the topic.

Click on Run to start the simulation.

Monitor the simulation metrics and logs in the Simulation tab. You can view the number of messages sent and received, the message throughput, and the behavior logs for each device.

Add additional devices, behaviors, and topics to simulate a more complex IoT network.

With the knowledge of how to simulate the monitoring of networks, we can forge ahead to understand the metrics that can affect how we configure them.

Early detection of issues: Continuous monitoring of IoT networks enables organizations to detect issues early and resolve them before they turn into major problems. This helps prevent system downtime, reduce maintenance costs, and enhance the overall system performance.

Improved system performance: Real-time monitoring of IoT networks can identify performance bottlenecks and help optimize the system for better performance. This leads to faster response times, improved system reliability, and enhanced user experience.

Better decision-making: IoT monitoring solutions provide real-time data and insights that can inform effective decision-making. Organizations can use the data to make informed decisions that improve operational efficiency, reduce costs, and enhance overall business performance.

Enhanced security: Continuous monitoring of IoT networks helps identify security vulnerabilities and potential threats. This enables an organization to take proactive measures to prevent attacks and protect sensitive data, ensuring the safety and security of the system.

Predictive maintenance: Continuous monitoring of IoT networks can identify patterns and trends that can inform predictive maintenance. This helps an organization identify potential failures before they occur, reducing maintenance costs and increasing the overall lifespan of the system.

Scalability: Continuous monitoring solutions can scale to meet the needs of expanding IoT networks. This enables an organization to handle large volumes of data and maintain a comprehensive view of the system’s performance, even as the network expands.

On the other hand, it is important to understand how the monitoring framework is done over AWS and in general. In Figure 9.1, we can see how this framework can be visualized and stepped through for our needs:

Figure 9.1 – IoT network monitoring framework

Here, we have the framework that we will walk through step by step to understand what each step encompasses:

Targeted testing: This step involves testing specific areas of the IoT network to identify potential vulnerabilities or weaknesses. Testing may involve performing a penetration test or using specialized tools to identify vulnerabilities in the network. In a smart home IoT network, targeted testing might involve using a network scanning tool such as Nmap to identify open ports on devices such as smart thermostats or security cameras.

Risk assessment: In this step, the results of the targeted testing are analyzed to identify potential risks and threats to the IoT network. A risk assessment helps to prioritize potential vulnerabilities based on their likelihood and potential impact on the network. After identifying vulnerabilities in the smart home network, a risk assessment could determine that an unpatched security camera poses a high risk due to its accessibility from the internet and the potential for it to be used as a gateway to access other devices on the network.

Interviews and data review: This step involves interviewing key stakeholders and reviewing data from various sources, such as system logs and incident reports. The goal is to gather additional information about potential vulnerabilities and risks to the IoT network. Interviews with the smart home’s residents could reveal that they are unaware of the need to regularly update device firmware. Reviewing system logs might show repeated attempts to access devices from unrecognized IP addresses, indicating potential security threats.

Gap analysis: This step involves comparing the results of the previous steps to the organization’s security policies and procedures. This helps to identify any gaps in the security posture of the IoT network and determine areas where improvements are needed. Comparing the current security measures of the smart home network with industry best practices might reveal gaps such as a lack of regular firmware updates, an absence of strong password policies, or a failure to segment the network to isolate critical devices from one another.

Application of forensic tests: The final step involves conducting forensic tests on the network to gather additional information about potential vulnerabilities and risks. Forensic tests may include analyzing system logs or performing a deep dive into specific areas of the network to identify potential issues. Forensic analysis of the smart home network could involve examining the security camera’s logs to trace back to the origin of unauthorized access attempts. It might also include a deep dive into network traffic to identify any unusual patterns that could indicate a breach or an ongoing attack.

By following these five steps, organizations can ensure that their IoT networks are monitored effectively and continuously, helping to minimize potential risks and threats and ensuring the overall security and efficiency of their networks.

In today’s fast-paced digital landscape, maintaining the continuous operation of IoT networks is more critical than ever. IoT networks are being used in a wide range of industries, from manufacturing and logistics to healthcare and retail, to collect and process real-time data and automate processes. As such, even a brief downtime can result in significant losses in revenue, productivity, and customer satisfaction. Therefore, it is essential to ensure that IoT networks remain operational 24/7, with little to no disruption in service.

In this section, we will explore the concept of continuous operation in IoT networks, discussing the challenges and benefits of maintaining uptime, as well as the strategies and best practices for achieving this goal. We will cover topics such as redundancy and failover mechanisms, monitoring and alerting systems, regular maintenance and updates, automation, machine learning, and KPI tracking. By the end of this section, readers will have a solid understanding of the importance of continuous operation in IoT networks and the tools and strategies needed to achieve this goal.

Challenges and benefits of maintaining continuous operation

Maintaining continuous operation of monitoring solutions can be challenging, particularly in the context of IoT networks where the volume of data being generated and transmitted can be significant. The following are key challenges that need to be looked at:

Managing data volumes: The massive influx of data generated by IoT devices can be overwhelming for organizations. Managing and processing the data in real time becomes a challenge, and requires effective data management strategies to ensure that data is processed accurately and efficiently.

Ensuring data accuracy: IoT monitoring data must be reliable and accurate to enable effective decision-making. Any inaccuracies or inconsistencies must be identified promptly, and mechanisms must be in place to rectify them.

Integrating with existing systems: Integration of IoT monitoring solutions into the existing architecture can pose a challenge. An organization needs to ensure that IoT monitoring solutions are compatible with existing systems and that the data from different systems is integrated to provide a comprehensive view of the overall system performance.

Balancing monitoring with system performance: Monitoring solutions generate huge volumes of data, which can consume resources and impact system performance. Organizations must balance monitoring requirements with system performance and implement effective resource management to avoid any adverse impact.

Maintaining security: IoT monitoring systems must be safeguarded from cyberattacks, as they can become a gateway for attackers to gain access to sensitive data. Security protocols must be in place to ensure the safety of the monitoring systems and data.

The challenge of scalability: IoT networks are expanding rapidly, leading to an increase in the volume of data generated. Monitoring solutions must be scalable to handle the growing data volumes, and the monitoring infrastructure must be designed to ensure effective monitoring and management of the system.

Maintaining the continuous operation of IoT monitoring solutions brings several benefits to organizations, including the following:

You should see something like the following after you have uploaded your code and start moving in front of the PIR motion sensor. When the bot is first started up, you will get the text Bot activated, and when you make a movement, you will get the text Motion detected!:

Figure 8.10 – Expected output on the Telegram bot

And that’s it; you’ve made your interoperable solution! Now, upload the code to GitHub and see if you can also make these modifications to your hardware/code. Now, for further understanding and practice on the concepts that you have learned through this practical, you can try doing the following.

Can you add an LED bulb to the circuit to also react when you receive a movement on the PIR motion sensor?

Can you replace the motion sensor with a DHT11 sensor to send messages to Telegram when the temperature rises above 30°C?

Feel free to use the documentation from the Super Starter Kit to also help you navigate the use cases of each sensor and how to properly use them.

Summary

In this chapter, we learned about what interoperability is, why it is important for IoT, and how we can architect solutions for it while navigating through the challenges that are being posed. We then looked further into how it can be beneficial with our practical in building a Telegram chatbot that alerts you to motion detection based on your ESP32, showing how interoperability is imperative to functioning solutions.

Through the discussions and practical exercises in this chapter, readers have gained a comprehensive understanding of interoperability and its significance in IoT. This understanding forms a foundation for creating robust, scalable, and adaptable IoT solutions. Additionally, by exploring how to architect solutions and navigate challenges, readers have acquired valuable insights and strategies that can be applied to their own projects. This equips readers with a broader toolkit to address the complex demands of IoT environments, making them better prepared to contribute to projects in this domain, or even lead initiatives that require a deep understanding of interoperability.

In the next chapter, we will look at operating and monitoring IoT networks.

Further reading

For more information about what was covered in this chapter, please refer to the following links:

Read more on smart home and how interoperability can support it: https://www.iotforall.com/smart-home-interoperability-fragmented-landscape

Explore more insights in IoT interoperability from a governance perspective: https://www2.gov.bc.ca/assets/gov/british-columbians-our-governments/services-policies-for-government/information-management-technology/information-security/information-security-awareness/its_7am_do_you_know_whats_on_your_network_forescout.pdf

Explore more on the passive infrared sensor from the Adafruit documentation: https://cdn-learn.adafruit.com/downloads/pdf/pir-passive-infrared-proximity-motion-sensor.pdf

Learn more on how to use Telegram from its official website: https://core.telegram.org/

Interaction with the Telegram bot will be facilitated using the Universal Arduino Telegram Bot Library, a tool created by Brian Lough that simplifies access to the Telegram Bot API. We will ensure that our Arduino IDE has these libraries and procure them if not. Proceeding from here, we can follow the following steps:

First, we need to download the Universal Arduino Telegram Bot library. We can find it at https://github.com/witnessmenow/Universal-Arduino-Telegram-Bot/archive/master.zip.

We then need to include the library. We need to navigate to Sketch > Include Library > Add. Zip the library and add the library:

Figure 8.7 – Pop-up window to add in the Universal Arduino Telegram Bot library

Important note

You should not be installing the library with the Arduino Library Manager, as a deprecated version may be installed instead.

We then must install the ArduinoJson library. To do this, we navigate to Sketch > Include Library > Manage Libraries.

We then search for arduinojson and install its latest version:

Figure 8.8 – Adding in the ArduinoJson library

With the libraries installed, we can now prepare the hardware.

Hardware setup

We will connect the ESP32 to the PIR motion sensor according to the following diagram. It is a motion sensor that detects movement by sensing changes in infrared radiation emitted by warm objects, such as humans or animals, in its field of view:

Figure 8.9 – Circuit diagram for PIR motion sensor

On the PIR to the ESP32, we will connect the negative terminal to GND, the positive terminal to 5V, and the supply terminal to GPIO 27.

Coding it up

We are now ready to start coding up the necessary code to run the program on our Arduino IDE.

We first declare the necessary libraries for the program. The WiFi.h library is used to connect to the internet over Wi-Fi. The WiFiClientSecure.h library establishes a secure client connection to ensure the data communication is encrypted. The UniversalTelegramBot.h library is for controlling the bot on Telegram, and the ArduinoJson.h library handles the JSON data format used by the Telegram bot:
#include <ArduinoJson.h>
#include <UniversalTelegramBot.h>
#include <WiFiClientSecure.h>
#include <WiFi.h>

We then create a struct to hold our network credentials (SSID and password), and you need to replace the “YOUR_SSID_HERE” and “YOUR_PASSWORD_HERE” placeholder values with your own values. Afterward, a TelegramBot class is defined that encapsulates the functionality of the UniversalTelegramBot library, simplifying our use of it later in the code. After this, we instantiate a NetworkCredentials object with bot_token and chat_id values that you need to replace with the personalized token you received for your bot and the user ID you received for your telegram account respectively, wifi_client as an instance of WiFiClientSecure to handle secure connections, and telegramBot as an instance of our TelegramBot class using the bot token and the secure client:
typedef struct {
    const char* network_id = “YOUR_SSID_HERE”;
    const char* network_pass = “YOUR_PASSWORD_HERE”;
} NetworkCredentials;
class TelegramBot {
public:
    TelegramBot(const char* botToken, WiFiClientSecure& client) : bot(botToken, client) {}
    void sendMessage(const char* chatId, const char* msg) {
        bot.sendMessage(chatId, msg, “”);}
private:
    UniversalTelegramBot bot;};
NetworkCredentials networkCredentials;
const char* bot_token = “6344540752:AAHN_xoPfRipHbAf2d5cbceWLnYvxd2uRiI”;
const char* chat_id = “6394755694”;
WiFiClientSecure wifi_client;
TelegramBot telegramBot(bot_token, wifi_client);

Following that, we set up the PIR sensor pin and a Boolean flag to track whether motion is detected. The detectMotion function will be called whenever the sensor pin detects a rising voltage (that is, motion), setting movementDetected to true:
constexpr int PIR_SENSOR_PIN = 27;
volatile bool movementDetected = false;
void IRAM_ATTR detectMotion() {
    movementDetected = true;}

We then need to create a connectWiFi function that sets the ESP32 to operate in Station (STA) mode and then attempts to connect it to the Wi-Fi network using the credentials we provided earlier. It also sets the certificate root on the secure client. It then waits until the ESP32 is connected before continuing the program:
void connectWiFi() {
    WiFi.mode(WIFI_STA);
    WiFi.begin(networkCredentials.network_id, networkCredentials.network_pass);
    wifi_client.setCACert(TELEGRAM_CERTIFICATE_ROOT);
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);}}

Finally, we create setup() and loop() functions. The setup() function initializes serial communication, sets the PIR sensor pin as an input with a pull-up resistor, and attaches an interrupt to it. It then connects to the Wi-Fi and sends a message indicating that the bot is active. The loop() function is the main loop of the program, which constantly checks if motion has been detected. If so, it sends a message and resets the flag:
void setup() {
    Serial.begin(115200);
    pinMode(PIR_SENSOR_PIN, INPUT_PULLUP);
   attachInterrupt(digitalPinToInterrupt(PIR_SENSOR_PIN), detectMotion, RISING);
    connectWiFi();
    telegramBot.sendMessage(chat_id, “Bot activated”);}
void loop() {
    if (movementDetected) {
        telegramBot.sendMessage(chat_id, “Motion detected!”);
        movementDetected = false;}}

As per usual, verify the code to ensure that you have entered everything correctly. Remember that there are four fields you must personally modify with your own information. If everything is done correctly, you should see the upload be successfully completed and your Telegram bot start churning messages after you have clicked Start on it.

And with that, we are ready to test our implementation.