Technical requirements – Working with Data and Analytics

by Keitra Marche AWS Certification Exam, Monitoring for security purposes, Monitoring ingested data

Managing data and performing analytics on it is a crucial aspect of any Internet of Things (IoT) deployment. It allows you to gain insights based on the large amounts of data generated by IoT devices and make appropriate decisions based on data to improve operations, increase efficiency, and reduce costs. With Amazon Web Services (AWS) and other cloud providers, there are a variety of services that you can use to analyze and visualize data that you have obtained from your IoT devices, from simple data storage and retrieval options that you can configure without much difficulty to more complex analytics and machine learning (ML) tools, which you may have to learn and fine-tune, that you perform as part of the analysis.

Often, data analytics is the piece of the puzzle that completes the picture that we are trying to architect with our IoT networks, as even with edge networks in which we process data on the edge nodes to reduce costs, there is usually always further processing and storage that we want to perform when the data reaches the cloud. We want to do so while still optimizing based on the options that we have within AWS and looking further into how we can adhere to best practices within AWS’ Well-Architected Framework to make the best use of our resources. The link to the framework can be found at the end of the chapter.

In this chapter, we’re going to cover the following main topics:

Introduction to data analysis at scale

Analysis on the cloud and outside

Practical – smart home insights with AWS IoT Analytics

Industrial data analytics

Practical – creating a data pipeline for end-to-end data ingestion and analysis

Technical requirements

This chapter will require you to have the following software installed

Arduino IDE

AWS account

We will be running our programs on Python and have a bit of Structured Query Language (SQL) syntax, a standardized programming language used for managing and manipulating relational databases, that we need to use as part of querying data in this chapter; again, don’t worry if you don’t understand some of the code — we will walk you through it and get you down to understanding how each part of the code works in no time.

You can access the GitHub folder for the code that is used in this chapter at https://github.com/PacktPublishing/IoT-Made-Easy-for-Beginners/tree/main/Chapter10.