COMPARATIVE ANALYSIS OF IOT DATA PROCESSING ARCHITECTURES FOR ENVIRONMENTAL MONITORING ON RASPBERRY PI 5

Abstract

The paper represents a comparative study of the performance of hybrid micro-batch processing strategies for environmental data using the Raspberry Pi 5 microcomputer. The experiment is based on a real stream of data from BME688 and SCD41 sensors, collected over 142 hours. The study compares six data storage systems of different architectural classes: SQLite, PostgreSQL, DuckDB, Polars, ClickHouse, and Delta Lake. The experiment involved micro-batch data processing with the accumulation of 50-record batches and the recording of system performance metrics. Statistical parameters of recording time, processor load, and RAM usage, including mean values, standard deviation, and quantile latency, as well as the amount of disk space required to store data, were used to analyse the results. The obtained experimental measurements allow a direct comparison of the behaviour of different data storage architectures under identical operating conditions on a resource-constrained edge device. The analysis focuses not only on average execution performance but also on the variability of delays that may occur during data ingestion operations. Such an approach makes it possible to evaluate the stability and reliability of each system in real-time data processing scenarios. The results show significant performance differences between the systems studied. The lowest delays in recording micro-batches were recorded for SQLite, ClickHouse, and Polars, while the use of Delta Lake is accompanied by significant overhead costs of computing resources on peripheral equipment. Additional observations indicate that lightweight or embedded data processing solutions demonstrate better suitability for edge computing environments compared to complex distributed architectures that require additional processing layers. The study provides an empirical basis for choosing optimal data storage architectures in real-time environmental monitoring systems running on devices with limited hardware resources and contributes to the development of efficient IoT data processing pipelines for edge-based monitoring infrastructures.