Panagiotis Koilakos

Associate Operational Data Management Officer, UNHCR

MSc in Data Science - Deciphering Big Data

In general

The Data Collection Process

 Data volume production has increased exponentially throughout the years. While only two zettabytes of data were produced in 2010, in 2016, the data production volume reached 26 zettabytes, and by 2025 it is estimated to reach 181 zettabytes. With nearly 80 zettabytes in data connections projected by 2025 through IoTs, almost 50% of worldwide data volume will come from such systems (Statista, 2022).

 Such volume presents unlimited opportunities but simultaneously creates many risks and challenges closely associated with technical limitations. As we live in an era where data protection is becoming more and more trending, with Data Protection Officers being required for every company under the General Data Protection Regulations applied in the EU and being embedded in the national legislation of most, if not all, EU countries. Ensuring that data are always safe, especially for interconnected and "always-available" devices, is one of the most critical data protection-related challenges (Ramachandran et al., 2018). In addition to the security issues, several constraints, such as bandwidth limits, processing speed, battery capacity and availability, especially in devices with integrated computing power, make using IoTs a decision that needs to be well thought out (Ramachandran et al., 2018; Dian et al., 2020).

 On the other hand, the same elements that create the challenges make using IoTs a popular option. Even though data protection presents a current challenge (coupled with the limited relevant regulation), anonymity can be achieved through blockchain technologies. Moreover, the broad spectrum of applications makes IoTs a go-to solution for many industries that aim for automation. Finally, the data production, linked with the relatively low cost of such devices, makes their use in many applications appealing, as the data wealth can be enormous, especially for machine learning applications (Ramachandran et al., 2018; Statista, 2022; Shafique et al., 2018).

 Data show that IoT devices are used daily, assisting companies and individuals in their daily routines. Even though the opportunities are virtually endless, one must remember that such applications are not a panacea, and the advantages must be cogitated against the disadvantages.

Back to contents

Dian, J., Vahidnia, R. & Rahmati, A. (2020) Wearables and the Internet of Things (IoT), Applications, Opportunities and Challenges: A Survey. IEEE Access 8: 69200-69211. DOI: 10.1109/ACCESS.2020.2986329.

Ramachandran, G. & Krishnamachari, B. (2018) Blockchain for the IoT: Opportunities and Challengers . s.I: s.n. Available from: https://www.researchgate.net/publication/325033863_Blockchain_for_the_IoT_Opportunities_and_Challenges [Accessed 20 October 2022].

Shafique, M. et al. (2018) An Overview of Next-Generation Architectures for Machine Learning: Roadmap, Opportunities and Challenges in the IoT Era. Design, Automation & Test in Europe Conference & Exhibition n.K: 827-832. DOI: 10.23919/DATE.2018.8342120.

Statista (2022) Data volume of internet of things (IoT) connections worldwide in 2019 and 2025. Available from: https://www.statista.com/statistics/1017863/worldwide-iot-connected-devices-data-size/ [Accessed 15 October 2022].

Statista (2022) Volume of data/information created, captured, copied, and consumed worldwide from 2010 to 2020, with forecasts from 2021 to 2025. Available from: https://www.statista.com/statistics/871513/worldwide-data-created/ [Accessed 17 October 2022].


Development Project: Proposed Database Design

Abstract

 In 2021, the Ministry of Transport and Economy for Wales launched the Llwybr Newydd – the Wales Transport Strategy 2021. This 20-year strategy aims to reshape the transport system and focuses on people and climate.

 KTK Consultants (KTK) have been commissioned to design a logical database that will be the backbone of the integrated ticketing system. This system will allow passengers to purchase tickets and use them across all modes of transport provided by Transport for Wales (TFW) (one card, all services). KTK is pleased to present a proposal design report for the Transport for Wales (TFW) data management system.

View embedded View in new page Download and View PDF file
Back to contents

Development Project: Executive Summary

Abstract

 In an ever-increasing digital age, where customers are used to convenience and quick turnarounds, TfW aims to improve how passengers purchase tickets across all their various transport offerings. TfW's primary goal is to provide a ticketing system that allows passengers to conveniently purchase tickets from retail outlets, TfW stations, and digital platforms. The system will be supported through a real-time geolocating system that tracks bus and train arrival times, busy routes, travel patterns and passenger habits (Finzgar & Trebar, 2011). The goal is to use this data to inform future transport planning and resource allocation across Wales.

View embedded View in new page Download and View PDF file
Back to contents

Task: Normalisation and Data Build

Abstract

 In this exercise, we are provided with an initial data table in an un-normalised form. The task is to normalise the table down to the 3rd Normal Form by showing and explaining each subsequent step (1st Normal Form, 2nd Normal Form, 3rd Normal Form). Following the normalisation, the data build task requires to create the actual relational database with the proper schema, linked tables mapped through the foreign keys, and primary key properly recorded. The database should enforce referential integrity.

View embedded View in new page Download and View PDF file
Back to contents

Feautured Resources