This module provides focuses on the acquisition and preprocessing of data in a large-scale, distributed environment. The module will describe the challenge of acquiring data from a variety of sources, which may differ in their connectivity, data format and volume, reliability, certainty, accuracy, and precision.
The concepts, techniques, methodologies, and tools used to retrieve and interpret data will be explored and evaluated in an industry relevant manner. The teaching and learning will be predominantly practical in nature with a strong focus on problem-solving and hands-on programming of functional code.
The module will utilise state of the art software frameworks for data acquisition (e.g., RESTful or web services) and the most appropriate data formatting/interpretation approach (e.g., JSON, XML).
Basic network programming –sockets, synchronous/asynchronous communications.
Data processing – data structures, data formats, data interpretation, data filtering.
Message-oriented architecture – integration of independent services and their inter-communication.
RESTful service – setup, connectivity, communication, JSON.
Web services – setup, connectivity, communication, DOM tree.
Distributed transactions – techniques to handle fault tolerance for data retrieval, eventual consistency, commitment protocols.
Big data implications – techniques for streaming, caching, paging, and storage.
Data validation – techniques for handling data uncertainty, accuracy, and precision.
Lectures/labs, discussion, practical examples, problem-solving exercises, project work, self-directed learning.
Note: computer labs must have the relevant software installed and available to students.
| Module Content & Assessment | |
|---|---|
| Assessment Breakdown | % |
| Other Assessment(s) | 100 |