week 7

3.7 Data dimensions and maintenance

3.7.1 Know the definitions of the six Vs (dimensions) and understand the six Vs (dimensions) of Big Data and their impact on gathering, storing, maintaining and processing:

Big Data refers to large, complex datasets that traditional data processing tools struggle to handle effectively. The “Six Vs” represent the key dimensions that define Big Data and influence how it is gathered, stored, maintained, and processed. These six dimensions - Volume, Variety, Velocity, Veracity, Variability, and Value - highlight the challenges and opportunities involved in working with massive data collections. Together, they shape the strategies organisations use to collect and manage data efficiently, ensuring it can be transformed into meaningful insights that support business and technological decisions.

Volume

This refers to the sheer amount of data being generated and stored. Modern systems collect data from multiple sources—such as social media, IoT devices, sensors, and business transactions—resulting in massive quantities of data that must be processed. Managing large volumes requires scalable storage solutions such as cloud systems and distributed databases to ensure data can be efficiently stored and accessed when needed.

Variety

Variety relates to the different types and formats of data available. Data can be structured (e.g. databases and spreadsheets), semi-structured (e.g. XML or JSON), or unstructured (e.g. videos, social media posts, and images). Handling such diverse formats requires flexible data management tools capable of integrating, converting, and analysing multiple data types to create a complete picture.

CD Catalogue xlm Example

Super Hero json Example

Velocity

Velocity refers to the speed at which data is generated, transmitted, and processed. In the digital age, data streams in real-time from online transactions, smart devices, and live monitoring systems. Organisations must employ high-speed processing techniques, such as stream analytics and edge computing, to capture and respond to information quickly and make timely decisions.

Veracity

Veracity focuses on the quality, accuracy, and trustworthiness of data. With data coming from various sources, it is crucial to validate and clean it to remove inconsistencies or errors. Poor data quality can lead to false insights, so effective verification and governance mechanisms are essential for maintaining reliability in analysis and reporting.

Variability

Variability concerns the inconsistency and fluctuation of data flows. Data volumes and formats can change unpredictably, especially during peak events like marketing campaigns or global news trends. Systems need to adapt to handle these irregular patterns and ensure stability in storage and processing performance.

Value

Value represents the ultimate purpose of Big Data—extracting meaningful insights that provide real-world benefits. Not all collected data holds equal importance; therefore, identifying data that supports decision-making and business improvement is vital. The true success of Big Data lies in its ability to generate measurable outcomes, such as improving efficiency, customer satisfaction, or innovation.

Case Study: Big Data in the Retail Industry – Tesco’s Smart Data Analytics System

Tesco, one of the UK’s largest supermarket chains, uses Big Data analytics to better understand customer behaviour, optimise stock levels, and improve marketing strategies. Through its Clubcard loyalty scheme, online shopping platforms, and in-store systems, Tesco gathers enormous amounts of data daily. This case study explains how each of the Six Vs applies to Tesco’s operations and decision-making processes.

Volume

Tesco gathers data from over 19 million Clubcard members, capturing details of every transaction, including items purchased, time, location, and payment method. This results in terabytes of data generated each day. To manage this volume, Tesco employs cloud storage and distributed databases to scale up as data grows, ensuring it can process customer insights efficiently.

Impact: Large volumes of data enable Tesco to predict buying patterns and adjust stock levels in real time, reducing waste and improving supply chain efficiency.

Variety

Data comes in many forms: structured data (sales figures, barcodes, inventory logs), semi-structured data (customer emails, loyalty data), and unstructured data (social media posts, customer reviews, CCTV footage). Tesco’s analytics systems integrate these diverse data types to gain a complete understanding of customer behaviour and store performance.

Impact: The ability to combine and analyse multiple data types helps Tesco personalise offers and target promotions more effectively.

Velocity

Tesco processes data at high speed from checkout systems, mobile apps, and online orders. Real-time analytics allow the company to monitor stock movement, detect supply issues, and respond quickly—for example, replenishing fast-selling items before they run out.

Impact: High data velocity enables Tesco to maintain a smooth shopping experience and optimise operations dynamically, ensuring customers always find what they need.

Veracity

Not all data collected is accurate—errors can occur due to scanning mistakes, incomplete customer profiles, or out-of-date information. Tesco employs data validation and cleaning tools to filter out duplicates, check accuracy, and ensure the data it relies on for decision-making is trustworthy.

Impact: Reliable data ensures that business decisions, such as pricing and promotions, are based on accurate insights rather than flawed information.

Variability

Customer buying behaviour changes frequently due to factors such as weather, holidays, or economic conditions. For example, during hot weather, Tesco sees spikes in sales of barbecue items and cold drinks. Data patterns fluctuate, so Tesco’s systems must adapt to handle these surges and seasonal trends.

Impact: By analysing variable data patterns, Tesco can forecast demand more accurately and adjust marketing campaigns accordingly.

Value

Ultimately, Tesco’s Big Data system generates value by transforming information into actionable insights. By understanding what products customers buy, when, and why, Tesco tailors promotions to individual shoppers, enhances customer satisfaction, and increases revenue. It also reduces waste through better inventory management.

Impact: Big Data creates measurable business value by improving efficiency, profitability, and customer loyalty.

Tesco’s use of Big Data demonstrates how the Six Vs influence every stage of data management:

• Gathering: Multiple data sources (tills, apps, sensors, social media)

• Storing: Cloud-based scalable databases

• Maintaining: Data cleaning and validation

• Processing: Real-time analytics to extract insights

By addressing the challenges and opportunities of Volume, Variety, Velocity, Veracity, Variability, and Value, Tesco maintains a competitive edge in the retail industry through data-driven decision-making.

Understanding the Six Vs of Big Data
Scenario:
You have been asked by a digital data consultancy company to explore how organisations use Big Data to improve their services and operations. Your manager wants you to demonstrate your understanding of the Six Vs (Volume, Variety, Velocity, Veracity, Variability, and Value) by applying them to a real-world organisation.

Your Task:
You are to work independently to research and create a short written or visual report (around one page or one presentation slide per section) explaining how a company of your choice applies the Six Vs of Big Data in its operations.

You must:
1. Choose an organisation that uses Big Data (for example: Amazon, Netflix, Tesco, NHS, Transport for London, or Spotify).
2. Describe how each of the Six Vs applies to your chosen company - explain what type of data they collect, how they manage it, and what impact it has on decision-making.
3. Explain how the Six Vs affect the organisation’s ability to gather, store, maintain, and process its data.
4. Conclude your task by identifying the value and benefits gained by the organisation from using Big Data.

Presentation Format Options:
You may present your findings in one of the following formats:
 - A one-page infographic showing each of the Six Vs with examples.
 - A PowerPoint or Google Slides presentation (6 slides minimum - one per “V”).
 - A short written report using subheadings for each “V.”

Extension Challenge:
Reflect on the challenges your chosen organisation might face if one of the Six Vs was not managed effectively. For example, what would happen if the data lacked veracity (accuracy) or velocity (speed)?

Time Allocation: 25–30 minutes

Success Criteria:
Each of the Six Vs is clearly explained in relation to a real company.
You use appropriate technical language (e.g., data accuracy, scalability, analytics).
You demonstrate understanding of how Big Data impacts decision-making and operations.

3.7.2 Know the definition of Big Data and understand that it has multiple dimensions.

Big Data is more than just large datasets - it is multi-dimensional, shaped by the Six Vs that define its complexity and usefulness. Each dimension affects how data is collected, stored, processed, and used to create value. By managing these dimensions effectively, organisations like Netflix, Amazon, and Tesco can turn vast amounts of raw information into powerful insights that drive innovation and competitive advantage.

3.7.3 Understand the impact of each dimension on how data is gathered and maintained.

3.7.4 Know the definitions of data quality assurance methods and understand their purpose and when each is used:

Data quality assurance refers to the processes and methods used to ensure that data is accurate, consistent, reliable, and suitable for its intended purpose. These methods are vital for maintaining trust in information systems and supporting effective decision-making. Without quality assurance, data can become misleading, duplicated, or corrupted - leading to costly mistakes in business or digital systems. The key methods include validation, verification, reliability, consistency, integrity, and redundancy management. Each plays a specific role in checking, maintaining, and safeguarding data throughout its lifecycle - from collection and storage to processing and analysis. These methods are applied at different stages depending on the goal: ensuring data entered is correct, confirming it has not changed unintentionally, and maintaining stable, accurate datasets over time.

Validation

Definition: Validation ensures that data entered into a system meets defined rules and criteria.
Purpose: It prevents incorrect or incomplete data from being stored. For example, ensuring a date of birth is in the correct format or that an email address contains “@”.
When Used: During data entry or import, when new data is collected or updated.
Example: An online form that rejects a postcode if it doesn’t match the UK format (e.g., “ME4 4QF”).

Verification

Definition: Verification checks that data entered into a system matches the original source or intended value.
Purpose: To confirm that information has been transferred or recorded correctly.
When Used: Typically used after data entry or data transfer between systems.
Example: Double-entry verification where a user must type an email address twice to confirm accuracy, or comparing paper forms to digital entries.

Reliability

Definition: Reliability measures how dependable and consistent the data is over time.
Purpose: To ensure that the same data gives the same results whenever it is accessed or used.
When Used: During testing, auditing, or repeated analysis phases.
Example: A system that consistently returns the same sales figures when queried shows reliable data.

Consistency

Definition: Consistency ensures that data values remain uniform across different systems or databases.
Purpose: It prevents conflicting information from existing in multiple locations.
When Used: During data synchronisation, database integration, or migration.
Example: A customer’s address should be identical in both the billing and shipping databases.

Integrity

Definition: Data integrity ensures that information remains accurate, complete, and unaltered during its storage or transmission.
Purpose: To maintain trustworthy and secure data that has not been tampered with or corrupted.
When Used: Throughout the entire data lifecycle, especially during transfers or updates.
Example: Using encryption or checksums to confirm that data has not changed during transfer.

Redundancy

Definition: Redundancy involves managing and reducing unnecessary duplication of data.
Purpose: To prevent wasted storage space, confusion, or outdated information being used.
When Used: During database design or maintenance, such as when normalising data.
Example: Removing repeated customer details stored across multiple tables in a database.

Case Study: Data Quality in the NHS Patient Records System

The National Health Service (NHS) manages vast amounts of patient data, from personal details to medical histories. To maintain high-quality, reliable information, the NHS employs strict data quality assurance methods:

• Validation is used at the point of entry when staff input patient data into electronic systems. For instance, fields such as NHS number, postcode, and date of birth must follow strict validation rules.

• Verification takes place when transferring records between hospitals to ensure information matches across systems.

• Reliability is ensured through regular audits that check for missing or duplicate patient records.

• Consistency ensures that updates made in one department’s database are reflected across others.

• Integrity is maintained through strong encryption and access controls to protect sensitive data.

• Redundancy is reduced by using a centralised database that links patient data, preventing unnecessary duplication.

By applying these methods, the NHS ensures patient information remains accurate, secure, and consistent across the UK healthcare system, enabling effective and safe medical care.

Checking Data Quality
Scenario:
You have been asked to act as a Data Quality Analyst for a fictional company, TechHealth Ltd, which stores patient and device data. Your job is to identify and correct poor-quality data in a small dataset.

Task Instructions:
You are to:
1.Review the sample dataset (provided by your teacher or created in Excel/Google Sheets).

Download file

2. Identify examples of poor data quality, such as:
 - Missing entries (e.g., blank postcodes)
 - Duplicated data (e.g., same patient ID repeated)
 - Incorrect data formats (e.g., “12/45/2024” as a date)

3.For each issue found, decide which data quality assurance method (validation, verification, reliability, consistency, integrity, or redundancy) could solve the problem.

4. Create a short written explanation or table showing:
 - The problem identified
 - The appropriate method to fix it
 - Why that method is suitable

Extension Challenge:
Suggest two ways TechHealth Ltd could automate data quality assurance in the future (e.g., automated validation scripts, database constraints).


Success Criteria:
You correctly identify at least 3–5 data quality issues.
You match each issue with the correct assurance method.
You clearly explain how each method helps maintain accurate and reliable data.

3.7.5 Know and understand factors that affect how data is maintained:

Maintaining data effectively involves keeping it accurate, up to date, secure, and accessible throughout its lifecycle. Several factors influence how well an organisation can manage its data — including time, skills, and cost. Data maintenance is not a one-off process; it requires continuous monitoring, updating, and validation to ensure the information remains relevant and reliable. Poorly maintained data can lead to errors, inefficiencies, and compliance risks, especially in sectors where accuracy is critical, such as healthcare, finance, or education. Successful data maintenance depends on allocating the right resources, staff training, budgeting for maintenance tools, and dedicating sufficient time to review and update records.

Time

Explanation:
Data maintenance requires ongoing time investment. Regular reviews must be scheduled to update outdated information, delete unnecessary records, and back up data securely. If data is not maintained in a timely manner, it can quickly become obsolete or misleading.
Example:
TfL schedules automatic daily updates to its Oyster card usage database to ensure that passenger numbers and travel patterns are current.
Impact:
Allocating sufficient time for regular updates prevents the accumulation of errors and supports real-time decision-making.

Skills

Explanation:
Skilled staff are essential for effective data maintenance. Employees need training in data management tools, security protocols, and database systems to ensure that updates and checks are performed correctly.
Example:
TfL employs data engineers and analysts trained in database administration, cybersecurity, and analytics tools such as SQL and Power BI.
Impact:
Without the right skills, mistakes can occur—such as deleting important data or failing to detect errors—which can damage data reliability.

Cost

Explanation:
Maintaining data has both direct and indirect costs, including software licences, staff wages, security systems, and hardware storage. Organisations must balance the value of maintaining accurate data with the cost of implementing it.
Example:
TfL invests heavily in cloud storage and predictive maintenance systems, which reduce long-term operational costs by improving reliability and performance.
Impact:
While high-quality data maintenance can be expensive, it saves money in the long term by preventing inefficiencies, downtime, and poor decision-making.

Case Study: Data Maintenance at Transport for London (TfL)

Transport for London (TfL) collects and manages enormous amounts of data daily - including Oyster card usage, contactless payments, GPS bus tracking, and maintenance schedules for underground services. To ensure that this data remains accurate and useful, TfL must balance time, skills, and cost effectively. The data is used to plan routes, predict passenger flow, and manage safety systems. However, without regular maintenance - such as updating route changes, deleting old data, and verifying passenger numbers - the system could become inaccurate, leading to delays, poor planning, and wasted resources. TfL’s ongoing investment in skilled data analysts and automated systems ensures that its transport network runs smoothly and efficiently.

Analysing Data Maintenance Factors
Scenario:
You have been asked to act as a Data Administrator for a company called EcoPower Ltd, which manages renewable energy sites across the UK. The company is facing issues with inaccurate energy output reports and incomplete maintenance records.

Your Task:
You are to work independently to investigate and explain how time, skills, and cost influence the company’s ability to maintain accurate data.

Instructions:
1. Read the scenario carefully.
2. Write a short report (around 250–300 words) that includes:
 - An explanation of how each factor (time, skills, and cost) affects data maintenance in EcoPower Ltd.
 - Suggestions for how the company could improve its data maintenance processes.
 - Examples of digital tools or systems (e.g., cloud storage, automated backups, data validation scripts) that could support efficient maintenance.
3. Conclude by identifying which factor you think is the most important and justify your choice.

Outcome:
By the end of the activity, you will have demonstrated your understanding of:
How time, skills, and cost impact data maintenance.
The practical decisions organisations must make to ensure data reliability.
How to propose realistic solutions for improving data maintenance within a business context.

Success Criteria:
Each factor (time, skills, cost) is clearly explained with examples.
You demonstrate understanding of how these affect data accuracy and accessibility.
You provide realistic and practical recommendations for improvement.

3.7.6 Understand the interrelationships between the dimensions of data, quality assurance methods and factors that impact how data is maintained and make judgements about the suitability of maintaining, transforming and quality assuring data in digital support and security.

Last Updated
2025-11-18 14:58:37