Chapter 2 — Data Representations, Models, and Contextual Quality Dimensions

Metadata

Opening Quote

MSQE educational framing: A representation can be valid and still misrepresent the fact that a decision depends on.

Opening Story

The following illustrative scenario concerns Atlas Commerce, a fictional retailer. The product team launches a fulfilment dashboard intended to show orders that can be packed today. The dashboard receives a field called delivery_date from an order event and displays “Today” for 1,126 orders.

The implementation appears sound. Every event contains a non-empty date in the format 2026-08-11. A schema check passes, the dashboard renders, and a small set of orders agrees with the customer-facing application.

Two warehouses use different local time zones. The order event’s delivery_date is a date derived from a promised delivery timestamp in the shopper’s time zone, while the packing decision must use the warehouse’s operational day. For orders near midnight, “today” means one day to the shopper and another day to the warehouse. The field is structurally valid, but its meaning is unsuitable for the packing decision.

The team also discovers that a subset of older orders uses NULL for the promised timestamp and a status code of PENDING_DATE. A non-null dashboard value was generated by defaulting the missing date to the ingestion date. This makes the dashboard look complete while hiding uncertainty about whether an order is packable.

The issue is not a failure to parse a date. It is a failure to preserve and communicate the representation, source, time zone, and uncertainty that the operational decision requires.

Why This Chapter Matters

Chapter 1 established that a data-quality claim must name its consumer, population, source, evidence, limitation, and residual risk. This chapter explains why those elements cannot be assessed without understanding representation.

The same business fact can appear in a user interface, an API payload, a relational row, a message, an exported file, and an aggregate. Every representation makes choices about type, identifier, precision, nullability, units, codes, time, and constraints. Those choices can clarify meaning, lose meaning, or create ambiguity. A field may satisfy a schema while violating a business expectation; two systems may both be internally consistent while representing different concepts.

Data-quality dimensions such as correctness, completeness, consistency, uniqueness, validity, timeliness, and integrity give teams a useful vocabulary. They are not independent boxes that can be marked complete for every dataset. This chapter treats them as contextual lenses for a decision. Chapter 3 will use those lenses when designing query-based evidence about populations, keys, relationships, and unmatched records. This chapter does not teach data modelling notation, database administration, or a product-specific schema-management tool.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish conceptual business meaning from logical and physical data representations;
• identify representation risks involving types, domains, identifiers, nullability, units, precision, rounding, coded values, and temporal values;
• explain how structured and semi-structured data can carry different quality risks for the same business fact;
• apply correctness, completeness, consistency, uniqueness, validity, timeliness/freshness, integrity, and semantic fitness as contextual quality lenses;
• explain why a structurally valid record can remain semantically unsuitable for a consumer decision;
• assess how schema constraints help, and where they provide insufficient evidence; and
• produce a representation-aware analysis that states assumptions, limitations, and residual risk rather than a generic quality score.

Meaning, Models, and Representations

A business fact is not its storage format

A conceptual model describes business facts and relationships without committing to a particular storage or transport technology. “An order has a customer, a promised delivery time, and a monetary amount” is conceptual. A logical model describes the structure used to organise those facts, such as entities, attributes, relationships, and identifiers. A physical representation describes how a system stores or transmits them: columns, JSON properties, files, message fields, encodings, indexes, or partitions.

These layers are related but not interchangeable. A product manager may say “delivery date” to mean the day a customer expects a parcel. A data warehouse may store a UTC timestamp, an event stream may carry a local-date string, and a dashboard may derive a business-day label. Each can be appropriate for a purpose. Problems occur when a consumer assumes that one representation carries a meaning that belongs to another.

A useful quality review traces a decision back through these layers. It does not assume a column name, JSON property, or report label is a complete definition.

Structured and semi-structured data

Structured data follows a predefined organisation whose fields and relationships can be consistently addressed, such as rows in a relational table. Semi-structured data carries some organisation but can vary by record, such as JSON events with optional nested fields. Neither form is inherently better quality.

Structured data can provide useful constraints and predictable query paths. It can still contain stale, incorrectly mapped, or semantically ambiguous values. Semi-structured data can preserve source detail and accommodate evolution. It can also make omissions, inconsistent field names, nested meanings, and version differences harder to discover.

The appropriate question is not “is JSON less reliable than a table?” It is “what representation assumptions does this consumer rely on, and what evidence would show whether those assumptions hold?”

Schema is a contract of structure, not a full contract of meaning

A schema specifies expected structural properties of data, such as fields, types, allowed values, relationships, requiredness, and sometimes constraints. A schema can prevent or reveal many defects. It does not normally express every business definition, historical exception, consumer need, or quality trade-off.

For example, a schema may state that currency_code is a three-character string. It may constrain the string to an allowed code list. It cannot by itself establish that the amount uses the same currency as the order, that conversion occurred at the intended point, or that a finance report applies the correct rate for the reporting period.

Schema validation is therefore valuable evidence at a structural boundary. It should be described as such. Chapter 8 will later examine data contracts, ownership, and evolution; this chapter concentrates on representation-aware reasoning.

Representation changes are quality changes

Changing a representation can change the meaning available to a consumer even when a migration completes without errors. Renaming a field, reducing decimal precision, changing an enum, replacing a nullable field with a default, or deriving a date from a timestamp can alter the facts that downstream code and people infer. A backwards-compatible payload can still be semantically disruptive if an old value is reinterpreted.

Before a representation change, identify the affected business fact, consumers, source-to-target mappings, historical records, and exception paths. Define how old and new values coexist, how unknown values are displayed, and what evidence will show that a critical population retained its intended meaning. This is not a full migration or data-contract process; it is the representation-risk question that should precede one.

For Atlas Commerce, replacing PENDING_DATE with a default calendar date might simplify a dashboard filter. It also removes a distinction between “not scheduled” and “scheduled on this date.” The change should be judged against the packing decision, not only whether existing consumers can parse the new field.

Types, Domains, and Missingness

Types answer some questions and conceal others

A type communicates an expected form: integer, decimal, string, Boolean, date, timestamp, array, object, and so on. A domain further limits meaningful values, such as a status enumeration, a range of valid quantities, or a recognised currency code. Choosing types and domains deliberately reduces ambiguity and supports validation.

But a technically compatible type can still be a poor representation. A customer identifier stored as a number may lose leading zeroes. A monetary value stored as a floating-point number can introduce representation and rounding behaviour inappropriate for an accounting decision. A free-text status can accept a misspelling that no consumer understands. A string can represent a date without identifying its calendar, time zone, or precision.

The engineer should ask what distinctions must survive. If a consumer must distinguish “not provided,” “not yet known,” “not applicable,” and “intentionally withheld,” one generic empty value is not enough.

Null is a state, not an explanation

In many data systems, NULL represents the absence of a value. Its precise behaviour differs by language, database, serialization format, and query semantics. It is not automatically equivalent to an empty string, zero, false, omitted property, or unknown business fact.

The operational risk comes from collapsing distinct meanings. At Atlas Commerce:

Treating all of these as “non-null after transformation” can make a dataset appear complete while removing information needed to assess uncertainty. A quality rule should name the business condition, not only require a non-null database value.

Validity is contextual

Validity asks whether data conforms to a relevant rule or allowable domain. The relevant rule might be a structural type, a regular expression, a business rule, a reference list, or a relationship constraint. Validity is useful because it makes expectations executable or reviewable.

However, validity is not the same as truth. The date 2026-08-11 can match the expected format while being the wrong date. A recognised ISO currency code can be valid while being attached to the wrong amount. A customer identifier can exist in a database but belong to a different legal entity. State the rule, its purpose, and its boundary before claiming that data is “valid.”

Identity, Keys, and Relationships

An identifier must identify something deliberately

An identifier is a value used to distinguish an entity, event, record, or version. A key is an identifier used in a data model or relation to establish uniqueness, access, or relationship. A key can be natural, such as a stable business reference, or surrogate, such as a generated technical identifier. Neither choice removes the need to define what is identified and at what scope.

order_id may uniquely identify an order across the Atlas Commerce platform. order_line_id may identify a line only within an order. payment_attempt_id may identify an attempt rather than a settled payment. An event identifier may identify a delivery notification, not the current order state. Duplicate detection that ignores these distinctions can report false defects or conceal real ones.

Uniqueness is therefore a contextual question: unique according to which key, within which population, during which time period, and for which lifecycle state? A historical table may legitimately contain several versions of the same business entity. A current-state table may not.

Integrity includes relationships and meaning

Integrity is used in several technical contexts. In this chapter it refers broadly to the preservation of expected data relationships, constraints, and meaning. Referential integrity is the more specific condition that a relationship reference points to an existing, appropriate parent record according to the model’s rules.

For example, an order row that references a non-existent customer may violate referential integrity. An order that references an existing customer from the wrong tenant may satisfy a simple existence check while violating a business boundary. A refund that refers to a valid payment attempt but exceeds its settled amount may preserve one relationship while violating another invariant.

Relationships should be evaluated in the context of lifecycle and timing. A child event can arrive before its parent record in an asynchronous system. That may be an expected temporary state, a replay condition, or evidence of loss. The quality claim must name which interpretation matters for the consumer decision.

Units, Precision, and Rounding

Units are part of the fact

A quantity without its unit is incomplete for many decisions. 1200 might mean 1,200 cents, 1,200 euros, 1,200 grams, or 1,200 milliseconds. A field name can suggest a unit, but a reliable representation preserves or governs it explicitly enough for the consumer.

Unit-conversion defects frequently look plausible. A dashboard can display an amount that is internally consistent but one hundred times too large because cents were interpreted as dollars. A shipment weight can pass a numeric range check while kilograms are treated as pounds. A Quality Engineer should identify unit assumptions at input, transformation, storage, and display boundaries.

Precision and rounding encode policy

Precision is the amount of detail represented; rounding is the rule used when a value cannot or should not retain all detail. Both can affect equality, aggregation, reconciliation, and customer outcomes.

Consider an order with three line-item discounts calculated to more decimal places than a currency display. Rounding each line before summing can produce a different total from summing first and rounding the total. Neither process is automatically wrong. The business, accounting, or contractual rule must state which result is intended, how adjustment is handled, and which representation a consumer uses.

Avoid using approximate floating-point values as a generic model for currency without understanding the language and storage behaviour. This is not a mandate for one database type or programming language. It is a reminder that technical representation choices must support the intended arithmetic and auditability.

Coded Values, Dates, and Time

Enumerations and codes need owned meaning

An enumeration is a controlled set of named values. A coded value may be a compact representation of an enumeration, such as S, P, and R for shipment states. Controlled values can improve consistency and enable validation, but they need definitions, ownership, and evolution rules.

If one service uses CANCELLED to mean customer cancellation and another uses it to include stock unavailability, a cross-system report can be consistent in spelling but inconsistent in meaning. A code list also changes over time. An unrecognised new value should not silently become a default category unless that behaviour is explicitly appropriate for the decision.

A date is not a timestamp

A date represents a calendar day. A timestamp represents a point in time, often with a time zone or offset. A local time without offset can be ambiguous during daylight-saving transitions. Event time, processing time, reporting time, and display time can all differ.

The opening story shows why an ISO-looking date is insufficient for a warehouse decision. A consumer needs to know the calendar and time-zone convention used to derive it, the source event or rule, and the time at which the representation was current. Chapter 7 will examine temporal and streaming data in depth; here, establish the habit of asking what time concept a field represents.

Timeliness and freshness depend on a decision window

Timeliness or freshness describes whether data is available and current enough for its purpose. A seven-day-old product catalog may be acceptable for historical analysis and harmful for checkout inventory. A payment feed delayed by two hours may be tolerable for a daily close and unacceptable for a fraud decision.

Do not describe data as fresh without naming the consumer, expected update cadence, cutoff, and consequence of delay. A recently updated record can still be stale if its source has not received a relevant event. Conversely, an older record can be correct if the underlying fact has not changed.

Quality Dimensions as Contextual Lenses

ISO/IEC 25012 provides a formal data-quality model for structured data.¹ Teams also use overlapping terminology in data management, product, and engineering work. The following lenses are practical teaching terms, not a claim that every standard uses identical definitions or that one composite score can represent quality.

Dimensions interact; they are not a menu of independent checks

An Atlas Commerce finance report can be complete for settled card payments yet incomplete for the revenue claim if bank transfers are in scope. A timestamp can be valid in format but semantically unfit if it uses processing time where the metric needs event time. A report can contain no duplicate rows while double-counting revenue because several different rows refer to one commercial transaction.

The appropriate lenses arise from the risk. A customer-notification job may prioritise correct contact preference and timely delivery. An audit record may prioritise traceability, historical integrity, and preservation of original values. A product dashboard may accept delayed aggregation while requiring a clear “as of” time. Make trade-offs visible rather than claiming all dimensions are equally important.

A representation and quality-dimension analysis

The following analysis is an original MSQE educational framing for a focused review:

1. Name the consumer decision and the business fact it relies on.
2. Trace that fact through conceptual, logical, physical, and consumer representations.
3. Identify representation-sensitive assumptions: type, identifier, missingness, unit, precision, code, time, relationship, and schema version.
4. Select the quality lenses most consequential for the decision.
5. Define the relevant rule, population, source, and evidence boundary for each lens.
6. Record limitations, unresolved ambiguity, owner, and residual risk.

This is not a universal data-model review checklist. It is a way to ensure that a structural check does not silently stand in for a semantic conclusion.

Supporting asset (Pass 2, planned): One Business Fact, Many Representations will trace a promised-delivery fact from order event through storage and dashboard decision, identifying where meaning can change.

Engineering Perspective

Representation choices are engineering decisions. Developers, data engineers, product managers, analysts, and Quality Engineers should be able to discuss which facts must remain distinguishable, what a field means, which system owns it, how it evolves, and what consumer decision it supports. This discussion is often most valuable before a schema, mapping, or dashboard is implemented.

In a change review, ask whether a new default value hides missingness, whether an identifier is unique at the needed scope, whether a timestamp has a defined time zone, and whether rounding rules are compatible with downstream use. These are quality and maintainability questions, not merely data-team implementation details.

Industry Perspective

Formal data-quality models help teams avoid using one term to mean several things. ISO/IEC 25012 provides a general model for structured-data quality.¹ ISO/TS 8000-82 describes creating data rules as part of data-quality assessment.² These references can support terminology and rule design, but they do not select the business definition, threshold, or trade-off for a particular consumer.

Interoperable representations also rely on explicitly shared specifications. For example, RFC 3339 defines a profile of date and time formats for Internet protocols.³ A conforming timestamp format remains only one part of an evidence argument: its business interpretation, source, and temporal role must still be established.

Common Misconceptions

“A schema-valid record is correct.”

Schema validity establishes structural conformance to the checked schema. It does not establish source truth, business semantics, complete population coverage, or consumer fitness.

“NULL means the value is missing.”

It may mean unknown, not applicable, not yet calculated, omitted by a producer, or an implementation default. Preserve or document distinctions that a consumer needs.

“A primary key guarantees uniqueness everywhere.”

A key is unique within the model and scope for which it is defined. History, tenancy, versioning, event delivery, and external identifiers may require a different uniqueness claim.

“Dates are simpler than timestamps.”

A date avoids some precision, but it introduces calendar and time-zone questions when derived from a timestamp or used in a distributed business process.

“Quality dimensions can be added into one definitive score.”

Scores can be useful local signals. They should not conceal the consumer, population, definition, trade-offs, and evidence limitations that determine fitness for purpose.

Summary

Data representations shape what a quality claim can mean. Types, domains, nullability, identifiers, relationships, units, precision, coded values, and time conventions carry information that a consumer may need to make a sound decision. A schema or format check is valuable but bounded evidence.

Quality dimensions provide useful lenses when they are tied to a defined population, consumer, rule, and consequence. The next chapter turns these concepts into query-based evidence about counts, distinct values, nulls, duplicates, keys, and relationships.

Key Takeaways

• Business meaning, logical models, physical representations, and consumer displays are related but distinct.
• Types and schemas reduce ambiguity, but structural conformance does not prove semantic fitness.
• Missingness, identifiers, units, precision, codes, and time conventions need explicit meaning when a decision depends on them.
• Uniqueness and integrity are scoped claims about keys, relationships, populations, and lifecycle conditions.
• Correctness, completeness, consistency, validity, freshness, and related dimensions are contextual lenses, not an independent checklist or universal score.
• A valid representation can still be incomplete, stale, inconsistent, or unsuitable for the consumer decision.
• Representation-aware evidence makes assumptions and residual risk inspectable before they become a harmful conclusion.

Review Questions

1. Distinguish a conceptual business fact, a logical model, and a physical representation.
2. Why can a structurally valid record be semantically unfit for a consumer?
3. Give three distinct business meanings that a NULL or omitted field might carry.
4. Why is “unique” incomplete without an identifier, scope, and lifecycle context?
5. How can a currency value be valid in type and still be wrong for a report?
6. What information is needed before a date can safely support a time-sensitive decision?
7. Explain why completeness is not a universal property of a dataset.
8. Which quality lenses would you prioritise for a warehouse packing decision, and why?

Interview Questions

1. How would you assess a field that passes all schema checks but produces a misleading dashboard metric?
2. Describe a representation issue involving identifiers, nullability, or time that could create a customer-impacting defect.
3. What questions would you ask before defining a uniqueness rule for an event stream or history table?
4. How do you decide whether data is fresh enough for a particular consumer?
5. How would you explain the limits of a data-quality score to a stakeholder who requests one?

Practical Exercise

Data Representation and Quality-Dimension Analysis: Atlas Commerce Packing View

Objective: Analyse whether the delivery_date representation in the opening story is fit for the warehouse packing decision.

Scenario: Atlas Commerce supplies this fictional representation map:

Tasks:

1. State the business fact and decision consumer for pack_today.
2. Trace the fact across conceptual, logical, physical, and consumer representations.
3. Identify at least five representation assumptions or ambiguities that could affect the decision.
4. Select the four most consequential quality lenses and explain their decision-specific meaning.
5. Propose bounded evidence for each selected lens, including what the evidence would not prove.
6. Recommend a representation or display change that preserves uncertainty rather than hiding it, and write a short residual-risk statement.

Expected artifact: A two-page Data Representation and Quality-Dimension Analysis containing a representation map, contextual dimension profile, evidence plan, recommended clarification, and residual-risk statement.

Reflection: Which issue would be invisible if the team measured only non-null values and schema conformance?

Portfolio relevance: This artefact demonstrates that you can translate a business decision into representation-aware quality reasoning. Use fictional or safely anonymised data only.

Further Reading

• ISO/IEC 25012:2008 — Data Quality Model
• ISO/TS 8000-82:2022 — Data quality assessment: Creating data rules
• RFC 3339 — Date and Time on the Internet: Timestamps
• Part II — Programming for Quality Engineers
• Part IV — API Quality Engineering
• Chapter 3 — Query-Based Data Evidence, Integrity, and Relationships

References

Chapter Checklist

Before moving on, confirm that you can:

• Distinguish a business fact from its logical, physical, and consumer representations.
• Explain the decision relevance of type, nullability, identifier, unit, precision, code, and temporal choices.
• Apply data-quality dimensions as contextual questions rather than universal checkboxes.
• Identify when structural validity is insufficient evidence of semantic fitness.
• Produce a representation-aware analysis that states assumptions, limitations, and residual risk.