In a nutshell
Imagine you are packing up a house. A relational database is like insisting that every belonging goes into the same size of rectangular box — perfect for books and crockery that stack neatly, absurd for a bicycle, a houseplant, or a rolled-up rug. Non-relational (NoSQL) storage says the opposite: pick the container that fits the shape of the thing. A self-describing document for a product whose attributes vary from item to item; a key-value locker for “give me item #42, instantly”; a graph for “who is connected to whom”; a column-family shelf for billions of sparse, time-stamped sensor readings. You stop forcing every shape into a table and instead choose the store that matches the data’s natural shape.
That is the first half of this lesson. The second half answers a different question. Once your data is scattered across all those containers — plus relational databases, loose files and SaaS apps — how do you turn the whole messy pile into a decision? That is analytics: gather the raw data into one cheap place, clean and join it, shape it into something query-friendly, and finally draw a chart a human can act on before their coffee goes cold. If storage is about keeping the data, analytics is about making it mean something.
Both halves map directly onto two objectives of the DP-900: Azure Data Fundamentals exam — “working with non-relational data on Azure” and “an analytics workload on Azure”. You write no production code here. You learn the vocabulary and the map of which Azure service does which job, so the tangle of brand names finally sorts itself into an order you can recite.
Level: Beginner · Time: ~34 min
Before you start, it helps to have met a few ideas from the two earlier Data Fundamentals lessons — structured vs semi-structured vs unstructured data, OLTP vs OLAP, and batch vs streaming — and to know roughly what a table, a row and a JSON object are. Nothing more; every term here is defined as it appears.
After this lesson you will be able to:
- Read a one-line description of a workload and pick which of the four NoSQL shapes (document, key-value, column-family, graph) fits — and name an Azure service for it.
- Explain Azure Cosmos DB to a colleague: its five APIs, global distribution, the five consistency levels, and why everything is priced in Request Units.
- Draw the five-stage analytics pipeline — ingest → store → transform → serve → visualise — from memory and drop the right Azure service onto each stage.
- Say confidently where Microsoft Fabric and OneLake sit, and why they are the current strategic direction for new analytics work.
- Answer DP-900-style questions on ETL vs ELT, batch vs stream, data lake vs warehouse, and the four Azure Storage services.
In the previous lesson we put data into neat, predictable tables and queried it with SQL. That model is superb when your data is regular and your relationships are well known — but a great deal of the world’s data is neither. A product catalogue where every item has different attributes, a stream of clickstream events arriving a million an hour, a social graph of who-follows-whom, a folder of PDFs and videos — none of these fit comfortably into rows and columns. This is the territory of non-relational (often called NoSQL) data stores, and Azure has a rich family of them.
There is also a second, separate question this lesson answers: once an organisation has data scattered across dozens of relational databases, NoSQL stores, files and SaaS apps, how does it bring everything together to make decisions? That is the job of an analytics pipeline — ingest the data, land it cheaply in a data lake, transform it into something clean and joined-up, serve it in a query-friendly shape, and finally visualise it in a report a human can act on. These two themes — non-relational storage and the analytics pipeline — are exactly the two halves of this lesson, and together they round out the storage-and-analytics portion of the DP-900: Azure Data Fundamentals certification.
We assume you have read the core data concepts lesson (structured vs semi-structured vs unstructured, OLTP vs OLAP, batch vs streaming) and the relational data on Azure lesson. Everything here is taught from first principles, but those two give you the vocabulary the rest of this builds on.
Learning objectives
By the end of this lesson you can:
- Explain when a non-relational store beats a relational one, and name the four common NoSQL data models (key-value, document, column-family, graph).
- Describe Azure Cosmos DB at a fundamentals level — its five APIs (NoSQL, MongoDB, Cassandra, Gremlin, Table), global distribution, multi-region writes, consistency levels and request units (RUs) as a throughput/cost currency.
- Describe the four Azure Storage services — Blob, Table, File and Queue — and say what each is for.
- Lay out the modern analytics pipeline — ingest → store in a data lake → transform → serve → visualise — and name the Azure service that owns each stage.
- Tell ETL from ELT and explain why the cloud pushed everyone towards ELT.
- Pick the right analytics tool for a job — Data Factory, Data Lake Storage Gen2, Synapse Analytics, Microsoft Fabric, Azure Databricks, Power BI — using a clear comparison table.
Prerequisites & where this fits
You need only basic IT literacy and the two earlier Data Fundamentals lessons noted above; no Azure account is required to follow the concepts, though the optional lab uses a free Cosmos DB tier and a few rupees of Blob storage. This is the third Data Fundamentals lesson in the Azure Zero-to-Hero course. It deliberately stays at fundamentals depth — broad coverage, every term defined — and then hands off to the advanced Data Factory, Synapse & Fabric deep-dive and the Cosmos DB partition-key & RU optimisation deep-dive for engineers who need to build, not just understand.
Part 1 — Non-relational data
When NoSQL beats relational
A relational database earns its keep when data is structured and relationships are stable: every row has the same columns, foreign keys join tables reliably, and you need ACID transactions (the guarantee that a multi-step change either fully happens or not at all). For an order-entry system or a bank ledger, that is exactly right.
Non-relational stores trade some of those guarantees for flexibility and scale. They shine when one or more of the following is true:
| Signal in your workload | Why relational struggles | Why NoSQL fits |
|---|---|---|
| Variable / evolving schema | Adding columns and migrating is painful; sparse rows waste space. | Each item carries its own fields; no migration to add one. |
| Massive horizontal scale | A single SQL server has a ceiling; sharding is hard to bolt on. | Built to partition (shard) data across many nodes from day one. |
| Very high write throughput / low latency | Locking and joins add overhead. | Simple lookups by key are extremely fast and predictable. |
| Global users needing local latency | Geo-replication is add-on and usually single-write-region. | Some stores replicate multi-region with multi-write natively. |
| Naturally hierarchical or connected data | Deep joins to model trees/graphs get ugly. | Document and graph models store the shape directly. |
The trade-off is real: many NoSQL stores favour availability and partition tolerance over strict consistency (the famous CAP theorem — you cannot have perfect consistency, availability and partition tolerance all at once), and they generally do not do rich cross-entity JOINs or multi-table transactions the way SQL does. The senior-architect’s rule of thumb: use relational by default for transactional business data; reach for NoSQL when schema flexibility, horizontal scale, global low latency, or a graph/document shape is the dominant requirement.
The four NoSQL data models
“NoSQL” is an umbrella over four quite different shapes. Knowing them is a guaranteed DP-900 question:
| Model | Stores data as… | Good for | Azure service |
|---|---|---|---|
| Key-value | A dictionary: a unique key → an opaque value. | Caching, session state, simple lookups by id. | Azure Cosmos DB for Table, Azure Table storage |
| Document | Self-describing documents (usually JSON), each with its own fields. | Catalogues, user profiles, content, event payloads. | Azure Cosmos DB for NoSQL / for MongoDB |
| Column-family (wide-column) | Rows that can each have different, very many columns, grouped into families. | Time-series, IoT, huge sparse tables. | Azure Cosmos DB for Apache Cassandra |
| Graph | Nodes (entities) joined by edges (relationships), both with properties. | Social networks, recommendations, fraud, knowledge graphs. | Azure Cosmos DB for Apache Gremlin |
Notice that Azure Cosmos DB appears in every row. That is the headline: Cosmos DB is a single, fully managed, multi-model database service that can present any of these models through a choice of API.
Azure Cosmos DB
Azure Cosmos DB is Azure’s flagship globally distributed, multi-model NoSQL (and now also relational-via-PostgreSQL) database. As a managed PaaS service you never patch a server or manage a cluster; you pick an API, set your throughput, choose your regions, and Azure runs the rest with guaranteed single-digit-millisecond latency and SLAs covering availability, latency, throughput and consistency.
The five APIs
You choose an API when you create the account, and it is effectively permanent for that account. The API decides the data model, the wire protocol, and the query language your application speaks — which matters enormously, because it lets you lift an existing app onto Cosmos DB with little or no code change.
| API | Data model | Speaks the protocol of… | Pick it when… |
|---|---|---|---|
| API for NoSQL (formerly SQL/Core) | Document (JSON) | Cosmos DB’s native SQL-like query over JSON | It is a new project — gets every new feature first; the default and recommended choice. |
| API for MongoDB | Document (BSON) | MongoDB | You are migrating a MongoDB app or your team knows MongoDB tooling/drivers. |
| API for Apache Cassandra | Column-family | Cassandra (CQL) | You are moving a Cassandra workload and want a managed, elastic backend. |
| API for Apache Gremlin | Graph | Apache TinkerPop / Gremlin | You have graph data — relationships are the point (social, recommendations, fraud). |
| API for Table | Key-value | Azure Table storage | You want a premium, globally distributed, low-latency upgrade for an Azure Table storage app. |
The exam framing to remember: the API is chosen at account creation, lets you reuse existing drivers/skills, and you cannot mix models in one account. For greenfield work the answer is almost always API for NoSQL.
Global distribution and multi-region writes
The defining Cosmos DB capability is turnkey global distribution. With a tick-box on a world map you replicate your data to any number of Azure regions, and Cosmos transparently routes each client to the nearest one for low latency. Two modes matter:
- Single-region write, multi-region read — one region accepts writes; the rest are read replicas. Simpler and cheaper.
- Multi-region writes (multi-master) — every region accepts writes locally, giving the lowest write latency and surviving a regional outage with no failover. The cost is that write conflicts can occur, which Cosmos resolves with policies (last-writer-wins by default, or custom).
This is what people mean when they call Cosmos DB “globally distributed”: data and write capability follow your users around the planet, with a 99.999% availability SLA when configured multi-region with multi-write.
Consistency levels
Distributed systems must trade consistency (always seeing the latest write) against latency and availability. Cosmos DB is unusual in offering five well-defined consistency levels as a simple setting, from strongest to weakest:
| Level | Guarantee (plain English) | Trade-off |
|---|---|---|
| Strong | Every read sees the most recent committed write, everywhere. | Highest latency; limits multi-region writes. |
| Bounded staleness | Reads lag the latest write by at most K versions or T seconds — a bounded “behind”. | Tunable freshness vs performance. |
| Session (default) | Within a single client session you always read your own writes, in order. | Best balance for most apps — the sensible default. |
| Consistent prefix | You never see writes out of order, but may see an older snapshot. | Cheaper, lower latency. |
| Eventual | Replicas converge “eventually”; reads may be stale and unordered. | Lowest latency and cost; weakest guarantee. |
You rarely need Strong globally; Session is the workhorse default and the one to quote in an interview.
Request Units (RUs) — the throughput and cost currency
This is the single most important Cosmos DB concept for DP-900. Cosmos DB does not bill you per CPU or per query type. Instead, every operation — a read, a write, a query — costs some number of Request Units (RUs), a normalised currency that blends CPU, memory and IOPS into one number. Reading a 1 KB item costs 1 RU; writes and complex queries cost more.
You provision throughput as RUs per second (RU/s) on a container (or database), and that is what you pay for. There are three capacity modes:
| Mode | How you pay | Best for |
|---|---|---|
| Provisioned throughput | You reserve a fixed RU/s (e.g. 400 RU/s), billed whether used or not. | Steady, predictable traffic. |
| Autoscale | You set a maximum; Cosmos scales between 10% and 100% of it automatically. | Spiky or unpredictable traffic — no manual tuning. |
| Serverless | You pay per RU consumed, nothing when idle. | Dev/test, intermittent or low-traffic workloads. |
If you exceed your provisioned RU/s, requests are throttled (HTTP 429 “request rate too large”) and must back off and retry — so RU planning and good partition-key design (so load spreads evenly) are the heart of running Cosmos DB well. That depth is the subject of the dedicated Cosmos DB partition-key & RU optimisation lesson; for DP-900 you simply need to know RUs are the currency, you provision RU/s, and over-running them causes throttling.
Azure Storage
Before Cosmos DB existed, and still for an enormous range of jobs, the workhorse non-relational store on Azure is the humble storage account. A single storage account is a namespace that bundles four distinct data services, each a different shape of unstructured or semi-structured storage:
| Service | What it stores | Typical use | Access pattern |
|---|---|---|---|
| Blob storage | Binary Large OBjects — any file: images, video, backups, logs, Parquet. | The default place for unstructured data and the data lake (see Part 2). | REST/HTTPS, SDKs; URL per blob. |
| Table storage | A simple key-value / wide-column NoSQL store (partition key + row key). | Cheap, massive, schemaless lookup tables; metadata. | Key lookups; no joins. |
| File storage (Azure Files) | Fully managed SMB/NFS file shares in the cloud. | Lift-and-shift file shares; shared config; mount as a drive. | Mounted like a network drive. |
| Queue storage | A simple message queue for asynchronous work. | Decoupling app tiers; buffering work items. | Put/get messages, ~64 KB each. |
A few fundamentals to know for the exam:
- Blob access tiers let you match cost to how often data is read: Hot (frequent access, higher storage cost, low access cost), Cool (infrequently accessed, ~30+ days), Cold (rarely accessed, ~90+ days), and Archive (offline, cheapest storage but must be rehydrated over hours before reading). Moving older data down the tiers is a classic cost lever.
- Blob types: block blobs (files), append blobs (logging), page blobs (random-access, used for VM disks).
- Redundancy decides how many copies Azure keeps and where: LRS (3 copies in one datacentre), ZRS (across availability zones), GRS/GZRS (replicated to a paired region for regional-disaster protection). This is your durability/availability dial.
Azure Storage is covered exhaustively in the storage accounts deep-dive; here, the fundamentals point is simply one account, four services — Blob, Table, File, Queue — and Blob is where your data lake lives.
Part 2 — Analytics on Azure
We now switch from storing operational data to making sense of it all together. This is analytics: turning raw, scattered data into insight a human or a model can act on.
The modern analytics pipeline
Every analytics platform on Azure — whatever the branding — implements the same five-stage pipeline. Fix this mental model and the services fall into place:
| Stage | What happens | Azure services that own it |
|---|---|---|
| 1. Ingest | Pull/copy data from many sources (databases, APIs, files, streams) into the platform. | Azure Data Factory, Synapse pipelines, Microsoft Fabric Data Factory; Event Hubs / Stream Analytics for streaming. |
| 2. Store | Land it cheaply and at scale in a data lake — raw, before any cleaning. | Azure Data Lake Storage Gen2 (Blob + hierarchical namespace); Fabric OneLake. |
| 3. Transform | Clean, join, deduplicate, aggregate — turn raw into trustworthy. | Synapse Spark/SQL pools, Azure Databricks, Data Factory data flows, Fabric. |
| 4. Serve | Present the cleaned data in a query-friendly shape (a warehouse / model). | Synapse dedicated SQL pool / Fabric Warehouse / a relational warehouse. |
| 5. Visualise | Build reports and dashboards humans read and act on. | Power BI (reports, dashboards). |
Read it as a sentence: ingest the data, store it in a lake, transform it into something clean, serve it in a warehouse, and visualise it in Power BI. Almost every interview question about “the Azure data platform” is really asking you to recite and place services onto this pipeline.
The data lake (stage 2 in depth)
A data lake is a single, massively scalable store that holds data of any structure — structured tables, JSON, images, Parquet — in its raw form, cheaply, before you decide what to do with it. On Azure the lake is Azure Data Lake Storage Gen2 (ADLS Gen2), which is simply a Blob storage account with a “hierarchical namespace” turned on (giving it real directories and file-level security, which big-data engines need).
The lake is usually organised into the medallion architecture — three quality layers data flows through:
- Bronze — raw, as-ingested, untouched (your immutable landing zone).
- Silver — cleaned, de-duplicated, conformed (trustworthy and joined-up).
- Gold — business-level aggregates, ready to serve to reports and models.
Contrast the lake with a data warehouse: a lake stores raw, any-shape data cheaply and applies structure on read (“schema-on-read”); a warehouse stores cleaned, structured data with structure defined on write (“schema-on-write”) for fast SQL analytics. Modern platforms use both — lake for cheap raw storage and flexibility, warehouse to serve curated data fast — and the blend is increasingly called a lakehouse.
ETL vs ELT
The transform stage comes in two flavours, and the difference is a perennial exam favourite. Both move data from sources into a destination and clean it; they differ in when the transform happens relative to the load.
| ETL (Extract → Transform → Load) | ELT (Extract → Load → Transform) | |
|---|---|---|
| Order | Transform data before loading it into the destination. | Load raw data first, then transform inside the destination. |
| Where transform runs | A separate processing engine en route. | The powerful destination (data lake / warehouse) itself. |
| Raw data kept? | Often not — only the transformed result lands. | Yes — raw lands first (great for re-processing and audit). |
| Best when | Sensitive data must be cleansed/masked before it lands; smaller, on-prem-style volumes. | Big data and the cloud — scale the cheap lake, transform with elastic compute. |
| Classic on Azure | Data Factory mapping data flows / SSIS. | Land in ADLS Gen2, transform with Spark/Synapse/Databricks. |
Why the cloud pushed everyone to ELT: cloud storage is cheap and effectively limitless, and cloud compute is elastic, so it is now cheaper and more flexible to dump everything raw into the lake first and transform later with on-demand power — keeping the raw copy so you can always re-derive results when requirements change. ETL still wins when governance or compliance demands that data be masked or cleansed before it ever lands in the platform. The one-line answer: ETL transforms before load (clean-then-store); ELT loads then transforms (store-then-clean); the cloud favours ELT because cheap, scalable storage and elastic compute make load-first the natural pattern.
The Azure analytics services
Here are the services you must recognise at DP-900 level, each placed on the pipeline:
- Azure Data Factory (ADF) — the cloud ingest-and-orchestrate service. Visual, low-code pipelines of activities copy data from 90+ connectors and schedule/trigger the whole flow. It is the “mover and conductor”.
- Azure Data Lake Storage Gen2 — the store stage: the scalable, cheap lake (Blob + hierarchical namespace) where raw and curated data live.
- Azure Synapse Analytics — an integrated analytics platform that combines pipelines (ingest), Spark pools and SQL pools (transform), a dedicated SQL pool data warehouse (serve), and a studio to tie them together. The previous-generation unified analytics service.
- Microsoft Fabric — Microsoft’s newest, all-in-one SaaS analytics platform. It unifies Data Factory, data engineering (Spark), a warehouse, real-time analytics and Power BI over a single shared lake called OneLake, billed as one capacity. It is the strategic direction for new analytics work.
- Azure Databricks — a first-party Apache Spark platform optimised for large-scale data engineering, data science and machine learning (the lakehouse pioneers). Reach for it for heavy Spark/ML workloads and notebook-driven teams.
- Power BI — the visualise stage: build interactive reports and pin visuals to dashboards that business users explore. The “last mile” that turns curated data into decisions, and itself part of Fabric.
Which tool for which job?
This table is the payoff — the one to internalise for both the exam and real architecture conversations:
| You need to… | Use | Why |
|---|---|---|
| Copy / move / orchestrate data from many sources on a schedule | Azure Data Factory | Purpose-built, low-code ingest and pipeline orchestration with 90+ connectors. |
| Store raw data of any shape, cheaply, at scale | Azure Data Lake Storage Gen2 | The lake — cheap, limitless, big-data-engine friendly. |
| Run a unified analytics platform on the previous generation | Azure Synapse Analytics | Pipelines + Spark + SQL warehouse in one studio. |
| Start a new analytics project with everything unified as SaaS | Microsoft Fabric | All-in-one over OneLake; Microsoft’s strategic direction. |
| Do heavy Spark data engineering / data science / ML | Azure Databricks | Best-in-class managed Spark + collaborative notebooks. |
| Build reports and dashboards for business users | Power BI | The visualisation and self-service BI layer. |
| Process real-time streams | Azure Stream Analytics / Event Hubs / Fabric Real-Time | Continuous query over data in motion (the streaming counterpart). |
The architect’s summary: Data Factory ingests, Data Lake Gen2 stores, Synapse/Fabric/Databricks transform and serve, and Power BI visualises — with Fabric the unified, SaaS, go-forward choice for greenfield. The full depth lives in the Data Factory, Synapse & Fabric deep-dive.
The diagram above stitches both halves together: the non-relational stores on the left (Cosmos DB’s five APIs and the four Azure Storage services) feeding the five-stage analytics pipeline on the right — ingest, lake, transform, serve, and finally Power BI.
Going deeper
Everything above is enough to pass the exam. This section is for the reader who wants the why underneath the names — the internals, the trade-offs, and the current direction of travel. It stays at fundamentals altitude, but each topic goes one level below the summary tables.
The four NoSQL families, one level down
The four models are not four brands of the same thing; they optimise for genuinely different access patterns. The way to hold them in your head is to ask “what is the unit I fetch, and what do I fetch it by?”
- Key-value — the unit is an opaque value, fetched by an exact key. There is deliberately no query over the contents of the value; the store does not look inside it. That constraint is the point: a pure hash lookup is O(1), predictable, and trivially shardable by hashing the key. Session state, feature flags, shopping-cart contents and cache entries live here. On Azure: Azure Table storage and Azure Cosmos DB for Table.
- Document — the unit is a document (JSON/BSON), fetched by id or by querying fields inside it. Unlike key-value, the engine indexes the document’s contents, so you can ask “all products where
category = 'books'andprice < 500”. Each document carries its own shape, so a catalogue where every product type has different attributes needs no schema migration. On Azure: Cosmos DB for NoSQL (native) and Cosmos DB for MongoDB. - Column-family (wide-column) — the unit is a row addressed by a key, but each row can hold thousands of sparse, dynamically named columns grouped into families, and the physical storage is column-oriented. This is built for enormous, sparse, write-heavy tables — IoT telemetry, time-series, event logs — where a row may have three columns or three hundred. On Azure: Cosmos DB for Apache Cassandra.
- Graph — the units are nodes and the edges between them, both carrying properties. The engine is optimised to traverse relationships (“friends of friends who bought X”), a query that would be an expensive multi-join in SQL and gets exponentially worse with depth. Social graphs, recommendation engines, fraud rings and knowledge graphs. On Azure: Cosmos DB for Apache Gremlin.
A quiet but important truth: these are overlapping, not exclusive. A document store can serve key-value lookups (just fetch by id and ignore the query engine). The skill the exam tests is choosing the model whose dominant access pattern matches your dominant workload — not finding the one model that is technically capable.
Cosmos DB: what “multi-model” actually means
The five APIs are not five databases. Under the hood there is one storage and indexing engine (the “atom-record-sequence” model) that every API projects a different face onto. That is why a single service can speak document, key-value, column-family and graph: the wire protocol and query language differ, but the engine, the global-distribution machinery, the SLAs and the RU billing are shared.
| API | Model | Wire protocol you reuse | The honest “pick it when” |
|---|---|---|---|
| NoSQL (native, formerly SQL/Core) | Document | Cosmos-native SQL-over-JSON | Greenfield — first to get every new feature (change feed, analytical store, vector search). |
| MongoDB | Document | MongoDB drivers/tools | Migrating a Mongo app, or the team already knows Mongo. (Also a vCore flavour for lift-and-shift of larger Mongo estates.) |
| Cassandra | Column-family | CQL | Moving a Cassandra workload onto a managed, elastic backend. |
| Gremlin | Graph | Apache TinkerPop/Gremlin | Relationships are the query. |
| Table | Key-value | Azure Table storage SDK | A premium, global, single-digit-ms upgrade for an existing Table-storage app. |
The DP-900 line to remember: the API is chosen at account creation and is effectively permanent; it lets you reuse existing drivers and skills; and for new work the answer is almost always API for NoSQL. (There is also a separate Cosmos DB for PostgreSQL — distributed relational, powered by the Citus extension — which is why you sometimes hear Cosmos described as “not only NoSQL”. It is out of scope for the four NoSQL models but worth recognising by name.)
The RU arithmetic that DP-900 hints at. A Request Unit normalises CPU, memory and IOPS into one number so that any operation has a single price. The anchor everyone memorises: a point read of a 1 KB item = 1 RU. From there, rough intuition:
- A 1 KB write costs about 5 RU — writes touch the index, so they cost more than reads.
- A query that filters on an indexed field and returns a few items costs a handful of RUs; a query that scans many items (no useful index, or a cross-partition fan-out) can cost hundreds.
- You provision RU/s on a container. A container at 400 RU/s sustains roughly 400 one-KB reads per second. Ask for more than you provisioned and Cosmos returns HTTP 429 – “request rate too large”; the SDK’s job is to back off and retry.
The three capacity modes map cleanly onto traffic shapes: provisioned (fixed RU/s, pay whether idle or not — steady traffic), autoscale (set a maximum; Cosmos floats between 10% and 100% of it — spiky traffic, no babysitting), and serverless (pay per RU actually consumed, nothing at idle — dev/test and bursty low-volume apps).
Why partition-key design is the real skill. RU/s is provisioned across the container, but the data is split into logical partitions by your partition key, and those are packed onto physical partitions (each capped at roughly 10,000 RU/s and 50 GB). If one partition-key value is far hotter than the rest — say you partition orders by country and 80% are IN — that one logical partition’s traffic lands on a single physical partition and gets throttled even though the container’s total RU/s looks generous. A good key is high-cardinality and spreads both storage and request load evenly. DP-900 only needs the idea (“pick an evenly distributed partition key; a hot partition causes 429s”); the container-and-physical-partition mechanics are the subject of the dedicated partition-key deep-dive.
Consistency, framed as a dial. The five levels — Strong, Bounded staleness, Session, Consistent prefix, Eventual — are a single knob trading freshness against latency, availability and cost. Strong guarantees every reader everywhere sees the latest committed write, but it forces synchronous replication and is incompatible with multi-region writes. Session (the default) guarantees you read your own writes within a client session — the sweet spot for the vast majority of apps. Eventual is cheapest and fastest and is fine for like-counts and view-counters where a brief lag is invisible. The exam wants you to know the ordering and that Session is the sensible default.
Table storage vs Cosmos DB for Table — when cheap wins
These two speak the same API but sit at opposite ends of the price/performance range, and DP-900 likes to probe the difference:
| Azure Table storage | Cosmos DB for Table | |
|---|---|---|
| Positioning | Bargain-basement key-value in a storage account | Premium, globally distributed key-value |
| Latency | Best-effort, tens of ms | Guaranteed single-digit ms, SLA-backed |
| Distribution | Single region (+ optional read-only geo replica) | Turnkey multi-region, optional multi-write |
| Throughput | Account-level, best-effort | Provisioned/autoscale/serverless RU/s |
| Cost | Very low (per GB + per transaction) | Higher (you pay for the guarantees) |
The rule: start cheap with Table storage; upgrade to Cosmos DB for Table only when you actually need the global reach, the latency guarantee, or the elastic throughput. Many “metadata table” workloads never need to leave Table storage.
Blob, ADLS Gen2 and the lakehouse, one level down
Blob storage underpins nearly everything else — it is where the data lake, backups, images, logs and Parquet files live. Three details earn their exam keep:
- Block vs append vs page blobs. Block blobs are the default (files uploaded in blocks); append blobs are optimised for append-only logging; page blobs back VM disks with random-access reads/writes. Most data-platform work is block blobs.
- Access tiers — Hot (frequent reads, cheap access, dearer storage), Cool (~30+ days idle), Cold (~90+ days idle), Archive (offline, cheapest storage, but must be rehydrated over hours before a single byte can be read). Lifecycle-management rules move ageing blobs down the tiers automatically — a classic, unglamorous cost win.
- Redundancy — LRS (3 copies, one datacentre), ZRS (across availability zones in a region), GRS/GZRS (asynchronously copied to a paired region for regional-disaster protection), with RA-GRS adding read access to that secondary. This is your durability/availability dial and a favourite exam contrast.
Azure Data Lake Storage Gen2 (ADLS Gen2) is not a separate product — it is a Blob storage account with the hierarchical namespace (HNS) switched on. HNS gives real directories and POSIX-style ACLs, which big-data engines rely on for efficient directory operations and file-level security. Enable HNS when you create the account: it has historically been a create-time-only choice, and although Microsoft now offers a one-directional upgrade of an existing flat account, it is irreversible and best avoided by designing for the lake up front.
The lake is usually laid out as the medallion architecture — bronze (raw, immutable landing), silver (cleaned, conformed), gold (business-ready aggregates). Pair the cheap, schema-on-read lake with a fast, schema-on-write warehouse to serve curated data, and you have a lakehouse — the pattern every modern Azure analytics platform now assumes.
The analytics stack, service by service
Fix the five-stage pipeline — ingest → store → transform → serve → visualise — and slot each service onto it:
- Azure Data Factory (ADF) — the ingest-and-orchestrate service. Visual, low-code pipelines of activities copy from 90+ connectors and schedule the whole flow with triggers (schedule, tumbling-window, event). An integration runtime is the compute that actually moves the bytes: Azure IR (cloud-to-cloud), self-hosted IR (reaches into on-premises or a private network), and SSIS IR (lifts-and-shifts legacy SSIS packages). Mapping data flows add Spark-backed, code-free transformation for teams that prefer not to write Spark.
- Azure Synapse Analytics — the previous-generation unified platform, and the one whose three compute engines you must be able to tell apart:
- Dedicated SQL pool (formerly SQL Data Warehouse) — a provisioned, massively-parallel (MPP) relational warehouse. It distributes each table across 60 distributions, is billed in DWUs, and can be paused to stop compute charges. Schema-on-write; the serve stage for large, structured warehouse workloads.
- Serverless SQL pool — always-on, nothing to provision, billed per terabyte of data your queries scan. You point T-SQL (
OPENROWSET/external tables) straight at Parquet/CSV/JSON files in the lake, in place — schema-on-read ad-hoc exploration with no cluster to manage. - Apache Spark pool — managed Spark clusters for notebook-driven big-data transform and ML (PySpark, Scala, Spark SQL, .NET).
- Plus Synapse Pipelines (ADF embedded) and Synapse Link (near-real-time analytics over Cosmos DB or SQL with no ETL — the HTAP idea).
- Azure Databricks — a first-party, optimised Apache Spark platform and the lakehouse pioneer: Delta Lake brings ACID transactions and time-travel to files on the lake, Unity Catalog governs data across workspaces, MLflow manages the ML lifecycle, and the Photon engine speeds SQL. Reach for it for heavy custom Spark, data science and notebook-centric teams.
- Microsoft Fabric — the current strategic direction: an all-in-one SaaS platform (GA since late 2023) that folds ingestion, data engineering, warehousing, real-time analytics, data science and Power BI into one product billed as a single capacity (F-SKU capacity units). Its foundation is OneLake — one tenant-wide logical data lake, “OneDrive for data”, built on ADLS Gen2 and storing everything in the open Delta Parquet format so every engine reads the same copy. Shortcuts virtualise external data (other ADLS accounts, Amazon S3, Google Cloud Storage) into OneLake without copying it. Its workloads are branded as experiences: Data Factory (pipelines + Dataflows Gen2), Data Engineering (Lakehouse + Spark), Data Warehouse (T-SQL), Real-Time Intelligence (Eventstream + KQL Eventhouse, formerly Real-Time Analytics), Data Science, Databases (a Fabric SQL database), and Power BI — with DirectLake letting Power BI read OneLake’s Delta tables directly, at import speed but without an import copy.
- Power BI — the visualise last mile, and itself part of Fabric. Author in Power BI Desktop, publish to the Power BI Service; a semantic model (dataset) feeds interactive, multi-page reports and single-page dashboards (pinned tiles, Service-only). Connectivity is Import (fast, cached, needs scheduled refresh), DirectQuery (live queries to the source, always current, slower), or DirectLake (the Fabric-native best-of-both over OneLake).
The go-forward summary: Data Factory ingests, ADLS Gen2 / OneLake stores, Synapse or Databricks or Fabric transform and serve, Power BI visualises — and for greenfield, Fabric over OneLake is Microsoft’s unified, SaaS answer, with Synapse retained for existing estates and Databricks for heavy custom Spark.
Batch vs stream — the latency dial for the whole pipeline
The same pipeline runs in two tempos, and choosing between them is a recurring exam and architecture question:
| Batch processing | Stream processing | |
|---|---|---|
| Data shape | Bounded — a finite chunk (yesterday’s sales) | Unbounded — an endless flow of events |
| When it runs | Scheduled (hourly, nightly) | Continuously, per event or per small window |
| Latency | Minutes to hours | Sub-second to seconds |
| Throughput/cost | Highest throughput, lowest cost per record | Lower per-record efficiency, higher complexity |
| Azure services | ADF + Spark/Synapse → warehouse | Event Hubs (ingest) → Azure Stream Analytics / Fabric Real-Time Intelligence → Power BI |
| Fits | Reports, model training, periodic aggregation | Fraud detection, live dashboards, IoT alerts |
Stream ingestion on Azure usually lands on Event Hubs (high-throughput event ingestion) or IoT Hub (device telemetry, bidirectional). Azure Stream Analytics then runs a SQL-like query continuously over the flow, using windowing functions — tumbling (fixed, non-overlapping), hopping (fixed, overlapping), sliding (event-triggered), session (activity-bounded) — to aggregate “the last N seconds” and push results to Power BI, storage, SQL or Cosmos DB. Fabric’s Real-Time Intelligence (Eventstream + KQL) is the newer, SaaS counterpart. Do not confuse Event Hubs (a stream of many data points) with Event Grid (routing of discrete events, e.g. “a blob was created”) — DP-900 likes that distinction.
The honest trade-off: stream when the decision must be made now; batch when “within the hour” is fine — batch is cheaper, simpler and easier to reason about. Many real platforms run both (a batch layer for accuracy plus a speed layer for freshness — the Lambda pattern), but for the exam the distinction and the example services are what matter.
How this maps onto the DP-900 exam
The two domains this lesson covers — non-relational data and analytics workloads — together account for a large slice of the exam, and the questions follow a small number of recognisable shapes:
- Match the service to the job — “Which service copies data from many sources on a schedule?” (Data Factory) / “…queries Parquet in the lake with T-SQL, paying per TB scanned?” (Synapse serverless SQL pool) / “…builds interactive dashboards?” (Power BI).
- Pick the Cosmos API / model from a workload description (graph feature → Gremlin; migrate Mongo → API for MongoDB; greenfield → API for NoSQL).
- RU / throttling — what an RU measures, what a 429 means, and which capacity mode fits which traffic.
- ETL vs ELT, batch vs stream, lake vs warehouse — definition-and-contrast questions.
- Storage services — Blob vs Table vs File vs Queue, access tiers, redundancy.
If you can recite the five-stage pipeline, place every service on it, and explain RUs and the five Cosmos models, you have the non-relational-and-analytics half of DP-900 covered.
Hands-on lab
A tiny, free hands-on to make both halves concrete: create a free-tier Cosmos DB account and a Blob container (your “data lake” landing zone), and confirm both work. We use the Azure CLI so the steps are copy-pasteable; everything here stays inside free limits or costs a few rupees.
1. Create a resource group
az group create \
--name rg-dp900-nosql \
--location centralindia
2. Create a free-tier Cosmos DB account (API for NoSQL)
The --enable-free-tier true flag gives you the first 1000 RU/s and 25 GB free on one account per subscription — perfect for learning.
az cosmosdb create \
--name kvcosmos$RANDOM \
--resource-group rg-dp900-nosql \
--locations regionName=centralindia \
--enable-free-tier true \
--default-consistency-level Session
Note the account name printed in the output. Expected output: a JSON object with "provisioningState": "Succeeded" and "enableFreeTier": true.
3. Create a database and a container with autoscale throughput
# Replace <account> with the name from step 2
az cosmosdb sql database create \
--account-name <account> \
--resource-group rg-dp900-nosql \
--name RetailDB
az cosmosdb sql container create \
--account-name <account> \
--resource-group rg-dp900-nosql \
--database-name RetailDB \
--name Products \
--partition-key-path "/category" \
--max-throughput 1000
Here --partition-key-path "/category" chooses category as the partition key (how data is spread for scale), and --max-throughput 1000 sets autoscale up to 1000 RU/s — inside the free allowance.
4. Create a Blob storage “data lake” landing container
az storage account create \
--name kvlake$RANDOM \
--resource-group rg-dp900-nosql \
--location centralindia \
--sku Standard_LRS \
--kind StorageV2
# Use the storage-account name printed above
az storage container create \
--account-name <storage-account> \
--name bronze \
--auth-mode login
Validation
# Confirm the Cosmos container and its throughput
az cosmosdb sql container show \
--account-name <account> --resource-group rg-dp900-nosql \
--database-name RetailDB --name Products \
--query "{name:name, pk:resource.partitionKey.paths}" -o table
# Confirm the Blob container exists
az storage container show \
--account-name <storage-account> --name bronze \
--auth-mode login --query name -o tsv
You should see the Products container with partition key /category, and bronze returned for the Blob container. You have just built, in miniature, a non-relational store and the landing layer of a data lake.
Cleanup — delete the whole resource group so nothing keeps billing:
az group delete --name rg-dp900-nosql --yes --no-wait
Cost note (INR): the Cosmos DB account is on the free tier (₹0 for the first 1000 RU/s and 25 GB). The Standard LRS storage account costs roughly ₹1.6–₹2 per GB per month for Hot blobs, and you stored nothing, so the lab total is effectively ₹0 if you clean up the same day. Always run the cleanup step — an idle provisioned-throughput Cosmos container (outside free tier) is the classic surprise on a learner’s bill.
Common mistakes & troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| HTTP 429 “request rate too large” from Cosmos DB | You exceeded provisioned RU/s (often a hot partition). | Raise RU/s or switch to autoscale; choose a better-distributed partition key; implement retry-with-backoff. |
| Want to change a Cosmos account from MongoDB to NoSQL API | The API is fixed at account creation. | Create a new account with the desired API and migrate the data. |
| Blob “Archive” data won’t read | Archive tier is offline; blobs must be rehydrated first. | Rehydrate to Hot/Cool (hours) before reading, or keep frequently read data in Hot/Cool. |
| Big-data engine can’t do directory operations on your lake | You used a plain Blob account, not ADLS Gen2 (no hierarchical namespace). | Recreate with hierarchical namespace enabled (it cannot be toggled on later). |
| Data warehouse load is slow and expensive | You forced ETL on huge volumes through a small engine. | Switch to ELT: land raw in the lake, transform with elastic Spark/SQL compute. |
| Power BI report is stale | Dataset refresh not scheduled, or source not refreshed. | Configure scheduled refresh; for big models consider DirectQuery / DirectLake. |
| Surprise Cosmos bill on an idle dev database | Provisioned RU/s bills whether used or not. | Use serverless or autoscale for dev/test; delete idle resources. |
| Choosing Synapse for a brand-new project | Defaulting to the previous generation. | For greenfield, evaluate Microsoft Fabric (the strategic direction) first. |
Common beginner mistakes
These are misconceptions, not error messages — the wrong mental model that quietly leads you to the wrong answer. (For symptom → cause → fix, see the troubleshooting table above.)
- “NoSQL means SQL is obsolete.” No. Relational databases remain the correct default for transactional business data that needs ACID guarantees and rich joins. NoSQL is not a replacement; it is the right tool when a specific pressure dominates — variable schema, horizontal scale, global low-latency, or a document/graph shape. The mature architect reaches for NoSQL deliberately, not reflexively.
- “Cosmos DB is a JSON/document database.” It is multi-model. One engine presents document, key-value, column-family and graph faces through the five APIs. Calling it “a document database” is like calling a Swiss Army knife “a blade” — true of one mode, wrong about the tool.
- “A data lake replaces the data warehouse.” They solve different problems. A lake stores raw, any-shape data cheaply with schema-on-read; a warehouse stores cleaned, structured data with schema-on-write for fast SQL. Modern platforms use both — that combination is the lakehouse. Picking one and discarding the other is the classic beginner over-simplification.
- “Microsoft Fabric is just a rebrand of Synapse (or of Power BI).” Fabric is a new SaaS platform that unifies Data Factory, data engineering, warehousing, real-time analytics, data science and Power BI over a single lake (OneLake), billed as one capacity. Synapse still exists as a distinct PaaS service; Fabric is the convergence and go-forward direction, not a new label on an old product.
- “More RUs always means faster — just crank up provisioned throughput.” Throughput is spread across partitions by your partition key. A hot partition (one key value taking most of the traffic) throttles regardless of how high the container’s total RU/s is — you cannot exceed a single physical partition’s ceiling. Fix the key, not just the number. And idle provisioned RU/s quietly bills around the clock.
- “Choose Strong consistency to be safe.” Strong is the most expensive setting: highest latency and incompatible with multi-region writes. Session is the sensible default and is what almost every app should use; strengthen only where a concrete requirement demands it. Defaulting to Strong “to be safe” trades away performance you probably needed.
- “Real-time is always better than batch.” Streaming is more complex and more expensive per record. If the decision can wait “until the hourly run”, batch is the right, cheaper choice. Match the tempo to the latency the business actually needs, not to what sounds more impressive.
- “ADLS Gen2 is a different product from Blob storage.” It is a Blob storage account with the hierarchical namespace switched on — same service, one extra capability (real directories and file-level ACLs) that big-data engines need. Enable it at account creation; retrofitting it is an irreversible migration you want to avoid.
Best practices
- Pick the right store for the shape of the data. Document → Cosmos DB for NoSQL; graph → Gremlin; cheap files/lake → Blob/ADLS Gen2; simple key-value at scale → Table storage. Don’t force everything into one model.
- Design the partition key first. In Cosmos DB (and Table storage) an even-spreading partition key is the single biggest factor in performance and cost; a “hot” partition wastes RUs.
- Use autoscale or serverless for variable/dev workloads and reserve fixed provisioned throughput only for steady, predictable traffic.
- Default to Session consistency in Cosmos DB; only strengthen it where a specific requirement demands.
- Land raw, then transform (ELT) and keep the bronze layer immutable so you can always re-derive curated data when business rules change.
- Tier your blobs (Hot/Cool/Cold/Archive) with lifecycle-management rules so cold data costs less automatically.
- For new analytics, evaluate Microsoft Fabric first; keep Synapse/Databricks where existing estates or heavy custom Spark justify them.
Security notes
- Prefer Microsoft Entra ID (identity) over keys. Both Cosmos DB and Storage support Entra ID with RBAC; use it (and managed identities for apps) instead of account keys or connection strings wherever possible, and rotate any keys you must use.
- Encryption is on by default. Data is encrypted at rest automatically; you can supply customer-managed keys (CMK) in Key Vault for extra control, and everything is encrypted in transit over HTTPS/TLS.
- Restrict the network. Lock storage accounts and Cosmos DB to private endpoints or selected VNets/firewall rules rather than leaving them open to the public internet; disable public blob access unless a container must be public.
- Govern the lake. As data from many sources lands in one place, classify and govern it (e.g. with Microsoft Purview) and apply least-privilege access at the container/folder level (ADLS Gen2 ACLs).
- Mind data residency. Global distribution replicates data to the regions you choose — pick regions deliberately to honour sovereignty and compliance requirements (for India, keep data in
centralindia/southindiawhere required).
Interview & exam questions
- When would you choose a non-relational store over a relational one? When the schema is variable/evolving, you need massive horizontal scale or very low-latency key lookups, you need global multi-write low latency, or the data is naturally a document/graph — and you don’t need rich cross-table JOINs or multi-table ACID transactions.
- Name the four NoSQL data models and an Azure service for each. Key-value (Table storage / Cosmos Table), document (Cosmos DB for NoSQL/MongoDB), column-family (Cosmos DB for Cassandra), graph (Cosmos DB for Gremlin).
- What are the five Cosmos DB APIs and when is each chosen? NoSQL (new projects, default), MongoDB (migrate Mongo apps), Cassandra (migrate Cassandra), Gremlin (graph data), Table (upgrade Azure Table apps). The API is fixed at account creation.
- What is a Request Unit (RU)? A normalised currency blending CPU, memory and IOPS; every operation costs RUs (a 1 KB read = 1 RU). You provision RU/s; exceeding it causes 429 throttling.
- Explain Cosmos DB consistency levels. Strong, Bounded staleness, Session (default), Consistent prefix, Eventual — a spectrum trading freshness for latency/availability. Session is the usual default.
- What is global distribution / multi-region writes? Replicating data to multiple regions for local latency; multi-region writes let every region accept writes (multi-master), surviving regional outages at the cost of conflict resolution.
- What are the four Azure Storage services? Blob (objects/files), Table (key-value NoSQL), File (SMB/NFS shares), Queue (messages).
- What are Blob access tiers and why do they matter? Hot, Cool, Cold, Archive — they match storage/access cost to access frequency; Archive is offline and must be rehydrated. They are a major cost lever.
- Describe the five stages of an analytics pipeline and name a service per stage. Ingest (Data Factory), store (Data Lake Gen2), transform (Synapse/Databricks/Fabric), serve (warehouse), visualise (Power BI).
- ETL vs ELT — what’s the difference and why did the cloud favour ELT? ETL transforms before loading; ELT loads raw then transforms in the destination. The cloud favours ELT because cheap, limitless storage plus elastic compute make “load-first” cheaper and more flexible, while keeping the raw copy.
- What is a data lake, and how does it differ from a data warehouse? A lake stores raw, any-shape data cheaply with schema-on-read; a warehouse stores cleaned, structured data with schema-on-write for fast SQL. ADLS Gen2 is the Azure lake; combining both is a lakehouse.
- Synapse, Fabric, or Databricks — which for a new project? For greenfield, evaluate Microsoft Fabric first (the unified SaaS, go-forward direction over OneLake); use Databricks for heavy custom Spark/ML; keep Synapse for existing estates.
Quick check
- Which Cosmos DB API would you choose for a brand-new application, and why?
- True or false: you can change a Cosmos DB account’s API after it is created.
- What does a Request Unit measure, and what happens if you exceed your provisioned RU/s?
- Put these in pipeline order: serve, ingest, visualise, transform, store.
- In ELT, does the transform happen before or after the data is loaded into the destination?
Answers
- API for NoSQL — it is the native, default API and receives every new feature first.
- False — the API is fixed at account creation; migrate to a new account to change it.
- An RU is a normalised throughput currency (CPU + memory + IOPS); exceeding provisioned RU/s causes HTTP 429 throttling, requiring back-off and retry.
- Ingest → store → transform → serve → visualise.
- After — ELT loads raw data first, then transforms it inside the destination (lake/warehouse).
Exercise
Imagine a retail company with: (a) a fast-growing product catalogue where items have wildly different attributes; (b) a clickstream of millions of website events per hour; © a folder of product images and PDFs; and (d) a need for executives to see daily sales dashboards.
For each of (a)–(d), write down which Azure service you would use and one sentence of justification. Then sketch — in five boxes — the end-to-end analytics pipeline that would carry the clickstream from the website all the way to an executive dashboard, labelling each stage. (Suggested answer: (a) Cosmos DB for NoSQL — flexible document schema; (b) ingest via Event Hubs/Data Factory into ADLS Gen2 — scale and cheap raw storage; © Blob storage — unstructured files; (d) Power BI — interactive dashboards. Pipeline: ingest → data lake (bronze) → transform (Spark/Synapse to silver/gold) → serve (warehouse) → visualise (Power BI).)
Practice challenges
Six DP-900-style multiple-choice questions, escalating from beginner to advanced. Read the scenario, commit to an answer before expanding the solution, and pay attention to the why — the exam rewards knowing why the distractors are wrong as much as why the answer is right.
1. (Beginner) A social app needs a “people you may know” feature that walks connections several hops deep — friends of friends of friends. Which Azure Cosmos DB API best fits?
- A. API for NoSQL
- B. API for Table
- C. API for Apache Gremlin
- D. API for MongoDB
<details><summary>Show answer</summary>
C — API for Apache Gremlin. “People you may know” is a multi-hop graph traversal, exactly what a graph model is optimised for. The same query in a document or key-value model would be an expensive, deepening chain of joins/lookups. Why not the others: NoSQL and MongoDB are document stores (great for the user profiles, not for traversing relationships); Table is key-value with no relationship traversal at all. </details>
2. (Beginner–intermediate) A team is moving an existing MongoDB application to a fully managed Azure database and wants to keep their current drivers and tooling with minimal code change. Which option is the best fit?
- A. Azure SQL Database
- B. Azure Cosmos DB for MongoDB
- C. Azure Table storage
- D. Azure Cosmos DB for NoSQL
<details><summary>Show answer</summary>
B — Azure Cosmos DB for MongoDB. Because it speaks the MongoDB wire protocol, existing drivers, tools and most application code work with little or no change — that protocol compatibility is the whole reason the non-NoSQL Cosmos APIs exist. Why not the others: NoSQL is also a document store but uses Cosmos’s own SQL-over-JSON, so it would mean rewriting data access; Azure SQL is relational; Table storage is key-value. </details>
3. (Intermediate) A Cosmos DB container is provisioned at a fixed 400 RU/s. It runs fine most of the day but returns HTTP 429 during short afternoon spikes. Leadership wants to stop the errors without paying peak rates 24×7. What is the best change?
- A. Switch the container to autoscale with a suitable maximum RU/s
- B. Raise provisioned throughput permanently to the peak value
- C. Move the account to Strong consistency
- D. Add more regions to the account
<details><summary>Show answer</summary>
A — switch to autoscale. Autoscale floats between 10% and 100% of a maximum you set, absorbing the spike automatically and dropping back down (and back to a lower bill) when it passes. Why not the others: raising fixed provisioned throughput removes the 429s but pays peak rates all day — exactly what was ruled out; consistency levels affect freshness/latency, not throughput; extra regions add reach and cost but do nothing for a per-partition throughput ceiling. (If the spikes were rare and the baseline near-idle, serverless would be the alternative to weigh.) </details>
4. (Intermediate) An analyst wants to run ad-hoc T-SQL queries directly over Parquet files sitting in ADLS Gen2, paying only for the data each query scans, with no cluster to provision or manage. Which Azure Synapse capability fits?
- A. Dedicated SQL pool
- B. Apache Spark pool
- C. Serverless SQL pool
- D. Synapse Pipelines
<details><summary>Show answer</summary>
C — Serverless SQL pool. It is always available, needs nothing provisioned, is billed per terabyte scanned, and queries files in the lake in place with T-SQL — the definition of schema-on-read ad-hoc exploration. Why not the others: a dedicated SQL pool is a provisioned MPP warehouse (you pay for reserved DWUs, and you load data in first); a Spark pool means notebooks and clusters, not plain T-SQL; Pipelines move and orchestrate data, they do not query it. </details>
5. (Intermediate–advanced) A company is starting a brand-new analytics platform. Leadership wants a single SaaS product with one bill, a single lake in an open format that every engine shares, and Power BI built in — not a set of separately managed services stitched together. Which Azure choice matches?
- A. Azure Synapse Analytics
- B. Azure Databricks
- C. Microsoft Fabric
- D. Azure Data Factory + Power BI, wired together manually
<details><summary>Show answer</summary>
C — Microsoft Fabric. Fabric is the all-in-one SaaS platform over OneLake (one logical lake, open Delta Parquet, shared by every workload) with Power BI as a native experience and a single capacity-based bill — precisely the requirements, and Microsoft’s stated strategic direction for new work. Why not the others: Synapse is the capable previous generation but is PaaS assembled from pieces; Databricks is superb for heavy Spark/ML but is a Spark platform, not an all-in-one with built-in Power BI; option D is the manual stitching the scenario explicitly rejects. </details>
6. (Advanced) A payments team needs to score transactions for fraud in under a second as they occur and light up a live Power BI tile. Which ingest-and-process combination fits the requirement?
- A. Azure Data Factory copies transactions nightly into a dedicated SQL pool
- B. Azure Event Hubs ingests the stream and Azure Stream Analytics scores it in real time, outputting to Power BI
- C. Blob storage lifecycle rules tier the data to Cool
- D. Azure Table storage with a scheduled batch export
<details><summary>Show answer</summary>
B — Event Hubs → Stream Analytics → Power BI. Sub-second scoring of an unbounded flow is a stream-processing job: Event Hubs is the high-throughput ingestion front door, Stream Analytics runs a continuous SQL-like query (with windowing) over the flow, and it can push straight to a real-time Power BI tile. (In a modern Fabric estate, Real-Time Intelligence is the equivalent.) Why not the others: A and D are batch — nightly or scheduled export cannot meet a sub-second requirement; C is a storage cost-management feature, not stream processing at all. </details>
Certification mapping
This lesson maps to the DP-900: Microsoft Azure Data Fundamentals certification, principally the exam areas “Describe considerations for working with non-relational data on Azure” (Cosmos DB, its APIs, and Azure Storage — Blob, Table, File, Queue) and “Describe an analytics workload on Azure” (the ingest → store → transform → serve → visualise pipeline, ETL vs ELT, data lakes, Data Factory, Synapse, Fabric, Databricks and Power BI). It is also a useful on-ramp to DP-203 / DP-700 (data engineering) and PL-300 (Power BI data analyst).
Is DP-900 still current? (2026 check)
This lesson’s exam framing was verified against the official Microsoft Learn certification page (page last updated July 2026):
- The exam is live and current. Microsoft Certified: Azure Data Fundamentals, earned by passing Exam DP-900, is an active Beginner-level Fundamentals certification with no prerequisites.
- It does not expire. Unlike role-based certifications (which need periodic renewal), Fundamentals credentials such as DP-900 are effectively lifetime.
- The four skill areas are unchanged and match this lesson’s coverage: core data concepts, relational data on Azure, non-relational data on Azure, and an analytics workload on Azure. The last two — the whole subject of this lesson — together make up a large share of the exam.
- Format: a proctored exam (Pearson VUE test centre or online), on the order of 40–60 questions in roughly 45 minutes of assessment time (allow about an hour of total seat time), scored 1–1000 with 700 to pass. Price varies by country/region.
- Fabric is the moving part to watch. Microsoft refreshes the DP-900 study guide periodically, and the analytics content increasingly foregrounds Microsoft Fabric and OneLake as the strategic direction (Synapse remains examinable for existing estates). On exam day, treat the official DP-900 study guide — not any third-party dump site — as the single source of truth for the exact skills list and current weightings.
Treat this as a freshness stamp, not a correction: nothing in the certification’s current structure invalidates the material in this lesson.
Glossary
-
NoSQL / non-relational — data stores that do not use the relational table model; optimised for flexible schema, scale or specific shapes (document, key-value, column-family, graph).
-
Cosmos DB — Azure’s globally distributed, multi-model, fully managed NoSQL database service.
-
API (Cosmos DB) — the protocol/data-model a Cosmos account speaks (NoSQL, MongoDB, Cassandra, Gremlin, Table), fixed at creation.
-
Request Unit (RU) — the normalised currency for Cosmos DB throughput; you provision RU/s.
-
Consistency level — the freshness-vs-latency setting in Cosmos DB (Strong, Bounded staleness, Session, Consistent prefix, Eventual).
-
Global distribution / multi-region writes — replicating Cosmos data to multiple regions, optionally with every region accepting writes (multi-master).
-
Storage account — an Azure namespace bundling Blob, Table, File and Queue services.
-
Blob — Binary Large Object; any file stored in Blob storage; tiered Hot/Cool/Cold/Archive.
-
Data lake / ADLS Gen2 — a cheap, scalable store for raw data of any shape (Blob + hierarchical namespace).
-
Medallion architecture — organising a lake into bronze (raw), silver (cleaned), gold (business-ready) layers.
-
Data warehouse — a store of cleaned, structured data optimised for fast SQL analytics (schema-on-write).
-
Lakehouse — an architecture combining a data lake and warehouse capabilities.
-
ETL / ELT — Extract-Transform-Load (transform before load) vs Extract-Load-Transform (transform after load, in the destination).
-
Azure Data Factory — cloud data-ingestion and pipeline-orchestration service.
-
Synapse Analytics — integrated analytics platform (pipelines + Spark + SQL warehouse).
-
Microsoft Fabric — Microsoft’s unified SaaS analytics platform over OneLake; the strategic direction.
-
Azure Databricks — first-party managed Apache Spark platform for data engineering and ML.
-
Power BI — Microsoft’s reporting and dashboard (data-visualisation) tool.
-
Multi-model database — one database engine that can present several data models (document, key-value, column-family, graph); Cosmos DB is the Azure example.
-
Partition key — the field Cosmos DB (and Table storage) uses to spread data and load across partitions; an even-spreading key is the biggest single factor in performance and cost.
-
Hot partition — a single partition-key value taking a disproportionate share of traffic, causing throttling even when total provisioned throughput looks ample.
-
Throttling (HTTP 429) — “request rate too large”; the response when operations exceed provisioned RU/s, requiring back-off and retry.
-
Capacity mode — how Cosmos DB throughput is billed: provisioned (fixed RU/s), autoscale (10–100% of a set maximum), or serverless (pay per RU consumed).
-
Schema-on-read / schema-on-write — apply structure when data is queried (lakes) vs when it is written (warehouses).
-
Hierarchical namespace (HNS) — the Blob-account capability that turns a flat blob store into ADLS Gen2, adding real directories and file-level ACLs; a create-time choice.
-
Access tier — Blob cost/access setting: Hot, Cool, Cold, Archive (Archive is offline and must be rehydrated before reading).
-
Redundancy — how many copies Azure keeps and where: LRS, ZRS, GRS/GZRS (and read-access RA-GRS).
-
Integration runtime — the compute Data Factory uses to move/transform data: Azure IR, self-hosted IR (on-premises reach), or SSIS IR.
-
Dedicated SQL pool — Synapse’s provisioned MPP relational warehouse (billed in DWUs, pausable); the schema-on-write “serve” engine.
-
Serverless SQL pool — Synapse’s always-on, pay-per-TB-scanned engine that queries lake files in place with T-SQL.
-
Spark pool — managed Apache Spark clusters (in Synapse, Fabric or Databricks) for big-data transformation and ML.
-
OneLake — Microsoft Fabric’s single, tenant-wide logical data lake (“OneDrive for data”), built on ADLS Gen2 and storing data in open Delta Parquet.
-
Delta / Parquet — the open, columnar table format Fabric and lakehouses store data in, giving every engine one shared copy (Delta adds ACID transactions).
-
Shortcut (OneLake) — a pointer that virtualises external data (other ADLS, S3, GCS) into OneLake without copying it.
-
DirectLake — Power BI mode that reads OneLake Delta tables directly: import-level speed with no import copy. (Contrast Import and DirectQuery.)
-
Semantic model (dataset) — the modelled, reusable data layer a Power BI report queries.
-
Report vs dashboard — a report is a multi-page interactive Power BI document; a dashboard is a single-page canvas of pinned tiles (Power BI Service only).
-
Event Hubs — high-throughput streaming-ingestion service (the front door for stream processing); distinct from Event Grid, which routes discrete events.
-
Azure Stream Analytics — managed real-time stream processor running SQL-like queries over data in motion, using windowing (tumbling, hopping, sliding, session).
-
Real-Time Intelligence — Fabric’s SaaS streaming experience (Eventstream + KQL Eventhouse), the newer counterpart to Stream Analytics.
-
Batch vs stream processing — processing bounded data on a schedule (high throughput, higher latency) vs unbounded data continuously (low latency).
-
CAP theorem — a distributed store cannot simultaneously guarantee perfect consistency, availability and partition tolerance; the trade-off behind consistency levels.
Next steps
You now understand non-relational data and the analytics pipeline at fundamentals depth — the last big pillar of DP-900’s storage-and-analytics content. To go deeper:
- Next lesson: Azure Data Integration & Analytics: Data Factory, Synapse & Microsoft Fabric — the advanced, build-it version of Part 2.
- Go deeper on Cosmos DB: Cosmos DB partition-key design & RU optimisation.
- Go deeper on storage: Azure Storage accounts deep-dive — every option.
- Revisit the foundations: Core data concepts, roles & workloads and Relational data on Azure.