Guide · 2026
Azure AI Foundry: a practical guide (2026)
Azure AI Foundry is Microsoft's platform for building, evaluating and operating generative-AI applications on Azure. It organises work into hubs (shared, governed infrastructure) and projects (individual workloads), and bundles a model catalogue, prompt and evaluation tooling, content safety and deployment. This guide explains how it fits together and how to move a use case from pilot to production without the cost and risk surprises.
- —AI Foundry hubs hold shared, governed infrastructure; projects are individual AI workloads that inherit it.
- —The biggest production risks are cost (token/PTU), security (private networking, identity) and governance (evaluations, content safety).
- —Choose provisioned throughput (PTU) for steady, high-volume traffic and pay-as-you-go for spiky or early-stage workloads.
- —Put an AI landing zone in place before you scale — retrofitting guardrails is far more expensive.
What Azure AI Foundry is
Azure AI Foundry is Microsoft's platform for building generative-AI applications on Azure. It brings the model catalogue, prompt engineering, evaluations, content safety, deployment and monitoring into one place, and organises everything into hubs and projects.
- A hub is shared, governed infrastructure — networking, security, connections to data and models, and compute — that your teams build on.
- A project is an individual AI workload (a RAG app, an agent, a document-intelligence pipeline) that lives inside a hub and inherits its governance.
This separation is what lets a platform team set guardrails once and let product teams ship safely on top.
Why the foundation matters more than the model
It is easy to demo a large language model. It is hard to run one in production — securely, affordably and in a way you can trust in front of customers. The difference is the foundation:
- Security: private networking to model endpoints, Microsoft Entra identity and RBAC, no keys in code.
- Cost: the token-versus-provisioned-throughput decision, model right-sizing, caching and monitoring.
- Governance: evaluations, content safety, logging and data-residency controls aligned to the EU AI Act and ISO 42001.
Retrofitting these onto a live workload is painful and expensive. Putting an AI landing zone in place first is the single highest-leverage decision most organisations make.
Pay-as-you-go vs provisioned throughput (PTU)
Azure OpenAI can be billed per token (pay-as-you-go) or by reserving provisioned throughput units (PTU):
| Pay-as-you-go | Provisioned throughput (PTU) | |
|---|---|---|
| Best for | Spiky, early-stage or low-volume traffic | Steady, high-volume production traffic |
| Cost model | Per 1K tokens | Reserved capacity, fixed monthly |
| Latency | Shared, variable | Dedicated, predictable |
| Commitment | None | Monthly / yearly reservation |
Most workloads start pay-as-you-go and move to PTU once traffic is steady enough that reserved capacity is cheaper. Getting this decision right is one of the biggest levers on your AI bill.
From pilot to production
A dependable path looks like this:
- Assess — an AI-readiness review of your estate, data and goals.
- Design — an AI landing zone: identity, networking, policy and responsible-AI guardrails, as code.
- Build — the use case, with evaluations, content safety and CI/CD.
- Operate — ongoing cost and platform management so it stays cheap, secure and current.
Getting help
North Peak Cloud builds production Azure AI Foundry workloads for organisations across the UK, US and Canada — from the landing zone to the use case to ongoing cost control. If you have an AI idea and need it to actually run, that is exactly what we do.
Asked and answered.
What is the difference between a hub and a project in Azure AI Foundry?+
A hub is shared, governed infrastructure — networking, security, connections and compute — that multiple teams build on. A project is an individual AI workload that lives inside a hub and inherits its governance and connections. You typically have one hub per environment and many projects.
Azure AI Foundry vs Copilot Studio — which should I use?+
Use Azure AI Foundry when you need full control over models, data, evaluation and deployment for a custom AI application. Use Copilot Studio for lower-code conversational copilots built on Microsoft 365. Many organisations use both; North Peak Cloud helps you choose per use case.
How do I control costs in Azure AI Foundry?+
Pick the right pricing model (pay-as-you-go vs provisioned throughput), right-size the model to the task, cache and batch requests, and monitor token usage for anomalies. Building this in from the start is far cheaper than retrofitting it.