Where to Start When You Have a Lot of Data and Want to Be AI-Native

If you have a ton of data and leadership wants the company to be AI-native, the question that actually matters is not "which model is best?" It is: where do I start, what does the first phase cost, and what setup is cheapest while ROI is still unproven?

That first phase is almost always a blind-faith investment — you are spending to discover what you have before you can prove what AI will return. The pragmatic move is to keep that spend reversible: index and query the corpus, learn what is usable, do the structural lifting to get records tagged and ready for vectors, graphs, or agents later — and defer fixed infrastructure until routing and volume are measured on real work.

The short answer for most CEOs: start with routed API access (cheaper models on bulk extraction, frontier models only on escalation), not tenant hardware or on-prem unless confidentiality or regulation removes the choice. The reference case study in Q2 prices that path on a 1.5 TB corpus; the decision matrix shows when on-prem or tenant wins instead.

Your CTO needs Q1–Q5 for token math, routing rules, and utilization. You need the sequence table below and the routed lean row in Q2.

The sequence: from raw data to AI-ready

Before anyone talks about embeddings, knowledge graphs, or chatbots, the work runs in order:

The cost table in Q2 prices phase 3 (LLM inference) on a corpus that has already been through phases 1 and 2. It does not include storage, OCR, transcription, ETL, or engineer time for phases 1–2. On engagements like the one below, phases 1 and 2 often run 2–5× the LLM line in the table — sometimes more — before a single frontier token is spent. Skipping straight to phase 4 is why "AI-native" initiatives fail: there is nothing reliable to vectorize or graph yet.

If you are working through model choice for the first time, the guide to choosing the right AI model for your business covers the task-tier framework that sits underneath the cost question. Read that first, then come back here for the numbers.

---

Q1: What is the honest first question an IT leader should ask before any compute decision?

The concept of the token is almost comical given how powerful these models are. You are paying for every bit and byte that flows through the model. That generates a real cost at scale.

When you have a model on your own hardware, it may not do everything a frontier model can. It may not be as fast or as polished. But the business goal is not to use what is cool. The goal is to drive value at minimal cost.

Every IT leader should ask: what is the most effective approach for this specific workload? If the processing is straightforward, will run for many hours, and is better suited to repeating a simple task than reasoning from scratch each time, the answer may be a smaller model, locally hosted. The sharper version of the question is: do we actually need the speed, agility, and reasoning that comes with paying by the byte?

Teams get the sequencing backwards because they benchmark on the headline cost of a frontier API, see the per-million-token price, and assume local is cheaper without modeling full utilization. The benchmark that matters is total cost per unit of work delivered, not the API sticker price.

---

Q2: What do the actual numbers look like for a data ingestion and structuring workload?

This section is the reference case study behind the framework above — a real engagement shape, not a generic TCO exercise. The question it answers for your CTO: given a corpus this size, what does the LLM layer cost under disciplined routing versus token-maxing — and where do tenant, API, or on-prem fit?

Here is the scenario: a company needs to process 1.5 TB of data, covering 1.3 million call recordings, 6 million documents and PDFs, emails with attachments, tens of thousands of image files, and financial records. The goal is to become AI-first, but the first step is to listen before you speak. You need to index, mine and structure your data to figure out what you have and learn everything you can long before you consider if and how you have what you need to build AI with it — and long before you vectorize, graph, or ship an agent on top of it.

The scenario budgets below reflect the LLM inference layer only (phase 3), after the corpus has been indexed and mined. They do not include OCR, storage, database infrastructure, ETL orchestration, call transcription, engineer time, or human QA for phases 1–2. Budget phases 1–2 separately — on a corpus this size, expect roughly 2–5× the LLM figures in the table before you reach routed extraction at scale. In most real projects, those non-model items matter more than the token bill.

Corpus indexed and worked through: 1.5 TB total: 1.3 million call recordings, 6 million documents and PDFs, emails with attachments, tens of thousands of image files, and financial records.

LLM token budgets for the project (structured extraction and enrichment across the mined corpus, not raw file bytes): Lean: 1.0 billion tokens; Base: 1.5 billion; Heavy: 2.0 billion.

The table below has two sections. The first three rows are routed budgets — what a disciplined ingestion project actually plans for after indexing, deduplication, and model routing. The last row is a token-maxing check: what happens if leadership decides the safest path is one Opus pass on every processable record in the corpus. That row is illustrative, but the order of magnitude is real, and it is the mistake we see when "use the best model everywhere" replaces a routing strategy.

API list pricing (per million tokens): Claude Opus 4.6 — $5.00 input / $25.00 output; Claude Sonnet 4.6 — $3.00 input / $15.00 output; Kimi 2.6 — $0.95 input / $4.00 output (Anthropic, Moonshot/Kimi). Routed totals use a 90% input / 10% output blend — $7.00, $4.20, and $1.26 blended per million tokens respectively (Kimi rounds from $1.255) — typical for bulk structured extraction where outputs are compact JSON. Project total = token budget in billions × 1,000 × blended rate (e.g. 1.0B × $7.00/M = $7,000).

*Calls transcription (routed tiers only): 5,000 audio hours in each tier (300,000 minutes). Lean, base, and heavy differ by per-minute transcription rate ($0.0025, $0.0043, and $0.0077/min respectively), not by hours processed. Transcription is billed separately from LLM token budgets.

†Token-maxing check — conservative ~12 billion tokens: one Opus-class extraction pass on every processable record after transcription/OCR, with no cheaper-model routing — roughly 4 million document-class records × 2,000 tokens (8 billion) + 800,000 call transcripts × 1,500 tokens (1.2 billion) + emails, financial records, and image-derived text (2.8 billion). This assumes not every file in the 1.5 TB corpus is equally text-rich; token-maxing teams often spend more than this in practice through longer prompts, richer outputs, and re-runs. At list pricing, 12 billion Opus tokens cost roughly 12× the routed lean budget ($84,000 vs ~$7,000) — volume and model choice compounding, not a change in per-token list price. Infrastructure columns are marked "see note" because tenant and local stacks are not priced per token; even at this conservative estimate, API billing alone is ~$84,000 before OCR, transcription, storage, or QA.

What this means in practice:

The token-maxing trap. The bottom row is the check every leadership team should run before approving a "use Opus everywhere" strategy. Routed lean at 1.0 billion tokens is a planning budget — roughly 200,000 LLM jobs on high-value routes after indexing, not a blind pass on every file. Token-maxing at a conservative ~12 billion tokens is what happens when routing discipline disappears: 6× the token volume on the heaviest routed tier, and 12× the Opus spend versus routed lean (~$84,000 vs $7,000). Opus is both the most expensive model per token and the model teams reach for when they skip routing — so the cost compounds twice. Even Kimi at full-corpus volume ($15,120) exceeds the entire routed lean Opus budget. The defensible path is not "never use Opus." It is route first, then escalate.

Claude Opus 4.6 is the most expensive option at every tier. At list pricing and a 90/10 input/output mix, a 1.0-billion-token routed lean budget runs about $7,000 for the LLM layer alone — not a monthly subscription, but total variable API spend for the project at that token volume. Opus is also the easiest to defend if output quality is the primary success criterion. If you need to take a document extraction result to a board and explain why you chose the model, Opus gives you a defensible answer. Corporate culture rewards the most defensible choice. That is not always the right reason to pick a model, but it is a real force in large organizations.

Claude Sonnet 4.6 sits between Opus and Kimi on cost — roughly 40% less than Opus at the same token budget ($4,200 vs $7,000 at lean). For the majority of structured extraction and classification work in a mixed corpus, Sonnet is often the practical default: strong enough for ambiguous documents, materially cheaper than routing everything through Opus, and still a first-party Anthropic API if vendor consolidation matters.

Kimi 2.6 delivers a large cost advantage at every tier, particularly when outputs are compact and structured. For bulk extraction where the cognitive task is clear, Kimi at roughly six times cheaper than Opus (~$1,260 vs ~$7,000 at lean) is a serious option.

Azure Qwen 4B on a tenant GPU stack shifts your cost profile from variable token spend to fixed infrastructure spend. At the lean and base tiers, that shift is strongly unfavorable: you are paying $3,500 to $4,500 per month for infrastructure that API billing would cover for roughly $1,260 to $10,500 in total LLM project spend (Kimi lean through Opus base). Over a typical 90-day ingestion window, three months of tenant hosting alone is $10,500 to $13,500 before engineer time — comparable to or above the full Opus base token budget. Where tenant hosting wins is predictable throughput, data control, and the ability to amortize the system across future ingestion projects. The crossover is only meaningful if you have consistent, high-volume demand.

Mac Studio at $3,999 upfront is attractive for a proof of concept or secure local preprocessing. For 6 million documents, it is generally throughput-limited unless the project is slow and batch-heavy. It works well as a sandboxed preprocessing layer before higher-cost inference.

Research on local inference with current-generation GPU hardware shows electricity-only costs of $0.001 to $0.04 per million tokens, which is 40 to 200 times below budget-tier cloud APIs.¹ The catch is that electricity is only one line in the cost model.

When to use which approach

The table above is a cost model. The decision matrix below is how we apply it in practice — including cases where on-prem wins even when API pricing looks cheaper on paper.

Manufacturing example: A plant continuously ingests sensor logs, operator notes, and machine outputs that encode how the line actually runs — trade-secret detail that cannot go to a public API. The cognitive work is often repetitive: flag drift, classify fault codes, extract structured fields from shift reports. Demand on the model is real but not "frontier-scale chat volume." On-prem or dedicated edge hardware with a smaller open-weight model fixes cost, keeps data inside the facility, and runs at high utilization on a bounded stream. That is a different problem from the 1.5 TB document-and-call ingestion in the table — and a different answer than tenant or Opus API billing.

---

Q3: How do you route documents of varying cognitive difficulty across a mixed corpus?

Ingestion and structuring covers a wide range of cognitive difficulty. A 1.3-million-call corpus, 6 million documents, and a set of financial records are not the same extraction problem. The routing decision determines whether a small local model handles it or whether you pay for frontier reasoning.

The routing logic depends on two variables: the quality of the source data, and the level of reasoning effort needed to extract and structure it cleanly.

If the text is clear, well-formatted, and the extraction task is mechanical, cheaper models work well. A structured invoice PDF with consistent field placement does not need Opus. A call recording from a noisy environment with multiple speakers, overlapping dialogue, and domain-specific terminology might.

The mistake teams make is treating this as an engineering call when it is actually a business case call. A business owner looks at an invoice PDF and decides it is straightforward. An engineer who has actually run extraction on that class of document knows whether the formatting is consistent across 50,000 instances or whether 20 percent of them are scanned images with degraded OCR quality. The engineer makes the better routing call. The business leader needs to fund the diagnostic work to find out which category each document type falls into, rather than assuming uniformity.⁶

The practical framework: sample 200 to 500 documents from each source type, run them through a cheaper model, and measure extraction accuracy and clean-up effort. Where accuracy falls below an acceptable threshold, route to a more capable model. That empirical data is the business case for each routing decision.

---

Q4: When the workload is batch ingestion rather than real-time inference, how does the utilization math change?

This is where most cost models go wrong. When you are buying compute by the hour, whether that is a tenant GPU stack on Azure or a local Mac Studio, you want to ensure you are running it as effectively as possible and turning it off when it is not needed.

The critical insight for batch ingestion workloads is that much of the data work does not require LLM inference at all. Query operations, Python-based parsing, format conversion, deduplication, and schema mapping are standard engineering tasks. Running those on an LLM inference stack is wasteful. If the workflow is properly organized from the outset, you can confine LLM spend to the tasks that actually require probabilistic reasoning and handle the rest with conventional code.²

This changes the break-even calculation significantly. A tenant GPU stack that looks expensive at $3,500 to $4,500 per month becomes more defensible if it is running at 70 to 80 percent utilization on genuine inference tasks, not sitting idle while Python scripts run. A stack running at 20 percent utilization on a mixed workload is hard to justify against per-token API billing.

The four-line TCO model that most teams ignore covers: GPU rent (typically 60 to 70 percent of total), serving-stack operations (5 to 10 percent), inference engineer time (25 to 30 percent), and build-out opportunity cost.⁴ At 100 million tokens per month, API access wins decisively. At 1.2 billion tokens per month for chat-type workloads, self-hosted begins to cross over. At 5 billion tokens per month, self-hosting can cost as little as 0.14 times the equivalent API bill.

For the batch ingestion project in Q2, even the heavy tier's 2.0 billion tokens spread over a 90-day window averages roughly 22 million tokens per day — below the 10 to 30 million per day range where self-hosting starts to win on TCO alone. API billing remains the rational default until volume, utilization, or compliance forces infrastructure ownership.

---

Q5: What is the most expensive mistake, and what is the minimum viable business case for any compute decision?

There are too many variables to give a single answer, but the expensive mistake follows a consistent pattern.

The expensive mistake is when a champion inside the organization believes the hype and wants to use a frontier model for everything, or wants to ensure the highest quality when the task does not require it. Remember: corporate culture celebrates the most defensible choice. The best offence is a defence. Routing every workload through Opus because someone can defend it to leadership is a legitimate organizational pressure — and on a corpus the size described in Q2, even a conservative full-corpus pass is a ~$84,000 LLM bill before transcription, OCR, or QA, not a ~$7,000 to $14,000 routed plan. That is not a cost-effective strategy.

The equally expensive mistake on the infrastructure side is buying hardware or committing to a tenant stack because it signals technical sophistication, not because utilization and task requirements justify it. When we ask a leadership team why they want to own the infrastructure, the right answers are regulatory constraints, IP risk, trade-secret operational data, or data residency requirements that legally prohibit third-party processing. If the answer is closer to "we want control" or "it feels more secure," we raise a yellow flag.

On-premises hardware makes genuine sense for specific scenarios — including the manufacturing case in Q2: a facility continuously ingesting confidential machine and process data that defines competitive advantage. The AI work is often steady monitoring and structured extraction, not frontier reasoning on every record. Demand on the model may be modest, but it never stops; data cannot leave the building; and a fixed local cost beats metering tokens. You fix the hardware cost, run a smaller model at high utilization, and the security case is real. That is a different situation from a 90-day batch document ingestion project where API routing is the rational default.³

The minimum viable business case before any compute decision gets signed off should answer four questions:

1. What is the actual token volume, measured from a real sample, not estimated?
2. What is the projected utilization rate, including the hours when the system will be idle?
3. What regulatory or contractual constraints apply to the data being processed?
4. What is the engineer overhead required to operate and maintain the stack, including model updates?

If you cannot answer all four with real numbers, you are not ready to sign a compute contract. You are ready to run a diagnostic sprint and produce the data that makes the decision defensible.

The ROI checkpoint for CEOs: treat phases 1–3 as the discovery investment — you are buying the right to know what your data is worth to AI before you commit to phase 4 at scale. The cheapest credible path for most companies is routed API spend in the ~$7,000 to $14,000 LLM range (plus non-LLM ingestion costs), not ~$84,000 token-maxing and not $10,000+ in fixed monthly infrastructure before you have proof. Once a routed slice produces measurable extraction quality and a use case leadership will fund, you have the business case to scale tokens, add vectors or a graph, or — if volume and compliance require it — move to tenant or on-prem.

If you want to work through this kind of decision with structured support, the AI Sprint is designed to take a leadership team from a compute question like this one to a signed-off architecture decision in five days. The deliverable is a business case you can take to the board, not a vendor recommendation.

Separately, if the question is about ongoing execution rather than a single project decision, the AI Department retainer covers the kind of continuous model routing, utilization monitoring, and cost governance that keeps these decisions from drifting over time.

The broader strategic framing for this class of decision is covered in what every CEO needs to know before starting an AI initiative, which addresses sequencing and governance before the infrastructure question even arises.

---

---

Frequently Asked Questions

At what monthly token volume does local hosting become cheaper than a frontier API?

Self-hosting starts to win between 10 million and 30 million tokens per day, according to SitePoint's 2026 TCO analysis. Below that threshold, API billing is almost always cheaper when you count the full stack: GPU rent, serving-stack operations, and inference engineer time. Above it, savings of 40 to 60 percent over equivalent commercial API spend become achievable, but only when hardware utilization is high and idle time is minimized through disciplined scheduling.

Is a 7B to 70B parameter open-source model sufficient for enterprise AI tasks?

For roughly 80 percent of enterprise workloads, yes. Most tasks fall into what practitioners call Tier 1 or Tier 2 cognitive demand: structured extraction, classification, summarization, and routine Q&A. A well-configured 7B to 70B parameter model handles those reliably. The remaining 20 percent, complex legal reasoning, nuanced financial analysis, or multi-step synthesis across ambiguous inputs, genuinely benefit from frontier-grade capability. The mistake is routing everything to a frontier model because it feels safer, not because the task requires it.

What hidden costs make self-hosted LLMs more expensive than API pricing suggests?

Four costs get missed most often. First, inference engineer time: someone has to deploy, monitor, and update the serving stack. Second, idle capacity: a GPU instance running at 20 percent utilization is expensive dead weight. Third, model-swap cycles: when a better open-source model ships, local deployments need re-quantization, smoke tests, and staged rollouts. Fourth, build-out opportunity cost: every engineering hour spent on model ops is an hour not spent on the product. API providers absorb all four of these costs and spread them across thousands of customers.

Should we start with API access or invest in local inference infrastructure first?

Start with API access — especially while ROI is still unproven. The reasoning is simple: API access is reversible; infrastructure investment is not. Use the API phase to ingest, index, and query a representative slice, measure actual token volumes, identify which tasks genuinely need frontier reasoning versus structured extraction, and build the utilization model that justifies a hardware decision. Most teams that jump to local hosting early discover their real workload is smaller than projected, their utilization rate is lower than assumed, and the engineer overhead is higher than budgeted. Validate demand before fixing the cost base.

How do data residency and regulatory requirements change the API versus local hosting calculation?

Regulatory or IP risk is one of the few reasons that genuinely justifies owning infrastructure before you have the token volume to make it economical. If your data cannot leave your jurisdiction, if your contracts prohibit third-party model processing, or if your IP exposure from sending proprietary documents to an external API is material, on-premises or dedicated tenant hosting moves from optional to required. The question to ask leadership is: do you need to own this for compliance reasons, or does it just feel safer? One answer changes the math; the other does not.

---

About the Author: Issy is the AI Orchestrator at Aspiro AI Studio — translates strategy into executable delivery; writes about what actually works.

References

1. arXiv: Private LLM Inference on Consumer Blackwell GPUs: A Practical Guide for Cost-Effective Local Deployment in SMEs
2. SitePoint: Open-Source vs Commercial LLMs: The Complete Guide (2026)
3. SitePoint: Local LLMs vs Cloud APIs: 2026 Total Cost of Ownership Analysis
4. Digital Applied: Self-Hosting Frontier AI Models: 2026 TCO Analysis
5. M. Gobbi (Substack): The Real Cost of Running Frontier AI Models in 2026
6. Michael Hannecke: Your Enterprise AI Doesn't Need a Frontier Model
7. Anuj Paryemalani: Frontier LLM vs Open-Source: How do you actually decide?