AI Project Scoping Checklist: A 1-2 Week Paid Feasibility Framework
A 50-item AI project scoping checklist plus the 1-2 week paid feasibility framework DevStudio uses before any $14k-$85k engagement. Free to use, $700-$2,800 if you want us to run it.
Direct Answer
An AI project scoping engagement is a paid 1-2 week feasibility study that delivers a written go/no-go recommendation, a 50-item readiness checklist, an Eval plan, a unit-cost model, and a draft architecture — before any code is written. DevStudio runs this engagement for $700-$2,800. About one in four engagements ends in “do not build this yet,” which is precisely the value: catching a doomed project in week one is cheaper than discovering it in month six.
TL;DR
- Free discovery calls fail at AI scoping. Too short, sales-biased, and produce no written deliverable you can take to your CFO.
- Paid Scoping is a fixed-price diagnostic, not a sales pitch. $700-$2,800, 1-2 weeks, written deliverables, no commitment to a build engagement.
- Four modules cover the failure surface: Business Alignment, Data Readiness, Technical Solution Architecture, and Commercial Feasibility.
- The 50-item checklist below is the same one we run internally. Use it free. If you want help running it on a real project, that is the Scoping engagement.
- About 25% of Scopings recommend NOT building. That is the point. The other 75% convert into project-based engagements at $14k-$85k over 4-10 weeks with Eval Week 1 and a 6-month warranty.
Why Free Discovery Calls Don’t Work for AI Projects
Most AI vendor “discovery calls” are 30-60 minute sales conversations dressed up as consulting. They have four structural problems that make them useless for an AI feasibility decision.
They are too short. AI projects fail at the intersection of workflow choice, data readiness, eval design, and unit cost — four substantive engineering questions. None of them can be diagnosed in a single call.
They are sales-biased. A free discovery call is paid for by the next engagement. The structural incentive is to find a way to say yes. We have walked away from $40k engagements during paid Scoping because the workflow was wrong. We could not have walked away during a free call — there was no contract for “no” to live inside of.
They produce no written deliverable. Notes, maybe a deck. Nothing your CFO can read.
They skip data and unit-cost questions entirely. Free calls almost always focus on the workflow itself and skip the corpus inspection, the labeling sample, and the cost-per-task model — three places where most pilots actually die. See Why 60% of Enterprise AI Pilots Die.
The Paid Scoping Model: 1-2 Weeks, $700-$2,800, Concrete Deliverables
DevStudio Scoping is a fixed-price 1-2 week engagement. Pricing scales with project complexity:
| Tier | Scope | Price | Duration |
|---|---|---|---|
| Lite Scoping | Single workflow, ≤2 integrations, no compliance | $700-$1,200 | 3-5 working days |
| Standard Scoping | Single agent or small RAG, 3-5 integrations | $1,200-$2,000 | 1 week |
| Deep Scoping | Multi-agent, large RAG corpus, regulated industry | $2,000-$2,800 | 2 weeks |
Every tier delivers four written artifacts: a Scoping Report (written go/no-go + key risks), a Readiness Checklist Results (50 items scored), an Eval Plan (50 labeled cases co-built with your domain owner), and a Cost Model with one-page architecture sketch.
If you decide to proceed with a build engagement, the Scoping fee credits 100% toward the project. If you decide not to, you still keep all four artifacts and can take them to any other vendor.
The 4 Modules Inside a DevStudio Scoping Engagement
Module 1: Business Alignment
Goal: Confirm the project moves a specific business metric, has an executive sponsor, and ROI math that survives a CFO review.
Activities: 60-minute working session with sponsor; map workflow to 1-2 specific KPIs (revenue, cost-to-serve, cycle time, error rate); back-of-envelope ROI model with three scenarios (conservative, expected, optimistic); identify success metric and failure threshold.
Deliverable: Two-page Business Alignment memo with KPI mapping, ROI model, success/failure thresholds, and named sponsor.
Module 2: Data Readiness Assessment
Goal: Verify that the data needed by the AI system actually exists, is accessible, and is clean enough to retrieve from.
Activities: Inventory of data sources; quality scorecard (30 sample queries traced against the corpus, scored on answer presence, currency, unambiguity); gap analysis (missing, stale, duplicated, locked behind permissions); recommended remediation work — often a content-ops sprint before the AI sprint.
Deliverable: Data Readiness Scorecard with named owners, blocker list, and recommended sequence.
Module 3: Technical Solution Architecture
Goal: Decide the technical approach (RAG vs. fine-tuning vs. prompt engineering vs. hybrid), model selection, integration map, and Eval plan.
Activities: Architecture sketch (components, data flow, model placement, retrieval design); model routing recommendation (frontier / strong / fast / open-source tiers — see AI Agent Token Cost Audit); Eval plan (50 starter cases, scoring rubric, CI gating — see Eval Framework Week 1); integration map.
Deliverable: One-page architecture diagram + Eval plan + integration map.
Module 4: Commercial Feasibility
Goal: Produce a defensible cost model, timeline, risk register, and a build-vs-buy recommendation.
Activities: Unit-cost ceiling (max acceptable cost per resolved query / per generated document / per workflow run); blended token cost projection at expected traffic; timeline estimate (4-10 weeks typical for production-grade); risk register with top 5 risks; build vs. buy comparison if a commercial alternative exists.
Deliverable: Cost Model with three-scenario forecast and a written go/no-go recommendation.
The 50-Item Scoping Checklist
Use this yourself. Each item is a question you must be able to answer “yes” to with evidence. “Probably,” “we think so,” and “we’ll figure it out” all count as no.
Category 1: Business Goal & Success Metrics (10 items)
- Specific KPI named. One named KPI with a current baseline. Red flag: “Increase efficiency” with no number attached.
- Executive sponsor identified. Named sponsor at VP+ level with budget authority. Red flag: “Whoever the CTO assigns.”
- Success threshold defined. Numeric target with deadline. Red flag: “We’ll know it when we see it.”
- Failure threshold defined. A kill metric with a date. Red flag: No defined exit criteria.
- ROI model built. One-page model with three scenarios. Red flag: “AI will save us money” without arithmetic.
- Timeline aligned with business cycle. Specific quarter or campaign. Red flag: “ASAP.”
- Counterfactual considered. Documented status quo cost or risk. Red flag: “We just have to do AI.”
- User research completed. ≥5 user interviews in the last 90 days. Red flag: User research = team intuition.
- Workflow tolerates probabilistic output. Written tolerance threshold or named reviewer process. Red flag: Zero tolerance with no reviewer.
- Build/buy decision made consciously. Compared ≥2 commercial alternatives. Red flag: “We just want our own.”
Category 2: Data & Knowledge Sources (10 items)
- Data inventory complete. Spreadsheet with system, owner, format, volume. Red flag: “It’s all in SharePoint somewhere.”
- Data access cleared. AI service account provisioned with documented scope. Red flag: “We’ll figure that out later.”
- Document currency reviewed. For top 30 queries, the right answer is the latest version. Red flag: No version control on source docs.
- Contradictions surfaced. List of known contradictions and resolution plan. Red flag: “We don’t have any” without a check.
- Structured vs. unstructured ratio known. Numeric breakdown. Red flag: “Mostly unstructured” with no number.
- OCR / extraction pipeline scoped. Named pipeline (e.g., Textract, custom). Red flag: PDFs without OCR considered “ready.”
- Data freshness SLA defined. Numeric SLA (hourly / daily / weekly). Red flag: “Real-time” without justification.
- Knowledge-base governance owned. Named owner and weekly cadence. Red flag: No one named.
- Sensitive fields identified. Field-level inventory. Red flag: “We don’t have PII” without an audit.
- Sample query trace completed. Pick 30 real user queries — for how many is the right answer present, current, and unambiguous? Acceptance: ≥80% pass rate. Red flag: Below 70% pass rate.
Category 3: Technical Constraints & Stack (10 items)
- Hosting environment named. Specific provider and region. Red flag: “Cloud” with no provider.
- Latency budget defined. Numeric ms target. Red flag: “As fast as possible.”
- Throughput projection done. Expected requests-per-minute at launch and year-1. Red flag: “We’ll scale as needed.”
- Model selection scoped. Frontier / strong / fast / open-source ladder defined. Red flag: “GPT-4 for everything.”
- RAG vs. fine-tune vs. prompt decided. One-paragraph rationale. Red flag: “We’ll fine-tune” with no data on labeled examples.
- Integration map drawn. Diagram with auth model. Red flag: “It needs to talk to Salesforce” without auth design.
- Existing code reviewed. Code review summary if hardening a demo. Red flag: “It’s fine, just productionize it” with no review.
- Eval plan drafted. 50+ starter cases and a scoring rubric. Red flag: “We’ll add Evals later.”
- Observability stack chosen. Named tools (LangSmith, Datadog, OpenTelemetry). Red flag: “We’ll figure it out.”
- Failure-handling pattern defined. Fallback model, deterministic path, or graceful degradation. Red flag: “It just works.”
Category 4: Compliance, Security, Privacy (10 items)
- Regulatory profile mapped. Named regulations and applicable clauses. Red flag: “We’re probably fine.”
- Data residency boundary set. Documented region constraint. Red flag: No residency review on a regulated workload.
- PII redaction strategy defined. Pipeline diagram with redaction step. Red flag: PII in vendor logs.
- Vendor data-usage opt-out confirmed. Documented setting per vendor. Red flag: Default settings with sensitive data.
- DSR (Data Subject Request) workflow. Named workflow for user-data deletion. Red flag: No DSR plan on consumer data.
- Auth model defined for tools. Per-tool auth scopes. Red flag: Shared service account with full access.
- Secrets management. Vault + quarterly rotation policy. Red flag: Keys in
.envchecked into git. - Prompt-injection guardrails. Named guardrail (commercial or in-house). Red flag: No guardrail on a public-facing agent.
- Rate limiting defined. Per-user, per-tenant, per-IP limits. Red flag: No limits.
- Incident response runbook drafted. Named on-call and top 12 failure classes. Red flag: “Whoever picks up Slack.”
Category 5: Operations, Maintenance, Cost (10 items)
- Unit-cost ceiling set. Numeric target. Red flag: “We’ll watch the bill.”
- Cost-per-task instrumented. Per-task cost logged in production from day one. Red flag: Only total spend visible.
- Token budget per request. Per-route limits. Red flag: Unbounded generation.
- Caching strategy. Documented cache tier. Red flag: No caching on FAQ-shaped traffic.
- System prompt budget. Token budget + review cadence. Red flag: Prompt grows monotonically.
- Monitoring & alerting. Named dashboards. Red flag: “We check it sometimes.”
- Eval cadence post-launch. Nightly + pre-deploy. Red flag: “When we remember.”
- Quarterly token audit booked. Calendar entry every 90 days. Red flag: No post-launch review.
- Retainer / handoff plan. Named retainer or in-house owner. Red flag: “Whoever built it.”
- Source-code ownership clear. Customer owns source code, deployment docs, and runbook in writing. Red flag: Vendor-locked code.
Scoring rule of thumb:
- 45-50 yes: Ready to build. Proceed.
- 35-44 yes: Fixable. Address gaps in Scoping or pre-build sprint.
- 25-34 yes: Doomed without intervention. Run a paid Scoping or postpone.
- <25 yes: Do not build. Either remediate the foundation first, or pick a different workflow.
When Paid Scoping Recommends NOT Building
About one in four DevStudio Scopings ends in “do not build, or do not build yet.” The most common reasons:
- Workflow is wrong. Emotional-labor heavy, data-chaos heavy, or stakes-too-high without a reviewer budget.
- Data is not ready. Corpus has <70% sample-query pass rate. 2-4 weeks of content-ops first is faster end-to-end.
- No business sponsor. No VP+ owns the outcome. The pilot will run out of political support.
- A commercial alternative beats build. $20-200/seat SaaS handling 80% of the workflow rarely loses on TCO to a $14k-$85k custom build.
- Unit cost cannot work at scale. Blended token cost projection exceeds defensible per-task economics. Pilot can ship; cannot scale.
We say no because the alternative — taking the engagement and discovering the same thing in month four — is worse for both sides. This is part of how we keep our scope edges clean as a Hangzhou-based, ex-Tencent engineering team that delivers project-rate engagements at $14k-$85k over 4-10 weeks with Eval Week 1, a 6-month QA window with quarterly token audits, and full source-code ownership on handover.
Pricing & Timeline
| Engagement | Price | Duration | Deliverables |
|---|---|---|---|
| Lite Scoping | $700-$1,200 | 3-5 days | 4 artifacts, single workflow |
| Standard Scoping | $1,200-$2,000 | 1 week | 4 artifacts + full Eval plan + architecture diagram |
| Deep Scoping | $2,000-$2,800 | 2 weeks | 4 artifacts + extended Data Readiness audit + multi-agent architecture review |
| Project-based delivery | $14k-$85k | 4-10 weeks | Production-grade build with Eval Week 1, 6-month QA window, full source code + deployment docs + runbook |
If you proceed to a build engagement, the Scoping fee credits 100% toward the project.
Frequently Asked Questions
How is paid Scoping different from a free discovery call?
A free discovery call is a 30-60 minute sales conversation with no written deliverable. Paid Scoping is a 1-2 week engagement with four written artifacts: Business Alignment memo, Data Readiness Scorecard, Architecture and Eval Plan, and Cost Model with risk register. Paid Scoping costs $700-$2,800 and can legitimately end in "do not build" — about 25% of ours do. The Scoping fee credits 100% toward the build engagement if you proceed.
Can I run this 50-item checklist myself instead of hiring DevStudio?
Yes. The checklist is the same one we use internally. If you have an experienced AI lead in-house with 5-10 days of capacity, run it yourself. Score below 35 yes / 50 = either internal remediation or external Scoping. The checklist is free; the structured engagement is the paid product.
How long does Scoping take if my project spans multiple workflows?
Multi-workflow Scoping typically lands at the Deep Scoping tier (2 weeks, $2,000-$2,800). Each workflow gets a separate Module 2 (Data Readiness) and Module 3 (Architecture) treatment. Module 1 Business Alignment and Module 4 Commercial Feasibility are usually combined.
What if my company will not approve a paid engagement before the project starts?
Two paths. First, frame Scoping as risk mitigation: a $1k-$3k engagement with 25% chance of saying no-go is the cheapest insurance against a $50k-$200k mistake. Second, structure Scoping as the first paid milestone of the build engagement, with explicit go/no-go gate between Scoping and build. Total contract is signed for the full project, but build budget is released only after Scoping says go.
Does Scoping work for a demo we already shipped?
Yes — about a third of our Scopings are audit engagements on shipped demos. The four modules apply directly: Business Alignment confirms KPI relevance, Data Readiness re-runs on the live corpus, Architecture review compares what was built vs. what should have been, Commercial Feasibility re-prices unit cost. Bring 30 days of production logs (redacted) for sharper Data Readiness analysis.
What happens if Scoping recommends not building?
You keep all four written artifacts. Take the Business Alignment memo to your board, the Data Readiness Scorecard to your CDO, the Architecture sketch to whichever vendor you pick next, the Cost Model to your CFO. The Scoping fee is not refundable, but the deliverables are designed to be useful regardless of who builds. We typically offer a free 30-minute follow-up 90 days later to re-evaluate the recommendation.
Is the 50-item checklist also available as a downloadable PDF?
A downloadable PDF version is in production. In the meantime, the full checklist on this page is yours to use, copy into your project tooling, or adapt for internal review. We do not gate the checklist behind a form. If you want help running it on your specific project, that is the Scoping engagement.