CCAR-P practice questions, free.

You are designing an agent that can modify customer records. Where do approval gates and audit trails belong?

• In the observability stack, with traces sampled at 10% and reviewed in a weekly governance meeting
• In a post-launch hardening phase, once task completion is stable enough to show which code paths actually matter
• In the tool layer from the start, with a log per tool call and an approval gate on every irreversible write
• In the system prompt, with instructions to confirm before writing and to summarize every action at the end of each run

How do you decide which tools an autonomous agent may call without a person?

• Let the model rate the risk of each call itself and request approval whenever its stated confidence falls below a configured threshold
• Gate every tool at launch and remove the gate from whichever tools users approve most often during the first month
• Allow every tool inside the agent's own sandbox and require approval only for calls that cross a network boundary
• Classify each action by blast radius: read-only and reversible calls run freely, irreversible or external ones need approval

A client engagement involves personal data. What must the architecture cover beyond the model calls?

• A data processing agreement with the model provider plus a zero-retention setting on the API organization
• Encryption at rest and in transit for every store the agent touches, with keys held in the client's own key management service
• An anonymized eval set so that test runs and CI pipelines contain no real customer records or identifiers
• Retention limits, PII redaction where possible, isolation from other clients' data, and access control on the logs

An executive asks whether to fund the agent system. What framing lands?

• Cost per completed task, measured reliability, the known failure modes, how each is caught, and what fixing them takes
• The eval harness design, benchmark scores against the previous model, and the token budget per task
• A live demo of the happy path with a projection of headcount savings once the agent reaches full autonomy
• The architecture diagram, the tool inventory, and the prompt techniques that produced the accuracy gains

A new model version is available. What is the responsible upgrade path for a production agent?

• Run the new model in shadow mode on live traffic for a quarter and adopt it only if no user reports a regression
• Run the golden eval set on the new model, compare against the current baseline, then roll out gradually with monitoring
• Pin the current model until the provider's deprecation date, then switch in one release with a rollback tag ready
• Point staging at the new model on release day and promote to production after a week with no complaints from the engineers using it

Which metrics tell you a production agent is drifting?

• Ticket volume from users, time to first response from the team, and the count of open bugs tagged to the agent
• Tool error rate, cost per completed task, latency, and eval scores tracked over time
• Token volume per day, API error codes, and the number of prompt versions shipped in the quarter
• Uptime of the agent service, p99 latency of the model endpoint, and the provider's status page history

A multi-agent system fails in cascades when one worker misbehaves. What architectural controls apply?

• Redundant workers per task with majority voting, so a single misbehaving worker is outvoted by the others
• Timeouts, bounded retries, circuit breakers per worker, and a coordinator that can stop the run
• Stricter output schemas and a shorter system prompt per worker, reducing the chance that any worker misbehaves
• A larger context window for the coordinator so it can hold every worker transcript and spot problems itself

Prompts are edited by several engineers and quality varies release to release. What fixes the process?

• Freeze the prompts for a quarter at a time and batch all edits into a single tested release
• Move the variable parts of each prompt into a config service so engineers edit settings rather than prose
• Version prompts in source control with review, and run the eval set in CI on every change
• Assign one owner per prompt who approves edits by hand and keeps a changelog in the team wiki

When should a deployed agent system be retired?

• When the model it was built on is deprecated, since a migration costs about as much as a rebuild
• When the original sponsor leaves and no team is willing to take over on-call for it
• When its measured value no longer covers cost and upkeep, and a simpler alternative covers the need
• When its eval scores fall below the launch baseline for two consecutive review periods

A stakeholder wants "an agent for everything". What do you establish before building?

• Success metrics, the tasks in scope, and which decisions stay with people
• A pilot with the widest possible scope so that real usage reveals which tasks are worth keeping
• The full tool inventory the agent would need and the MCP servers that expose each system
• The model tier, the token budget per task, and the evaluation harness for the first release

A multi-agent system has a planner, three workers, and a verifier. The verifier rejects 40% of worker output. Where do you look first?

• The planner's decomposition, adding a second verifier pass so borderline output gets a tie-breaking judgment
• The verifier's rubric, since a strict rubric applied to reasonable output is the more common cause of a high rejection rate in practice
• The worker model tier, since a stronger model usually cuts the rejection rate faster than editing prompts does
• The worker briefs and acceptance criteria, since a high rejection rate usually means the task or the criteria are unclear

Your agent's tool surface has grown to 60 tools and accuracy is falling. Best structural fix?

• Group tools behind fewer well-scoped interfaces or route to specialist subagents with small tool sets
• Rewrite all 60 descriptions with examples and add a tool-selection few-shot block to the system prompt
• Move to a larger model tier, since tool selection accuracy scales with capability at this tool count
• Order the tool list by call frequency so the most used tools appear first in the model's context

A long-running agent must survive process restarts. What is required?

• Idempotent tools, so that re-running the task from the first turn after a restart produces the same result
• A larger context window so the full history fits when the process reloads the last transcript
• Durable state: the plan, progress, and pending tool results persisted outside the process
• A supervisor that restarts the process and re-sends the original request with a note to continue where it left off

Which design keeps a coordinator's context from growing without bound over hundreds of subtasks?

• Each subagent appends its full transcript so the coordinator can re-check work without re-running it
• Subagents return compact results; the coordinator keeps a structured ledger and compacts periodically
• Restart the coordinator on a fixed cadence and hand the new instance the last user message plus the task list
• Stream subagent output straight to the user and keep the coordinator stateless between subtasks

You need a prompt that behaves the same across model upgrades. What practice helps most?

• Few-shot examples drawn from the previous model's best outputs so the new model imitates them
• A detailed persona and tone section, since style stability is what users notice most across model versions
• Explicit, testable instructions plus a golden eval set that runs on every model change
• Pin the model version in config and treat any upgrade as a new project with a freshly written prompt

An MCP server exposes tools with side effects to several agents. What must the server enforce itself?

• Authorization per caller and idempotency on side-effecting operations
• Structured logging of every call with the caller's agent id so misuse can be traced after the fact
• Rate limits per client and a global concurrency cap so no single agent can exhaust the backend
• Tool descriptions that state the side effects clearly so calling agents gate them appropriately

Which caching strategy suits a system with a large stable policy document and per-user variable content?

• Skip caching, since per-user variation means most requests miss and the cache write premium is wasted
• Cache the policy document as the prefix and put per-user content after the breakpoint
• Cache each user's full conversation as its own prefix so that repeat visits hit the cache
• Place the user content first so the model sees it before the policy, then cache the policy block

An agent's plan step produces good plans but execution drifts. What is the architectural response?

• Increase the planning depth so the plan covers more of the edge cases the executor might wander into during long runs
• Add a retry at the end of execution that reruns the task when the output fails validation
• Drop the separate planning step and let a single stronger model plan and execute in one pass
• Make the plan an explicit artifact that execution checks against, with a review step when it deviates

Which of these is the right place for a hard spending limit on an agent?

• The system prompt, as a stated token budget the model tracks and stops at on its own
• The runtime: a budget enforced by the loop or a hook that stops the run
• The tool descriptions, with a cost annotation so the model prefers cheaper calls
• The billing dashboard, with an alert to the on-call engineer at 80% of budget

A system uses one prompt for five customer segments and quality varies by segment. Approach?

• Add few-shot examples from the weakest segment until its scores match the others
• Raise the model tier for all segments, since the variance reflects a capability limit
• Expand the single prompt with a section per segment so the model applies the right guidance
• Route by segment to specialized prompts and evaluate each route separately

What is the main risk of letting subagents spawn their own subagents without limits?

• Duplicate work, since sibling subagents cannot see each other's results without a shared store
• Loss of detail, since each level summarizes upward and precision degrades with depth
• Rate limit exhaustion at the API layer, which the provider handles with backoff in any case
• Unbounded cost and untraceable behavior, so depth and fan-out need caps

A retrieval-augmented agent returns confident answers from stale documents. Fix?

• A system prompt instruction telling the model to hedge when it is unsure whether a source is current
• A larger retrieval window so that newer documents are more likely to appear among the results
• Freshness metadata in the index, ranking or filtering by recency, and surfacing dates in the answer
• A reranker tuned on relevance so the most on-topic chunk wins regardless of its age

Which is the strongest reason to keep tool results out of the system prompt?

• System prompts have a hard length cap that tool results would exceed on most real tasks
• Tool results in the system prompt are billed at a higher rate than the same tokens in messages
• The system prompt is for instructions; data belongs in turns so it stays data and can be managed
• Prompt caching only applies to the system prompt, so adding data there invalidates the cache

When should an architect prefer a deterministic function over a model call inside a workflow?

• Only for arithmetic and date math, where model error rates are known to be high
• When the step is on the critical path, keeping model calls for background work
• Only when latency matters, since a model call on a small tier is comparable in cost
• Whenever the logic is fully specifiable; code is cheaper, faster, and testable

A Claude Code deployment across a 200-engineer org needs consistent behavior. Which lever scales?

• Each engineer's user-level settings and memory, tuned during an onboarding workshop
• An internal wiki with recommended prompts and a Slack channel for questions
• A central prompt-engineering team that reviews a sample of each engineer's session transcripts weekly
• Checked-in CLAUDE.md, shared skills, hooks, and project-scoped MCP config

What should an evaluator in an evaluator-optimizer loop output?

• A single numeric score, keeping the evaluator cheap and its output easy to threshold
• A pass or fail flag plus the full reasoning trace so that humans can audit the decision later without re-running it
• A corrected version of the output, so the loop converges in one step instead of several
• A structured verdict against explicit criteria with specific feedback the generator can act on

Which context strategy suits an agent that reads thousands of tickets a day?

• One agent per ticket with no shared state, relying on the ticket system for any aggregation
• Embed all tickets and let the agent retrieve as needed with no persistent working state
• Load the full day's tickets into one long-context call so the model can spot cross-ticket patterns
• Process in batches with per-batch summaries and a rolling state document

A team asks whether to expose a tool as one call with many options or several narrow tools. Guidance?

• Prefer one tool with an options schema, since fewer tools keep the definitions block small
• Mirror the backend REST endpoints one to one so that the mapping between each tool and its API call is obvious to maintainers
• Prefer many tools in every case, since each extra tool adds a few tokens and selection is cheap
• Prefer narrow, clearly described tools; combine only when the model routinely needs them together

Which artifact should exist before an agent is allowed to act on production systems?

• A risk register listing each action, its blast radius, its gate, and its rollback
• A load test showing the agent stays within the production API's rate limits
• A signed change request from the system owner covering the agent's service account
• A passing eval run above the launch threshold on the current golden set

An audit asks why the agent took a specific action last month. What lets you answer?

• The agent's memory store, which records the facts it had learned by that date
• A replay of the request on today's system with the same inputs to reproduce the decision
• Per-call logs of inputs, tool calls, results, and decisions, retained per policy
• The prompt version deployed that month and the eval results attached to its release

What is a kill switch in an agent system?

• A tool the model calls when it detects it is outside policy, ending the run
• A feature flag that routes new requests to a fallback and lets in-flight runs finish
• An operator control that halts the system immediately, independent of the model
• A per-run timeout that stops any task exceeding its expected duration

A client wants the agent to use customer data from another client's engagement to improve results. Response?

• Agree if the other client's contract permits data reuse for service improvement
• Refuse; client data isolation is a governance boundary, not an optimization trade-off
• Anonymize the records and proceed, since de-identified data is no longer client data
• Use the other client's data only to build the eval set, not in production prompts

How should personal data be handled in prompts where possible?

• Included in full so the model has context, with zero data retention configured on the API
• Encrypted inside the prompt text so the model processes it without seeing plaintext
• Duplicated in the system prompt and the user turn so that retrieval cannot miss a field
• Minimized or redacted, with only what the task needs

Which change to an agent requires a fresh risk review?

• Changing the model tier used for a read-only summarization step
• Rewriting a tool description to improve selection accuracy
• Raising the retry count on a read-only search tool from two to three
• Adding a tool that can send, delete, or pay

What is the purpose of a staged rollout for an agent change?

• Meeting the launch date while the remaining tests continue to run in parallel
• Collecting user feedback on the interface before committing to the design
• Reducing eval cost by testing on live traffic instead of maintaining a golden set
• Limiting blast radius while monitoring real behavior before full exposure

A CFO asks 'what does this agent cost us per month and what happens if it breaks?' The right answer includes:

• Cost per task and per month, failure modes, how they are detected, and time to recover
• Total tokens per month and the provider's published availability figures
• Model names, token counts per task, and the prompt caching rate that keeps the monthly bill down
• A live demo plus a commitment to SLA-backed uptime from the model provider

Which metric set belongs on an agent's monthly review?

• Token volume, cache hit rate, and API error rate by endpoint
• User count, sessions per user, and feature requests logged
• Completion rate, quality score, cost per task, incidents, and time to recover
• Prompt versions shipped, eval pass rate, and model release notes reviewed

A stakeholder wants a feature that the evals show is unreliable. Your move?

• Ship it behind a feature flag so that it can be turned off quickly if users begin to complain about it
• Show the eval results, state the risk, and propose a gated rollout or scope reduction
• Raise the eval threshold and ship once a stronger model tier clears it
• Decline the request and escalate to the sponsor with the eval numbers attached

How should prompt changes be communicated to stakeholders of a production system?

• Only when the change alters user-visible behavior, via the release notes
• Through the change ticket that the engineer files when editing the prompt
• As versioned releases with eval results and a rollback plan
• In the monthly review, summarized as the number of prompt edits made

Which event should trigger a lifecycle review of an agent system?

• A model upgrade, a change in tool capabilities, or a shift in the task's inputs
• A new release of the agent framework or SDK that the runtime uses, since dependencies change behavior
• A change in the on-call rotation or the team that owns the agent
• A quarter end, so that reviews line up with the budget cycle

A production agent's accuracy dropped after a tool provider changed its response format. What should have caught it?

• A larger eval set run before each of your own releases, covering more of the tool paths
• Contract tests on tool responses plus a scheduled eval that alerts on drift
• Pinning the provider's API version in the client so that format changes could not reach production
• Schema validation in the tool wrapper that coerces unexpected fields to safe defaults

Which design keeps a multi-tenant agent from leaking one customer's context into another's?

• Per-tenant context, memory, and tool credentials, with tenant id enforced in every tool call
• A system prompt instruction that names the current tenant and forbids any reference to others
• Random session ids per conversation so that contexts cannot be guessed across tenants
• A shared memory store partitioned by a tenant tag that the model includes in each query

A coordinator agent spends most of its tokens re-reading subagent transcripts. Fix?

• A larger context window on the coordinator so that full transcripts fit without truncation or summarization
• Prompt caching on each transcript so that repeated reads cost less per token
• Subagents return structured summaries; the coordinator keeps a ledger instead of transcripts
• Fewer, larger subagents so there are fewer transcripts for the coordinator to read

How should an architect decide between extended thinking and a second verification call?

• By task type: extended thinking for reasoning-heavy tasks, verification for extraction and formatting work
• Extended thinking by default, since it keeps the workflow to one call and one bill
• A verification call by default, since a second independent pass catches more errors
• By measuring: run both on the eval set and compare accuracy, cost, and latency for that task

Which practice makes a prompt library maintainable across a large team?

• A single senior owner for the whole library who personally approves every change
• Ownership per prompt, versioning, an eval per prompt, and a change process
• A shared document where any engineer can edit, with a weekly review of the accumulated diff
• A freeze on all prompts outside a monthly release window, with edits batched

What is the failure mode of caching aggressively in an agent with per-user personalization?

• Cached prefixes go stale, so users keep seeing policy text from before the last deployment
• The cache lifetime expires in the middle of a conversation, forcing a full uncached re-read of the prefix on the very next turn
• Personal content placed before the breakpoint defeats the cache, or a shared prefix built wrong leaks user data
• Cache writes cost more than reads, so low-repeat traffic pays a premium for no benefit

A client wants the agent to remember everything forever. Your design answer?

• Append every transcript to a vector index and retrieve the top matches into each prompt
• Store durable facts and decisions in a memory store with retention rules; load what a task needs
• Decline memory entirely and require the user to restate context at the start of each session
• Keep full transcripts in the prompt and rely on a long-context model tier to hold them

An MCP server is shared by several teams' agents and performance degrades under load. First architectural move?

• Add a second server instance and split the teams by name across the two
• Rate limits and per-client quotas on the server, plus caching of read-heavy resources
• Reduce the number of tools exposed so each call carries less schema overhead
• Switch transports from HTTP to stdio so that each agent gets its own dedicated server process

Which signal tells you a workflow step should become a model call rather than code?

• It is slow in code, and a small model tier would return an answer faster
• It runs on every request, so a model call's consistency matters more than its cost
• Its inputs are unstructured or ambiguous and rules keep failing on edge cases
• It is the step users complain about most, regardless of what the complaints say

How do you keep a 60-tool integration surface manageable for an agent?

• Merge related tools into a few generic ones with a mode parameter that selects the behavior
• Sort the tool list by usage and drop the bottom 40, since rarely used tools add noise
• Give the model all 60 with a routing table in the system prompt that maps each user intent to the right tool
• Partition tools by task into specialist subagents or dynamic tool loading, each with a small relevant set

What belongs in the audit record for an agent action that changed a customer's data?

• The final state of the record and the user id of the session owner
• Who or what triggered it, the inputs, the tool call and result, the approval if any, and a timestamp
• The full conversation transcript, stored against the ticket for that customer
• The before and after state of the record plus the model version and prompt version that produced the change

A regulator asks how the agent avoids discriminatory outcomes in a decision-support use case. What must exist?

• Use of the largest available model tier, which carries the strongest fairness training
• A disclaimer in the output stating that the recommendation is advisory only
• A system prompt instructing the model to ignore protected attributes when reasoning
• Documented bias evaluation, human review on decisions, and monitoring with defined thresholds

Which is the right response to discovering the agent logged full credit card numbers?

• Encrypt the log store at rest and add the finding to the next quarterly risk review
• Add redaction to the logger and let the existing records age out under the retention policy
• Restrict log access to the security team and shorten the retention window to seven days
• Stop the logging, purge the records, assess exposure, and add redaction with tests

What is a sensible policy for agent access to production databases?

• Full read access on the primary with writes blocked at the network layer
• A dedicated agent role that mirrors the application's service account so the agent can do what the app does
• No database access; the agent files tickets that a human executes by hand
• Reads through scoped views by default; writes through audited, gated operations

How should an architect treat a model's refusal behavior in a regulated workflow?

• As a signal to tune away with the system prompt so the workflow completes reliably
• As the primary control, documented in the compliance file as the enforcement mechanism
• As one safety layer among several, with policy enforced in code and review regardless
• As a reason to route the refused requests to a less restrictive model tier

What should a change-management process require before an agent's tool set expands?

• A code review of the tool wrapper and a passing run of the existing eval set
• A risk review of the new capabilities, updated gates, tests, and a staged rollout
• A load test of the new tool and a rate limit configured on the MCP server
• Approval from the tool's system owner and an updated tool description

Which incident-response element is specific to agent systems?

• The ability to halt autonomous actions immediately and replay decisions from logs
• A public status page updated while the incident is in progress
• An on-call rotation with paging tied to error rate and latency alerts on the agent service
• A blameless postmortem held within a fixed window after resolution

A client asks for a single number: 'how accurate is it?' What do you give them?

• A range spanning the best and worst eval runs from the past quarter
• Accuracy on a defined eval set with the date, the failure categories, and what happens on failure
• The eval pass rate as a single percentage, so the client has the number they asked for
• A comparison to human accuracy on the same task, drawn from the client's own quality assurance records for the past year

How do you set expectations for an agent's first month in production?

• A commitment to the eval accuracy figure as a production SLA from day one
• A soft launch to a friendly team with no metrics until usage has settled down
• Full autonomy on the low-risk tasks from day one and a promise to review the remaining tasks after 30 days
• A staged plan with metrics, human review levels, and the criteria for expanding autonomy

A stakeholder wants to skip evals to hit a launch date. Your position as the architect?

• Ship with production monitoring in place of evals, since live traffic is the more realistic test
• Hold the launch until the full eval suite passes, whatever that does to the date
• Agree, provided the stakeholder signs a written risk acceptance and the full evals run during the week after launch
• Explain the risk concretely, offer a reduced-scope launch that keeps the eval gate, and document the decision

Which artifact keeps a long-running agent engagement healthy across team changes?

• The chat history between the team and the client, archived and searchable
• The original proposal and statement of work, kept current through change orders
• A living runbook: architecture, gates, evals, known failure modes, and on-call procedures
• Recorded handover sessions from each departing engineer, stored with the project

When a model provider announces a deprecation, what is the lifecycle response?

• Switch to the successor in production at once, since the provider has already validated it
• Stay on the deprecated model until the retirement date to avoid churn, then switch
• Freeze the agent's scope and run both models in parallel for the foreseeable future
• Plan the migration: run evals on the successor, compare, canary, cut over, and update docs

How should success be reported to executives for an agent that handles support tickets?

• The number of tickets the agent touched, the average handling time, and the uptime of the service
• Resolution rate, customer satisfaction, cost per ticket, escalation rate, and incidents, trended
• Tokens per day, cache hit rate, and average latency per response
• Prompt versions shipped and the eval pass rate per release