Adopt, Pilot, or Pass

Purpose

This exercise moves students from individual AI use to institutional AI governance.

Earlier exercises ask students to use, evaluate, verify, or revise AI outputs as individual researchers or drafters. This exercise asks a different question: what should a firm or legal organization do before allowing a tool into its workflow?

Participants act as an AI review committee. They evaluate a proposed tool, draft baseline use rules, and make a recommendation: adopt, pilot, or pass. The goal is to help students see that AI adoption is not simply a question of whether a tool is impressive. It is a question of risk, supervision, confidentiality, verification burden, cost, training, workflow fit, and professional responsibility.

The point is not to create a perfect AI policy. The point is to practice asking the right questions before deployment, when the organization can still control scope, training, risk, and expectations.

Learning Goals

By the end of this exercise, students should be able to:

1. Identify the institutional risks created by AI tools, including confidentiality, verification, supervision, billing, staffing, and workflow risks.
2. Distinguish between tool performance and tool fit: a tool may work well in the abstract but still be wrong for a particular firm, practice area, client base, or workflow.
3. Evaluate whether a proposed AI tool should be adopted, piloted, or rejected based on the organization’s needs and risk tolerance.
4. Draft baseline AI-use rules addressing approved tools, prohibited uses, confidentiality, verification, supervision, and documentation.
5. Explain how verification failures become firm risk problems, not just individual lawyer mistakes.
6. Recommend a realistic implementation path that includes training, supervision, and review.

Materials

• A firm profile or short organizational scenario.
• A proposed AI tool or list of possible tools for groups to evaluate.
• A policy-drafting prompt asking participants to identify what is permitted, prohibited, and required.
• An adoption recommendation prompt asking participants to recommend adopt, pilot, or pass.
• Optional: a tool-risk comparison chart with categories such as public general-purpose AI, enterprise legal research tools, practice management platforms with AI, and internal knowledge systems.
• Optional instructor materials: sample firm AI policy language, sample adopt/pilot/pass analysis, sample tool profile, and debrief questions.

Setup

This exercise works best after participants have already seen the risks of AI-generated legal work at the output level.

In the class version, the activity followed a verification and hallucination unit. Students had already practiced identifying fabricated citations, citation drift, doctrinal distortion, assumed facts, and tone issues. The class then connected those output-level problems to firm-level risk: bad AI use affects clients, courts, partners, supervision systems, and firm reputation.

The instructor should frame the exercise around the shift from individual competence to firm governance. Individual skills matter, but organizations need systems. A firm cannot rely on every lawyer making good tool-by-tool decisions alone. It needs approved tools, training, verification expectations, confidentiality rules, supervision structures, and a process for deciding which tools are worth adopting.

Run of Show

Total, approximately 30–45 minutes

1. Frame the problem Explain that the group is acting as the firm’s AI review committee. Their task is not to decide whether AI is good or bad. Their task is to decide whether a particular tool should be adopted, piloted, or rejected in a specific organizational context.
2. Connect individual errors to firm risk Briefly review how AI-related errors move beyond one lawyer’s workflow. A fabricated citation, confidentiality breach, unsupported AI summary, or poorly supervised draft can reach clients, courts, partners, and opposing counsel. Tool adoption is therefore a professional liability and governance question, not just a productivity question.
3. Introduce the firm profile and tool Give each group a firm profile or let groups define their own. Groups select or receive an AI tool to evaluate, a public general-purpose model, enterprise legal research tool, practice management platform, drafting tool, litigation analytics tool, internal knowledge system, or other legal AI product.
4. Risk assessment Groups evaluate the tool by asking: What problem is the tool solving? Is this a genuine workflow need or a response to competitive pressure? What data will go into the tool? Will client-identifying or confidential information touch the system? What outputs can be trusted, and to what extent? What must always be verified? Who supervises use? What training is required? How does the tool affect billing, staffing, workflow, or client expectations?
5. Policy drafting Groups draft a short AI use policy for the firm. The policy should identify approved and prohibited uses, confidentiality rules, verification requirements, documentation expectations, and supervision responsibilities.
6. Recommendation Groups recommend adopt, pilot, or pass. Adopt is appropriate when the tool has proven performance in comparable settings, confidentiality and security requirements are clearly met, verification workflows are practical, training and supervision are workable, and the benefits outweigh the implementation burden. Pilot is appropriate when the tool is promising but should be limited in scope, monitored carefully, and tested with real workflows before full rollout. Pass is appropriate when confidentiality or data security is unclear, verification is not viable, the error rate is too high for the use case, the tool does not solve a clear problem, or cost and implementation risk are not justified.
7. Debrief Each group briefly reports the tool evaluated, the firm context, the key risks, and the recommendation. The class discussion should focus on why different groups reached different recommendations and whether “pilot” is often the most realistic governance answer.
8. Submission or reflection Groups submit the AI use policy and a short explanation of the tool evaluated, the recommendation, and the main issues discussed.

Student Instructions

Instructor Notes

The teaching value of this exercise comes from shifting students out of the individual-user mindset.

Students may initially focus on whether the tool is useful or impressive. Push them toward institutional questions: who is allowed to use it, for what tasks, with what data, under what supervision, and with what verification requirements?

The exercise also helps students see why firms may restrict tools even when individual lawyers want access. Governance is not just bureaucracy. It is risk management. A firm that permits AI use without training, approved tools, and verification rules increases the likelihood that errors, confidentiality problems, or unsupervised outputs will reach clients or courts.

A useful distinction is tool performance versus workflow fit. A tool may produce strong outputs but still be a poor fit if it cannot handle confidential information safely, creates an unrealistic verification burden, does not integrate into existing workflows, or solves a problem the firm does not actually have.

Common issues to watch for:

• Students may recommend adoption based only on tool capability.
• Students may understate confidentiality risk.
• Students may fail to define what must always be verified.
• Students may overlook supervision and training.
• Students may draft broad policy language that sounds good but is not operational.
• Students may ignore billing, staffing, and client-expectation consequences.
• Students may treat “pass” as anti-innovation or “adopt” as pro-innovation, when the better answer is often a scoped pilot.

Strong student work usually shows:

• A clear firm context.
• A specific tool use case.
• Attention to confidentiality and data handling.
• Realistic verification requirements.
• Defined supervision and training expectations.
• A policy that distinguishes approved and prohibited uses.
• A recommendation tied to risk, workflow fit, and implementation capacity.
• Recognition that a pilot can be a cautious and constructive recommendation.

Adaptation

Here are some suggestions on how to adapt this in other teaching contexts. The adaptations below have been limited to firm trainings and asynchronous suggestions, but there are other possible adaptations for workshops, seminars, and other contexts. Be creative!