Overview
What this is: Helping members collectively interpret, test, and guide the responsible use of AI based on real library materials and practices. Because AI-based approaches may operate across multiple dimensions of library practice, evaluation treats AI as an infrastructure, up-skilling, and readiness issue rather than a standalone technology or a simple replacement for current workflows. The program helps members keep pace with change while avoiding premature commitments to particular tools, vendors, or architectures.
CRL’s role is not to build production systems or develop technologies. Instead, CRL acts as a network-level intermediary, translating technical developments into practical guidance for governance and decision-making. CRL may host carefully scoped testing environments or sandboxes, especially where CRL collections and workflows are relevant, to support shared evaluation and learning rather than ongoing service delivery. CRL may also connect members with AI projects seeking content, screening requests against member criteria, and helping developers locate appropriate materials.
Intended outcomes / Steady state: CRL is a trusted venue where the community can collectively assess emerging AI methods before widespread adoption, and coordinate to influence the directions of interest for these technologies. Outputs include:
- Consensus guidance on responsible and effective AI use in library contexts
- Documented evidence about error patterns, staffing implications, and cost ranges
- Practical decision tools such as evaluation rubrics, procurement questions, data-preparation checklists, and exit criteria.
- Limited prototype implementations or evaluation environments for feasible approaches
Approaches that demonstrate durable value are incorporated into CRL’s own services, so that CRL remains usable, visible, and credible as AI-mediated research practices develop. Inconclusive or unsuccessful approaches are documented and retired, preserving institutional learning without ongoing cost. The goal is not to accelerate adoption for its own sake, but to improve collective judgment and coordinated adoption.
How it relates: Extends CRL’s role as a coordinator and broker of shared knowledge and services. Aligns with the strategic framework’s emphasis on resilience, coordinated stewardship of collections and content, and collective action across the membership. Reduces duplicated effort in evaluating AI and provides a clearer “voice of the many” in conversations with vendors, technologists, and research partners. Members benefit from shared evidence and guidance enabling each member to focus on specific local evaluation questions. For this role, CRL’s identity as a working library rather than a technology innovation organization is a strength: if something is credible and viable for CRL, it is broadly relevant and usable for research libraries of many types.
Problem or opportunity statement
AI-enabled methods are advancing faster than research libraries can reasonably assess them, and the capacity to experiment and understand these technologies is unevenly distributed across the community. While some institutions can pilot new approaches, many libraries lack the staff, infrastructure, or risk tolerance required to experiment directly with evolving tools whose long-term value and implications are uncertain.
At the same time, early experimentation often shapes broader professional practice. Decisions about metadata standards, discovery expectations, licensing terms, and data preparation methods tend to propagate through the ecosystem quickly. Libraries that move later may find themselves adapting to de facto standards and practices shaped by early adopters or vendor assumptions rather than collective professional priorities.
This creates a shared strategic risk. Moving too slowly risks marginalizing library collections and services in AI-mediated research environments. Moving too quickly risks accruing technical debt and locking institutions into tools, workflows, or vendor relationships that are difficult to reverse and may not align with long-term stewardship values. A shared intermediary helps bridge this divide: innovators gain a venue to evaluate and interpret results in a community context, while others gain visibility into emerging practices and a clearer path to adopt approaches that prove durable.
CRL is positioned to address this gap. As a trusted network organization serving a broad membership, CRL can act as an intermediary that concentrates shared questions, convenes expertise, and produces evidence-based guidance grounded in real collections and workflows. By organizing evaluation and interpretation at the network level, CRL can reduce duplicative effort, surface risks constructively, and help members make more effective decisions about when to adopt, adapt, or defer emerging AI approaches.
Practical Example(s)
Shared guidance and consensus documents
CRL convenes working groups from across the membership to develop shared guidance on emerging issues such as:
- Effective uses of AI in metadata creation and enhancement
- Guardrails for AI-mediated discovery interfaces
- Documentation and data preparation standards for AI-ready collections
- Differentiation of requirements for retrieval, other research use, and model training.
- Risk assessment and governance frameworks for AI adoption
Outputs include white papers, evaluation frameworks, and consensus guidance that institutions can use to inform local policy and procurement decisions.
Comparative evaluation using CRL collections
CRL coordinates limited evaluations of AI-enabled approaches using selected collections and workflows. These may include:
- AI-assisted metadata for different formats and collection types
- Retrieval-augmented discovery using CRL catalog data and digital collections
- Preparation of selected corpora for reliable use in AI-mediated research tools
The emphasis is showing where approaches perform well, where they introduce unacceptable errors, and what preparation or governance conditions are required for responsible use. Findings are documented in comparable formats, so institutions can assess feasibility, cost implications, and operational impacts before committing locally.
Shared experimentation environments
Where helpful, CRL may host bounded testing environments that allow members or partners to evaluate approaches using shared content. Examples might include:
- Small-scale sandboxes where members can test AI-assisted metadata tools, on their own collections or on CRL provided content
- Limited environments where innovators can experiment with CRL datasets under defined conditions
- Pilot implementations that demonstrate potential workflows without creating permanent infrastructure
These environments are limited in scope and temporary, designed to support shared learning rather than ongoing service provision. These environments do not have to be CRL run, necessarily; this could be a trial-version program where CRL adds value by screening offers against go/no-go criteria established by members to alleviate risks.
Content connector
CRL has experience and capabilities that could be converted into a useful role connecting the optimal content to the right AI development effort. Our experience vetting offers against model licenses, coupled to our capabilities in collections analysis and overarching goal to have strong awareness of where distinctive content of all kinds is held all play into this. For content-holders, CRL could serve as a filter, evaluating request for access to content against a rubric of required terms. For content-seekers, CRL could serve as a discovery mechanism to reduce the work of locating relevant content and based on standard terms, providing better assurance of a successful match.
Implementation and Development
Implementation Requirements
Staffing, organizational capacity, and professional development
The program requires dedicated staff time for coordination, design and documentation of processes, and synthesis of findings. Staff responsibilities include convening working groups, preparing collections and workflows for evaluation, managing short-term projects, and translating technical outcomes into practical guidance for member institutions. The emphasis is on strengthening CRL’s existing competencies and up-skilling staff—metadata expertise, collections analysis, discovery practices, and program coordination—rather than building internal product development capacity.
Access to external expertise
CRL will rely on targeted, time-limited engagement with specialists from member institutions, iSchools, and partner organizations. These contributors provide depth in specific domains (e.g., retrieval-augmented discovery, OCR improvement, metadata normalization) while CRL retains responsibility for coordination and interpretation. This could include or augment CRL’s planned research fellowship program.
Compute, storage, and technical services
Modest, flexible computing and storage resources are required to support benchmarking, testing, and comparative evaluation. Infrastructure is selected to be vendor-agnostic and disposable by design. Go/no-go criteria include the ability to sunset environments cleanly once evaluation goals are met, preserving documentation and results without ongoing operational cost. This would include cash funding.
Development Process
Phase 1: Framing and Preparation: Guidance from existing CRL committees, supplemented by ad hoc groups for this initiative, select use-cases aligned with core services (e.g. metadata, discovery, or collections analysis) and determine appropriate content to include and technologies to evaluate. Preparation focuses on normalization, rights review, and documentation so that investments add long-term value regardless of experimental outcomes. Evaluation criteria, success measures, and exit conditions are defined at the outset, including governance and stewardship considerations.
Phase 2: Experimentation and evaluation: Experiments are conducted within clearly bounded scopes and timelines. Evaluation emphasizes comparative performance, error profiles, cost and staffing implications, and alignment with existing professional practice. Work is designed for reproducibility and interpretation, not optimization or scale. Throughout this phase, findings are documented in a form suitable for reuse, reassessment, and governance review.
Phase 3: Synthesis, integration, and dissemination: All evaluated approaches are documented with comparable rigor. When evidence supports adoption, CRL translates results into guidance and reference workflows. When evidence indicates limitations or misalignment with current needs, documentation focuses on what was learned, what remains unresolved, and what future conditions would warrant reconsideration—without creating ongoing maintenance obligations. Approaches demonstrating clear, durable value may be incorporated into CRL services. Others are formally concluded, preserving institutional learning while maintaining program focus and sustainability.
Governance and Oversight
Across governance and oversight mechanisms, this program should aim to upskill or enhance existing structures in CRL wherever possible. Although the specific technologies involved in this program may or may not be adopted, the program as a whole is intended to make sure CRL is organizationally ready for the AI era.
Program ownership and accountability
The program operates within CRL, with primary responsibility housed in Discovery, Access, and Technology and advisory input from Collections, Acquisitions, and Licensing on stewardship and policy considerations. Program leadership is accountable for scope control, alignment with strategic priorities, and transparent reporting of outcomes.
Advisory and working groups
Time-limited advisory and working groups drawn from the membership provide input on use-case selection, evaluation criteria, and interpretation of results. These groups guide direction and context without assuming operational responsibility, ensuring broad participation while preserving CRL’s ability to manage execution and governance.
Policy and risk alignment
Policy and risk assessment—e.g. data use, licensing implications, and ethical constraints—can draw on relevant licensing working groups in DCAL, technology groups in DCAT, CRL counsel, and the CSC.
Reporting and transparency
CRL provides regular reporting to members and the Board on active evaluations, completed projects, concluded efforts, and resulting guidance. Reporting includes technical performance but emphasizes lessons learned, decision rationale, and implications for member practice and CRL services, as well, to support informed decision-making and development around AI in libraries.