KB Sections:
Overview
AI Agents at UW-Green Bay are custom-built assistants that provide automated answers and guidance using approved University information sources. To ensure accuracy and security, all AI Agent requests are a partnership between GBIT (Information Technology) and the requesting department’s Subject Matter Expert (SME).
Each AI Agent goes through phases of setup, content integration, testing, and deployment. IT manages infrastructure and system performance, while SMEs guide content accuracy and user testing.
IT Responsibilities
Setup and Configuration
GBIT is responsible for creating, configuring, and securing the AI Agent within UW-Green Bay’s approved AI infrastructure. This includes:
- Provisioning the AI Agent environment and ensuring GBIT retains ownership and administrative control of the system.
- Implementing access controls and data compliance safeguards.
- Assisting with deployment of the AI Agent to approved university channels.
Training and Testing Environment
IT supports the agent’s initial training and testing environment by configuring workflows, logging, and system integrations that allow for safe experimentation. This environment is used for:
- Testing how the AI Agent generates and structures answers.
- Running performance and error checks before deployment.
- Ensuring the model aligns with UW-Green Bay’s data and acceptable use policies.
Answer Generation and System Oversight
IT verifies that responses are being generated based on approved, secure data sources. The team will ensure that answers are drawn from verified repositories (ex: UWGB websites, official KBs, and University policies) and that technical updates are made as needed.
SME Responsibilities
User Acceptance Testing (UAT)
Subject Matter Experts validate the AI Agent’s content accuracy and usefulness through structured User Acceptance Testing. SMEs should test for clarity, correctness, and tone, providing feedback directly to IT.
Content Ownership and Quality
SMEs are the content owners responsible for providing, maintaining, and updating the information used by the AI Agent. This includes:
- Submitting reviewed and approved content for inclusion.
- Ensuring information reflects current policies and procedures.
- Collaborating with IT to fix or improve inaccurate or unclear responses.
Error Reporting and Updates
When issues arise (such as incorrect answers, missing data, or inconsistent tone), SMEs should report these to GBIT for investigation and correction through the ticketing process.
Collaboration and Testing Process
AI Agent development is an iterative process that depends on close collaboration. Typical workflow:
- Kickoff Meeting: Define goals, audiences, and core questions.
- Setup: IT configures the environment and connects data sources.
- Training and Review: SMEs provide validated content and review test outputs.
- User Acceptance Testing: SMEs test responses, IT adjusts configurations.
- Launch and Monitor: After approval, the agent goes live with joint oversight.
Ongoing Monitoring and Improvement
After deployment, both GBIT and SMEs share responsibility for monitoring performance and accuracy. Regular reviews should check for:
- Broken links or outdated references.
- Inaccurate or incomplete responses.
- Emerging topics requiring new content.
GBIT manages the technical performance and logs, while SMEs ensure continued content validity. Together, they sustain an effective, compliant, and user-friendly AI Agent experience for campus users.
Note: For technical assistance, contact the GBIT Service Desk.