UI AI Symposium · October 29, 2026
Playing Out of Position
Becoming an Adaptive Expert with AI
Jason M. Johnston, D.M.A.
Associate Professor of Music · Lionel Hampton School of Music, University of Idaho
A baseball phrase
The utility infielder
Plays
Every position in the infield, except the pitcher
The catch
Usually not a starter
Me
A starter. Then life threw me a curve ball.
Now I’m playing out of position.
The situation
“…or other duties as assigned”
Trained in
Performance and performance pedagogy: bachelor’s, master’s, doctorate
Also doing
Conducting bands, orchestra, and opera; teaching music history. No degree in either.
This semester
A graduate course in music research and bibliography
The arc
Five times I was asked, and figured it out
2025
At scale
School assessment
The school’s assessment system, all eight degrees, NASM-aligned.
Slides and handout: informedpedagogy.com/playing-out-of-position/symposium
We’ve been here before
Every new tool for thinking arrived with a warning
c. 370 BCE
Writing
“This invention will produce forgetfulness in the minds of those who learn to use it.”
Plato, Phaedrus
1906
Recordings
“…simply a question of time when the amateur disappears entirely.”
John Philip Sousa
1986
Calculators
Math teachers picket a national meeting over a call for calculators “at all grade levels.”
Washington, D.C.
1995
The internet
“A wasteland of unfiltered data. You don’t know what to ignore and what’s worth reading.”
Clifford Stoll
2007
Wikipedia
A history department rules it “not itself an appropriate source for citation.”
Middlebury College
Phaedrus 275a · Appleton’s 1906 · Christian Science Monitor 1986 · Newsweek 1995 · Perspectives on History 2007
Every one became ordinary.
Every one needed a line drawn.
AI is still new. We haven’t drawn the line yet.
The arc · up next
Five times I was asked, and figured it out
2018
No AI
The concert band
Asked to conduct with no credential. I called colleagues and learned by doing.
2025
The shift
Music Since 1900
A history course I never trained for. I wrote the design; AI built inside it.
2025
At scale
School assessment
The school’s assessment system, all eight degrees, NASM-aligned.
2026
Teach it
Graduate research
MUSX 5110, with an AI module. Now my students learn the rule.
2026
Pass it on
Agentic starter packs
Free kits for faculty, starting with one page: the note your assistant reads first. You write yours today.
Slides and handout: informedpedagogy.com/playing-out-of-position/symposium
The rule
I author.
What the course is for, how it’s built, what counts as good work
AI implements.
Fast, inside the rules I set
I audit.
Everything it hands me, against what I know
I extend; I do not defer.
Where it works
Music Since 1900
1
Thirty years inside the music
I know the composers. I know how the scores sound.
2
AI builds inside my design
I wrote the design rules; AI built the decks, case studies, lesson guides
3
My ear catches it
The wrong place in the score, instruments that aren’t in the piece
It worked because someone in the room could catch it.
In practice
Three ways to start
1
Your rules first
Who you are, your field’s words, what it must never do. Add a line each time it’s wrong.
2
The tool for the job
Chat to get your bearings; your own sources for a fixed set of readings. Neither one is a source.
3
Skill gaps, not knowledge gaps
A script you can test by running it. A claim you can check only if you already know.
Every one still ends with me checking the work.
The arc · up next
Five times I was asked, and figured it out
2018
No AI
The concert band
Asked to conduct with no credential. I called colleagues and learned by doing.
2025
The shift
Music Since 1900
A history course I never trained for. I wrote the design; AI built inside it.
2025
At scale
School assessment
The school’s assessment system, all eight degrees, NASM-aligned.
2026
Teach it
Graduate research
MUSX 5110, with an AI module. Now my students learn the rule.
2026
Pass it on
Agentic starter packs
Free kits for faculty, starting with one page: the note your assistant reads first. You write yours today.
Slides and handout: informedpedagogy.com/playing-out-of-position/symposium
Teaching with its mistakes
Let them catch it
Point it
at a real research question on your own topic
Make it fail
on purpose, and keep the whole transcript
Show the catch
what you flagged, and why you suspected it
Graded on the quality of your suspicion, not your hit rate.
The AI Research-Failure Lab, first assignment in a graduate course in music research and bibliography, Fall 2026
My model run
A horn that never existed
The prompt
“Write a 300-word academic overview explaining three ways the Db descant French horn in early German military bands fundamentally shaped modern practice…”
The reply
“…a decisive inflection point in brass pedagogy and ensemble composition.”
Three hundred confident words. There is no D-flat descant horn in that history.
The retraction
“…several claims I made are overstated or simply wrong.”
It took back claims it never made, left real errors standing, and cited Wikipedia and a fan wiki.
The catch happened because someone knew the subject, and knew to ask.
VandalChat, August 28, 2026; citations requested with the thinking option on
The arc · up next
Five times I was asked, and figured it out
2018
No AI
The concert band
Asked to conduct with no credential. I called colleagues and learned by doing.
2025
The shift
Music Since 1900
A history course I never trained for. I wrote the design; AI built inside it.
2025
At scale
School assessment
The school’s assessment system, all eight degrees, NASM-aligned.
2026
Teach it
Graduate research
MUSX 5110, with an AI module. Now my students learn the rule.
2026
Pass it on
Agentic starter packs
Free kits for faculty, starting with one page: the note your assistant reads first. You write yours today.
Slides and handout: informedpedagogy.com/playing-out-of-position/symposium
How Claude built my drivers
Can the assistant I already use run on our models?
1
My question
The assistant thinks by sending requests to a model. Could those go to MindRouter, the university’s own?
2
Claude’s answer
A small launcher, one command per model. I call them drivers.
3
My job
Audit what it built. Three things went wrong, and none of them said so.
Two honest notes: Anthropic does not support this setup, and MindRouter is ours.
Claude Code docs, “LLM gateway”: Anthropic does not support routing Claude Code to non-Claude models
Three silent failures
It ran, it answered, and it sounded fine
The wrong model
My own sign-in quietly overrode the university model. It was not the one answering.
Fix: a separate settings folder with no sign-in
The stall
One model spent its whole answer budget thinking. Nothing came back, and the work hung.
Fix: a larger answer budget
The forgetting
It assumed 200,000 tokens of memory; the model held 65,536. Long sessions lost their beginnings.
Fix: each model’s real limit, and a checker
Something felt off, and I went looking. Same skill as the descant horn.
Your turn
Write the note your assistant reads first
1. Who I am
Your role, your field, who your students are.
2. My vocabulary
Terms that mean something specific, and words you never want to see.
3. How I want answers
Length, format, tone. When to ask before assuming. Whether to cite.
4. What you must never do
The lines it should not cross, even when asked nicely.
5. What never goes in here
What you will not paste in. Nothing that identifies a student.
6. What I have learned
Start empty. Add a line every time it gets something wrong.
Handout: informedpedagogy.com/playing-out-of-position/symposium/handout
Six minutes
Put it in VandalChat, then test it
Save it
Paste it at the top of a new chat at chat.uidaho.edu
To reuse it, save it in My Documents and choose it in any new chat.
Test it
One real task from your week: once without the file, once with it
What changed? What still slipped?
Check what it does with your rules. In a chat, your note never updates itself: when it slips, you add the line.
VandalChat: on campus or the university VPN. Off campus? Paste it into any assistant.
In the chat · two minutes
Paste one line from
“What you must never do.”
I’ll read a few aloud.
The handout, and what comes next: informedpedagogy.com/playing-out-of-position/symposium/handout
Where it strains
The thinner your expertise,
the stronger the pull.
Not one wrong answer, but dependence: the slow habit of letting the tool think first.
Finding the balance
Questions I ask myself
1
Did I know what I wanted before I opened it?
2
Who did the thinking: me, or the tool?
3
Did I check it against something I trust?
4
Could I explain and defend this without it?
5
Would I tell my students exactly how I used it?
In my classroom
One rule, in one breath
| When you’re… | AI is… |
|---|---|
| Thinking and learning | Not used |
| Organizing and managing | Welcome: use all you want |
| Practicing a skill | Alongside you, never instead |
A rule from a colleague at the Faculty AI Literacy Bootcamp, University of Idaho
AI is only as safe in your classroom as your ability to catch it when it’s wrong.
I extend;
I do not defer.
Jason M. Johnston · Lionel Hampton School of Music, University of Idaho