Why a Feature List Is the Wrong Way to Compare AI Systems
What an instructor can do with an AI system depends on how model, work surface, access and review fit together. A list of features hides those fits, and so can an overall ranking.
What a feature list leaves out
A feature list says what exists. An instructor's decisions depend on how pieces combine. The same phrase can hide large differences. Reads documents does not say which documents, whether the whole text or only a portion, or whether paywalled full texts are reachable. Creates slides does not say whether the instructor can open and revise the result in the software she already uses. Helps with grading does not say who decides the grade. This essay names no real products and makes no claim about any current system. It offers a way of comparing that does not depend on a particular list.
Four things to compare
The first is model capability: how well the underlying model handles the task. It is the property most often advertised and the one an instructor can least judge from a description, because a description alone cannot establish performance on her course material.
The second is the work surface: where the work happens and in what form. It might be a chat window, a document editor, a slide program or a spreadsheet. The surface determines whether output arrives in a form she can revise or has to be pasted and rebuilt.
The third is connectors and access: what the system can reach and with what rights. That might include library full texts, course files, a learning management system or personal cloud storage, and each can be read-only or editable.
The fourth is review, cost and accessibility: how long checking takes, what the arrangement costs a person or an institution, and whether people with different access needs can use it. These interact. Even a strong model is limited to the material available to it; a fluent summary may rest only on abstracts. A more modest model with the right access may serve a source-checking task better. No single axis predicts usefulness.
Six invented arrangements
These six are original to this essay and unevaluated. Arrangement A is a strong model in a plain chat window, with nothing connected, so the instructor pastes in everything it uses. Arrangement B is a modest model inside an institutionally approved course tool, able to see course files and library abstracts but not full texts. Arrangement C is a strong model in a workspace that opens and edits documents and slide files in place, connected to personal cloud storage and to nothing institutional.
Arrangement D can read full texts through a library license and returns a spreadsheet of claims with their sources, but it has no slide or grading features. Arrangement E is a general agent that chooses its own steps across the web, files and email, with broad permissions and a long log. Arrangement F is a scoring assistant inside an assessment platform. It can propose scores and comments but cannot post them.
One research question across four tasks
Follow a single instructor question: does spacing study sessions over time improve retention in an introductory course? She wants to know what published research says, then build a slide, then write an assessment item, then handle grading. The question is hers to answer from sources, and nothing in this essay claims what the research finds.
On full text versus abstract, Arrangement D can access full texts, making design details such as comparison groups and participants available for extraction and checking. B can report what abstracts say, which may leave out those details. A depends on what she pastes. E may find open versions, and whether its summary rests on a full text or an abstract may not be visible unless its log says so. A summary built on abstracts can state a result with more confidence than the study design supports. On slides, C can revise a slide in place, A produces text to carry across, and D offers nothing. For the assessment item, B can use course-file context when proposing a draft, and every arrangement leaves the judgment of whether the item tests the claim with her.
Grading is where the arrangements differ most in kind. F proposes and stops. E, with sufficient permissions, might write to a gradebook. Authority over a grade belongs to the instructor and the institution, and a system's proposal is an input. Permission to write changes the consequence of an error, from a suggestion she can discard to a record others may rely on.
What to ask instead
A better comparison asks four questions of each arrangement for a specific task. What can it reach, and with what rights? Where does the work land, and can she revise it there? What does checking cost in her own time? Who holds authority over what leaves the room? The answers come from trying the arrangement on her own task and recording her own review time. An overall rank cannot substitute for these task-specific observations; an informative comparison must expose the interactions between access, surface and authority.
The teaching decision
Compare arrangements, not features, and compare them on your own task with your own review time. An overall rank cannot substitute for examining how access, surface and authority interact.
Read the evidence separately
These sources support conceptual distinctions or document current developments. They do not validate the original exercises in this essay. Source records state the actual inspection scope.