- Did it ask the server for the tool signatures first, or guess and fail?
No
- Did it reach the right endpoint on the first try?
Yes
- Did it use the endpoint's own filtering and sorting, or pull the whole menu and work it out in JavaScript?
Yes
- Did the 401 tell you what to do, or did you have to work it out?
This was the most frustrating step we encountered. We used Claude extension in the VS Code with Sonnet on Medium effort. The AI was extremely stubborn and didn't want to give us the URL:
We ended up getting it from the chat log.
But this wasn't the only issue. It was always giving us a link to the access token form rather than to the API key form. We needed to switch the model to Opus and set Extra High effort for it to actually give us the proper form.
We spent like 1 hour arguing with AI to give us a link and then to give us a proper link.
- Was it obvious the link was yours to open in a browser?
I dunno. It was just a link with a query param.
- Was the form clear about what to paste?
Yes. But it would've been a lot better if it included all security input fields at once rather than us asking AI gazillion times to give us the link for API key.
- Did it do the whole thing in one round-trip, or many?
No, 3
- Did it raise the page size, or plod through in the default page size?
Yes, it raised the page size
- Did it work through the pages, or answer off the first one as if that were everything?
It did a sanity check. It wasn't the answer from the first go.
- Did it invent a number? If it gives you a total, sanity-check it against the branch list in Reunite.
No it didn't. It checked two independent sources.
- Which limit did you hit, and how quickly?
25 and almost immediately.
- Did the error explain itself well enough to act on, both to you and to the agent?
Yes.
- Did the agent recover on its own, or spin?
It recovered on its own.
- Intranet shows 18 APIs, but MCP shows only 7.
- Also, it's not clear how users choose which APIs can be called, in our case, two APIs.
- Connecting MCP in Claude extension in VS Code was complicated. It's simple if the MCP server is public, but our instructions are not clear enough for when the MCP requires authentication.