We’re at an interesting but confusing point in time.
AI agents are good enough to be useful but setting them up can be frustratingly hard.
And setting them up to be both useful and secure? Good luck with that.
I’ve been trying to figure out how best to set up my own AI agents and still haven’t found an approach that I’m entirely happy with.
A lot of what you need comes down to what we might call ‘agent orchestration’ and ‘agent harnesses’ (if we define that a little more broadly than is becoming common now).
The growing jumble of possibilities includes agent harnesses such as OpenClaw, Claude Code, Claude Cowork and Hermes as well as more orchestration-focussed products such as n8n, CrewAI, Zapier and Paperclip.
Here are some of the aspects I currently see as important in any agent orchestration / harness setup:
1. Triggers
Chat interfaces are great but I don’t want to have to trigger my agents manually every time I want them to do something. I want the platform to support, at least, the following different triggers:
- Scheduled: I want some actions to happen on a regular basis, e.g. every day or every week.
- On receipt of an inbound event: e.g. receipt of an email or WhatsApp message (it should ideally be easy to hook up different inbound channels).
- Manually: Sometimes I’m doing something ad hoc or developing a new agent or workflow. In those cases, I do still want to be able to trigger things manually.
2. Availability and Responsiveness
- 24×7 availability: I want the agents to be able to respond to me and/or other triggers 24×7 and to not be reliant on my laptop being on.
- Fast responses: And when I’m interacting with it (e.g. via a chat interface) I want it to respond quickly.
3. Cost Effectiveness
- Low marginal cost per workflow: The cost overhead of each extra agentic workflow I set up should be little more than the extra tokens it uses.
- Subscription-friendly: Ideally, I should be able to take advantage of the cheap tokens available through subscriptions such as Claude Pro/Max and ChatGPT Plus/Pro.
- Cost visibility: It should be easy for me to see how much I’m spending.
- API cost efficiency: The platform should use LLM APIs effectively to avoid unnecessary costs. For example:
- Model choice: It should be possible to use cheaper models where they are good enough for a task.
- Cache-friendliness: It should be possible to take advantage of the cost benefits of caching.
- Token efficiency: The platform should be efficient in the number of tokens it uses for tasks.
4. Security
The platform should help in maintaining a reasonable level of security. There are many aspects to this, but some that I think may be particularly important:
- Minimise agents’ access to secrets irrelevant to their task: Agents shouldn’t, for example, be able to scan your laptop and dig out important credentials from .env or .zshrc files or use Gmail to read all the emails you’ve ever sent or received. Ideally, they shouldn’t have access to any 3rd party credentials at all and should only have access to the specific information they need to perform their tasks well.
- Platform security: It should be easy to keep the platform itself up to date and free from known security issues. For example, I don’t want to worry that my agent harness is running on a server that hasn’t had security patches applied for the last year. And it’s no use keeping important credentials away from agents if the place we’re keeping them is insecure.
5. Support for Popular Patterns
Some patterns have emerged around AI agents that people seem to be finding useful. The platform should provide good support for those patterns.
For example:
- Filesystem/shell use
- Skills
- MCP
- Subagents
- Code execution
6. Flexibility
The ecosystem of models, libraries, agent harnesses, etc. is messy and evolving very quickly.
The platform should make it easy to try existing things out and flexible enough to work with whatever new things that emerge.
7. Configuration Management
As far as possible I want to be able to easily understand my current setup: how agents are configured (perhaps including important aspects of their ‘memories’), what triggers are in place, etc.
And, ideally, I want good ways to track that configuration over time; perhaps using software-style version control that allows me to see what’s changed and undo changes that are proving problematic.
8. Simplicity
Ideally, the platform should be simple to use so that I don’t need to spend a lot of time trying to understand it, troubleshooting it, etc.
9. Good Long-Term Outlook
Whatever platform I use, I want it to have a good chance of being around and well-supported for the long term to minimse the chances I’ll have the pain of switching over to another platform in six months or a year’s time.
Closing Thoughts
This list reflects my rough current thoughts on what’s important in an agent orchestration platform.
I haven’t tried to be comprehensive with my list. In particular, I haven’t tried to cover things that I’m sure would be important in a more corporate context.
That said, I hope these thoughts are helpful to you if you’re also trying to figure out how to set up your AI agents.
I’d love to connect with more people working with AI agents. If that’s you, you can find me on X here.
Leave a Reply