The colleague needs boundaries

The personal AI I find interesting would remember why a task matters. It would know which decision remains open, which piece of work depends on it and what information would help move it forward. It would also know when to leave me alone. That last capability may deserve more engineering attention than it usually receives.

Calling such a system a colleague is a useful working metaphor. It encourages us to think about continuity, delegated responsibility and the quality of a handover. It should not lead us to assume human understanding, consciousness or a human relationship where those things have not been established.

The practical ambition is straightforward: less repeated explanation, better preparation and fewer important things falling between conversations. A system that remembers context could make a short instruction useful because the surrounding purpose is already available. The difficulty is deciding what it should remember, how confidently it should interpret it and what that knowledge permits it to do.

Memory is especially easy to romanticise. We imagine perfect recall as an uncomplicated advantage. In practice, a remembered detail can be outdated, inferred incorrectly or relevant only to one particular situation. Someone choosing an early flight once has not necessarily expressed a permanent enthusiasm for airports before dawn.

A useful memory should carry some account of where it came from and when it applied. The W3C’s work on provenance provides a formal vocabulary for tracking the origins of information. Applied more simply to personal AI, the principle is that an explicit instruction, a tentative inference and an old preference should remain distinguishable. Otherwise, a helpful system can become remarkably consistent at repeating its first misunderstanding.

I would want correction to be easy. Tell the assistant that a preference has changed and the old assumption should stop governing new decisions. Ask why it believes something and it should be able to identify the relevant evidence. Ask it to forget something and the product should explain clearly what deletion covers, including any limits. Trust depends partly on whether the person can inspect and change the picture being held of them.

Context also needs boundaries. An assistant may understand why a difficult meeting matters without being entitled to share that reason with everyone attending. Access to information and permission to disclose it are separate decisions. A system that can read a calendar should not infer unlimited authority over the relationships represented there.

Security already has a useful discipline for this. NIST’s zero-trust architecture rejects implicit trust based merely on where a user or system sits, and focuses on protecting resources through explicit access decisions. Personal assistants need similarly deliberate limits: access suited to the task, clear authority for consequential actions and a reliable way to withdraw it.

That need not mean asking for approval every time the system rearranges its own notes. Good delegation distinguishes preparation from commitment. Researching options, organising evidence and drafting a response can often proceed within an agreed scope. Sending the response, spending money or sharing private information crosses a different boundary. The user should be able to understand those boundaries without reading a security manual.

Reliability needs the same practical treatment. NIST’s AI Risk Management Framework places risk management across the use and development of AI systems. In a personal assistant, I would translate that into ordinary questions. What could go wrong here? How would we notice? Can the action be reversed? Who remains responsible for checking the outcome?

The best answer will differ between tasks. A mistaken restaurant suggestion is inconvenient. A mistaken statement sent to a customer can have wider consequences. The assistant’s confidence should never be the only factor determining how much freedom it receives.

There is also a quieter measure of quality: whether delegation actually reduces the user’s work. An assistant that produces a constant stream of updates can become another inbox. One that acts silently beyond its authority creates a different problem. Useful judgement sits between those failures, surfacing decisions when needed and carrying out agreed preparation without demanding applause.

I want personal AI to make it easier to remain attentive and accountable. Remember the context, prepare the next step, explain uncertainty and respect the boundary. That would be a substantial improvement on software that merely waits for the next instruction, while leaving the person firmly in charge of what happens next.