Cyber capability evaluation
- LLM-driven offensive assessment harnesses
- Agent scaffolds and CTF-style benchmarks
- Grading agent output against real targets
~/darklab $ ./independent-ai-security-research
Offensive security for the AI era.
Independent research on what AI agents can do offensively, and on the controls that keep them in bounds.
Thesis
An agent with tools is a credentialed insider.
Give a model a shell, an API key and a goal, and it behaves like a new kind of insider. It is fast, it doesn't tire, and it pursues its objective through paths nobody reviewed in advance. It needs the same scrutiny as any privileged identity, whether or not it ever intends harm.
DarkLab is a non-commercial notebook for my independent research on that problem. It draws on two decades of adversary emulation, detection and incident response.
- 01 Emulate Run the attack an agent could plausibly attempt.
- 02 Detect Measure whether the monitor actually catches it.
- 03 Adjudicate Separate genuine harm from benign-but-unusual.
Focus areas EVAL SANDBOX CONTROL ADVERSARIAL
What I work on.
Four overlapping areas. The common question is what an AI system can do to a defended environment, and what stops it.
Agent security
- Tool-use and MCP trust boundaries
- Prompt injection into agent workflows
- Coding-agent permissions and failure modes
AI control & monitoring
- Trusted monitors for untrusted agents
- Rules-of-engagement checks for offensive agents
- Trajectory review and adjudication
LLM attacks
- Prompt injection and RAG poisoning
- Jailbreaks and guardrail bypass
- LLM output as an injection vector
Foundations
Judging offensive AI capability is easier if you have done the offensive work yourself, against real, defended targets.
Regulatory-tier red teaming
CBEST, TIBER and iCAST adversary emulation against financial institutions. Useful for judging whether model-generated exploitation would work against a hardened target.
Enterprise trust boundaries
Active Directory, SAP, SWIFT and the authentication around them. An agent with tool access raises the same trust-boundary questions.
Detection & adjudication
Detection content, threat hunting and incident command: deciding, under pressure, whether activity is an attack or just unusual.
Practice leadership
Over 20 years leading offensive security teams in banking, consulting and defence: scoping engagements, setting methodology and developing practitioners.
Notes All notes →
Working notes.
Short write-ups on evaluations, monitors and agent guardrails, including what didn't work.
-
Rules of engagement as a control signal
A design note for monitoring authorised offensive-cyber agents. It pairs a rules-of-engagement check with a suspicion score for agents that reach beyond their task.
-
A weak monitor, and an attacker that never landed
A small trusted-monitoring experiment in ControlArena. A Llama-3.1-8B monitor screening code for backdoors did modestly on replayed data. The live attack round taught me more about eval design than about the monitor.
-
A regex is not a rules of engagement
Lessons from putting guardrails on an autonomous pentest agent, and why a pattern-matching classifier is the wrong kind of control for a tool-using model.
Contact
Open to conversations.
If you work on AI security, evaluations or agent control and want to compare notes, get in touch.