From tribal knowledge to AI Teammate
Teaching AI 15 years of Umbraco
@zajkowskimarcin
Has anything changed?
Let's blame AI for that!
“roughly one in six people worldwide now using generative AI tools” - Microsoft AI Economy Institute, Global AI Adoption in 2025, Jan 2026
Is it a lot?
We're in the front row.
But not everyone likes popcorn...
~13 times
~4.1 years
Avg. jobs changes + median number of years with current employer - US Bureau of Labor, Sep 2026
We're all gonna die.
Console.WriteLine("Hello, World!");
Console.WriteLine("Hello, World!");
Console.WriteLine("Goodbye, World!");
The truck factor is:
"the number of people on your team that have to be hit by a truck (or quit) before the project is in serious trouble"
- L. Williams and R. Kessler, Pair Programming Illuminated. Addison Wesley, 2003.
Almost half of the projects have a bus factor of 2 or less.
Source: https://metabase-public.metabaseapp.com/public/dashboard/552f3868-5f09-4b0b-a403-67089952d32c
I know you're on holiday, but...
Brent isn't the problem.
The system that made Brent indispensable is...
The model already
knows Umbraco.
Should we trust AI?
"We can know more
than we can tell"
Turning tacit into explicit is the hardest step.
Michael Polanyi, The Tacid Dimension (1966)
Chesterton's fence
2nd July 2019
- Duration: 29 minutes
- Symptom: 100% CPU exhaustion
- The Root Cause: A single new regex rule...
Public
Docs, release notes, open source code.
Repo
Code, configs, history, additional "context" data.
Tribal
Why, when not, regrets, "best practises".
Umbraco Healthcheck <3
Give us:
- Access to CMS backoffice (login, Cloud access) or we'll hack it anyway
- Access to source code or we'll decompile it anyway
- Access to the database or we'll pull it from the live site anyway
- Time...
Takeover, refactor, rebuild or... run!
v10 Headless - 2024
- 111 checks
- repo access, db copy, team interview, documentation (!)
- 86 of 111 (77%) reach a detector today
v7 MVC - 2023
- 90 checks
- no repo access, db copy, backoffice, team interview
- 71 of 90 (78%) reach a detector today
v7 MVC - 2022
- 103 checks
- repo access, db copy, team interview
- 83 of 103 (80%) reach a detector today
(Craft) Healthchecks Selection
20 hours...
What can be automated?
Idea: The tool does the job, I drink coffee and watch YouTube.
Additional context: DO NOT involve AI until necessary or REALLY helpful, gather and cite, let me decide. Make no mistakes ;)
- 52 checks fully automatable
- 77 partly
- 7 manual only!
Automation didn't
replace the expert.
We became... partners <3
42 of 136 checks were inferred by the model...
38 recovered from my own prose, 4 with no anchor at all.
No evidence = no confident recommendation.
Every blocker carries file:line and a docs citation. A missing tool or a check it can't run becomes an open question, never a guess.
knowledge-mcp
WHY it matters, when it does not apply. Reviewed by a HUMAN.
umbraco-docs-mcp
Vendor info, cached and cited.
umbraco-legacy-mcp
Deterministic tools.
Detectors, plain code, read-only.
Facts → Judgment → Decision
Project structure
Sample detector
Sample detector's finding
Knowledge Unit
Knowledge confidence check
Fact and reason joined as finding
Other project decisions worth knowing...
The registration gate
Scanning for facts...
Finding things...
MCP output converted into blocker
Claude, MCPs and nothing else...
Should we trust AI?
Three REAL client solutions
Reports, reports...
Damn fast!
Adjustable and "dynamic" for growth
But...?!
Captured judgement
can be wrong too.
"False positives"?
70%
The draft said 70% of the site was duplicate content. After the fix: 13%.
13%
Why not just CLAUDE.md?
Or skill(s), plugins etc.
Who and where is Jev?!
Narrate
Generative agent
Ranked report, upgrade path
Expensive, slow, used sparingly
✅
Decide (new)
Calibrated scoring model
A probability for each option
Cheap: ~$0.04 per million i/t
???
Detect
Deterministic MCP tool
Facts: files, lines, counts
No mode, ever!
✅
Advisor was designed in three layers before “decision models” were a product category. Calibrated scorers such as Microsoft-Decision-1 and Jev fit the middle layer without changing the one rule that matters.
What's NEXT?!
An AI audit of my AI-built tool found a CRITICAL path-jail gap and four detector branches no test proved.
Built by agents, audited by agents, signed off by me.
Power & Control
Observability, Flexibility, Extensibility...
But...?!
I became Brent...
What can you do
on Monday?
Mine, mine, mine...
- PR comments
- postmortems
- Junior's questions (R.I.P. Juniors <3)
- The Brent Interview (scan QR code, hope this one works ;))
Format ladder
- Instructions files (individual)
- Skill
- Shared skills
- MCP server(s)
- Harness'
- Autonomous, self-fixing client sites and business'...
AI doesn't fix a team.
It amplifies what's
already there.
If your knowledge lives in Brent's head, AI amplifies and catalize Brent's absence.
Start with ONE thread.
Not "all company knowledge".
Marcin Zajkowski
zajkowskimarcin
mzajkowski
zajkowskimarcin
promptingforbabies.com
Country Manager @ Novicell PL
Solution Architect @ Novicell UK
CTO @ Novicell NO
CTO / CKO @ WOW School
Marcin @ umbraCoffee (Last Friday's 11:30 on YT)
Blogger @ udfnd.pl
YT Wannabe / Podcaster @ The Busy Programmer
https://bit.ly/umbuk26-brent
