Agent Skills and OKF: Skills, Knowledge and Trust

A skill tells an agent how to perform a task. SKILL.md combines a small YAML header containing a name and description with instructions and references to supporting material. The model sees the description first, loads the instructions when the skill is activated, and retrieves other resources as needed. This saves context, but makes descriptions part of selection behavior. An overly broad skill can crowd out more useful ones. Explicit invocation and automatic discovery serve different purposes; the speakers do not declare either universally superior.

A polished file is not evidence of improvement. Discussing SkillBench, Smirnov contrasts generated skills with curated, iteratively refined ones: the former did not reliably help, while the latter could. He also sees skills as a way to improve smaller models and keep multistep execution within useful boundaries. The implication is to test the actual configuration, rather than expect a consistent benefit across models.

Sharing skills inside a company quickly becomes a quality management problem. People contribute generated folders to a catalog and compare downloads or likes, while basic questions remain: which tasks does each skill suit, does it follow company rules, and who updates it? External catalogs also introduce the risk of malicious instructions. Popularity does not establish suitability for a particular workflow.

Smirnov connects this to the familiar difficulty of reusable components: each new consumer still needs changes. Narrow instructions express expectations more precisely but travel less easily. Polomodov illustrates this with a favorite coffee recipe: adding everyone's preferences can turn it into generic advice the model already knows. An architecture decision also needs more than a template. Technology selection requires the organization's technology radar; decomposition requires knowledge of its systems. Independently running agents need a shared source of those facts, distinct from the procedure that uses them.

One set of checks determines whether today's document or code is acceptable. Another tests whether a changed skill still performs well across a collection of tasks. Combining these instructions can send the agent into the wrong evaluation process. The speakers recommend making the distinction explicit. A basic completion check is useful, but creating the expected files does not establish that their contents are correct.

Skill Creator is discussed as an accessible aid for preparing and running evaluations, with token costs and limits on transferring results between models. The unit under test includes the skill, model, and surrounding agent software. Context loading, skill discovery, and history compaction can all affect behavior. An unchanged SKILL.md therefore does not guarantee unchanged results after an agent update.

A viewer asks how to challenge old rules and skills generated by a weaker model when the team prefers to leave working tools alone. The proposed approach starts with actual failures. Collect tasks where a person had to correct the output and define what success should have looked like. Reconstruct a clean environment so the agent cannot find the finished solution in repository history or receive clues unavailable in the original task.

Compare the old skill, a run without a skill, and an improved version. Use deterministic checks where possible, such as tests for the bug being fixed. For other properties, use a model judge with explicit criteria. Inspect the execution trace and measure time, tokens, and cost alongside success. These comparisons can reveal both useful instructions and unnecessary work. A major model release is a practical trigger for retesting: some old guidance may have become redundant. Evaluations themselves consume resources, so their timing should be deliberate.

The knowledge discussion starts with the motivation behind LLM Wiki: repeated questions often require parsing the same PDFs, tables, and articles again. Instead, information can be extracted and structured once, then queried through an intermediate wiki. A schema describes page and relationship types. Three operations surround the collection: ingesting new material, answering queries, and maintaining integrity.

OKF is presented as a specification that expresses this approach through Markdown, typed metadata, and relationships between pages. It builds on familiar ideas and can work with tools such as Obsidian. Implementations may expose operations through MCP. The speakers see value in a shared description of knowledge for agents; correctly arranged files alone do not ensure accurate extraction, fresh content, or identical behavior across implementations.

Asked whether OKF replaces RAG, the speakers describe a spectrum of tasks. A lightweight structured collection can serve skills within a bounded domain. Graph databases address larger connected datasets, while broad collections and heavy query workloads can justify retrieval infrastructure. This is a guide to choosing an approach, not a universal document-count threshold. Organizational search also needs index updates, access control, and relevance evaluation; a storage format does not settle those issues.

Polomodov describes asking for company-wide strategy and receiving a well-written strategy for a small department. The document exists, but its scope does not match the request. Selecting knowledge requires attention to the user's role and the question's scope. Wikis also decay as pages become outdated and links break. A promising agent task is to check and repair the collection regularly. Such maintenance may slow deterioration without guaranteeing permanent order.

The discussion of OKF v0.2 shifts from data organization to provenance. Source identity, freshness, and the process that produced a derived article are separate concerns. A link and timestamp are insufficient: the timestamp itself may need attestation. Smirnov highlights verification of the algorithm's execution behind a result. The episode discusses this direction in the specification without demonstrating an operational verification system.

The closing synthesis combines three elements: skills specify work, domain knowledge supplies its factual basis, and evaluations and execution receipts help assess outcomes and provenance. Multiple specialized agents also need a shared domain model and consistent validation rules. Autonomous operation depends on engineering these pieces together; neither an instruction format nor a knowledge format provides it by itself.