Oscar, an open-source contributor agent architecture (AI column)

Oscar is an interesting project. from Google about the use of LLM agents in programming. However, the goal is not to generate code, but rather to eliminate the routine work of maintaining a large open source project. The bottom line is that writing code is an interesting part of creating software, but processing incoming issues, matching questions with existing documentation or checking reports is all an uninteresting part of the job and it would be good to automate it. Oscar itself is now an experiment and part of the golang project, but could become a standalone project in the future. The original purpose was

• Reduce maintainer effort to resolve issues [note that resolve does not always mean fix]
• Reduce maintainer effort to resolve change lists (CLs) or pull requests (PRs) [note that resolve does not always mean submit/merge]
• Reduce maintainer effort to resolve forum questions
• Enable more people to become productive maintainers To solve these problems, the authors decided that Oscar should have three key opportunities.

1. Indexing and displaying the associated project context during participants’ interactions.
2. Use natural language to control deterministic tools.
3. Analysis of problem reports, Change list/pull requests and group discussions to improve them in real time during or shortly after sending, and for their appropriate labeling and routing of reports

In general, the authors’ approach is to use LLM in what they are strong at, or rather in semantic analysis of natural language and its transformation into calls of deterministic code to do the rest of the work.

1) Indexing and displaying the associated project context Here, the authors suggest using embeddings and vector databases to index and search for relevant information. This was also mentioned in other documents: Fowler’s article.Emerging Patterns in Building GenAI Products" (my short story here) and Whitepaper "Agents" by Google (story here). Oscar authors note the following advantages of using agents The agent surfaces related context to contributors - contributors are reported to have similar problems, allowing them not to duplicate problems The agent surfaces related context even to project maintainers - Maintainers can be useful to see similar bugs and collect more accurate information, which allows you to correctly prioritize, close or reopen the bug

• The agent interacts with bug reporters immediately - immediate feedback to the person who reported the bug allows you to collect more relevant information, as reporterts is ready to provide additional information in the moment or to investigate the problem behavior more deeply if you ask him to do so immediately. 2) Using natural language to control deterministic tools Here, the authors of the bot left a point for expansion, but have not yet integrated this into the bot. But, in fact, it describes the use of tools from the framework about agents, described in the whitepaper "Agents", about which I am already wrote. For example, there may be a fix of the text of the report or a proposal how to fix it. For example, here. here There's a whitepaper from Google's "Evaluating Agent-based Program Repair at Google." 3) Analyzing issue reports and CLs/PRs Here again, the authors left the expansion point under the action of putting the correct label issue or checking the correct completion of the error report. In principle, everything is quite clear here.

Inside go community Oscar presented as a bot Gaby ("Go AI bot")One that deals with new issues. Interestingly, there’s a gopherbot inside the go community, but it’s based on code that reacts determistically to commands.

#AI #ML #Engineering #Software #Architecture #SystemDesign #DistributedSystems