We used to regularly invite our clients and partners to technical presentations, known as ‘Get Togethers’. I particularly remember the event on 29 November 2017. Bitcoin was hot topic at the time, and my presentation was titled ‘Trust Shifts – Trust in the Digital Age’. It explored how trust works in a social context and within organisations, the facades we use to abstract and simplify, how poorly trust scales in large organisations, and how effective processes contribute to a company’s integrity.
The topic has come back into focus with the AI agents. There’s a new façade. Much like the Google search bar, there is a system hidden behind the agents, one that I either come to trust or simply do not. Which answers do I trust, and which do I not? What do I need to check before I base my life on them? Or what are the consequences of the autonomous actions of this genie (I like to call the agents ‘genie’, like Kent Beck), who fulfils my wishes?
For us software engineers, this question is of vital importance: we earn our living by taking responsibility for digital processes. We sit down with clients and try to specify a digital process as closely as possible to reality, and then implement it. Our reputation depends on us doing this with the utmost diligence. We break down complexity, assess the consequences and isolate risks.
Trust the Genie
Nowadays, it is not only humans who are writing more code, but AI agents as well. This presents us with a problem: the pace and scale of software development are increasing. How do we, as software engineers, keep up with this? How do we make sound decisions about whether to take on responsibility for increasingly complex software?
As a software engineering firm that prides itself on thoroughness, we have always relied on tried-and-tested methods for building trust in code. Key factors include, for example, how many stars a software library has on GitHub, or how many open, unresolved issues it has. We often know the maintainers personally from conferences. We also use technical metrics, such as automated tests with code coverage. For us, it’s always 100 per cent – much like double-entry bookkeeping, but for software. In code reviews, too, it has always mattered who makes the change: the intern? Or a senior software engineer with 10 years’ experience?
But what if a Genie is behind the code change? What if an intern – or even the client themselves – is now producing code that is thoroughly tested, with 100 per cent test coverage, and looks plausible?
In our experience, building trust does not scale well in this context, much like the situation with large organisations discussed in my 2017 talk. The tension lies between the increased effort required for code reviews and the client’s expectation of faster and more results. Faster, more and better do not naturally scale alongside a sense of responsibility. New approaches are needed, and we have a few ideas.
The Path of the Shikigami
Since the end of last year, we have been developing an AI integration for Redmine, our ticketing system. The customer writes their ticket as usual, but instead of assigning it to a human, it is assigned to “Jean Claude Opus” and set to “To Start”. Jean Claude takes on the ticket and implements it. When he is finished, he returns it to the customer with the status “QA”, which means that a human code review is required. Internally, we call the tool “Shikigamiya”, the house of the Shikigami. The main idea is that Redmine gives us control over the development process in a way that we, as engineers, can take responsibility for. The software development process is exactly the same as if a human were carrying it out – only faster.
Werbeeinschub: Advertisement: If anyone is interested in the Shikigamiya tool – for example, because you run your own Redmine instance – we can also extend the beta; simply book a lunch meeting with Samuel Steiner.
The perfect pull request
Code changes are proposed via pull requests (usually on GitHub). So far, we have been assessing the quality of these proposed changes using automated tests, linters and critical self-reviews. This also includes a reference to a clearly specified issue. We want to further improve the quality of our pull requests using automated review apps. This involves automatically creating a copy of the current staging application, where the review engineer can examine the changes without having to set up the project locally themselves. With a hosting provider such as Deploio, review apps are particularly easy to set up because the apps there are 12-factor compliant.
Mark out the perimeter
The specification – for which a human is always responsible – also always serves as the basis for automated tests. In software development, it is often said that ‘the specification IS the test’, and this is known as Test-Driven Development, or TDD for short. The most important point here is that the tests are written first. Otherwise, there is a bias towards the implementation. This leads to bugs or feature creep. AI agents are very poor at this, even when instructed to do so. This is due to the design of the context window: the agent cannot forget or deliberately ignore information. We would now like to introduce methods to separate the writing of automated tests from the implementation as much as possible. This is similar to the scientific method: first formulate the expectation, then measure whether it is met. Within this area, there are also older methods, such as Design-by-Contract, which we can dust off to set runtime expectations for a programme. Furthermore, we also want to define boundaries within the code using algorithmic policies. There should be areas that the ‘genie’ is not allowed to touch, and where I can trust that they have not been tampered with, so that I can skip these checks during the code review.
This still requires a bit of collective thinking within Renuo. The Learning Week in September 2026 is exactly the right place for this. There, we can explore this together as people and grow from the experience.
