The selection of methodologies should be adapted to the control risks linked to the compliance measures put in place by the audited provider, as well as to the detection risks, namely the risk not to detect misstatements in the information the provider makes available to the auditing organisation. For example, where an audited obligation could involve the audit of an algorithmic system based on personalisation for individual recipients of the audited service and on recurrent updates of the algorithmic system, such as the disclosure obligations for recommender systems pursuant to Article 27 of Regulation (EU) 2022/2065, the choice of methodology should allow the auditing organisation to design the appropriate tests to minimise detection risks. Similarly, where the auditing organisation seeks to assess whether all relevant risks were mitigated in the design, functioning and use of applications based on large-scale language models, such as chat functionalities or recommender systems deployed by the audited provider, the auditing organisation should first assess the appropriateness of the controls put in place by the provider. The choice of tests should be informed by the robustness of those internal controls. In particular, but not limited to the cases where the internal controls are weak, incomplete or inconclusive to assess whether the rules are complied with when considering the population of recipients of the audited service, the audit procedures should rely on a combination of methodologies. For example, methodologies could include substantive analytical procedures, such as the analysis of the interactions between all algorithmic systems involved in the recommender systems and related decision-making rules and processes for establishing the main parameters of those recommender systems, observations of digital records and logs. Methodologies should also include tests of the system, such as tests in simulated environments.
Recital 25, Regulation (EU) 2024/436
In forcesince 20 Oct 2023