AI tools hold great promise for DataOps. They can help teams identify the best apps to migrate, identify optimum cloud infrastructure and even validate decisions. But how can an IT team ensure the AI tools it uses are actually advancing its DataOps goals?
A hard pill to swallow is that even the best-designed algorithmic decision-making and AI tools are prone to errors. Fortunately, this is unlike errors in software development, which are largely caused by development. AI errors emerge from poorly defined project goals and the organization’s tendency to treat the technology as if it’s magic. Without solid requirements, model monitoring and regular check-ins with reality, it is almost impossible for AI to deliver.
This is why audits must be a mandate when introducing any AI tool that greatly influences business decisions. However, before the audit is conducted, ensure that the following points are met:
Ensure the AI Tool is ‘Auditable’
This step ensures that organizations bring in the right AI tools to their arsenal.
An AI tool is auditable when it is:
- Transparent: When AI tools are used to enhance the decisions a business makes, stakeholders must be able to peer into the system to ensure the legitimacy of the algorithm decision. Ensure the AI tool you choose has a logging system that makes it transparent as to how the system is making a decision, what factors most influence it and how confident it is in that decision. This allows decision-makers to trust in the system, as well as creates a feedback loop for the developers to take in real-world data and enhance and fine-tune the system to your needs.
- Built with best practices: Just as security engineers follow best practices when securing a network, AI models and tools built with best practices in mind fare better when in use. As a general rule of thumb, all AI models must keep detailed access and process logs, incorporate failback mechanisms and generate regular health reports.
Set Up Markers for Success
Don’t approach an AI audit as though it’s a sandbox question. If teams are clear as to what an AI tool must achieve and how to measure its success, only then will an audit be able to judge the performance of the tool.
It is helpful to set up markers that describe success. This could be the number of cases the AI model has rightly predicted, the time it takes, the number of edge cases it is able to handle and the number of times the AI decision collided with a human decision.
The Audit
Once the AI tool has been integrated into the organization and is in active use for a defined period, audits must occur. They help you answer the following questions:
- Is the algorithm suitably transparent to end-users?
- Is it likely to be used in a socially acceptable way?
- Might it produce undesirable psychological effects or inadvertently exploit natural human frailties?
- Is the algorithm being used for a deceptive purpose?
- Is there evidence of internal bias or incompetence in its design?
- Is it adequately reporting how it arrives at its recommendations and indicating its level of confidence?
(HBR: “Why we need to audit algorithms”)
Assess the Algorithm’s Fitness for its Purpose
From a use-case perspective, AI algorithms in modern software applications tend to fall into two main categories. One is offering insights that help humans make decisions—this is what AI-driven analytics is all about. The other is allowing software applications to undertake automated actions entirely on their own, as AIOps tools increasingly do.
A basic step in auditing your algorithm is to determine which type of use case it is supposed to support, then evaluate how well it achieves it. This is important to do regularly, because AI use cases can drift as applications evolve and the original purpose of an algorithm might change over time. (It’s common for algorithms to be implemented originally as a way to offer insights, but for applications to be updated with the expectation that the algorithms can support automated decision-making.)
Compare and Contrast Algorithmic Decisions Against Human Decisions
Though some of the inner workings of an AI model are a black box to even data scientists, the result of those inner workings is not. An AI audit must set up a process and team that regularly and actively monitors the decisions AI makes and compares them against decisions that a human would make.
This is useful in identifying early on whether the AI tool has internalized biases. With an AI system, input greatly affects learning. For example, if a tool used to forecast purchase orders is incorporated by the organization during peak sales season, the tool will assume that this is the organization’s baseline performance. This causes erratic predictions in the long run.
Active Experimentation
Though an audit should uncover any bugs, biases or security concerns of an AI tool, it also could be used to test system variables. An AI tool is generally built on assumptions of the problem borders and trends. However, assumptions are very fragile in reality. The baselines of reality could change, causing the AI to produce questionable results.
An audit also must be able to revisit AI tools with new assumptions, annually or biannually, to uncover any new parameters that could help improve the accuracy of the tool.
Interviewing Users of the System
Ultimately, an AI tool is put into place to either reduce the workload of humans or enhance it. Daily users have vast knowledge and perspective into not just how the tool works but also its limitations and quirks.
Conclusion
More often that not, the mystique of an AI model is used as an excuse for erratic results and poor performance. By building a robust and transparent audit system, your organizations can ensure your AI tools are actually working for your teams—and if not, it can at least uncover rare scenarios and edge cases and help mitigate them.
This sponsored article was written on behalf of Unravel.
