Effective oversight can significantly reduce the likelihood of an AI implementation failing

To fully comprehend the possible advantages and hazards of AI's newest developments, deep knowledge and background are necessary in the technology itself and the industry or business workflows where it will be applied.

For those that are accountable for budgets and protecting your brand, understand the gap between hype and the realities of AI (Artificial Intelligence). Using AI technology to achieve a business benefit requires practical expertise. In order to take advantage of the mature aspects of AI in a beneficial way, partner with a trusted expert. There are no shortcuts to a fully successful implementation and the on-going maturation of the models required by the technology. Often these types of projects can take a year or more.

Recent Failures Underscore the Risks

Although every company wants to successfully deploy technology in a beneficial way, when it comes to AI, there are obstacles to be overcome. In the CIO article from June 2023 titled “8 famous analytics and AI disasters”, the author, Thor Olavsrud, shared projects that had unexpected results that were detrimental to their given objectives. Those examples demonstrated the negative impacts that occur when technology is not well understood, output isn’t verified and the data models or algorithms aren’t properly monitored and managed.

In a recent podcast, Burke Autrey, CEO of Fortium Partners, provided a “CEO’s Guide to AI'“. A few key points from his presentation:

AI has been in development since 1956. There are mature aspects of AI. However, the recent Generative AI “Boom” began with OpenAI (2016) with Generative Pre-Trained Transformer models. These aspects of AI are not mature.
A web-based user interface has simplified the access to the ‘not as mature’ functionality of AI. Be aware of how this is being used within the company.
Referencing Gartner’s Hype Cycle, Generative AI is at the peak. However, hype today becomes tomorrow’s standard/expected features so awareness needs to exist.
CEOs should rely on their functional leadership team to lead key efforts, while holding the team accountable for awareness, understanding and appropriate action.

Since many small and medium size companies have limited resources and expertise, they should look to vendors to incorporate the more mature aspects of AI/ML solutions into their offerings.

With ChatGPT (and the others like it), great tools, but the output must be validated. Output from asking ‘ChatGPT’ (morning of 9/11/2023) : “Can I trust the output of ChatGPT?”

Provided 7 ‘warnings’ and key points of awareness and then summarized with this statement:

In summary, while ChatGPT can be a valuable tool for generating information and ideas, it should be used as a starting point rather than a definitive source, especially for critical decisions or situations that require high levels of accuracy. Always verify important information from reliable and up-to-date sources and use your judgment when assessing the trustworthiness of its responses.

Note: Overtime, the same question will provide different responses as the repository of information continues to grow or change.

ML (Machine Learning) - Many aspects are very mature

Robotics have been in use in warehouses since back in the 80s and 90s. There are now many vendors providing robotic solutions in this area. Recommend a buy rather than build solution.

Amazon has been developing forecasting models to assist in their own inventory management. An output of this effort is Amazon Forecast, “a fully managed time-series forecasting service based on machine learning (ML) and built for business metrics analysis.” Use of the tool/api does not require any ML experience. Another opportunity to buy rather than relying on internally built spreadsheet models.

Before you start your AI project

Accountability

Assign an executive or business leader as a project sponsor that will oversee and be accountable for the effort. Put together a cross-functional team that will report into the project sponsor and have a vested interest in the success of the effort.

Objectives

Be clear on the business problem you are trying to solve. Understand the business processes or procedures that are targeted for improvement. These processes should have key performance indicators that can be used as a baseline or representative of the current state. The results can then be compared to the baseline to properly document improvement.

Assess Internal Capabilities

Plan to have in-house capabilities to support on-going efforts. Before you start an AI project with a partner, assess internal capabilities and capacity. The objective should be to engage a partner to assist you with plan/build/implement and at the conclusion of the project, a proper handoff to your internal resources occurs. Determine whether additional internal resources are required or what level of education needs to occur prior to supporting and maintaining the new technology.

Define your goals and requirements

The extent of engagement or scope of work for the partner should be determined by considering the in-house proficiencies, the accessibility and availability of in-house personnel. Be very clear on what you want to achieve with the AI project and what capabilities you need a partner to provide. This will help narrow your search. Include an exit strategy for the partner on this particular effort to ensure proper handoffs occur and timelines are properly respected.

Evaluation of a New Partner

Since adding a new provider of services to your portfolio of partners, vendors, or suppliers (hereafter will just say ‘partner’) takes on additional overheard, start with your existing partners to determine if they have the ‘demonstrated’ expertise to accommodate your needs. If you determine a new partner is required, use a systematic approach to make the selection.

Don’t be in a rush. There are numerous companies using the descriptor of ‘AI experts’. Take the time to thoroughly evaluate potential partners against your most important criteria. Prioritize must-have capabilities and cultural fit. Cultural fit is often neglected in lieu of technology. Don’t underestimate the contribution cultural alignment makes to ensuring a successful engagement. Make sure your organizational values and priorities align for smooth collaboration. Communication and transparency are key. Below are a additional relevant areas. Prioritize based on your situation.

Leadership

Look at their leadership team and advisors. Do they have experts in AI/ML and business leadership? Verify who will be working on your account. Strong cross functional teams require leadership to ensure the new technology is not limited due to legacy constraints. However, balance must occur so legacy processes aren’t broken because of the changes being made.

Validate cross-functional teams

Opt for partners with multidisciplinary teams in their company that span AI research, engineering, project management, and subject matter expertise. This needs to be more than just about understanding the features and functionality. They need to be able to integrate with existing business processes.

Industry knowledge

Select a partner that understands your business domain and industry use cases. Domain expertise is invaluable. It takes too long and is too costly to educate a partner on your business processes when they lack industry knowledge. Do they have a focused vertical or use case they have solved for? Or are they spreading themselves too thin attempting to be relevant in every industry? Having a defined industry focus is better. When checking references, validate this expertise.

Examine their technology

Assess tools and infrastructure. Choose a partner with the computational resources, frameworks, and platforms needed to support your AI initiatives. Cloud-based resources are common.

Do they have any research papers, patents, or technical details available? Legitimate AI companies should have some technical foundations to evaluate. Look for technical expertise. Find partners with solid experience building and deploying the type of AI solutions you require. Review their past projects and client success stories.

Evaluate Data Assets

AI is all about data. The Partner will need to help you with your data. Verify that they have access to quality datasets and the ability to prepare data for model development. This is especially important for training machine learning models. Your internal resources engaged in the project should also be well versed in your data architecture and repositories.

Business model

There are many new start-ups providing AI services. Important to understand how they plan to generate revenue and scale the business. Assess if the model makes sense. It is important that they have the staying power to complete your project and then provide on-going support as needed. Recommend smaller scale projects to validate assumptions.

Partnerships

Reputable partners can validate the company's capabilities and market fit. Lack of partnerships may be a red flag. Ensure they have a strong relationship with your public cloud provider of choice.

References

Most importantly, check references and reputation. Talk to past clients and research reviews. A trusted partner is critical for AI success. Can you find any examples of clients adopting their solutions? Real world validation is key. References within your industry may be a challenge due to the competitive nature of the marketplace. Identify references with similar size project, implementation of similar technology and similar size company regardless of the industry.

Project methodology

Have them provide an overview of their structured approach to delivering AI projects iteratively and incrementally. Agile methods are popular. This demonstrates the level of maturity that exists in their ability to execute.

Check for transparency

A key objective for the provider is to assist you in developing in-house expertise. Does this align with their approach? Do they share information about their progress, technologies, and business operations? Lack of transparency is a concern. They should be very open about the technology they are implementing in your environment. Transparency should be validated when checking with references.

Compare costs

Evaluate project pricing models (fixed-bid, time & materials) and assess the overall value for investment.

Summary

Keeping up with today's marketplace requires a bias towards speed. However, quality while protecting the brand, customers and employees cannot be set aside for speed. Follow a systematic approach to ensure business objectives are met.

For those that are accountable for the success of the AI project, ensure you are partnering with trusted experts. In-house resources are critical to have engaged, however, a smaller percentage of companies have the depth of expertise or available resources to take on this type of effort internally without outside help.

Partners will be a valuable component in meeting your AI objectives. However, they aren't accountable for the success. Start with a smaller portion of the overall objective to ensure assumptions are validated in both the objectives and the partner. Then expand the rollout as appropriate and manage the exit strategy to completion.

If you are in need of assistance in maturing your partner selection process, schedule a time to discuss.

AI – Partner with a Trusted Expert