5 Practical Considerations for Implementing Gen AI in Enterprises | by Anagha Donde | Jan, 2025


How to navigate the challenges and maximize the value of Generative AI for your organization
Enterprises, big and small, are actively assembling AI teams to employ generative AI (Gen AI) to accelerate innovation and gain efficiencies. McKinsey Global Institute estimates that generative AI could add the equivalent of $2.6 trillion to $4.4 trillion in value to the global economy and has the potential to automate work activities that consume 60–70% of employees’ time today.
Generative AI offers transformative opportunities for enterprises, but its implementation comes with unique challenges that require careful planning and consideration for successful deployment. These challenges span three critical areas: business, people, and technology. This article explores five key practical considerations from business and people perspectives. The technology aspects will be addressed in a subsequent, separate article.
- Prioritize the Right Use Cases
Generative AI and LLMs are democratizing access to artificial intelligence, finally sparking the beginnings of truly enterprise-wide AI. Powered by the potential of newly emerging use cases, AI is finally moving from pilot projects and “islands of excellence” to a generalized capability integrated into the fabric of organizational workflows.
There is a lot of enthusiasm and curiosity for the potential of Gen AI to transform enterprise workflows. However, identifying the right use cases where Gen AI will have the most impact can be difficult. Not all functions in an enterprise will benefit equally from Gen AI, so prioritizing where to apply Gen AI is critical for maximizing returns.
According to the McKinsey report, about 75% of the value that Gen AI use cases could deliver falls across four areas: customer operations, marketing and sales, software engineering, and R&D. Quantifying benefits obtained through Gen AI applied to use cases in these functions may also seem fairly uncomplicated if data collection mechanisms are in place. For example, the benefits of an AI assistant for customer support agents can be measured through metrics such as calls handled per hour, time per call, and customer satisfaction. These functions therefore offer the most promise as the first targets to apply Gen AI. However, consider other factors, especially organization-specific factors in your prioritization efforts. These may include readiness for Gen AI, cultural response to change, technical capability, security and compliance requirements, availability of data, and more. Irrespective of what factors play a role in the prioritization and what function you choose as the first candidate for Gen AI implementations, begin with narrow, well-defined, quantifiable use cases that can be implemented quickly to demonstrate business and customer value, managed from a risk perspective, and help build momentum for future implementations.
2. Quantify the Business Value
We are now in The Between Times for AI — between the demonstration of the technology’s capability and the realization of its promise reflected in widespread adoption.
— Authors of ‘Power and Prediction. The Disruptive Economics of Artificial Intelligence’ (HBR Press)
Interest in Gen AI is high, but implementations require significant investments. Without clear, measurable ROI, justifying and obtaining serious investment in Gen AI implementations beyond proof-of-concept and pilot projects is challenging. Executives need confidence that the benefits of implementing Gen AI will outweigh the costs and risks.
Quantifying the business value isn’t always an easy task. The choice of the metric is as important as the choice of the use case. Proposals for Gen AI implementations face the same challenges as any other automation when choosing the right metrics to quantify benefits. For example, an often stated benefit of Gen AI-based assistants is improved employee productivity and a frequently chosen metric to quantify the benefit is employee time savings. However, employee time savings will only translate to enhanced employee productivity if the captured time savings accelerate the delivery of priority initiatives or are re-employed towards tasks that drive business objectives. So, making that connection between employee time savings and a tangible business outcome is important, though often also difficult.
Throughput is another, at times more appropriate, metric for measuring Gen AI benefits. However, throughput is more reliably measurable for repetitive, standardized tasks, making it challenging to apply to the diverse and variable tasks that many information workers perform daily. That said, linking increased throughput to a tangible business outcome may be easier in certain use cases. The key takeaway: Quantified and well-articulated business value is essential for securing executive buy-in and avoiding delays in Gen AI initiatives
3. Assemble a Talented, Multidisciplinary Team
There is a great deal of anxiety in the industry around a shortage of skilled professionals who can build, manage, and optimize AI systems, particularly for large-scale enterprise implementations. Upskilling and re-skilling your existing developer pool and augmenting it with data scientists, AI/ML experts, Cloud engineers, and Site Reliability Engineers, as required for your implementations, might be a prudent strategy in many cases.
Getting it right with Gen AI implementations will also require assembling a multidisciplinary team that includes members from IT, Legal, Security, HR, Compliance, Procurement, and users from business teams. Start with a core team of engineers and product managers and grow the team as needed. Don’t overlook the value of experience developing, deploying, and scaling large non-AI applications as Gen AI applications involve a great deal of standard automation. Once an initial team is assembled, it will be important for management to provide the support and sponsorship, and have the patience to allow time for exploration and experimentation.
4. Manage Gen AI Risks
If business value or ROI is top-of-mind for executives when considering Gen AI implementations, concerns over Gen AI risks rank a close second. These include potential for data privacy violations, IP leaks, data security breaches, non-compliance with laws and regulations, inaccurate or fabricated Gen AI output, harmful bias, and more.
While the promise of Gen AI is alluring, implementing Gen AI in enterprises comes with a unique set of challenges that need to be thoughtfully addressed for a successful implementation.
One good resource for exploring the AI risk landscape is the AI Risk Management Framework published by the National Institute of Standards and Technology (NIST) in July 2024. NIST also signed collaboration agreements in August with Anthropic and OpenAI regarding AI safety research, testing, and evaluation that promise good things.
AI risk and mitigation landscape is still evolving. This is a critical and broad topic that cannot be entirely covered in this short discussion, but here is a potential roadmap. To compile an AI risk mitigation plan, begin with a thorough review of your enterprise’s data and compliance policies. Enlist help from Legal, Security, and Compliance teams to compile a list of data privacy, security, and compliance requirements for the proposed solution. Add to that list any AI-specific risks applicable to the proposed solution. For reference, see NIST’s AI RMF mentioned above. Plan measures to address or mitigate identified risks and articulate these clearly in your proposal to your executives. Update this plan as you progress through the implementation phases and define a process for periodic risk assessment and audit when in production.
Some risks can be addressed through existing technology solutions — defense-in-depth security, strong encryption, role-based access control (RBAC), anonymizing sensitive data, etc. If purchasing AI solutions from third-party vendors, understand privacy and security controls offered or if building in-house, design appropriate privacy and security controls to be built into the solution. Test the feasibility of such controls in your proof-of-concepts and pilots. Some examples of such controls may be user notifications, explicit consent for certain actions, AI-specific control measures — for example, to prevent prompt injection attacks, and more.
Although a related but different topic, the discoverability of AI-generated content in litigation is an important consideration. Work with your organization’s legal department to ensure discoverability requirements are assessed and met.
5. Plan for Employee Trust and Adoption
What does employee trust mean when it comes to Gen AI? Employees typically have two primary concerns with Gen AI implementations — job displacement and trust in Gen AI output. Think about this — for many use cases, Gen AI is not just another tool employees use to assist them with their jobs. Instead, Gen AI is now producing part of the outcome originally produced by the employee. This means that employees have to trust Gen AI to do part of their job role — such as creating code snippets or drafting ad copy or contracts and this requires a different, higher sort of trust. Ultimately, employing Gen AI brings enhanced productivity and creativity benefits to employees, but the concerns must be satisfactorily addressed for employees to trust using Gen AI.
So how do you build such trust? Here are some suggestions. Actively involve employees from the start of the project and plan for and encourage ongoing participation and feedback. Invest in employee education to increase Gen AI literacy. Communicate transparently about project statuses, features, how roles will change, and what skills will need to evolve. Focus on how Gen AI will enhance and complement rather than replace human work. Conduct POCs and pilots to demonstrate value and provide an opportunity for feedback. Incorporating Human-In-The-Loop (HITL) approach into Gen AI solutions also helps gain employee trust by reassuring employees that they will be involved in validating and refining Gen AI outputs, by providing an opportunity for feedback, and also because HITL workflows help demonstrate that AI complements human expertise, not replace it.
Other considerations based on individual enterprises’ circumstances may include engagement with Works Councils, customer trust and adoption of customer-facing enterprise solutions, and other appropriate measures.
I will share two final thoughts.
One, agentic workflows are gaining traction in Gen AI and other AI applications. While useful in many applications, implementing agentic workflows in employee-facing Gen AI applications needs careful planning and user education. If you are in the early phases of introducing Gen AI applications to your organizations, begin with explainable, HITL workflows and build trust for appropriate agentic workflows in future phases.
And two, although I have not specifically discussed change management in the five practical considerations discussed above, it is a common theme across all of them. AI implementation leaders will need to plan for and drive change to succeed.
The era of generative AI is just beginning. Excitement over this technology is palpable, and early pilots are compelling. But a full realization of the technology’s benefits will take time, and leaders in business and society still have considerable challenges to address. These include managing the risks inherent in generative AI, determining what new skills and capabilities the workforce will need, and rethinking core business processes such as retraining and developing new skills.
Implementing Generative AI applications in the enterprise offers significant rewards, despite the challenges you may need to overcome. What other considerations or challenges do you see in implementing Gen AI-based applications in your organizations?
I work at Oracle. The views and opinions expressed here are my own and do not reflect or represent those of my employer.
