We recently hosted a number of local CIOs here at our offices to chat a bit about where we’re at with generative AI, and when it will start being operationalized. The F500 has already kicked off a plethora of enterprise POCs in the space and are hard at work extracting the more important enterprise use case needs. While generative AI may not be 100% enterprise ready yet, in the next 6-12 month we’re going to start seeing more serious (potentially customer-facing) deployments. Budgets are being defined for larger-scale 2024 deployments, with the rest of 2023 being the experimentation phase. Our group chimed in to provide some color on where we’re heading.
Where is Enterprise Generative AI at Today?
- Back to Basics – People, Process, Technology hasn’t really changed
- Architecture Matters Once More – There is so much at stake and the pieces need to come together properly
- A Long-Term Vision is Necessary – How software is built in the next 20 years will completely change – it won’t just be efficiency, it will eventually become true innovation. The big transformations of the last 30 years (for the enterprise) will be the internet -> mobile -> LLMs – so a cohesive game plan for the 2020s will be required
- AI Will Initially Look a Lot Like RPA++
- AI for Internal Use & AI in the Product – CEOs of every company (not just tech companies) will want to know how this is embedded in the product (“we have AI technology”). Over the next few years, companies will need to be embedding generative AI features into all of their products to compete – that planning starts now
- There Will Be More Losers Than Winners – so maybe hold off on any major investments – no contracts of >1 year. People are asking for cost premiums today on things that will be commodities tomorrow – now is the time for small bets and exploration
- Work Hard to Keep Things Contained Until This is More Enterprise-Ready – Avoid IP and sensitive data leakage
- Begin Early Exploration with Large Partners Like Salesforce and Microsoft – What brings benefit and what doesn’t? It’s easy to piggyback on their IP
- There Will Be New Legal and Regulatory Hurdles to Tackle – How do you explain how these new tools came to their conclusions? How do you explain how a 30B parameter model came to its conclusion?
- It’s Time to Review Vendor Renewals With a Closer Lens – Are you opting out of sharing your data? Are you opting out of your data being used for training? Especially if that data is proprietary. Many existing software providers are attempting to use your data to train their models. Make vendors prove that they’re retiring your data
- Data Will be the Key for Impact and Utility, Not Technology (Which Will be a Me-Too Product) – (1) The provenance of your data is critical in regulated industries (e.g. in healthcare, if one data source is wrong, people could die), (2) Data quality will be the key to advancing AI efficiencies – you need something that will validate the data that you’re getting, (3) The ecosystem around enterprise data warehouses is going to continue to explode , (4) The Chief Data Officer will be replaced by the Chief AI Officer, (5) Early use cases should be lightweight and not break anything major
- It’s Crucial to Keep and Eye Towards Real Deployment – Don’t leave things in the sandbox forever while your competitors pull ahead
- Generative AI Will Ultimately Be Rolled Into Standard Product Offerings – It won’t be something that you pay for separately as it becomes increasingly commoditized – competition will drive the shift
- AI Startups Will Need to do a Better Job Complying to Enterprise Standards – Play by the rules, be transparent, and build trust
- There Will Be Many Competitive Opportunities for Vertical Plays – Open source is going to be great for industry-specific or niche build use cases (vector in your own data etc.)
- Companies Will Need to Gain Comfort Around Which LLMs Are the Right LLMs for Them – What should they be looking for? How should they be evaluating? VCs and other thought leaders can help evangelize here
- Real Production Use Cases Will Begin to Emerge in 2024
How Will Generative AI Change the Security Landscape?
- Teams are Excited and No One is Thinking About Security
- Data Security is a Rising Issue – As soon as the data leaves to a third party it’s a huge risk (company recommendation for this: https://www.private-ai.com/). The security team responsible for third party risk is going to have to change their process a little bit (show me you are deleting my data, insist on positive consent in contracts)
- Threat Vectors Will Be Smarter Than They Used to Be – Security solutions will have to be smarter too
- More Available Data -> Increase in Cyber Crime
How Should Leadership Be Handling the People Element?
- Education, Training, and Communication is Key – You have to focus on educating your teams, educating the board, educating your customers and educating regulators – the language people use today is inconsistent or inaccurate
- Try to Enable Internal Teams – Otherwise they are going to be doing these things on their own. Employees are not cautious. Collect internal use cases by division or persona
- Establish Guard Rails and Weekly AI Office Hours – Figure out how to partner with the business, not just block them
- Create a Sandbox for Employee Use Cases
How Effective Will Early Generative AI Use Cases Be?
- The Tech Looks Interesting, but the Deployment Isn’t There Yet – There are issues with data, security, leakage, etc. So the opportunity is present, but still nascent
- The Type of Use Case Will Matter A Lot – Regulated industries and areas like security are going to require a lot more caution
- Code Generation is Still a Challenge – It impacts IP directly (is AI generated code your code?)
- KPIs are Needed for Each Use Case – Figure out how it’s improving things either via TTV, eliminating/freezing staff, etc.
- There is More Scrutiny from Regulators Now – This will slow things down
What Early Use Cases Are Looking Strong?
- The best use cases today are the ones where we already know what good looks like (e.g. RPA)
- RFP processes – E.g. Transforming/automating the RFP, RFI, DDQ and security questionnaire processes
- Customer service/customer facing efficiency and knowledge sharing and enterprise search (e.g. Glean++)
- Many companies are seeing modest, but measurable, productivity gains with copilot (15-25%)
- Document (job descriptions, emails, etc.), communications, and content generation
- QA
- Forecasting
- New product development (e.g. a genAI tutor for math)
- Code conversion (vs. generation) – e.g. Gosu into Python
- Contract management and summarization
- Translation
- Internal security
AI Council Best Practices
- Who Should Be On It? Head of Data/AI, Head of Product, R&D, Legal, CIO (representing GTM, finance, HR, etc.), Engineering, Program Manager
- Have people submit needs, group them, and give one team a chance to drive things forward for the whole company (vs. having 100 siloed conversations)
- This could also be a group that drives what evolution in the product will be (maybe have a “working group” for product/eng)
Buy or Build?
- Hybrid model, or buy and train
- Take generic models and fine tune them – but the value has to be there relative to the cost (costs will come down over time)
Will CIOs Get More Budget?
- No
- Good news: GPT-3 cost $15M to build, Mosaic built the equivalent with $500K