Imagine a world where every piece of data, whether it’s an image, text, or a blog, can be represented by numbers. It may sound far-fetched, but it’s actually possible. By converting data into vectors and capturing the relationships between different words, we can create something called an embedding. This allows us to turn search and classification into a math problem.
In this multi-dimensional space, text can be represented as mathematical vectors. This creates clusters where words with similar meanings are grouped together. For example, in the screenshot above, words related to “database” are clustered together, making it easy to respond to queries containing that word. Embeddings can be used to create text classifiers and enhance semantic search.
With a trained model, you can even ask it to generate specific images, like “a cat flying through space in an astronaut suit,” and it will produce the image within seconds. This magic is made possible by large clusters of GPUs and CPUs that process vast amounts of data, constantly updating the model’s weights and biases. These trained models are already increasing productivity and reducing the need for additional hires.
Now, let’s talk about competitive advantages. Just like the show Ted Lasso attracted new customers to AppleTV, having a trained model can drive users to your product. As they use the product and provide data through text prompts, the model can continuously improve.
Once the data -> training -> fine-tuning -> training cycle begins, it becomes a sustainable competitive advantage for businesses. Vendors have been focused on building larger models for better performance, pushing the boundaries of what’s possible. Why settle for a ten-billion-parameter model when you can train a massive general-purpose model with 500 billion parameters that can answer questions from any industry?
However, there’s a growing realization that the size of a model may not always correlate with productivity gains. For specific use cases, smaller models trained on highly specific data may be more effective. BloombergGPT, for example, is a 50 billion-parameter language model trained on financial data that only Bloomberg can access. It demonstrates that use-case specific models can outperform their larger counterparts.
When it comes to choosing a generative AI model, there is no one-size-fits-all solution. The best model for an enterprise will depend on the specific use case, whether it’s large or small, open-source or closed-source. Evaluations have shown that smaller models can sometimes outperform larger ones. Decision-makers must consider various factors before implementing generative AI.
Complexity of operationalOptimizing foundation models: Training a model is an ongoing process that involves updating the weights and biases through fine-tuning. It’s a continuous journey of improvement and refinement.
Training and inference costs: There are different options available for training models, each with varying costs. Training a large language model from scratch can be expensive, costing up to $10 million. Using a public model from a vendor can also accumulate API usage costs. Alternatively, fine-tuning a smaller proprietary or open-source model requires continuous updates.
It’s important to consider the costs associated with calling the model’s API. For example, if a model is used to customize each email in an email blast, it can significantly increase costs and impact gross margins.
Confidence in wrong information: Language models can sometimes generate answers with high confidence, even if they are incorrect. This phenomenon, known as hallucination, poses a challenge for the adoption of language models in the enterprise.
Teams and skills: To effectively manage the vast amount of data in today’s companies, team restructuring is often necessary. A centralized team that handles data for both analytics and machine learning analytics has proven to be an efficient structure. This approach works well for both predictive AI and generative AI.
Security and data privacy: Sharing critical code or proprietary information with a language model can pose security and privacy risks. Vendors can use shared data to update their models, potentially compromising sensitive information and inviting regulatory action.
Predictive AI vs. generative AI considerations: Operationalizing machine learning has been a challenge for many teams. However, generative AI offers advantages over predictive AI in terms of time-to-value. It can provide value across various verticals without the need for extensive training or fine-tuning. Tasks like generating code for web applications can now be done in seconds, saving time and resources.
Future opportunities: Generative AI presents a significant opportunity for innovation, similar to the emergence of cloud technology in 2008. This opens doors for founders to create impactful products as the entire stack is still being built.
Security for AI: Addressing the challenges of securing AI involves preventing malicious actors from manipulating model weights and ensuring that code doesn’t contain hidden vulnerabilities. These attacks can be highly sophisticated and difficult to detect, even for experts.>LLMOps: Integrating generative AI into daily workflows is still a complex challenge for organizations large and small. There is complexity regardless of whether you are chaining together open-source or proprietary LLMs. Then the question of orchestration, experimentation, observability and continuous integration also becomes important when things break. There will be a class of LLMOps tools needed to solve these emerging pain points.
AI agents and copilots for everything: An agent is basically your personal chef, EA and website builder all in one. Think of it as an orchestration layer that adds a layer of intelligence on top of LLMs. These systems can let AI out of its box. For a specified goal like: “create a website with a set of resources organized under legal, go-to-market, design templates and hiring that any founder would benefit from,” the agents would break it down into achievable tasks and then coordinate to achieve the objective.
Compliance and AI guardrails: Regulation is coming. It is just a matter of time before lawmakers around the world draft meaningful guardrails around this disruptive new technology. From training to inference to prompting, there will need to be new ways to safeguard sensitive information when using generative AI.
LLMs are already so good that software developers can generate 60-70% of code automatically using coding copilots. This number is only going to increase in the future. One thing to keep in mind though is that these models can only produce something that’s a derivative of what has already been done. AI can never replace the creativity and beauty of a human brain, which can think of ideas never thought before. So, the code poets who know how to build amazing technology over the weekend will find AI a pleasure to work with and in no way a threat to their careers.
Final thoughts
Generative AI for the enterprise is a phenomenal opportunity for visionary founders to build the FAANG companies of tomorrow. This is still the first innings that is being played out. Large enterprises, SMBs and startups are all figuring out how to benefit from this innovative new technology. Like the California gold rush, it might be possible to build successful companies by selling picks and shovels if the perceived barrier to entry is too high.
