The Aible Augmented Analysis will also show what underlying groups are contributing to these shifts, removing the need to wait on additional information before you take action on important behavior changes. The key insights are presented via end-user-customizable generative AI summaries.

Now, organizations can try out many different combinations of settings, from the selected Language Model, model settings, tone, prompt augmentation, few shot learning, or VectorDB settings to identify the best combinations that maximize end-user satisfaction. They can continuously deploy, compare and monitor improvements to chat apps with such Aible A/Z testing.

We introduced such caching for Natural Language use cases last Halloween. Now caching works for all kinds of questions. Aible now detects either exact or conceptual matches for user questions and retrieves the answer from cache if possible. The organization saves a significant amount of money, while the user gets the question answered instantaneously. Users can easily adjust the cache settings or turn it off for specific questions.

Some organizations prefer small models because they can run fully within their cloud instead of being hosted by the cloud provider. Unfortunately, the operating cost benefits of these models disappear once you need to use a hefty GPU-powered server to run them. For GCP customers, Aible now runs small models like Mistral in Quantized form, serverless without requiring GPUs. This significantly reduces the operating cost per chat, but reduces the overall cost by more than 100X when compared to running such small models on dedicated servers. This feature is currently only available on GCP for a subset of models such as Mistral and Llama. We expect to add more clouds and models soon.

The Aible GenAI dashboard starts the user off with several good questions to riff from. The starting questions can be selected manually by central analytics/IT teams, based on popular questions from peers, or automatically recommended by Aible analysis.

The problem is that most business users don’t know how to read SQL and would have no way to know whether it is correct. Given high hallucination rates for the GenAI models, there is a significant chance that the model misunderstands the question and translates it into the wrong answer. As long as the answer is plausible, the end user has no way to check on the answer short of learning SQL. Aible actually uses two models – one to understand the user’s question and answer it exactly, and the other to generate a SQL query that is slightly broader than the user’s question and includes information that provides additional context to the user to enable them to do a gut-check of the response. In our user testing we found that, for example, if a user asks for revenue from Germany that year and the AI gives a single number answer, it is hard for the user to detect errors. However, if the AI gives the answer but also includes the revenue numbers for Germany for a few more years for context, or includes the revenue numbers from other countries for context, the user is able to spot AI errors much more effectively. In essence, context is key in helping business users detect GenAI hallucinations.

Aible however detects that essentially the same question has been asked recently and responds from a cache without incurring the unnecessary cost. The average response time is also much faster as a result. Of course, users can always turn off the caching response if they want more up to date answers. Others then benefit from the more up to date answers when they ask similar questions.

Aible automatically detects shifts in behavior and frequency of millions of variables to detect why the KPI has shifted during time periods and explains the insights in a narrative customizable for individual business users. Users can then ask follow-on questions to explore the insights further.

Aible immediately leverages an end user’s feedback to improve the model for them via a technique called Few Shot Learning. The user can immediately see the impact of their feedback and adjust the feedback as appropriate to get better results. They can also immediately share their Few Shot Learning improvements with other users. Finally, experts can eventually aggregate the best performing feedback across users to retrain the model for all users.

Users can simply tooltip over the highlighted sections to see exactly where the GenAI got the relevant information from.

Aible automatically detects the document snippets most relevant to their question whatever the type of the original document - and answers the question based on the most relevant content.

The total amount of information sent by the VectorDB (the average length of snippets multiplied by the number of snippets) is actually constrained by the GenAI’s context length or context window which is the total amount of information that can be sent to the GenAI from the VectorDB while leaving room for the user prompt and the response.

For example, if you are trying to answer questions based on a directory, you would want the VectorDB to return short snippets. This is because adjacent entries in a directory (think old school yellow books for example) do not contain relevant information. At the same time, because a directory might contain many examples related to the question, you want to return many individual snippets. The settings would be very different if you are trying to answer questions based on a news article. Here the snippets should be longer because adjacent sections of text typically contain related information but we may only have room for fewer snippets so as to not go over the constraints of the context window.

Aible runs a completely serverless VectorDB such that there can be different available-on-demand VectorDBs for individual use cases. This allows Aible users to use the right settings for each use case without affecting other use cases. Aible chat templates incorporate the best practices VectorDB settings for common use cases.

These can indicate a case where the relevant user is doing something problematic, but can also just mean they have adopted a new best practice others have not learnt yet. Organizations can review user specific anomalous prompt patterns to flag problematic use, recommend training, or promote best practices.

analytics chat template that understands how Salesforce tags custom variables with _c. This enables better user experience because the LLM becomes aware of which variables are custom for the organization. A specific Salesforce customer may then derive further from the general Salesforce analytics chat template to incorporate information specific to their organization, such as the definition of their fiscal quarter. Quite soon you may end up with multiple chat templates derived from each other and you need a mechanism by which changes made in a parent template can percolate down to child templates. The Aible Chat Template Lineage provides that mechanism. It also makes it easy to see the actual lineage of any template just like you can easily see traditional data lineage. This is crucial for proper governance of this crucial genAI capability.

When underlying technology such as LLMs change, we need to update the Chat Templates to compensate for the change so that the use case works better than before, based on the updated tech. The Prompt Augmentation and recommended settings for GPT-3.5 are significantly different from those for GPT-4 for example. So when GPT-4 was released we had to update each of the relevant chat templates to make them work with GPT-4 as well. With the Aible Chat Template Lineage mechanism, we need to typically make such changes only once in the original parent template and all use-case-specific child templates derived from the parent immediately inherit all the improvements and can then start benefiting from the new technology. In the absence of this lineage technology IT organizations will find it really difficult to manually keep use cases updated as underlying technology changes.