“Techniques like Item Response Theory from AWS promises to help with one of the more tricky aspects of RAG, measuring the effectiveness of the information retrieved before sending it to the model,” Shimmin said, adding that with such optimizations at the ready, enterprises can better optimize their inferencing overhead by sending the best information to a model rather than throwing everything at the model at once.
On the other hand, model size is only one factor influencing the performance of foundation models, Forrester’s Dai said.
“Enterprises should take a systematic approach for foundation model evaluation, spanning technical capabilities (model modality, model performance, model alignment, and model adaptation), business capabilities (open source support, cost-effectiveness, and local availability), and ecosystem capabilities (prompt engineering, RAG support, agent support, plugins and APIs, and ModelOps),” Dai explained.
