Prof. Shafiq Joty, Salesforce
Title: Unleash the Potential of LLMs through Task and Data Engineering
Abstract:
Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. While the proliferation of LLMs has enhanced numerous applications, a significant number of high-performing models remain proprietary, impeding the progress of scientific exploration. LLMs are also susceptible to hallucinations, generating seemingly credible yet factually inaccurate information that can impact their broad acceptance and integration. In this seminar, I will commence by introducing our newly released Xgen—an advanced open-source LLM with a parameter scale of 7 billion. I will delve into its pre-training process and present its results on standard benchmarks. Subsequently, I will discuss our work involving factual reasoning with LLMs, democratizing them for low-resource languages, and distilling knowledge from a larger (175B) proprietary LLM to a smaller (7B) model in a personalized manner. Finally, I will conclude by addressing some limitations of LLMs, emphasizing that scaling alone might not suffice as a solution and that new innovations are needed to tackle these challenges.