LLM Factory

A Specialized, Customizable, Secure Self-Service Tool

LLM Factory is a self-service tool developed by the University of Kentucky’s Center for Applied Artificial Intelligence (CAAI).
Use LLM Factory to fine-tune your own LLM. Train a model with your own data in a secure environment controlled by the University of Kentucky. Configure a model to meet your needs and produce more accurate and effective AI applications. Cutting edge models like Llama 3 8B and Whisper are accessible via the chat interface or through OpenAI compatible API. Soon, Llama 3.1 70B & the Nomic-Embed embeddings model will be integrated into the system. The platform is designed to be accessible to most industry-standard libraries and tools.

Citation

Read more about the development of LLM Factory in the paper linked below.

How LLM Factory Works

An Introduction to fine-tuning, data security, API Requests, and hardware

CAAI’s LLM Factory operates on the cutting edge of AI advancements, namely Parameter Efficient Fine Tuning (PEFT) and Low Rank Adaption (LoRA). LLM Factory leverages the latest and greatest open-source models, including Llama 3.1 405B and Nemotron-4 340B.

Techniques like Parameter Efficient Fine-Tuning and Low Rank Adaption have revolutionized the way we leverage pre-trained knowledge in LLMs. Layering additional information, called adapters, on top of a base model enables fine-tuning on only a small subset of parameters. Previously, custom model training required significant computational resources. Now, with LoRA methods, we can achieve comparable performance to traditional full fine-tuning, but with significantly reduced memory usage and trainable parameters. This process is more efficient and less expensive. Fine-tuned models developed through LoRA methods are comparable to that of traditional full fine-tuned models, but they are much easier to create and scale.

Building on the efficiency of PEFT and LoRA, we take fine-tuning to the next level by harnessing the power of the latest state-of-the-art models as foundational base models. These models offer an immense capacity for pre-trained knowledge, requiring equally impressive computing capabilities. CAAI uses an on-site NVIDIA DGX computing cluster, boasting 3.2TB of VRAM, that enables us to host a variety of base models and a multitude of adapters. Rather than hosting a bunch of different, separate models, users interface with the system programmatically. This is computationally and cost efficient.

Users have the power to create LLMs that not only are informed on the vast amounts of data that foundational models have been trained on but that also learn from project specific data. With LLM Factory, users have access to cutting-edge models and the ability to customize those models to meet their unique needs. It’s our hope this drives unprecedented achievements and efficient workflows.

Users can easily integrate their fine-tuned models, or base models available through LLM Factory, with OpenAI’s ecosystem. LLM Factory’s API endpoints are OpenAI compatible. This means that all other libraries, tools, and systems that are OpenAI compatible (the industry standard) can be integrated seamlessly with LLM Factory. LLM Factory’s User Guide goes in-depth and covers things like adapter training, tool/function calling, embedding, and transcription. The User Guide also just walks you through how to navigate the platform. You don’t have to be an experienced developer to use LLM Factory.

LLM Factory offers efficient, scalable fine-tuning on a user-friendly platform controlled by the University of Kentucky. Alternative fine-tuning options, like OpenAI or Anthropic, don’t have clear or customizable data restriction policies. With LLM Factory, your data and interactions are secure. Only you and your team members have access to your trained adapters. Data used for fine-tuning, conversation history through the chat interface, and API calls are all secure. Individual, private, HIPAA compliant instances are available by request and evaluated on a needs-based basis. If you would like to learn more, please reach out to ai@uky.edu.

Access

LLM Factory is available through collaboration with CAAI. You must be granted the necessary permissions from a CAAI Administrator in order to access the platform. Please contact us for access, or fill out our collaboration form.

HIPAA Compliance

LLM Factory V1.3 (July 2024) is not HIPAA Compliant. Private, HIPAA compliant instances are available on an individual basis! If you would like to learn more, please reach out to ai@uky.edu

Collaborative Projects using LLM Factory

The following tools are in development: