آموزش استقرار مدل‌های زبانی بزرگ (LLM) در محیط عملیاتی: vLLM، FastAPI، Modal و چت‌بات هوشمند - آخرین آپدیت

سرفصل ها و درس ها

Basic Building Blocks: Building up Modal Platform Understanding

• Course Repository

• Start Strong with Modal: Environment, Installation, and API Setup

• Basics of Python Scripts in Modal: From Local Testing to Remote Deployment

• Ephemeral Apps in Modal: Deployment, Invocation, and Lifecycle Management

• Deployment Basics: Setting Up Infrastructure and Exploring Local vs. Remote Runs

• Building Web APIs with Modal: FastAPI Integration and REST Endpoint Deployment

• Class-Based Deployment: Lifecycle Hooks, Resources and Dynamic Management

Practical Example with Weights Volume Mapping in a Real World Application

• Configuring Modal for AI: Secrets, Authentication and Environment Setup

• Infrastructure as Code for LLM Deployment: Container Setup, GPUs, and APIs

• Efficient Deployment and Runtime Management: Full Workflow, Containers Cron Jobs

• Model Deployment Best Practices: Saving and Loading Models Using Modal Volumes

• The Role of Volume Mapping in Application Containers

• Preparation for Production Deployment

vLLM Inferencing Endpoint with Chatbot, Volumes and Bot Memory App Example

• How to map storage volumes to efficiently manage model weghts

• Understanding VLLM Framework and Volume Management

• How to integrate APIs using FastAPI and vLLM for serving machine learning model

• Creating a FastAPI application and defining API routes

• Utilizing VLLM for efficient model loading and weight management

• Conducting chat sessions through API interactions to test model responses in rea

• Developing a local Chat App to interface with a cloud-deployed language model