Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scal
Install
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add Orchestra-Research/AI-Research-SKILLs --skill modal -a cursor -y
Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.
Questions
How do I install modal-serverless-gpu?
Download 09-infrastructure/modal/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does modal-serverless-gpu live?
The publisher keeps it at https://github.com/Orchestra-Research/AI-Research-SKILLs.
Which agents can use modal-serverless-gpu?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.
Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.
name: modal-serverless-gpu
description: Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Infrastructure, Serverless, GPU, Cloud, Deployment, Modal]
dependencies: [modal>=0.64.0]
Modal Serverless GPU
Comprehensive guide to running ML workloads on Modal's serverless GPU cloud platform.
When to use Modal
Use Modal when:
Running GPU-intensive ML workloads without managing infrastructure
Deploying ML models as auto-scaling APIs
Running batch processing jobs (training, inference, data processing)
Need pay-per-second GPU pricing without idle costs
# Single GPU
@app.function(gpu="A100")
# Specific memory variant
@app.function(gpu="A100-80GB")
# Multiple GPUs (up to 8)
@app.function(gpu="H100:4")
# GPU with fallbacks
@app.function(gpu=["H100", "A100", "L40S"])
# Any available GPU
@app.function(gpu="any")
Container images
# Basic image with pip
image = modal.Image.debian_slim(python_version="3.11").pip_install(
"torch==2.1.0", "transformers==4.36.0", "accelerate"
)
# From CUDA base
image = modal.Image.from_registry(
"nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04",
add_python="3.11"
).pip_install("torch", "transformers")
# With system packages
image = modal.Image.debian_slim().apt_install("git", "ffmpeg").pip_install("whisper")
Persistent storage
volume = modal.Volume.from_name("model-cache", create_if_missing=True)
@app.function(gpu="A10G", volumes={"/models": volume})
def load_model():
import os
model_path = "/models/llama-7b"
if not os.path.exists(model_path):
model = download_model()
model.save_pretrained(model_path)
volume.commit() # Persist changes
return load_from_path(model_path)
@app.cls(gpu="A100")
class Model:
@modal.enter() # Run once at container start
def load(self):
self.model = load_model() # Load during warm-up
@modal.method()
def predict(self, x):
return self.model(x)
Parallel processing
@app.function()
def process_item(item):
return expensive_computation(item)
@app.function()
def run_parallel():
items = list(range(1000))
# Fan out to parallel containers
results = list(process_item.map(items))
return results
Common configuration
@app.function(
gpu="A100",
memory=32768, # 32GB RAM
cpu=4, # 4 CPU cores
timeout=3600, # 1 hour max
container_idle_timeout=120,# Keep warm 2 min
retries=3, # Retry on failure
concurrency_limit=10, # Max concurrent containers
)
def my_function():
pass
Debugging
# Test locally
if __name__ == "__main__":
result = my_function.local()
# View logs
# modal app logs my-app
Common issues
Issue
Solution
Cold start latency
Increase container_idle_timeout, use @modal.enter()
GPU OOM
Use larger GPU (A100-80GB), enable gradient checkpointing
Download 09-infrastructure/modal/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does modal-serverless-gpu live?
The publisher keeps it at https://github.com/Orchestra-Research/AI-Research-SKILLs.
Which agents can use modal-serverless-gpu?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.