Module 6: Modal — Serverless deployment of LLMs
Account and CLI setup
Before you can deploy an LLM to Modal, you need three things working: an account, the CLI installed on your machine, and a token that connects them. In this capsule you'll have all three ready and you'll verify them by running a trivial remote function.
By the end you'll be able to:
- Create your Modal account and understand what the initial credits give you
- Install and authenticate the CLI from your terminal
- Run a Python function in Modal's cloud with a single command
- Diagnose the most common setup errors (token, version, authentication)
Why does this capsule matter?
It's the only capsule in the module that isn't strictly "AI". Setup seems boring, but 90% of the problems you'll see in the following capsules come from a badly done setup: an expired token, an old version of the CLI, a forgotten environment variable.
Doing it right here saves you hours of debugging later.
Step 1 — Create the account
Modal uses authentication with GitHub or Google. There's no traditional email/password flow.
- Open modal.com/signup
- Choose "Sign up with GitHub" (recommended if you plan to use Modal in open-source projects) or "Sign up with Google"
- Authorize access (reading your email; it doesn't ask for private repos)
- Verify the email if it asks you to
When you create the account you receive free credits (as of early 2026 it's $30/month recurring on the free tier, enough for this whole module and personal experiments). The credits renew every calendar month. If you run out, your account doesn't get blocked — you simply can't run more until the next cycle or until you add a payment method.
Visual verification: after signup, you should see the dashboard at https://modal.com/apps (empty, no apps yet).
Step 2 — Install the CLI
Modal is administered from Python. The CLI comes as part of the modal package on PyPI.
Requirements:
- Python 3.10 or higher
piporuvinstalled
Verify Python:
python --version
# You should see Python 3.10.x or higher
If you have Python 3.9 or lower, update before continuing. Modal doesn't support older versions.
Recommended: use a virtual environment. Mixing the global modal with other projects ends badly. Create a venv specific to this module:
mkdir modal-tutorial
cd modal-tutorial
python -m venv .venv
source .venv/bin/activate # macOS/Linux
# .venv\Scripts\activate # Windows PowerShell
Install:
pip install modal
This downloads the modal package (and dependencies like grpclib, synchronicity, typer). It takes about 20-30 seconds.
Verify installation:
modal --version
# modal client version: 0.64.x (or similar)
If the modal command isn't recognized, your venv isn't activated or the installation failed. We'll see troubleshooting at the end.
Step 3 — Authenticate the CLI with a token
The CLI needs a token that proves it's you. Modal generates it by opening the browser and asking you to confirm.
modal token new
This opens a tab in your browser with a URL like https://modal.com/token-flow/.... Confirm "Authorize CLI" and you return to the terminal.
Expected output:
Web authentication finished successfully!
Token written to ~/.modal.toml in profile 'default'.
The ~/.modal.toml file ends up with something like:
[default]
token_id = "ak-XXXXXXXXXXXXXXXXXXXXXX"
token_secret = "as-XXXXXXXXXXXXXXXXXXXXXX"
active = true
Important: this file is a secret. Don't commit it to git. Don't paste it in Slack or in public issues. If you expose it by accident, run modal token rotate to invalidate it and generate a new one.
Step 4 — Your first remote function
Let's test that everything works with a minimal "hello world". It uses no GPU or LLM — that comes in the following capsules. The goal of this capsule is only to verify the setup.
Create hello.py:
# hello.py
import modal
app = modal.App("hello-modal")
@app.function()
def greet(name: str) -> str:
import platform
return f"Hello {name}, greetings from {platform.node()}"
@app.local_entrypoint()
def main():
local_result = greet.local("Mike")
remote_result = greet.remote("Mike")
print("Local:", local_result)
print("Remote:", remote_result)
What each line does:
modal.App("hello-modal")— defines your application's "namespace" in Modal. The name will appear in your dashboard.@app.function()— decorator that marksgreetas executable in Modal's cloud.greet.local(...)— runs the function on your machine, like any Python function.greet.remote(...)— packages the code, sends it to Modal, runs it there, returns the result to you.@app.local_entrypoint()— marksmainas the entry point when you runmodal run hello.py.
Run:
modal run hello.py
Expected output (the first time):
✓ Initialized. View run at https://modal.com/apps/.../hello-modal
✓ Created objects.
├── 🔨 Created mount /Users/.../hello.py
└── 🔨 Created function greet.
✓ App finished.
Local: Hello Mike, greetings from your-laptop.local
Remote: Hello Mike, greetings from modal-container-xxxxxx
Notice two details:
- The hostname changes between local and remote — that confirms the remote version really ran in a Modal container, not on your machine.
- The first execution took a few seconds (10-15s typically). That's the cold start: Modal spun up a container from scratch. If you run
modal run hello.pyagain within the next few minutes, it'll be instant because the container stays "warm" for a while.
Common traps in the setup
Trap 1 — command not found: modal
Your venv isn't activated or the pip install modal ran on another Python. Solutions:
# Confirm which Python is active
which python
# Reactivate the venv if needed
source .venv/bin/activate
# Reinstall
pip install --force-reinstall modal
Trap 2 — Token has expired or unauthenticated
Your token expired or ~/.modal.toml is wrong. Solutions:
modal token new # Generates a new token (overwrites the old one)
# Or if you need to delete and start clean:
rm ~/.modal.toml
modal token new
Trap 3 — You paste the token into an environment variable and it doesn't work
Modal prioritizes the MODAL_TOKEN_ID and MODAL_TOKEN_SECRET variables over ~/.modal.toml. If you put old values in your .bashrc / .zshrc, the CLI uses them even if you regenerated the token. Solutions:
# Check whether they're set
echo $MODAL_TOKEN_ID
echo $MODAL_TOKEN_SECRET
# If they have an old value, unset them
unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET
# And remove them from your shell rc for good
Trap 4 — "Function is taking forever the first time"
It's not an error: it's the cold start building the image. The first time, Modal has to download the base Linux image + dependencies, which can take 30-60s. The following runs reuse the image and are fast. We cover how to minimize this in the capsule 06-autoscaling.md.
Trap 5 — I want to use Modal without installing Python locally
You can't. Modal requires local Python because your source code lives there; what gets "uploaded" to Modal is the code your Python serializes. There's no pure web client.
Verification exercise
Before moving on to capsule 03, prove to yourself that you have the setup complete. Modify hello.py to:
- Add a second parameter
repeat: int = 1togreet - Have it return the greeting concatenated
repeattimes - Call
greet.remote("your-name", repeat=3)frommain
Expect to see your greeting three times in a row.
See solution
# hello.py
import modal
app = modal.App("hello-modal")
@app.function()
def greet(name: str, repeat: int = 1) -> str:
return " ".join([f"Hello {name}!" for _ in range(repeat)])
@app.local_entrypoint()
def main():
print(greet.remote("Ana", repeat=3))
# Output: Hello Ana! Hello Ana! Hello Ana!
If this works, your setup is complete.
Summary
You now have:
- ✅ A Modal account with free credits available
- ✅ The
modalCLI installed in your venv - ✅ An authentication token in
~/.modal.toml - ✅ A Python function running remotely
Checkpoint before moving on: if modal run hello.py returns a greeting and ~/.modal.toml exists with your token, you're ready.
Next capsule
In 03 — Your first serverless function we'll go deeper into what happened "behind the scenes" when you ran greet.remote: how Modal built the image, what the cold start is, and how to add Python dependencies to a remote function. It's the foundation you need before putting LLM models in the next step.
Resources
- Modal — Getting Started — official step-by-step guide.
- Modal — CLI Reference — all available commands.
- Modal — Free tier details — monthly credits and free tier limits.
- Python venv documentation — virtual environments if you never used them.