Google Colab CLI Hands-On: Free A100 Runs Gemma4 Inference at 179 tok/s

Google recently folded Colab's advanced compute into the Google AI Pro subscription, so A100/H100 are no longer exclusive to enterprise customers. A pile of similarly named CLI tools has popped up on PyPI, but only one actually works. This article documents the full process from installation, authentication, and instance creation through running Gemma4 inference — and works out exactly how long 200 compute units will last.

Two CLI Tools — Don't Install the Wrong One

There's an old package on PyPI called colab-cli (version 2.2.5) that has nothing to do with Google. Installing it force-downgrades click to 7.1.2 and typer to 0.1.1, wrecking your environment. A look at its source reveals it's just a small upload utility from the pydrive of years ago — it syncs a local .ipynb to Google Drive and opens a browser tab. That's all it does.

The real official tool is google-colab-cli, released in 2026 (currently 0.7.4, requires Python 3.12+). Install it with uv:

uv tool install --python 3.12 google-colab-cli

After installing, run colab --help and you'll see a full set of modern commands: new, exec, ssh, usage, log, and more. The google- prefix in the name is the only distinguishing marker — look carefully when searching.

Why are there two tools with the same name? Once a package name is registered on PyPI, it's occupied long-term — it isn't automatically released even if the original project stops being maintained. The maintainer of colab-cli wrote that upload script years ago to sync local Jupyter Notebooks to Google Drive. It has no connection to Google, but because it grabbed the name early, it ranks higher in search results, creating an obvious "name trap" for new users unfamiliar with the ecosystem. Google didn't release an official CLI until 2026, so the package name was forced to add a google- prefix to distinguish itself. The way to tell an official tool is simple: check whether the package homepage links to a google.dev or colab.research.google.com domain (Google's official developer tool domain is now unified under google.dev), and whether the installed command is colab rather than colab-cli.

What uv tool install means: uv is a Python package manager written in Rust by Astral. Its tool install subcommand installs a CLI and its dependencies into an isolated environment, without polluting the current project's site-packages. This is especially important here — if you pip install directly, google-colab-cli could conflict with the click/typer versions already in your project, repeating the colab-cli disaster. The --python 3.12 flag forces the 3.12 interpreter, because the official CLI depends on several type hint syntax features added in 3.12.

OAuth Authentication and A100 Instance Creation

The first run of colab usage triggers Google OAuth authorization. The terminal prints an authorization URL; log into the browser with the account subscribed to Google AI Pro, approve the permissions, and paste the authorization code back. After successful authentication, you can see your initial balance:

Current balance: 200.00 compute units
Usage rate: 0.00/hr
Active assignments: 0

Then request an A100 instance with a single command:

colab new --session gemma-test --gpu A100

About 5 seconds later the terminal returns [colab] Session READY.. That's much faster than clicking through the web console, and the advantage is especially obvious when you're repeatedly starting and stopping instances.

Google OAuth Authorization Code Flow is the authorization model Google designed for CLI-type applications that can't host a callback server. It splits the standard OAuth "redirect to get a code" into two steps: the app first requests that Google generate a one-time authorization link; after the user logs in and agrees to the permissions in the browser, the page displays a short-lived authorization code like 4/0AX4Xf...; the user manually pastes it back into the terminal, and the app uses that code to exchange for a refresh token. The whole flow corresponds to OAuth 2.0's Authorization Code Grant. The token is usually valid for 1 hour, while the refresh token is long-lived. Note that the authorization link is one-time — if you refresh the browser tab midway, you must re-run the command to generate a new link, otherwise you'll get an invalid_grant error. The authorization scopes typically include colab.research.google.com and cloud-platform, and you can revoke them at any time on your Google account's "Security > Third-party apps" page.

A100 and the relativity of "old card": The A100 is based on the Ampere architecture (released in 2020), a 7nm process. The 40GB HBM2e version has about 1555 GB/s of memory bandwidth and roughly 156 TFLOPS of TF32 compute. Compared with the newer H100 (Hopper, HBM3 at about 3.35 TB/s bandwidth), there is indeed a generational gap, but in 2026 it's still one of the most cost-effective platforms for inference and fine-tuning. 40GB of VRAM leaves plenty of context cache room for a Q4_K_M quantized 26B model (roughly 15–20 GB of weights). The name gpu-a100-s-kkb-usc1f1 here contains three layers of information: a100-s indicates an A100 standard slice, usc1f1 is the codename for Google's data center (US Central zone 1, facility 1), and the random string at the end is this session's instance ID. Common slices in third-party Colab environments also include a100-l, t4, l4, and so on. The CLI's hardware subcommand lists the hardware granted to the current account.

Gemma4 Inference Benchmark: 100% GPU Load, 179 tok/s

The A100 may be an old card, but 40GB of VRAM is more than enough for medium-sized models. Using colab exec, deploy Ollama on the remote instance and pull gemma4:26b (GGUF Q4_K_M) for testing.

First check the VRAM allocation:

nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv
# Output: NVIDIA A100-SXM4-40GB, 40960 MiB, 20072 MiB

The model occupies 20,072 MiB of VRAM, GPU load is 100%, and the host side still has 83.5 GB RAM and 20 GB of VRAM buffer. Input a clinical genetics question:

What is the role of BRCA1 in breast cancer? Give a 2-sentence clinical summary.

After the model triggers a chain of thought and completes about 270 tokens of reasoning, it produces a well-structured two-sentence conclusion (BRCA1 repairs double-strand DNA breaks through homologous recombination, and pathogenic mutations significantly increase the risk of hereditary breast and ovarian cancer). Performance data:

• Generation time: 2.16 seconds (369 tokens)
• Generation speed: 179.42 tok/s
• Context window: Ollama default 32,768 tokens

That speed is quite impressive for a quantized 26B model, and the A100's memory bandwidth advantage becomes even more pronounced in long-context scenarios.

Instance Monitoring: Six Ways to See Remote Status

The Colab CLI provides a complete set of monitoring commands, from point balance to GPU physical sensors.

Compute billing status (colab usage):

Current balance: 197.28 compute units
Usage rate: 5.30/hr
Active assignments: 1

From 200 points down to 197.28 points, the whole process took about 30 minutes, with the deduction rate locked at 5.30 points/hour.

Instance lifecycle (colab status -s gemma-test):

[gemma-test] gpu-a100-s-kkb-usc1f1-ryarjq9kw3qd | Hardware: A100 | Shape: Standard | Variant: GPU | Status: IDLE
  Last Execution: stdin at 2026-10-12 08:26:15

Status is RUNNING while a task runs, and switches to IDLE when idle.

Hardware-level monitoring (colab exec -s gemma-test "nvidia-smi") returns:

• Temperature and power: 32°C, 47.19W at idle, spiking above 180W under inference (hardware limit 400W)
• VRAM: 20,072 MiB / 40,960 MiB (only 6 MiB at idle)
• Host: 12-core CPU, 83.47 GB RAM, 235.68 GB NVMe disk

Service-layer monitoring (colab exec -s gemma-test "ollama ps"):

NAME          ID              SIZE      PROCESSOR    CONTEXT    RUNNER
gemma4:26b    001e5dafc3c7    1.2 GB    100% GPU     32768      llamacpp

By default, Ollama automatically releases VRAM after 5 minutes of idle time, avoiding persistent background resource usage.

Execution history export is one of the official CLI's most valuable features. Run:

colab log -s gemma-test -o colab_a100_gemma4_replay.ipynb

All input, output, and logs from the half hour are automatically packaged into a 728 KB standard .ipynb file. Anyone opening it can see the output of each cell, fully reproducing the experiment. This is extremely practical for scenarios where you need to keep an experiment record or share results.

Real-time web dashboard: get the dedicated Notebook address with colab url -s gemma-test. Open it in a browser and the top-right corner shows a bar chart of hardware resources (system RAM, disk, GPU RAM), letting you directly watch 20 GB of VRAM being carved out in real time.

Point Pricing and the Pay As You Go Mechanism

Type colab pay in the terminal, or click "Learn more" on the Resources tab in the Colab interface, and the official Colab paid plans and pricing page opens.

Google's Colab compute billing has two layers:

Subscription — Google AI Pro / Colab Pro distribute a fixed allotment of compute points each month (Google AI Pro grants 200 points), unlock higher-tier hardware quotas (A100, TPU v5e, etc.), and give you higher queue priority than the free tier.

Pay As You Go add-ons — When 200 points are used up by heavy fine-tuning, you can buy one-off add-ons (100 points for about $9.99). Add-on points don't expire as long as your account is active, and are deducted automatically after your monthly free allowance.

Converted to real time:

200 points ÷ 5.30 points/hour ≈ 37.7 hours

Continuous running lasts 37.7 hours. If each test only takes 20 minutes, 200 points is enough for more than 100 complete inference sessions.

The most point-efficient operation is the Ephemeral Job Runner the CLI provides:

colab run -s gemma-eval --gpu A100 my_benchmark.py

It boots in the cloud, runs your script, retrieves the output, and immediately shuts down automatically to release resources — you don't even need to manually colab stop, so not a single second of points is wasted.

Who This Combination Is For

If your Colab usage is "occasionally running small models, doing a benchmark, or validating an inference script from a paper" rather than long-term training of large models, then google-colab-cli plus the 200 compute points from Google AI Pro is currently a highly cost-effective option. Terminal operation means it can plug into scripts, CI pipelines, or other automation tools, and the reproducible notebook exported by colab log fills in the last piece of the experiment-record puzzle.

It's certainly not for everyone. Tasks that need long continuous training or multi-GPU parallelism still belong on a proper cloud platform. But for individual developers and small teams, the 37 hours of A100 time that 200 points buys already covers most validation work.

One-line Summary

Hands-on test: google-colab-cli under a Google AI Pro subscription spins up an A100 to run Gemma4 inference at 179 tok/s, with 200 points lasting 37.7 hours.

Paragraph Summary

This article documents the full process of running Gemma4 inference from scratch using the official google-colab-cli under a Google AI Pro subscription. It first clarifies that PyPI has a misleading same-named package, colab-cli (actually an old pydrive upload tool that downgrades dependencies and breaks the environment), while the real official tool is google-colab-cli (0.7.4, requires Python 3.12+). It then covers OAuth authorization, creating an instance in about 5 seconds with colab new --gpu A100, deploying Ollama remotely to pull gemma4:26b GGUF Q4_K_M, and measuring a generation speed of 179.42 tok/s with 20 GB of VRAM usage. The article also walks through six monitoring methods — usage, status, nvidia-smi, ollama ps, log, url — and works out the billing model: 200 points deducted at 5.30 points/hour, equivalent to about 37.7 hours of continuous A100 use, with Pay As You Go add-ons at 100 points for about $9.99.