Serverless GPU, GPU Grants and Much More 🤯

Matichon Maneegard

May 20, 2025 — 3 min read

Hi Everyone. Mati is Here 👋👋

It have been a while since latest update. (One-click deployment)

Today, I have a very big update about Float16.cloud.

Serverless GPU

Firstly, we are proud to announce our "Serverless GPU" service, powered by H100.

Key Features

Zero code changes required—say goodbye to Docker images 👋
The world's fastest cold start, under 100ms
Deployment mode for AI inference (Please see the examples)
Spot mode for AI training

The Main Differentiator Between Our Serverless GPU and Others

Design Principle, We designed our Serverless GPU to be compatible with traditional server scripts, such as FastAPI server scripts.

Developers don't need to change their code to use our Serverless GPU.

In addition to zero code changes, you can say goodbye to Docker images. While some serverless GPU platforms also offer zero code changes, they require you to ship the Docker image to their server. This process can be very cumbersome and slow down deployment speed. We recognized this problem and have already eliminated this trade-off.

Our service requires only the script and a requirements.txt file to perform the tasks for you. The secret to this process is our base environment, which comes pre-built with the necessary libraries and dependencies, such as:

Transformer Engine (for NVIDIA NeMo and mixed-precision training)
PyTorch-based NVIDIA NGC
NVIDIA Triton Inference Server, NVIDIA Rapids Stack, NVIDIA Curator
Transformers (Hugging Face)
LlamaCPP, ExllamaV2, OpenCV

This setup provides an experience similar to using Google Colab in terms of pre-built dependencies.

Serverless GPU Examples

Examples are essential for getting started with any service. We have created a public GitHub repository to demonstrate how to use our service.

In addition to our "official" examples, we welcome "contributors" who want to demonstrate how to use our Serverless GPU. We also offer GPU Grants for "contributors" (see next section).

GPU Grants

We aim to accelerate impact within the ecosystem, benefiting both researchers and engineers.

We have decided to announce a Request for Grants (RFG) to provide GPU credits for "ANY" type of research. This also covers developers who want to create blog posts, content, benchmarks, etc.