Serverless GPU, GPU Grants and Much More 🤯
Hi Everyone. Mati is Here 👋👋
It have been a while since latest update. (One-click deployment)
Today, I have a very big update about Float16.cloud.
Serverless GPU
Firstly, we are proud to announce our "Serverless GPU" service, powered by H100.
Key Features
- Zero code changes required—say goodbye to Docker images 👋
- The world's fastest cold start, under 100ms
- Deployment mode for AI inference (Please see the examples)
- Spot mode for AI training
The Main Differentiator Between Our Serverless GPU and Others
Design Principle, We designed our Serverless GPU to be compatible with traditional server scripts, such as FastAPI server scripts.
Developers don't need to change their code to use our Serverless GPU.
In addition to zero code changes, you can say goodbye to Docker images. While some serverless GPU platforms also offer zero code changes, they require you to ship the Docker image to their server. This process can be very cumbersome and slow down deployment speed. We recognized this problem and have already eliminated this trade-off.
Our service requires only the script and a requirements.txt file to perform the tasks for you. The secret to this process is our base environment, which comes pre-built with the necessary libraries and dependencies, such as:
- Transformer Engine (for NVIDIA NeMo and mixed-precision training)
- PyTorch-based NVIDIA NGC
- NVIDIA Triton Inference Server, NVIDIA Rapids Stack, NVIDIA Curator
- Transformers (Hugging Face)
- LlamaCPP, ExllamaV2, OpenCV
This setup provides an experience similar to using Google Colab in terms of pre-built dependencies.
Serverless GPU Examples
Examples are essential for getting started with any service. We have created a public GitHub repository to demonstrate how to use our service.
In addition to our "official" examples, we welcome "contributors" who want to demonstrate how to use our Serverless GPU. We also offer GPU Grants for "contributors" (see next section).
float16-cloudExamples and Contribute template
Details about Serverless GPU
GPU Grants
We aim to accelerate impact within the ecosystem, benefiting both researchers and engineers.
We have decided to announce a Request for Grants (RFG) to provide GPU credits for "ANY" type of research. This also covers developers who want to create blog posts, content, benchmarks, etc.
Discords link : https://discord.gg/j2DVTMjr67
Call for Partners
We are looking for partners in 3 categories:
Consultant Agencies
How we can help:
- Use cases
- Reference architecture
- Demo tools
Software Houses and System Integrators (SI)
How we can help:
- GPU credits
- Reference architecture
- Developer relations
Learning Platforms
How we can help:
- Matching discounts
- Course outlines
- Domain expertise
Please direct contact to me : matichon[dot]man[at]float16[dot]cloud
Trust center
We already accomplish SOC2 and Trust Center with Vanta.
Contact Float16
- Medium : Float16.cloud
- Facebook : Float16.cloud
- X : Float16.cloud
- Discord : Float16.cloud
- Youtube : Float16.cloud
