Inference Endpoints documentation

Create an Endpoint

Join the Model Database community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Create an Endpoint

After your first login, you will be directed to the Endpoint creation page. As an example, this guide will go through the steps to deploy distilbert-base-uncased-finetuned-sst-2-english for text classification.

1. Enter the Model Database Repository ID and your desired endpoint name:

select repository

2. Select your Cloud Provider and region. Initially, only AWS will be available as a Cloud Provider with the `us-east-1` and `eu-west-1` regions. We will add Azure soon, and if you need to test Endpoints with other Cloud Providers or regions, please let us know.

select region

3. Define the [Security Level](security) for the Endpoint:

define security

4. Create your Endpoint by clicking Create Endpoint. By default, your Endpoint is created with a medium CPU (2 x 4GB vCPUs with Intel Xeon Ice Lake) The cost estimate assumes the Endpoint will be up for an entire month, and does not take autoscaling into account.

create endpoint

5. Wait for the Endpoint to build, initialize and run which can take between 1 to 5 minutes.

overview

6. Test your Endpoint in the overview with the Inference widget 🏁 🎉!

run inference

←Access the solution (UI) Send Requests to Endpoints→