Model Database's logo
Join the Model Database community

and get access to the augmented documentation experience

to get started

Add custom Dependencies

Inference Endpoints’ base image includes all required libraries to run inference on 🤗 Transformers models, but it also supports custom dependencies. This is useful if you want to:

  • customize your inference pipeline and need additional Python dependencies
  • run a model which requires special dependencies like the newest or a fixed version of a library (for example, tapas (torch-scatter)).

To add custom dependencies, add a [requirements.txt](https://huggingface.co/philschmid/distilbert-onnx-banking77/blob/main/requirements.txt) file with the Python dependencies you want to install in your model repository on the Model Database Hub. When your Endpoint and Image artifacts are created, Inference Endpoints checks if the model repository contains a requirements.txt file and installs the dependencies listed within.

optimum[onnxruntime]==1.2.3
mkl-include
mkl

Take a look at the requirements.txt files in the following model repositories as an example:

For more information, take a look at how you can create and install dependencies when you use your own custom container for inference.