Model Database's logo

PEFT documentation

Models

Model Database's logo
Join the Model Database community

and get access to the augmented documentation experience

to get started

Models

PeftModel is the base model class for specifying the base Transformer model and configuration to apply a PEFT method to. The base PeftModel contains methods for loading and saving models from the Hub, and supports the PromptEncoder for prompt learning.

PeftModel

class peft.PeftModel

< >

( model: PreTrainedModel peft_config: PeftConfig adapter_name: str = 'default' )

Parameters

  • model (PreTrainedModel) — The base transformer model used for Peft.
  • peft_config (PeftConfig) — The configuration of the Peft model.
  • adapter_name (str) — The name of the adapter, defaults to "default".

Base model encompassing various Peft methods.

Attributes:

  • base_model (PreTrainedModel) — The base transformer model used for Peft.
  • peft_config (PeftConfig) — The configuration of the Peft model.
  • modules_to_save (list of str) — The list of sub-module names to save when saving the model.
  • prompt_encoder (PromptEncoder) — The prompt encoder used for Peft if using PromptLearningConfig.
  • prompt_tokens (torch.Tensor) — The virtual prompt tokens used for Peft if using PromptLearningConfig.
  • transformer_backbone_name (str) — The name of the transformer backbone in the base model if using PromptLearningConfig.
  • word_embeddings (torch.nn.Embedding) — The word embeddings of the transformer backbone in the base model if using PromptLearningConfig.

create_or_update_model_card

< >

( output_dir: str )

Updates or create model card to include information about peft:

  1. Adds peft library tag
  2. Adds peft version
  3. Adds quantization information if it was used

disable_adapter

< >

( )

Disables the adapter module.

forward

< >

( *args: Any **kwargs: Any )

Forward pass of the model.

from_pretrained

< >

( model: PreTrainedModel model_id: Union[str, os.PathLike] adapter_name: str = 'default' is_trainable: bool = False config: Optional[PeftConfig] = None **kwargs: Any )

Parameters

  • model (PreTrainedModel) — The model to be adapted. The model should be initialized with the from_pretrained method from the 🤗 Transformers library.
  • model_id (str or os.PathLike) — The name of the PEFT configuration to use. Can be either:
    • A string, the model id of a PEFT configuration hosted inside a model repo on the Model Database Hub.
    • A path to a directory containing a PEFT configuration file saved using the save_pretrained method (./my_peft_config_directory/).
  • adapter_name (str, optional, defaults to "default") — The name of the adapter to be loaded. This is useful for loading multiple adapters.
  • is_trainable (bool, optional, defaults to False) — Whether the adapter should be trainable or not. If False, the adapter will be frozen and use for inference
  • config (PeftConfig, optional) — The configuration object to use instead of an automatically loaded configuation. This configuration object is mutually exclusive with model_id and kwargs. This is useful when configuration is already loaded before calling from_pretrained. kwargs — (optional): Additional keyword arguments passed along to the specific PEFT configuration class.

Instantiate a PEFT model from a pretrained model and loaded PEFT weights.

Note that the passed model may be modified inplace.

get_base_model

< >

( )

Returns the base model.

get_nb_trainable_parameters

< >

( )

Returns the number of trainable parameters and number of all parameters in the model.

get_prompt

< >

( batch_size: int task_ids: Optional[torch.Tensor] = None )

Returns the virtual prompts to use for Peft. Only applicable when peft_config.peft_type != PeftType.LORA.

get_prompt_embedding_to_save

< >

( adapter_name: str )

Returns the prompt embedding to save when saving the model. Only applicable when peft_config.peft_type != PeftType.LORA.

print_trainable_parameters

< >

( )

Prints the number of trainable parameters in the model.

save_pretrained

< >

( save_directory: str safe_serialization: bool = False selected_adapters: Optional[List[str]] = None **kwargs: Any )

Parameters

  • save_directory (str) — Directory where the adapter model and configuration files will be saved (will be created if it does not exist).
  • kwargs (additional keyword arguments, optional) — Additional keyword arguments passed along to the push_to_hub method.

This function saves the adapter model and the adapter configuration files to a directory, so that it can be reloaded using the PeftModel.from_pretrained() class method, and also used by the PeftModel.push_to_hub() method.

set_adapter

< >

( adapter_name: str )

Sets the active adapter.

PeftModelForSequenceClassification

A PeftModel for sequence classification tasks.

class peft.PeftModelForSequenceClassification

< >

( model peft_config: PeftConfig adapter_name = 'default' )

Parameters

Peft model for sequence classification tasks.

Attributes:

  • config (PretrainedConfig) — The configuration object of the base model.
  • cls_layer_name (str) — The name of the classification layer.

Example:

>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForSequenceClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "SEQ_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForSequenceClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117

PeftModelForTokenClassification

A PeftModel for token classification tasks.

class peft.PeftModelForTokenClassification

< >

( model peft_config: PeftConfig = None adapter_name = 'default' )

Parameters

Peft model for token classification tasks.

Attributes:

  • config (PretrainedConfig) — The configuration object of the base model.
  • cls_layer_name (str) — The name of the classification layer.

Example:

>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForTokenClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "TOKEN_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForTokenClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForTokenClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117

PeftModelForCausalLM

A PeftModel for causal language modeling.

class peft.PeftModelForCausalLM

< >

( model peft_config: PeftConfig adapter_name = 'default' )

Parameters

Peft model for causal language modeling.

Example:

>>> from transformers import AutoModelForCausalLM
>>> from peft import PeftModelForCausalLM, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "CAUSAL_LM",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 1280,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 20,
...     "num_layers": 36,
...     "encoder_hidden_size": 1280,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForCausalLM.from_pretrained("gpt2-large")
>>> peft_model = PeftModelForCausalLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 1843200 || all params: 775873280 || trainable%: 0.23756456724479544

PeftModelForSeq2SeqLM

A PeftModel for sequence-to-sequence language modeling.

class peft.PeftModelForSeq2SeqLM

< >

( model peft_config: PeftConfig adapter_name = 'default' )

Parameters

Peft model for sequence-to-sequence language modeling.

Example:

>>> from transformers import AutoModelForSeq2SeqLM
>>> from peft import PeftModelForSeq2SeqLM, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "SEQ_2_SEQ_LM",
...     "inference_mode": False,
...     "r": 8,
...     "target_modules": ["q", "v"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.1,
...     "fan_in_fan_out": False,
...     "enable_lora": None,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> peft_model = PeftModelForSeq2SeqLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 884736 || all params: 223843584 || trainable%: 0.3952474242013566

PeftModelForQuestionAnswering

A PeftModel for question answering.

class peft.PeftModelForQuestionAnswering

< >

( model peft_config: PeftConfig = None adapter_name = 'default' )

Parameters

Peft model for extractive question answering.

Attributes:

  • config (PretrainedConfig) — The configuration object of the base model.
  • cls_layer_name (str) — The name of the classification layer.

Example:

>>> from transformers import AutoModelForQuestionAnswering
>>> from peft import PeftModelForQuestionAnswering, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "QUESTION_ANS",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForQuestionAnswering.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForQuestionAnswering(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 592900 || all params: 108312580 || trainable%: 0.5473971721475013

PeftModelForFeatureExtraction

A PeftModel for getting extracting features/embeddings from transformer models.

class peft.PeftModelForFeatureExtraction

< >

( model peft_config: PeftConfig = None adapter_name = 'default' )

Parameters

Peft model for extracting features/embeddings from transformer models

Attributes:

Example:

>>> from transformers import AutoModel
>>> from peft import PeftModelForFeatureExtraction, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "FEATURE_EXTRACTION",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }
>>> peft_config = get_peft_config(config)
>>> model = AutoModel.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForFeatureExtraction(model, peft_config)
>>> peft_model.print_trainable_parameters()
←IA3 Configuration→