xtuner

/

internlm-7b-qlora-sql

like 0

Model

internlm-7b-qlora-sql is fine-tuned from internlm-7b with sql-create-context dataset by XTuner.

Quickstart

Usage with Model Database libraries

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from transformers.generation import GenerationConfig

tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-7b', trust_remote_code=True)
quantization_config = BitsAndBytesConfig(load_in_4bit=True, load_in_8bit=False, llm_int8_threshold=6.0, llm_int8_has_fp16_weight=False, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type='nf4')
model = AutoModelForCausalLM.from_pretrained('internlm/internlm-7b', quantization_config=quantization_config, device_map='auto', trust_remote_code=True).eval()
model = PeftModel.from_pretrained(model, 'xtuner/internlm-7b-qlora-sql')
gen_config = GenerationConfig(max_new_tokens=512, do_sample=True, temperature=0.1, top_p=0.75, top_k=40)

prompt_template = 'If you are an expert in SQL, please generate a good SQL Query for Question based on the CREATE TABLE statement.\n### Question: {input}\n### Query: '

text = 'CREATE TABLE weather (zip_code VARCHAR, mean_visibility_miles INTEGER)\n找到mean_visibility_miles最大的zip_code。'
inputs = tokenizer(prompt_template.format(input=text), return_tensors='pt')
inputs = inputs.to(model.device)
pred = model.generate(**inputs, generation_config=gen_config)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
"""
SELECT zip_code FROM weather ORDER BY mean_visibility_miles DESC LIMIT 1
""""

Usage with XTuner CLI

Installation

pip install xtuner

Chat

xtuner chat internlm/internlm-7b --adapter xtuner/internlm-7b-qlora-sql --prompt-template sql

Fine-tune

Use the following command to quickly reproduce the fine-tuning results.

xtuner train internlm_7b_qlora_sql_e3