AI LLM Models pretrained

Here I will explore already trained and ready to use out of the box NN text models.

description artificial nets models with artificial intelligence and Development|lower

READY AI MODELS

1. What types we have: Image classification, Text Generation, Object detection, Text classification, Image segmentation

 

TEXT

LARGE LANGUAGE MODELS

1. Used for text generation, question answering, chat bots, grammar correction

GEMMA

0. If you on CPU use device_map="cpu" otherwise it will take a lot more RAM and time.

1. Usage from disc. Install $ pip install transformers

1.1 Class AutoModelForCausalLM doesn't have from_pretrained method // I need to install PyTorch

from transformers import AutoModelForCausalLM

g = AutoModelForCausalLM.from_pretrained("google/gemma-7b", device_map="auto")

1.2. model.generate(**input_token) takes too long

1.2.1. Try torch_dtype=torch.uint8 // No. Only float

1.2.2. Use torch_dtype=torch.float16 2b instead of bfloat16 (4b) // Doesn't work

2. While g = keras_nlp.models.GemmaCausalLM.from_preset("path") got an error there is no such method in the class.

3. Gemma-b2 is not working actually (maybe I have to change temperature or topK somehow), but gemma-1.1 is ok.

outputs = self.model.generate(**input_ids, max_new_tokens=30, temperature=4 )

4. gemma-it version suited for question answering and chatbots. It could remember dialogs if you use special formatting. And to make chatbot you have to implement algorithm which just collects all history of questions and responses and feed to model all the history every time you generate new answer.

5. Quantization with transformers. // You have to install package $ pip install -q -i https://pypi.org/simple/ bitsandbytes

from transformers import AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(

load_in_4bit = True,

bnb_4bit_quant_type="nf4",

bnb_4bit_compute_dtype=torch.bfloat16,

bnb_4bit_use_double_quant=True )

model = AutoModelForCausalLM.from_pretrained( MODEL, quantization_config=quantization_config )

5.1. Got an error Only GPU is needed for quantization. i've added AutoModelForCausalLM.from_pretrained(device_map="auto") // Got error pip install accelerate is needed

5.2. Try

device = torch.device("cpu")

model = AutoModelForCausalLM.from_pretrained().to(device) // Doesn't help

5.3. bitsandbytes doesn't support CPU. i need another quantization method AQLM

6. To increase performance try: // Nothing

AutoModelForCausalLM.from_pretrained(

ptmodel,

device_map="cpu", 

load_in_4bit=True, 

torch_dtype=torch.float16 )

LLAMA

1. From Meta (Facebook) corp. It is closed commercial network and honestly not the best one. It is not a good idea to spend my time on it. Probably they have trained the model on low IQ facebook posts. It is better to use Mistral which is much faster and precise.

MISTRAL

1. Effective model takes 15 GB RAM. Works with transformers.

T5

1. From Google from 77M to 11.3B parameters.

2. To use T5 I have to install tokenizer SentencePiece which is segmented words // $ pip install sentencepiece

3. This model is way faster then gemma.

4. It could be used for translation purposes but small and large model are bad, maybe xxl is good enough but I didn't test it.

 

Token classification

NER models

1. dbmdz/bert-large-cased-finetuned-conll03-english or huggingface-course/bert-finetuned-ner from HF.

1.1. For transformers use >>> from transformers import AutoModelForTokenClassification

2. What tasks NER could realize

People including fictional

  • Nationalities or religious political groups
  • Buildings, airports, highways, bridges
  • Companies
  • Countries, cities, states
  • Non-GPE locations, mountain ranges, bodies of water
  • Vehicles, weapons, foods
  • Named hurricanes, battles, wars, sports events
  • Titles of books, songs,
  • Named documents made into laws
  • Any named language
  • Absolute or relative dates or periods
  • Times smaller than a day
  • Percentage (including “%”)
  • Monetary values, including unit
  • Measurements such as weight or distance
  • “first”, “second”
  • Numerals that do not fall under another type

 

Summarization

1. To estimate summarization effectiveness ROUGE metrics is used.

2. Use transformer's AutoModelForSeq2SeqLM.from_pretrained(model_from_hf)

3. Example of usage >>> sum = pipeline("summarization", model=hub_model_id)

4. It is possible to generate title or description this way.