Neuigkeiten von trion.
Immer gut informiert.

!pip install -U torch
!pip install -U transformers
!pip install -U einops
!pip install -U git+https://github.com/huggingface/accelerate.git

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = 'tiiuae/falcon-7b'

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    'text-generation',
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto"
)

prompt = 'What is The Answer to the Ultimate Question of Life, the Universe, and Everything?'
sequences = pipeline(
  prompt,
  max_length=100,
  do_sample=True,
  top_k=10,
  num_return_sequences=2,
  eos_token_id=tokenizer.eos_token_id
)
for seq in sequences:
    print(f"DeepThought's Result: {seq['generated_text']}")

Das Orakel sagt: What is The Answer to the Ultimate Question of Life, the Universe, and Everything?
The Answer to the Ultimate Question of Life, the Universe, and Everything in a 7-word equation is 42!
If you are a fan of Douglas Adams and his Hitchhiker's Guide to the Galaxy or a fan of the 5th Element or any sort of science fiction or a nerd or a dork, then you know exactly what I'm talking about.
If not
Das Orakel sagt: What is The Answer to the Ultimate Question of Life, the Universe, and Everything?
The answer is...42
The answer is 42.
I know that answer because it was on the back of a hitchhikers guide to the galaxy :)
42
I have been reading a lot of sci-fi books, and 42 is the answer to all the big questions. So, if someone asks, I can answer that question, but I can't explain it.