Transformer Neural Networks
Summary:
Large Language Models
How to download LLAMA models?
2. You may need access in both HuggingFace and Meta Website. 3. Once approved you receive email from both (normall its quick within minutes) 4. Specially with email from meta, you get the link to download. 5. Then just follow, these steps– https://github.com/meta-llama/llama-models/blob/main/README.md Links: In Google colab: !cp -r /root/.llama/checkpoints/Llama-2-7b /content/drive/MyDrive/
LoRA
What is Lora? How Fine-Tuning works?
Example code: Fine-tunning LLM using LoRA
Best resources:
Building Large Language Models (LLMs)
Pre-Training. Post-Training. Language Modeling. P(the, mouse, ate, the, cheese) = 0.02 – syntactic knowledgeP(the, the, mouse, ate,cheese) = 0.0001 – semantic knowledgeP(…) Auto-Reggressive (AR) language model: Predict next word. Steps: she likely prefers: tokenize -> 1 -she, 2-likely, 3-prefers => pass to blackbox model => get probability distribution over next word prediction – sample & ... Read More
Fine Tuning LLMs
Requirements: Challenges: Parameter Tuning: References:
What is Transformer?
Used by: Primary Innovations: Key Terms: Visual Explaination: Example:https://aws.amazon.com/what-is/transformers-in-artificial-intelligence/ References: