Model Variations And File Formats

LLaMA and Llama-2 Local Hardware Requirements

Model Variations and File Formats

LLaMa-2 offers several model variations with different file formats, including GGML, GGUF, GPTQ, and HF.

Hardware Requirements

Running LLaMA and Llama-2 locally requires significant hardware capabilities:

LLaMA: Requires a minimum of 4 NVIDIA A100 GPUs or equivalent with 16GB VRAM each.

Llama-2: Hardware requirements vary depending on the model variation:

• Llama-2-13b-chatggmlv3q8_0bin: 4 NVIDIA A100 GPUs or equivalent with 80GB VRAM
• Llama-2-67b-chatggmlv3q8_0bin: 8 NVIDIA A100 GPUs or equivalent with 160GB VRAM
• Llama-2-13b-chatgptq_0bin: 4 NVIDIA A100 GPUs or equivalent with 80GB VRAM

Open Source and Free for Research and Commercial Use

Both LLaMA and Llama-2 are open source and available for free research and commercial use.

Dell Infohub

Github