# Liquid Foundation Models 2

Liquid Foundation Models 2 (LFM2) are a family of small, open-weight models from Liquid AI focused on quality, speed, and memory efficiency. Liquid AI released text-only LFM2 and text+vision LFM2-VL models.

LFM2 features a new hybrid Liquid architecture with multiplicative gates, short-range convolutions, and grouped query attention, enabling fast training and inference.

This guide shows how to fine-tune both the LFM2 and LFM2-VL models with Axolotl.

Thanks to the team at LiquidAI for giving us early access to prepare for these releases.

## Getting Started

1. Install Axolotl following the installation guide.

   Here is an example of how to install from pip:

       # Ensure you have a compatible version of Pytorch installed
       pip3 install packaging setuptools wheel ninja
       pip3 install --no-build-isolation 'axolotl[flash-attn]>=0.12.0'

2. Run one of the finetuning examples below.

   LFM2

       # FFT SFT (1x48GB @ 25GiB)
       axolotl train examples/LiquidAI/lfm2-350m-fft.yaml

   LFM2-VL

       # LoRA SFT (1x48GB @ 2.7GiB)
       axolotl train examples/LiquidAI/lfm2-vl-lora.yaml

   LFM2-MoE

       pip install git+https://github.com/huggingface/transformers.git@0c9a72e4576fe4c84077f066e585129c97bfd4e6

       # LoRA SFT (1x48GB @ 16.2GiB)
       axolotl train examples/LiquidAI/lfm2-8b-a1b-lora.yaml

### TIPS

- Installation Error: If you encounter `ImportError: ... undefined symbol ...` or `ModuleNotFoundError: No module named 'causal_conv1d_cuda'`, the `causal-conv1d` package may have been installed incorrectly. Try uninstalling it:

      pip uninstall -y causal-conv1d

- Dataset Loading: Read more on how to load your own dataset in our documentation.

- Dataset Formats:

  - For LFM2 models, the dataset format follows the OpenAI Messages format as seen here.
  - For LFM2-VL models, Axolotl follows the multi-content Messages format. See our Multimodal docs for details.

## Optimization Guides

- Optimizations Guide

## Related Resources

- LFM2 Blog
- LFM2-VL Blog
- LFM2-MoE Blog
- Axolotl Docs
- Axolotl GitHub
- Axolotl Discord