HAMMAD YOUSUF

FREE RESOURCE · FROM THE "EDGE" REEL

Edge0 local AI: setup, memory limits and mobile options

By · 8 November 2026

Edge0 local AI streams selected model experts from storage and uses a trained router to predict which experts come next. Start with the smaller model, then measure memory, speed and answer quality on your own device.

Your first run

  1. 1. Check the platform and model files

    The Python route needs Apple Silicon and macOS, with Python 3.10 or newer; upstream recommends Python 3.12. Read the current README before installing. Leave space for the model and its adapter files: roughly 4.2 GB for the smaller 8B checkpoint and 23 GB for 35B.

  2. 2. Install the Python development route

    Clone the official repository and create a separate environment. The commands below follow the upstream Python route, not an app-store installer.

    step 2
    git clone https://github.com/Edge0-AI/Edge0.git
cd Edge0/python
python3.12 -m venv .venv
.venv/bin/pip install -e '.[dev,fetch]'

  3. 3. Fetch the smaller model for a first test

    This downloads model files. Check available storage and the model licence, then keep the model, prerouter and LoRA adapter files together.

    step 3
    .venv/bin/python scripts/fetch_models.py --tier edge0-8b --target-dir models

  4. 4. Run a bounded demo

    Begin with a short prompt and short output. Record load time, tokens per second and total process memory separately. Inspect the answer before increasing context length.

    step 4
    .venv/bin/python -m edge0 demo models/edge0-8b

  5. 5. Choose a mobile route only after checking requirements

    The Android guide lists 8 GB+ phone RAM for 8B, 12–16 GB for 35B and at least 30 GB free for model sources and converted builds. The iOS route uses an Xcode project and model-pack workflow. These are developer setup paths, with device-specific limits.

What the memory headline actually measures

Active allocation
The 2.9 GiB figure is peak MLX allocation in the published benchmark. Operating-system memory, file-backed pages, tokenizer, caches and context also need RAM.
Storage streaming
Only selected experts enter active memory while the other weights remain on storage. Disk space and read speed still matter.
Predictive routing
A trained prerouter predicts upcoming experts so loading can overlap computation. This is the distinction from our earlier general storage-offload explainer.
Model-specific quality
The report describes average evaluation results. It does not promise byte-identical output to an ordinary full-precision run.
Free engine code
Edge0 is Apache-2.0 code. Hardware, storage and model-specific licences remain part of your setup.

Review before real work

  • Confirm the model tier, adapter files and device requirements before downloading.
  • Measure total process memory separately from the active allocation headline.
  • Increase prompt and context size gradually; record whether speed and output quality remain useful.
  • Use the upstream README when commands or dependencies change.

About the video: The EDGE reel explains active-memory streaming and predictive routing. Use this guide for the installation steps and device limits. Documentation checked on 8 November 2026.

Keep going