On-device RAG performs document chunking, indexing, retrieval and generation entirely locally, and limited by phone memory and compute, the document scale and retrieval accuracy it can handle are usually weaker than cloud-based solutions.