Background
I maintain an enterprise software ecosystem that has run for over a decade. Over its lifecycle, the project has accumulated roughly 24,000 work items, system notes, code modifications, and thousands of annotated troubleshooting screenshots split between our modular backend, web client and a companion mobile app.
I wanted to see if I could build a conversational technical assistant using a local LLM that could accurately map out the evolution of our software features over time, answer design questions, and trace historical architecture shifts cleanly.
I could not find anything that did what I envisioned. Finding a solution that could handle the overlapping timeline of a software repository without uploading all of our data to a cloud service was more difficult than I expected. I thought this was a standard tool but since I could not find anything, I rolled my own locally using the available GPU. It required writing a custom data pipeline from scratch.
Why Standard RAG Fails on Repositories
Most generic RAG frameworks are built for static data—like a flat directory of PDFs or wiki manuals. They assume all text chunks are equally valid. In an active software repository, the data layout is a multi-user chronological stream.
- The Temporal Problem: Requirements change. A ticket from 2014 might state that a feature works one way, but a ticket from 2016 completely modifies it. Standard vector lookups pull both text segments, see that they contradict, and cause the LLM to hallucinate.
- The Multimodal Disconnect: Developers rarely report complex bugs using plain text alone. They upload screenshots covered in hand-drawn red arrows, highlighting boxes, and sequence numbers. Standard OCR reads the characters but misses the spatial relationships entirely.
To fix this, the pipeline has to shift from simple "semantic word searching" to building a Time-Aware Lifecycle Ledger.
The Architecture Setup
The pipeline is split into a two-phase data processing loop.
- Phase 1 scrapes the repository endpoints sequentially and handles asset extraction.
- Phase 2 chunks, maps, and indexes the records into a high-performance vector store.
- Phase 3 makes the data from the vector database available in a Gemma 4 based chatbot.
[ GitLab REST API ]
│
(Keyset Pagination Stream Loop)
│
▼
[ process_attachment_list() ]
├── .pdf/.docx ──► Local Extraction Handlers
└── .png/.jpeg ──► Qwen 2.5 VL Server (Port 8090)
│
▼
[ ./rag_chunks (JSON Ledgers) ]
│
▼
[ Ingestion Engine (bge-large-1024) ]
│
▼
[ Qdrant Vector Store ]
│
▼
[ Gemma 4 Chatbot ]
1. Keyset pagination & resumable checkpoints
Because the overall project consists of 2 Gitlab projects where each spans 12,000+ items per project, using standard offset pagination load a lot of data into memory. I decided to loop using a strict keyset streaming engine ordered by creation date.
To protect long-running execution windows (which can take upwards of 20 hours (per Gitlab project) when running heavy image transformations (each workitem can have multiple files (images, pdf, doc) and each file can have multiple images), I implemented a state tracker file (rag_progress.json). Every time a issue/workitem finishes processing, its ID and a creation timestamp are saved to disk. If the network drops or the loop breaks, it reads the tracking file on reboot and injects a created_after flag into the REST payload, jumping straight to the exact minute it left off.
2. Extracting visual information from images
To keep the data state perfectly grouped, I embedded our extraction engines directly into the main parsing pass.
When the file download stream hits an image extension (image uploaded to a workitem), it converts the binary buffer into a Base64 string and POSTs it to a parallel instance of llama.cpp running unsloth--Qwen2.5-VL-7B-Instruct-GGUF locally. The vision engine decomposes the layout, decodes the manual drawings, and returns dense markdown text describing what is depicted on the image.
I updated our local PDFHandler and DocxHandler factories to catch embedded images. The classes now unpack the underlying page object layers, save binary image objects to a temporary cache directory, stream them to the llama.cpp running unsloth--Qwen2.5-VL-7B-Instruct-GGUF, and stitch the visual text description below the baseline text paragraphs that referenced them.
3. Deploying Qdrant in Debian WSL2
For a pool of this size (generating roughly 300,000 individual text chunks) I started with ChromaDB but faced some challenges that led me to using a Qdrant database (running in a Docker container in WSL2).
I initialized the Qdrant collection space using a 1024-dimension layout using the high-resolution BAAI/bge-large-en-v1.5 embedding model to ensure the system can map subtle relationships between backend api and frontend clients - web or mobile.
4. Constructing the time-aware prompt reasoner
To prevent the model from getting confused by historical contradictions, a two-step dual-instruction prompt for our main text server running Gemma 4 locally (optimized with single-slot focus --slots 1 and 8-bit quantized attention caches to eliminate latency).
SYSTEM_PROMPT = "
You are an expert technical assistant for the AppinuxCare and dialogevent software repositories.
Your task is a two-step process: First, find the factual answer. Second, format or translate it as requested.
STEP 1: FACTUAL GROUNDING & LOGICAL FUSION (STRICT)
- Look at the project context below to see if it answers the user's question.
- If the core information cannot be found in the context, say exactly: "I don't know based on the provided project files."
TEMPORAL & CUMULATIVE RULES FOR STEP 1:
1. SPECIFICATIONS ARE ADDITIVE: Requirements accumulate over time. If file A defines "Buttons A and B" and file B later states "Add Button C", you must MERGE them. The present answer is "Buttons A, B, and C".
2. CONFLICT RESOLUTION: If a newer specification explicitly modifies an older rule, the newer rule completely replaces the old one for that specific feature.
3. HISTORICAL ANCHORING: If the user explicitly asks about a past era (e.g., "back in 2014"), you must isolate your analysis to the state of the specs at that exact point in time, ignoring future modifications.
STEP 2: TRANSLATION & FORMATTING (FLEXIBLE)
- If the information IS found, you may translate the final answer into another language if requested. Keep raw variable names and SQL schemas in their original syntax.
"
A local conversational message history array was added directly to our payload engine and set "stream": True inside the requests post-processor. This reads incoming network token lines sequentially as they generate in VRAM, allowing the model to type out complex timeline answers onto the terminal immediately.
Results & Summary
By structuring the pipeline this way, I have achieved total local data isolation. If I query the chatbot globally regarding a feature state, it reads the timeline data, combines the fragmented historical issues, and provides a clear description of the current application rules. If I specifically prompt it with "back in 2014, how did this module function?", it executes our historical anchoring override rule, ignores subsequent code commits, and accurately recovers the decade-old specification.
Building it local-first shows that with the right combination of keyset data ingestion, local vision engines, and temporal prompting constraints, it worked better than I expected and was able to contruct a highly accurate description of how features from 2014 have evolved over time into how they look in 2026. All code changes and diffs are part of the knowledge and it is pretty accurate. As with all LLMs, they can occasionally go off the rails, and they do not "understand" things, workitems, commits, code the same way as I (and I guess most other people) do. But still a very useful tool.
