Sharang Parnerkar
7e12d1433a
Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com> Reviewed-on: #4
80 lines
3.2 KiB
Markdown
80 lines
3.2 KiB
Markdown
# AI Chat (RAG)
|
|
|
|
The AI Chat feature lets you ask natural language questions about your codebase. It uses Retrieval-Augmented Generation (RAG) to find relevant code and provide accurate, source-referenced answers.
|
|
|
|
## How It Works
|
|
|
|
1. **Code graph** is built for the repository (functions, classes, modules)
|
|
2. **Embeddings** are generated for each code symbol using an LLM embedding model
|
|
3. When you ask a question, your query is **embedded** and compared against code embeddings
|
|
4. The **top 8 most relevant** code snippets are retrieved
|
|
5. These snippets are sent as context to the LLM along with your question
|
|
6. The LLM generates a response **grounded in your actual code**
|
|
|
|
## Getting Started
|
|
|
|
### 1. Select a Repository
|
|
|
|
Navigate to **AI Chat** in the sidebar. You'll see a grid of repository cards. Click one to open the chat interface.
|
|
|
|
### 2. Build Embeddings
|
|
|
|
Before chatting, you need to build embeddings for the repository:
|
|
|
|
1. Click **Build Embeddings**
|
|
2. Wait for the process to complete — a progress bar shows `X/Y chunks`
|
|
3. Once the status shows **Embeddings ready**, the chat input is enabled
|
|
|
|
::: info
|
|
Embedding builds require:
|
|
- A code graph already built for the repository (via the Graph feature)
|
|
- A configured embedding model (`LITELLM_EMBED_MODEL`)
|
|
|
|
The default model is `text-embedding-3-small`.
|
|
:::
|
|
|
|
### 3. Ask Questions
|
|
|
|
Type your question in the input area and press Enter (or click Send). Examples:
|
|
|
|
- "How does authentication work in this codebase?"
|
|
- "What functions handle database connections?"
|
|
- "Explain the error handling pattern used in this project"
|
|
- "Where are the API routes defined?"
|
|
- "What does the `process_scan` function do?"
|
|
|
|
## Understanding Responses
|
|
|
|
### Answer
|
|
|
|
The AI response is a natural language answer to your question, grounded in the actual source code of your repository.
|
|
|
|
### Source References
|
|
|
|
Below each response, you'll see source references showing exactly which code was used to generate the answer:
|
|
|
|
- **Symbol name** — The qualified name of the function/class/module
|
|
- **File path** — Where the code is located, with line range
|
|
- **Code snippet** — The first ~10 lines of the relevant code
|
|
- **Relevance score** — How closely the code matched your question (0.0 to 1.0)
|
|
|
|
## Conversation Context
|
|
|
|
The chat maintains conversation history within a session. You can ask follow-up questions that reference previous answers. The system sends the last 10 messages as context to maintain coherence.
|
|
|
|
## Configuration
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| `LITELLM_URL` | LiteLLM proxy URL | `http://localhost:4000` |
|
|
| `LITELLM_API_KEY` | API key for the LLM provider | — |
|
|
| `LITELLM_MODEL` | Model for chat responses | `gpt-4o` |
|
|
| `LITELLM_EMBED_MODEL` | Model for code embeddings | `text-embedding-3-small` |
|
|
|
|
## Tips
|
|
|
|
- **Be specific** — "How does the JWT validation middleware work?" is better than "Tell me about auth"
|
|
- **Reference filenames** — "What does `server.rs` do?" helps the retrieval find relevant code
|
|
- **Ask about patterns** — "What error handling pattern does this project use?" works well with RAG
|
|
- **Rebuild after changes** — If the repository has been updated significantly, rebuild embeddings to include new code
|