Compare commits

..

2 commits

Author SHA1 Message Date
Pratik Narola
971548321f references updated for qdrant 2025-09-09 13:05:16 +05:30
Pratik Narola
f625f8f556 Migrated to Qdrant 2025-09-09 12:54:46 +05:30
9 changed files with 98 additions and 134 deletions

View file

@ -5,11 +5,9 @@ OPENAI_BASE_URL=https://your-openai-compatible-endpoint.com/v1
EMBEDDER_API_KEY=AIzaSy-your-google-gemini-api-key-here
# Database Configuration
POSTGRES_DB=mem0_db
POSTGRES_USER=mem0_user
POSTGRES_PASSWORD=mem0_password
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
QDRANT_HOST=qdrant
QDRANT_PORT=6333
QDRANT_COLLECTION_NAME=mem0
# Neo4j Configuration
NEO4J_AUTH=neo4j/mem0_neo4j_password
@ -36,4 +34,5 @@ EXPERT_MODEL=o3 # Expert-level comprehensive analysis
# - Ensure all models are available on your OpenAI-compatible endpoint
# - Verify model availability: curl -H "Authorization: Bearer $API_KEY" $BASE_URL/v1/models
# - Neo4j must be version 5.18+ for vector.similarity.cosine() function
# - Ollama must be running locally with nomic-embed-text:latest model
# - Qdrant vector database for embeddings storage (replaces PostgreSQL+pgvector)
# - Ollama must be running locally with nomic-embed-text:latest model

View file

@ -1,12 +1,12 @@
# Mem0 Interface - Production Ready
A fully operational Mem0 interface with PostgreSQL and Neo4j integration, featuring intelligent model routing, comprehensive memory management, and production-grade monitoring.
A fully operational Mem0 interface with Qdrant and Neo4j integration, featuring intelligent model routing, comprehensive memory management, and production-grade monitoring.
## Features
### Core Memory System
- ✅ **Mem0 OSS Integration**: Complete hybrid datastore (Vector + Graph + KV storage)
- ✅ **PostgreSQL + pgvector**: High-performance vector embeddings storage
- ✅ **Qdrant**: Purpose-built vector database for high-performance embeddings storage
- ✅ **Neo4j 5.18**: Graph relationships with native vector similarity functions
- ✅ **Google Gemini Embeddings**: Enterprise-grade embedding generation
- ✅ **Memory Operations**: Store, search, update, delete memories with semantic search
@ -62,7 +62,7 @@ curl http://localhost:8000/health
### Core Components
- **FastAPI Backend**: Production-ready API with comprehensive monitoring
- **Mem0 OSS**: Hybrid memory management (vector + graph + key-value)
- **PostgreSQL + pgvector**: Vector embeddings storage and similarity search
- **Qdrant**: Purpose-built vector database for embeddings storage and similarity search
- **Neo4j 5.18**: Graph relationships with native vector functions
- **Google Gemini**: Enterprise-grade embedding generation
@ -203,13 +203,13 @@ curl http://localhost:8000/graph/relationships/alice
### Service Dependencies
- **Neo4j**: Must start before backend for vector functions
- **PostgreSQL**: Required for vector storage initialization
- **Qdrant**: Required for vector storage initialization
- **Ollama**: Must be running locally on port 11434
- **API Endpoint**: Must have valid models available
## Production Notes
- **Memory Usage**: Neo4j and PostgreSQL require adequate RAM for vector operations
- **Memory Usage**: Neo4j and Qdrant require adequate RAM for vector operations
- **API Rate Limits**: Monitor usage on custom endpoint
- **Data Persistence**: All data stored in Docker volumes
- **Scaling**: Individual services can be scaled independently
@ -217,4 +217,4 @@ curl http://localhost:8000/graph/relationships/alice
## Development
See individual README files in `backend/` and `frontend/` directories for development setup.
See individual README files in `backend/` and `frontend/` directories for development setup.

View file

@ -9,7 +9,7 @@ A production-ready FastAPI backend that provides intelligent memory integration
1. **Mem0Manager** (`mem0_manager.py`)
- Central orchestration of memory operations
- Integration with custom OpenAI-compatible endpoint
- Memory persistence across PostgreSQL and Neo4j
- Memory persistence across Qdrant and Neo4j
- Performance timing and operation tracking
2. **Configuration System** (`config.py`)
@ -39,8 +39,8 @@ A production-ready FastAPI backend that provides intelligent memory integration
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
PostgreSQL │ │ Neo4j │ │ Custom LLM │
(pgvector) │ │ (APOC) │ │ Endpoint │
Qdrant │ │ Neo4j │ │ Custom LLM │
(Vector DB) │ │ (APOC) │ │ Endpoint │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ • Vector Store │ │ • Graph Store │ │ • claude-sonnet-4│
│ • Embeddings │ │ • Relationships │ │ • Gemini Embed │
@ -134,11 +134,9 @@ backend/
| `OPENAI_COMPAT_API_KEY` | Your custom endpoint API key | - | ✅ |
| `OPENAI_BASE_URL` | Custom endpoint URL | - | ✅ |
| `EMBEDDER_API_KEY` | Google Gemini API key for embeddings | - | ✅ |
| `POSTGRES_HOST` | PostgreSQL host | postgres | ✅ |
| `POSTGRES_PORT` | PostgreSQL port | 5432 | ✅ |
| `POSTGRES_DB` | Database name | mem0_db | ✅ |
| `POSTGRES_USER` | Database user | mem0_user | ✅ |
| `POSTGRES_PASSWORD` | Database password | - | ✅ |
| `QDRANT_HOST` | Qdrant vector database host | qdrant | ✅ |
| `QDRANT_PORT` | Qdrant vector database port | 6333 | ✅ |
| `QDRANT_COLLECTION_NAME` | Qdrant collection name | mem0 | ✅ |
| `NEO4J_URI` | Neo4j connection URI | bolt://neo4j:7687 | ✅ |
| `NEO4J_USERNAME` | Neo4j username | neo4j | ✅ |
| `NEO4J_PASSWORD` | Neo4j password | - | ✅ |
@ -283,7 +281,7 @@ docker exec mem0-neo4j cypher-shell -u neo4j -p mem0_neo4j_password \
### Key Integration Points Verified
- ✅ **Ollama Embeddings**: nomic-embed-text:latest working for vector generation
- ✅ **PostgreSQL + pgvector**: Vector storage and similarity search operational
- ✅ **Qdrant**: Vector storage and similarity search operational
- ✅ **Neo4j 5.18**: Graph relationships and native vector functions working
- ✅ **Custom LLM Endpoint**: All 4 models accessible and routing correctly
- ✅ **Memory Persistence**: Data survives container restarts via Docker volumes
@ -390,7 +388,7 @@ backend:
#### Dependency Requirements
- Neo4j 5.18+ (for vector.similarity.cosine function)
- Ollama running locally with nomic-embed-text:latest
- PostgreSQL with pgvector extension
- Qdrant vector database
- Valid API keys for custom LLM endpoint
### Debugging Commands
@ -447,4 +445,4 @@ docker logs mem0-backend --tail 20 -f
## 📄 License
This POC is designed for demonstration and evaluation purposes.
This POC is designed for demonstration and evaluation purposes.

View file

@ -10,18 +10,16 @@ class Settings(BaseSettings):
"""Application settings loaded from environment variables."""
# API Configuration
openai_api_key: str = Field(..., env="OPENAI_API_KEY")
openai_base_url: str = Field(..., env="OPENAI_BASE_URL")
openai_api_key: str = Field(..., env="OPENAI_COMPAT_API_KEY")
openai_base_url: str = Field(..., env="OPENAI_COMPAT_BASE_URL")
embedder_api_key: str = Field(..., env="EMBEDDER_API_KEY")
# ollama_base_url: str = Field(..., env="OLLAMA_BASE_URL")
# Database Configuration
postgres_host: str = Field("localhost", env="POSTGRES_HOST")
postgres_port: int = Field(5432, env="POSTGRES_PORT")
postgres_db: str = Field("mem0_db", env="POSTGRES_DB")
postgres_user: str = Field("mem0_user", env="POSTGRES_USER")
postgres_password: str = Field("mem0_password", env="POSTGRES_PASSWORD")
qdrant_host: str = Field("localhost", env="QDRANT_HOST")
qdrant_port: int = Field(6333, env="QDRANT_PORT")
qdrant_collection_name: str = Field("mem0", env="QDRANT_COLLECTION_NAME")
# Neo4j Configuration
neo4j_uri: str = Field("bolt://localhost:7687", env="NEO4J_URI")
@ -35,10 +33,6 @@ class Settings(BaseSettings):
# Model Configuration - Ultra-minimal (single model)
default_model: str = Field("claude-sonnet-4", env="DEFAULT_MODEL")
@property
def postgres_url(self) -> str:
"""Build PostgreSQL connection URL."""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}:{self.postgres_port}/{self.postgres_db}"
@property
def cors_origins_list(self) -> List[str]:
@ -51,4 +45,4 @@ class Settings(BaseSettings):
# Global settings instance
settings = Settings()
settings = Settings()

View file

@ -39,14 +39,13 @@ class Mem0Manager:
}
},
"vector_store": {
"provider": "pgvector",
"provider": "qdrant",
"config": {
"dbname": settings.postgres_db,
"user": settings.postgres_user,
"password": settings.postgres_password,
"host": settings.postgres_host,
"port": settings.postgres_port,
"embedding_model_dims": 2560
"collection_name": settings.qdrant_collection_name,
"host": settings.qdrant_host,
"port": settings.qdrant_port,
"embedding_model_dims": 2560,
"on_disk": True
}
},
"graph_store": {

View file

@ -9,8 +9,7 @@ openai
google-genai
# Database
psycopg2-binary
pgvector
qdrant-client
neo4j
langchain-neo4j
rank-bm25
@ -30,4 +29,4 @@ python-json-logger
# CORS and Security
python-jose[cryptography]
passlib[bcrypt]
passlib[bcrypt]

View file

@ -1,66 +0,0 @@
-- Initialize PostgreSQL database for Mem0 with pgvector extension
-- Create the pgvector extension for vector operations
CREATE EXTENSION IF NOT EXISTS vector;
-- Create user if not exists (in case it's needed)
DO
$do$
BEGIN
IF NOT EXISTS (
SELECT FROM pg_catalog.pg_roles
WHERE rolname = 'mem0_user') THEN
CREATE ROLE mem0_user LOGIN PASSWORD 'mem0_password';
END IF;
END
$do$;
-- Grant necessary permissions
GRANT ALL PRIVILEGES ON DATABASE mem0_db TO mem0_user;
GRANT ALL ON SCHEMA public TO mem0_user;
-- Create table for vector embeddings (if needed by Mem0's pgvector implementation)
CREATE TABLE IF NOT EXISTS embeddings (
id SERIAL PRIMARY KEY,
user_id VARCHAR(255),
content TEXT,
embedding VECTOR(2560), -- OpenAI embedding dimension
metadata JSONB,
created_at TIMESTAMP DEFAULT NOW()
);
-- Create index for efficient vector similarity search
CREATE INDEX IF NOT EXISTS embeddings_embedding_idx ON embeddings
USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
-- Create index for user_id lookups
CREATE INDEX IF NOT EXISTS embeddings_user_id_idx ON embeddings (user_id);
-- Create index for metadata queries
CREATE INDEX IF NOT EXISTS embeddings_metadata_idx ON embeddings USING GIN (metadata);
-- Grant permissions on the table
GRANT ALL PRIVILEGES ON TABLE embeddings TO mem0_user;
GRANT USAGE, SELECT ON SEQUENCE embeddings_id_seq TO mem0_user;
-- Create table for memory history tracking
CREATE TABLE IF NOT EXISTS memory_history (
id SERIAL PRIMARY KEY,
memory_id VARCHAR(255),
user_id VARCHAR(255),
action VARCHAR(50),
previous_value TEXT,
new_value TEXT,
metadata JSONB,
created_at TIMESTAMP DEFAULT NOW()
);
-- Create indexes for memory history
CREATE INDEX IF NOT EXISTS memory_history_memory_id_idx ON memory_history (memory_id);
CREATE INDEX IF NOT EXISTS memory_history_user_id_idx ON memory_history (user_id);
CREATE INDEX IF NOT EXISTS memory_history_created_at_idx ON memory_history (created_at);
-- Grant permissions
GRANT ALL PRIVILEGES ON TABLE memory_history TO mem0_user;
GRANT USAGE, SELECT ON SEQUENCE memory_history_id_seq TO mem0_user;

View file

@ -1,20 +1,17 @@
services:
# PostgreSQL with pgvector extension for vector storage
postgres:
image: pgvector/pgvector:pg17-trixie
container_name: mem0-postgres
environment:
POSTGRES_DB: ${POSTGRES_DB:-mem0_db}
POSTGRES_USER: ${POSTGRES_USER:-mem0_user}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-mem0_password}
# Qdrant vector database for vector storage
qdrant:
image: qdrant/qdrant:latest
container_name: mem0-qdrant
expose:
- "5432"
- "6333"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./config/postgres-init.sql:/docker-entrypoint-initdb.d/init.sql
- qdrant_data:/qdrant/storage
command: >
sh -c "apt-get update && apt-get install -y curl && ./entrypoint.sh"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-mem0_user} -d ${POSTGRES_DB:-mem0_db}"]
interval: 5s
test: ["CMD", "curl", "-f", "http://localhost:6333/"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
@ -57,11 +54,9 @@ services:
OPENAI_API_KEY: ${OPENAI_COMPAT_API_KEY}
OPENAI_BASE_URL: ${OPENAI_COMPAT_BASE_URL}
EMBEDDER_API_KEY: ${EMBEDDER_API_KEY:-AIzaSyA_}
POSTGRES_HOST: postgres
POSTGRES_PORT: 5432
POSTGRES_DB: ${POSTGRES_DB:-mem0_db}
POSTGRES_USER: ${POSTGRES_USER:-mem0_user}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-mem0_password}
QDRANT_HOST: qdrant
QDRANT_PORT: 6333
QDRANT_COLLECTION_NAME: ${QDRANT_COLLECTION_NAME:-mem0}
NEO4J_URI: bolt://neo4j:7687
NEO4J_USERNAME: ${NEO4J_USERNAME:-neo4j}
NEO4J_PASSWORD: ${NEO4J_PASSWORD:-mem0_neo4j_password}
@ -71,7 +66,7 @@ services:
ports:
- "${BACKEND_PORT:-8000}:8000"
depends_on:
postgres:
qdrant:
condition: service_healthy
neo4j:
condition: service_healthy
@ -81,7 +76,7 @@ services:
command: ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
volumes:
postgres_data:
qdrant_data:
neo4j_data:
neo4j_logs:
neo4j_import:
@ -89,4 +84,4 @@ volumes:
networks:
default:
name: mem0-network
name: mem0-network

View file

@ -37,7 +37,14 @@
padding: 20px;
background: #fafafa;
border-bottom: 1px solid #e0e0e0;
display: flex;
justify-content: space-between;
align-items: center;
}
.chat-header-content {
text-align: center;
flex: 1;
}
.chat-header h1 {
@ -51,6 +58,31 @@
font-size: 14px;
}
.clear-chat-btn {
background: #f8f9fa;
color: #666;
border: 1px solid #e0e0e0;
padding: 8px 12px;
border-radius: 6px;
cursor: pointer;
font-size: 12px;
font-weight: 500;
display: flex;
align-items: center;
gap: 5px;
transition: all 0.2s ease;
}
.clear-chat-btn:hover {
background: #e9ecef;
border-color: #ced4da;
color: #495057;
}
.clear-chat-btn:active {
background: #dee2e6;
}
.chat-messages {
flex: 1;
overflow-y: auto;
@ -242,8 +274,13 @@
<!-- Chat Section -->
<div class="chat-section">
<div class="chat-header">
<h1>What can I help you with?</h1>
<p>Chat with your memories - User: pratik</p>
<div class="chat-header-content">
<h1>What can I help you with?</h1>
<p>Chat with your memories - User: pratik</p>
</div>
<button class="clear-chat-btn" id="clearChatBtn" title="Clear chat history">
🗑️ Clear Chat
</button>
</div>
<div class="chat-messages" id="chatMessages">
@ -281,6 +318,7 @@
const memoriesList = document.getElementById('memoriesList');
const memoryCount = document.getElementById('memoryCount');
const refreshButton = document.getElementById('refreshMemories');
const clearChatBtn = document.getElementById('clearChatBtn');
// Chat history in localStorage
let chatHistory = JSON.parse(localStorage.getItem('chatHistory') || '[]');
@ -296,6 +334,7 @@
if (e.key === 'Enter') sendMessage();
});
refreshButton.addEventListener('click', loadMemories);
clearChatBtn.addEventListener('click', clearChatWithConfirmation);
});
// Load chat history from localStorage
@ -493,6 +532,13 @@
}, 5000);
}
// Clear chat history with confirmation
function clearChatWithConfirmation() {
if (confirm('Are you sure you want to clear the chat history? This action cannot be undone.')) {
clearChatHistory();
}
}
// Clear chat history (for debugging)
function clearChatHistory() {
chatHistory = [];