think-bigger/docs/plans/checklists/development-setup.md

Development Setup Checklist

This checklist ensures a consistent development environment across all team members and provides a quick start guide for new contributors.

Prerequisites

• Python 3.10+ installed
• Node.js 18+ installed (for frontend development)
• Docker and Docker Compose installed
• Git configured with SSH keys
• VS Code or preferred IDE installed

Environment Setup

Backend Setup

• Clone repository: git clone <repo-url>
• Create virtual environment: python -m venv venv
• Activate environment: source venv/bin/activate (Linux/Mac) or venv\Scripts\activate (Windows)
• Install dependencies: pip install -r requirements.txt
• Install dev dependencies: pip install -r requirements-dev.txt
• Copy environment template: cp .env.example .env
• Configure API keys in .env:
  • OPENAI_API_KEY
  • OPENALEX_API_KEY (if using)
  • HUGGINGFACE_TOKEN (if using local models)

Frontend Setup

• Navigate to frontend directory: cd frontend/
• Install dependencies: npm install
• Copy environment template: cp .env.example .env.local
• Configure frontend environment variables

Database Setup

• Choose database: Neo4j/Dgraph/local alternative
• Install database locally or configure cloud instance
• Update connection settings in .env
• Run database migrations/initialization scripts

Development Tools

Code Quality

• Install pre-commit hooks: pre-commit install
• Configure linting: npm run lint (frontend) or appropriate backend linter
• Set up type checking: mypy for Python, TypeScript for frontend
• Configure code formatting: Black for Python, Prettier for frontend

Testing

• Install test dependencies
• Configure test database
• Run initial test suite: pytest (Python) or npm test (frontend)
• Set up test coverage reporting

Documentation

• Install documentation tools (Sphinx for Python, etc.)
• Configure API documentation generation
• Set up documentation deployment (ReadTheDocs, etc.)

Project Structure Validation

Backend Structure

• Verify src/ directory structure matches architecture
• Confirm config/settings.py loads correctly
• Test basic imports: python -c "import src.episodic_memory"
• Validate configuration loading

Frontend Structure

• Verify component structure
• Test basic build: npm run build
• Confirm routing configuration
• Validate TypeScript compilation

API and Integration Testing

Backend API

• Start development server: uvicorn api.server:app --reload
• Test health endpoint: curl http://localhost:8000/health
• Verify API documentation: http://localhost:8000/docs

Frontend-Backend Integration

• Start frontend dev server: npm run dev
• Test basic API calls from frontend
• Verify CORS configuration
• Test file upload endpoints

Data Pipeline Testing

Document Processing

• Place test PDF in data/raw_documents/
• Run document parser: python -c "from src.episodic_memory.document_parser import DocumentParser; parser = DocumentParser(); chunks = parser.parse_directory('data/raw_documents'); print(f'Parsed {len(chunks)} chunks')"
• Verify chunk creation and metadata extraction

Embedding Service

• Test embedding generation
• Verify vector storage
• Test basic similarity search

Agent Development Setup

Dana Runtime

• Verify Dana language runtime installation
• Test basic Dana code execution
• Confirm REPL functionality

Agent Framework

• Test agent loading and initialization
• Verify basic agent communication
• Confirm agent configuration loading

Performance and Monitoring

Logging

• Configure structured logging
• Set up log aggregation (if needed)
• Test error tracking and reporting

Performance Monitoring

• Set up basic performance profiling
• Configure health checks
• Test resource usage monitoring

Security and Privacy

Local Data Sovereignty

• Verify all data paths are user-configurable
• Test file system permissions
• Confirm no external data leakage

API Security

• Configure basic authentication (if needed)
• Test input validation
• Verify secure file handling

Deployment Preparation

Docker

• Build Docker image: docker build -t second-brain .
• Test container execution: docker run -p 8000:8000 second-brain
• Verify Docker Compose setup

Production Configuration

• Create production environment template
• Configure production logging
• Set up production database connections

Final Validation

• Run full test suite: pytest --cov=src --cov-report=html
• Perform integration testing
• Test end-to-end user workflows
• Validate performance requirements
• Confirm documentation is current

Troubleshooting

Common Issues

• Import errors: Check virtual environment activation and PYTHONPATH
• Database connection failures: Verify connection strings and credentials
• API key issues: Confirm environment variables are loaded correctly
• Build failures: Check Node.js/Python versions and dependency conflicts

Getting Help

• Check existing issues in repository
• Review documentation in docs/
• Contact team lead for environment-specific issues

Next Steps

After completing this checklist:

1. Start with Phase 1 foundation work
2. Set up daily standups and progress tracking
3. Begin implementing core backend services
4. Schedule regular architecture reviews docs/plans/checklists/development-setup.md