HomeBase

Self-hosted Node.js Docker application with automated SSH deployment and long-term image storage with corruption detection and tagging for ML training.

Features

• 🚀 Express.js server
• 🐳 Docker containerized with persistent volumes
• 🔄 Automated deployment via SSH
• 📸 Long-term image storage with SQLite database
• 🏷️ Tag-based organization for machine learning datasets
• 🛡️ Corruption detection using SHA256 checksums
• 🔍 RESTful API for image management
• 🧪 Automatic fetching every 2-3 minutes from web services
• ✅ Health check endpoints
• 🔐 SSH key-based authentication

Prerequisites

On your local machine:

• Node.js (for local testing)
• Docker (for local testing)
• SSH access to beepc server

On the remote server (beepc):

• Docker installed
• Docker Compose installed
• SSH access configured for user spencer

Quick Start

Local Development

# Install dependencies
npm install

# Initialize the image storage system
node setup.js init

# Configure image sources (edit image-sources.json)
# Set "enabled": true for sources you want to fetch from

# Run locally
npm start

# Visit http://localhost:3001

Image Storage Features

Once running, manage images via the REST API:

# List all images
curl http://localhost:3001/api/images

# Add an image manually
curl -X POST http://localhost:3001/api/images \
  -H "Content-Type: application/json" \
  -d '{"source_url":"https://example.com/image.jpg","tags":["test"]}'

# Get storage statistics
curl http://localhost:3001/api/stats

# Verify all images for corruption
curl -X POST http://localhost:3001/api/verify

📖 Full documentation:

• 📸 Image Storage Guide - Complete API reference, configuration, and usage
• 🐳 Docker Deployment Guide - Deployment to home lab with persistent storage

Deployment

Quick Deployment to Home Lab (PowerShell)

From your local Windows machine:

# Deploy to home lab (beepc)
.\scripts\deploy.ps1

This will:

1. Copy all project files to your home lab server
2. Build the Docker image
3. Create and start the container with persistent storage
4. Set up systemd service for auto-start

The app will be available at http://homebase.sketchferret.com

Persistent Data Storage

Images and database are stored in a Docker volume (homebase-data) that persists across container restarts and updates.

For detailed deployment instructions, see DOCKER_DEPLOYMENT.md

Option 2: Manual Deployment

Manual deployment steps are documented in DOCKER_DEPLOYMENT.md

SSH Setup

Ensure you have SSH key-based authentication set up:

# Generate SSH key if you don't have one
ssh-keygen -t ed25519 -C "your_email@example.com"

# Copy key to remote server
ssh-copy-id spencer@beepc

# Test connection
ssh spencer@beepc

Remote Server Setup

On beepc, ensure Docker is installed:

# Install Docker (Ubuntu/Debian)
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add user to docker group
sudo usermod -aG docker spencer

# Install Docker Compose
sudo apt-get update
sudo apt-get install docker-compose-plugin

# Logout and login again for group changes to take effect

API Endpoints

Status & Health

• GET /api/status - App status and version
• GET /health - Health check endpoint

Image Management

• GET /api/images - List all images (with pagination & filtering)
• GET /api/images/{id} - Get image details
• GET /api/images/{id}/download - Download image file
• POST /api/images - Fetch and save new image
• DELETE /api/images/{id} - Delete image

Tagging

• POST /api/images/{id}/tags - Add tags to image
• GET /api/tags - List all available tags

Administration

• GET /api/stats - Storage statistics
• POST /api/verify - Verify all images for corruption
• POST /api/cleanup - Clean up old images
• GET /api/fetcher/status - Image fetcher status

See IMAGE_STORAGE_GUIDE.md for detailed API documentation.

Project Structure

HomeBase/
├── scripts/
│   ├── deploy.ps1                    # PowerShell deployment script
│   ├── deploy.sh                     # Bash deployment script
│   └── deploy.bat                    # Windows batch deployment script
├── lib/
│   ├── database.js                   # SQLite operations & schema
│   ├── storage.js                    # File storage & checksums
│   └── image-fetcher.js             # Image fetching service
├── routes/
│   └── images.js                     # Image API endpoints
├── public/
│   ├── index.html
│   ├── app.js
│   └── styles.css
├── .github/
│   └── workflows/
│       └── deploy.yml                # GitHub Actions workflow
├── .dockerignore                     # Docker ignore file
├── .gitignore                        # Git ignore file
├── Dockerfile                        # Docker configuration
├── docker-compose.yml                # Docker Compose with volumes
├── homebase.service                  # Systemd service file
├── package.json                      # Node.js dependencies
├── server.js                         # Express server
├── setup.js                          # Setup & CLI tool
├── image-sources.json                # Image source configuration
├── IMAGE_STORAGE_GUIDE.md           # Complete image storage docs
├── DOCKER_DEPLOYMENT.md             # Docker deployment guide
└── README.md                         # This file

Deployment Process

The deployment script does the following:

1. Creates remote directory if it doesn't exist
2. Syncs files to remote server via rsync/scp
3. Stops existing Docker container
4. Builds new Docker image
5. Starts new container
6. Verifies container is running

Troubleshooting

Check logs on remote server

ssh spencer@beepc
cd /home/spencer/homebase
docker compose logs -f

Check container status

ssh spencer@beepc "docker ps"

Restart container

ssh spencer@beepc "cd /home/spencer/homebase && docker compose restart"

Remove and rebuild

ssh spencer@beepc "cd /home/spencer/homebase && docker compose down && docker compose up -d --build"

Accessing the App

Once deployed, access the app at:

• http://homebase.sketchferret.com (reverse proxied, no port needed)
• http://beepc:3001 (direct access from local network)
• http://localhost:3001 (direct access on the server)

Note: The app runs on port 3001 internally but is accessed via reverse proxy at the standard HTTP port.

Auto-Start on Boot

The deployment script automatically sets up a systemd service that:

• Starts the Docker container on server boot
• Restarts the container if it crashes
• Runs as the spencer user

To manually manage the service:

# Check status
ssh spencer@beepc "sudo systemctl status homebase"

# Stop service
ssh spencer@beepc "sudo systemctl stop homebase"

# Start service
ssh spencer@beepc "sudo systemctl start homebase"

# Restart service
ssh spencer@beepc "sudo systemctl restart homebase"

# Disable auto-start
ssh spencer@beepc "sudo systemctl disable homebase"

Configuration

Image Sources

Edit image-sources.json to configure which image URLs to fetch from:

{
  "sources": [
    {
      "name": "Webcam Feed",
      "url": "https://your-service.com/image.jpg",
      "tags": ["webcam", "monitoring"],
      "enabled": true
    }
  ],
  "fetchInterval": 0.033
}

Fetch Intervals - Edit fetchInterval (in minutes):

• 0.0167 = 1 second
• 0.033 = 2 seconds (default, recommended for steady updates)
• 0.05 = 3 seconds
• 1 = 1 minute
• 2.5 = 2.5 minutes (original default)

The fetcher will automatically start pulling images from enabled sources at this interval.

Environment Variables

Customize settings in docker-compose.yml:

environment:
  - PORT=3001
  - DOMAIN=homebase.sketchferret.com
  - NODE_ENV=production

Security Notes

• Uses SSH key authentication (no passwords)
• Container runs as non-root user
• Only production dependencies installed in Docker image
• Health check endpoint for monitoring

License

MIT