Ana Sayfa / Kod ve Scriptler / Morphés - Self-Hosted Training Data Pipeline

Morphés - Self-Hosted Training Data Pipeline

Morphés; kendi eğitim altyapınız için JSONL eğitim verilerini toplar, temizler, inceler ve dışa aktarır.

  • Yayın tarihi22 Ekim 2026
  • Nerede çalışırPython 3.13
  • VeritabanıPostgreSQL 16, SQLite 3.x, Redis
  • Paket içeriği.py, .css, .html, .jsx, .json, .webp, .md, .yml/.yaml, package.json, Dockerfile, JavaScript .js
  • Son güncelleme23 Ekim 2026
  • Canlı demoVar
Morphés - Self-Hosted Training Data Pipeline

Overview

Morphés is a self-hosted application for the data-preparation side of a model-training workflow. It guides an operator from a domain description to reviewed, exportable JSONL training data, and can hand that data off to training infrastructure the buyer controls. What it is not: Morphés does not train or fine-tune models itself and is not a hosted multi-tenant service. Training runs execute on the buyer's own machines through a Local Agent, SSH or REST connector. The buyer provides hosting, source accounts, an optional LLM provider account, and training compute.

How it works

  1. Intake – validated domain, volume tier and source selection.
  2. Collection – Kaggle, public Hugging Face Datasets, web search and custom URLs, with a source-transparency log. No synthetic fallback data.
  3. Cleaning – HTML and boilerplate removal, text normalization, junk rejection and near-duplicate detection, with a keep/discard reason per document.
  4. Validation – optional LLM-assisted relevance, licensing-signal and category checks. If no LLM is configured, a deterministic fallback is used and is never presented as a model judgment.
  5. Organize, review, export – category grouping, a human review gate, and JSONL export in instruction, chat or raw shape.

A run cannot reach "ready" with zero usable records, and a training handoff is blocked until the run is reviewed. The demo shows a simulated walkthrough of the interface with synthetic data. It has no backend and does not call providers, run a pipeline or train a model.

Features

  • Guided domain and volume intake, persistent run history, resumable jobs
  • Pluggable sources: Kaggle, public Hugging Face Datasets (explicit dataset/config/split/column selection), web search, custom URLs
  • Per-record cleaning decisions and source provenance you can inspect
  • Optional LLM-assisted validation with your own provider profile (OpenAI, Anthropic, Google, NVIDIA NIM or any OpenAI-compatible endpoint)
  • Category review dashboard with samples, summaries and date ranges
  • JSONL export in instruction, chat or raw format
  • Training handoff over Local Agent, SSH (host-key verification) or REST (webhook HMAC verification); real progress only, no fake percentages
  • Re-training safety: blocks sending records that overlap with earlier Morphés runs, with a reason-recorded admin override
  • Automatic holdout set and an evaluator script that runs on your own training host; measured results only
  • Optional Domain + Status Advisor chat that proposes an intake configuration (the operator must apply it; it never starts a job)
  • Browser login, one-time admin bootstrap, 4 roles (admin, operator, reviewer, viewer), CSRF protection, rate limits
  • Encrypted provider credentials, SSRF and egress protection, redirect blocking
  • Docker Compose stack: FastAPI, React/Vite, PostgreSQL, Redis, RQ workers, nginx
  • Readable source, Alembic migrations, 500+ regression tests, deployment runbook and security documentation
  • No activation system, license check, domain lock or remote validation

Requirements

  • Docker Engine with Docker Compose v2 (Linux host or compatible environment)
  • PostgreSQL and Redis are bundled in the Compose stack
  • For manual development setup: Python 3.13+ and Node.js 20+
  • Your own credentials for any source, LLM provider or training host you choose to use (all optional except the sources you collect from)
  • For internet-facing deployments: TLS termination, firewall, backups and secret management are the operator's responsibility