DLP protocol v2

Pinchana DLP is the optional asynchronous download service used by Pinchana Web for ordinary YouTube URLs. It is not a normal /v1/scrape module and is never exposed directly to the browser.

DLP exists because long-running YouTube downloads, format merging, large output files, and optional user cookies need a stricter resource/security boundary than normal scraper modules.

Public flow

Browser
  │ same-origin /api/dlp/...
  ▼
Pinchana Web BFF
  │ signed Web session
  ▼
pinchana-server /web/dlp/...
  │ internal service token + owner identity
  ▼
DLP API
  │ job state / ciphertext
  ├── Redis
  │
  ▼
Orchestrator ── creates one ephemeral worker
                     │
                     ▼
                 DLP VPN
                     │
                     ▼
                  YouTube

A valid Pinchana Web session is the job-ownership boundary. The gateway derives an opaque owner identifier and ensures one browser session cannot access another owner’s job.

Protocol v2 job lifecycle

1. Allocate — the gateway asks DLP to create a job. Allocation returns a job ID, expiry metadata, and a short-lived X25519 worker public key plus key ID.
2. Prepare options — Web selects only capability-advertised fixed values for video/audio quality, codec/container, dubbed audio, subtitles, and filenames.
3. Optional cookies — if the user explicitly selected an unlocked Cookie Vault profile, the browser filters cookies for the submitted URL and encrypts them for the allocated worker.
4. Submit — fixed options and optional ciphertext are forwarded through the BFF/gateway. The BFF and gateway do not decrypt cookie contents.
5. Run — the orchestrator creates one hardened worker for the job. The worker holds the corresponding X25519 private key and performs yt-dlp/FFmpeg processing through the DLP VPN.
6. Poll — Web reads job progress/state.
7. Download — when ready, the completed file is streamed through the owner-bound gateway/BFF route to the browser download manager.
8. Expire — worker/container and temporary job data are removed according to job lifetime/cleanup rules.

Cookie encryption

Anonymous DLP jobs omit cookies completely. Cookies are used only when a visitor explicitly selects a Cookie Vault profile.

For a cookie-bearing job, browser-side crypto uses:

• X25519 ephemeral key agreement with the worker’s allocation public key;
• HKDF to derive the symmetric key;
• AES-256-GCM for the cookie envelope;
• an independent 32-byte HKDF salt;
• a 12-byte AES-GCM IV;
• authenticated associated data:

pinchana-dlp:v2:{jobId}:{keyId}

The envelope contains the protocol version, key ID, browser X25519 public key, salt, IV, and ciphertext.

Only the job worker has the private key needed to decrypt the envelope. It writes plaintext cookies to a RAM-backed /run/cookies location for the job. Redis stores metadata/ciphertext only; the DLP API and orchestrator do not need plaintext cookies.

Allowed sources and options

DLP v2 accepts YouTube hosts only. Callers cannot supply arbitrary yt-dlp format strings, output templates, shell arguments, or remote post-processing commands.

The current protocol supports validated fixed choices for:

• video quality ceiling, including Best and advertised resolution ceilings;
• codec preference: Auto, H.264, AV1, VP9;
• container: Auto, MP4, WebM, MKV;
• audio output: Best, MP3, OGG, WAV, Opus;
• advertised audio bitrates;
• better-audio preference;
• preferred dubbed language;
• preferred embedded subtitle language;
• filename style.

Web reads /web/capabilities and shows only choices advertised by the selected deployment. Codec/dub/subtitle choices are preferences with safe fallback behavior rather than guarantees that every source contains that exact stream.

Selected subtitles prefer creator-provided subtitles and fall back to automatic captions where the worker supports the selected language. Explicit video container choices are remuxed when possible rather than forcing a full video transcode.

Internal DLP API

The trusted gateway talks to the internal service using x-dlp-service-token and an opaque x-job-owner. The DLP API’s protocol routes are:

• POST /v2/jobs
• POST /v2/jobs/{jobId}/submit
• GET /v2/jobs/{jobId}
• GET /v2/jobs/{jobId}/file

These internal routes are not browser/public API endpoints. Browser-facing access remains under gateway /web/dlp/... and Web same-origin /api/dlp/....

Runtime isolation

DLP API

Validates service authentication, owner identity, job state, URL, options, limits, and ciphertext. It does not have the Docker socket.

Orchestrator

The only DLP component with Docker-socket access. It creates and removes the per-job worker with constrained mounts/networks/resources.

Worker

Holds the job private key, decrypts cookies only when supplied, runs pinned yt-dlp/FFmpeg components, writes output into that job’s directory, and is removed after completion/expiry.

Workers use a read-only root filesystem, dropped capabilities, no-new-privileges, bounded CPU/memory/PID settings, RAM-backed temporary storage, a forced lifetime, and an output-only job mount.

Redis

Stores job metadata/state and ciphertext. It is internal-only and configured without persistence in the production profile because jobs are temporary.

Default limits

Current production defaults include:

• 3 concurrent jobs/workers;
• 8 GiB maximum final output;
• 45-minute execution timeout;
• 2-hour job lifetime;
• 5-minute worker key lifetime.

During merge operations, temporary working files can exceed the final output limit within bounded headroom, but the completed artifact is still rejected above the final limit.

Self-hosters can tune these values, but larger limits directly increase disk, memory, bandwidth, and abuse exposure.

Streaming the finished file

Pinchana Web does not buffer a completed DLP artifact into a giant browser Blob. The /api/dlp/jobs/{jobId}/file BFF route forwards range metadata and streams the file into the browser’s normal download manager.

This is distinct from normal scrape ZIP/audio conversion, which can use browser memory.

Capability gating

DLP_ENABLED=false is the safe API default. The gateway advertises protocol v2 only after the feature flag is enabled and the internal service is healthy.

Deploy and canary the infrastructure first, then enable the public capability. See DLP operations and security for the production sequence.