SFTP Source

Poll an SFTP server for new files with support for password and private key authentication.

Overview

The SFTP source connects to a remote SFTP server and polls a specified directory for new files at a configurable interval. When files matching the pattern are found, their contents are downloaded and passed into the channel pipeline.

SFTP is widely used in healthcare for secure file transfers between organizations, particularly for batch lab results, insurance claims, pharmacy data feeds, and clinical document exchange. Unlike plain FTP, SFTP encrypts all data in transit over SSH.

Tip Use private key authentication instead of passwords for automated integrations. Store the key path in an environment variable for portability across environments.

Configuration

yaml

listener:
  type: sftp
  sftp:
    host: sftp.lab-corp.example.com
    port: 22
    poll_interval: 5m
    directory: /outbound/results
    file_pattern: "*.csv"
    move_to: /outbound/archive
    error_dir: /outbound/errors
    auth:
      type: private_key
      username: intu-service
      private_key_path: ${SFTP_KEY_PATH}
    sort_by: date

Properties

host string required

Hostname or IP address of the SFTP server.

port int optional

SSH port. Defaults to 22.

poll_interval string optional

How often to check for new files. Accepts Go duration strings such as 30s, 5m. Defaults to 1m.

directory string required

Remote directory path to poll for files.

file_pattern string optional

Glob pattern to filter files. Examples: *.csv, RESULTS_*.txt. Defaults to *.

move_to string optional

Remote directory to move files after successful processing.

error_dir string optional

Remote directory to move files that fail processing.

auth object required

Authentication configuration. Supported types: password and private_key.

sort_by string optional

Order in which files are processed. One of name, date, or size. Defaults to name.

Auth Sub-Properties

Type	Properties	Description
`password`	`username`, `password`	Standard username/password authentication over SSH.
`private_key`	`username`, `private_key_path`, `passphrase` (optional)	SSH key-based authentication. The passphrase is only required if the key is encrypted.

Complete Example

Poll a remote SFTP server for CSV lab result files every 5 minutes, archive processed files, and route failures to an error directory.

yaml

id: sftp-lab-results
enabled: true

listener:
  type: sftp
  sftp:
    host: sftp.lab-corp.example.com
    port: 22
    poll_interval: 5m
    directory: /outbound/results
    file_pattern: "*.csv"
    move_to: /outbound/archive
    error_dir: /outbound/errors
    auth:
      type: private_key
      username: intu-service
      private_key_path: ${SFTP_KEY_PATH}
    sort_by: date

validator:
  runtime: node
  entrypoint: validator.ts

transformer:
  runtime: node
  entrypoint: transformer.ts

destinations:
  - lab-results-fhir

TypeScript Transformer Example

This transformer parses a CSV file containing lab results and converts each row into a FHIR DiagnosticReport resource.

typescript

import { Message, TransformResult } from "@intu/sdk";

interface LabResult {
  patientMrn: string;
  testCode: string;
  testName: string;
  value: string;
  unit: string;
  referenceRange: string;
  status: string;
  collectedDate: string;
}

function parseCsv(content: string): LabResult[] {
  const lines = content.trim().split("\n");
  const headers = lines[0].split(",").map((h) => h.trim());
  const results: LabResult[] = [];

  for (let i = 1; i < lines.length; i++) {
    const values = lines[i].split(",").map((v) => v.trim());
    results.push({
      patientMrn: values[headers.indexOf("patient_mrn")] || "",
      testCode: values[headers.indexOf("test_code")] || "",
      testName: values[headers.indexOf("test_name")] || "",
      value: values[headers.indexOf("value")] || "",
      unit: values[headers.indexOf("unit")] || "",
      referenceRange: values[headers.indexOf("reference_range")] || "",
      status: values[headers.indexOf("status")] || "",
      collectedDate: values[headers.indexOf("collected_date")] || "",
    });
  }

  return results;
}

export default function transform(msg: Message): TransformResult {
  const csvContent = msg.body as string;
  const results = parseCsv(csvContent);

  if (results.length === 0) {
    return { success: false, error: "No lab results found in CSV" };
  }

  const bundle = {
    resourceType: "Bundle",
    type: "batch",
    entry: results.map((lab) => ({
      resource: {
        resourceType: "DiagnosticReport",
        status: lab.status === "final" ? "final" : "preliminary",
        code: {
          coding: [
            {
              system: "http://loinc.org",
              code: lab.testCode,
              display: lab.testName,
            },
          ],
        },
        subject: {
          identifier: {
            system: "urn:oid:2.16.840.1.113883.19.5",
            value: lab.patientMrn,
          },
        },
        effectiveDateTime: lab.collectedDate,
        result: [
          {
            display: `${lab.testName}: ${lab.value} ${lab.unit}`,
          },
        ],
      },
    })),
  };

  return {
    success: true,
    body: JSON.stringify(bundle),
    contentType: "application/fhir+json",
  };
}