Skip to content

email-ingest-lambda

Status: Current | Last reviewed: 2026-08-09

The email coordinator accepts SES-delivered messages, forwards Salesforce-bound mail exactly once, stores attachments, and publishes one correlated email.received workflow event.

Source

apps/email-ingest-lambda

Runtime

Aspect Value
Language Python 3.12
Trigger SES receipt rule → encrypted S3 object → SNS → Lambda
State DynamoDB lease and idempotency ledger
Failure sink SQS dead-letter queue
Deployment infra/modules/aws/email-ingest

The same handler package backs two functions:

  • processing/ mail is parsed and published directly to workflow ingress.
  • salesforce/ mail is first re-sent through SES to the configured Salesforce Email-to-Case address, then published with the SES delivery id Salesforce stores as EmailMessage.MessageIdentifier.

Processing contract

For each raw-message S3 key, the coordinator:

  1. Acquires a short DynamoDB lease. completed and suppressed records are terminal.
  2. Parses the RFC-822 message and suppresses delivery reports, auto-submitted mail, bulk/list mail, and mailer-daemon bounces before any forward or event publication.
  3. For Salesforce mail, forwards the original MIME content with attachments and preserves the customer address in Reply-To and X-TrueServe-Original-* headers.
  4. Stores attachments under a SHA-256 content-addressed key in the encrypted attachments bucket.
  5. Builds a deterministic UUIDv5 event id from the raw S3 key.
  6. Publishes email.received to the AI Manager workflow ingress.
  7. Records the workflow event id and Salesforce delivery message id as completed.

An exception records failed, releases the lease for retry, and is re-raised to SNS. The Salesforce delivery id is stored as soon as forwarding succeeds, so a later workflow-ingress retry does not forward the email again.

Event payload

The payload includes:

  • parsed sender, recipients, subject, text, HTML, selected headers, and original RFC Message-ID;
  • a normalized routing_address derived from the first envelope recipient, used for recipient-specific workflow routing;
  • raw-message S3 URI;
  • attachment filename, content type, size, checksum, bucket, key, and S3 URI;
  • spam and virus verdicts when available;
  • salesforce_delivery_message_id and the angle-bracketed salesforce_message_identifier used for the exact Salesforce join;
  • stable event and correlation ids.

The ingress token is read from SSM and sent as a bearer token. The Salesforce forwarding list is also held in SSM so an Email-to-Case address can be rotated without rebuilding the Lambda. Vanguard accepts both salesforce@trueserve.ai and the dedicated medtech@trueserve.ai alias through the same idempotent Salesforce forwarding path; the workflow uses routing_address to select the MedTech record type.

Limits and retention

Defaults are 25 attachments, 25 MiB per attachment, a 120-second processing lease, and a 30-day idempotency record TTL. Raw-message and attachment lifecycle periods are Terraform inputs. S3 public access is blocked and server-side encryption is enabled.

Development

Lambdas do not run under Overmind. Run the focused suite from the repository root:

python -m pytest apps/email-ingest-lambda/test_handler.py

Apply the Vanguard stack from infra/stacks/email-ingest-vanguard. Populate the API token and Salesforce forwarding SSM parameters after the first apply; Terraform intentionally ignores later secret-value rotation.

Operations

Check the DynamoDB row first when an email appears to be missing or duplicated:

  • processing: the current lease may still be active;
  • forwarded: Salesforce forwarding succeeded but event publication did not finish;
  • failed: inspect last_error, then allow or trigger an SNS retry;
  • completed: forwarding and event publication both completed;
  • suppressed: inspect suppression_reason; no Salesforce Case or workflow run should exist.

The SQS dead-letter queue should remain empty. CloudWatch logs include the inbound id, deterministic event id, Salesforce message id, and attachment count without logging message bodies.

See also