Source Connectors - Setup Runbooks¶

Pull-based, read-only connectors that sync content and identity from source systems through the standard ingest path. As built: Entra ID (identity), SharePoint/OneDrive, Exchange/Outlook, Slack, Salesforce (content). Each is a thin Python package under workers/ on the shared workers_common harness.

Normative contract (source-connector-framework spec, approved v0.3; as-built v0.4-v0.8):

Connectors are read-only against the source. They never write, move, tag, or annotate source content.
Each runs as a sync:{source} job chain: a run leases one job whose payload carries the delta cursor, processes the window, and enqueues the successor. The job chain IS the connector state - there is no separate registry.
Items are submitted through POST /v1/ingest with an options.sourceMeta envelope; parsing stays in the format workers.
A failing item is skipped and audited, never aborts the run.

How a connector process authenticates¶

Two credentials, two wires:

Job leasing (gRPC). The harness polls core's WorkerService with a signed token. For connectors this must be a connector token (audience "connector", tenant-scoped AND named): it leases only that tenant's sync: jobs, and its name is the delete namespace.

telha api-key connector-token --connector sharepoint --tenant <uuid> --org <uuid>
# requires grpc.token_key in core config; token TTL is 300 s

Because tokens expire in 300 s, prefer TELHA_WORKER_TOKEN_CMD (the harness re-mints by running the command) over a static TELHA_WORKER_TOKEN.

Writes (REST). Ingest, record upserts, lookups, and deletes go over REST with TELHA_API_KEY (a normal tenant API key). Deletes issued under a connector token are namespace-checked (below); API keys remain unrestricted tenant-admin credentials.

Common runtime environment (workers_common harness)¶

Env var	Meaning	Default
`TELHA_URL`	Core REST base URL	(required)
`TELHA_API_KEY`	Tenant API key for REST writes	(required)
`TELHA_GRPC_TARGET`	Core gRPC `host:port`	`127.0.0.1:7626`
`TELHA_WORKER_TOKEN`	Static signed token for job leasing	-
`TELHA_WORKER_TOKEN_CMD`	Command that prints a fresh token (preferred)	-
`TELHA_WORKER_ID`	Worker identity in leases	package name
`TELHA_HEALTH_PORT`	`/healthz` port for orchestration	`8090`
`TELHA_LOG_LEVEL`	Python log level	`INFO`

Run a connector:

cd workers && pip install -e workers_common -e sharepoint_connector -e ../sdk/python
python -m workers_common.run sharepoint_connector

A half-configured process fails loudly at startup listing exactly the missing env vars.

Seeding and inspecting sync chains¶

The chain is seeded once per source per tenant with the operator CLI. These commands open the data dir directly - run them with the server stopped (or from a one-shot container on the same volume).

# seed (payload defaults to {"v":1}, a fresh cursor)
telha connector run sharepoint --tenant <uuid> --org <uuid> --data-dir <dir>
# seeded sync:sharepoint chain: job <uuid>

# refuse-double-seed: one live chain step per source is enforced
# list every sync job across states
telha connector ls --tenant <uuid> --org <uuid> --data-dir <dir>

# latest chain step for one source: state, payload (cursor), event log
telha connector status sharepoint --tenant <uuid> --org <uuid> --data-dir <dir>

After the seed, restart telha serve; the running connector process leases the job, syncs a window, and the completion enqueues the next step at now + interval. Re-seed only after a chain has drained (completed/dead with nothing pending or leased).

Connector: Entra ID (identity)¶

Makes Entra the canonical source of PERSON/GROUP identity: verified aliases (aad_oid, upn, mail/proxyAddresses), MEMBER_OF membership, REPORTS_TO manager edges, delta-synced. Content connectors' placeholder persons (email-only) merge into canonical ones via ALIAS_OF edges. Run this connector before the clarify-bot: it is the authority substrate for clarification routing and binding.

App registration (Azure portal):

Entra ID > App registrations > New registration (single tenant).
API permissions > Microsoft Graph > Application permissions: User.Read.All, Group.Read.All, GroupMember.Read.All. Grant admin consent.
Certificates & secrets > New client secret; record the value.

The connector authenticates app-only against the v2.0 token endpoint with scope https://graph.microsoft.com/.default and calls /users/delta, /groups/delta, /users/{id}/manager (read-only, throttle-aware, Retry-After honored; expired delta link = clean full resync).

Environment (exact names from entra_connector/config.py):

Env var	Meaning	Default
`ENTRA_TENANT_ID`	Entra directory (tenant) id	(required)
`ENTRA_CLIENT_ID`	App registration client id	(required)
`ENTRA_CLIENT_SECRET`	Client secret value	(required)
`ENTRA_INTERVAL_MINUTES`	Sync interval	`60`
`ENTRA_SYNC_GROUPS`	Sync groups + membership	`true`
`ENTRA_SYNC_MANAGER`	Sync manager edges	`true`

telha connector run entra --tenant <uuid> --org <uuid> --data-dir <dir>
python -m workers_common.run entra_connector

Connector: SharePoint / OneDrive¶

Graph drive-delta chains per allowlisted site; files route by extension (pdf/docx/pptx to docling, csv/xlsx to tabular, json to the bypass; others skip with a stat). Full sourceMeta: author/lastModifiedBy as aad_oid principals, owner from the shared.owner facet, ACL from permissions (group grants stored as the group id, resolved via MEMBER_OF at gate time). Source deletes propagate as tombstones.

App registration: as for Entra, but Graph Application permissions Sites.Read.All, Files.Read.All (admin consent).

Env var	Meaning	Default
`SP_TENANT_ID`	Entra directory id	(required)
`SP_CLIENT_ID`	App registration client id	(required)
`SP_CLIENT_SECRET`	Client secret	(required)
`SP_SITE_IDS`	Comma-separated site-id allowlist; empty = sync nothing	(empty)
`SHAREPOINT_INTERVAL_MINUTES`	Sync interval	`30`
`SHAREPOINT_MAX_ITEM_MB`	Oversize skip threshold	`50`

Recorded v1 limitation: Graph's composite site-id form (hostname,siteCollectionGuid,siteGuid) contains commas and cannot ride the comma-separated allowlist; supply one Graph-addressable site id per entry.

Connector: Exchange / Outlook¶

Graph messages/delta chains per allowlisted mailbox. Every message is fetched as raw MIME (/messages/{id}/$value) and ingested as format="email"; attachments fan out projection-side and deduplicate by content hash across mailboxes. Immutable message ids (Prefer: IdType="ImmutableId") keep identity stable across folder moves. @removed deltas propagate as tombstones. ACL is the mailbox-owner principal (the mailbox is the container).

App registration: Graph Application permission Mail.Read (admin consent).

Env var	Meaning	Default
`EX_TENANT_ID`	Entra directory id	(required)
`EX_CLIENT_ID`	App registration client id	(required)
`EX_CLIENT_SECRET`	Client secret	(required)
`EXCHANGE_MAILBOXES`	Comma-separated mailbox (UPN) allowlist; empty = sync nothing	(empty)
`EXCHANGE_INTERVAL_MINUTES`	Sync interval	`30`
`EXCHANGE_MAX_ITEM_MB`	Oversize skip threshold (checked post-fetch)	`50`

Connector: Slack (content)¶

Per-channel cursor chains over conversations.history; one thread (parent + replies, via conversations.replies) becomes one format="json" document labeled SLACK_THREAD with HAS_MESSAGES children; files attached to messages route by extension. A per-channel daily ceiling batches overflow threads into a rollup document.

Slack app: create a bot, install to the workspace, and grant scopes covering the methods the connector calls: conversations.history, conversations.replies, users.info (profile emails for authorship), and file downloads via url_private. (Scope names are not pinned in code; grant what those methods require, read-only.)

Env var	Meaning	Default
`SLACK_BOT_TOKEN`	Bot token (`xoxb-...`)	(required)
`SLACK_CHANNELS`	Comma-separated channel-id allowlist; empty = sync nothing	(empty)
`SLACK_INTERVAL_MINUTES`	Sync interval	`30`
`SLACK_MAX_ITEM_MB`	Oversize skip threshold	`50`
`SLACK_ROLLUP_MAX_PER_DAY`	Thread-docs per channel per UTC day before rollup	`50`

Recorded v1 stances: no ACL mirroring (channel membership churns), and no delete propagation (polling surfaces no delete delta). Authors ride as email principals only when the Slack profile exposes an email; Slack identities never create binding authority (see the Entra verified-match rule).

Connector: Salesforce¶

Per-object windows over the REST updated/deleted feeds; records ride format=json (describe-shaped scalar content), labeled with the UPPER_SNAKE object API name. Deletes propagate. A stale cursor (past the feeds' 30-day reach) triggers an in-place full resync via paged SOQL id enumeration, which doubles as the initial backfill.

Connected app: enable OAuth client-credentials flow with a run-as integration user; record consumer key/secret. REST API version pinned v62.0.

Env var	Meaning	Default
`SF_INSTANCE_URL`	`https://<org>.my.salesforce.com`	(required)
`SF_CLIENT_ID`	Connected app consumer key	(required)
`SF_CLIENT_SECRET`	Connected app consumer secret	(required)
`SF_USERNAME`	Operator metadata only (client-credentials runs as the connected app's run-as user)	optional
`SF_OBJECTS`	Comma-separated SObject allowlist; empty = sync nothing	(empty)
`SALESFORCE_INTERVAL_MINUTES`	Sync interval	`30`
`SALESFORCE_MAX_ITEM_MB`	Oversize skip threshold	`50`

Notes/Files sync is opt-in: allowlist ContentDocument in SF_OBJECTS. Recorded v1 stances: no ACL mirroring (record-level sharing is not cheaply exportable; Telha RBAC is the only gate), and a full resync carries no delete feed.

What sourceMeta and authority routing buy you¶

Every synced item carries options.sourceMeta (camelCase, as built): {connector (required), externalId, containerPath, sourceUrl, createdAt, modifiedAt, author?, lastModifiedBy?, owner?, acl?, extra?} with principals as {kind: aad_oid|email|slack_user|sf_user, value}. Core resolves principals to PERSON nodes at apply time (canonical when an alias matches, placeholder-by-email otherwise) and writes AUTHORED_BY / LAST_MODIFIED_BY / OWNED_BY edges from the projection root.

That buys you:

Clarification authority routing. When Telha is unsure about a fact from a document, it asks the document's last-modified-by/author/ owner people, in that authority order, instead of a generic queue.
Provenance deep links. sourceUrl lands in traces and evidence, so "open it in SharePoint" is one click, behind the source's own login.
Temporal grounding. createdAt/modifiedAt become source-record time.
Point lookups by source identity. GET /v1/records?externalId=...&connector=... resolves a source item to its projection root (connector tokens infer the namespace).

ACL semantics: floor-only, advisory¶

Mirrored source ACLs (acl in sourceMeta, stored as acl_json on the source version) are advisory and floor-only: they may only NARROW who is shown content (the clarification gate's source-side check), and never widen access beyond Telha workspace RBAC. Both directions hold: a person in Telha RBAC but absent from the item ACL is skipped for that item, and a person in the ACL but absent from RBAC is also skipped. An unparseable acl_json fails closed. Telha RBAC remains the single enforcement point; ACLs go stale between syncs by design.

Deletion is namespace-scoped¶

A connector token may tombstone only records its own namespace synced: core checks the root node's source_connector property on every connector-token DELETE /v1/records/:id and rejects mismatches with 403 CONNECTOR_NAMESPACE_MISMATCH plus a security audit event. A compromised SharePoint connector can never delete Slack-synced, uploaded, or manually created records. API keys are not narrowed; they remain tenant-admin credentials.