Introduce a source-neutral ingestion model for external feeds #25

New issue

Closed

opened 2026-09-28 16:54:12 +00:00 by Michael Westergaard · 1 comment

Michael Westergaard commented 2026-09-28 16:54:12 +00:00

Owner

• Copy link
• Copy Markdown

Problem

Links and AutoBoost currently model ActivityPub actors and Announce of remote object IDs. RSS, Atom, and direct Bluesky need a common path for user-added sources and imported items without putting protocol logic in link, UI, or federation transport code.

Scope

• Define application/domain contracts for a linked source, normalized item, stable source/item identity, provenance, timestamps, URL, text, and media metadata. Preserve the source URL and author attribution.
• Add provider adapters behind those contracts; keep source-specific fetch/parse logic in Services and persistence behind Application repository interfaces. The Web layer only configures sources and shows status.
• Persist per-source polling/checkpoint state and item identity. Handle initial import limits, edits, duplicates, removals, disabled/deleted sources, transient failures, and bounded retries without duplicate publication.
• Apply outbound-fetch protections (HTTP(S) only, redirects/DNS/private-network restrictions, response-size and timeout limits) to user-supplied feed URLs. Treat external content as untrusted.
• Specify ownership/verification semantics: adding a public feed is not proof that the user owns it; verified-source claims require a separate mechanism.

Acceptance

Functional and contract tests pin normalization, idempotent polling, source lifecycle, error isolation, and fetch safety. Keep unit tests in regular CI; use manually run integration tests only for real HTTP/database boundaries. Document the boundary and migration of current ActivityPub-link behavior without regressions.

Dependency for RSS, Atom, and Bluesky source tickets. Publication to followers is tracked separately.

## Problem Links and AutoBoost currently model ActivityPub actors and Announce of remote object IDs. RSS, Atom, and direct Bluesky need a common path for user-added sources and imported items without putting protocol logic in link, UI, or federation transport code. ## Scope - Define application/domain contracts for a linked source, normalized item, stable source/item identity, provenance, timestamps, URL, text, and media metadata. Preserve the source URL and author attribution. - Add provider adapters behind those contracts; keep source-specific fetch/parse logic in Services and persistence behind Application repository interfaces. The Web layer only configures sources and shows status. - Persist per-source polling/checkpoint state and item identity. Handle initial import limits, edits, duplicates, removals, disabled/deleted sources, transient failures, and bounded retries without duplicate publication. - Apply outbound-fetch protections (HTTP(S) only, redirects/DNS/private-network restrictions, response-size and timeout limits) to user-supplied feed URLs. Treat external content as untrusted. - Specify ownership/verification semantics: adding a public feed is not proof that the user owns it; verified-source claims require a separate mechanism. ## Acceptance Functional and contract tests pin normalization, idempotent polling, source lifecycle, error isolation, and fetch safety. Keep unit tests in regular CI; use manually run integration tests only for real HTTP/database boundaries. Document the boundary and migration of current ActivityPub-link behavior without regressions. Dependency for RSS, Atom, and Bluesky source tickets. Publication to followers is tracked separately.

Michael Westergaard referenced this issue 2026-09-28 16:54:34 +00:00

Publish imported source items as local ActivityPub posts #26

Michael Westergaard referenced this issue 2026-09-28 16:55:00 +00:00

Import generic RSS feeds into profile feeds #27

Michael Westergaard referenced this issue 2026-09-28 16:55:18 +00:00

Import generic Atom feeds into profile feeds #28

Michael Westergaard referenced this issue 2026-09-28 16:55:37 +00:00

Synchronize an owned Bluesky account directly into the profile feed #29

Michael Westergaard referenced this issue 2026-09-28 16:56:07 +00:00

Add service-specific Atom setup and presentation (YouTube first) #30

Michael Westergaard commented 2026-09-29 10:43:29 +00:00

Author

Owner

• Copy link
• Copy Markdown

Implemented in commits 883e7f5 and 6430cea on michael. The external-source model, durable polling and checkpointing, item deduplication, and lifecycle handling are covered by unit and repository contract tests. Final Forgejo CI run 1101 and integration run 1102 passed. Publication as local ActivityPub posts is tracked in issue 26.

Implemented in commits 883e7f5 and 6430cea on michael. The external-source model, durable polling and checkpointing, item deduplication, and lifecycle handling are covered by unit and repository contract tests. Final Forgejo CI run 1101 and integration run 1102 passed. Publication as local ActivityPub posts is tracked in issue 26.

Michael Westergaard closed this issue 2026-09-29 10:43:29 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

michael

main

features/single-user-instance

0.9.16

0.9.15

0.9.14

0.9.13

0.9.12

0.9.11

0.9.10

0.9.9

0.9.8

0.9.7

0.9.6

0.9.5

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.0.3

0.0.2

0.0.1

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

klafbang/fediprofile#25

No description provided.