FEP 7888 serving up an OrderedCollection
-
Just wrapped up a call with @pfefferle@mastodon.social and @jesseplusplus@mastodon.social to review their implementations of FEP 7888, specifically in relation to conversational backfill.
individual objects serve a
context
property
that
context
property is a URL that resolvesOne of the concerns raised was related to the OrderedCollection of items served by the
context
. Specifically, if the items presented in the collection were not in chronological order, NodeBB failed at importing some of the items as theinReplyTo
referenced an object that did not exist.The solution to this was to ensure that the collection items were in chronological order from oldest to newest. Once fixed:
the context resolved to an OrderedCollection containing objects
NodeBB was able to pull in the entire conversation
NodeBB used to guard against this by ordering all received items by chronological order, but I realized that while this worked 99%+ of the time, there are some fun (ahem...) individuals who send objects with timestamps way in the future.
Personally I think removing the sorting just to fix one edge case was premature. At the same time, I think specifying that the OrderedCollection be sorted in chronological order should be a requirement.
-
@julian @pfefferle @jesseplusplus shouldn't the order basically be "whatever the owner gives you"? i can see there being a SHOULD on forward-chron maybe but it's also possible for reverse chron, order of acknowledgement, or anything else. by default Adding to a collection might append a thing to either end of it. probably the simplest thing is the latest inserted item becomes the first, and the previous list becomes the rest. Reordering of collections, sorting of collections, etc is not possible
-
@julian @pfefferle @jesseplusplus shouldn't the order basically be "whatever the owner gives you"? i can see there being a SHOULD on forward-chron maybe but it's also possible for reverse chron, order of acknowledgement, or anything else. by default Adding to a collection might append a thing to either end of it. probably the simplest thing is the latest inserted item becomes the first, and the previous list becomes the rest. Reordering of collections, sorting of collections, etc is not possible
@julian @pfefferle @jesseplusplus but you should be able to handle an object where inReplyTo doesn't fully dereference (yet or ever). it shouldn't break processing
-
@julian Are the future posts scheduled posts. Do they have a "scheduled" flag or something similar? Is timezone info included by default? Those are the two situations I can think of where this would occur with "timestamps in the future".
-
@julian @pfefferle @jesseplusplus shouldn't the order basically be "whatever the owner gives you"? i can see there being a SHOULD on forward-chron maybe but it's also possible for reverse chron, order of acknowledgement, or anything else. by default Adding to a collection might append a thing to either end of it. probably the simplest thing is the latest inserted item becomes the first, and the previous list becomes the rest. Reordering of collections, sorting of collections, etc is not possible
@trwnh@mastodon.social yeah that was what I was afraid of. Maybe a SHOULD?
What we discussed on the call was "keep it simple" for implementors, which would be to have implementors order by forward chron.
End of the day since NodeBB is the one having trouble with the order I think it's on me to re-add re-ordering so other implementors can send in whatever order they want.
cc @pfefferle@mastodon.social @jesseplusplus@mastodon.social
-
@julian Are the future posts scheduled posts. Do they have a "scheduled" flag or something similar? Is timezone info included by default? Those are the two situations I can think of where this would occur with "timestamps in the future".
@harmonicarichard@techhub.social if an object gets federated out with a future timestamp but is meant to be scheduled, I think that's a bug.
I wouldn't trust other implementors to respect post scheduling
-
@julian @pfefferle @jesseplusplus i've got some prior fep work on OrderedCollection.orderType and SortedCollection.sortedBy|sortOrder (1985, 1863 but not submitted yet)
but there are deeper issues when trying to do anything with Collections other than just read them as-is... i've filed several issues on w3c/activitypub etc but tldr idk what the best solution is here. the existing abstraction of as2 collections falls short when you are trying to represent data structures other than a basic set.
-
@julian I have a classicpress and a wordpress instance where I could test scheduled posts. It might help to check instance logs. I can test scheduled posts from classicpress with ease.
-
@julian @pfefferle @jesseplusplus @trwnh +1 for chronological order requirement.
Are those implementations public? I'd like to test my context resolver against them too -
@julian @pfefferle @jesseplusplus @trwnh +1 for chronological order requirement.
Are those implementations public? I'd like to test my context resolver against them too@silverpill@mitra.social yes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
We were testing against these URLs from @pfefferle@mastodon.social's personal blog:
- Top level
Article
: https://notiz.blog/2025/02/11/fedidem/ - Mid-level reply
Note
: https://notiz.blog/?c=2045174
@jesseplusplus@mastodon.social had a test URL but NodeBB fell over because it encountered an Object in
next
instead of a URL, so that's my bad: - Top level
-
@julian @silverpill @pfefferle Right now my fork’s implementation has just a Collection rather than an OrderedCollection, but I’m going to change that to an OrderedCollection based on today’s discussion
-
@julian @pfefferle @jesseplusplus @harmonicarichard @trwnh
>yes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
My server can retrieve both kinds of collections
I had concerns about diverging / conflicting implementations in the past, but the solution was found...
>We were testing against these URLs from @pfefferle@mastodon.social's personal blog
This context is working
>@jesseplusplus@mastodon.social had a test URL but NodeBB fell over because it encountered an Object in next instead of a URL
I have a problem with this one because the
first
page doesn't have anid
. I can adjust my code but the absence ofid
is unusual. For example, there is anext
page (currently 404), and if we navigate to it, howprev
would look like if the first page is anonymous?>All top level or mid-level objects should report a resolvable context
Do you mean replies made by the context owner specifically? I think remote mid-level replies should not be required to have
context
(that would prevent non-implementing servers from participating). -
@julian @pfefferle @jesseplusplus @harmonicarichard @trwnh
>yes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
My server can retrieve both kinds of collections
I had concerns about diverging / conflicting implementations in the past, but the solution was found...
>We were testing against these URLs from @pfefferle@mastodon.social's personal blog
This context is working
>@jesseplusplus@mastodon.social had a test URL but NodeBB fell over because it encountered an Object in next instead of a URL
I have a problem with this one because the
first
page doesn't have anid
. I can adjust my code but the absence ofid
is unusual. For example, there is anext
page (currently 404), and if we navigate to it, howprev
would look like if the first page is anonymous?>All top level or mid-level objects should report a resolvable context
Do you mean replies made by the context owner specifically? I think remote mid-level replies should not be required to have
context
(that would prevent non-implementing servers from participating). -
@julian @pfefferle @jesseplusplus @harmonicarichard @trwnh
>yes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
My server can retrieve both kinds of collections
I had concerns about diverging / conflicting implementations in the past, but the solution was found...
>We were testing against these URLs from @pfefferle@mastodon.social's personal blog
This context is working
>@jesseplusplus@mastodon.social had a test URL but NodeBB fell over because it encountered an Object in next instead of a URL
I have a problem with this one because the
first
page doesn't have anid
. I can adjust my code but the absence ofid
is unusual. For example, there is anext
page (currently 404), and if we navigate to it, howprev
would look like if the first page is anonymous?>All top level or mid-level objects should report a resolvable context
Do you mean replies made by the context owner specifically? I think remote mid-level replies should not be required to have
context
(that would prevent non-implementing servers from participating).@silverpill @julian @pfefferle @jesseplusplus @harmonicarichard the context SHOULD be copied if you want to participate in the same context, but the owner MAY Add whatever they want to an arbitrary Collection
context-unaware impls aren't prevented from participating but this will lead to a degraded experience if you're not careful. likely you would start with some graph source, filter for context, then optionally crawl replies for anything missing or otherwise optionally reverse query inReplyTo
-
@silverpill @julian @pfefferle @jesseplusplus @harmonicarichard the context SHOULD be copied if you want to participate in the same context, but the owner MAY Add whatever they want to an arbitrary Collection
context-unaware impls aren't prevented from participating but this will lead to a degraded experience if you're not careful. likely you would start with some graph source, filter for context, then optionally crawl replies for anything missing or otherwise optionally reverse query inReplyTo
@trwnh@mastodon.social @silverpill@mitra.social I'd argue that the broad baseline implementation is to allow anyone to participate in the same context... mostly because you can't guard against other implementors specifying your
context
value.However, there's nothing stopping a future FEP ("context ownership"/"reply controls") from recommending that implementors check against the context first before locally adding a received object that provides
context
. -
@julian @silverpill right, it's a bidirectional link that should be verified both ways ideally. so you have an object claiming to be included, but you also ideally have a reverse claim of inclusion
i think you can mostly get away with doing something like:
- if context is present and it has a canonical collection, you can browse it directly. ignore the original object
- if someone declares your context and you become aware of it somehow, you can optionally add it to the canonical collection -
@julian @silverpill @pfefferle Right now my fork’s implementation has just a Collection rather than an OrderedCollection, but I’m going to change that to an OrderedCollection based on today’s discussion
@jesseplusplus@mastodon.social @pfefferle@mastodon.social I fixed up NodeBB's janky handling of collections and now I am able to import an entire Frequency conversation via its
context
I think @silverpill@mitra.social is right though, your
context.first
needs anid
. I worked around it but it's better to have it so it can be referenced against by another page'sprev
. -
@julian @silverpill right, it's a bidirectional link that should be verified both ways ideally. so you have an object claiming to be included, but you also ideally have a reverse claim of inclusion
i think you can mostly get away with doing something like:
- if context is present and it has a canonical collection, you can browse it directly. ignore the original object
- if someone declares your context and you become aware of it somehow, you can optionally add it to the canonical collection@julian @silverpill the third point which is more contentious:
- deleting a context implies all objects included in it are now orphans and can be garbage-collected (deleted, updated, moved, whatever)
-
@julian @silverpill the third point which is more contentious:
- deleting a context implies all objects included in it are now orphans and can be garbage-collected (deleted, updated, moved, whatever)
> deleting a context implies...
@trwnh@mastodon.social that would be yet another FEP
-
@silverpill@mitra.social yes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
We were testing against these URLs from @pfefferle@mastodon.social's personal blog:
- Top level
Article
: https://notiz.blog/2025/02/11/fedidem/ - Mid-level reply
Note
: https://notiz.blog/?c=2045174
@jesseplusplus@mastodon.social had a test URL but NodeBB fell over because it encountered an Object in
next
instead of a URL, so that's my bad:@julianyes, but they serve objects because we're radical implementors who don't do the whole activities thing
sorry in advance.
What if only the activity is signed (as is often the case) and the object is not fetchable due to let's say some network error? - Top level