Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
ekk

ekk

  1. Home
  2. Categories
  3. ActivityPub
  4. Backfilling Conversations: Two Major Approaches

Backfilling Conversations: Two Major Approaches

Scheduled Pinned Locked Moved ActivityPub
activitypubfep7888f228171b
26 Posts 8 Posters 21 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • silverpill@mitra.socialS [email protected]

    @mikedev @julian

    hen conversation containers are working correctly, you never need to backfill a conversation. It is all delivered to you.

    I think there is one case where backfill is necessary: public conversations that are not discovered through following (e.g. by retrieving an object by its ID).

    julian@community.nodebb.orgJ This user is from outside of this forum
    julian@community.nodebb.orgJ This user is from outside of this forum
    [email protected]
    wrote last edited by
    #11

    [email protected] [email protected] correct. Backfill is important even when you have good synchronization systems in place.

    One example I use is Lemmy's use of 1b12. It is exceedingly good at keeping subscribers in sync, but if you discover a new node or leaf, then backfill is required to get you the conversation up to that point.

    1 Reply Last reply
    0
    • silverpill@mitra.socialS [email protected]

      @mikedev @julian

      hen conversation containers are working correctly, you never need to backfill a conversation. It is all delivered to you.

      I think there is one case where backfill is necessary: public conversations that are not discovered through following (e.g. by retrieving an object by its ID).

      mikedev@fediversity.siteM This user is from outside of this forum
      mikedev@fediversity.siteM This user is from outside of this forum
      [email protected]
      wrote last edited by
      #12
      True, but fetch one collection and you've got it all. Might be paged, and with Mastodon that means another fetch for every ten activities (seriously?), but those are just implementation details.
      1 Reply Last reply
      0
      • silverpill@mitra.socialS [email protected]

        @julian @trwnh @mikedev

        neither approach conflicts with the other

        I don't fully agree with this statement, because these "threading paradigms" suggest two different solutions to the problem of moderation. If the OP is the single source of truth, they can moderate the entire conversation (represented by context collection: Streams). If not, then each reply is independent and authors moderate only the direct replies (represented by replies collections: GoToSocial).

        In theory two solutions can be combined, but at the cost of significantly increased complexity.

        julian@community.nodebb.orgJ This user is from outside of this forum
        julian@community.nodebb.orgJ This user is from outside of this forum
        [email protected]
        wrote last edited by
        #13

        [email protected] said:
        > If the OP is the single source of truth, they can moderate the entire conversation (represented by context collection: Streams). If not, then each reply is independent and authors moderate only the direct replies (represented by replies collections: GoToSocial).

        That is a good point. The approaches are broadly compatible when top-down moderation by the context owner is not assumed.

        In a moderated scenario, crawling the reply tree would not be useful unless paired with some sort of "is member of" validation with the context owner... at which point the served collection would be more performant.

        It could be useful for discovery by the context owner itself though.

        1 Reply Last reply
        0
        • mikedev@fediversity.siteM This user is from outside of this forum
          mikedev@fediversity.siteM This user is from outside of this forum
          [email protected]
          wrote last edited by
          #14
          I think a couple of folks have mentioned trying to consolidate both of these approaches into one. I once used something that resembled 1b12 (long before there was a "threadiverse"), but as I recall it didn't really work well with private groups and aspects/circles -  where you're often dealing with third-party permissions. You can only relay public activities to third parties via an Announce, and so conversations with restricted audiences don't work out very well for viewers on Mastodon. The third party does not have permission to access the activity from its author, only from the conversation owner. Once you've run into this issue, you are likely to more fully understand the advantages and disadvantages of these two approaches. Container operations are pure relays and work correctly with third-party access control, assuming you're using signed objects (which everybody should be using, but that's a hill to die on another day).
          silverpill@mitra.socialS 1 Reply Last reply
          0
          • mikedev@fediversity.siteM [email protected]
            I think a couple of folks have mentioned trying to consolidate both of these approaches into one. I once used something that resembled 1b12 (long before there was a "threadiverse"), but as I recall it didn't really work well with private groups and aspects/circles -  where you're often dealing with third-party permissions. You can only relay public activities to third parties via an Announce, and so conversations with restricted audiences don't work out very well for viewers on Mastodon. The third party does not have permission to access the activity from its author, only from the conversation owner. Once you've run into this issue, you are likely to more fully understand the advantages and disadvantages of these two approaches. Container operations are pure relays and work correctly with third-party access control, assuming you're using signed objects (which everybody should be using, but that's a hill to die on another day).
            silverpill@mitra.socialS This user is from outside of this forum
            silverpill@mitra.socialS This user is from outside of this forum
            [email protected]
            wrote last edited by
            #15

            @mikedev @julian

            but as I recall it didn't really work well with private groups and aspects/circles

            Last time I heard about 1b12 private groups, the proposed solution was to use a "collection inclusion endpoint" to verify that actor is a member of a group

            julian@community.nodebb.orgJ 1 Reply Last reply
            0
            • silverpill@mitra.socialS [email protected]

              @mikedev @julian

              but as I recall it didn't really work well with private groups and aspects/circles

              Last time I heard about 1b12 private groups, the proposed solution was to use a "collection inclusion endpoint" to verify that actor is a member of a group

              julian@community.nodebb.orgJ This user is from outside of this forum
              julian@community.nodebb.orgJ This user is from outside of this forum
              [email protected]
              wrote last edited by
              #16

              [email protected] do you still need to if you're not using a shared inbox?

              silverpill@mitra.socialS 1 Reply Last reply
              0
              • scott@loves.techS This user is from outside of this forum
                scott@loves.techS This user is from outside of this forum
                [email protected]
                wrote last edited by
                #17
                @julian It should be noted that a platform receiving a moderated conversation thread does not have to honor it for its own local users. Whether this is desired or not is another discussion.

                In this case, the owner of the thread (either the forum or the person who started the thread) tells you what comments are part of the thread. Some comments may be removed due to moderator actions or user-initiated blocks.

                But as a remote platform importing the thread, you may be aware of other replies that are part of the reply tree, but not in the official moderated version of the conversation according to the thread owner.

                As a remote platform, you have an option. You can honor the thread owner's official version of the thread and only display the moderated version, or you can modify it. You may remove replies from actors blocked on your server, for example. But you could also add comments from the reply tree that are not part of the moderated version of the conversation.
                1 Reply Last reply
                0
                • julian@community.nodebb.orgJ [email protected]

                  [email protected] do you still need to if you're not using a shared inbox?

                  silverpill@mitra.socialS This user is from outside of this forum
                  silverpill@mitra.socialS This user is from outside of this forum
                  [email protected]
                  wrote last edited by
                  #18

                  @julian @mikedev Yes, if you receive an Announce(Create), and Create is not signed, then you need to retrieve this Create from its origin. When that origin server receives your signed GET request, it needs to verify that you belong to the group, but it might not have that information.

                  1 Reply Last reply
                  0
                  • scott@loves.techS This user is from outside of this forum
                    scott@loves.techS This user is from outside of this forum
                    [email protected]
                    wrote last edited by
                    #19
                    Just thought of something interesting. In the case of moderated threads, it may be useful to tell other platforms that you know about a particular comment, but have removed it on purpose from the official moderated version of the thread. Because there is a difference between "I didn't know about that reply due to a technical issue" and "this content was removed by a moderator."
                    julian@community.nodebb.orgJ 1 Reply Last reply
                    0
                    • scott@loves.techS [email protected]
                      Just thought of something interesting. In the case of moderated threads, it may be useful to tell other platforms that you know about a particular comment, but have removed it on purpose from the official moderated version of the thread. Because there is a difference between "I didn't know about that reply due to a technical issue" and "this content was removed by a moderator."
                      julian@community.nodebb.orgJ This user is from outside of this forum
                      julian@community.nodebb.orgJ This user is from outside of this forum
                      [email protected]
                      wrote last edited by
                      #20

                      That'd be accomplished with a Remove activity, most likely.

                      For those expressing the context collection as a set of objects, then removal from the set should suffice. There are probably better signals to send.

                      scott@loves.techS 1 Reply Last reply
                      0
                      • julian@community.nodebb.orgJ [email protected]

                        That'd be accomplished with a Remove activity, most likely.

                        For those expressing the context collection as a set of objects, then removal from the set should suffice. There are probably better signals to send.

                        scott@loves.techS This user is from outside of this forum
                        scott@loves.techS This user is from outside of this forum
                        [email protected]
                        wrote last edited by
                        #21
                        @julian Wouldn't a remove would remove it from everywhere, including the server of the person who posted it. That may be desired, but also could lead to confusion, since on many platforms like Mastodon, they can't see threads and don't realize their comment can be deleted everywhere (including their own copy).
                        julian@community.nodebb.orgJ 1 Reply Last reply
                        0
                        • scott@loves.techS [email protected]
                          @julian Wouldn't a remove would remove it from everywhere, including the server of the person who posted it. That may be desired, but also could lead to confusion, since on many platforms like Mastodon, they can't see threads and don't realize their comment can be deleted everywhere (including their own copy).
                          julian@community.nodebb.orgJ This user is from outside of this forum
                          julian@community.nodebb.orgJ This user is from outside of this forum
                          [email protected]
                          wrote last edited by
                          #22

                          [email protected] not necessarily, a remove merely represents that it has been removed from a collection. A Delete would instruct the recipient servers to purge the object, and that can't be done unless the actor is same-origin.

                          scott@loves.techS 1 Reply Last reply
                          0
                          • julian@community.nodebb.orgJ [email protected]

                            [email protected] not necessarily, a remove merely represents that it has been removed from a collection. A Delete would instruct the recipient servers to purge the object, and that can't be done unless the actor is same-origin.

                            scott@loves.techS This user is from outside of this forum
                            scott@loves.techS This user is from outside of this forum
                            [email protected]
                            wrote last edited by
                            #23
                            @julian Okay, that makes sense.
                            1 Reply Last reply
                            0
                            • projectmoon@forum.agnos.isP This user is from outside of this forum
                              projectmoon@forum.agnos.isP This user is from outside of this forum
                              [email protected]
                              wrote last edited by
                              #24

                              [email protected] said in Backfilling Conversations: Two Major Approaches:
                              > A number of implementors follow this approach to backfill, including NodeBB, Discourse, WordPress, Frequency, Mitra, and Streams. Additional implementors like Lemmy and Piefed have expressed interest.

                              Is this implemented currently? One weakness I have noticed in NodeBB's current federation is that posts which are in reply to a topic (e.g. a Lemmy comment) show up as individual threads until (or if) the root post of that topic shows up in the local NodeBB. It's a bit confusing of a UX, I think. Because you think:

                              1. (Before root post) Why is this a post that seems to be just a random comment?
                              2. (After root post) Why did that other thread disappear and why am I seeing the same comment again?
                              julian@community.nodebb.orgJ 1 Reply Last reply
                              0
                              • projectmoon@forum.agnos.isP [email protected]

                                [email protected] said in Backfilling Conversations: Two Major Approaches:
                                > A number of implementors follow this approach to backfill, including NodeBB, Discourse, WordPress, Frequency, Mitra, and Streams. Additional implementors like Lemmy and Piefed have expressed interest.

                                Is this implemented currently? One weakness I have noticed in NodeBB's current federation is that posts which are in reply to a topic (e.g. a Lemmy comment) show up as individual threads until (or if) the root post of that topic shows up in the local NodeBB. It's a bit confusing of a UX, I think. Because you think:

                                1. (Before root post) Why is this a post that seems to be just a random comment?
                                2. (After root post) Why did that other thread disappear and why am I seeing the same comment again?
                                julian@community.nodebb.orgJ This user is from outside of this forum
                                julian@community.nodebb.orgJ This user is from outside of this forum
                                [email protected]
                                wrote last edited by
                                #25

                                > One weakness I have noticed in NodeBB's current federation is that posts which are in reply to a topic (e.g. a Lemmy comment) show up as individual threads until (or if) the root post of that topic shows up in the local NodeBB.

                                No, Lemmy does not implement either strategy, they rely on 1b12 only.

                                If NodeBB is receiving parts of a topic that don't resolve up to the root-level post that might be something we can fix. I'll try to take a look at it.

                                1 Reply Last reply
                                0
                                Reply
                                • Reply as topic
                                Log in to reply
                                • Oldest to Newest
                                • Newest to Oldest
                                • Most Votes


                                • Login

                                • Don't have an account? Register

                                • Login or register to search.
                                Powered by NodeBB Contributors
                                • First post
                                  Last post
                                0
                                • Categories
                                • Recent
                                • Tags
                                • Popular
                                • World
                                • Users
                                • Groups