Documentation Wiki

Metadata

Objective

The main purpose of this document is to establish the final API for version 1.0 of the library.

This includes all the modules that the GoActivityPub library is currently composed of.

Background

Since its inception the GoActivityPub library developed has been driven by a continuous loop of assessing the specifications it implements, developing functionality for that and, using the resulting features in reference applications, which in turn expose incompatibilities or rough edges.

We’re finally in a place where the loop has run enough times that there are no more obvious places that require added functionality, and we can focus on unifying the existing elements into a robust and polished version.

The way we want to signal this is by tagging a stable version.

Goals

We want to change the structure of the models from what it is today. This needs to be done because functionality has shifted between modules, some have been simplified to the point that they don’t make sense independently, and some of them can be consolidated.

This is the simplified plan:

  1. The activitypub module will be renamed to vocab (or perhaps lexicon) and contain the vanilla types described in the Activity Vocabulary specification.
  2. The processing module will become the new activitypub module, and it will contain the state machine for processing client to server and server to server activities.
    • It will include sub-packages for convenience functionality to provide better interoperation with the wider Go ecosystem: HTTP handlers and middleware
    • The auth module moves as a sub-package here. Now that the osin server has been removed, maybe we can even rename it to something else. Its main function currently is to load from various authorization headers the corresponding Actor entities.
  3. The client module mostly stays as it is. Perhaps we move the OAuth2 and HTTP-Signatures round trippers to the authorization module.
    • Another option is to move it as a sub-package of processing.
  4. Add context.Context arguments to functions and methods to allow canceling:
    • The storage methods
    • The Processing methods and functions

Non-Goals

A very large part of making the GoActivityPub library compatible with the wider Fediverse is to allow for highly dynamic Activity Vocabulary objects.

As this contradicts the “statically typedness” of the Go programming language and the specific goal of the library to provide data types identical with their on the wire representation, the solution is to build a code generation tool that can build these dynamic types at compile time based on their JSON-LD context documents.

This code generation tool is not going to be part of version 1.0 of the GoActivityPub library. A separate document will be created for it at a later date, ideally after all the goals of this document have been reached.

We consider that all the work done under this RFC is smoothing the way for this future development.

We have started documenting the requirements for this future RFC.

Processing activity methods

We want the processing of activities be cancelable, so we need to accept a context argument.

 ProcessActivity(it vocab.Item, author vocab.Actor, receivedIn vocab.IRI) (vocab.Item, error)
 // becomes
 ProcessActivity(ctx context.Context, it vocab.Item, author vocab.Actor, receivedIn vocab.IRI) (vocab.Item, error)

Additionally the processing.Store interface will receive a context so that the implementations can cancel storage operations.

type Store interface {
	// Load returns an Item or an ItemCollection from an IRI
	// after filtering it through the FilterFn list of filtering functions. Eg ANY()
	Load(vocab.IRI, ...filters.Check) (vocab.Item, error)

	// Save saves the incoming vocabulary Object, and returns it together with any properties
	// populated by the method's side effects. (eg, Published property can point to the current time, etc.).
	Save(vocab.Item) (vocab.Item, error)
	// Delete completely deletes from storage the vocabulary Object, this is usually
	// a side effect of an Undo activity.
	Delete(vocab.Item) error

	// AddTo adds "it" element to the "col" collection.
	AddTo(vocab.IRI, ...vocab.Item) error
	// RemoveFrom removes "it" item from "col" collection
	RemoveFrom(vocab.IRI, ...vocab.Item) error
}
// becomes

type Store interface {
	// Load returns an Item or an ItemCollection from an IRI
	// after filtering it through the FilterFn list of filtering functions. Eg ANY()
	Load(context.Context, vocab.IRI, ...filters.Check) (vocab.Item, error)

	// Save saves the incoming vocabulary Object, and returns it together with any properties
	// populated by the method's side effects. (eg, Published property can point to the current time, etc.).
	Save(context.Context, vocab.Item) (vocab.Item, error)
	// Delete completely deletes from storage the vocabulary Object, this is usually
	// a side effect of an Undo activity.
	Delete(context.Context, vocab.Item) error

	// AddTo adds "it" element to the "col" collection.
	AddTo(context.Context, vocab.IRI, ...vocab.Item) error
	// RemoveFrom removes "it" item from "col" collection
	RemoveFrom(context.Context, vocab.IRI, ...vocab.Item) error
}

The way we want this to be used is similar to the example in this slide:


func (s *store) Load(ctx context.Context, iri vocab.IRI, f ...filters.Check) (vocab.Item, error) {
	ctx, cancel := context.WithCancel(ctx)
	defer cancel()

	c := make(chan vocab.Item)
	go s.asyncLoad(c, iri, f...)

	select {
	case <-ctx.Done():
		return nil, ctx.Err()
	case it := <-c:
		return it, nil
	}
	return nil, errors.New()
}

General module rules

When applicable, each module needs to have a single main data type that operates on, this is not entirely idiomatic with the Go language suggestion of keeping everything related in the same package.

The initializer can return un-exported data types.

The vocabulary/lexicon

Currently all the types present in the Activity Vocabulary specification are present and usable from the GoActivityPub library.

There are no changes required at present.

The ActivityPub state machine

We split the logic of the new activitypub module into three distinct pieces of logic:

The ActivityPub state machine needs to operate the following:

  1. Validate received Activity.
    1. Determine which properties need to be dereferenced.
    2. Dereference properties and save them to local storage.
    3. Validate dereferenced objects.
  2. Persist flattened activity to storage.
  3. Operate Activity side-effects.
  4. Aggregate recipients.
  5. Dereference recipients.(?)
  6. Disseminate to local recipients.
  7. Disseminate to remote recipients.

Data types

We need some interfaces that unifies behaviour between client to server and server to server logic.

We should establish if the current authorized actor needs to be part of the API, or we can leave it as an implementation detail.

The implementation for the Dereferencer would be API agnostic and would probably serve in other modules.

For an early example of how the dereferencing state machine would work see the ActivityPub client example.

For example, it could be used in auth module for dereferencing Actor public keys.

The implementations for the Processor and Validator would be scoped per each section of the specification:

// dereferencer is a helper interface that gets used
// by the [Validator] to enrich an activity with additional information.
type dereferencer interface {
	// Add adds the IRIs to the fetch process.
	Add(iri ...vocab.IRI)
	// OnFetched calls fn function with the resulting IRI, Item pair
	OnFetched(fn func(vocab.IRI, vocab.Item))
	// OnFetchError calls fn function with the resulting IRI, error pair
	OnFetchError(fn func(vocab.IRI, error))
}

// disseminator is a helper interface that gets used
// by the [Processor] to submit activities to recipients.
type disseminator interface {
	ToCollection(colIRI vocab.IRI, it vocab.Item) error
}

// readStore is a helper interface that gets used
// by the [Processor] or [Validator] to retrieve 
// objects and activities from *local* storage.
type readStore interface {
	Load(iri vocab.IRI, ff ...filters.Check) (vocab.Item, error)
}

// readStore is a helper interface that gets used
// by the [Processor] to save objects and collections to *local* storage.
type writeStore interface {
	// Save saves the incoming ActivityStreams Object, and returns it together with any properties
	// populated by the method's side effects. (eg, Published property can point to the current time, etc.).
	Save(vocab.Item) (vocab.Item, error)
	// Delete deletes completely from storage the ActivityStreams Object, this is usually
	// a side effect of an Undo activity.
	Delete(vocab.Item) error

	// AddTo adds "it" elements to the "col" collection.
	AddTo(vocab.IRI, ...vocab.Item) error
	// RemoveFrom removes "it" item from "col" collection
	RemoveFrom(vocab.IRI, ...vocab.Item) error
}

// store is a helper interface that gets used
// by the [Processor] or [Validator] to retrieve and/or store 
// objects and activities from *local* storage.
// It's implemented by individual storage backends.
type store interface {
	readStore
	writeStore
}

type Validator interface {
	// Validate validates an activity.
	// It dereferences all properties that need to be dereferenced and validates it.
	// The "it" parameter gets enriched with the dereferenced properties. 
	Validate(ctx context.Context, it vocab.Item) error
}

type Processor interface {
	// Process processes the incoming activity
	// The "it" parameter gets enriched with whichever additional information operating 
	// the activity adds.
	// Eg. In the case of a Create activity operated in an outbox collection, its Object 
	// would receive an ID, or the Published property gets added if they're missing.
	Process(ctx context.Context, it vocab.Item) error
}

The ActivityPub client

Currently the client module consists of a wrapper for a standard library http.Client which has additional functionality for the ActivityPub specific operations.

Additionally there are http.RoundTripper implementations that are used for authorization in the Social API and with the Federation Protocol.

We must decide if we want the RoundTripper implementations for s2s and proxy, maybe even debug round trippers should be moved to something else. According to the Go developers cramming this kind of logic into RoundTrippers is not the way.