Usage¶
Entities¶
Federation has it’s own base entity classes. When incoming messages are processed, the protocol specific entity mappers transform the messages into our base entities. In reverse, when creating outgoing payloads, outgoing protocol specific messages are constructed from the base entities.
Entity types are as follows below.
Protocol entities¶
Each protocol additionally has it’s own variants of the base entities, for example
Diaspora entities in federation.entities.diaspora.entities
. All the protocol
specific entities subclass the base entities so you can safely work with for example
DiasporaPost
and use isinstance(obj, Post)
.
When creating incoming objects from messages, protocol specific entity classes are returned. This is to ensure protocol specific extra attributes or methods are passed back to the caller.
For sending messages out, either base or protocol specific entities can be passed to the outbound senders.
If you need the correct protocol specific entity class from the base entity,
each protocol will define a get_outbound_entity
function.
Federation identifiers¶
All entities have an id
to guarantee them a unique name in the network.
The format of the id
depends on the protocol in question.
ActivityPub: maps to the object
id
(whether wrapped in an Activity or not)Diaspora: maps to
guid
for the entity.
Profiles¶
Profiles are uniquely identified by the id
as above. Additionally for Diaspora they always have a handle
.
ActivityPub profiles can also have a handle
but it is optional.
A handle will always be in email like format, without the @ prefix found on some platforms. This will be added to outgoing payloads where needed.
Creator and owner identifiers¶
All entities except Profile
have an actor_id
which tells who created this object or activity. The format
depends on the protocol
in question.
ActivityPub: maps to Object
attributedTo
or Activityactor_id
.Diaspora: maps to entity
author
Activity identifiers¶
Entities which are an activity on something, for example creating, updating, deleting, following, etc, should have
an activity_id
given to be able to send out to the ActivityPub protocol.
Mentions¶
Entities store mentions in the list _mentions
. The list is a simple list of strings which will be either
an URL format profile.id
or handle, as per above examples.
The syntax for a mention in text is URL format @{<profile.id>}
or @{<profile.handle>}
.
The GUID format profile.id
cannot be used for a mention.
Examples:
# profile.handle
Hello @{user@domain.tld}!
# profile.id in URL format
Hello @{https://domain.tld/user}
It is suggested profile.handle
syntax is used always for textual mentions unless handles are not available.
Inbound¶
Mentions are added to the entity _mentions
list when processing inbound entities. For ActivityPub they will be
extracted from Mention
tags and for Diaspora extracted from the text using the Diaspora mention format.
Outbound¶
Mentions can be given in the _mentions
list. If not given, they will be extracted from the textual content
using the above formats in the example.
For ActivityPub they will be added as Mention
tags before sending. If the mention is in handle format,
a WebFinger fetch will be made to find the profile URL format ID.
For Diaspora they will be added to the text in the correct format, if not found. If they are found in the text in non-Diaspora format, they will be converted before sending.
Discovery¶
Federation provides many generators to allow providing discovery documents. They have been made as Pythonic as possible so that library users don’t have to meddle with the various documents and their internals.
The protocols themselves are too complex to document within this library, please consult protocol documentation on what kind of discovery documents are expected to be served by the application.
Generators¶
Helper methods¶
- federation.hostmeta.generators.generate_host_meta(template=None, *args, **kwargs)[source]¶
Generate a host-meta XRD document.
Template specific key-value pairs need to be passed as
kwargs
, see classes.- Parameters:
template – Ready template to fill with args, for example “diaspora” (optional)
- Returns:
Rendered XRD document (str)
- federation.hostmeta.generators.generate_legacy_webfinger(template=None, *args, **kwargs)[source]¶
Generate a legacy webfinger XRD document.
Template specific key-value pairs need to be passed as
kwargs
, see classes.- Parameters:
template – Ready template to fill with args, for example “diaspora” (optional)
- Returns:
Rendered XRD document (str)
- federation.hostmeta.generators.generate_hcard(template=None, **kwargs)[source]¶
Generate a hCard document.
Template specific key-value pairs need to be passed as
kwargs
, see classes.- Parameters:
template – Ready template to fill with args, for example “diaspora” (optional)
- Returns:
HTML document (str)
- federation.hostmeta.generators.generate_nodeinfo2_document(**kwargs)[source]¶
Generate a NodeInfo2 document.
Pass in a dictionary as per NodeInfo2 1.0 schema: https://github.com/jaywink/nodeinfo2/blob/master/schemas/1.0/schema.json
- Minimum required schema:
- {server:
baseUrl name software version
} openRegistrations
Protocols default will match what this library supports, ie “diaspora” currently.
- Returns:
dict
- Raises:
KeyError on missing required items
- federation.hostmeta.generators.get_nodeinfo_well_known_document(url, document_path=None)[source]¶
Generate a NodeInfo .well-known document.
See spec: http://nodeinfo.diaspora.software
- Parameters:
url – The full base url with protocol, ie https://example.com
document_path – Custom NodeInfo document path if supplied (optional)
- Returns:
dict
Generator classes¶
- class federation.hostmeta.generators.DiasporaHostMeta(*args, **kwargs)[source]¶
Diaspora host-meta.
Required keyword args:
webfinger_host (str)
- class federation.hostmeta.generators.DiasporaWebFinger(handle, host, guid, public_key, *args, **kwargs)[source]¶
Diaspora version of legacy WebFinger.
Required keyword args:
handle (str) - eg user@domain.tld
host (str) - eg https://domain.tld
guid (str) - guid of user
public_key (str) - public key
- class federation.hostmeta.generators.DiasporaHCard(**kwargs)[source]¶
Diaspora hCard document.
Must receive the required attributes as keyword arguments to init.
- class federation.hostmeta.generators.MatrixClientWellKnown(homeserver_base_url: str, identity_server_base_url: str | None = None, other_keys: Dict | None = None)[source]¶
Matrix Client well-known as per https://matrix.org/docs/spec/client_server/r0.6.1#server-discovery
- class federation.hostmeta.generators.MatrixServerWellKnown(homeserver_domain_with_port: str)[source]¶
Matrix Server well-known as per https://matrix.org/docs/spec/server_server/r0.1.4#server-discovery
- class federation.hostmeta.generators.NodeInfo(software, protocols, services, open_registrations, usage, metadata, skip_validate=False, raise_on_validate=False)[source]¶
Generate a NodeInfo document.
See spec: http://nodeinfo.diaspora.software
NodeInfo is unnecessarely restrictive in field values. We wont be supporting such strictness, though we will raise a warning unless validation is skipped with skip_validate=True.
For strictness, raise_on_validate=True will cause a ValidationError to be raised.
See schema document federation/hostmeta/schemas/nodeinfo-1.0.json for how to instantiate this class.
- class federation.hostmeta.generators.RFC7033Webfinger(id: str, handle: str, guid: str, base_url: str, profile_path: str, hcard_path: str = '/hcard/users/', atom_path: str | None = None, search_path: str | None = None)[source]¶
RFC 7033 webfinger - see https://tools.ietf.org/html/rfc7033
A Django view is also available, see the child
django
module for view and url configuration.- Parameters:
id – Profile ActivityPub ID in URL format
handle – Profile Diaspora handle
guid – Profile Diaspora guid
base_url – The base URL of the server (protocol://domain.tld)
profile_path – Profile path for the user (for example /profile/johndoe/)
hcard_path – (Optional) hCard path, defaults to
/hcard/users/
.atom_path – (Optional) atom feed path
- Returns:
dict
- class federation.hostmeta.generators.SocialRelayWellKnown(subscribe, tags=(), scope='all', *args, **kwargs)[source]¶
A .well-known/social-relay document in JSON.
For apps wanting to announce their preferences towards relay applications.
See WIP spec: https://wiki.diasporafoundation.org/Relay_servers_for_public_posts
Schema see schemas/social-relay-well-known.json
- Parameters:
subscribe – bool
tags – tuple, optional
scope – Should be either “all” or “tags”, default is “all” if not given
Fetchers¶
High level utility functions to fetch remote objects. These should be favoured instead of protocol specific utility functions.
Inbound¶
High level utility functions to pass incoming messages to. These should be favoured instead of protocol specific utility functions.
- federation.inbound.handle_receive(request: RequestType, user: UserType | None = None, sender_key_fetcher: Callable[[str], str] | None = None, skip_author_verification: bool = False) Tuple[str, str, List] [source]¶
Takes a request and passes it to the correct protocol.
- Returns a tuple of:
sender id
protocol name
list of entities
NOTE! The returned sender is NOT necessarily the author of the entity. By sender here we’re talking about the sender of the request. If this object is being relayed by the sender, the author could actually be a different identity.
- Parameters:
request – Request object of type RequestType - note not a HTTP request even though the structure is similar
user – User that will be passed to protocol.receive (only required on private encrypted content) MUST have a private_key and id if given.
sender_key_fetcher – Function that accepts sender handle and returns public key (optional)
skip_author_verification – Don’t verify sender (test purposes, false default)
- Returns:
Tuple of sender id, protocol name and list of entity objects
Outbound¶
High level utility functions to pass outbound entities to. These should be favoured instead of protocol specific utility functions.
- federation.outbound.handle_send(entity: BaseEntity, author_user: UserType, recipients: List[Dict], parent_user: UserType | None = None, payload_logger: callable | None = None) None [source]¶
Send an entity to remote servers.
Using this we will build a list of payloads per protocol. After that, each recipient will get the generated protocol payload delivered. Delivery to the same endpoint will only be done once so it’s ok to include the same endpoint as a receiver multiple times.
Any given user arguments must have
private_key
andfid
attributes.- Parameters:
entity – Entity object to send. Can be a base entity or a protocol specific one.
author_user – User authoring the object.
recipients –
A list of recipients to delivery to. Each recipient is a dict containing at minimum the “endpoint”, “fid”, “public” and “protocol” keys.
For ActivityPub and Diaspora payloads, “endpoint” should be an URL of the endpoint to deliver to.
The “fid” can be empty for Diaspora payloads. For ActivityPub it should be the recipient federation ID should the delivery be non-private.
The “protocol” should be a protocol name that is known for this recipient.
The “public” value should be a boolean to indicate whether the payload should be flagged as a public payload.
TODO: support guessing the protocol over networks? Would need caching of results
For private deliveries to Diaspora protocol recipients, “public_key” is also required.
For example [
- {
“endpoint”: “https://domain.tld/receive/users/1234-5678-0123-4567”, “fid”: “”, “protocol”: “diaspora”, “public”: False, “public_key”: <RSAPublicKey object> | str,
}, {
”endpoint”: “https://domain2.tld/receive/public”, “fid”: “”, “protocol”: “diaspora”, “public”: True,
}, {
”endpoint”: “https://domain4.tld/sharedinbox/”, “fid”: “https://domain4.tld/profiles/jack/”, “protocol”: “activitypub”, “public”: True,
}, {
”endpoint”: “https://domain4.tld/profiles/jill/inbox”, “fid”: “https://domain4.tld/profiles/jill”, “protocol”: “activitypub”, “public”: False,
}, {
”endpoint”: “https://matrix.domain.tld”, “fid”: “#@user:domain.tld”, “protocol”: “matrix”, “public”: True,
}
]
parent_user – (Optional) User object of the parent object, if there is one. This must be given for the Diaspora protocol if a parent object exists, so that a proper
parent_author_signature
can be generated. If given, the payload will be sent as this user. For Activitypub, the parent_user’s private key will be used to generate the http signature if the author_user is not a local user.payload_logger – (Optional) Function to log the payloads with.
Django¶
Some ready provided views and URL configuration exist for Django.
Note! Django is not part of the normal requirements for this library. It must be installed separately.
- federation.hostmeta.django.generators.rfc7033_webfinger_view(request, *args, **kwargs)[source]¶
Django view to generate an RFC7033 webfinger.
- federation.hostmeta.django.generators.matrix_client_wellknown_view(request, *args, **kwargs)[source]¶
- federation.hostmeta.django.generators.matrix_server_wellknown_view(request, *args, **kwargs)[source]¶
Configuration¶
To use the Django views, ensure a modern version of Django is installed and add the views to your URL config for example as follows. The URL’s must be mounted on root if Diaspora protocol support is required.
url(r"", include("federation.hostmeta.django.urls")),
Some settings need to be set in Django settings. An example is below:
FEDERATION = {
"base_url": "https://myserver.domain.tld,
"federation_id": "https://example.com/u/john/",
"get_object_function": "myproject.utils.get_object",
"get_private_key_function": "myproject.utils.get_private_key",
"get_profile_function": "myproject.utils.get_profile",
"matrix_config_function": "myproject.utils.matrix_config_funct",
"nodeinfo2_function": "myproject.utils.get_nodeinfo2_data",
"process_payload_function": "myproject.utils.process_payload",
"search_path": "/search/?q=",
"tags_path": "/tags/:tag:",
}
base_url
is the base URL of the server, ie protocol://domain.tld.federation_id
is a valid ActivityPub local profile id whose private key will be used to create the HTTP signature for GET requests to ActivityPub platforms.get_object_function
should be the full path to a function that will return the object matching the ActivityPub ID for the request object passed to this function.get_private_key_function
should be the full path to a function that will accept a federation ID (url, handle or guid) and return the private key of the user (as an RSA object). Required for example to sign outbound messages in some cases.get_profile_function
should be the full path to a function that should return aProfile
entity. The function should take one or more keyword arguments:fid
,handle
,guid
orrequest
. It should look up a profile with one or more of the provided parameters.matrix_config_function
(optional) function that returns a Matrix configuration dictionary, with the following objects:
{
# Location of the homeserver (not server name)
"homeserver_base_url": "https://matrix.domain.tld",
# Homeserver domain and port (not server domain)
"homeserver_domain_with_port": "matrix.domain.tld:443",
# Homeserver name
"homeserver_name": "domain.tld",
# Appservice details
"appservice": {
# Unique ID to register with at the homeserver. Don't change this after creating.
"id": "uniqueid",
# Short code (a-z only), used for various things like namespacing
"shortcode": "federatedapp",
# Secret token for communication
"token": "secret_token",
},
# (Optional) location of identity server
"identity_server_base_url": "https://id.domain.tld",
# (Optional) other keys to include in the client well-known (must be a dictionary)
"client_wellknown_other_keys": {
"org.foo.key" "barfoo",
},
# (Optional) registration shared secret
"registration_shared_secret": "supersecretstring",
}
nodeinfo2_function
(optional) function that returns data for generating a NodeInfo2 document. Once configured the path/.well-known/x-nodeinfo2
will automatically generate a NodeInfo2 document. The function should return adict
corresponding to the NodeInfo2 schema, with the following minimum items:
{server:
baseUrl
name
software
version
}
openRegistrations
process_payload_function
(optional) function that takes in a request object. It should returnTrue
if successful (or placed in queue for processing later) orFalse
in case of any errors.search_path
(optional) site search path which ends in a parameter for search input, for example “/search?q=”tags_path
(optional) path format to view items for a particular tag.:tag:
will be replaced with the tag (without#
).
Protocols¶
The code for opening and creating protocol messages lives under each protocol module in federation.protocols
.
Each protocol defines a protocol.Protocol
class under it’s own module. This is expected to contain certain methods that are used by the higher level functions that are called on incoming messages and when sending outbound messages. Everything that is needed to transform an entity into a message payload and vice versa should be here.
Instead of calling methods directly for a specific protocol, higher level generic functions should be normally used.
Utils¶
Various utils are provided for internal and external usage.
ActivityPub¶
Diaspora¶
- federation.utils.diaspora.fetch_public_key(handle)[source]¶
Fetch public key over the network.
- Parameters:
handle – Remote handle to retrieve public key for.
- Returns:
Public key in str format from parsed profile.
- federation.utils.diaspora.get_fetch_content_endpoint(domain, entity_type, guid)[source]¶
Get remote fetch content endpoint.
See: https://diaspora.github.io/diaspora_federation/federation/fetching.html
- federation.utils.diaspora.get_private_endpoint(id: str, guid: str) str [source]¶
Get remote endpoint for delivering private payloads.
- federation.utils.diaspora.get_public_endpoint(id: str) str [source]¶
Get remote endpoint for delivering public payloads.
- federation.utils.diaspora.parse_profile_from_hcard(hcard: str, handle: str)[source]¶
Parse all the fields we can from a hCard document to get a Profile.
- Parameters:
hcard – HTML hcard document (str)
handle – User handle in username@domain.tld format
- Returns:
federation.entities.diaspora.entities.DiasporaProfile
instance
- federation.utils.diaspora.retrieve_and_parse_content(id: str, guid: str, handle: str, entity_type: str, cache: bool = True, sender_key_fetcher: Callable[[str], str] | None = None)[source]¶
Retrieve remote content and return an Entity class instance.
This is basically the inverse of receiving an entity. Instead, we fetch it, then call “handle_receive”.
- Parameters:
sender_key_fetcher – Function to use to fetch sender public key. If not given, network will be used to fetch the profile and the key. Function must take handle as only parameter and return a public key.
- Returns:
Entity object instance or
None
- federation.utils.diaspora.retrieve_and_parse_diaspora_webfinger(handle)[source]¶
Retrieve a and parse a remote Diaspora webfinger document.
- Parameters:
handle – Remote handle to retrieve
- Returns:
dict
- federation.utils.diaspora.retrieve_and_parse_profile(handle)[source]¶
Retrieve the remote user and return a Profile object.
- Parameters:
handle – User handle in username@domain.tld format
- Returns:
federation.entities.Profile
instance or None
Matrix¶
- federation.utils.matrix.register_dendrite_user(username: str) Dict [source]¶
Shared secret registration for Dendrite.
Note uses the legacy route, see https://github.com/matrix-org/dendrite/issues/1669
Currently compatible with Django apps only.
- Returns:
- {
‘user_id’: ‘@username:domain.tld’, ‘access_token’: ‘randomaccesstoken’, ‘home_server’: ‘domain.tld’, ‘device_id’: ‘randomdevice’
}
Network¶
- federation.utils.network.fetch_document(url=None, host=None, path='/', timeout=10, raise_ssl_errors=True, extra_headers=None, cache=True, **kwargs)[source]¶
Helper method to fetch remote document.
Must be given either the
url
orhost
. Ifurl
is given, only that will be tried without falling back to http from https. Ifhost
given, path will be added to it. Will fall back to http on non-success status code.- Parameters:
url – Full url to fetch, including protocol
host – Domain part only without path or protocol
path – Path without domain (defaults to “/”)
timeout – Seconds to wait for response (defaults to 10)
raise_ssl_errors – Pass False if you want to try HTTP even for sites with SSL errors (default True)
extra_headers – Optional extra headers dictionary to add to requests
:arg kwargs holds extra args passed to requests.get :returns: Tuple of document (str or None), status code (int or None) and error (an exception class instance or None) :raises ValueError: If neither url nor host are given as parameters
- federation.utils.network.send_document(url, data, timeout=10, method='post', *args, **kwargs)[source]¶
Helper method to send a document via POST.
Additional
*args
and**kwargs
will be passed on torequests.post
.- Parameters:
url – Full url to send to, including protocol
data – Dictionary (will be form-encoded), bytes, or file-like object to send in the body
timeout – Seconds to wait for response (defaults to 10)
method – Method to use, defaults to post
- Returns:
Tuple of status code (int or None) and error (exception class instance or None)
Protocols¶
Exceptions¶
Various custom exception classes might be returned.
- exception federation.exceptions.EncryptedMessageError[source]¶
Encrypted message could not be opened.
- exception federation.exceptions.NoSenderKeyFoundError[source]¶
Sender private key was not available to sign a payload message.