.. _githubwebhooks:
===============
GitHub Webhooks
===============
Mozilla collects and republishes
`GitHub Webhooks `_ for a number
of Mozilla's organizations and projects.
Overall Architecture
====================
GitHub Webhooks are configured at the organizational or project level
to publish ``application/json`` payloads to
https://3abyt2fapj.execute-api.us-west-2.amazonaws.com/prod/webhook.
These HTTP requests are delivered to an Amazon API Gateway service
operated by the Developer Productivity team. Each webhook request
invokes an AWS Lambda function which does the following:
* Publishes the record to an AWS Kinesis Firehose
* Publishes the record to an *all* AWS SNS topic and optionally a a
*public* AWS SNS topic if the event is non-private.
Data published to the Kinesis Firehose is flushed to Amazon S3 for
long-term storage and to facilitate analytics.
Additional AWS Lambda functions consume the *public* SNS topic and
republish events to other channels, such as Pulse.
Private Events
--------------
While the ingestion server often receives *all* events for an organization
or repository, not all events are republished in public channels.
The following events are excluded from the public:
* Any event belonging to a *private* repository
* Team membership changes (``membership`` and ``team_add`` events)
* Transition of repository from private to public (``public`` event)
* Repository creation, deletion, or public/private transitions (``repository``
event)
* Any new events GitHub adds that aren't in a list of allowed events
Pulse Notifications
===================
`Pulse `_ is a
RabbitMQ exchange operated by Mozilla that serves as a nexus of event
publishing for various systems.
GitHub Webhook events are republished to the
`exchange/github-webhooks/v1 `_
exchange.
The routing key for each message is of the form ``/``
where ```` is the GitHub account/organization + repository
and ```` is the GitHub event name. e.g. ``mozilla/gecko-dev/push``
or ``servo/servo/issues``.
The JSON message published to pulse has the following relevant keys:
``event``
GitHub event name. e.g. ``push``, ``issues``, or ``status``.
``request_id``
UUID uniquely identifying this message. The ID is generated by GitHub.
``payload``
The payload of the GitHub event. The formats are documented at
https://developer.github.com/v3/activity/events/types/.
Delivery of GitHub events to Pulse is best effort. If Pulse is down, data
may fail to publish.
SNS Topic
=========
Non-private GitHub events are published to the
``arn:aws:sns:us-west-2:699292812394:github-webhooks-public`` AWS SNS topic.
Kinesis Firehose and S3 Access
==============================
Access to the streaming GitHub data in Kinesis Firehose and the historical
data retained in S3 can be granted on a per-case basis. If interested,
email ``developer-services@mozilla.org``.