left arrow Back to posts

GitHub syncing to a DB near you

Randy Coulman
2 min read
main image of the article

We’re excited to announce support for one of our most requested sources: GitHub!

Use your database to read from and write to your repositories, PRs, and issues. 

The tool we all use

We use GitHub all the time. It’s a source we’ve wanted to add to Sequin for a while. Like most software teams, pull requests are central to our product development workflow. And we use GitHub actions to run our CI/CD pipeline.

Now with GitHub in your Postgres database, you can augment your team’s git flow with internal tools and workflows faster than ever before.

An API you want to avoid

Like many APIs, GitHub’s API has its quirks. A few barbs you avoid when building with Sequin:

Partial objects on list

When you request a list of objects (e.g. /orgs/{org}/repos), GitHub’s API only returns partial objects.

This means you need to make a subsequent GET request to retrieve the complete objects for each object in the list. That’s 100 follow-up requests for every list request. 😵‍💫

Just webhooks

GitHub doesn’t provide an `/events` endpoint. To detect changes, you need to set up a webhook listener. If your server goes down or you miss a webhook, you need to trigger re-deliveries on the GitHub dashboard.

Apps or Personal Tokens

While it’s easy to start using GitHub’s API with Personal Access Tokens (PAT), it’s not a good solution for integrations across an organization. PATs are tied to a specific user’s permissions, so they can easily break. They also mean all actions made via the API are attributed to a single user. If you don’t want to use PATs (which, in most cases, you shouldn’t), you’ll need to set up a GitHub App, which is easier said than done. Let us do this for you!

Build tools so you can build faster

Once you have your GitHub repos syncing to your database with Sequin, you can begin to build with your GitHub data using select, insert, and delete.

To demonstrate a powerful example of this, we built an OpenAI-powered search tool on top of our GitHub data. The search tool uses embeddings to allow a user to search semantically instead of literally. So a user can type “serialize and deserialize structs into ecto jsonb” and pull up all Pull Requests related to that topic – even if the PR description doesn’t contain any exact literal matches for those words!

Check out how we built it, with instructions on how to build a tool like this yourself:

Sync GitHub now

GitHub syncs are available to spin up today. Learn more about how our GitHub syncs work in our docs.