AI Code Reviewer: I Built a Self-Hostable GitHub PR Reviewer with FastAPI, Redis, and Docker

The Problem

Manual code reviews are slow. Developers often wait hours for a teammate to check their code, only to find simple syntax errors or missing edge cases. I wanted a system that handles the repetitive "sanity checks" automatically so humans can focus on high-level architecture.

What I Built

• Automated Feedback: Every qualifying PR gets one comment with free-text review notes.
• Security Notes: The review comments on potential vulnerabilities before they merge.
• Self-Hostable: Can be deployed via Docker on your own VPS.

Architecture

• Backend: FastAPI (Python)
• Task Queue: Redis + worker
• AI: Gemini 2.5 Flash, hardcoded model name
• Database: PostgreSQL for review status and stored diff and review text
• Deployment: Docker Compose
• Auth to GitHub: a personal access token, not a GitHub App

Hard Parts

Tuning the prompt to be helpful without being annoying. AI can sometimes find "bugs" that aren't actually bugs. There is no confidence threshold, so the model's full output gets posted as-is.

How does a review get triggered?

Only pull request actions "opened" and "synchronize" trigger a review. Other actions do not. There is no language or file type filtering, so every file in the diff is reviewed.

How does the queue work?

GitHub's webhook timeout is about 3 seconds and LLM calls take 5 to 15 seconds. That gap is why Redis sits between the webhook and the worker: the webhook returns fast, the worker does the slow LLM work later. One worker handles one job at a time, so reviews run serially.

How does it handle large pull requests?

The whole diff is fetched and sent to Gemini in one prompt. There is no chunking, no truncation, no size limit, and no file filtering. Very large PRs go in one shot. So a huge PR is one big request to the model.

What does the review comment look like?

The review is posted as a single PR comment under an "AI Code Review" heading. The body is free text from Gemini: a summary, bugs, security points, and suggestions. There are no severity labels, no per-line comments, and no confidence threshold. Whatever the model returns is what gets posted.

What happens when a review fails?

There are no retries and no explicit timeouts. If the Gemini call or the GitHub post fails, the review is marked "failed" in PostgreSQL, nothing is posted, and the job is not requeued. On success it stores the diff, the review text, and status "completed". Tokens, latency, and cost are not recorded anywhere.

What does it not do yet?

Stated plainly:

• No confidence threshold. Every model finding is posted.
• No per-line comments. One comment for the whole PR.
• No severity labels.
• No retries and no explicit timeouts on failure.
• No chunking, truncation, size limit, or file filtering for large diffs.
• No language or file type filtering. Every file is reviewed.
• Only "opened" and "synchronize" trigger a review.
• One worker handles one job at a time.
• Setup uses a GitHub personal access token, not a GitHub App.
• Tokens, latency, and cost are not recorded.