Apache Kafka Documentation Website

This repository contains the source for the Apache Kafka documentation website. The site is built using Hugo with the Docsy theme, providing a modern, maintainable, and feature-rich documentation experience.

Structure of the Website

Documentation Versioning

The documentation is organized by Kafka versions in the content/en directory:

content/en/
├── _index.md                 # Landing page
├── 40/                       # Latest version (4.0)
│   ├── apis/
│   ├── configuration/
│   ├── design/
│   └── ...
├── 39/                       # Previous version (3.9)
├── 38/                       # Version 3.8
└── ...

Each version directory contains the complete documentation for that specific Kafka release. The latest version (currently 4.0) is the default documentation shown to users.

Important: The version-specific documentation (under directories like 40/, 39/, etc.) is sourced from the corresponding release branches in the apache/kafka repository. The docs directory in each branch serves as the source of truth. During the website build process, this content is copied to the appropriate version directory. For more details, see KIP-1133.

Key Directories

• assets/: Contains customizations and overrides

  • scss/: Custom styling and theme variables
  • icons/: Custom icons and branding
  • json/: Search index configuration

• layouts/: Custom Hugo templates and overrides

  • _default/: Base templates
  • partials/: Reusable template components
  • shortcodes/: Custom Hugo shortcodes

• data/: JSON data files for dynamic content

  • testimonials.json: Powers the testimonials page
  • committers.json: Powers the committers page

Features and Customizations

1. Offline Search: Enabled via offlineSearch: true in hugo.yaml, providing fast client-side search functionality
2. Version Selector: Allows users to switch between different Kafka versions
3. Custom Shortcodes: Located in layouts/shortcodes/ for enhanced content formatting
4. Custom Styling: SCSS customizations in assets/scss/

Updating the Documentation Website

Adding documentation for a new release

When releasing a new documentation version (e.g., version 4.2 / “42”), follow these steps:

1. Create Content Directory

Create a new directory in content/en/ for the new version (e.g., 42/ for version 4.2)

2. Update Version Parameters in hugo.yaml

Locate the Version Configuration block at the top of the params section (around line 245). Update the following fields:

1. latest_version: Set to the new version string (e.g., “42”).
2. latest_version_number: Set to the new version number (e.g., “4.2”).
3. version: Update to the new version (e.g., 4.2).
4. url_latest_version: Update the link (e.g., /42/).
5. versions:
   • Add the new version to the top of the list.
   • Mark the previous version as archived_version: true.

  # Latest documentation version - UPDATE THIS WHEN RELEASING NEW VERSION
  latest_version: "42"
  latest_version_number: "4.2"
  
  # ...
  
  version: 4.2

  # ...

  url_latest_version: /42/

  # ...

  versions:
    - version: "4.2"
      url: /42/
    - version: "4.1"
      url: /41/
      archived_version: true

What Updates Automatically

Once you update the parameters above, the following are automatically updated:

Menu Items:

• DOCS → /42/
• Getting Started → /42/getting-started/
• APIs, Configuration, Design, Implementation, Operations, Security, Kafka Connect, Kafka Streams → all automatically updated

Search Index:

• The search index automatically includes the latest version directory (no manual update needed)
• Configured via offline search template assets/json/offline-search-index.json

Managing Testimonials and Committers

Adding a New Testimonial

1. Add the company's logo to static/images/powered-by/
2. Add an entry to data/testimonials.json:

   {
     "link": "https://company-website.com/",
     "logo": "company-logo.png",
     "logoBgColor": "#FFFFFF",
     "description": "Description of how the company uses Apache Kafka."
   }
   

Adding a New Committer

1. Add the committer's photo to static/images/
2. Add an entry to data/committers.json:

   {
     "image": "/images/committer-photo.jpg",
     "name": "Committer Name",
     "title": "Committer, and PMC member",
     "linkedIn": "https://www.linkedin.com/in/committer/",
     "twitter": "https://twitter.com/committer",
     "github": "https://github.com/committer"
   }
   

The website uses Hugo's data templates to automatically generate the testimonials and committers pages from these JSON files. The templates are located in:

• layouts/testimonials/: Templates for rendering testimonials
• layouts/community/: Templates for rendering committer information

Updating Content

1. For version-specific documentation:

   • Make changes in the appropriate version directory
   • Test changes locally before committing

2. For common content (e.g., landing page, community docs):

   • Edit files directly in content/en/

Front Matter Guide

Detailed information about the Front Matter fields used in this site:

• title: The title of the page or post.
• linkTitle: (Optional) Short title used in sidebars and menus.
• date: Publication date (YYYY-MM-DD).
• author: Author name (often with GitHub handle). Typically used for blogs.
• weight: (Optional) Controls ordering in lists/menus (lower numbers appear first).
• description: (Optional) Brief summary for SEO and previews.
• type: Used to specify the layout type (e.g., type: docs for documentation pages).
• aliases: (Optional) List of old URLs that should redirect to this page.
• url: (Optional) Overrides the default URL path constructed from the filename.

For more details, see the Hugo Front Matter Documentation.

Adding a New Blog Post

Blog posts are located in content/en/blog/. The most common use case is adding a release announcement.

Adding a Release Blog Post

1. Create a new markdown file in content/en/blog/releases/ using kebab-case for the filename (e.g., ak-4.3.0.md).
2. Add the required Front Matter. See the Front Matter Guide for details on fields.

Example Front Matter for a Release Post:

---
date: 2025-01-01
title: "Apache Kafka 4.3.0 Release Announcement"
linkTitle: "AK 4.3.0"
author: "Author Name (@github_handle)"
---

3. Write your content below the Front Matter.

Build and Test

Prerequisites

• Docker (20.10.0 or newer)
• Make

Local Development

1. Start the development server with hot-reload:

   make serve
   

   This will:

   • Build the Hugo Docker image
   • Start a development server on http://localhost:1313
   • Watch for changes and automatically rebuild
   • Enable drafts and future posts

2. Build the site without starting the server:

   make build
   

   The built site will be available in the output directory.

Production Build

1. Build and test the production image locally:

   make prod-run
   

   The site will be available at http://localhost:8080

2. Build production image for deployment:

   make prod-image
   

   This creates a multi-architecture Nginx image (ARM64 and AMD64) optimized for production.

Cleaning Up

Remove built files and Docker images:

make clean

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test locally using make serve
5. Submit a pull request

For more details about the migration to Markdown and the overall architecture, see KIP-1133.