Pelican Builds

This website is built using Pelican. Configure the build using the pelicanconf.py settings.

Pelican Theme

# Theme
THEME = './theme/apache'

See theme template for details about this site's theme.

Plugins

The Pelican environment is enhanced with plugins. Our environment has its own copy of the asf plugins, while the pelican-build.py script provides pelican-gfm.

# Pelican Plugins
# pelican-gfm is installed in the buildbot as part of build_pelican.py. It is an ASF Infra custom plugin.
# other plugins are discoverable and can be installed via pip by mentioning them in requirements.txt
# You can find plugins here: https://github.com/pelican-plugins
# Plugins that are custom for this site are found in PLUGIN_PATHS.
PLUGIN_PATHS = ['./theme/plugins']
PLUGINS = ['asfgenid', 'asfdata', 'pelican-gfm', 'asfreader']

1. Data Model. The asfdata.py plugin builds a metadata model that is shared with every page.
2. GFM Content. The pelican-gfm plugin reads .md, .markdown, .mkd, and .mdown files and converts the GFM Markdown into HTML.
3. EZMD Content. The asfreader.py plugin reads .ezmd files, injects data, translates ezt, and converts the GFM Markdown into HTML.
4. Generate ID. The asfgenid.py plugin performs a number of enhancements to the HTML.

See process for the steps signaled. See plugins for the Python code.

Tree Structure

Pages and static content are stored in the same tree. Generated content is output with the same relative path, except with an html extension. These are the necessary settings.

PATH = 'content'
# Save pages using full directory preservation
PAGE_PATHS = ['.']
# Path with no extension
PATH_METADATA = '(?P<path_no_ext>.*)\..*'
# We are not slugifying any pages
ARTICLE_URL = ARTICLE_SAVE_AS = PAGE_URL = PAGE_SAVE_AS = '{path_no_ext}.html'
# We want to serve our static files mixed with content
STATIC_PATHS = ['.']
# we want any html to be served as-is
READERS = {'html': None}
# ignore README.md files in the content tree and the interviews and include folders
IGNORE_FILES = ['README.md','interviews','include']

Process

Pelican uses signals as it goes through the process of reading and generating content. Our plugins:

Read Source

From pelican-gfm

    def read_source(self, source_path):
        "Read metadata and content from the source."
	...
	# Fetch the source content, with a few appropriate tweaks
        with pelican.utils.pelican_open(source_path) as text:

            # Extract the metadata from the header of the text
            lines = text.splitlines()
            for i in range(len(lines)):
                line = lines[i]
                match = GFMReader.RE_METADATA.match(line)
                if match:
                    name = match.group(1).strip().lower()
		    ...
                    metadata[name] = value
                elif not line.strip():
                    # blank line
                    continue
                else:
                    # reached actual content
                    break
	    ...
            # Reassemble content, minus the metadata
            text = '\n'.join(lines[i:])

            return text, metadata

Example:

Title: ASF Export Classifications and Source Links
license: https://www.apache.org/licenses/LICENSE-2.0
asf_headings: False

#### ASF Project
...

The first three lines specify three metadata key-value pairs. There is a blank line and the rest is the text.

Model Metadata

From asfreader.py

    def add_data(self, text, metadata):
        "Mix in ASF data as metadata"

        asf_metadata = self.settings.get('ASF_DATA', { }).get('metadata')
        if asf_metadata:
            metadata.update(asf_metadata)
            # insert any direct references
            m = 1
            while m:
                m = METADATA_RE.search(text)
                if m:
                    this_data = m.group(1).strip()
                    format_string = '{{{0}}}'.format(this_data)
                    try:
                        new_string = format_string.format(**metadata)
                        print(f'{{{{{m.group(1)}}}}} -> {new_string}')
                    except Exception:
                        # the data expression was not found
                        new_string = format_string
                        print(f'{{{{{m.group(1)}}}}} is not found')
                    text = re.sub(METADATA_RE, new_string, text, count=1)
        return text, metadata

Metadata Examples

We extend EZT syntax to do metadata substitution prior to EZT translation. This allows for a more natural and direct representation than with EZT sequences.

|  |  |  |
|-----------|-----------|-------------|
| [{ board[0].name }] | [{ board[1].name }] | [{ board[2].name }] |
| [{ board[3].name }] | [{ board[4].name }] | [{ board[5].name }] |
| [{ board[6].name }] | [{ board[7].name }] | [{ board[8].name }] |

| Office    | Individual  |
|-----------|-------------|
| Board Chair |  [{ ci[boardchair][roster] }] |
| Vice Chair |  [{ ci[vicechair][roster] }] |
| President |  [{ ci[president][roster] }] |
| Exec. V.P |  [{ ci[execvp][roster] }] |
| [[]Treasurer](https://treasurer.apache.org/) |  [{ ci[treasurer][roster] }] |
| Assistant Treasurer |  [{ ci[assistanttreasurer][roster] }] |
| Secretary |  [{ ci[secretary][roster] }] |
| Assistant Secretary |  [{ ci[assistantsecretary][roster] }] |
| V.P., [[]Legal Affairs](/legal/) |  [{ ci[legal][chair] }] |
| Assistant V.P., [[]Legal Affairs](/legal/) |  [{ ci[assistantvplegalaffairs][roster] }] |

EZT Translation

*ezmd Pages files are ezt templates that create Markdown and HTML output. See EZT Syntax for a full description of the directives.

EZT Examples

EZT

| Office    | Individual  |
|-----------|-------------|[for projects]
| V.P., [if-any projects.site][[][end]Apache [projects.display_name][if-any projects.site]]([projects.site])[end] | [projects.chair] |[end]

[for featured_projs]<li [if-index featured_projs first]class="active"[end]>
     <a href="#[featured_projs.key_id]" data-toggle="tab">[featured_projs.display_name]</a>
</li>[end]

Content

Content is in GitHub Flavored Markdown (GFM) with ASF specific enhancements for Apache CMS-style annotations.

Generate ID

We use the asfgenid plugin to perform modifications on the generated content that mimics the markdown extensions in the Apache CMS. Many of these ASF-specific enhancements are controlled in pelican settings in the ASF_GENID dictionary.

Data

Data is placed into GFM content with a combination of EZT and ASF Python-style directives.

Data Model

The data model file specifies three types of data:

1. Constants. These metadata are made available to ezmd, asfgenid, and pelican templates.

- [{ code_lines }]+ lines of code in stewardship

2. Sequences. EZT directives and the ASF Python-style directives use these metadata.

Include and Insert

Templates

Pelican uses HTML templates. Templates have Pelican metadata and Pelican settings.

Theme

Templates for details.

1. Data Model How the global data model works and how to enhance the data availble for your content.

2. EZT How EZ Templates are used to convert global data into content.

3. Markdown How your content is converted to HTML.

4. Processing How the process works to create the HTML.

5. Preview Branching How to work on complex changes like updated templates, new data sources, and alternative content.

6. Local Builds How to set up your local environment to work on a branch.

7. Error Analysis How to find errors in your build.