podlings.xml is still masterHere is an analysis of the many ways podling status is tracked by the Incubator. Currently there are five components to podling status.
What follows is a description of each of these processes. This will be followed by some ideas about how we should proceed.
The podling catalog is in this file: podling.xml
Values are stored for all Incubator podlings in a single xml file each in a podling node. Not all of the data pertain to current podlings.
A current podling:
<podling name="Crail" status="current" resource="crail" sponsor="Incubator" startdate="2017-11-01">
<description>Crail is a storage platform for sharing performance critical data in distributed data processing jobs at very high speed.</description>
<reporting group="3"/>
<champion availid="lresende">Luciano Resende</champion>
<mentors>
<mentor username="jhyde">Julian Hyde</mentor>
<mentor username="lresende">Luciano Resende</mentor>
</mentors>
</podling>
A brand new podling:
<podling name="DataSketches" status="current" resource="datasketches" sponsor="Incubator" startdate="2019-03-30">
<description>DataSketches is an open source, high-performance library of stochastic streaming algorithms commonly called "sketches" in the data sciences. Sketches are small, stateful programs that process massive data as a stream and can provide approximate answers, with mathematical guarantees, to computationally difficult queries orders-of-magnitude faster than traditional, exact methods.</description>
<reporting group="2" monthly="true">May, June, July</reporting>
<champion availid="jbonofre">Jean-Baptiste Onofré</champion>
<mentors>
<mentor username="chenliang613">Liang Chen</mentor>
<mentor username="kenn">Kenneth Knowles</mentor>
<mentor username="kamaci">Furkan Kamaci</mentor>
</mentors>
</podling>
A podling graduated to a TLP:
<podling name="CXF" status="graduated" resource="cxf" sponsor="Incubator" startdate="2006-08-15" enddate="2008-04-16">
<description>The CXF project will create a SOA services framework by merges the ObjectWeb Celtix project and the Codehaus XFire project.</description>
<resolution tlp="true"/>
<mentors>
<mentor username="jim">Jim Jagielski</mentor>
<mentor username="jstrachan">James Strachan</mentor>
</mentors>
</podling>
A podling sponsored by a PMC that graduated into the PMC:
<podling name="Derby" status="graduated" resource="derby" sponsor="DB" startdate="2004-08-15" enddate="2005-07-18">
<description>Java relational database</description>
<resolution link="DB Derby" url="http://db.apache.org/derby/"/>
<mentors>
<mentor username="coar">Ken Coar</mentor>
</mentors>
</podling>
A podling that graduated with a name change:
<podling name="OpenOffice.org" status="graduated" resource="openofficeorg" resourceAliases="ooo" sponsor="Incubator" startdate="2011-06-13" enddate="2012-10-17">
<description>OpenOffice.org is comprised of six personal productivity applications: a word processor (and its web-authoring component), spreadsheet, presentation graphics, drawing, equation editor, and database.</description>
<resolution link="OpenOffice" url="https://openoffice.apache.org/"/>
<mentors>
<mentor username="jim">Jim Jagielski</mentor>
<mentor username="rubys">Sam Ruby</mentor>
<mentor username="danese">Danese Cooper</mentor>
<mentor username="curcuru">Shane Curcuru</mentor>
<mentor username="noirin">Noirin Plunkett</mentor>
<mentor username="joes">Joe Schaefer</mentor>
<mentor username="grobmeier">Christian Grobmeier</mentor>
<mentor username="rgardler">Ross Gardler</mentor>
</mentors>
</podling>
A podling that retired:
<podling name="Provisionr" status="retired" resource="provisionr" sponsor="Incubator" startdate="2013-03-07" enddate="2013-11-22">
<description>Provisionr provides a service to manage pools of virtual machines on multiple clouds.</description>
<resolution>Failed to grow a community. Retired at request of PPMC.</resolution>
<champion availid="tomwhite">Tom White</champion>
<mentors>
<mentor username="rvs">Roman Shaposhnik</mentor>
<mentor username="tomwhite">Tom White</mentor>
<mentor username="mnour">Mohammad Nour El-Din</mentor>
</mentors>
</podling>
A podling that retired and moved development elsewhere:
<podling name="Heraldry" status="retired" resource="heraldry" sponsor="Incubator" startdate="2005-07-14" enddate="2007-06-09">
<description>Identity for the rest of us.</description>
<resolution link="OpenID.net" url="http://openid.net/">Project retired. Some activity moved to OpenID.net</resolution>
<mentors>
<mentor username="ben">Ben Laurie</mentor>
<mentor username="pquerna">Paul Querna</mentor>
<mentor username="twl">Ted Leung</mentor>
<mentor username="farra">J. Aaron Farr</mentor>
<mentor username="wrowe">William Rowe</mentor>
</mentors>
</podling>
More information is found on the project status page in xml wrapped html. For the most part this is descriptive text which has data embedded tagged with id attributes. There are also very descriptive check lists.
Here is the template for new podling status pages. You can see that it is a tedious process to copy this to add a new file for each podling.
This data is available and in some cases is extracted from these files by processes including the Clutch Analysis.
The status pages are sporadically maintained by a majority of podlings after bootstrap.
In the podling yml file the following parameters have values:
podling.resource upper cased.You can see that there is quite a bit of overlap with information from the podling status page. This is intentional since the yml files are a replacement data store.
The key values are asfCopyright and distributionRights
The initial report template is created by clutch2report.py. Look for the perproject variable. The values $name and $description are taken indirectly from podlings.xml via the clutch2.pkl created by clutch2.py. There is much more information available that can be injected into the template.
The podling maturity section has checkboxes which parallel some of the work items and checklists on the podling status page.
[ ] Initial setup [ ] Working towards first release [ ] Community building [ ] Nearing graduation [ ] Other:
Clutch analysis is performed using the clutch2.sh shell script. The key programs in the process are the clutch2.py python script which performs analysis pulling information from the following resources:
podlings.xmlThe results of the clutch analysis are published daily and appear on the Clutch Report. The shell script also runs clutch2status.py which produces clutch analysis pages. The data is also written as a clutch2.json file.
This program was recently modernized and returned to relevancy.
We have robust status information available for all the podlings, but have various gaps and incomplete transformations.
podlings.xmlWe have to decide how best to handle the data that is in the Podlings Status page and the Podlings YML file. We should narrow this down to what is critical to keep and then decide where it should go.
These items seem to be the most critical:
There are design choice here. Do we fold all of these additional pieces of information into podlings.xml, or do we normalize the Podling YML file to only include these fields. A third option is to replace the Podling YML with a data file in another format.
Eliminating the need to hand edit files without removing the ability is a goal. Let's discuss the actions that need to be recorded. Work was started in this direction in Whimsy and this can be continued.
New Podling - begins incubation. This adds the podling record to podlings.xml
Podling Roster / Status - this page exists and currently handles roster changes. It could be expanded to cover updates:
Also, the page can show the clutch analysis for the podling.
Rename Podling - this expensive operation needs to be discouraged, but when it is needed appropriate guidance should be provided.
Exit Incubation - a page that covers the several transitions a podling may make as it exits the Incubator by graduating or retiring.
The Incubator site can be simplified as follows:
/projects//projects/ page to be the place to find deeper status with links to Whimsy pages for the podling. Each podling has an anchor in the tables..htaccess file to redirect project links to the anchor.The Podling Status pages in xml will no longer be updated nor will they be shown. They can remain in svn for archival purposes.
The checklists should be moved into the Cookbook
These tasks are done to setup infrastructure for the podling and prepare the team to develop in The Apache Way.