main.actor-architecture-change.md

Overview

Move responsibility for building documentation hosted on main.actor from main.actor to library maintainers.

Current Architecture

At the moment, main.actor is responsible for building documentation for "hosted" packages. This raises some issues:

• Documentation builds can only handle known archetypes
• Allowing for ways to handle these can open up security concerns where we run arbitrary code in an environment that has write access to Ponylang repos.

You can see build-missing-docs.sh for a look into the code.

Proposed Change

Push responsibility for building documentation off onto library maintainers. The general outline of what I'm imagining is...

We provide code to help ease various aspect of the process as part of the library-project-starter. By default, if nothing special needs to happen with a library, using the library-project-starter would result in all steps below being completed automated without the library maintainers having to intervene.

• Library maintainer sets up CI for their project
• Includes Ponylang supplied code for building documentation
• Includes Ponylang supplied code for doing releases
• Tag is verfied to be main.actor compatible (x.y.z)
• As part of release, documentation is generated for the tag being released
• A PR is opened against main.actor-package-markdown
• CI on main.actor-package-markdown validates that the change is good and the gatsby site will build with the included code
• PR is auto-merged if it passes
• CI runs again due to merge to master. That CI task triggers netlify to build main.actor via the Netlify API.

Additionally, if library maintainers wanted to, they could skip using all the automated tools we provide and could open PRs for new documentation by hand if they wanted to.

Pros:

• No exposure of tokens that allow access to all Pony repos to arbitrary user code
• Library maintainers have complete control over the build environment
• Library maintainers don't have to provide a Makefile to install to build the docs
• Removes the current main.actor GitHub only limitation
• No need to add support for every possible VCS hosting service that library maintainers might use.
• Ponyc version compatibility is no longer an issue when building documentation.
• No longer have to worry about github API limits
• No longer have any issues with "max packages to pull down"

Cons:

• If we have to regenerate all the docs for main.actor, we have no easy way to do that as the build environment isn't under our control
• We'll have to deal programmatically with cloning the main.actor-package-markdown repo and opening a PR.
• Dealing with the fork/PR aspect programmatically could be a little wonky.
• Potentially more work/to diagnose for library maintainers

Problematic with either approach:

• As we add functionality to main.actor, we have to maintain compatibility with older generated documentation as it is generated from a tag and therefore immutable.

Other implications:

• Library maintainers would no longer need to PR manifest files to a main.actor repo to get their library included.

Alternatives

We could using something like the existing environment except, we run on a VM where we can run docker and library maintainers need to provide a docker container we can run on their source to build the documentation.

This has the pro of having far fewer security implications compared to the current architecture. However, it is also quite a bit more complicated to setup and maintain on our side.