8. Kinkdoc: API Documentation system

Kinkdoc is an API documentation system for Kink programs. Like Javadoc, godoc, and Doxygen, documentation texts for kinkdoc are written in program comments.

8.1. Tutorial

Let's make a set of modules, and write kinkdoc documentation for them. In this tutorial, we will make following modules:

• math/RAT : for rational numbers.

• math/COMPLEX : for complex numbers.

Here, we make a library in the following directory structure:

• mathlib/ : the project directory

  • src/ : the root of modules

    • math/

      • RAT.kn : program file of math/RAT module

      • COMPLEX.kn : program file of math/COMPLEX module

  • build/ : the directory of generated files

Let's start from src/math/RAT.kn.

:NUM.require_from('kink/')

:new <- {(:Numer :Denom)
  :Desc = 'RAT.new(Numer Denom)'
  NUM.is?(Numer) && Numer.int? || raise(
    '{}: Numer must be an int num, but was {}'.format(Desc Numer.repr))
  NUM.is?(Denom) && Denom.int? && Denom != 0 || raise(
    '{}: Denom must be a non-zero int num, but was {}'.format(Desc Denom.repr))
  new_val(
    .. Rat_trait
    'Numer' Numer
    'Denom' Denom
  )
}

:Rat_trait <- [
  'numer' {[:R] R.Numer }
  'denom' {[:R] R.Denom }
  'repr' {[:R] '(rat {} {})'.format(R.numer R.denom) }
]

8.1.1. Functions

First, add a description of new function. The first line of a kinkdoc comment chunk starts with ##, and subsequent lines start with #.

## RAT.new(Numer Denom)
#
# `new` makes a `rat` value
# which represents a rational number.
#
# `Numer` is the numerator, and `Denom` is the denominator.
:new <- {(:Numer :Denom)
  ,,,
}

The text of the first line after ## and a whitespace is handled as the title of the section. Following lines are handled as the body of the section. The body text is separated to paragraph blocks by an empty comment line.

The kinkdoc comment chunk above can be translated to HTML like this:

<h3>RAT.new(Numer Denom)</h3>

<p>`new` makes a `rat` value which represents a rational number.</p>

<p>`Numer` is the numerator, and `Denom` is the denominator.</p>

Note that use of backticks ` for quotation is just a convention. It has no special meaning in the kinkdoc system.

You can add an example code to the section. A code block is indented by two whitespaces.

## RAT.new(Numer Denom)
#
# `new` makes a `rat` value
# which represents a rational number.
#
# `Numer` is the numerator, and `Denom` is the denominator.
#
#   :RAT.require_from('org/example/')
#
#   :Rat <- RAT.new(1 2)
#   stdout.print_line(Rat.repr)  # => (rat 1 2)
:new <- {(:Numer :Denom)
  ,,,
}

The kinkdoc comment chunk above can be translated to HTML like this:

<h3>RAT.new(Numer Denom)</h3>

<p>`new` makes a `rat` value which represents a rational number.</p>

<p>`Numer` is the numerator, and `Denom` is the denominator.</p>

<pre>
:RAT.require_from('org/example/')

:Rat &lt;- RAT.new(1 2)
stdout.print_line(Rat.repr)  # =&gt; (rat 1 2)
</pre>

You can add a heading inside the section body. If the block starts with == and a whitespace, and ends with a whitespace and ==, it is regarded as a heading.

## RAT.new(Numer Denom)
#
# `new` makes a `rat` value
# which represents a rational number.
#
# `Numer` is the numerator, and `Denom` is the denominator.
#
# == Usage ==
#
#   :RAT.require_from('org/example/')
#
#   :Rat <- RAT.new(1 2)
#   stdout.print_line(Rat.repr)  # => (rat 1 2)
:new <- {(:Numer :Denom)
  ,,,
}

The kinkdoc comment chunk above can be translated to HTML like this:

<h3>RAT.new(Numer Denom)</h3>

<p>`new` makes a `rat` value which represents a rational number.</p>

<p>`Numer` is the numerator, and `Denom` is the denominator.</p>

<p><em class="heading">Usage</em></p>

<pre>
:RAT.require_from('org/example/')

:Rat &lt;- RAT.new(1 2)
stdout.print_line(Rat.repr)  # =&gt; (rat 1 2)
</pre>

Note that the heading is not translated to h3, h4, or h5 element, but a p element. It is because a heading does not affect the tree structure of the document.

8.1.2. Types and methods

Second, document rat type.

Kinkdoc system parses only comment lines. So you can add a kinkdoc comment anywhere in the program file. In this case, the best location to document rat type is before the assignment of Rat_trait variable.

## type rat
#
# `rat` is a type of rational numbers.
:Rat_trait <- [
  'numer' {[:R] R.Numer }
  'denom' {[:R] R.Denom }
  'repr' {[:R] '(rat {} {})'.format(R.numer R.denom) }
]

Then, add kinkdoc comments of the methods.

## type rat
#
# `rat` is a type of rational numbers.
:Rat_trait <- [

  ## R.numer
  #
  # `numer` returns the numerator of the rational number `R`.
  'numer' {[:R] R.Numer }

  ## R.denom
  #
  # `denom` returns the denominator of the rational number `R`.
  'denom' {[:R] R.Denom }

  'repr' {[:R] '(rat {} {})'.format(R.numer R.denom) }
]

Here, comment chunks of R.numer and R.denom are indented from the comment chunk of type rat. Thus, the sections of R.numer and R.denom are regarded as children of the section of type rat. So the above fragment can be translated to HTML as follows.

<h3>type rat</h3>

<p>`rat` is a type of rational numbers.</p>

<h4>R.numer</h4>

<p>`numer` returns the numerator of the rational number `R`.</p>

<h4>R.denom</h4>

<p>`denom` returns the denominator of the rational number `R`.</p>

8.1.3. Modules

To document the module, add a kinkdoc comment chunk at the beginning of the program file, and leave the title empty.

##
# This module provides calculation of rational numbers.

:NUM.require_from('kink/')

## RAT.new(Numer Denom)
#
# `new` makes a `rat` value
# which represents a rational number.
#
# `Numer` is the numerator, and `Denom` is the denominator.
#
# == Usage ==
#
#   :RAT.require_from('org/example/')
#
#   :Rat <- RAT.new(1 2)
#   stdout.print_line(Rat.repr)  # => (rat 1 2)
:new <- {(:Numer :Denom)
  :Desc = 'RAT.new(Numer Denom)'
  NUM.is?(Numer) && Numer.int? || raise(
    '{}: Numer must be an int num, but was {}'.format(Desc Numer.repr))
  NUM.is?(Denom) && Denom.int? && Denom != 0 || raise(
    '{}: Denom must be a non-zero int num, but was {}'.format(Desc Denom.repr))
  new_val(
    .. Rat_trait
    'Numer' Numer
    'Denom' Denom
  )
}

## type rat
#
# `rat` is a type of rational numbers.
:Rat_trait <- [

  ## R.numer
  #
  # `numer` returns the numerator of the rational number `R`.
  'numer' {[:R] R.Numer }

  ## R.denom
  #
  # `denom` returns the denominator of the rational number `R`.
  'denom' {[:R] R.Denom }

  'repr' {[:R] '(rat {} {})'.format(R.numer R.denom) }
]

8.1.4. Kinkdoc toolchain

Document generation is done in two steps:

• Parsing: DOC_PARSE_TOOL module generates a JSON file from kinkdoc comments of program files.

• Rendering: A renderer generates an end result from the JSON file. For example, HTML_RENDER_TOOL module generates an HTML file, and SPHINX_RENDER_TOOL module generates Sphinx source files.

Dataflow of the toolchain:

If you want to generate an HTML file, you can run a command chain like this:

$ kink mod:kink/doc/DOC_PARSE_TOOL src | kink mod:kink/doc/render/html/HTML_RENDER_TOOL > doc.html

DOC_PARSE_TOOL module parses program files of public modules under the specified directory, src, and writes the JSON data to the standard output. HTML_RENDER_TOOL module reads the JSON data from the standard input, then writes an HTML document to the standard output like:

<!DOCTYPE html>
<html>
<head>
  <title>API documentation</title>
</head>
<body>

<h1>API documentation</h1>

<h2>math/COMPLEX</h2>

<p>This module provides calculation of complex numbers.</p>

,,,

<h2>math/RAT</h2>

<p>This module provides calculation of rational numbers.</p>

<h3>RAT.new(Numer Denom)</h3>

,,,

</body>
</html>

8.1.5. Document title and the overview

You can give the title of the entire document by --title option of DOC_PARSE_TOOL module. If --title is not specified, “API documentation” is used as the default title.

You can also give the overview text of the document. First, make a program file containing kinkdoc comment chunks, then specify the file to --overview option of DOC_PARSE_TOOL module. The toplevel section of the program file is used as the overview text. The second and lower level sections are used as as module-level and lower level sections.

Let's make src/overview.kn as follows:

##
# This library provides various systems of numbers.

Then run the command chain:

$ kink mod:kink/doc/DOC_PARSE_TOOL src \
    --title 'Math library' \
    --overview src/overview.kn \
    | kink mod:kink/doc/render/html/HTML_RENDER_TOOL \
    > doc.html

doc.html looks like:

<!DOCTYPE html>
<html>
<head>
  <title>Math library</title>
</head>
<body>

<h1>Math library</h1>

<p>This library provides various systems of numbers.</p>

<h2>math/COMPLEX</h2>

,,,

<h2>math/RAT</h2>

,,,

</body>
</html>

8.1.6. Vim folding support

If the title line contains {{{, which is the default start marker of folding in vim, the marker and the text after that are ignored.

## type rat {{{
#
# `rat` is a type of rational numbers.
:Rat_trait <- [

  ## R.numer {{{
  #
  # `numer` returns the numerator of the rational number `R`.
  'numer' {[:R]
    R.Numer
  } # }}}

  ## R.denom {{{
  #
  # `denom` returns the denominator of the rational number `R`.
  'denom' {[:R]
    R.Denom
  } # }}}

  'repr' {[:R] '(rat {} {})'.format(R.numer R.denom) }

] # }}}

# vim: fdm=marker

8.2. Data model

A kinkdoc document is parsed from program files as nested sections. A section consists of three properties:

• title: a string of the section tile

• blocks: an array of the blocks in the section

• subsections: an array of child sections

The outermost section corresponds to the document itself. Second level sections usually correspond to modules. Third and deeper sections usually correspond to types, functions, and methods.

8.2.1. JSON Schema

A kinkdoc document is encoded in JSON following the next JSON schema.

{ "type": "object",
  "required": ["title", "blocks", "subsections"],
  "properties": {
    "title": {
      "type": "string",
      "pattern": "[^\\u0000-\\u0020\\u007f]( *[^\\u0000-\\u0020\\u007f])*"
    },
    "blocks": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["type", "text"],
        "properties": {
          "type": {
            "enum": ["paragraph", "code", "heading"]
          },
        },

        "if": { "properties": { "type": { "const": "code" } } },
        "then": {
          "properties": {
            "text": {
              "type": "string",
              "pattern": "( *[^\\u0000-\\u0020\\u007f]+( +[^\\u0000-\\u0020\\u007f]+)*\\n)(\\n?( *[^\\u0000-\\u0020\\u007f]+( +[^\\u0000-\\u0020\\u007f]+)*\\n))*"
            }
          }
        },
        "else": {
          "properties": {
            "text": {
              "type": "string",
              "pattern": "[^\\u0000-\\u0020\\u007f]( *[^\\u0000-\\u0020\\u007f])*"
            }
          }
        }
      }
    },
    "subsections": {
      "type": "array",
      "items": { "$ref": "#" }
    }
  }
}

8.3. Kinkdoc parsing from program files

This section defines the process for parsing a kinkdoc document from program files.

Terms used in this section:

Program file

A program file is a file which contains a valid Kink program encoded in UTF-8.

Source program

A source program is a Unicode string read from a program file and decoded by UTF-8, which is a valid Kink program.

Whitespace character

The whitespace character is the codepoint U+0020.

Line break sequence

A line break sequence is CR+LF (U+000d, U+000a) or LF (U+000a).

Line

A line is a range in a source program which meets one of the following conditions:

• If the source program includes one or more line break sequences:

  • From the start of the source program, to the position just before the first line break sequence.

  • From the position just after a line break sequence which is not the last one, to the position just before the next line break sequence.

  • From the position just after the last line break sequence, to the end of the source program.

• If the source program includes no line break sequence:

  • From the start of the source program, to the end of the source program.

Number sign

The number sign is the codepoint # (U+0023).

Comment delimiter

The comment delimiter of a comment is the longest sequence of number signs inclusively from the number sign which starts the comment.

Comment only line

A comment only line is a line which meets the following conditions:

• The line contains a comment.

• All the codepoints in the line before the comment delimiter are whitespace characters.

Comment text

A comment text is extracted from a comment only line as follows:

• Extract codepoints from the position just after the comment delimiter, to the end of the line.

• Replace every ASCII control character (U+0000-U+001f, U+007f) to a whitespace character.

• Drop the longest sequence of whitespace characters at the end.

Empty comment line

An empty comment line is a comment only line whose comment text is empty.

Comment chunk

A comment chunk is a sequence of comment only lines, which is not preceded by a comment only line, and not followed by a comment only line.

8.3.1. Kinkdoc comment chunk

If a comment chunk meets the following conditions, it is called a kinkdoc comment chunk.

• The number of whitespace characters at the beginning of each line is the same for all the lines.

• The comment delimiter of the first line is exactly two number signs ##.

• The comment delimiter of the second and onwards is exactly one number sign #.

• If the comment text of a line is not empty, the first codepoint of the comment text is a whitespace character.

From a kinkdoc comment chunk, a section is generated.

8.3.2. Title

The title of the section is extracted from the comment text of the first line of the kinkdoc comment chunk as follows.

1. Match the comment text of the first line by regex [ ]*(?<Title>.*?)[ ]*(\{\{\{.*)?.

2. Use the group Title as the title.

8.3.3. Blocks

The second and following lines of the kinkdoc comment chunk are split to blocks by empty comment lines.

There are three types of blocks:

• Code block

• Heading block

• Paragraph block

8.3.3.1. Code block

If the comment text of every line of a block starts with three or more whitespace characters, the block is a code block.

The text of the code block is generated from comment texts as follows.

1. Remove three whitespace characters from the beginning of the comment text of each line.

2. Add a line feed character (U+000a) to the end of the comment text of each line.

3. Concatenate comment texts in order.

If a code block is directly followed by another code block, those code blocks are combined into a single code block. The text of the code block is made by joining the texts of the combined code blocks with a line feed character (U+000a).

8.3.3.2. Heading block

If a block meets following conditions, it is a heading block.

• The block is not a code block.

• The block consists of a a single line.

• The comment text of the line matches the regex [ ]+==[ ]+(?<Heading>[^ ].*?)[ ]+==.

The group Heading is used as the text of the heading block.

8.3.3.3. Paragraph block

If a block is neither a code block nor a heading block, it is a paragraph block.

The text of the paragraph block is generated as follows.

1. Remove whitespace characters from the beginning of the comment text of each line.

2. Concatenate the comment texts of the lines in order, inserting a whitespace character between lines.

8.3.4. Structure of program-level sections

DOC_PARSE_TOOL module generates a program-level section from kinkdoc comment chunks of a program file.

8.3.4.1. The program-level section

If the source program has one or more kinkdoc comment chunks, and the first kinkdoc comment chunk of the source program has an empty title, the blocks of the kinkdoc comment chunk are used as the blocks of the program-level section.

If the source program has no kinkdoc comment chunk, or the title of the first kinkdoc comment chunk is not empty, the program-level section is created with no blocks.

If the program file is one specified by --overview option, the title is specified by --title option, or a default title is used.

If the program file is a source of a module, the module name is used as the title.

8.3.4.2. Nested subsections

Kinkdoc comment chunks which are not of the program-level section are parsed as nested subsections, as shown in the following pseudo code.

program_level_section := «the program level section»
chunks := «queue of kinkdoc comment chunks not of the program-level section»

read_children(program_level_section, 0)

def read_children(parent, min_whitespaces) {
  loop {
    if empty?(chunks)
      return

    chunk := peek(chunks)
    if chunk.leading_whitespaces < min_whitespaces
      return

    dequeue(chunks)
    section := new_section(
      title: if empty?(chunk.title) then «replacement title» else chunk.title,
      blocks: chunk.blocks,
      subsections: []
    )
    parent.subsections := parent.subsections + [section]
    read_children(section, chunk.leading_whitespaces + 1)
  }
}

8.3.5. Structure of document-level sections

DOC_PARSE_TOOL module generates a document-level section for each execution, as shown in the following pseudo code.

doc_title := if option_given?('--title') then option('--title') else 'API documentation'
doc_section := ( if option_given?('--overview')
  then parse_page_level_section(title: doc_title, file: option('--overview'))
  else new_section(title: doc_title, blocks: [], subsections: [])
)

mod_pages := «tuples of (mod_name, file_name)»
mod_pages := sort(mod_pages by mod_name alphabetically)
for page in mod_pages {
  page_section := parse_page_level_section(title: page.mod_name, file: page.file_name)
  doc_section.subsections := doc_section.subsections + [page_section]
}