Incorrect display of some elements on documentation pages. Broken links.

[sabina-talipova]

Description

• On some documentation pages, elements such as [notes] or [hint] are displayed as plain text instead of being properly styled.
• It is necessary to check all documentation for problems with styles.

Some examples:

• https://docs.silverstripe.org/en/5/developer_guides/templates/syntax/. Search for [note]
• https://docs.silverstripe.org/en/5/developer_guides/forms/introduction/. Search for [how_tos/handle_nested_data].
• https://docs.silverstripe.org/en/5/developer_guides/forms/field_types/htmleditorfield/ Search for [hint].

Related issue

• Documentation does not render lists inside of [info] tags correctly #280
  • specifically Documentation does not render lists inside of [info] tags correctly #280 (comment)

Updates

There is another related issue with links. The relative links to the articles on https://docs.silverstripe.org/ is not rendered correctly If they are in content of callout blocks. If link is part of plain text of paragraph everything works.
E.g. : [Environment Types](/developer_guides/debugging/environment_types) will be processed in https://docs.silverstripe.org/developer_guides/debugging/environment_types instead of https://docs.silverstripe.org/en/5/developer_guides/debugging/environment_types/.

Acceptance criteria

• Our MD parser is updated to use GitHub block quotes for all call outs.
• All pre-existing call outs are converted to the new format.
  • All changes are applied to CMS 4 branches and to the User help.
• Our doc article about updating docs is updated to use the new call out format.

Notes

• Call outs in the CMS 3 doc will definitely break. We've decided not to care.

PRs

Assign back to Guy after merging so they can double check everything is working correctly

• ENH Render the new callout block style #284
• DOC Update syntax for callout blocks developer-docs#449
• DOC Update syntax for callout blocks developer-docs#452
• DOC Update syntax for callout blocks silverstripe-userhelp-content#180
• DOC Update syntax for callout blocks silverstripe-userforms#1263
• DOC Update syntax for callout blocks developer-docs#455
• FIX Resolve final callout block issues #285

[emteknetnz]

Talked about internally, essentially we need to either update a regex which is liable to just create issues elsewhere, or switch to a different system such as:

• github blockquotes
• some other way to parse the markdown

[GuySartorelli]

Callout blocks are converted as follows:

old syntax	new syntax
info	NOTE
note	NOTE
hint	TIP
warning	WARNING
notice	WARNING
alert	CAUTION

[GuySartorelli]

I used the below script for catching automatic changes (ran it separately over developer-docs, this copy is for userhelp but the only difference is the $modules array):

Click to see code

<?php

$modules = [
    'silverstripe/userhelp-content' => [
        'branch' => '4',
        'remote' => '[email protected]:creative-commoners/silverstripe-userhelp-content.git',
    ],
    'symbiote/silverstripe-advancedworkflow' => [
        'branch' => '5',
        'remote' => '[email protected]:creative-commoners/silverstripe-advancedworkflow',
    ],
    'silverstripe/registry' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-registry',
    ],
    'silverstripe/contentreview' => [
        'branch' => '4',
        'remote' => '[email protected]:creative-commoners/silverstripe-contentreview',
    ],
    'silverstripe/blog' => [
        'branch' => '3',
        'remote' => '[email protected]:creative-commoners/silverstripe-blog',
    ],
    'silverstripe/userforms' => [
        'branch' => '5',
        'remote' => '[email protected]:creative-commoners/silverstripe-userforms',
    ],
    'silverstripe/subsites' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-subsites',
    ],
    'silverstripe/taxonomy' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-taxonomy',
    ],
    'silverstripe/iframe' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-iframe',
    ],
    'silverstripe/versionfeed' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-versionfeed',
    ],
    'dnadesign/silverstripe-elemental' => [
        'branch' => '4',
        'remote' => '[email protected]:creative-commoners/silverstripe-elemental',
    ],
    'bringyourownideas/silverstripe-maintenance' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-maintenance',
    ],
    'silverstripe/sharedraftcontent' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-sharedraftcontent',
    ],
    'silverstripe/documentconverter' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-documentconverter',
    ],
    'silverstripe/ckan-registry' => [
        'branch' => '1',
        'remote' => '[email protected]:creative-commoners/silverstripe-ckan-registry',
    ],
    'silverstripe/mfa' => [
        'branch' => '4',
        'remote' => '[email protected]:creative-commoners/silverstripe-mfa',
    ],
    'silverstripe/securityreport' => [
        'branch' => '2',
        'remote' => '[email protected]:creative-commoners/silverstripe-securityreport',
    ],
    'silverstripe/sitewidecontent-report' => [
        'branch' => '3',
        'remote' => '[email protected]:creative-commoners/silverstripe-sitewidecontent-report',
    ],
    'silverstripe/session-manager' => [
        'branch' => '1',
        'remote' => '[email protected]:creative-commoners/silverstripe-session-manager',
    ],
];

$vendorDir = __DIR__ . '/vendor/';

// Execute a command, throw a wobbly if something goes wrong, and return the result otherwise.
function cmd(string $command): string
{
    $output = [];
    exec($command, $output, $res);
    $out = implode("\n", $output);
    if ($res !== 0) {
        throw new RuntimeException("Failed to execute command: $out");
    }
    return $out;
}

// Makes a request to the github API
function github_api($url, $data = [], $httpMethod = '')
{
    // silverstripe-themes has a kind of weird redirect only for api requests
    $url = str_replace('/silverstripe-themes/silverstripe-simple', '/silverstripe/silverstripe-simple', $url);
    // info("Making curl request to $url");
    $token = ''; // PUT YOUR TOKEN HERE
    $jsonStr = empty($data) ? '' : json_encode($data, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, !empty($data));
    if ($httpMethod) {
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $httpMethod);
    }
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'User-Agent: silverstripe-module-standardiser',
        'Accept: application/vnd.github+json',
        "Authorization: Bearer $token",
        'X-GitHub-Api-Version: 2022-11-28'
    ]);
    if ($jsonStr) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonStr);
    }
    $response = curl_exec($ch);
    $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);
    if ($httpcode >= 300) {
        echo "HTTP code $httpcode returned from GitHub API\n";
        echo $response . "\n";
        echo "Failure calling github api: $url\n";
    }
    return json_decode($response, true);
}

// -_____- EXECUTION STARTS HERE
$prsCreated = [];
foreach ($modules as $moduleDir => $data) {
    $branch = $data['branch'];
    $remote = $data['remote'];
    $org = dirname($moduleDir);
    preg_match('#^git@github\.com:creative-commoners/(?<repo>.*)\.git$#', $remote, $matches);
    $repo = $org . '/' . $matches['repo'];
    $fullPath = $vendorDir . $moduleDir;
    if (!is_dir($fullPath)) {
        throw new LogicException("$moduleDir is missing!");
    }
    echo "------------------\n";
    echo "Starting on $moduleDir\n";

    // Swap to correct branch, and create a new PR branch
    chdir($fullPath);
    cmd('git checkout ' . $branch);
    $prBranch = "pulls/$branch/update-callout-syntax";
    cmd("git checkout -b $prBranch");         //// UNCOMMENT THIS WHEN YOU DO IT!!!!

    // Find and replace for all markdown files
    $iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($fullPath));
    $mdIterator = new RegexIterator($iterator, '/^.+\.md$/i', RecursiveRegexIterator::GET_MATCH);
    foreach ($mdIterator as $filePath => $_) {
        // echo "File: $filePath\n";
        $fileContent = file_get_contents($filePath);
        $regex = '/\h*\[(?<type>hint|warning|info|alert|notice|note)\](?<content>.*?)\[\/\1\]\h*/s';
        $callback = function(array $matches): string {
            // Find out what type of callout this is
            $realType = match ($matches['type']) {
                'info' => 'NOTE',
                'note' => 'NOTE',
                'hint' => 'TIP',
                'warning' => 'WARNING',
                'notice' => 'WARNING',
                'alert' => 'CAUTION',
                default => throw new RuntimeException('unexpected type: ' . $matches['type'])
            };
            $content = $matches['content'];
            //skip this - it may be nested
            if (str_starts_with($content, ' ')) {
                return $matches[0];
            }
            // trim and add callout marker
            $content = trim($content);
            $result = ["> [!$realType]"];
            // Loop over each line and blockquote-ify it
            foreach (explode("\n", $content) as $line) {
                $result[] = rtrim('> ' . $line);
                $line = strtok("\n");
            }
            return implode("\n", $result);
        };
        // This is the line that actually triggers the content getting replaced
        $newFileContent = preg_replace_callback($regex, $callback, $fileContent);
        if ($newFileContent === null) {
            throw new RuntimeException('Failed to replace content!');
        }
        file_put_contents($filePath, $newFileContent);
    }

    // If there's no changes, move on to the next repo
    $diff = trim(cmd('git diff --name-only'));
    if (!$diff) {
        echo "No diff - no PR needed for this repo.\n";
        continue;
    }

    // If there have been changes, commit them and make a PR
    cmd('git add . && git commit -m "DOC Update syntax for callout blocks"');
    cmd("git remote add cc $remote");
    cmd("git push cc $prBranch");
    $prDesc = <<<DESC
    ## IMPORTANT

    This should not be merged until all of the associated PRs have all been approved. Only merge when all of the PRs are ready to go, to minimise the amount of time spent with broken callout blocks.

    ## Description
    Converts all of the legacy callout blocks to the new syntax. Note that we're not doing CMS 3. There will also not be a separate CMS 5 PR for this repo - the merge up should be pretty clean.

    The target branch was chosen based on the list in https://github.com/silverstripe/doc.silverstripe.org/blob/master/sources-user.js

    Note that the callout blocks will be converted as per https://github.com/silverstripe/doc.silverstripe.org/issues/282#issuecomment-1915939874.

    ## Issues
    - https://github.com/silverstripe/doc.silverstripe.org/issues/282
    DESC;
    $responseJson = github_api("https://api.github.com/repos/$repo/pulls", [
        'title' => 'DOC Update syntax for callout blocks',
        'body' => $prDesc,
        'head' => "creative-commoners:$prBranch",
        'base' => $branch,
    ]);
    $prsCreated[] = $responseJson['html_url'];
}

echo "---------------\n";
if (empty($prsCreated)) {
    echo "No PRs created:\n";
} else {
    echo "PRs created:\n";
    foreach ($prsCreated as $url) {
        echo "- $url\n";
    }
}

The regex is based off the regex that's currently used to render callout blocks. Since we know that doesn't perfectly catch all callout blocks I also manually ran a text search to look for any that the regex missed.

Note that most of the userhelp docs didn't have any changes, so there's no PR for them.

[emteknetnz]

@GuySartorelli Assigned back to you to do follow up PR

[emteknetnz]

@GuySartorelli have merged PRs, assigning back to you to check things work as expected

[GuySartorelli]

Things render correctly now! 🎉