Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MDN zimit 2 ZIM has issue with demo output iframe on Android #4084

Open
benoit74 opened this issue Nov 14, 2024 · 13 comments
Open

MDN zimit 2 ZIM has issue with demo output iframe on Android #4084

benoit74 opened this issue Nov 14, 2024 · 13 comments
Assignees
Labels
Milestone

Comments

@benoit74
Copy link

Describe the bug
I have a test ZIM with an iframe with has an srcdoc attribute instead of the classical src (i.e. code is "inline").

The iframe stays blank.

This is the same issue as kiwix/kiwix-apple#1027 but Android-side.

Expected behavior

iFrame content displays

Steps to reproduce the behavior:

Test ZIM: tests_eng_mdn-page_2024-11.zim.zip (remove .zip extension added to please GitHub)

Open the ZIM and see the blank "Output" area

Screenshots

Image

Environment

Logs
NA

@MohitMaliFtechiz
Copy link
Collaborator

@benoit74, It is a bit weird the HTML content is changing after going to the webView. There is an empty <title> so the whole script is going inside the <title> tag.

The incoming HTML from the ZIM file seems correct.

<!doctype html>
<html lang="en">
<head>


    <!-- WB Insert -->
    <script src="../../../_zim_static/wombat.js"></script>
    <script src="../../../_zim_static/wombatSetup.js"></script>
    <script>
        // Get the current url we are serving from.
        // We cannot know it at zim creation as it depends of our server.
        const current_url = (function () {
          var href = new URL(window.location.href);
          href.hash = "";

          return href.toString();
        })();

        const wbinfo = wombatSetup.getWombatInfo(
          current_url,
          "interactive-examples.mdn.mozilla.net",
          "https",
          "https://interactive-examples.mdn.mozilla.net/pages/tabbed/section.html",
          current_url.substring(0, current_url.length - ("interactive-examples.mdn.mozilla.net/pages/tabbed/section.html".length))
        );

        if (window && window._WBWombatInit) {
          window._WBWombatInit(wbinfo);
        }
    </script>


    <!-- End WB Insert -->
    <meta charset="UTF-8"/>
    <meta http-equiv="x-ua-compatible" content="ie=edge"/>
    <title>HTML Demo: &lt;section></title>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <link href="../../css/codemirror.css%3Fv%3D341af68" rel="stylesheet"/>
    <link href="../../css/editor-tabbed.css%3Fv%3D341af68" rel="stylesheet"/>
</head>

<body>
<header class="output-header border-rounded-top">
    <h4 class="output-heading">HTML Demo: &lt;section></h4>
    <button id="reset" class="reset" type="button">Reset</button>
</header>
<noscript>
    <div id="warning-no-script" class="warning-container">
        <div class="warning">
            The interactive example cannot be shown because JavaScript is
            disabled.
        </div>
    </div>
</noscript>
<div id="warning-mathml-not-supported" class="warning-container hidden">
    <div class="warning">
        The interactive example cannot be shown because MathML is not supported
        by your browser.
    </div>
</div>
<div id="editor-container" class="editor-container tabbed-standard hidden border-rounded-bottom"
     data-editor-type="tabbed">
    <section id="tab-container" class="tabs">
        <div class="tab-list" id="tablist" role="tablist">
            <button role="tab" aria-selected="false" aria-controls="html-panel" id="html"
                    class="hidden">
                HTML
            </button>
            <button role="tab" aria-selected="false" aria-controls="css-panel" id="css"
                    class="hidden" tabindex="-1">
                CSS
            </button>
            <button role="tab" aria-selected="false" aria-controls="js-panel" id="js" class="hidden"
                    tabindex="-1">
                JavaScript
            </button>
        </div>
        <section id="html-panel" tabindex="0" role="tabpanel" aria-labelledby="html" class="hidden"
                 aria-hidden="true">
            <div id="html-editor">
                                                                                                                <pre><code>&lt;h1>Choosing an Apple&lt;/h1>
                                                                                                    &lt;section>
                                                                                                      &lt;h2>Introduction&lt;/h2>
                                                                                                      &lt;p>This document provides a guide to help with the important task of choosing the correct Apple.&lt;/p>
                                                                                                    &lt;/section>

                                                                                                    &lt;section>
                                                                                                      &lt;h2>Criteria&lt;/h2>
                                                                                                      &lt;p>
                                                                                                        There are many different criteria to be considered when choosing an Apple — size, color, firmness, sweetness,
                                                                                                        tartness...
                                                                                                      &lt;/p>
                                                                                                    &lt;/section>
                                                                                                    </code></pre>
            </div>
        </section>
        <section id="css-panel" tabindex="0" role="tabpanel" aria-labelledby="css" class="hidden"
                 aria-hidden="true">
            <div id="css-editor">
                                                                                                                <pre>
                                                                                                                <code>h1,
                                                                                                    h2 {
                                                                                                      margin: 0;
                                                                                                    }
                                                                                                    </code>
                                                                                                    </pre>
            </div>
        </section>
        <section id="js-panel" tabindex="0" role="tabpanel" aria-labelledby="js" class="hidden"
                 aria-hidden="true">
            <div id="js-editor">
                <pre><code></code></pre>
            </div>
        </section>
    </section>
    <div id="output" class="output-container">
        <h4 class="output-label">Output</h4>
        <iframe id="output-iframe"></iframe>
    </div>
</div>

<section class="console-container hidden" aria-hidden="true">
    <h4 class="console-label">Console Output</h4>
    <button type="button" id="clear" class="clear">
        <svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 877.714 1024" class="icon-clear">
            <path class="path0" fill="currentColor"
                  d="M749.714 510.286c0-62.286-18.286-120-49.714-168.571L269.143 772.001c49.143 32 107.429 50.857 169.714 50.857 171.429 0 310.857-140 310.857-312.571zM178.857 681.143l431.429-430.857c-49.143-33.143-108-52-171.429-52-171.429 0-310.857 140-310.857 312 0 63.429 18.857 121.714 50.857 170.857zm698.857-170.857c0 243.429-196.571 440.571-438.857 440.571S0 753.714 0 510.286c0-242.857 196.571-440 438.857-440s438.857 197.143 438.857 440z"/>
        </svg>
        clear console
    </button>
    <div id="console" class="console"><code></code></div>
</section>
<template id="output-body">
    <div id="html-output" class="output editor-tabbed">
        %html-content%
    </div>

    <script>
        function executeExample() {
          'use script';
          try {
          %js-content%
          } catch(e) {
            console.error(e);
          }
        }
    </script>
</template>
<template id="output-head">
    <meta charset="UTF-8" />
    <title></title>
    <meta http-equiv="x-ua-compatible" content="ie=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <link href="../../css/editor-tabbed.css%3Fv%3D341af68" rel="stylesheet" />
    <style>
                                                                                                            
    </style>
    <style id="css-output">
      %css-content%
    </style>
</template>
<script src="../../js/codemirror.js%3Fv%3D341af68"></script>
<script src="../../js/editor-tabbed.js%3Fv%3D341af68"></script>
</body>
</html>

and the HTML that is changing is this which is inside the srcdoc.

<template id="output-head">
    <meta charset="UTF-8" />
    <title></title>
    <meta http-equiv="x-ua-compatible" content="ie=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <link href="../../css/editor-tabbed.css%3Fv%3D341af68" rel="stylesheet" />
    <style>
                                                                                                            
    </style>
    <style id="css-output">
      %css-content%
    </style>
</template>
<template id="output-body">
    <div id="html-output" class="output editor-tabbed">
        %html-content%
    </div>

    <script>
        function executeExample() {
          'use script';
          try {
          %js-content%
          } catch(e) {
            console.error(e);
          }
        }
    </script>
</template>

The output-head is changing after going into the webView. It becomes this.

<iframe id="output-iframe" srcdoc="
<!DOCTYPE html>
<html id=&quot;output-root&quot;>
<head>
      <meta charset=&quot;UTF-8&quot;>
      <meta http-equiv=&quot;x-ua-compatible&quot; content=&quot;ie=edge&quot;>
      <title><..editor-tabbed.css%3Fv%3D341af68&quot; rel=&quot;stylesheet&quot;>
      <style>
        
      <style>
    </head>
<body>
      <div id=&quot;html-output&quot; class=&quot;output editor-tabbed&quot;>
        <h1>Choosing an Apple</h1>
<section>
  <h2>Introduction</h2>
  <p>This document provides a guide to help with the important task of choosing the correct Apple.</p>
</section>

<section>
  <h2>Criteria</h2>
  <p>
    There are many different criteria to be considered when choosing an Apple — size, color, firmness, sweetness,
    tartness...
  </p>
</section>

      <script>
    </body>
</html>
"></iframe>

Image

As you can see there are two <style> tags but without closing tags. Also, there <link> is going inside the <title> tag, and the closing </title> tag is missing. Because of these issues, the content is not displaying.

If I properly close these tags the content starts displaying.

Image

So it might be two issues, either the issue is inside the script from where this HTML code is changing(editor-tabbed.js) since from here we are changing the HTML.

Image

or it is a webView problem it is not reading the content.

So I have started my research on this and found there is an empty title tag in the <template id="output-head"> which causing the problem. If I remove the title tag from the ZIM content then it starts the loading content.

Image

but still, you can see the example is not perfectly working, there is h1,h2 { margin: 100; } showing in the output which is wrong. It is because of style tag is going inside the link tag as you can see.

Image

So now we know the title tag is not creating a problem the problem is something else.

If I look at the loaded content in webView it is correctly loading as you can see in the below screenshots,

With title tag Without Title tag
Image Image

But after changing it via script the output-head HTML is corrupted. So there is some issue inside the script which is causing this error.

@benoit74
Copy link
Author

Thank you for the detailed analysis and feedback!

It is a bit weird the HTML content is changing after going to the webView.

This is unfortunately the original website behavior, and it is kept unmodified in the ZIM.

What is weird is that the same ZIM works in all other Kiwix readers (besides Apple which had also an issue but it is mostly solved now), so HTML/JS code is probably not that bad.

Can you please try to load original URL https://developer.mozilla.org/en-US/docs/Web/HTML/Element/section into a Webview and see if we have the same problem or not? It could help to indicate it is a bug linked to bad cooperation between MDN code and the webview, or if we have a problem more linked to wombat / warc2zim / specific code path on Android webview in the JS (would be weird, but why not ...). Thanks

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Nov 19, 2024

@benoit74 Directly loading the original URL in webView working fine. So it is not a compatibility issue with webView.

Image

Image

@benoit74 benoit74 changed the title Reader does not support iframes with srcdoc attribute MDN zimit 2 ZIM has issue with demo output iframe on Android Nov 20, 2024
@benoit74
Copy link
Author

OK, so this is clearly a problem of MDN + Android. Let's park this for now, I'm not sure it is really worth the effort digging into their JS more than you have.

@kelson42
Copy link
Collaborator

@benoit Not sure to follow you, considering online web site works, either there is a bug in the ZIM or in the Reader. Should be fixed IMHO, either sooner than later. Postponing to next milestone but should be still treated in prio once current milestone closed.

@kelson42 kelson42 modified the milestones: 3.12.0, 3.13.0, 3.12.1 Nov 20, 2024
@benoit74
Copy link
Author

For me since online website is working inside the webview, ZIM is working on all other readers, and fixing some HTML manually help it to load, it means this bug is a rare edge case linked to bad cooperation between the ZIM HTML / JS (especially wombat.js) and the Android reader. And analysis done so far tends to prove it is really nasty. From my PoV, I consider it is not worth investigating until we get repro on other websites to get a better grasp of the problem. I consider we have more urgent issues to tackle which would benefit the whole ecosystem, and not just MDN ZIM. But if you have the resources / will to investigate and fix, it will be to my pleasure of course !

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 2, 2024

@benoit74 I have tried to debug the wombat.js how it behaves differently with the Android reader, and I found that overrideHtmlAssignSrcDoc is responsible for writing the content. I have tried to print the incoming HTML there, and I see that the incoming HTML to this method is wrong. See the screenshots below.

Image
Image

So my question is: where is it getting the HTML for writing or overriding? Is it taking it directly from the WebView? if yes, so the webView is correctly loading the content as in my previous analysis? Then why it is coming here like this? Is it stripping out some empty HTML tags before passing it?

Edited

I have just noticed that directly loading the website in webView is working but it has the updated js, and CSS files, and in our ZIM file we are using the old js, and CSS file.

We are using the editor-tabbed.css?v=341af68, and editor-tabbed.js?v341af68 files.

Image

And MDN website uses the editor-tabbed.css?v=220ae8b, and editor-tabbed.js?=220ae8b files.

Image

@benoit74
Copy link
Author

benoit74 commented Dec 3, 2024

The fact that you've found version discrepancy is not a surprise, the ZIM is a capture of the website at a given moment. Do you mean that they might have fixed the problem? Worth to give it a try indeed, I will look into it.

The HTML which is fed to wombat comes from MDN JS which source it from the web page AFAIK.

@benoit74
Copy link
Author

benoit74 commented Dec 3, 2024

New ZIM: tests_eng_mdn-page-20241203_2024-12.zim.zip (remove .zip extension as usual)

Same problem, still working on all readers (even PWA on Android) but not in Android Kiwix app.

@MohitMaliFtechiz
Copy link
Collaborator

Do you mean that they might have fixed the problem? Worth to give it a try indeed, I will look into it.

Probably, not sure about this, but I can view the content with updated JS code in webView so it is worth trying.

The HTML which is fed to wombat comes from MDN JS which source it from the web page AFAIK.

Okay, so it is coming from the MDN JS code, so the issue is somewhere there in JS.

New ZIM: tests_eng_mdn-page-20241203_2024-12.zim.zip (remove .zip extension as usual)

Thanks for the ZIM file.

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 3, 2024

The error is in the JS code of the editor-tabbed file, I have debugged it and found when it is returning the HTML from this JS file then it is corrupting the HTML.

For example:
Here we are getting the content via the innerHTML tag and returning it.

Image

But this tag corrupts the HTML of the webView see the below screenshot.

Image

If I print the whole e, and t tag then HTML looks correct.

Image

Edited

I don't know why it is not properly supporting the innerHTML tag inside the template tag, but getting HTML via another method works in this.

Image

@benoit74
Copy link
Author

benoit74 commented Dec 3, 2024

Thank you! So to rephrase your findings:

  • MDN js code rely on document.getElementById("output_body").innerHTML
  • for some reason inside Android Webview this method is not returning correct HTML (while it seems like it does on other readers)
  • this issue is hence mostly a bad cooperation between MDN JS and Android Webview, or maybe even a bug of Android Webview JS engine

Are these statements correct?

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 4, 2024

@benoit74 Yes, your points are correct.

 ! function() {
                    const e = document.getElementById("clear"),
                        t = document.getElementById("editor-container"),
                        n = document.getElementById("tab-container"),
                        c = document.getElementById("output"),
                        a = document.querySelector(".output-header"),
                        d = document.getElementById("css-editor"),
                        s = document.getElementById("html-editor"),
                        l = document.getElementById("js-editor"),
                        u = d.querySelector("pre"),
                        m = s.querySelector("pre"),
                        f = l.querySelector("pre"),
                        y = document.getElementById("output-iframe"),
                        h = function() {
                            const e = document.getElementById("output-head"),
                                t = document.getElementById("output-body");
                                const templateHeadContent = e.content;
                                const templateBodyContent = t.content;
                                const serializedHeadContent = Array.from(templateHeadContent.childNodes)
                                    .map(node => node.outerHTML || node.textContent)
                                    .join('');
                                const serializedBodyContent = Array.from(templateBodyContent.childNodes)
                                                           .map(node => node.outerHTML || node.textContent)
                                                           .join('');
                                    console.log("HEAD_DATA\n",e, serializedHeadContent, "\nBODY_DATA", serializedBodyContent);
                            return `\n<!DOCTYPE html>\n<html id="output-root">\n<head>${serializedHeadContent}</head>\n<body>${serializedBodyContent}</body>\n</html>\n`
                        }(),

If I use this code then it correctly loading the content in webView.

Image

@kelson42 kelson42 modified the milestones: 3.13.0, 3.14.0 Dec 13, 2024
@kelson42 kelson42 modified the milestones: 3.14.0, 3.15.0 Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants