newlines treated differently in pure java implementation

[tmm1]

kind of a stupid bug to be reporting, but its breaking some tests that assume the MRI behavior.

in MRI, newlines appear to be stripped:

$ ruby -rubygems -ve' require "nokogiri"; p Nokogiri::VERSION; p Nokogiri::HTML::DocumentFragment.parse("<p>hi</p>\n").to_html '
ruby 1.8.7 (2010-12-23 patchlevel 330) [i686-darwin10.5.0]
"1.5.0.beta.4"
"<p>hi</p>"

in the java version, the newline is passed through to the output:

$ ruby -rubygems -ve' require "nokogiri"; p Nokogiri::VERSION; p Nokogiri::HTML::DocumentFragment.parse("<p>hi</p>\n").to_html '
jruby 1.6.0 (ruby-1.8.7-p330) (2011-04-06 aa7d946) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_24) [darwin-x86_64-java]
"1.5.0.beta.4"
"<p>hi</p>\n"

[yokolet]

This is not a stupid bug. This is a rather serious parsing problem. Java version creates Text node of "\n" as a sibling of element p, while libxml version doesn't. I'll have a look.

[headius]

If I remember right, there's modes in most of the Java parsers to preserve all whitespace or not. I think that's what's at play here.

[yokolet]

@headius Xerces does have that option. But, xerces needs schema (grammar) to treat whitespaces. If no schema is given, ignore whitespace option does nothing. Xerces is really strict.

[yokolet]

@tmm1 The bug is fixed in rev. 3f2e575

When no grammar is given, trailing whitespace is cut out from a fragment string. If possible, would you try this using master branch?

[yokolet]

This should be closed.