[Laszlo-dev] [Updated] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding
Raju Bitter
rajubitter at me.com
Mon Jun 8 16:08:22 PDT 2009
Tucker, Wanli,
I've done some testing with OL 4.0.15 and Webtop running 1.5.2 running
on OL 4.0.15. While the fix works for OL trunk, it doesn't work for
4.0.15. Here's the debug output for test file "test/css/encoding/
utf8_with_BOM_no_charset_attr.lzx". The UTF-8 encoded data is read
correctly.
http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
> compiler.StyleSheetCompiler - StyleSheetCompiler.compile called!
> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
> compiler.StyleSheetCompiler - @charset=utf-8 found on stylesheet tag
> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
> compiler.StyleSheetCompiler - reading in stylesheet from
> src="utf8_with_BOM_no_charset_attr.css"
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
> Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
> Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
> Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/src/
> svn/openlaszlo/4.0.15/test/css/encoding/
> utf8_with_BOM_no_charset_attr.css
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
> addFile Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/
> encoding/utf8_with_BOM_no_charset_attr.css
> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
> creating CSSHandler
> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
> Trying to parse CSS with charset setting of utf-8
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
> Read the following bytes: text[name='german'] {
> backgroundColor: #ffcc00;
> buttonText: "öäüÖÄܧ";
> }
>
> text[nam
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
> Testing for UTF-8 BOM!
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
> Found BOM on file, encoding is UTF-8
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
> Opening CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
> css/encoding/utf8_with_BOM_no_charset_attr.css using encoding UTF-8
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
> Skip first 3 bytes containing UTF-8 BOM
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - compiling CSSHandler using new unique
> names
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Conditional selector: [name="german"]
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Attribute condition
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - simple selector:text
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
> attrvalue: "german", simpleselector: "text"}
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Conditional selector: [name="korean"]
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Attribute condition
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - simple selector:text
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
> attrvalue: "korean", simpleselector: "text"}
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Conditional selector: [name="chinese"]
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Attribute condition
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - simple selector:text
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
> attrvalue: "chinese", simpleselector: "text"}
> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
> compiler.StyleSheetCompiler - whole stylesheet as css $lzc
> $style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
> "german", simpleselector: "text"}, {backgroundColor: 0xFFCC00,
> buttonText: "???????"}));
> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
> "korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
> buttonText: "? ?? ??? ?? ?? ?? ?? ??"}));
> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
> "chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
> buttonText: "???????????????"}))
But the debug console in the browser shows the following error message:
> ERROR: Unknown cohort for rule: #0 * (undefined)
> ERROR: Unknown cohort for rule: #1 * (undefined)
> ERROR: Unknown cohort for rule: #2 * (undefined)
> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
> node «lz.text#3| #german» for property name buttonText
> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
> node «lz.text#3| #german» for property name backgroundColor
> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
> node «lz.text#4| #korean» for property name buttonText
> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
> node «lz.text#4| #korean» for property name backgroundColor
> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
> node «lz.text#5| #chinese» for property name buttonText
> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
> node «lz.text#5| #chinese» for property name backgroundColor
Any idea what this might be?
- Raju
On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
> [cc-ing Wanli]
>
> Wanli,
>
> Maybe you also want to review (or have someone on your team review)
> as this issue affects you?
>
> ---
>
> Raju,
>
> This looks good.
>
> I think you might have trouble trying checking in your utf-16
> examples to svn. I'm not sure svn will do the right thing. It
> looks like it is treating them as binary files. Maybe that will
> work. I don't know.
>
> It looks to me like you may also have some tab characters in your
> sources. You need to expand tabs to check in, otherwise the pre-
> commit filter will reject your checkin.
>
> I approve this change. You might want to check in your fix
> separately from your tests, just in case the UTF-16 files cause svn
> issues.
>
> On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
>
>> Added a "+" in debug output string concatenation.
>>
>> Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de on
>> 2009-06-08 20:09:02 CEST
>> in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
>> for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk
>>
>> Summary: Fix for CSS parser uses incorrect file encoding
>>
>> New Features: Adds an optional @charset to the stylesheet tag, in
>> case the user wants to use a CSS file in a different encoding then
>> utf-8
>>
>> Bugs Fixed: LPP-8045
>>
>> Technical Reviewer: ptw
>> QA Reviewer: (pending)
>> Doc Reviewer: (pending)
>>
>> Documentation:
>>
>> Release Notes:
>>
>> Details:
>> + StyleSheetCompiler.java: Handling for @charset on stylesheet tag
>> added. Added a
>> 2nd parameter with the encoding value to the CSSHandler.parse() call.
>>
>> + CSSHandler.java:
>> parse() method takes the encoding from the LZX stylesheet tag as a
>> 2nd parameter.
>> getInputSource() method does a few more things now:
>> 1) checks for a possible BOM on the CSS file
>> 2) checks if a possible BOM conflicts with the value of the
>> stylesheet tag's @charset
>> 3) if there's a BOM, the BOM bytes are removed from the input stream
>>
>> + FileUtils.java:
>> Added method public static String
>> detectBOMEncoding(BufferedInputStream in)
>> The method returns the BOM marker interpreted as one of the
>> following strings:
>> UTF-8
>> UTF-16LE
>> UTF-16BE
>>
>> Tests:
>> + test files in folder test/css/encoding
>> The following test exist:
>> 1) iso8859-1_with_charset_attr.lzx
>> Reading an iso-8859-2 encoded CSS file with some German special
>> chars
>> 2) utf16BE_with_BOM.lzx
>> Reading an utf-16 BE CSS file with BOM marker
>> 3) utf16LE_with_BOM.lzx
>> Reading an utf-16 LE CSS file with BOM marker
>> 4) utf8_with_BOM_no_charset_attr.lzx
>> Reading an utf-8 CSS file with BOM and no charset attribute on
>> the stylesheet tag
>> 5) utf8_with_BOM_conflicting_charset_attr.lzx
>> Reading a CSS with @charset value of utf-16, but CSS having a
>> UTF-8 BOM marker, will
>> throw a compile error
>>
>> Files:
>> A test/css/encoding
>> A test/css/encoding/iso8859-1_with_charset_attr.lzx
>> A test/css/encoding/utf16BE_with_BOM.lzx
>> A test/css/encoding/utf8_with_BOM_no_charset_attr.css
>> A test/css/encoding/utf16LE_with_BOM.css
>> A test/css/encoding/utf8_no_BOM_no_charset_attr.css
>> A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
>> A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>> A test/css/encoding/utf16LE_with_BOM.lzx
>> A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
>> A test/css/encoding/iso8859-1_with_charset_attr.css
>> A test/css/encoding/utf16BE_with_BOM.css
>> A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
>> M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
>> M WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
>> M WEB-INF/lps/server/src/org/openlaszlo/compiler/
>> StyleSheetCompiler.java
>>
>> Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
>
More information about the Laszlo-dev
mailing list