[Laszlo-dev] [Updated] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding
Amy Muntz
amuntz at laszlosystems.com
Tue Jun 9 07:41:43 PDT 2009
Hi Raju,
Please check in the trunk fix for 4.4 right away. It may take an
additional fix for 4.0.x, but that's just fine.
We can re-open the bug for 4.0.x and it can be fixed at a later time.
Thanks,
Amy
On Mon, Jun 8, 2009 at 7:08 PM , Raju Bitter wrote:
> Tucker, Wanli,
>
> I've done some testing with OL 4.0.15 and Webtop running 1.5.2 running
> on OL 4.0.15. While the fix works for OL trunk, it doesn't work for
> 4.0.15. Here's the debug output for test file
> "test/css/encoding/utf8_with_BOM_no_charset_attr.lzx". The UTF-8
> encoded data is read correctly.
>
>
> http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>> compiler.StyleSheetCompiler - StyleSheetCompiler.compile called!
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>> compiler.StyleSheetCompiler - @charset=utf-8 found on stylesheet tag
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>> compiler.StyleSheetCompiler - reading in stylesheet from
>> src="utf8_with_BOM_no_charset_attr.css"
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>> Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
>> Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>> Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/src/
>> svn/openlaszlo/4.0.15/test/css/encoding/
>> utf8_with_BOM_no_charset_attr.css
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
>> addFile Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/
>> encoding/utf8_with_BOM_no_charset_attr.css
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>> creating CSSHandler
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>> Trying to parse CSS with charset setting of utf-8
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>> Read the following bytes: text[name='german'] {
>> backgroundColor: #ffcc00;
>> buttonText: "öäüÖÄܧ";
>> }
>>
>> text[nam
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>> Testing for UTF-8 BOM!
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>> Found BOM on file, encoding is UTF-8
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>> Opening CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
>> css/encoding/utf8_with_BOM_no_charset_attr.css using encoding UTF-8
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>> Skip first 3 bytes containing UTF-8 BOM
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - compiling CSSHandler using new unique
>> names
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Conditional selector: [name="german"]
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Attribute condition
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - simple selector:text
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>> attrvalue: "german", simpleselector: "text"}
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Conditional selector: [name="korean"]
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Attribute condition
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - simple selector:text
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>> attrvalue: "korean", simpleselector: "text"}
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Conditional selector: [name="chinese"]
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Attribute condition
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - simple selector:text
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>> attrvalue: "chinese", simpleselector: "text"}
>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>> compiler.StyleSheetCompiler - whole stylesheet as css $lzc
>> $style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>> "german", simpleselector: "text"}, {backgroundColor: 0xFFCC00,
>> buttonText: "???????"}));
>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>> "korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
>> buttonText: "? ?? ??? ?? ?? ?? ?? ??"}));
>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>> "chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
>> buttonText: "???????????????"}))
>
> But the debug console in the browser shows the following error
> message:
>> ERROR: Unknown cohort for rule: #0 * (undefined)
>> ERROR: Unknown cohort for rule: #1 * (undefined)
>> ERROR: Unknown cohort for rule: #2 * (undefined)
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
>> node «lz.text#3| #german» for property name buttonText
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found for
>> node «lz.text#3| #german» for property name backgroundColor
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
>> node «lz.text#4| #korean» for property name buttonText
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found for
>> node «lz.text#4| #korean» for property name backgroundColor
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
>> node «lz.text#5| #chinese» for property name buttonText
>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found for
>> node «lz.text#5| #chinese» for property name backgroundColor
>
> Any idea what this might be?
>
> - Raju
>
>
> On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
>
>> [cc-ing Wanli]
>>
>> Wanli,
>>
>> Maybe you also want to review (or have someone on your team review)
>> as this issue affects you?
>>
>> ---
>>
>> Raju,
>>
>> This looks good.
>>
>> I think you might have trouble trying checking in your utf-16
>> examples to svn. I'm not sure svn will do the right thing. It
>> looks like it is treating them as binary files. Maybe that will
>> work. I don't know.
>>
>> It looks to me like you may also have some tab characters in your
>> sources. You need to expand tabs to check in, otherwise the pre-
>> commit filter will reject your checkin.
>>
>> I approve this change. You might want to check in your fix
>> separately from your tests, just in case the UTF-16 files cause svn
>> issues.
>>
>> On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
>>
>>> Added a "+" in debug output string concatenation.
>>>
>>> Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de on
>>> 2009-06-08 20:09:02 CEST
>>> in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
>>> for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk
>>>
>>> Summary: Fix for CSS parser uses incorrect file encoding
>>>
>>> New Features: Adds an optional @charset to the stylesheet tag, in
>>> case the user wants to use a CSS file in a different encoding then
>>> utf-8
>>>
>>> Bugs Fixed: LPP-8045
>>>
>>> Technical Reviewer: ptw
>>> QA Reviewer: (pending)
>>> Doc Reviewer: (pending)
>>>
>>> Documentation:
>>>
>>> Release Notes:
>>>
>>> Details:
>>> + StyleSheetCompiler.java: Handling for @charset on stylesheet tag
>>> added. Added a
>>> 2nd parameter with the encoding value to the CSSHandler.parse()
>>> call.
>>>
>>> + CSSHandler.java:
>>> parse() method takes the encoding from the LZX stylesheet tag as a
>>> 2nd parameter.
>>> getInputSource() method does a few more things now:
>>> 1) checks for a possible BOM on the CSS file
>>> 2) checks if a possible BOM conflicts with the value of the
>>> stylesheet tag's @charset
>>> 3) if there's a BOM, the BOM bytes are removed from the input stream
>>>
>>> + FileUtils.java:
>>> Added method public static String
>>> detectBOMEncoding(BufferedInputStream in)
>>> The method returns the BOM marker interpreted as one of the
>>> following strings:
>>> UTF-8
>>> UTF-16LE
>>> UTF-16BE
>>>
>>> Tests:
>>> + test files in folder test/css/encoding
>>> The following test exist:
>>> 1) iso8859-1_with_charset_attr.lzx
>>> Reading an iso-8859-2 encoded CSS file with some German special
>>> chars
>>> 2) utf16BE_with_BOM.lzx
>>> Reading an utf-16 BE CSS file with BOM marker
>>> 3) utf16LE_with_BOM.lzx
>>> Reading an utf-16 LE CSS file with BOM marker
>>> 4) utf8_with_BOM_no_charset_attr.lzx
>>> Reading an utf-8 CSS file with BOM and no charset attribute on
>>> the stylesheet tag
>>> 5) utf8_with_BOM_conflicting_charset_attr.lzx
>>> Reading a CSS with @charset value of utf-16, but CSS having a
>>> UTF-8 BOM marker, will
>>> throw a compile error
>>>
>>> Files:
>>> A test/css/encoding
>>> A test/css/encoding/iso8859-1_with_charset_attr.lzx
>>> A test/css/encoding/utf16BE_with_BOM.lzx
>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.css
>>> A test/css/encoding/utf16LE_with_BOM.css
>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.css
>>> A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>>> A test/css/encoding/utf16LE_with_BOM.lzx
>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
>>> A test/css/encoding/iso8859-1_with_charset_attr.css
>>> A test/css/encoding/utf16BE_with_BOM.css
>>> A test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
>>> M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
>>> M WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
>>> M WEB-INF/lps/server/src/org/openlaszlo/compiler/
>>> StyleSheetCompiler.java
>>>
>>> Changeset:
>>> http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
>>
More information about the Laszlo-dev
mailing list