[Laszlo-dev] [Updated] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding
Raju Bitter
rajubitter at me.com
Tue Jun 9 15:56:47 PDT 2009
Sorry, Amy, just saw that mail now. I'll do that tomorrow.
Best,
Raju
On Jun 9, 2009, at 4:41 PM, Amy Muntz wrote:
> Hi Raju,
> Please check in the trunk fix for 4.4 right away. It may take an
> additional fix for 4.0.x, but that's just fine.
> We can re-open the bug for 4.0.x and it can be fixed at a later time.
> Thanks,
> Amy
>
>
> On Mon, Jun 8, 2009 at 7:08 PM , Raju Bitter wrote:
>
>> Tucker, Wanli,
>>
>> I've done some testing with OL 4.0.15 and Webtop running 1.5.2
>> running on OL 4.0.15. While the fix works for OL trunk, it doesn't
>> work for 4.0.15. Here's the debug output for test file "test/css/
>> encoding/utf8_with_BOM_no_charset_attr.lzx". The UTF-8 encoded data
>> is read correctly.
>>
>> http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>> compiler.StyleSheetCompiler - StyleSheetCompiler.compile called!
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>> compiler.StyleSheetCompiler - @charset=utf-8 found on stylesheet tag
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>> compiler.StyleSheetCompiler - reading in stylesheet from
>>> src="utf8_with_BOM_no_charset_attr.css"
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>>> Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
>>> Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>>> Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/
>>> src/ svn/openlaszlo/4.0.15/test/css/encoding/
>>> utf8_with_BOM_no_charset_attr.css
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
>>> addFile Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
>>> css/ encoding/utf8_with_BOM_no_charset_attr.css
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>>> creating CSSHandler
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>>> Trying to parse CSS with charset setting of utf-8
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>> Read the following bytes: text[name='german'] {
>>> backgroundColor: #ffcc00;
>>> buttonText: "öäüÖÄܧ";
>>> }
>>>
>>> text[nam
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>> Testing for UTF-8 BOM!
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>> Found BOM on file, encoding is UTF-8
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>>> Opening CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
>>> css/encoding/utf8_with_BOM_no_charset_attr.css using encoding UTF-8
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>>> Skip first 3 bytes containing UTF-8 BOM
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - compiling CSSHandler using new
>>> unique names
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Conditional selector: [name="german"]
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Attribute condition
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - simple selector:text
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>>> attrvalue: "german", simpleselector: "text"}
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Conditional selector: [name="korean"]
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Attribute condition
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - simple selector:text
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>>> attrvalue: "korean", simpleselector: "text"}
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Conditional selector: [name="chinese"]
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Attribute condition
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - simple selector:text
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - Cond string: { attrname: "name",
>>> attrvalue: "chinese", simpleselector: "text"}
>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>> compiler.StyleSheetCompiler - whole stylesheet as css $lzc
>>> $style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>>> "german", simpleselector: "text"}, {backgroundColor: 0xFFCC00,
>>> buttonText: "???????"}));
>>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>>> "korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
>>> buttonText: "? ?? ??? ?? ?? ?? ?? ??"}));
>>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>>> "chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
>>> buttonText: "???????????????"}))
>>
>> But the debug console in the browser shows the following error
>> message:
>>> ERROR: Unknown cohort for rule: #0 * (undefined)
>>> ERROR: Unknown cohort for rule: #1 * (undefined)
>>> ERROR: Unknown cohort for rule: #2 * (undefined)
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
>>> for node «lz.text#3| #german» for property name buttonText
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
>>> for node «lz.text#3| #german» for property name backgroundColor
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
>>> for node «lz.text#4| #korean» for property name buttonText
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
>>> for node «lz.text#4| #korean» for property name backgroundColor
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
>>> for node «lz.text#5| #chinese» for property name buttonText
>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
>>> for node «lz.text#5| #chinese» for property name backgroundColor
>>
>> Any idea what this might be?
>>
>> - Raju
>>
>>
>> On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
>>
>>> [cc-ing Wanli]
>>>
>>> Wanli,
>>>
>>> Maybe you also want to review (or have someone on your team
>>> review) as this issue affects you?
>>>
>>> ---
>>>
>>> Raju,
>>>
>>> This looks good.
>>>
>>> I think you might have trouble trying checking in your utf-16
>>> examples to svn. I'm not sure svn will do the right thing. It
>>> looks like it is treating them as binary files. Maybe that will
>>> work. I don't know.
>>>
>>> It looks to me like you may also have some tab characters in your
>>> sources. You need to expand tabs to check in, otherwise the pre-
>>> commit filter will reject your checkin.
>>>
>>> I approve this change. You might want to check in your fix
>>> separately from your tests, just in case the UTF-16 files cause
>>> svn issues.
>>>
>>> On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
>>>
>>>> Added a "+" in debug output string concatenation.
>>>>
>>>> Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de
>>>> on 2009-06-08 20:09:02 CEST
>>>> in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
>>>> for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk
>>>>
>>>> Summary: Fix for CSS parser uses incorrect file encoding
>>>>
>>>> New Features: Adds an optional @charset to the stylesheet tag, in
>>>> case the user wants to use a CSS file in a different encoding
>>>> then utf-8
>>>>
>>>> Bugs Fixed: LPP-8045
>>>>
>>>> Technical Reviewer: ptw
>>>> QA Reviewer: (pending)
>>>> Doc Reviewer: (pending)
>>>>
>>>> Documentation:
>>>>
>>>> Release Notes:
>>>>
>>>> Details:
>>>> + StyleSheetCompiler.java: Handling for @charset on stylesheet
>>>> tag added. Added a
>>>> 2nd parameter with the encoding value to the CSSHandler.parse()
>>>> call.
>>>>
>>>> + CSSHandler.java:
>>>> parse() method takes the encoding from the LZX stylesheet tag as
>>>> a 2nd parameter.
>>>> getInputSource() method does a few more things now:
>>>> 1) checks for a possible BOM on the CSS file
>>>> 2) checks if a possible BOM conflicts with the value of the
>>>> stylesheet tag's @charset
>>>> 3) if there's a BOM, the BOM bytes are removed from the input
>>>> stream
>>>>
>>>> + FileUtils.java:
>>>> Added method public static String
>>>> detectBOMEncoding(BufferedInputStream in)
>>>> The method returns the BOM marker interpreted as one of the
>>>> following strings:
>>>> UTF-8
>>>> UTF-16LE
>>>> UTF-16BE
>>>>
>>>> Tests:
>>>> + test files in folder test/css/encoding
>>>> The following test exist:
>>>> 1) iso8859-1_with_charset_attr.lzx
>>>> Reading an iso-8859-2 encoded CSS file with some German special
>>>> chars
>>>> 2) utf16BE_with_BOM.lzx
>>>> Reading an utf-16 BE CSS file with BOM marker
>>>> 3) utf16LE_with_BOM.lzx
>>>> Reading an utf-16 LE CSS file with BOM marker
>>>> 4) utf8_with_BOM_no_charset_attr.lzx
>>>> Reading an utf-8 CSS file with BOM and no charset attribute on
>>>> the stylesheet tag
>>>> 5) utf8_with_BOM_conflicting_charset_attr.lzx
>>>> Reading a CSS with @charset value of utf-16, but CSS having a
>>>> UTF-8 BOM marker, will
>>>> throw a compile error
>>>>
>>>> Files:
>>>> A test/css/encoding
>>>> A test/css/encoding/iso8859-1_with_charset_attr.lzx
>>>> A test/css/encoding/utf16BE_with_BOM.lzx
>>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.css
>>>> A test/css/encoding/utf16LE_with_BOM.css
>>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.css
>>>> A test/css/encoding/
>>>> utf8_with_BOM_conflicting_charset_attr.css
>>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>>>> A test/css/encoding/utf16LE_with_BOM.lzx
>>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
>>>> A test/css/encoding/iso8859-1_with_charset_attr.css
>>>> A test/css/encoding/utf16BE_with_BOM.css
>>>> A test/css/encoding/
>>>> utf8_with_BOM_conflicting_charset_attr.lzx
>>>> M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
>>>> M WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
>>>> M WEB-INF/lps/server/src/org/openlaszlo/compiler/
>>>> StyleSheetCompiler.java
>>>>
>>>> Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
>>>
More information about the Laszlo-dev
mailing list