[Laszlo-checkins] r8708 - openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc
dda@openlaszlo.org
dda at openlaszlo.org
Wed Apr 16 08:57:24 PDT 2008
Author: dda
Date: 2008-04-16 08:57:15 -0700 (Wed, 16 Apr 2008)
New Revision: 8708
Modified:
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/Compiler.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/JavascriptGenerator.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/ParseTreePrinter.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9External.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9Generator.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9ParseTreePrinter.java
openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/TranslationUnit.java
Log:
Change 20080415-dda-G by dda at lester.local on 2008-04-15 19:43:52 EDT
in /Users/dda/laszlo/src/svn/openlaszlo/trunk-a
for http://svn.openlaszlo.org/openlaszlo/trunk
Summary: Line number tracking and options for debugging compiler for development
New Features: New compilation options for lzc:
-DtrackLines
-DdumpASTInput=filename
-DdumpASTOutput=filename
Bugs Fixed: LPP-5784: Need option to track input line numbers to output code
AKA nexb bugs
#285 LZX: Relocate sourcelocator assigned donalda enhancement 1 - ASAP LZX: OpenLaszlo core (mostly fixed, see below))
#262 LZX: Wrong sourcelocator for <script> tags assigned donalda defect 2 - Must Be LZX: OpenLaszlo core (mostly fixed)
#290 LZX: sourcelocator line numbers messed up assigned donalda defect 1 - ASAP LZX: OpenLaszlo core (mostly fixed)
Technical Reviewer: ptw (pending)
QA Reviewer: promanik (pending)
Doc Reviewer: (pending)
Documentation:
Release Notes:
Details:
There are two parts to this change.
First - When 'trackLines' is set to 'true' in the compiler options, some extra /* -*- file: #XXX -*- */
entries will be emitted into the output script. This will happen at most(**) statements,
but will not happen when the line number can be otherwise deduced by counting
from the previous /* -*- file: #XXX -*- */ seen. Tracklines can be set in lzc via:
lzc -DtrackLines --runtime=dthml file.lzx
or can be set in the Compiler.Options object. It only has an effect when script output is
produced (dhtml or swf9).
The /* -*- file: #XXX -*- */ is an abbreviated format, we aren't showing the line or column number.
(**) The flaw is that not all statements are being tracked with line number information, so it
is possible to have some statements not be tracked right. The aberation in output is temporary - at the next
statement that is tracked, we see we are off, and emit a line number. An example of the flaw:
var y1 = 17; var y2 = 37; var y3 = 43;
var xx = new String("");
y1 += y2; y2 = y3; y3 += y1 + y2;
for (var i=0; i<arr.length; i++) {
emits:
var $3_y1 = 17;
/* -*- file: #30 -*- */
var $4_y2 = 37;
/* -*- file: #30 -*- */
var $5_y3 = 43;
var $6_xx = new String("");
$3_y1 += $4_y2;
$4_y2 = $5_y3;
$5_y3 += $3_y1 + $4_y2;
/* -*- file: #33 -*- */
for (var $7_i = 0;$7_i < $1_arr.length;$7_i++) {
Notice that the line beginning y += y2 does not track correctly for subsequent statements
on that line, but at the following 'for' statement, everything is okay.
Due to the need to get this out and in use, we're reviewing before these
flaws can be addressed. I hope to correct this with a followup changeset.
Second, in trying to track this problem, is was convenient to have a mechanism to dump the
compiler's AST to files before and after it is transformed by the generator. SWF9 had a
way to do this (it was always put in the temp build directory), I've pushed this capability into
all the compilers. There are now two options, invoked from lzc via:
-DdumpASTInput=filename
-DdumpASTOutput=filename
Tests:
Ran compiler on optarg.lzx and observed output.
Added unusually formatted input and observed output.
Ran usual regressions, including runlzunit and lztest
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/Compiler.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/Compiler.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/Compiler.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -348,8 +348,19 @@
profiler.enter("parse");
source = cg.preProcess(source);
SimpleNode program = new Parser().parse(source);
+
+ String astInputFile = (String)options.get(DUMP_AST_INPUT);
+ if (astInputFile != null) {
+ emitFile(astInputFile, program);
+ System.err.println("Created " + astInputFile);
+ }
profiler.phase("generate");
SimpleNode translated = cg.translate(program);
+ String astOutputFile = (String)options.get(DUMP_AST_OUTPUT);
+ if (astOutputFile != null) {
+ emitFile(astOutputFile, translated);
+ System.err.println("Created " + astOutputFile);
+ }
if (isScript) {
@@ -454,6 +465,8 @@
public static String DEBUG_BACKTRACE = "debugBacktrace";
public static String DEBUG_SIMPLE = "debugSimple";
public static String DEBUG_EVAL = "debugEval";
+ public static String DUMP_AST_INPUT = "dumpASTInput";
+ public static String DUMP_AST_OUTPUT = "dumpASTOutput";
public static String DISABLE_CONSTANT_POOL = "disableConstantPool";
public static String ELIMINATE_DEAD_EXPRESSIONS = "eliminateDeadExpressions";
public static String FLASH_COMPILER_COMPATABILITY = "flashCompilerCompatability";
@@ -476,6 +489,7 @@
public static String PRINT_INSTRUCTIONS = "printInstructions";
public static String RESOLVER = "resolver";
public static String SCRIPT_ELEMENT = "scriptElement";
+ public static String TRACK_LINES = "trackLines";
public static String VALIDATE_CACHES = "validateCaches";
public static String WARN_UNDEFINED_REFERENCES = "warnUndefinedReferences";
public static String WARN_GLOBAL_ASSIGNMENTS = "warnGlobalAssignments";
@@ -918,6 +932,86 @@
public static String nodeString(SimpleNode node) {
return ptp.text(node);
}
+
+ // TextEmitter and associated methods are used to create
+ // files useful for debugging the compiler only, so errors are ignored.
+
+ /**
+ * A source of text data.
+ */
+ public interface TextEmitter {
+ void emit(Writer writer)
+ throws IOException;
+ }
+
+ /**
+ * Emit a file using the TextEmitter as a source for the file text.
+ */
+ public static void emitFile(String filename, TextEmitter tw) {
+ FileWriter writer = null;
+ try {
+ File f = new File(filename);
+ f.delete();
+ writer = new FileWriter(f);
+ tw.emit(writer);
+ writer.close();
+ writer = null;
+ }
+ catch (IOException ioe) {
+ System.err.println("Cannot write to " + filename);
+ if (writer != null) {
+ try {
+ writer.close();
+ }
+ catch (IOException ioe2) {
+ // ignored.
+ }
+ }
+ }
+ }
+
+ /**
+ * emit a file with the given String text
+ */
+ public static void emitFile(String filename, final String txt) {
+ emitFile(filename, new TextEmitter() {
+ public void emit(Writer writer)
+ throws IOException {
+ writer.write(txt);
+ }
+ });
+ }
+
+ /**
+ * emit a file with the given node (to be dumped) as the text.
+ */
+ public static void emitFile(String filename, final SimpleNode node) {
+ emitFile(filename, new TextEmitter() {
+ public void emit(Writer writer)
+ throws IOException {
+ nodeFileDump(writer, "", node);
+ }
+ });
+ }
+
+ /**
+ * Helper method to dump a node into a file.
+ */
+ public static void nodeFileDump(Writer writer, String prefix, SimpleNode node)
+ throws IOException
+ {
+ writer.write(node.toString(prefix) + "\n");
+ SimpleNode[] children = node.getChildren();
+ if (children != null) {
+ for (int i = 0; i < children.length; ++i) {
+ SimpleNode n = (SimpleNode)children[i];
+ if (n != null) {
+ nodeFileDump(writer, prefix + " ", n);
+ }
+ }
+ }
+ }
+
}
/**
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/JavascriptGenerator.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/JavascriptGenerator.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/JavascriptGenerator.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -194,7 +194,8 @@
public List makeTranslationUnits(SimpleNode translatedNode, boolean compress, boolean obfuscate)
{
- return (new ParseTreePrinter(compress, obfuscate)).makeTranslationUnits(translatedNode);
+ boolean trackLines = options.getBoolean(Compiler.TRACK_LINES);
+ return (new ParseTreePrinter(compress, obfuscate, trackLines)).makeTranslationUnits(translatedNode);
}
public byte[] postProcess(List tunits) {
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/ParseTreePrinter.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/ParseTreePrinter.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/ParseTreePrinter.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -47,6 +47,7 @@
public static final boolean DEBUG_NODE_OUTPUT = false;
boolean compress;
+ boolean trackLines;
String SPACE;
String NEWLINE;
String COMMA;
@@ -71,10 +72,15 @@
this(compress, false);
}
+ public ParseTreePrinter(boolean compress, boolean obfuscate) {
+ this(compress, obfuscate, false);
+ }
+
// TODO: [2007-11-21 dda] if compress/obfuscate are on, probably
// can turn off generation of annotations.
- public ParseTreePrinter(boolean compress, boolean obfuscate) {
+ public ParseTreePrinter(boolean compress, boolean obfuscate, boolean trackLines) {
this.compress = compress;
+ this.trackLines = trackLines;
// Set whitespace
this.SPACE = compress ? "" : " ";
this.NEWLINE = obfuscate ? "" : "\n";
@@ -1032,14 +1038,32 @@
AnnotationProcessor ap = new AnnotationProcessor() {
TranslationUnit curtu = defaulttu;
+ boolean atBol = true;
+ int linenumDiff = 0;
public String notify(char op, String operand) {
switch (op) {
case ANNOTATE_OP_TEXT:
curtu.addText(operand);
+ if (trackLines) {
+ if (operand.endsWith("\n")) {
+ atBol = true;
+ }
+ else if (operand.length() > 0) {
+ atBol = false;
+ }
+ }
return "";
case ANNOTATE_OP_LINENUM:
- curtu.setInputLineNumber(Integer.parseInt(operand));
+ int linenum = Integer.parseInt(operand);
+ if (trackLines && atBol && linenum != 0) {
+ int newdiff = curtu.getTextLineNumber() - linenum;
+ if (newdiff != linenumDiff) {
+ curtu.addText("/* -*- file: #" + linenum + " -*- */\n");
+ linenumDiff = newdiff + 1; // account for the line just added
+ }
+ }
+ curtu.setInputLineNumber(linenum);
return "";
case ANNOTATE_OP_CLASSNAME:
curtu = new TranslationUnit();
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9External.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9External.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9External.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -448,12 +448,12 @@
String bigErrorString = "";
int bigErrorCount = 0;
- if (SWF9Generator.DEBUG_OUTPUT) {
- String buildsh = "#!/bin/sh\n";
- buildsh += "cd " + dir + "\n";
- buildsh += prettycmd + "\n";
- emitFile(workDirectoryName("build.sh"), buildsh);
- }
+ // Generate a small script (unix style) to document how
+ // to build this batch of files.
+ String buildsh = "#!/bin/sh\n";
+ buildsh += "cd " + dir + "\n";
+ buildsh += prettycmd + "\n";
+ Compiler.emitFile(workDirectoryName("build.sh"), buildsh);
Process proc = Runtime.getRuntime().exec(cmdstr, null, null);
try {
@@ -712,90 +712,6 @@
closeit(fos);
}
}
-
- // TextEmitter and associated methods are used to create
- // files useful for debugging only, so errors are ignored.
-
- /**
- * A source of text data.
- */
- public interface TextEmitter {
- void emit(Writer writer)
- throws IOException;
- }
-
- /**
- * Emit a file relative to the work directory,
- * using the TextEmitter as a source for the file text.
- */
- public static void emitFile(String filename, TextEmitter tw) {
- FileWriter writer = null;
- try {
- File f = new File(filename);
- if (!f.isAbsolute()) {
- throw new IllegalArgumentException("emitFile: file name cannot be relative");
- }
- f.delete();
- writer = new FileWriter(f);
- tw.emit(writer);
- writer.close();
- writer = null;
- }
- catch (IOException ioe) {
- System.err.println("Cannot write to " + filename);
- if (writer != null) {
- try {
- writer.close();
- }
- catch (IOException ioe2) {
- // ignored.
- }
- }
- }
- }
-
- /**
- * emit a file relative to the work directory with the given String text
- */
- public static void emitFile(String filename, final String txt) {
- emitFile(filename, new TextEmitter() {
- public void emit(Writer writer)
- throws IOException {
- writer.write(txt);
- }
- });
- }
-
- /**
- * emit a file relative to the work directory with the given node
- * (to be dumped) as the text.
- */
- public static void emitFile(String filename, final SimpleNode node) {
- emitFile(filename, new TextEmitter() {
- public void emit(Writer writer)
- throws IOException {
- nodeFileDump(writer, "", node);
- }
- });
- }
-
- /**
- * Helper method to dump a node into a file.
- */
- public static void nodeFileDump(Writer writer, String prefix, SimpleNode node)
- throws IOException
- {
- writer.write(node.toString(prefix) + "\n");
- SimpleNode[] children = node.getChildren();
- if (children != null) {
- for (int i = 0; i < children.length; ++i) {
- SimpleNode n = (SimpleNode)children[i];
- if (n != null) {
- nodeFileDump(writer, prefix + " ", n);
- }
- }
- }
- }
}
/**
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9Generator.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9Generator.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9Generator.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -34,12 +34,6 @@
public static final String PASSTHROUGH_TOPLEVEL = "toplevel";
- // TODO: [2007-12-12 dda] make DEBUG_OUTPUT a compiler option.
- /**
- * When set, some extra debugging files are saved.
- */
- public static final boolean DEBUG_OUTPUT = true;
-
/** The user 'main' class, which extends LFCApplication */
public final static String MAIN_APP_CLASSNAME = "LzApplication";
@@ -642,6 +636,7 @@
{
boolean buildSharedLibrary = options.getBoolean(Compiler.BUILD_SHARED_LIBRARY);
boolean debugEval = options.getBoolean(Compiler.DEBUG_EVAL);
+ boolean trackLines = options.getBoolean(Compiler.TRACK_LINES);
String mainClass;
if (buildSharedLibrary) {
@@ -650,7 +645,7 @@
mainClass = debugEval ? SWF9Generator.DEBUG_EVAL_CLASSNAME : SWF9Generator.MAIN_APP_CLASSNAME;
}
- return (new SWF9ParseTreePrinter(compress, obfuscate, mainClass, buildSharedLibrary)).makeTranslationUnits(translatedNode);
+ return (new SWF9ParseTreePrinter(compress, obfuscate, mainClass, buildSharedLibrary, trackLines)).makeTranslationUnits(translatedNode);
}
/** Implements CodeGenerator.
@@ -662,12 +657,6 @@
boolean buildSharedLibrary = options.getBoolean(Compiler.BUILD_SHARED_LIBRARY);
SWF9External ex = new SWF9External(options);
- if (DEBUG_OUTPUT) {
- ex.emitFile(ex.workDirectoryName("source.txt"), savedSource);
- ex.emitFile(ex.workDirectoryName("program.txt"), (new SWF9ParseTreePrinter()).text(savedProgram));
- ex.emitFile(ex.workDirectoryName("progdump.txt"), savedProgram);
- }
-
for (Iterator iter = tunits.iterator(); iter.hasNext(); ) {
TranslationUnit tunit = (TranslationUnit)iter.next();
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9ParseTreePrinter.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9ParseTreePrinter.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/SWF9ParseTreePrinter.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -66,20 +66,20 @@
}
public SWF9ParseTreePrinter() {
- this(false, false, null, false);
+ this(false, false, null, false, false);
}
public SWF9ParseTreePrinter(boolean compress) {
- this(compress, false, null, false);
+ this(compress, false, null, false, false);
}
public SWF9ParseTreePrinter(boolean compress, boolean obfuscate) {
- this(compress, obfuscate, null, false);
+ this(compress, obfuscate, null, false, false);
}
- public SWF9ParseTreePrinter(boolean compress, boolean obfuscate, String mainClassName, boolean sharedLibrary) {
+ public SWF9ParseTreePrinter(boolean compress, boolean obfuscate, String mainClassName, boolean sharedLibrary, boolean trackLines) {
// never compress or obfuscate
- super(false, false);
+ super(false, false, trackLines);
this.mainClassName = mainClassName;
this.islib = sharedLibrary;
}
Modified: openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/TranslationUnit.java
===================================================================
--- openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/TranslationUnit.java 2008-04-16 15:43:30 UTC (rev 8707)
+++ openlaszlo/trunk/WEB-INF/lps/server/src/org/openlaszlo/sc/TranslationUnit.java 2008-04-16 15:57:15 UTC (rev 8708)
@@ -93,6 +93,10 @@
linenum += countOccurence(s, '\n');
}
+ public int getTextLineNumber() {
+ return linenum;
+ }
+
public void addInsertStreamMarker(int streamNum) {
text.append(INSERT_STREAM_MARK + streamNum + INSERT_END_MARK);
if (streamNum > maxInserts)
More information about the Laszlo-checkins
mailing list