0.12.0 - Major refactoring, updated jtext-parser to latest version, switched from TextIteratorParser (previously TextParserImpl) to TextCharsParser and files are now ready and passed through the parsing workflow as char[] with offset and length instead of as a String. Also more clearly named classes to identify the two major parsing phases: tokenization and parsing/extracting. Moved/renamed a lot of classes/packages. Added more generic tokenization via CodeTokenizerBuilder.

Author: TeamworkGuy2

.classpath

@@ -9,7 +9,6 @@
 	<classpathentry kind="con" path="org.eclipse.jdt.USER_LIBRARY/Lombok"/>
 	<classpathentry kind="con" path="org.eclipse.jdt.USER_LIBRARY/TemplateUtil"/>
 	<classpathentry kind="con" path="org.eclipse.jdt.USER_LIBRARY/TestChecks"/>
-	<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/data-transfer/bin/data_transfer.jar"/>
 	<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jackson/jar/jackson-annotations-2.5.0.jar"/>
 	<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jackson/jar/jackson-core-2.5.0.jar"/>
 	<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jackson/jar/jackson-databind-2.5.4.jar"/>

CHANGELOG.md

@@ -4,7 +4,25 @@ This project does its best to adhere to [Semantic Versioning](http://semver.org/
 
 
 --------
-###[0.11.0](https://github.com/TeamworkGuy2/JParseCode/commit/76f3cf2eba9327659f6e454d8bfe3208695cdab5) - 2016-09-05
+###[0.12.0](N/A) - 2016-09-13
+#### Added
+* PerformanceTrackers, ParseTimes, TokenizeStepDetails in new twg2.parser.codeParser.tools.performance package - used for tracking performance
+
+#### Changed
+* biggest change is switching from jtext-parser's TextIteratorParser (previously: TextParserImpl) to TextCharsParser and files are read as char[] and stored in ParseInput and CodeFileSrc as char[] with offset and length, this will hopefully provide a small performance boost since each file's contents is copied one less time (no more new String(...) copy) and TextCharsParser is designed to take a char[] without any data copying
+* second large change is moving the parsing process toward a clearly defined two step process, the first step is called 'tokenization' and the second is called 'parsing/extracting'
+  * file tokenization logic has been split up.  Cs and Java FileTokenizer classes now return CodeTokenizerBuilder instances and CodeTokenizerBuilder contains all the generic logic for running the tokenization process
+* updated to new latest dependencies, especially jtext-parser
+* moved CodeFragment, DocumentFragment, and DocumentFragmentText from package twg2.parser.documentParser -> twg2.parser.fragment
+* moved CodeFileParsed, CodeFileSource, ParseInput, and ParserWorkflow from package twg2.parser.codeParser -> twg2.parser.workflow
+* moved/renamed twg2.parser.documentParser.DocumentParser -> twg2.parser.tokenizers.CodeTokenizerBuilder
+* moved IsParentChild from package twg2.parser.documentParser -> twg2.parser.tokenizers
+* CommentAndWhitespaceExtractor now drops the last trailing newlines from the comment strings
+* updated a number of unit tests
+
+
+--------
+###[0.11.0](https://github.com/TeamworkGuy2/JParseCode/commit/3029d0d08bda6cc308d3732eb09eb971fd0e6030) - 2016-09-06
 #### Added
 * __basic C# and Java enum parsing__
   * Added twg2.ast.interm.field FieldDef and FieldDefResolved to represent enum members (TODO could use some clarification/refactoring)

README.md

@@ -1,14 +1,14 @@
 JParseCode
 ==============
-version: 0.11.0
+version: 0.12.0
 
 In progress C#/Java/TypeScript parser tools built atop [JTextParser] (https://github.com/TeamworkGuy2/JTextParser), [Jackson] (https://github.com/FasterXML/jackson-core/) (core, databind, annotations) and half a dozen other utility libraries. 
 
-####Goals:
+### Goals:
 * A competent source code parser that can turn C#, Java, or JavaScript/TypeScript code into a simple AST like structure ('competent' meaning this project aims to support common use cases, not every syntatic feature of the supported languages). 
 * A code first parser aimed at manipulating the resulting AST and writing it back as source code or JSON.  With the goal of allowing simple language constructs like interfaces and data models to be transpiled to different languages. 
 
-####Not Goals: 
+### Not Goals:
 * NOT to create another compiler for C#, Java, or JS/TS. This project's parser expects valid code as input, the few error messages that are present are NOT design to highlight syntax errors in the input. 
 * NOT to create a valid AST for each supported language. Rather an AST like structure that supports the lowest common denominator between the targeted languages. This means many AST compromises are inevitable due to differences in language specs. 
 
@@ -179,7 +179,9 @@ JSON Result (printed to System.out):
 
 
 --------
-Command Line Interface (CLI):
+
+### Command Line Interface (CLI)
+
 A command line call looks like:
 ```
 path/to/java -jar path/to/jparse-code.jar 

bin/jparse_code-with-tests.jar

@@ -1,12 +1,11 @@
 {
-	"version" : "0.11.0",
+	"version" : "0.12.0",
 	"name" : "jparse-code",
 	"description" : "An in-progress suite of parsing tools for C#, Java, and TypeScript source code",
 	"homepage" : "https://github.com/TeamworkGuy2/JParseCode",
 	"license" : "MIT",
 	"main" : "./bin/jparse_code.jar",
 	"dependencies" : {
-		"data-transfer": "*",
 		"jackson-annotations": "~2.5.0",
 		"jackson-core": "~2.5.0",
 		"jackson-databind": "~2.5.0",

bin/jparse_code.jar

@@ -4,7 +4,7 @@
 import twg2.annotations.Immutable;
 import twg2.ast.interm.classes.ClassSig;
 import twg2.parser.codeParser.BlockType;
-import twg2.parser.documentParser.CodeFragment;
+import twg2.parser.fragment.CodeFragment;
 import twg2.treeLike.simpleTree.SimpleTree;
 
 /**

package-lib.json

@@ -60,10 +60,6 @@ public Impl(T_SIG signature, List<List<String>> usingStatements, List<? extends
 			@SuppressWarnings("unchecked")
 			val enumsCast = (List<T_ENUM>)enums;
 
-			if(enumsCast != null && enumsCast.size() > 0) {
-				System.out.println();
-			}
-
 			this.signature = signature;
 			this.usingStatements = usingStatements;
 			this.enumMembers = enumsCast;

src/twg2/ast/interm/block/BlockAst.java

@@ -11,7 +11,7 @@
 import twg2.parser.codeParser.AccessModifier;
 import twg2.parser.codeParser.extractors.DataTypeExtractor;
 import twg2.parser.codeParser.tools.NameUtil;
-import twg2.parser.documentParser.CodeFragment;
+import twg2.parser.fragment.CodeFragment;
 import twg2.parser.fragment.CodeFragmentType;
 import twg2.parser.output.WriteSettings;
 import twg2.text.stringEscape.StringEscapeJson;

src/twg2/ast/interm/classes/ClassAst.java

@@ -10,7 +10,7 @@
 import twg2.io.write.JsonWrite;
 import twg2.parser.codeParser.AccessModifier;
 import twg2.parser.codeParser.tools.NameUtil;
-import twg2.parser.documentParser.CodeFragment;
+import twg2.parser.fragment.CodeFragment;
 import twg2.parser.output.WriteSettings;
 import twg2.text.stringEscape.StringEscapeJson;
 import twg2.treeLike.simpleTree.SimpleTree;