0.10.1 - Merged DocumentFragmentRef with DocumentFragmentText. Added some documentation and fixed some warnings. Fixed annotation named parameter parsing and C# property access modifier parsing bug.

Author: TeamworkGuy2

versions.md CHANGELOG.md

@@ -1,47 +1,56 @@
---------
-####0.10.0
-date: 2016-4-13
+# Change Log
+All notable changes to this project will be documented in this file.
+This project does its best to adhere to [Semantic Versioning](http://semver.org/).
 
-commit: ?
 
-* Fixed minor bug in annotation named parameter parsing when annotation only had one parameter
+--------
+###[0.10.1](N/A) - 2016-05-13
+####Changed
+* Merged DocumentFragmentRef with DocumentFragmentText
+* Added some documentation
+* Fixed some code warnings
+* Moved full class parsing tests to twg2.parser.codeParser.test
 
+####Fixed
+* Bug in annotation named parameter parsing when annotation only had one parameter
+* Bug in C# property parsing not supporting field modifiers (i.e. 'private', 'protected', ...)
 
---------
-####0.10.0
-date: 2016-4-12
+####Removed
+* DocumentFragmentRef (merged with DocumentFragmentText)
 
-commit: 3e8a324ccada6af273339e6f29ae569795e3abcd
 
+--------
+###[0.10.0](https://github.com/TeamworkGuy2/JParserTools/commit/3e8a324ccada6af273339e6f29ae569795e3abcd) - 2016-04-12
+####Added
 * Added better annotation parsing, including support for negative numbers as arguments
-* Fixed ParserWorkflow to generate and group all results by destination file before writing (previously a writer was opened in overwrite mode for each destination group, thereby overwriting data written to the same file by a previous destination group during the same program execution)
 * Added CodeFragment which extends 'DocumentFragmentText<CodeFragmentType>' so don't have to keep typing that every time, updated most of the code to use CodeFragment
 * Added OperatorUtil and Operator (with C# and Java implementation enums) similar to the existing Keyword enums
+
+####Changed
 * Refactored how we use EnumSubSet and enum sub-categorization
    * Added CodeFragmentEnumSubSet with is() and parse() methods which accept CodeFragment nodes (we were starting to duplicate this parsing code in Keyword and the new Operator class, so moved it to a reusable class)
    * Removed KeywordUtil isXyzKeyword() and parseXyzKeyword() methods in favor of methods that return CodeFragmentEnumSubSet instances for each of the keyword categories (i.e. 'blockModifiers()' or 'operators()')
 * Moved twg2.parser.codeParser extractor classes (i.e. AccessModifierExtractor or BlockExtractor) to new twg2.parser.codeParser.extractors package
 
+####Fixed
+* ParserWorkflow now generates and groups all results by destination file before writing (previously a writer was opened in overwrite mode for each destination group, thereby overwriting data written to the same file by a previous destination group during the same program execution)
 
---------
-####0.9.0
-date: 2016-3-20
-
-commit: 679778bafd13a413854bd169cabe747b12bbc894
 
+--------
+###[0.9.0](https://github.com/TeamworkGuy2/JParserTools/commit/679778bafd13a413854bd169cabe747b12bbc894) - 2016-03-20
+####Added
 * Added commented parsing for comments attached to methods and fields (future TODO: add comment parsing for comments attached to classes and namespaces)
+
+####Changed
 * Renamed intermAst packages to 'twg2.ast.interm'
-* Removed 'interm' from most AST class names
+* Renamed most AST classes, removed 'interm' from the name
 * Moved type resolution out of AST classes into new 'twg2.parser.resolver' classes (i.e. ClassSigResolver, FieldSigResolver, etc.)
 * Created 'twg2.parser.language' package for code language management classes
 
 
 --------
-####0.8.0
-date: 2016-2-28
-
-commit: 32ee2a5ec5c218d3f90d1438f893a86e34b9c716
-
+###[0.8.0](https://github.com/TeamworkGuy2/JParserTools/commit/32ee2a5ec5c218d3f90d1438f893a86e34b9c716) - 2016-02-28
+####Changed
 Move from assuming that conditions can list the initial chars that match them (CharParser.WithMarks.getMatchFirstChars()) to CharParserMatchable and new getFirstCharMatcher() method which allows for a flexible definition of matching first chars
 * Moved/renamed ParserWorkFlow SourceInfo and LoadResult \(renamed to SourceFiles) nested classes and ParserMain.getFilesByExtension() to [JFileIo] (https://github.com/TeamworkGuy2/JFileIo) library
 * Moved twg2.parser.output JsonWrite and JsonWritable to JFileIO project's twg2.io.write package
@@ -51,87 +60,83 @@ Move from assuming that conditions can list the initial chars that match them (C
 
 
 --------
-####0.7.0
-date: 2016-2-24
-
-commit: 218036c37673615e6bced0eecfb8a9b7d6eb7808
-
+###[0.7.0](https://github.com/TeamworkGuy2/JParserTools/commit/218036c37673615e6bced0eecfb8a9b7d6eb7808) - 2016-02-24
+####Changed
 * Updated to latest version of JTextParser and JStreamish
 * Switched from StringLineSupplier for reading lines from a source string to CharLineSupplier (slightly less garbage generated due to less conversion between strings and char arrays)
-* Fixed toJson() not formatting generic types correctly
 * By default, annotation arguments map is include in toJson() output even if empty
 * ITrackSearchService.cs test file was using '\r' for newlines, replaced with '\n'
 * Moved twg2.parser.test package to separate test directory
 
+####Fixed
+* Fixed toJson() not formatting generic types correctly
 
---------
-####0.6.0
-date: 2016-2-21
-
-commit: 5ae0793feb0475654bbdf835ef5d350e91cdd438
 
+--------
+###[0.6.0](https://github.com/TeamworkGuy2/JParserTools/commit/5ae0793feb0475654bbdf835ef5d350e91cdd438) - 2016-02-21
+####Added
 * Added numeric literal parsing \(i.e. '23' or '1.5f')
 * Added field and method access modifier parsing \(i.e. public, static, synchronized, volatile)
-* Fields now writing their annotations when toJson\() is called
+
+####Changed
+* Fields now write their annotations when toJson\() is called
 * Changed JsonWrite method names to help differentiate their purposes
 * Update to use latest version of multiple libraries, including: JFileIo, JStreamish, JTextParser, and JParserDataTypeLike
 
 
 --------
-####0.5.0
-date: 2016-2-9
-
-commit: eea353c111f789b315ab5471661c6a305c0701d2
-
+###[0.5.0](https://github.com/TeamworkGuy2/JParserTools/commit/eea353c111f789b315ab5471661c6a305c0701d2) - 2016-02-09
+####Added
 * Added array type parsing, \(i.e. 'int\[]\[]')
+* Added some more tests
+
+#### Changed
 * Moved twg2.parser.codeParser.eclipseProject to another project
-* Added some more tests and moved none JUnit experiment files to 'miscellaneous' package
+* Moved none JUnit experiment files to 'miscellaneous' package
 
 
 --------
-####0.4.0
-date: 2016-1-16
-
-commit: ab23d86656221e6b1a540d7129446b08c808aca4
-
-* Fixed a regression in C# method signature parsing
-* Removed unused twg2.parser.documentParser.block package
+###[0.4.0](https://github.com/TeamworkGuy2/JParserTools/commit/ab23d86656221e6b1a540d7129446b08c808aca4) - 2016-01-16
+####Changed
 * Moved twg2.parser.condition.AstParser -> twg2.parser.baseAst.AstParser
 * Modified CodeLanguageOptions, so implementation class is a sub-class and CodeLanguageOptions contains only static fields and methods
 
+####Removed
+* Removed unused twg2.parser.documentParser.block package
 
---------
-####0.3.0
-date: 2016-1-16
+####Fixed
+* Fixed a regression in C# method signature parsing
 
-commit: 0b7128980ba31623d17f85d9f10bd4d99bd1288e
 
-* __Added Java parsing support and tests.__
+--------
+###[0.3.0](https://github.com/TeamworkGuy2/JParserTools/commit/0b7128980ba31623d17f85d9f10bd4d99bd1288e) - 2016-01-16
+####Added
+* __Java parsing support and tests.__
 * Added Keyword interface for generic language keyword operations such as isKeyword(), isBlockModifierKeyword(), isDataTypeKeyword(), etc.
+
+####Changed
 * Made interm parsing more generic, added a bunch of parser creator functions to AstExtractor.  Converted some of the C# interm parsers into more generic parsers to be used by C# and Java (see BaseFieldExtractor, BaseMethodExtractor, BaseMethodParametersParser)
 * Renamed getFullyQualifyingName() methods to getFullName(), (note: this may change again in future, possibly to getFqName())
 * Moved and renamed ParserCondition, Precondition -> ParserFactory, TokenParserCondition -> TokenParser, CharParserCondition -> CharParser, and ParserStartChars to the [JTextParser] (https://github.com/TeamworkGuy2/JTextParser) project
 * Simplified CharConditions and StringConditions sub-class names (i.e. StringConditions.StringStartFilter -> StringConditions.Start)
 
 
 --------
-####0.2.1
-date: 2016-1-13
-
-commit: a33f37ad6a116e7e697498af88327dfaa46709a0
-
-* Added simple C# parse example with resulting JSON.  Added and refactored some test cases.
+###[0.2.1](https://github.com/TeamworkGuy2/JParserTools/commit/a33f37ad6a116e7e697498af88327dfaa46709a0) - 2016-01-13
+####Added
+* Simple C# parse example with resulting JSON.  Added and refactored some test cases.
 
 
 --------
-####0.2.0:
-date: 2016-1-12
-
-commit: 5b692bc3476ca94c6dedb5b6424d1319fcad2057
-
+###[0.2.0](https://github.com/TeamworkGuy2/JParserTools/commit/5b692bc3476ca94c6dedb5b6424d1319fcad2057) - 2016-01-12
+####Added
 * __Finished command line interface (CLI) argument parsing for ['sources', 'destinations', 'log'] and ParserWorkflow.__
+
+####Changed
 * Simplified CodeFileSrc generic signature. Added interfaces and setup entire parsing process to be generic (added CodeLanguage.getExtractor() and AstExtractor interface).
 * Refactored C# classes to support this more generic approach.
-* Removed IntermClass.getBlockTree(), it should be tracked higher up in the parsing process.
 * Added Simple and Resolved sub-classes of ProjectClassSet. Renamed CsMain -> ParserMain.
 * Added some additional EclipseClasspathFile/Utils methods for finding project dependencies.
+
+####Removed
+* Removed IntermClass.getBlockTree(), it should be tracked higher up in the parsing process.

src/twg2/parser/baseAst/csharp/CsAstUtil.java

@@ -125,22 +125,35 @@ public boolean isKeyword(CodeFragment node, CsKeyword keyword1, CsKeyword keywor
 	}
 
 
+	/** Supports property blocks in the format:
+	 * <pre><code>{
+	 *   [ [access-modifier] get ( ; | {...} ) ]
+	 *   [ [access-modifier] set ( ; | {...} ) ]
+	 *}</code></pre>
+	 */
 	@Override
 	public boolean isFieldBlock(SimpleTree<CodeFragment> block) {
 		if(block == null) { return true; }
 		val childs = block.getChildren();
-		// properties must have at-least one indexer
+		// properties must have at-least one indexer (i.e. 'get' or 'set')
 		if(childs.size() == 0) { return false; }
 
+		val keywords = this.getLanguage().getKeywordUtil();
+
 		boolean prevWasGetOrSet = false;
-		for(val child : childs) {
+		for(int i = 0, size = childs.size(); i < size; i++) {
+			val child = childs.get(i);
+			val nextChild = i < size - 1 ? childs.get(i + 1) : null;
 			val frag = child.getData();
 			val fragType = frag.getFragmentType();
 			if(fragType == CodeFragmentType.COMMENT) {
 				continue;
 			}
-			val isGetOrSet = fragType == CodeFragmentType.IDENTIFIER && ("get".equals(frag.getText()) || "set".equals(frag.getText()));
-			if(isGetOrSet || (prevWasGetOrSet && (fragType == CodeFragmentType.BLOCK || fragType == CodeFragmentType.SEPARATOR))) {
+			val isGetOrSet = isGetOrSet(frag);
+			val isAccessMod = keywords.fieldModifiers().is(frag);
+			if(isGetOrSet ||
+					(prevWasGetOrSet && (fragType == CodeFragmentType.BLOCK || fragType == CodeFragmentType.SEPARATOR)) ||
+					(isAccessMod && nextChild != null && isGetOrSet(nextChild.getData()))) {
 				// allow
 			}
 			else {
@@ -151,4 +164,9 @@ public boolean isFieldBlock(SimpleTree<CodeFragment> block) {
 		return true;
 	}
 
+
+	private static boolean isGetOrSet(CodeFragment frag) {
+		return frag.getFragmentType() == CodeFragmentType.IDENTIFIER && ("get".equals(frag.getText()) || "set".equals(frag.getText()));
+	}
+
 }

src/twg2/parser/codeParser/tools/EnumSplitter.java

@@ -5,7 +5,7 @@
 import java.util.function.Function;
 import java.util.function.Predicate;
 
-/**
+/** Static methods for splitting/extracting subsets of values from enums
  * @author TeamworkGuy2
  * @since 2016-4-9
  */

src/twg2/parser/codeParser/tools/EnumSubSet.java

@@ -7,7 +7,9 @@
 import java.util.function.Function;
 import java.util.function.Predicate;
 
-/** A class and builder for a list of enum values and names and searching those names
+import twg2.collections.util.ToStringUtil;
+
+/** A class and builder for a list of enum values and names and find() method to retrieve the value associated with a name
  * @author TeamworkGuy2
  * @since 2016-2-20
  */
@@ -49,7 +51,13 @@ public E find(String name) {
 	}
 
 
-	/** Given a list of names in original order (names) and sorted order (namesSorted) and a list of values (values), create a sorted values array containing 'values'
+	@Override
+	public String toString() {
+		return ToStringUtil.toStringKeyValuePairs(enumNames, enumValues, enumNames.length, null).toString();
+	}
+
+
+	/** Given a list of names in original order (names) and sorted order (namesSorted) and a list of values (values), return a sorted 'values' array
 	 * in the same order as (namesSorted) by using names.indexOf() to determine which index from (values) goes into each index of the returned array
 	 */
 	private static <G> G[] sortSecondListIntoArrayMatchingFirst(List<String> names, String[] namesSorted, List<G> values) {
@@ -84,13 +92,22 @@ private static <G> G[] sortSecondListIntoArrayMatchingFirst(List<String> names,
 
 
 
+	/** Build a subset of names and values from a enum
+	 * @author TeamworkGuy2
+	 * @since 2016-2-20
+	 * @param <F> this is the type of enum
+	 */
 	public static class Builder<F> {
 		List<String> enumNameList = new ArrayList<>();
 		List<F> enumValuesList = new ArrayList<>();
 		Predicate<F> filter;
 		Function<F, String> getName;
 
 
+		/** Create a build for a subset of enum names and values
+		 * @param filter predicate to determine if an enum instance matches 
+		 * @param getName used to transform each matching enum's name into the form that will be matched by {@link EnumSubSet#find(String)}
+		 */
 		public Builder(Predicate<F> filter, Function<F, String> getName) {
 			this.filter = filter;
 			this.getName = getName;
@@ -106,6 +123,7 @@ public boolean add(F enm) {
 			return false;
 		}
 
+
 		public EnumSubSet<F> build() {
 			String[] enumNames = enumNameList.toArray(new String[enumNameList.size()]);
 			Arrays.sort(enumNames);
@@ -114,6 +132,7 @@ public EnumSubSet<F> build() {
 
 			return new EnumSubSet<>(enumNames, enumValues);
 		}
+
 	}
 
 }

src/twg2/parser/codeParser/tools/TokenListIterable.java

@@ -9,17 +9,23 @@
 import twg2.streams.EnhancedListBuilderIterator;
 import twg2.treeLike.simpleTree.SimpleTree;
 
-/**
+/** Create an iterator for AST nodes, which skips {@link CodeFragmentType} COMMENT and STRING nodes.
  * @author TeamworkGuy2
  * @since 2016-1-1
  */
 public class TokenListIterable implements Iterable<SimpleTree<CodeFragment>> {
 
+	/** A {@link SimpleTree}{@code <}{@link CodeFragment}{@code >} {@link Supplier}.  When given an {@link Iterable}, this creates
+	 * a supplier which returns them in order, returns null once all the values have been iterated over
+	 * @author TeamworkGuy2
+	 * @since 2016-1-1
+	 */
 	static class TokenSupplier implements Supplier<SimpleTree<CodeFragment>> {
 		Iterator<SimpleTree<CodeFragment>> childIter;
 		Predicate<? super CodeFragment> cond;
 		int i;
 
+
 		public TokenSupplier(Iterable<? extends SimpleTree<? extends CodeFragment>> childs) {
 			this(childs, null);
 		}