Skip to content

Commit 76f3cf2

Browse files
TeamworkGuy2TeamworkGuy2
committed
0.10.0 - Added C# and Java enum parsing (see new class field 'initializer'/'initializerExpression' JSON field), major refactoring (package and class moving/renaming), added unit tests.
1 parent a89e38e commit 76f3cf2

File tree

99 files changed

+1830
-973
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

99 files changed

+1830
-973
lines changed

‎.classpath

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jstreamish/bin/jstreamish.jar" sourcepath="/JStreamish"/>
3030
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtext-parser/bin/jtext_parser.jar" sourcepath="/JTextParser"/>
3131
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtext-template/bin/jtext_template.jar" sourcepath="/JTextTemplate"/>
32-
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtext-util/bin/jtext_util.jar" sourcepath="C:/Users/TeamworkGuy2/Documents/Java/Projects/JTextParser/src"/>
32+
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtext-util/bin/jtext_util.jar" sourcepath="/JTextUtil"/>
3333
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtree-walker/bin/jtree_walker.jar" sourcepath="/JTreeWalker"/>
3434
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtuples/bin/jtuples.jar" sourcepath="/JTuples"/>
3535
<classpathentry kind="lib" path="C:/Users/TeamworkGuy2/Documents/Java/Libraries/jtwg2-logging/bin/jtwg2_logging.jar" sourcepath="/JTwg2Logging"/>

‎CHANGELOG.md

+30-2
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,36 @@ This project does its best to adhere to [Semantic Versioning](http://semver.org/
44

55

66
--------
7-
###[0.10.8](N/A) - 2016-09-02
8-
#### Changed
7+
###[0.11.0](N/A) - 2016-09-05
8+
#### Added
9+
* __basic C# and Java enum parsing__
10+
* Added twg2.ast.interm.field FieldDef and FieldDefResolved to represent enum members (TODO could use some clarification/refactoring)
11+
* Added CsEnumMemberExtractor and JavaEnumMemberExtractor
12+
* Moved duplicate code from class that implemented AstParser into new AstParserReusableBase and AstMemberInClassParserReusable abstract classes (I know the names are a little awkward, suggestions are welcome)
13+
* Added C# and Java unit tests for enum parsing
14+
* Moved duplicate source code parsing logic for tests into new CodeFileAndAst class
15+
16+
#### Changed
17+
* moved and renamed several packages and class names
18+
* twg2.parser.baseAst and sub-packages split and moved to twg2.parser.codeParser and sub-packages
19+
* twg2.parser.baseAst.tools -> twg2.parser.fragment
20+
* twg2.parser.baseAst.CompoundBlock -> twg2.parser.codeParser.BlockType
21+
* CsClassParser -> CsClassTokenizer
22+
* JavaClassParser -> JavaClassTokenizer
23+
* twg2.parser.codeParser.parsers -> twg2.parser.tokenizers and *Parser class name prefix changed to *Tokenizer
24+
* AccessModifierExtractor renamed readAccessModifier() -> parseAccessModifier() and readAccessModifierFromIter() -> readAccessModifiers()
25+
* CsBlock and JavaBlock renamed fromKeyword() -> parseKeyword() and tryFromKeyword() -> tryParseKeyword()
26+
* TypeSig renamed nested classes:
27+
* Simple -> TypeSigSimple
28+
* SimpleBaseImpl -> TypeSigSimpleBase
29+
* SimpleGenericImpl -> TypeSigSimpleGeneric
30+
* ResolvedBaseImpl -> TypeSigResolvedBase
31+
* ResolvedGenericImpl -> TypeSigResolvedGeneric
32+
33+
34+
--------
35+
###[0.10.8](https://github.com/TeamworkGuy2/JParseCode/commit/a89e38e23341bb999ec136fc61212c8271ad4332) - 2016-09-02
36+
#### Changed
937
* Updated dependency, switched jparser-data-type-like (now deprecated/removed) to jparse-primitive which is a separate project containing just the primitive parsing code from jparser-data-type-like
1038
* Renamed project from JParserTools -> JParseCode
1139
* Moved plugin-js/ -> plugin/node-js/

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
JParseCode
22
==============
3-
version: 0.10.8
3+
version: 0.11.0
44

55
In progress C#/Java/TypeScript parser tools built atop [JTextParser] (https://github.com/TeamworkGuy2/JTextParser), [Jackson] (https://github.com/FasterXML/jackson-core/) (core, databind, annotations) and half a dozen other utility libraries.
66

‎bin/jparse_code-with-tests.jar

25.5 KB
Binary file not shown.

‎bin/jparse_code.jar

24.8 KB
Binary file not shown.

‎package-lib.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"version" : "0.10.8",
2+
"version" : "0.11.0",
33
"name" : "jparse-code",
44
"description" : "An in-progress suite of parsing tools for C#, Java, and TypeScript source code",
55
"homepage" : "https://github.com/TeamworkGuy2/JParseCode",

‎src/twg2/ast/interm/annotation/AnnotationSig.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
import lombok.Getter;
99
import lombok.val;
1010
import twg2.annotations.Immutable;
11-
import twg2.parser.baseAst.tools.NameUtil;
11+
import twg2.parser.codeParser.tools.NameUtil;
1212
import twg2.parser.output.JsonWritableSig;
1313
import twg2.parser.output.WriteSettings;
1414

‎src/twg2/ast/interm/block/BlockAst.java

+2-2
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
import lombok.Getter;
44
import twg2.annotations.Immutable;
55
import twg2.ast.interm.classes.ClassSig;
6-
import twg2.parser.baseAst.CompoundBlock;
6+
import twg2.parser.codeParser.BlockType;
77
import twg2.parser.documentParser.CodeFragment;
88
import twg2.treeLike.simpleTree.SimpleTree;
99

@@ -12,7 +12,7 @@
1212
* @since 2015-12-5
1313
*/
1414
@Immutable
15-
public class BlockAst<T_BLOCK extends CompoundBlock> {
15+
public class BlockAst<T_BLOCK extends BlockType> {
1616
private final @Getter ClassSig.SimpleImpl declaration;
1717
private final @Getter SimpleTree<CodeFragment> blockTree;
1818
private final @Getter T_BLOCK blockType;

‎src/twg2/ast/interm/classes/ClassAst.java

+31-11
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,16 @@
66
import lombok.Getter;
77
import lombok.val;
88
import twg2.annotations.Immutable;
9+
import twg2.ast.interm.field.FieldDef;
10+
import twg2.ast.interm.field.FieldDefResolved;
911
import twg2.ast.interm.field.FieldSig;
1012
import twg2.ast.interm.field.FieldSigResolved;
1113
import twg2.ast.interm.method.MethodSig;
1214
import twg2.ast.interm.method.ParameterSig;
1315
import twg2.ast.interm.method.ParameterSigResolved;
1416
import twg2.io.write.JsonWrite;
15-
import twg2.parser.baseAst.CompoundBlock;
16-
import twg2.parser.baseAst.tools.NameUtil;
17+
import twg2.parser.codeParser.BlockType;
18+
import twg2.parser.codeParser.tools.NameUtil;
1719
import twg2.parser.output.JsonWritableSig;
1820
import twg2.parser.output.WriteSettings;
1921
import twg2.text.stringUtils.StringJoin;
@@ -22,7 +24,7 @@
2224
* @author TeamworkGuy2
2325
* @since 2015-12-4
2426
*/
25-
public interface ClassAst<T_SIG extends ClassSig, T_METHOD extends JsonWritableSig, T_BLOCK extends CompoundBlock> extends JsonWritableSig {
27+
public interface ClassAst<T_SIG extends ClassSig, T_METHOD extends JsonWritableSig, T_BLOCK extends BlockType> extends JsonWritableSig {
2628

2729
public T_SIG getSignature();
2830

@@ -40,23 +42,31 @@ public interface ClassAst<T_SIG extends ClassSig, T_METHOD extends JsonWritableS
4042
* @since 2015-12-4
4143
*/
4244
@Immutable
43-
public static class Impl<T_SIG extends ClassSig, T_FIELD extends JsonWritableSig, T_METHOD extends JsonWritableSig, T_PARAM extends JsonWritableSig, T_BLOCK extends CompoundBlock>
45+
public static class Impl<T_SIG extends ClassSig, T_ENUM extends JsonWritableSig, T_FIELD extends JsonWritableSig, T_METHOD extends JsonWritableSig, T_PARAM extends JsonWritableSig, T_BLOCK extends BlockType>
4446
implements ClassAst<T_SIG, T_METHOD, T_BLOCK> {
4547
private final @Getter T_SIG signature;
4648
private final @Getter List<List<String>> usingStatements;
49+
private final @Getter List<T_ENUM> enumMembers;
4750
private final @Getter List<T_FIELD> fields;
4851
private final @Getter List<T_METHOD> methods;
4952
private final @Getter T_BLOCK blockType;
5053

5154

52-
public Impl(T_SIG signature, List<List<String>> usingStatements, List<? extends T_FIELD> fields, List<? extends T_METHOD> methods, T_BLOCK blockType) {
55+
public Impl(T_SIG signature, List<List<String>> usingStatements, List<? extends T_FIELD> fields, List<? extends T_METHOD> methods, List<? extends T_ENUM> enums, T_BLOCK blockType) {
5356
@SuppressWarnings("unchecked")
5457
val fieldsCast = (List<T_FIELD>)fields;
5558
@SuppressWarnings("unchecked")
5659
val methodsCast = (List<T_METHOD>)methods;
60+
@SuppressWarnings("unchecked")
61+
val enumsCast = (List<T_ENUM>)enums;
62+
63+
if(enumsCast != null && enumsCast.size() > 0) {
64+
System.out.println();
65+
}
5766

5867
this.signature = signature;
5968
this.usingStatements = usingStatements;
69+
this.enumMembers = enumsCast;
6070
this.fields = fieldsCast;
6171
this.methods = methodsCast;
6272
this.blockType = blockType;
@@ -77,6 +87,16 @@ public void toJson(Appendable dst, WriteSettings st) throws IOException {
7787
JsonWrite.joinStr(usingStatements, ", ", dst, (us) -> '"' + NameUtil.joinFqName(us) + '"');
7888
dst.append("],\n");
7989

90+
if(enumMembers != null && blockType.isEnum()) {
91+
dst.append("\t\"enumMembers\": [");
92+
if(enumMembers.size() > 0) {
93+
dst.append("\n\t\t");
94+
JsonWrite.joinStrConsume(enumMembers, ",\n\t\t", dst, (f) -> f.toJson(dst, st));
95+
dst.append("\n\t");
96+
}
97+
dst.append("],\n");
98+
}
99+
80100
dst.append("\t\"fields\": [");
81101
if(fields.size() > 0) {
82102
dst.append("\n\t\t");
@@ -112,11 +132,11 @@ public String toString() {
112132
* @since 2016-1-2
113133
*/
114134
@Immutable
115-
public static class SimpleImpl<T_BLOCK extends CompoundBlock> extends Impl<ClassSig.SimpleImpl, FieldSig, MethodSig.SimpleImpl, ParameterSig, T_BLOCK> {
135+
public static class SimpleImpl<T_BLOCK extends BlockType> extends Impl<ClassSig.SimpleImpl, FieldDef, FieldSig, MethodSig.SimpleImpl, ParameterSig, T_BLOCK> {
116136

117137
public SimpleImpl(ClassSig.SimpleImpl signature, List<List<String>> usingStatements, List<? extends FieldSig> fields,
118-
List<? extends MethodSig.SimpleImpl> methods, T_BLOCK blockType) {
119-
super(signature, usingStatements, fields, methods, blockType);
138+
List<? extends MethodSig.SimpleImpl> methods, List<? extends FieldDef> enums, T_BLOCK blockType) {
139+
super(signature, usingStatements, fields, methods, enums, blockType);
120140
}
121141

122142
}
@@ -129,11 +149,11 @@ public SimpleImpl(ClassSig.SimpleImpl signature, List<List<String>> usingStateme
129149
* @since 2015-12-4
130150
*/
131151
@Immutable
132-
public static class ResolvedImpl<T_BLOCK extends CompoundBlock> extends Impl<ClassSig.ResolvedImpl, FieldSigResolved, MethodSig.ResolvedImpl, ParameterSigResolved, T_BLOCK> {
152+
public static class ResolvedImpl<T_BLOCK extends BlockType> extends Impl<ClassSig.ResolvedImpl, FieldDefResolved, FieldSigResolved, MethodSig.ResolvedImpl, ParameterSigResolved, T_BLOCK> {
133153

134154
public ResolvedImpl(ClassSig.ResolvedImpl signature, List<List<String>> usingStatements, List<? extends FieldSigResolved> fields,
135-
List<? extends MethodSig.ResolvedImpl> methods, T_BLOCK blockType) {
136-
super(signature, usingStatements, fields, methods, blockType);
155+
List<? extends MethodSig.ResolvedImpl> methods, List<? extends FieldDefResolved> enums, T_BLOCK blockType) {
156+
super(signature, usingStatements, fields, methods, enums, blockType);
137157
}
138158

139159
}

‎src/twg2/ast/interm/classes/ClassSig.java

+6-6
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@
88
import twg2.annotations.Immutable;
99
import twg2.ast.interm.type.TypeSig;
1010
import twg2.io.write.JsonWrite;
11-
import twg2.parser.baseAst.AccessModifier;
12-
import twg2.parser.baseAst.tools.NameUtil;
11+
import twg2.parser.codeParser.AccessModifier;
12+
import twg2.parser.codeParser.tools.NameUtil;
1313
import twg2.parser.output.JsonWritableSig;
1414
import twg2.parser.output.WriteSettings;
1515

@@ -35,7 +35,7 @@ public interface ClassSig extends JsonWritableSig {
3535
public static class SimpleImpl implements ClassSig {
3636
private final @Getter List<String> fullName;
3737
/** This type's generic type parameters, if any */
38-
private final @Getter List<TypeSig.Simple> params;
38+
private final @Getter List<TypeSig.TypeSigSimple> params;
3939
/** The block's type (i.e. 'interface', 'class', 'enum', etc.) */
4040
private final @Getter AccessModifier accessModifier;
4141
private final @Getter String declarationType;
@@ -101,12 +101,12 @@ public String toString() {
101101
public static class ResolvedImpl implements ClassSig {
102102
private final @Getter List<String> fullName;
103103
/** This type's generic type parameters, if any */
104-
private final @Getter List<TypeSig.Resolved> params;
104+
private final @Getter List<TypeSig.TypeSigResolved> params;
105105
private final @Getter AccessModifier accessModifier;
106106
/** The block's type (i.e. 'interface', 'class', 'enum', etc.) */
107107
private final @Getter String declarationType;
108-
private final @Getter TypeSig.Resolved extendClass;
109-
private final @Getter List<TypeSig.Resolved> implementInterfaces;
108+
private final @Getter TypeSig.TypeSigResolved extendClass;
109+
private final @Getter List<TypeSig.TypeSigResolved> implementInterfaces;
110110

111111

112112
@Override

‎src/twg2/ast/interm/field/FieldDef.java

+103
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
package twg2.ast.interm.field;
2+
3+
import java.io.IOException;
4+
import java.util.List;
5+
6+
import lombok.Getter;
7+
import twg2.annotations.Immutable;
8+
import twg2.ast.interm.annotation.AnnotationSig;
9+
import twg2.ast.interm.type.TypeSig.TypeSigSimple;
10+
import twg2.io.write.JsonWrite;
11+
import twg2.parser.codeParser.AccessModifier;
12+
import twg2.parser.codeParser.extractors.DataTypeExtractor;
13+
import twg2.parser.codeParser.tools.NameUtil;
14+
import twg2.parser.documentParser.CodeFragment;
15+
import twg2.parser.fragment.CodeFragmentType;
16+
import twg2.parser.output.WriteSettings;
17+
import twg2.text.stringEscape.StringEscapeJson;
18+
import twg2.treeLike.simpleTree.SimpleTree;
19+
20+
/** A {@link FieldSig} with an initializer
21+
* @author TeamworkGuy2
22+
* @since 2016-09-04
23+
*/
24+
@Immutable
25+
public class FieldDef extends FieldSig {
26+
private final @Getter SimpleTree<CodeFragment> initializer;
27+
28+
29+
public FieldDef(String name, List<String> fullName, TypeSigSimple fieldType, List<AccessModifier> accessModifiers,
30+
List<AnnotationSig> annotations, List<String> comments, SimpleTree<CodeFragment> initializer) {
31+
super(name, fullName, fieldType, accessModifiers, annotations, comments);
32+
this.initializer = initializer;
33+
}
34+
35+
36+
@Override
37+
public void toJson(Appendable dst, WriteSettings st) throws IOException {
38+
dst.append("{ ");
39+
dst.append("\"name\": \"" + (st.fullFieldName ? NameUtil.joinFqName(fullName) : fullName.get(fullName.size() - 1)) + "\", ");
40+
41+
dst.append("\"type\": ");
42+
fieldType.toJson(dst, st);
43+
dst.append(", ");
44+
45+
dst.append("\"accessModifiers\": [");
46+
JsonWrite.joinStr(accessModifiers, ", ", dst, (acs) -> '"' + acs.toSrc() + '"');
47+
dst.append("], ");
48+
49+
dst.append("\"annotations\": [");
50+
JsonWrite.joinStrConsume(annotations, ", ", dst, (ann) -> ann.toJson(dst, st));
51+
dst.append("], ");
52+
53+
initializerToJson(initializer, dst, st);
54+
55+
dst.append("\"comments\": [");
56+
JsonWrite.joinStrConsume(comments, ", ", dst, (str) -> { dst.append('"'); dst.append(StringEscapeJson.toJsonString(str)); dst.append('"'); });
57+
dst.append("]");
58+
59+
dst.append(" }");
60+
}
61+
62+
63+
@Override
64+
public String toString() {
65+
return fieldType + " " + NameUtil.joinFqName(fullName);
66+
}
67+
68+
69+
/** Write a field initializer to a JSON field named 'initializer' if the value is a number, boolean, string, or null literal, else write it to a field named 'initializerExpression'
70+
* @throws IOException
71+
*/
72+
public static void initializerToJson(SimpleTree<CodeFragment> astNode, Appendable dst, WriteSettings st) throws IOException {
73+
CodeFragment data = null;
74+
boolean isNumOrBoolOrNull = false;
75+
if(astNode != null && !astNode.hasChildren() && (data = astNode.getData()) != null &&
76+
(data.getFragmentType() == CodeFragmentType.STRING ||
77+
(isNumOrBoolOrNull = (data.getFragmentType() == CodeFragmentType.NUMBER || DataTypeExtractor.isBooleanLiteral(data) || DataTypeExtractor.isNullLiteral(data))))) {
78+
dst.append("\"initializer\": ");
79+
if(isNumOrBoolOrNull) {
80+
dst.append(data.getText());
81+
}
82+
else {
83+
dst.append('"');
84+
dst.append(StringEscapeJson.toJsonString(data.getText()));
85+
dst.append('"');
86+
}
87+
dst.append(", ");
88+
}
89+
else if(astNode != null && astNode.hasChildren()) {
90+
dst.append("\"initializerExpression\": ");
91+
if((data = astNode.getData()) != null && !DataTypeExtractor.isNullLiteral(data)) {
92+
dst.append('"');
93+
dst.append(StringEscapeJson.toJsonString(data.getText()));
94+
dst.append('"');
95+
}
96+
else {
97+
dst.append(null);
98+
}
99+
dst.append(", ");
100+
}
101+
}
102+
103+
}

0 commit comments

Comments
 (0)