In the previous article, Supports function expressions. With the correspondence of function expressions, the type handled as a value is only ʻInteger`, and the function expression is also treated as a value. In the same way, we will handle strings as values.
Check what you want to do with string support.
For example, there is the following program.
A string enclosed in " is a string literal.
When a character string is treated as a logical value under a condition such as an if statement, a character string of length 0 is false.
Strings can be concatenated with +.
The println () method call at the end of the program aims to output Hello world!.
var object = ""
if (!object) {
object = "world"
}
println("Hello " + object + "!")
We will consider how to implement it in the order of lexical analysis (Lexer), parser (Parser), and interpreter (Interpreter).
Since there is no function to parse the string literal enclosed in " , add it.
String literals can be treated in the same way as numeric literals, so we'll add definitions according to numeric literals.
We will make it possible to handle String as a value in the same way as ʻInteger. The processing of unary and binary operators, which only considered ʻInteger,
Change it so that it can also handle String.
Move on to implementation. About lexical analysis (Lexer), parser (Parser), interpreter (Interpreter) Let's take a look at the changes and additions in order.
Lexer.java
An implementation of Lexer.java.
Adds the ability to parse strings enclosed in " .
Add the ʻisStringStart ()method. " `Detects the start of the enclosure.
Lexer.java
private boolean isStringStart(char c) {
return c == '"';
}
Add the string () method.
" Paruses up to the end of the box into one token.
Implement by referring to the JSON string below. http://json.org/json-ja.html The reference implementation of 4 Hexadecimal digits has been truncated.
The kind that represents a string literal should be string.
Lexer.java
private Token string() throws Exception {
StringBuilder b = new StringBuilder();
next();
while (c() != '"') {
if (c() != '\\') {
b.append(next());
} else {
next();
char c = c();
if (c == '"') {
b.append('"');
next();
} else if (c == '\\') {
b.append('\\');
next();
} else if (c == '/') {
b.append('/');
next();
} else if (c == 'b') {
b.append('\b');
next();
} else if (c == 'f') {
b.append('\f');
next();
} else if (c == 'n') {
b.append('\n');
next();
} else if (c == 'r') {
b.append('\r');
next();
} else if (c == 't') {
b.append('\t');
next();
} else {
throw new Exception("string error");
}
}
}
next();
Token t = new Token();
t.kind = "string";
t.value = b.toString();
return t;
}
Modify the nextToken () method.
Add the call to the added method to // Add.
You can now decompose it into a token of a string literal.
Lexer.java
public Token nextToken() throws Exception {
skipSpace();
if (isEOT()) {
return null;
} else if (isSignStart(c())) {
return sign();
} else if (isDigitStart(c())) {
return digit();
// Add
} else if (isStringStart(c())) {
return string();
} else if (isIdentStart(c())) {
return ident();
} else if (isParenStart(c())) {
return paren();
} else if (isCurlyStart(c())) {
return curly();
} else if (isSymbolStart(c())) {
return symbol();
} else {
throw new Exception("Not a character for tokens");
}
}
Parser.java
An implementation of Parser.java.
Add a definition of how it works for the meaning of the token.
Added " string " to // Update.
String literals, like numeric literals and variable names,
Since the syntax cannot be further decomposed, it is added to the factor classification.
The factor parsing has already been implemented, so this is the end of the parser implementation.
Parser.java
public Parser() {
degrees = new HashMap<>();
degrees.put("(", 80);
degrees.put("*", 60);
degrees.put("/", 60);
degrees.put("+", 50);
degrees.put("-", 50);
degrees.put("==", 40);
degrees.put("!=", 40);
degrees.put("<", 40);
degrees.put("<=", 40);
degrees.put(">", 40);
degrees.put(">=", 40);
degrees.put("&&", 30);
degrees.put("||", 30);
degrees.put("=", 10);
// Update
factorKinds = Arrays.asList(new String[] { "digit", "ident", "string" });
binaryKinds = Arrays.asList(new String[] { "sign" });
rightAssocs = Arrays.asList(new String[] { "=" });
unaryOperators = Arrays.asList(new String[] { "+", "-", "!" });
reserved = Arrays.asList(new String[] { "function", "return", "if", "else", "while", "break", "var" });
}
Interpreter.java
An implementation of Interpreter.java.
ʻExpression () method change. It is a process that branches according to the meaning (kind) of the token that represents the expression. Added a branch under // Add for the token string, which represents a string literal. Processing a string token simply returns its value`.
Interpreter.java
public Object expression(Token expr) throws Exception {
if (expr.kind.equals("digit")) {
return digit(expr);
// Add
} else if (expr.kind.equals("string")) {
return string(expr);
} else if (expr.kind.equals("ident")) {
return ident(expr);
} else if (expr.kind.equals("func")) {
return func(expr);
} else if (expr.kind.equals("fexpr")) {
return fexpr(expr);
} else if (expr.kind.equals("paren")) {
return invoke(expr);
} else if (expr.kind.equals("sign") && expr.value.equals("=")) {
return assign(expr);
} else if (expr.kind.equals("unary")) {
return unaryCalc(expr);
} else if (expr.kind.equals("sign")) {
return calc(expr);
} else {
throw new Exception("Expression error");
}
}
public String string(Token token) {
return token.value;
}
This is a modification of the value () method.
The value () method returns the value if the argument is the value itself, such as 1.
If it is a variable, it returns the value that the variable holds.
It also corresponded when the argument was String.
Interpreter.java
public Object value(Object value) throws Exception {
if (value instanceof Integer) {
return value;
} else if (value instanceof String) {
return value;
} else if (value instanceof Func) {
return value;
} else if (value instanceof Variable) {
Variable v = (Variable) value;
return value(v.value);
}
throw new Exception("right value error");
}
ʻInteger () method change. The ʻinteger () method guarantees that the argument is ʻInteger. Changed to implicitly try to convert to ʻInteger if the argument was String.
Interpreter.java
public Integer integer(Object value) throws Exception {
if (value instanceof Integer) {
return (Integer) value;
} else if (value instanceof String) {
return Integer.decode((String) value);
} else if (value instanceof Variable) {
Variable v = (Variable) value;
return integer(v.value);
}
throw new Exception("right value error");
}
Added the string () method.
The string () method guarantees that the argument is String.
Implicitly convert to String if the argument is ʻInteger`.
Interpreter.java
public String string(Object value) throws Exception {
if (value instanceof String) {
return (String) value;
} else if (value instanceof Integer) {
return value.toString();
} else if (value instanceof Variable) {
Variable v = (Variable) value;
return string(v.value);
}
throw new Exception("right value error");
}
ʻUnaryCalc () method change. The ʻunaryCalc () method is a method that handles unary operators.
Before the change, only ʻIntegerwas supported. In order to correspond toString, depending on the result of calling the value () method If it is ʻInteger, call the ʻunaryCalcInteger () method that handles the unary operator for ʻInteger.
If it is String, call the ʻunaryCalcString ()method that handles the unary operator forString`.
Interpreter.java
public Object unaryCalc(Token expr) throws Exception {
Object value = value(expression(expr.left));
if (value instanceof Integer) {
return unaryCalcInteger(expr.value, (Integer) value);
} else if (value instanceof String) {
return unaryCalcString(expr.value, (String) value);
} else {
throw new Exception("unaryCalc error");
}
}
public Object unaryCalcInteger(String sign, Integer left) throws Exception {
if (sign.equals("+")) {
return left;
} else if (sign.equals("-")) {
return -left;
} else if (sign.equals("!")) {
return toInteger(!isTrue(left));
} else {
throw new Exception("unaryCalcInteger error");
}
}
public Object unaryCalcString(String sign, String left) throws Exception {
if (sign.equals("!")) {
return toInteger(!isTrue(left));
} else {
throw new Exception("unaryCalcString error");
}
}
This is a modification of the calc () method.
The calc () method is a method that handles binary operators.
Before the change, only ʻIntegerwas supported. In order to correspond toString, depending on the result of calling the value () method on the left side, If it is ʻInteger, call the calcInteger () method that handles the binary operator for ʻInteger. If it is String, call the calcString () method that handles the binary operator for String`.
The right-hand side may be implicitly converted.
Interpreter.java
public Object calc(Token expr) throws Exception {
Object left = value(expression(expr.left));
Object right = value(expression(expr.right));
Integer ileft = null;
String sleft = null;
if (left instanceof Integer) {
ileft = (Integer) left;
} else if (left instanceof String) {
sleft = (String) left;
}
if (ileft != null) {
return calcInteger(expr.value, ileft, right);
} else if (sleft != null) {
return calcString(expr.value, sleft, right);
} else {
throw new Exception("calc error");
}
}
public Object calcInteger(String sign, Integer left, Object right) throws Exception {
if (sign.equals("+")) {
return left + integer(right);
} else if (sign.equals("-")) {
return left - integer(right);
} else if (sign.equals("*")) {
return left * integer(right);
} else if (sign.equals("/")) {
return left / integer(right);
} else if (sign.equals("==")) {
return toInteger(left == integer(right));
} else if (sign.equals("!=")) {
return toInteger(left != integer(right));
} else if (sign.equals("<")) {
return toInteger(left < integer(right));
} else if (sign.equals("<=")) {
return toInteger(left <= integer(right));
} else if (sign.equals(">")) {
return toInteger(left > integer(right));
} else if (sign.equals(">=")) {
return toInteger(left >= integer(right));
} else if (sign.equals("&&")) {
if (!isTrue(left)) {
return left;
}
return right;
} else if (sign.equals("||")) {
if (isTrue(left)) {
return left;
}
return right;
} else {
throw new Exception("calcIteger error");
}
}
public Object calcString(String sign, String left, Object right) throws Exception {
if (sign.equals("+")) {
return left + string(right);
} else if (sign.equals("==")) {
return toInteger(left.equals(string(right)));
} else if (sign.equals("!=")) {
return toInteger(!left.equals(string(right)));
} else if (sign.equals("<")) {
return toInteger(left.compareTo(string(right)) < 0);
} else if (sign.equals("<=")) {
return toInteger(left.compareTo(string(right)) <= 0);
} else if (sign.equals(">")) {
return toInteger(left.compareTo(string(right)) > 0);
} else if (sign.equals(">=")) {
return toInteger(left.compareTo(string(right)) >= 0);
} else if (sign.equals("&&")) {
if (!isTrue(left)) {
return left;
}
return right;
} else if (sign.equals("||")) {
if (isTrue(left)) {
return left;
}
return right;
} else {
throw new Exception("calcString error");
}
}
ʻIsTrue ()Method change. Changed so thatString` can be processed as a logical value.
Changed to judge a character string of length 0 as false.
Interpreter.java
public boolean isTrue(Object value) throws Exception {
if (value instanceof Integer) {
return 0 != ((Integer) value);
} else if (value instanceof String) {
return !"".equals(value);
} else if (value instanceof Func) {
return true;
} else {
return false;
}
}
The program below using the above implementation
var object = ""
if (!object) {
object = "world"
}
println("Hello " + object + "!")
And
The println () method call at the end of the program prints Hello world!.
Interpreter.java
public static void main(String[] args) throws Exception {
String text = "";
text += "var object = \"\"";
text += "if (!object) {";
text += " object = \"world\"";
text += "}";
text += "println(\"Hello \" + object + \"!\")";
List<Token> tokens = new Lexer().init(text).tokenize();
List<Token> blk = new Parser().init(tokens).block();
new Interpreter().init(blk).run();
// --> Hello world!
}
That's all for the implementation. Thank you very much.
The full source is available here.
Calc https://github.com/quwahara/Calc/tree/article-15-string-r3/Calc/src/main/java
There is a continuation article.
** Corresponds to method invocation ** http://qiita.com/quwahara/items/f1bddefe984c8d233e02
Recommended Posts