bartek/smnp-py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve a little bit string tokenizer

Browse Source
This commit is contained in:
Bartłomiej Pluta
2019-07-12 16:52:26 +02:00
parent c7e90b9fbd
commit b711b6a582
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
smnp/token/tokenizers/string.py
View File

@@ -13,5 +13,5 @@ def stringTokenizer(input, current, line):
char = input[current + consumedChars] char = input[current + consumedChars]
value += char value += char
consumedChars += 1 consumedChars += 1
return (consumedChars, Token(TokenType.STRING, value[1:len(value)-1], (line, current))) return (consumedChars, Token(TokenType.STRING, value[1:len(value)-1], (line, current), value))
return (0, None) return (0, None)