Sign upLogin

giganticode/codeprep

View on GitHub
Star
refactoring(matchers): not necessary convert to a list of parsed tokens #113
hlib authored a78e6678
complete00:00:17
Your first build completed successfully!
See the results

Congratulations

Your first build completed successfully!

See the Results

1
git clone
2
codeclimate validate-config
View output
codeclimate validate-config
No checked in config: nothing to validate.
3
codeclimate prepare
View output
codeclimate prepare
4
builder pull-engines
View output
determining required images
5
structure
View output
12
Parser process id: 12
codeclimate-parser socket not present
waiting 1s...
W, [2024-04-24T00:06:47.398283 #1]  WARN -- : Skipping test-data/test-corpus/jquery.min.js due to CC::Parser::Client::HTTPError
W, [2024-04-24T00:06:47.398533 #1]  WARN -- : Response status: 422
6
duplication
View output
12
Parser process id: 12
codeclimate-parser socket not present
waiting 1s...
W, [2024-04-24T00:06:52.617150 #1]  WARN -- : Skipping test-data/test-corpus/jquery.min.js due to CC::Parser::Client::HTTPError
W, [2024-04-24T00:06:52.617198 #1]  WARN -- : Response status: 422
I, [2024-04-24T00:06:52.769970 #1]  INFO -- : Skipping file ./codeprep/prepconfig.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 27
    EN_ONLY: str = 'enonly'
           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.771715 #1]  INFO -- : Skipping file ./codeprep/preprocess/tokens.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 27
    def __init__(self, over: Sequence[Any],
                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.783990 #1]  INFO -- : Skipping file ./codeprep/preprocess/metadata.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 15
    n_subtokens_per_token: List[int] = field(default_factory=list)
                         ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.785386 #1]  INFO -- : Skipping file ./codeprep/preprocess/result.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 15
    prepped_tokens: TokenSequence = field(default_factory=TokenSequence.empty)
                  ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.800282 #1]  INFO -- : Skipping file ./codeprep/preprocess/reprconfig.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 15
    bpe_data: Optional[BpeData],
            ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.816009 #1]  INFO -- : Skipping file ./codeprep/preprocess/core.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 13
    def to_repr_list(token_list: Sequence[ParsedToken], repr_config: ReprConfig) -> PreprocessingResult:
                               ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.818280 #1]  INFO -- : Skipping file ./codeprep/preprocess/codestructure.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 15
    subtokens_in_each_line: List[int]
                          ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.831560 #1]  INFO -- : Skipping file ./codeprep/preprocess/placeholders.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/python2/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/python2/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf7 in position 0: invalid start byte


I, [2024-04-24T00:06:52.844160 #1]  INFO -- : Skipping file ./codeprep/bpepkg/bpe_config.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 16
    CASE: str = 'case'
        ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.845272 #1]  INFO -- : Skipping file ./codeprep/bpepkg/merge.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 14
    def __init__(self, pair: Tuple[str, str], freq: int = None, priority: int = None):
                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.856811 #1]  INFO -- : Skipping file ./codeprep/bpepkg/wild_bpe.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 22
    def __init__(self, merges_done: int, time_for_last_merge: float,
                                  ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.859333 #1]  INFO -- : Skipping file ./codeprep/bpepkg/bpe_learn.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 20
    def get_stats(split_base_vocab: Dict[str, int]) -> PriorityCounter:
                                  ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.873207 #1]  INFO -- : Skipping file ./codeprep/bpepkg/bpe_encode.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 21
    def __init__(self, merges_cache=None, merges: MergeList=None):
                                                ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.897275 #1]  INFO -- : Skipping file ./codeprep/bpepkg/cache.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 21
    def read_bpe_cache(file: str) -> Dict[str, List[str]]:
                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.897952 #1]  INFO -- : Skipping file ./codeprep/noneng.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 22
    def replace_non_ascii_seqs(word:str, placeholder: str) -> str:
                                   ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.908901 #1]  INFO -- : Skipping file ./codeprep/util/misc.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 12
    def merge_dicts_(dict1, dict2) -> Tuple[Dict, List]:
                                   ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.913260 #1]  INFO -- : Skipping file ./codeprep/util/file.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 12
    def has_one_of_extensions(name: bytes, extensions: List[bytes]) -> bool:
                                  ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.922397 #1]  INFO -- : Skipping file ./codeprep/util/dir.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 17
    def walk(path:bytes, extension: Optional[bytes] = None) -> Generator[bytes, None, None]:
                 ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.935151 #1]  INFO -- : Skipping file ./codeprep/parse/matchers.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 22
    def match(self, token, value: str) -> bool:
                                ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.948841 #1]  INFO -- : Skipping file ./codeprep/parse/core.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 35
    def _convert(token, value: str) -> Union[ParsedToken, List[ParsedToken]]:
                             ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.950182 #1]  INFO -- : Skipping file ./codeprep/parse/subtokens.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 18
    def split_identifier(token: str) -> Identifier:
                              ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.961730 #1]  INFO -- : Skipping file ./codeprep/stemming.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 10
    def stem(word: str):
                 ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.963517 #1]  INFO -- : Skipping file ./codeprep/api/text.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 23
    def remove_trailing_newline(prep_tokens: List[ParsedToken]) -> List[ParsedToken]:
                                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.972626 #1]  INFO -- : Skipping file ./codeprep/api/corpus.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 26
    def _calc_n_tokens(params: Tuple[str,]) -> int:
                             ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.974568 #1]  INFO -- : Skipping file ./codeprep/api/common.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 11
    def create_split_value(split_type: str, bpe_codes_id: Optional[str] = None, full_strings: bool = False,
                                     ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:52.996318 #1]  INFO -- : Skipping file ./codeprep/pipeline/parse_projects.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 22
    def preprocess_and_write(params: Tuple[bytes, bytes]) -> None:
                                   ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.006355 #1]  INFO -- : Skipping file ./codeprep/pipeline/vocab.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 48
    def __init__(self, word_counts: Counter, chunk: int):
                                  ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.008314 #1]  INFO -- : Skipping file ./codeprep/pipeline/to_repr.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 37
    def get_global_bpe_data_if_available() -> Optional[BpeData]:
                                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.021876 #1]  INFO -- : Skipping file ./codeprep/pipeline/stages.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 19
    def run_parsing(dataset: Dataset) -> None:
                           ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.022343 #1]  INFO -- : Skipping file ./codeprep/pipeline/bpelearner.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 22
    def get_base_vocab(dataset: Dataset) -> Tuple[Dict[str, int], Dict[str, int]]:
                              ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.033605 #1]  INFO -- : Skipping file ./codeprep/pipeline/bperegistry.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 37
    def __init__(self, merge_list_id: str, n_merges: int, codes_file: str, cache_file: str):
                                    ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.036810 #1]  INFO -- : Skipping file ./codeprep/pipeline/vocabloader.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 18
    def all(merge_list_id: str) -> Dict[str, int]:
                         ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.048769 #1]  INFO -- : Skipping file ./codeprep/pipeline/dataset.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 35
    def __init__(self, dataset: 'Dataset', path: str, suffix: str = ''):
                              ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.065213 #1]  INFO -- : Skipping file ./codeprep/cli/spec.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 177
    dsc.main(app_name, f'{app_name} {version}', argv=args, exit_at_end=False)
                                             ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.080875 #1]  INFO -- : Skipping file ./codeprep/cli/impl.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 24
    def set_log_level(args: Dict[str, str]) -> None:
                          ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.082485 #1]  INFO -- : Skipping file ./codeprep/cli/vocab.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 16
    parser.add_argument('path_to_dataset', action='store', help=f'path to dataset')
                                                                                 ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.096284 #1]  INFO -- : Skipping file ./codeprep/tokentypes/noneng.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 16
    def __init__(self, processable_token: Identifier):
                                        ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.110095 #1]  INFO -- : Skipping file ./codeprep/tokentypes/whitespace.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 21
    return f'<{self.__class__.__name__}>'
                                        ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.111969 #1]  INFO -- : Skipping file ./codeprep/tokentypes/rootclasses.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 15
    def _wrap_in_metadata_for_full_word(self, tokens: List[str], n_additional_empty_line,
                                                    ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.127194 #1]  INFO -- : Skipping file ./codeprep/tokentypes/containers.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 19
    def set_all_tokens_type(token_seq: TokenSequence, t: Type) -> TokenSequence:
                                     ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.128873 #1]  INFO -- : Skipping file ./codeprep/tokentypes/numeric.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 14
    def __init__(self, val: str):
                          ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.143650 #1]  INFO -- : Skipping file ./codeprep/tokentypes/word.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 19
    return f'<{self.__class__.__name__}>'
                                        ^
SyntaxError: invalid syntax


I, [2024-04-24T00:06:53.143916 #1]  INFO -- : Skipping file ./codeprep/tokentypes/__init__.py due to exception (CC::Engine::Analyzers::ParserError): `python2 /usr/src/app/lib/cc/engine/analyzers/python/parser.py` exited with code 1:
Traceback (most recent call last):
  File "/usr/src/app/lib/cc/engine/analyzers/python/parser.py", line 51, in <module>
    print(json.dumps(to_json(ast.parse(source))))
  File "/usr/local/python2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 12
    importlib.import_module(f'{__package__}.{basename(f)[:-3]}')
                                                              ^
SyntaxError: invalid syntax