请输入您要查询的英文单词:

 

单词 Tokeniser
释义

Tokeniser

中文百科

词法分析 Lexical analysis

(重定向自Tokeniser)

词法分析英语:lexical analysis)是计算机科学中将字符串行转换为标记(token)串行的过程。进行词法分析的进程或者函数叫作词法分析器(lexical analyzer,简称lexer),也叫扫描器(scanner)。词法分析器一般以函数的形式存在,供语法分析器调用。

英语百科

Lexical analysis 词法分析

(重定向自Tokeniser)

In computer science, lexical analysis is the process of converting a sequence of characters (such as in a computer program or web page) into a sequence of tokens (strings with an identified "meaning"). A program that performs lexical analysis may be called a lexer, tokenizer, or scanner (though "scanner" is also used to refer to the first stage of a lexer). Such a lexer is generally combined with a parser, which together analyze the syntax of programming languages, web pages, and so forth.

随便看

 

英汉双解词典包含3607232条英汉词条,基本涵盖了全部常用单词的翻译及用法,是英语学习的有利工具。

 

Copyright © 2004-2022 Newdu.com All Rights Reserved
京ICP备09058993号 更新时间:2025/8/10 3:49:25