# 2.5 总结

不管对哪个语言，词法分析的内在工作原理都是相似的：根据语言的词法规则将输入字节流转换为 Token 流。唯一不同的是各语言的词法规则，所以通常情况下会使用一个词法生成器来构建词法分析器，[Lex](https://en.wikipedia.org/wiki/Lex_\(software\)) 是流行的词法生成器。但 Golang 选择了自己实现，其核心 `scanner.go` 只有 1000 行不到的代码，这也为我们学习 golang 的词法解析提供了好机会。

 scanner 最重要的对外接口就是 `next` 方法，其职责非常简单：根据 golang 的词法规则解析下一个 Token. 其虽然不是严格意义上的 DFA(处理 `..` 时需要回滚一个字符), 但大体工作原理是一致的， scanner 是后续语法分析器的重要组成部分。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://gocompiler.shizhz.me/2.-golang-bian-yi-qi-ci-fa-fen-xi/2.5-zong-jie.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.