├── .gitignore
├── AnalysisApi
├── ArgOverwritten
│ ├── .gitignore
│ ├── LICENSE
│ ├── README.md
│ ├── cmd
│ │ └── argoverwritten
│ │ │ └── main.go
│ ├── go.mod
│ ├── go.sum
│ └── passes
│ │ └── ArgOverwritten
│ │ ├── analyzer.go
│ │ ├── analyzer_test.go
│ │ └── testdata
│ │ ├── AnonymousFunction
│ │ └── main.go
│ │ ├── AssigningParamToAVariableFirst
│ │ └── main.go
│ │ ├── DecrementOperator
│ │ └── main.go
│ │ ├── EmptyBodyFunction
│ │ ├── add_amd64.s
│ │ └── main.go
│ │ ├── MultipleParamsOfSameType
│ │ └── main.go
│ │ ├── NoWarnings
│ │ └── main.go
│ │ ├── OverwritingParamFromOuterScope
│ │ └── main.go
│ │ ├── ShadowingVariable
│ │ └── main.go
│ │ └── SimpleOverwriting
│ │ └── main.go
└── README.md
├── CompilerFrontEndASTInGo
├── ArgsOverwriteAnalyzer.go
├── ArgsOverwriteAnalyzer_test.go
├── CodeExamples
│ ├── BadExpr.txt
│ ├── DifferentSpecsAndDecls.go
│ └── SimpleForLoop.go
├── README.md
├── result
│ └── ArgsOverwriteAnalyzer.go
└── testdata
│ ├── AnonymousFunction
│ └── main.go
│ ├── AssigningParamToAVariableFirst
│ └── main.go
│ ├── DecrementOperator
│ └── main.go
│ ├── EmptyBodyFunction
│ ├── add_amd64.s
│ └── main.go
│ ├── LhsMultipleValues
│ └── main.go
│ ├── MultipleParamsOfSameType
│ └── main.go
│ ├── NoWarnings
│ └── main.go
│ ├── OverwritingParamFromOuterScope
│ └── main.go
│ ├── ShadowingVariable
│ └── main.go
│ └── SimpleOverwriting
│ └── main.go
├── CompilerMiddleEndSSAInGo
├── CodeExamples
│ ├── ElseIf
│ │ └── ElseIf.go
│ └── Map
│ │ └── Map.go
├── README.md
├── TerminationInsideGoroutine.go
├── TerminationInsideGoroutine_test.go
└── testdata
│ ├── FatalInsideGoroutineAnonymousFlag
│ └── main.go
│ ├── FatalInsideGoroutineSimpleFlag
│ └── main.go
│ ├── FatalInsideRegularFunctionNoFlag
│ └── main.go
│ └── SkipInsideGoroutineSimpleFlag
│ └── main.go
├── Conclusion
└── README.md
├── Intro
└── README.md
├── README.md
├── go.mod
└── go.sum
/.gitignore:
--------------------------------------------------------------------------------
1 | *.o
2 | *.a
3 | *.so
4 | .idea
5 | .vscode
6 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/.gitignore:
--------------------------------------------------------------------------------
1 | *.o
2 | *.a
3 | *.so
4 | .idea
5 | .vscode
6 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2020 Amit Davidson
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/README.md:
--------------------------------------------------------------------------------
1 | # ArgOverwritten
2 |
3 | [](http://golang.org)
4 | [](http://golang.org)
5 | [](https://lbesson.mit-license.org/)
6 | [](http://makeapullrequest.com)
7 |
8 | ArgOverwritten finds function arguments being overwritten
9 |
10 | ## Quick Start:
11 |
12 | Download the package
13 |
14 | ``` bash
15 | go get github.com/amit-davidson/ArgOverwritten/cmd/argoverwritten
16 | ```
17 |
18 | Pass the entry point
19 |
20 | ``` bash
21 | go vet -vettool=$(which argoverwritten) ${path_to_file}
22 | ```
23 |
24 | ## Example:
25 | ``` go
26 | package testdata
27 |
28 | func body(a int) {
29 | _ = func() {
30 | a = 5
31 | }
32 | }
33 |
34 | func main() {
35 | body(5)
36 | }
37 | ```
38 | ``` bash
39 | go vet -vettool=$(which argoverwritten) /testdata/OverwritingParamFromOuterScope
40 | ```
41 |
42 | Output
43 | ```
44 | /testdata/OverwritingParamFromOuterScope/main.go:5:3: "a" overwrites func parameter
45 | ```
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/cmd/argoverwritten/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "github.com/amit-davidson/ArgOverwritten/passes/ArgOverwritten"
5 | "golang.org/x/tools/go/analysis/singlechecker"
6 | )
7 |
8 | func main() {
9 | singlechecker.Main(ArgOverwritten.Analyzer)
10 | }
11 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/go.mod:
--------------------------------------------------------------------------------
1 | module github.com/amit-davidson/ArgOverwritten
2 |
3 | go 1.15
4 |
5 | require golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e
6 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/go.sum:
--------------------------------------------------------------------------------
1 | github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
2 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
3 | golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
4 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
5 | golang.org/x/mod v0.3.0 h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
6 | golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
7 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
8 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
9 | golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
10 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
11 | golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
12 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
13 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
14 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
15 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
16 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
17 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
18 | golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
19 | golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e h1:4nW4NLDYnU28ojHaHO8OVxFHk/aQ33U01a9cjED+pzE=
20 | golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
21 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
22 | golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
23 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
24 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
25 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/analyzer.go:
--------------------------------------------------------------------------------
1 | package ArgOverwritten
2 |
3 | import (
4 | "fmt"
5 | "go/ast"
6 | "golang.org/x/tools/go/analysis"
7 | )
8 |
9 | var Analyzer = &analysis.Analyzer{
10 | Name: "ArgOverwritten",
11 | Doc: doc,
12 | Run: run,
13 | }
14 |
15 | const (
16 | doc = "ArgOverwritten finds function arguments being overwritten"
17 | )
18 |
19 | func report(pass *analysis.Pass, ident *ast.Ident) {
20 | message := fmt.Sprintf("\"%s\" overwrites func parameter", ident.Name)
21 | pass.Report(analysis.Diagnostic{
22 | Pos: ident.Pos(),
23 | Message: message,
24 | })
25 | }
26 |
27 | func run(pass *analysis.Pass) (interface{}, error) {
28 | visitor := func(node ast.Node) bool {
29 | var typ *ast.FuncType
30 | var body *ast.BlockStmt
31 | switch fn := node.(type) {
32 | case *ast.FuncDecl: // Regular function
33 | typ = fn.Type
34 | body = fn.Body
35 | case *ast.FuncLit: // Anonymous function
36 | typ = fn.Type
37 | body = fn.Body
38 | }
39 | if typ == nil || body == nil { // Exclude other types but also external functions with missing body
40 | return true
41 | }
42 | if len(typ.Params.List) == 0 {
43 | return true
44 | }
45 |
46 | for _, field := range typ.Params.List {
47 | for _, arg := range field.Names {
48 | obj := pass.TypesInfo.ObjectOf(arg)
49 | ast.Inspect(body, func(node ast.Node) bool {
50 | switch stmt := node.(type) {
51 | case *ast.AssignStmt:
52 | for _, lhs := range stmt.Lhs {
53 | ident, ok := lhs.(*ast.Ident)
54 | if ok && pass.TypesInfo.ObjectOf(ident) == obj {
55 | report(pass, ident)
56 | }
57 | }
58 | case *ast.IncDecStmt:
59 | ident, ok := stmt.X.(*ast.Ident)
60 | if ok && pass.TypesInfo.ObjectOf(ident) == obj {
61 | report(pass, ident)
62 | }
63 | }
64 | return true
65 | })
66 | }
67 | }
68 | return true
69 | }
70 | for _, f := range pass.Files {
71 | ast.Inspect(f, visitor)
72 | }
73 | return nil, nil
74 | }
75 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/analyzer_test.go:
--------------------------------------------------------------------------------
1 | package ArgOverwritten
2 |
3 | import (
4 | "fmt"
5 | "golang.org/x/tools/go/analysis/analysistest"
6 | "os"
7 | "testing"
8 | )
9 |
10 | func TestAnalyzer(t *testing.T) {
11 | var testCases = []struct {
12 | name string
13 | }{
14 | {name: "SimpleOverwriting"},
15 | {name: "AnonymousFunction"},
16 | {name: "OverwritingParamFromOuterScope"},
17 | {name: "AssigningParamToAVariableFirst"},
18 | {name: "MultipleParamsOfSameType"},
19 | {name: "ShadowingVariable"},
20 | {name: "EmptyBodyFunction"},
21 | {name: "NoWarnings"},
22 | {name: "DecrementOperator"},
23 | }
24 | for _, tc := range testCases {
25 | t.Run(tc.name, func(t *testing.T) {
26 | analysistest.Run(t, fmt.Sprintf("%s%s%s", analysistest.TestData(), string(os.PathSeparator), tc.name), Analyzer)
27 | })
28 | }
29 | }
30 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/AnonymousFunction/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func main() {
4 | _ = func(a int) {
5 | a = 5 // want `"a" overwrites func parameter`
6 | }
7 | }
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/AssigningParamToAVariableFirst/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a int) {
4 | b := a
5 | b = 5
6 | _ = b
7 | }
8 |
9 | func main() {
10 | body(5)
11 | }
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/DecrementOperator/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | func retry(fn func() error, retries int) error {
4 | for {
5 | if err := fn(); err != nil {
6 | if retries < 1 {
7 | return err
8 | }
9 | retries-- // want `"retries" overwrites func parameter`
10 | continue
11 | }
12 | return nil
13 | }
14 | }
15 | func main() {
16 | _ = retry(func() error {
17 | return nil
18 | }, 5)
19 | }
20 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/EmptyBodyFunction/add_amd64.s:
--------------------------------------------------------------------------------
1 | TEXT ·Add(SB),$0-24
2 | MOVQ a+0(FP), AX
3 | ADDQ b+8(FP), AX
4 | MOVQ AX, ret+16(FP)
5 | RET
6 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/EmptyBodyFunction/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | // Add returns the sum of a and b.
4 | func Add(a int64, b int64) int64
5 |
6 | func body(a int) {
7 | b := a
8 | b = 5
9 | _ = b
10 | }
11 |
12 | func main() {
13 | body(5)
14 | _ = Add(3, 4)
15 | }
16 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/MultipleParamsOfSameType/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a, b int, c int) {
4 | a = 5 // want `"a" overwrites func parameter`
5 | _ = b
6 | c = 3 // want `"c" overwrites func parameter`
7 | }
8 |
9 | func main() {
10 | body(1, 2, 3)
11 | }
12 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/NoWarnings/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | import "fmt"
4 |
5 | func closeBody(body int) {
6 | fmt.Print(body)
7 | }
8 |
9 | func main() {
10 | closeBody(1)
11 | }
12 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/OverwritingParamFromOuterScope/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a int) {
4 | _ = func() {
5 | a = 5 // want `"a" overwrites func parameter`
6 | }
7 | }
8 |
9 | func main() {
10 | body(5)
11 | }
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/ShadowingVariable/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a, b int, c int) {
4 | f := func(a int, b int) {
5 | a = 5 // want `"a" overwrites func parameter`
6 | }
7 | _ = f
8 | }
9 |
10 | func main() {
11 | body(1, 2, 3)
12 | }
13 |
--------------------------------------------------------------------------------
/AnalysisApi/ArgOverwritten/passes/ArgOverwritten/testdata/SimpleOverwriting/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func closeBody(body int) {
4 | body = 1 // want `"body" overwrites func parameter`
5 | }
6 |
7 | func main() {
8 | closeBody(1)
9 | }
10 |
--------------------------------------------------------------------------------
/AnalysisApi/README.md:
--------------------------------------------------------------------------------
1 | ## 4 Analysis API:
2 | ### 4.1 (tools/go/analysis]((https://pkg.go.dev/golang.org/x/tools/go/analysis))
3 | The package defines an [API](https://pkg.go.dev/golang.org/x/tools/go/analysis) for modular static analysis tools. In other words, it's a common interface for all static
4 | code analyzers.
5 |
6 | Why should you care? What's the difference between using the analysis API, and the way we wrote so far?
7 | The analysis API makes writing analyses easier by taking care of:
8 | - file parsing
9 | - testing
10 | - integration with go vet
11 | - many more
12 |
13 | Also, it enforces a single pattern for all the static analysis tools such as how analysis are structured and how
14 | warnings are reported
15 |
16 | ### 4.2 Analysis API members
17 | The primary type in the API is [`analysis.Analyzer`](https://pkg.go.dev/golang.org/x/tools/go/analysis#hdr-Analyzer).
18 | It describes an analysis function: its name, documentation, flags, relationship to other analyzers, and of course, it's logic.
19 |
20 | ``` go
21 | type Analyzer struct {
22 | Name string
23 | Doc string
24 | Requires []*Analyzer
25 | Run func(*Pass) (interface{}, error)
26 |
27 | ...
28 | }
29 | ```
30 |
31 | The `Name` and `Doc` are obvious. They are used to describe what the tool does.
32 |
33 | Another interesting is the `Requires` field. It specifies a list of analyses upon which this one depends and whose
34 | results it may access, and it constrains the order in which a driver may run analyses.
35 |
36 | > To use SSA in the analysis api, we would have to require the [SSA builder](https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/buildssa)
37 | > `Requires: []*analysis.Analyzer {
38 | buildssa.Analyzer
39 | }`
40 |
41 |
42 | The most important one is the `Run` function. It contains the logic that should is executed upon a single package.
43 | It takes as an argument [`*analysis.Pass`](https://pkg.go.dev/golang.org/x/tools/go/analysis#hdr-Pass) and returns a
44 | value to be used by other analyzers and an error.
45 |
46 | ``` go
47 | type Pass struct {
48 | Fset *token.FileSet
49 | Files []*ast.File
50 | Pkg *types.Package
51 | TypesInfo *types.Info
52 | Report func(Diagnostic)
53 |
54 | ...
55 | }
56 | ```
57 |
58 | A `Pass` describes a single unit of work: the application of a particular `Analyzer` to a particular package of Go code. The `Pass` provides information to the Analyzer's `Run` function about the analyzed package and provides operations to the `Run` function for reporting diagnostics and other information back to the driver. It also provides `Fset`, `Files`, `Pkg`, and `TypesInfo` that we know from earlier, so we don't have to take care of ourselves.
59 |
60 | The [`Report`](https://pkg.go.dev/golang.org/x/tools/go/analysis#Pass.Reportf) function emits a diagnostic, a message associated with a source position. For most analyses, diagnostics are their primary result. For convenience, `Pass` provides a helper method, `Reportf`, to report a new diagnostic by formatting a string. Diagnostic is defined as:
61 |
62 | ``` go
63 | type Diagnostic struct {
64 | Pos token.Pos
65 | Category string // optional
66 | Message string
67 | }
68 | ```
69 |
70 | ### 4.3 Project structure
71 | First let's define the project structure:
72 | ```
73 | │── README.md
74 | │── cmd
75 | │ └── analyzerName
76 | │ └── main.go
77 | │── go.mod
78 | │── go.sum
79 | └── passes
80 | └── passName
81 | │── pass.go
82 | │── pass_test.go
83 | └── testdata
84 | ```
85 |
86 | We create a directory where all of our passes reside in named `passes`. Each pass lives in its package, including its logic and tests.
87 | Then we define the usual `cmd` for our executables that contains all the analyzers the module has.
88 |
89 | Regarding our analyzers we wrote previously, we had to handle both the logic of the analyzer and the instrumentation around it.
90 | When converting our code to the analysis API, the AST traversal part (the logic) will sit under `passes` and loading the code
91 | part will be taken care of by the analysis API so we can ignore it.
92 |
93 | Next , we need a way to run the Analyzer and to test it.
94 |
95 | ### 4.4 Running our code
96 | inside `main.go`, we'll add the following code.
97 |
98 | ``` go
99 | package main
100 |
101 | import (
102 | "path/to/our/pass"
103 | "golang.org/x/tools/go/analysis/singlechecker"
104 | )
105 |
106 | func main() { singlechecker.Main(passName.Analyzer) }
107 | ```
108 | Analyzers are provided in the form of packages that a driver program is expected to import.
109 | The [`singlechecker`](https://pkg.go.dev/golang.org/x/tools/go/analysis/singlechecker) package provides the `main` function for a command that runs one Analyzer. By convention, each Analyzer should be accompanied by a singlechecker-based command defined in its entirety as: This code calls our Analyzer.
110 | If we wanted our command to run multiple analyzers, we would have to use [`multichecker`](https://pkg.go.dev/golang.org/x/tools/go/analysis/multichecker).
111 |
112 | Now we can run it using
113 | ``` bash
114 | go install path/to/analyzer
115 | go vet -vettool=$(which analyzername) path/to/files
116 | ```
117 |
118 |
119 | ### 4.5 Testing our code
120 | The [`analysistest`](https://godoc.org/golang.org/x/tools/go/analysis/analysistes) subpackage provides utilities for testing an Analyzer. Using `analysistest.Run`, it is possible to run an analyzer on a package of `testdata` files and check that it reported all the expected diagnostics.
121 | Expectations are expressed using "// want ..." comments in the input code, such as the following:
122 |
123 | ``` go
124 | package testdata
125 |
126 | func main() {
127 | _ = func(a int) {
128 | a = 5 // want `"a" overwrites func parameter`
129 | }
130 | }
131 | ```
132 |
133 | ### 4.6 Overviewing an analyzer!
134 | In this section, we'll convert our [`ArgsOverwrite`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/result/ArgsOverwriteAnalyzer.gohttps://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/result/ArgsOverwriteAnalyzer.go)
135 | Analyzer from earlier to the analysis API
136 |
137 | ### 4.7 Congratulations
138 | You have a good understanding of what the analysis API is and how to use it help us in writing analyses in the future.
139 |
140 | In the [next section](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/Conclusion/README.md)
141 | we'll conclude this workshop by touching a point regarding static code analyzers in general and take a look at other code
142 | analyzers written by the Go community.
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "fmt"
5 | "go/ast"
6 | "go/importer"
7 | "go/parser"
8 | "go/token"
9 | "go/types"
10 | "log"
11 | "os"
12 | )
13 |
14 | func main() {
15 | path := os.Args[1]
16 | fset, pkgs, err := loadProgram(path)
17 | if err != nil {
18 | log.Fatalf("Failed to parse dir %s: %s", path, err)
19 | }
20 |
21 | for _, pkg := range pkgs {
22 | outputs := analyzePackage(pkg, fset)
23 | for _, message := range outputs {
24 | fmt.Println(message)
25 | }
26 | }
27 | }
28 |
29 | func loadProgram(path string) (*token.FileSet, map[string]*ast.Package, error) {
30 | fset := token.NewFileSet()
31 | pkgs, err := parser.ParseDir(fset, path, nil, 0)
32 | if err != nil {
33 | return nil, nil, err
34 | }
35 | return fset, pkgs, nil
36 | }
37 |
38 | func populateTypes(conf types.Config, fset *token.FileSet, f *ast.File) (*types.Info, error) {
39 | info := &types.Info{
40 | Defs: make(map[*ast.Ident]types.Object),
41 | Uses: make(map[*ast.Ident]types.Object),
42 | }
43 | _, err := conf.Check("", fset, []*ast.File{f}, info)
44 | if err != nil {
45 | return nil, err
46 | }
47 | return info, nil
48 | }
49 |
50 | func analyzePackage(p *ast.Package, fset *token.FileSet) []string {
51 | var info *types.Info
52 | var err error
53 | outputs := make([]string, 0)
54 |
55 | visitor := func(node ast.Node) bool {
56 | var typ *ast.FuncType
57 | var body *ast.BlockStmt
58 | switch fn := node.(type) {
59 | case *ast.FuncDecl: // Regular function
60 | typ = fn.Type
61 | body = fn.Body
62 | case *ast.FuncLit: // Anonymous function
63 | typ = fn.Type
64 | body = fn.Body
65 | }
66 | if typ == nil || body == nil { // Exclude other types but also external functions with missing body
67 | return true
68 | }
69 | if len(typ.Params.List) == 0 {
70 | return true
71 | }
72 |
73 | for _, field := range typ.Params.List {
74 | for _, arg := range field.Names {
75 | obj := info.ObjectOf(arg)
76 | ast.Inspect(body, func(node ast.Node) bool {
77 | switch stmt := node.(type) {
78 | case *ast.AssignStmt:
79 | var ident *ast.Ident
80 | // -------------------------------------------------------------------------
81 | // info.ObjectOf requires *ast.ident. How can I extract *ast.ident from *ast.AssignStmt.Lhs?
82 | // The flow should be similar to how *ast.Ident is taken in the *ast.IncDecStmt case
83 | // -------------------------------------------------------------------------
84 | if info.ObjectOf(ident) == obj {
85 | outputs = append(outputs, output(ident, fset.Position(ident.Pos())))
86 | }
87 |
88 | case *ast.IncDecStmt:
89 | ident, ok := stmt.X.(*ast.Ident)
90 | if ok && info.ObjectOf(ident) == obj {
91 | outputs = append(outputs, output(ident, fset.Position(ident.Pos())))
92 | }
93 | }
94 | return true
95 | })
96 | }
97 | }
98 | return true
99 | }
100 |
101 | conf := types.Config{Importer: importer.Default()}
102 | for _, f := range p.Files {
103 | info, err = populateTypes(conf, fset, f)
104 | if err != nil {
105 | log.Fatal(err)
106 | }
107 | ast.Inspect(f, visitor)
108 | }
109 | return outputs
110 | }
111 |
112 | func output(ident *ast.Ident, pos token.Position) string {
113 | return fmt.Sprintf("\"%s\" overwrites func parameter in pos: %s:%d:%d", ident.Name, pos.Filename, pos.Line, pos.Column)
114 | }
115 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer_test.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "github.com/stretchr/testify/assert"
5 | "github.com/stretchr/testify/require"
6 | "go/ast"
7 | "go/parser"
8 | "go/token"
9 | "path/filepath"
10 | "testing"
11 | )
12 |
13 | func Test_analyzePackage(t *testing.T) {
14 | var testCases = []struct {
15 | name string
16 | result []string
17 | }{
18 | {name: "SimpleOverwriting", result: []string{"\"body\" overwrites func parameter in pos"}},
19 | {name: "AnonymousFunction", result: []string{"\"a\" overwrites func parameter in pos"}},
20 | {name: "OverwritingParamFromOuterScope", result: []string{"\"a\" overwrites func parameter in pos"}},
21 | {name: "AssigningParamToAVariableFirst", result: []string{}},
22 | {name: "MultipleParamsOfSameType", result: []string{"\"a\" overwrites func parameter in pos", "\"c\" overwrites func parameter in pos"}},
23 | {name: "ShadowingVariable", result: []string{"\"a\" overwrites func parameter in pos"}},
24 | {name: "EmptyBodyFunction", result: []string{}},
25 | {name: "NoWarnings", result: []string{}},
26 | {name: "DecrementOperator", result: []string{"\"retries\" overwrites func parameter"}},
27 | {name: "LhsMultipleValues", result: []string{"\"a\" overwrites func parameter", "\"b\" overwrites func parameter"}},
28 | }
29 | for _, tc := range testCases {
30 | t.Run(tc.name, func(t *testing.T) {
31 | path := filepath.Join(".", "testdata", tc.name)
32 | fset := token.NewFileSet()
33 | pkgs, err := parser.ParseDir(fset, path, nil, 0)
34 | require.NoError(t, err)
35 | require.Len(t, pkgs, 1)
36 | var testPkg *ast.Package
37 | for _, pkg := range pkgs {
38 | testPkg = pkg
39 | }
40 |
41 | outputs := analyzePackage(testPkg, fset)
42 | assert.Len(t, outputs, len(tc.result))
43 | for i := range outputs {
44 | assert.Contains(t, outputs[i], tc.result[i])
45 | }
46 | })
47 | }
48 | }
49 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/CodeExamples/BadExpr.txt:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | func main() {
4 | if {
5 | println("Hello, World!")
6 | }
7 | }
8 |
9 | // focus on the missing condition if and how it's represented in the AST
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/CodeExamples/DifferentSpecsAndDecls.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | const (
4 | a = iota
5 | b
6 | c = 3
7 | )
8 |
9 | type d int
10 |
11 | var (
12 | e = 7
13 | )
14 |
15 | func main() {
16 | const name = 8
17 | name2 = 7
18 | }
19 |
20 | // 1. See how each declaration is represented in the AST and the difference between them. Specifically look which type
21 | // implements each interface (spec or decl) by looking at the docs for go/ast: https://golang.org/pkg/go/ast/.
22 |
23 | // You can also look at the code, it gives a good overview: https://golang.org/src/go/ast/ast.go?s=26852:26891#L851
24 | // and here:https://golang.org/src/go/ast/ast.go?s=29093:29170#L929
25 |
26 | //2. Look at the difference between the "name" and "name2".
27 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/CodeExamples/SimpleForLoop.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | func main() {
4 | for n := 0; n <= 5; n++ {
5 | if n%2 == 0 {
6 | continue
7 | }
8 | }
9 | }
10 |
11 | //1. See the properties of the for loop
12 |
13 | //2. See the properties of the if condition
14 |
15 | //3. See how continue is represented in the code
16 |
17 | // You can go to https://golang.org/pkg/go/ast/ for the types and their documentation. It'll give you an explanation
18 | // about the properties of each type.
19 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/README.md:
--------------------------------------------------------------------------------
1 | ## Compiler front end, AST, and analysis introduction
2 | ### 2.1 Go packages overview
3 | There are six relevant packages regarding the compiler front end when talking about static analysis:
4 |
5 | - [token](https://golang.org/pkg/go/token/) - Package `token` defines constants representing the lexical tokens of Go
6 | - [scanner](https://golang.org/pkg/go/scanner/) - Package `scanner` implements a scanner for Go source text. It takes a `[]byte` as the source, which can then be tokenized through repeated calls to the `Scan` method.
7 | - [parser](https://golang.org/pkg/go/parser/) - Package `parser` implements a parser for Go source files. The output is an abstract syntax tree (AST) representing the Go source
8 | - [AST](https://golang.org/pkg/go/ast/) - Package `AST` declares the types used to represent syntax trees for Go packages.
9 | - [constant](https://golang.org/pkg/go/constant/) - Package `constant` implements Values representing untyped Go constants and their corresponding operations.
10 | - [types](https://golang.org/pkg/go/types/) - Package `types` declares the data types and implements the algorithms for type-checking of Go packages
11 |
12 | The `scanner` package is fed with `[]byte` representing the source code. Its output is a list of tokens defined by the
13 | `token` package, and the parser package uses them to create the `AST` tree. After the tree is constructed,
14 | the parser runs type-checking algorithms run over the tree, validates its correctness, and evaluates constants.
15 |
16 |
17 |
18 | ### 2.2 AST in Go
19 | An abstract syntax tree (AST) is a way of representing the syntax of a programming language as a hierarchical tree-like structure. Let's take a look at the following program for an explanation.
20 |
21 | ``` go
22 | package main
23 | import "fmt"
24 |
25 | func main() {
26 | fmt.Println("hello world")
27 | }
28 | ```
29 |
30 | We can use this [AST visualizer](http://goast.yuroyoro.net/) to view it's AST.
31 | ```
32 | 0 *ast.File {
33 | 1 . Package: 1:1
34 | 2 . Name: *ast.Ident {
35 | 3 . . NamePos: 1:9
36 | 4 . . Name: "main"
37 | 5 . }
38 | 6 . Decls: []ast.Decl (len = 2) {
39 | 7 . . 0: *ast.GenDecl {
40 | 8 . . . TokPos: 3:1
41 | 9 . . . Tok: import
42 | 10 . . . Lparen: -
43 | 11 . . . Specs: []ast.Spec (len = 1) {
44 | 12 . . . . 0: *ast.ImportSpec {
45 | 13 . . . . . Path: *ast.BasicLit {
46 | 14 . . . . . . ValuePos: 3:8
47 | 15 . . . . . . Kind: STRING
48 | 16 . . . . . . Value: "\"fmt\""
49 | 17 . . . . . }
50 | 18 . . . . . EndPos: -
51 | 19 . . . . }
52 | 20 . . . }
53 | 21 . . . Rparen: -
54 | 22 . . }
55 | 23 . . 1: *ast.FuncDecl {
56 | 24 . . . Name: *ast.Ident {
57 | 25 . . . . NamePos: 5:6
58 | 26 . . . . Name: "main"
59 | 27 . . . . Obj: *ast.Object {
60 | 28 . . . . . Kind: func
61 | 29 . . . . . Name: "main"
62 | 30 . . . . . Decl: *(obj @ 23)
63 | 31 . . . . }
64 | 32 . . . }
65 | 33 . . . Type: *ast.FuncType {
66 | 34 . . . . Func: 5:1
67 | 35 . . . . Params: *ast.FieldList {
68 | 36 . . . . . Opening: 5:10
69 | 37 . . . . . Closing: 5:11
70 | 38 . . . . }
71 | 39 . . . }
72 | 40 . . . Body: *ast.BlockStmt {
73 | 41 . . . . Lbrace: 5:13
74 | 42 . . . . List: []ast.Stmt (len = 1) {
75 | 43 . . . . . 0: *ast.ExprStmt {
76 | 44 . . . . . . X: *ast.CallExpr {
77 | 45 . . . . . . . Fun: *ast.SelectorExpr {
78 | 46 . . . . . . . . X: *ast.Ident {
79 | 47 . . . . . . . . . NamePos: 6:3
80 | 48 . . . . . . . . . Name: "fmt"
81 | 49 . . . . . . . . }
82 | 50 . . . . . . . . Sel: *ast.Ident {
83 | 51 . . . . . . . . . NamePos: 6:7
84 | 52 . . . . . . . . . Name: "Println"
85 | 53 . . . . . . . . }
86 | 54 . . . . . . . }
87 | 55 . . . . . . . Lparen: 6:14
88 | 56 . . . . . . . Args: []ast.Expr (len = 1) {
89 | 57 . . . . . . . . 0: *ast.BasicLit {
90 | 58 . . . . . . . . . ValuePos: 6:15
91 | 59 . . . . . . . . . Kind: STRING
92 | 60 . . . . . . . . . Value: "\"hello world\""
93 | 61 . . . . . . . . }
94 | 62 . . . . . . . }
95 | 63 . . . . . . . Ellipsis: -
96 | 64 . . . . . . . Rparen: 6:28
97 | 65 . . . . . . }
98 | 66 . . . . . }
99 | 67 . . . . }
100 | 68 . . . . Rbrace: 7:1
101 | 69 . . . }
102 | 70 . . }
103 | 71 . }
104 | 72 . Scope: *ast.Scope {
105 | 73 . . Objects: map[string]*ast.Object (len = 1) {
106 | 74 . . . "main": *(obj @ 27)
107 | 75 . . }
108 | 76 . }
109 | 77 . Imports: []*ast.ImportSpec (len = 1) {
110 | 78 . . 0: *(obj @ 12)
111 | 79 . }
112 | 80 . Unresolved: []*ast.Ident (len = 1) {
113 | 81 . . 0: *(obj @ 46)
114 | 82 . }
115 | 83 }
116 | ```
117 |
118 | Let's focus on the JSON under [`*ast.File`](https://golang.org/pkg/go/ast/#File) representing a Go source file. The file is the root node, and it contains all
119 | the top-level declarations in the file - the import and the main function. Under `mains'` body, we have a
120 | [`*ast.blockStmt`](https://golang.org/pkg/go/ast/#BlockStmt) containing a list of the function statements. Similar to HTML, the dependency of the nodes create a
121 | tree-like structure.
122 |
123 | The syntax is "abstract" in the sense that it does not represent every detail appearing in the real syntax, but rather
124 | just the structural or content-related details. For instance, grouping parentheses are implicit in the tree structure,
125 | so these are not represented as separate nodes.
126 |
127 | ### 2.3 go/ast Package members
128 | The AST package contains the types used to represent syntax trees in Go. We can divide the members into three categories:
129 | Interfaces, concrete types, and others.
130 |
131 | - Concrete Types: The full list is [long](https://golang.org/pkg/go/ast/#ArrayType). Those are the tree nodes, and they contain values such as: `FuncDecl`, `IncDecStmt`, `Ident`, `Comment`, and so on.
132 | - Interfaces: `Node`, `Decl`, `Spec`, `Stmt`, `Expr`
133 | - Others: `Package`, `File`, `Scope`, `Object`
134 |
135 | Well take a look at `AssignStmt`
136 | ```go
137 | type AssignStmt struct {
138 | Lhs []Expr // All the variables to left side of assign operator
139 | TokPos token.Pos // position of operator
140 | Tok token.Token // assignment token. `=`, `:=`, `+=`, `<<=` and so on...
141 | Rhs []Expr // Expressions to right of the assignment operator
142 | }
143 | ```
144 | For example, in the expression: `a := 5`,
145 | - Lhs is [a]
146 | - Rhs is [5]
147 | - TokPos [3] (the position of the ":" character)
148 | - Tok [:=]
149 |
150 | Pretty straight forward. Now we'll look at `Expr` which `AssignStmt` implements. `Expr` is common interface for
151 | everything that returns a value. As you can see, it only contains the node interface (which is implemented by all the nodes on the AST graph).
152 |
153 | ```go
154 | type Expr interface {
155 | Node
156 | // contains filtered or unexported methods
157 | }
158 | ```
159 |
160 | From the other's group we'll look at `File`.
161 | ```go
162 | type File struct {
163 | Doc *CommentGroup // associated documentation; or nil
164 | Package token.Pos // position of "package" keyword
165 | Name *Ident // package name
166 | Decls []Decl // top-level declarations; or nil
167 | Scope *Scope // package scope (this file only)
168 | Imports []*ImportSpec // imports in this file
169 | Unresolved []*Ident // unresolved identifiers in this file
170 | Comments []*CommentGroup // list of all comments in the source file
171 | }
172 |
173 | ```
174 |
175 | An `ast.File` is the root node of each of the files we analyze. When analyzing a program, we'll iterate over the files
176 | and for each we'll pass it to the `ast.Inspect` function for iteration.
177 |
178 | It's worth mentioning again that `ast` package contains only the "abstract" parts so it ignores parentheses, colon, etc...
179 |
180 | ### 2.4 Exercise:
181 | In the folder [`CompilerFrontEndASTInGo/CodeExamples`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/tree/master/CompilerFrontEndASTInGo/CodeExamples)
182 | there are some interesting programs (well... AST-wise). Using our [AST visualizer](http://goast.yuroyoro.net/)
183 | from earlier, take each of the program and look at their AST. I added comments explaining the important points.
184 |
185 | You should start with [`SimpleForLoop`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/CodeExamples/SimpleForLoop.go),
186 | move to [`BadExpr`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/CodeExamples/BadExpr.txt)
187 | and finish with [`DifferentSpecsAndDecls`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/CodeExamples/DifferentSpecsAndDecls.go)
188 |
189 | ### 2.5 Loading a program using the parser
190 | To load the program, we need to parse it first
191 | ``` go
192 | package main
193 |
194 | import (
195 | "fmt"
196 | "go/ast"
197 | "go/parser"
198 | "go/token"
199 | )
200 |
201 | func main() {
202 |
203 | src := `package main
204 |
205 | import "fmt"
206 |
207 | func main() {
208 | fmt.Println("hello world")}
209 | `
210 |
211 | fset := token.NewFileSet()
212 | f, err := parser.ParseFile(fset, "", src, 0)
213 | if err != nil {
214 | fmt.Println(err)
215 | return
216 | }
217 | visitor := func(node ast.Node) bool {
218 | strLit, ok := (node).(*ast.BasicLit)
219 | if ok && strLit.Value == "\"hello world\"" {
220 | pos := fset.Position(strLit.Pos())
221 | fmt.Printf("We found hello world in pos:%d:%d", pos.Line, pos.Column)
222 | return false
223 | }
224 | return true
225 | }
226 | ast.Inspect(f, visitor)
227 | }
228 | ```
229 |
230 |
231 | We first create a [`fileSet`](https://golang.org/pkg/go/token/#FileSet), representing a set of source files. `FileSet` has the properties `files` and `base`
232 | recording the used files and all the files' total size. Using the size property of `token.File`, we can easily determine
233 | in which file a statement is, given its position.
234 |
235 | ```go
236 | fset := token.NewFileSet()
237 | ```
238 |
239 |
240 |
241 | Then, we call the [`parser.ParseFile`](https://golang.org/pkg/go/parser/#ParseFile) function, providing it our `fileSet` to populate it, an empty path, a string as the
242 | source so the parser will use it instead of loading from a file, and a build mode - 0. In this example, we used 0 to
243 | fully load the program, but any other [mode](https://golang.org/pkg/go/parser/#Mode) can be used.
244 |
245 | ```go
246 | f, err := parser.ParseFile(fset, "", src, 0)
247 | if err != nil {
248 | fmt.Println(err)
249 | return
250 | }
251 | ```
252 | > Tip: Instead of iterating file by file, you can load an entire directory using [`parser.ParseDir`](https://golang.org/pkg/go/parser/#ParseDir)
253 |
254 | Finally, we define a visitor function that will be called with each node inside the AST. We pass our function to
255 | [`ast.Inspect`](https://golang.org/pkg/go/ast/#Inspect) to iterate over all the nodes in depth-first order and print a message when we reach the
256 | `Hello World` string literal with it's position in the code. We return `true` each iteration to keep traversing the tree until we found the desired
257 | node. Then, we print our message and return false to indicate we're done searching and to exit the traverse function.
258 |
259 | ```go
260 | visitor := func(node ast.Node) bool {
261 | strLit, ok := (node).(*ast.BasicLit)
262 | if ok && strLit.Value == "\"hello world\"" {
263 | pos := fset.Position(strLit.Pos())
264 | fmt.Printf("We found hello world in pos:%d:%d", pos.Line, pos.Column)
265 | return false
266 | }
267 | return true
268 | }
269 | ast.Inspect(f, visitor)
270 | ```
271 |
272 | ### 2.6 Exercise 2!
273 | In the file [`CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer.go`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer.go)
274 | we have an analyzer that checks if function arguments were modified as in the example below. The problem is that there
275 | are some parts missing from it. There are comments in the places where you should add your code according to the comment.
276 | You can run the tests [`CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer_test.go`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/ArgsOverwriteAnalyzer_test.go)
277 | to make sure your tests pass. You can also debug using the test to inspect the AST graph of this program.
278 |
279 | If you give up, you can see the result in [`CompilerFrontEndASTInGo/result/ArgsOverwriteAnalyzer.go`](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerFrontEndASTInGo/result/ArgsOverwriteAnalyzer.go) :)
280 |
281 |
282 | ### 2.7 Congratulations
283 | You have a good understanding of what AST is, the different Go packages used to create static code analyzers that
284 | interact with it and how to write such analyzers.
285 |
286 | In the [next section](https://github.com/amit-davidson/GopherCon2021IsraelStaticAnalysisWorkshop/blob/master/CompilerMiddleEndSSAInGo)
287 | we'll focus on the middle end level, and see how analyzer "operating" in this level work.
288 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/result/ArgsOverwriteAnalyzer.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "fmt"
5 | "go/ast"
6 | "go/importer"
7 | "go/parser"
8 | "go/token"
9 | "go/types"
10 | "log"
11 | "os"
12 | )
13 |
14 | func main() {
15 | path := os.Args[1]
16 | fset, pkgs, err := loadProgram(path)
17 | if err != nil {
18 | log.Fatalf("Failed to parse dir %s: %s", path, err)
19 | }
20 |
21 | for _, pkg := range pkgs {
22 | outputs := analyzePackage(pkg, fset)
23 | for _, message := range outputs {
24 | fmt.Println(message)
25 | }
26 | }
27 | }
28 |
29 | func loadProgram(path string) (*token.FileSet, map[string]*ast.Package, error) {
30 | fset := token.NewFileSet()
31 | pkgs, err := parser.ParseDir(fset, path, nil, 0)
32 | if err != nil {
33 | return nil, nil, err
34 | }
35 | return fset, pkgs, nil
36 | }
37 |
38 | func populateTypes(conf types.Config, fset *token.FileSet, f *ast.File) (*types.Info, error) {
39 | info := &types.Info{
40 | Defs: make(map[*ast.Ident]types.Object),
41 | Uses: make(map[*ast.Ident]types.Object),
42 | }
43 | _, err := conf.Check("", fset, []*ast.File{f}, info)
44 | if err != nil {
45 | return nil, err
46 | }
47 | return info, nil
48 | }
49 |
50 | func analyzePackage(p *ast.Package, fset *token.FileSet) []string {
51 | var info *types.Info
52 | var err error
53 | outputs := make([]string, 0)
54 |
55 | visitor := func(node ast.Node) bool {
56 | var typ *ast.FuncType
57 | var body *ast.BlockStmt
58 | switch fn := node.(type) {
59 | case *ast.FuncDecl: // Regular function
60 | typ = fn.Type
61 | body = fn.Body
62 | case *ast.FuncLit: // Anonymous function
63 | typ = fn.Type
64 | body = fn.Body
65 | }
66 | if typ == nil || body == nil { // Exclude other types but also external functions with missing body
67 | return true
68 | }
69 | if len(typ.Params.List) == 0 {
70 | return true
71 | }
72 |
73 | for _, field := range typ.Params.List {
74 | for _, arg := range field.Names {
75 | obj := info.ObjectOf(arg)
76 | ast.Inspect(body, func(node ast.Node) bool {
77 | switch stmt := node.(type) {
78 | case *ast.AssignStmt:
79 | for _, lhs := range stmt.Lhs {
80 | ident, ok := lhs.(*ast.Ident)
81 | if ok && info.ObjectOf(ident) == obj {
82 | outputs = append(outputs, output(ident, fset.Position(ident.Pos())))
83 | }
84 | }
85 | case *ast.IncDecStmt:
86 | ident, ok := stmt.X.(*ast.Ident)
87 | if ok && info.ObjectOf(ident) == obj {
88 | outputs = append(outputs, output(ident, fset.Position(ident.Pos())))
89 | }
90 | }
91 | return true
92 | })
93 | }
94 | }
95 | return true
96 | }
97 |
98 | conf := types.Config{Importer: importer.Default()}
99 | for _, f := range p.Files {
100 | info, err = populateTypes(conf, fset, f)
101 | if err != nil {
102 | log.Fatal(err)
103 | }
104 | ast.Inspect(f, visitor)
105 | }
106 | return outputs
107 | }
108 |
109 | func output(ident *ast.Ident, pos token.Position) string {
110 | return fmt.Sprintf("\"%s\" overwrites func parameter in pos: %s:%d:%d", ident.Name, pos.Filename, pos.Line, pos.Column)
111 | }
112 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/AnonymousFunction/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func main() {
4 | _ = func(a int) {
5 | a = 5
6 | }
7 | }
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/AssigningParamToAVariableFirst/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a int) {
4 | b := a
5 | b = 5
6 | _ = b
7 | }
8 |
9 | func main() {
10 | body(5)
11 | }
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/DecrementOperator/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | func retry(fn func() error, retries int) error {
4 | for {
5 | if err := fn(); err != nil {
6 | if retries < 1 {
7 | return err
8 | }
9 | retries--
10 | continue
11 | }
12 | return nil
13 | }
14 | }
15 |
16 | func main() {
17 | _ = retry(func() error {
18 | return nil
19 | }, 5)
20 | }
21 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/EmptyBodyFunction/add_amd64.s:
--------------------------------------------------------------------------------
1 | TEXT ·Add(SB),$0-24
2 | MOVQ a+0(FP), AX
3 | ADDQ b+8(FP), AX
4 | MOVQ AX, ret+16(FP)
5 | RET
6 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/EmptyBodyFunction/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | // Add returns the sum of a and b.
4 | func Add(a int64, b int64) int64
5 |
6 | func body(a int) {
7 | b := a
8 | b = 5
9 | _ = b
10 | }
11 |
12 | func main() {
13 | body(5)
14 | _ = Add(3, 4)
15 | }
16 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/LhsMultipleValues/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func main() {
4 | _ = func(a, b int) {
5 | a, b = 5, 7
6 | }
7 | }
8 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/MultipleParamsOfSameType/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a, b int, c int) {
4 | a = 5
5 | _ = b
6 | c = 3
7 | }
8 |
9 | func main() {
10 | body(1, 2, 3)
11 | }
12 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/NoWarnings/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | import "fmt"
4 |
5 | func closeBody(body int) {
6 | fmt.Print(body)
7 | }
8 |
9 | func main() {
10 | closeBody(1)
11 | }
12 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/OverwritingParamFromOuterScope/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a int) {
4 | _ = func() {
5 | a = 5
6 | }
7 | }
8 |
9 | func main() {
10 | body(5)
11 | }
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/ShadowingVariable/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func body(a, b int, c int) {
4 | f := func(a int, b int) {
5 | a = 5
6 | }
7 | _ = f
8 | }
9 |
10 | func main() {
11 | body(1, 2, 3)
12 | }
13 |
--------------------------------------------------------------------------------
/CompilerFrontEndASTInGo/testdata/SimpleOverwriting/main.go:
--------------------------------------------------------------------------------
1 | package testdata
2 |
3 | func closeBody(body int) {
4 | body = 1
5 | }
6 |
7 | func main() {
8 | closeBody(1)
9 | }
10 |
--------------------------------------------------------------------------------
/CompilerMiddleEndSSAInGo/CodeExamples/ElseIf/ElseIf.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "os"
5 | "strconv"
6 | )
7 |
8 | var a int
9 |
10 | func main() {
11 | a, _ = strconv.Atoi(os.Args[1])
12 | if a > 0 {
13 | a = 3
14 | } else if a == 0 {
15 | a = 4
16 | } else {
17 | a = 5
18 | }
19 | }
20 |
21 | // 1. In the source code, the condition can go to the if, the else if or the else blocks. Here, each block can have up to 2
22 | // successors. Notice that the first block can only jump to 1 (a=3) or 3 (a==0) and 3 evaluates the condition again and can go to
23 | // 4 (a=4) or 5(a=5)
24 |
25 | // 2. `a` is declared in the global scope (and not under the function scope). Because it's in the global scope, it might
26 | // be used in other places, so the compiler isn't sure if it could have side effects, (not considered dead) so it can't
27 | // optimize away it using dead code elimination.
28 |
--------------------------------------------------------------------------------
/CompilerMiddleEndSSAInGo/CodeExamples/Map/Map.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | func main() {
4 |
5 | m := make(map[string]int)
6 |
7 | m["k1"] = 7
8 | m["k2"] = 13
9 |
10 | v1 := m["k1"]
11 | _ = v1
12 |
13 | _, prs := m["k2"]
14 | _ = prs
15 |
16 | delete(m, "k2")
17 | }
18 |
19 | // See how different map operations are carried out in SSA:
20 | // 1. Creating a map
21 | // 2. Assigning a value to a map
22 | // 3. Reading from a map
23 | // 4. Reading from a map into multiple variables
24 | // 5. Deleting a key from a map
25 |
--------------------------------------------------------------------------------
/CompilerMiddleEndSSAInGo/README.md:
--------------------------------------------------------------------------------
1 | ## 3. Compiler middle end and static analysis with SSA In Go
2 | ### 3.1 What is SSA?
3 | [SSA](https://en.wikipedia.org/wiki/Static_single_assignment_form) stands for static single assignment. It's a property of an IR **that requires each variable to be assigned exactly once**, and every variable be defined before it is used.
4 | The primary usefulness of SSA comes from how it simplifies the properties of variables and improves compilers optimizations.
5 |
6 | For example, consider this piece of code:
7 | ```
8 | y := 1
9 | y := 2
10 | x := y
11 | ```
12 |
13 | In an SSA form, it'll be translated to:
14 | ```
15 | y1 := 1
16 | y2 := 2
17 | x1 := y2
18 | ```
19 | Humans can see that the first assignment is unnecessary and that the value of `y` used in the third line comes from the
20 | second assignment of `y`. In SSA form, both of these are immediate
21 |
22 | ### 3.2 go/ssa package members
23 | The package [`tools/go/ssa`](https://pkg.go.dev/golang.org/x/tools/go/ssa) defines the representation of elements of Go programs in SSA format.
24 | The key types form a hierarchical structure.
25 |
26 | #### [Program](https://pkg.go.dev/golang.org/x/tools/go/ssa#Program)
27 | A Program is a partial or complete Go program converted to an SSA form.
28 |
29 |
30 |
31 | #### [Package](https://pkg.go.dev/golang.org/x/tools/go/ssa#Package)
32 | A Package is a single analyzed Go package containing Members for all package-level functions, variables, constants, and types it declares.
33 |
34 |
35 |
36 | #### [Function](https://pkg.go.dev/golang.org/x/tools/go/ssa#Function)
37 | Function represents the parameters, results, and code of a function or method.
38 |
39 |
40 |
41 | #### [Basic Block](https://pkg.go.dev/golang.org/x/tools/go/ssa#BasicBlock)
42 | BasicBlock represents an SSA basic block. A set of instructions that are executed and can't jump somewhere else. Basic blocks are connected using conditions and goto statements.
43 |
44 |
45 |
46 | Control Flow Graph (CFG) - In a control-flow graph, each node in the graph represents a basic block.
47 | Together, they compose all paths that might be traversed through a program during its execution.
48 |
49 |
50 |
51 | #### [Instruction](https://pkg.go.dev/golang.org/x/tools/go/ssa#Instruction)
52 | a statement that consumes values and performs computation. For example, `Call`, `Return`, `TypeAssert`, etc
53 |
54 |
55 |
56 | #### [Value](https://pkg.go.dev/golang.org/x/tools/go/ssa#Value)
57 | an expression that yields a value. For example, function calls are both `Instruction` and `Value` since they both consume values and yield a value.
58 |
59 |
60 |
61 | And when combined:
62 |
63 |
64 |
65 | The package contains other [types](https://pkg.go.dev/golang.org/x/tools/go/ssa#pkg-overview) - Include language keywords such as `Defer`, `If` but also lower level primitives like `MakeChan` and `Alloc`.
66 |
67 |
68 | ### 3.3 Overviewing an analyzer!
69 | In this section we'll look over an analyzer that warns when `t.Fatal` is used inside a goroutine as described here:
70 | https://github.com/ipfs/go-ipfs/issues/2043
71 |
72 | ### 3.4 SSA vs AST
73 | The most important difference is that AST reasons about the structure of the code, where SSA reasons about how data
74 | flows in the code. Why do need both? Each "level" suits for a different problem. You can think of it as satellite vs
75 | terrain modes on maps. They both represent the same source map, but each mode solves a different problem.
76 |
77 | We can summarize the differences using the following table:
78 | | | SSA | AST |
79 | |----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
80 | | Why to Choose? |