├── practise ├── gorm-practise │ ├── README.md │ ├── update.go │ ├── models │ │ └── user.go │ ├── main.go │ └── dbstone │ │ ├── base.go │ │ └── user.go ├── cobra-practise │ ├── pixiuctl │ │ ├── pixiuctl-test │ │ ├── pixiuctl.go │ │ ├── util │ │ │ └── util.go │ │ ├── README.md │ │ ├── create │ │ │ ├── create_service.go │ │ │ └── create.go │ │ ├── apply │ │ │ └── apply.go │ │ └── plugin │ │ │ └── plugin.go │ └── demo-server │ │ ├── democonfig.yaml │ │ ├── demoserver.go │ │ ├── dao │ │ ├── factory.go │ │ └── db.go │ │ └── app │ │ ├── config │ │ └── config.go │ │ ├── options │ │ ├── configfile.go │ │ └── options.go │ │ └── server.go ├── gin-practise │ ├── tls │ │ ├── cert │ │ │ ├── tls.crt │ │ │ └── tls.key │ │ ├── README.md │ │ └── main.go │ ├── practise-log.log │ ├── practise-access.log │ ├── endpoint │ │ ├── optimise.go │ │ └── endpoint.go │ ├── hander │ │ └── hanlder.go │ ├── log │ │ ├── logging.go │ │ └── zaplogging.go │ ├── worker │ │ └── worker.go │ ├── optimise │ │ └── optimise.go │ ├── workqueue │ │ └── queue.go │ ├── middleware │ │ └── middleware.go │ ├── main.go │ └── k8s │ │ └── k8s.go ├── test-practise │ ├── add.go │ ├── README.md │ └── add_test.go ├── klog-practise │ ├── test │ │ └── test.go │ └── main.go ├── image-practise │ ├── config.yaml │ ├── image │ │ └── option.go │ └── main.go ├── template-practise │ ├── template │ │ ├── common.go │ │ └── template.go │ ├── service.yaml │ ├── merge-service.yaml │ └── main.go ├── decorator-practise.go ├── rbac-practise │ ├── model.conf │ └── main.go ├── http-practise │ ├── Dockerfile │ └── http-learning.go ├── loadconfig-practise │ ├── test.yaml │ ├── loadyaml.go │ ├── config │ │ └── config.go │ └── loadfile.go ├── excelize-practise │ ├── main.go │ └── excel │ │ └── excel.go ├── syscall-practise.go ├── grpc │ ├── tunnel │ │ ├── tunnel.proto │ │ └── tunnel_grpc.pb.go │ ├── client.go │ ├── README.md │ └── server.go ├── k8s-practise │ ├── app │ │ ├── client.go │ │ └── helper.go │ ├── node-metrics.go │ ├── annotation.go │ ├── localstorage.go │ ├── events.go │ ├── proxy.go │ ├── metrics.go │ ├── gin-informer.go │ ├── scale.go │ ├── gin-informer2.go │ ├── clients.go │ ├── infomer-practise.go │ └── monitor.go ├── grpc-practise │ ├── pixiu │ │ ├── pixiu.proto │ │ └── pixiu_grpc.pb.go │ ├── client │ │ └── client.go │ ├── server │ │ └── server.go │ ├── csiclient │ │ └── main.go │ └── README.md ├── error-practise.go ├── interface-practise │ ├── v1 │ │ └── pixiu.go │ ├── v2 │ │ └── scaler.go │ ├── test.go │ ├── people.go │ └── main.go ├── sync-practise │ ├── cond.go │ └── waitgroup.go ├── configflags-practise.go ├── logrus-practise.go ├── containerd-practise │ └── main.go ├── checker.go ├── etcd-practise.go ├── exec-practise │ └── main.go ├── http-practise.go ├── fsnotify-practise │ └── main.go ├── sftp-practise.go ├── gorestful-practise.go └── ws-practise.go ├── algorithm └── README.rst ├── examples ├── informer │ ├── README.md │ └── main.go ├── services │ ├── oncluster │ │ ├── README.md │ │ └── main.go │ └── incluster │ │ ├── Dockerfile │ │ ├── README.md │ │ ├── main.yaml │ │ └── main.go ├── README.md └── workqueue │ └── README.md ├── doc ├── kubernetes │ ├── scheduler-schedule.md │ ├── scheduler-start.md │ ├── pictures │ │ ├── cni0.png │ │ ├── crd.jpeg │ │ ├── lb.png │ │ ├── masq.png │ │ ├── return.png │ │ ├── router.png │ │ ├── endpoints.png │ │ ├── headless2.png │ │ ├── nodednat.png │ │ ├── nodeport.png │ │ ├── kube-post2.png │ │ ├── node-route3.png │ │ ├── nodeportcni.png │ │ ├── postrouting.png │ │ ├── tablesflow.png │ │ ├── KUBE-MARK-MASQ.png │ │ ├── defualt-router.png │ │ ├── iptablesflow.png │ │ ├── kube-forward.png │ │ ├── kube-forward2.png │ │ ├── kube-nodeport.png │ │ ├── kube-services.png │ │ ├── node-port-back.png │ │ ├── nodeport-back2.png │ │ ├── nodeport-mark.png │ │ ├── headless-service.png │ │ ├── kube-postrouting.png │ │ ├── node-port-filter.png │ │ ├── clusterprerouting.png │ │ ├── nodeport-masqerade.png │ │ ├── KUBE-SEP-EZ6FKUNEIONYFN4Z.png │ │ └── KUBE-SVC-4N57TFCL4MD7ZTDA.png │ ├── code-generator.md │ ├── controller.md │ ├── operator.md │ ├── csi.md │ └── network.md └── learning.md ├── .gitignore ├── shares └── 20220825 │ ├── demo.go │ ├── README.md │ └── parameter.go ├── hack ├── update-gofmt.sh └── verify-gofmt.sh ├── .github └── workflows │ └── main.yml ├── README.md └── go.mod /practise/gorm-practise/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /algorithm/README.rst: -------------------------------------------------------------------------------- 1 | # Algorithm 2 | -------------------------------------------------------------------------------- /examples/informer/README.md: -------------------------------------------------------------------------------- 1 | # Running 2 | -------------------------------------------------------------------------------- /doc/kubernetes/scheduler-schedule.md: -------------------------------------------------------------------------------- 1 | # Scheduler 的调度分析 2 | 3 | TODO -------------------------------------------------------------------------------- /doc/kubernetes/scheduler-start.md: -------------------------------------------------------------------------------- 1 | # Scheduler 的启动分析 2 | 3 | TODO 4 | -------------------------------------------------------------------------------- /examples/services/oncluster/README.md: -------------------------------------------------------------------------------- 1 | go build main.go 2 | ./main 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Files generated by JetBrains 2 | .idea/ 3 | .DS_Store 4 | .vscode/ 5 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/pixiuctl-test: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo test 4 | -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | # Go-learninng Examples 2 | 3 | 4 | 5 | ### 高级概念 6 | - [**Work queues**](./workqueue) -------------------------------------------------------------------------------- /practise/gin-practise/tls/cert/tls.crt: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | ... 3 | -----END CERTIFICATE----- 4 | -------------------------------------------------------------------------------- /practise/gin-practise/tls/cert/tls.key: -------------------------------------------------------------------------------- 1 | -----BEGIN PRIVATE KEY----- 2 | ... 3 | -----END PRIVATE KEY----- 4 | -------------------------------------------------------------------------------- /practise/test-practise/add.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | func Add(x, y int) int { 4 | return x + y 5 | } 6 | -------------------------------------------------------------------------------- /practise/gin-practise/practise-log.log: -------------------------------------------------------------------------------- 1 | 2021-09-11 23:28:07 info endpoint/endpoint.go:48 Post Practise log 2 | -------------------------------------------------------------------------------- /doc/kubernetes/pictures/cni0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/cni0.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/crd.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/crd.jpeg -------------------------------------------------------------------------------- /doc/kubernetes/pictures/lb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/lb.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/masq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/masq.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/return.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/return.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/router.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/router.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/endpoints.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/endpoints.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/headless2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/headless2.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodednat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodednat.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodeport.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodeport.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-post2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-post2.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/node-route3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/node-route3.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodeportcni.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodeportcni.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/postrouting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/postrouting.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/tablesflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/tablesflow.png -------------------------------------------------------------------------------- /practise/test-practise/README.md: -------------------------------------------------------------------------------- 1 | # 单元测试 2 | 3 | ### 基本规则 4 | - 测试文件以 `_test.go` 结尾 5 | - 测试函数名必须以 `Test` 开始 6 | - 测试以 `go test` 命令开始 7 | -------------------------------------------------------------------------------- /doc/kubernetes/pictures/KUBE-MARK-MASQ.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/KUBE-MARK-MASQ.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/defualt-router.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/defualt-router.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/iptablesflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/iptablesflow.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-forward.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-forward2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-forward2.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-nodeport.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-nodeport.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-services.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-services.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/node-port-back.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/node-port-back.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodeport-back2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodeport-back2.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodeport-mark.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodeport-mark.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/headless-service.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/headless-service.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/kube-postrouting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/kube-postrouting.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/node-port-filter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/node-port-filter.png -------------------------------------------------------------------------------- /practise/klog-practise/test/test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import klog "k8s.io/klog/v2" 4 | 5 | func Test1() { 6 | klog.InfoS("内部 aaaa") 7 | } 8 | -------------------------------------------------------------------------------- /doc/kubernetes/pictures/clusterprerouting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/clusterprerouting.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/nodeport-masqerade.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/nodeport-masqerade.png -------------------------------------------------------------------------------- /examples/services/incluster/Dockerfile: -------------------------------------------------------------------------------- 1 | From ubuntu:14.04 2 | COPY app /root/app 3 | RUN chmod +x /root/app 4 | WORKDIR /root 5 | ENTRYPOINT ["/root/app"] 6 | -------------------------------------------------------------------------------- /doc/kubernetes/pictures/KUBE-SEP-EZ6FKUNEIONYFN4Z.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/KUBE-SEP-EZ6FKUNEIONYFN4Z.png -------------------------------------------------------------------------------- /doc/kubernetes/pictures/KUBE-SVC-4N57TFCL4MD7ZTDA.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/caoyingjunz/go-learning/HEAD/doc/kubernetes/pictures/KUBE-SVC-4N57TFCL4MD7ZTDA.png -------------------------------------------------------------------------------- /practise/gin-practise/practise-access.log: -------------------------------------------------------------------------------- 1 | 2021-09-11 23:28:07 info middleware/middleware.go:38 | 200 | 978.344µs | 127.0.0.1 | POST | /practise/post | 2 | -------------------------------------------------------------------------------- /practise/image-practise/config.yaml: -------------------------------------------------------------------------------- 1 | default: 2 | push_kubernetes: false 3 | push_images: false 4 | 5 | kubernetes: 6 | version: v1.23.6 7 | 8 | images: 9 | - docker.io/nginx:latest 10 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/democonfig.yaml: -------------------------------------------------------------------------------- 1 | default: 2 | name: caoyingjun 3 | 4 | mysql: 5 | user: caoyingjun 6 | password: caoyingjun 7 | host: 8.8.8.8 8 | port: 3306 9 | name: test 10 | -------------------------------------------------------------------------------- /examples/services/incluster/README.md: -------------------------------------------------------------------------------- 1 | GOOS=linux go build -o ./app . 2 | docker build . 3 | kubectl apply -f main.yaml 4 | kubectl create clusterrolebinding default-view --clusterrole=view --serviceaccount=default:default 5 | -------------------------------------------------------------------------------- /practise/template-practise/template/common.go: -------------------------------------------------------------------------------- 1 | package template 2 | 3 | const ( 4 | CommonService = `common: 5 | apiVersion: {{ apiVersion }} 6 | kind: "{{ kind }}" 7 | metadata: { metadata } 8 | ` 9 | ) 10 | -------------------------------------------------------------------------------- /practise/test-practise/add_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import "testing" 4 | 5 | func TestAdd(t *testing.T) { 6 | x, y := 1, 2 7 | z := Add(x, y) 8 | if z != 3 { 9 | t.Errorf("Add(%d,%d)=%d,real=%d", x, y, z, 3) 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /examples/workqueue/README.md: -------------------------------------------------------------------------------- 1 | # Workqueue Example 2 | 3 | The example is based on https://git.k8s.io/community/contributors/devel/sig-api-machinery/controllers.md. 4 | 5 | ## Running 6 | 7 | ``` 8 | go run main.go -kubeconfig=/your/kubeconfig -logtostderr=true 9 | ``` 10 | -------------------------------------------------------------------------------- /practise/decorator-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | ) 6 | 7 | func TestA(a, b string, f func()) { 8 | fmt.Println(a) 9 | f() 10 | fmt.Println(b) 11 | } 12 | 13 | func main() { 14 | TestA("before", "end", func() { 15 | fmt.Println("中间执行") 16 | }) 17 | } 18 | -------------------------------------------------------------------------------- /practise/template-practise/service.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | apiVersion: v1 3 | kind: Service 4 | metadata: 5 | name: caoyingjun 6 | spec: 7 | emails: 8 | - test1@gmail.com 9 | - test2@gmail.com 10 | selector: 11 | app: name 12 | m1: v1 13 | m2: v2 14 | name: caoyingjunz 15 | age: "18" 16 | -------------------------------------------------------------------------------- /shares/20220825/demo.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "net/http" 5 | 6 | "github.com/gin-gonic/gin" 7 | ) 8 | 9 | func main() { 10 | //gin.SetMode(gin.ReleaseMode) 11 | 12 | r := gin.Default() 13 | r.GET("/ping", func(c *gin.Context) { 14 | c.JSON(http.StatusOK, gin.H{"message": "pong"}) 15 | }) 16 | 17 | _ = r.Run() 18 | } 19 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/demoserver.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "k8s.io/klog/v2" 5 | 6 | "go-learning/practise/cobra-practise/demo-server/app" 7 | ) 8 | 9 | func main() { 10 | cmd := app.NewDemoCommand() 11 | 12 | if err := cmd.Execute(); err != nil { 13 | klog.Fatalf("exec demo server failed %v", err) 14 | } 15 | } 16 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/pixiuctl.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // refer to https://github.com/spf13/cobra 4 | import ( 5 | "os" 6 | 7 | "go-learning/practise/cobra-practise/pixiuctl/app" 8 | ) 9 | 10 | func main() { 11 | cmd := app.NewDefaultPixiuCommand() 12 | 13 | if err := cmd.Execute(); err != nil { 14 | os.Exit(1) 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /practise/gin-practise/tls/README.md: -------------------------------------------------------------------------------- 1 | # How to Test 2 | 3 | ## Generate RSA private key and digital certificate 4 | 5 | ## Starting server with tls 6 | ```shell 7 | go run main.go 8 | ``` 9 | 10 | ## Curl with cert 11 | ```shell 12 | curl --cacert ./cert/tls.crt https://127.0.0.1:443/welcome 13 | 14 | # 正常回显 15 | {"status":"success"} 16 | ``` 17 | -------------------------------------------------------------------------------- /practise/rbac-practise/model.conf: -------------------------------------------------------------------------------- 1 | [request_definition] 2 | r = sub, obj, act 3 | 4 | [policy_definition] 5 | p = sub, obj, act 6 | 7 | [role_definition] 8 | g = _, _ 9 | 10 | [policy_effect] 11 | e = some(where (p.eft == allow)) 12 | 13 | [matchers] 14 | m = g(r.sub, p.sub) && keyMatch2(r.obj, p.obj) && regexMatch(r.act, p.act) || r.sub == "21220821" 15 | -------------------------------------------------------------------------------- /practise/gin-practise/tls/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "net/http" 5 | 6 | "github.com/gin-gonic/gin" 7 | ) 8 | 9 | func main() { 10 | r := gin.Default() 11 | r.GET("/welcome", func(c *gin.Context) { 12 | c.JSON(http.StatusOK, gin.H{ 13 | "status": "success", 14 | }) 15 | }) 16 | 17 | r.RunTLS(":443", "./cert/tls.crt", "./cert/tls.key") 18 | } 19 | -------------------------------------------------------------------------------- /practise/http-practise/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang:1.15-alpine3.12 as builder 2 | ARG GOPROXY 3 | ARG APP 4 | ENV GOPROXY=${GOPROXY} 5 | WORKDIR /go/pixiu 6 | COPY http-learning.go . 7 | RUN CGO_ENABLED=0 go build -a -o httpd http-learning.go 8 | 9 | FROM jacky06/static:nonroot 10 | ARG APP 11 | WORKDIR / 12 | COPY --from=builder /go/pixiu/httpd /usr/local/bin/httpd 13 | USER root:root -------------------------------------------------------------------------------- /practise/loadconfig-practise/test.yaml: -------------------------------------------------------------------------------- 1 | cache: 2 | enable : false 3 | list : [redis,mongoDB] 4 | mysql: 5 | user : root 6 | password : Tech2501 7 | host : 8.8.8.8 8 | port : 3306 9 | name : cwi 10 | student: 11 | caoyingjun1: 12 | school: zhiyuan 13 | age: 18 14 | caoyingjun2: 15 | school: zhiyuan 16 | age: 19 17 | caoyingjun3: 18 | school: sizhong 19 | name: yamltest 20 | -------------------------------------------------------------------------------- /practise/excelize-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "go-learning/practise/excelize-practise/excel" 5 | ) 6 | 7 | func main() { 8 | f := excel.NewExcelFile() 9 | defer f.Close() 10 | 11 | f.SetSheetName("Sheet1", "SheetName") 12 | 13 | _ = f.SetCellSlice([]string{"name1", "name2", "name3", "name4"}) 14 | _ = f.SetCellSlice([]string{"1", "2", "3", "4"}) 15 | 16 | f.SaveAs("Book1.xlsx") 17 | } 18 | -------------------------------------------------------------------------------- /practise/template-practise/merge-service.yaml: -------------------------------------------------------------------------------- 1 | common: 2 | apiVersion: {{ apiVersion }} 3 | kind: "{{ kind }}" 4 | metadata: { metadata } 5 | --- 6 | --- 7 | apiVersion: v1 8 | kind: Service 9 | metadata: 10 | name: caoyingjun 11 | spec: 12 | emails: 13 | - test1@gmail.com 14 | - test2@gmail.com 15 | selector: 16 | app: name 17 | m1: v1 18 | m2: v2 19 | name: caoyingjunz 20 | age: "18" 21 | -------------------------------------------------------------------------------- /practise/syscall-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "os/exec" 7 | "syscall" 8 | ) 9 | 10 | func main() { 11 | binary, err := exec.LookPath("pixiuctl-test") 12 | if err != nil { 13 | panic(err) 14 | } 15 | 16 | args := []string{"pixiuctl-test", "test"} 17 | 18 | if err = syscall.Exec(binary, args, os.Environ()); err != nil { 19 | panic(err) 20 | } 21 | 22 | fmt.Println("end") // 不会被执行 23 | } 24 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/dao/factory.go: -------------------------------------------------------------------------------- 1 | package dao 2 | 3 | import "github.com/jinzhu/gorm" 4 | 5 | type ShareDBFactory interface { 6 | Test() TestInterface 7 | } 8 | 9 | type shareDBFactory struct { 10 | db *gorm.DB 11 | } 12 | 13 | func (f *shareDBFactory) Test() TestInterface { 14 | return NewTest(f, f.db) 15 | } 16 | 17 | func NewDBFactory(db *gorm.DB) ShareDBFactory { 18 | return &shareDBFactory{ 19 | db: db, 20 | } 21 | } 22 | -------------------------------------------------------------------------------- /practise/grpc/tunnel/tunnel.proto: -------------------------------------------------------------------------------- 1 | syntax="proto3"; 2 | 3 | option go_package = "go-learning/practise/grpc-practise/tunnel/tunnel"; 4 | 5 | package tunnel; 6 | 7 | service Tunnel { 8 | // Client 调用此方法建立连接 9 | rpc Connect(stream Request) returns (stream Response); 10 | } 11 | 12 | message Request { 13 | string type = 1; // "client_call" 或 "server_call" 14 | bytes payload = 2; 15 | } 16 | 17 | message Response { 18 | bytes result = 1; 19 | } 20 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/app/config/config.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | type Config struct { 4 | Default DefaultConfig 5 | Mysql MysqlConfig 6 | } 7 | 8 | type DefaultConfig struct { 9 | Name string `yaml:"name"` 10 | } 11 | 12 | type MysqlConfig struct { 13 | User string `yaml:"user"` 14 | Password string `yaml:"password"` 15 | Host string `yaml:"host"` 16 | Port string `yaml:"port"` 17 | Name string `yaml:"name"` 18 | } 19 | -------------------------------------------------------------------------------- /practise/image-practise/image/option.go: -------------------------------------------------------------------------------- 1 | package image 2 | 3 | type Config struct { 4 | Default DefaultOption `yaml:"default"` 5 | Kubernetes KubernetesOption `yaml:"kubernetes"` 6 | Images []string `yaml:"images"` 7 | } 8 | 9 | type DefaultOption struct { 10 | PushKubernetes bool `yaml:"push_kubernetes"` 11 | PushImages bool `yaml:"push_images"` 12 | } 13 | 14 | type KubernetesOption struct { 15 | Version string `yaml:"version"` 16 | } 17 | -------------------------------------------------------------------------------- /examples/services/incluster/main.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: k8s-example3 5 | labels: 6 | app: k8s-example3 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: k8s-example3 12 | template: 13 | metadata: 14 | labels: 15 | app: k8s-example3 16 | spec: 17 | containers: 18 | - name: k8s-example3 19 | image: k8s/example3:latest 20 | imagePullPolicy: IfNotPresent 21 | -------------------------------------------------------------------------------- /practise/gorm-practise/update.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "fmt" 7 | "go-learning/practise/gorm-practise/dbstone" 8 | ) 9 | 10 | func main() { 11 | udb := dbstone.NewUserDB() 12 | 13 | // 以使用乐观锁的方式更新 user 14 | updates := make(map[string]interface{}) 15 | updates["age"] = 19 16 | if err := udb.OptimisticUpdate(context.TODO(), "caoyingjun", 6, updates); err != nil { 17 | if errors.Is(err, dbstone.RecordNotUpdated) { 18 | fmt.Println("hahaha not updated") 19 | } 20 | } 21 | } 22 | -------------------------------------------------------------------------------- /practise/loadconfig-practise/loadyaml.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | 7 | "gopkg.in/yaml.v2" 8 | 9 | "go-learning/practise/loadconfig-practise/config" 10 | ) 11 | 12 | // refer to https://github.com/go-yaml/yaml 13 | 14 | func main() { 15 | data, err := ioutil.ReadFile("test.yaml") 16 | if err != nil { 17 | panic(err) 18 | } 19 | 20 | config := new(config.PixiuConfiguration) 21 | if err = yaml.Unmarshal(data, config); err != nil { 22 | panic(err) 23 | } 24 | 25 | fmt.Println(config) 26 | } 27 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/util/util.go: -------------------------------------------------------------------------------- 1 | package util 2 | 3 | import ( 4 | "sync" 5 | ) 6 | 7 | type Factory interface { 8 | Client() string 9 | } 10 | 11 | type factoryImpl struct { 12 | client string 13 | 14 | // Caches OpenAPI document and parsed resources 15 | parser sync.Once 16 | getter sync.Once 17 | } 18 | 19 | func (f *factoryImpl) Client() string { 20 | return f.client 21 | } 22 | 23 | func NewFactory(kubeconfig string) Factory { 24 | c := kubeconfig 25 | f := &factoryImpl{ 26 | client: c, 27 | } 28 | 29 | return f 30 | } 31 | -------------------------------------------------------------------------------- /practise/k8s-practise/app/client.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "path/filepath" 5 | 6 | restclient "k8s.io/client-go/rest" 7 | "k8s.io/client-go/tools/clientcmd" 8 | "k8s.io/client-go/util/homedir" 9 | ) 10 | 11 | func BuildClientConfig(configFile string) (*restclient.Config, error) { 12 | if len(configFile) == 0 { 13 | configFile = filepath.Join(homedir.HomeDir(), ".kube", "config") 14 | } 15 | 16 | config, err := clientcmd.BuildConfigFromFlags("", configFile) 17 | if err != nil { 18 | return nil, err 19 | } 20 | return config, nil 21 | } 22 | -------------------------------------------------------------------------------- /practise/template-practise/template/template.go: -------------------------------------------------------------------------------- 1 | package template 2 | 3 | const ( 4 | ServiceTemplate = `--- 5 | apiVersion: v1 6 | kind: Service 7 | metadata: 8 | name: {{ .UserName }} 9 | spec: 10 | {{- if gt (len .Emails) 0 }} 11 | emails: 12 | {{- range .Emails }} 13 | - {{ . }} 14 | {{- end }} 15 | {{- end }} 16 | selector: 17 | {{- with .Friends }} 18 | {{- range . }} 19 | app: {{ .Name }} 20 | {{- end }} 21 | {{- end }} 22 | 23 | {{- range $m, $v := .Mods }} 24 | {{ $m }}: {{ $v }} 25 | {{- end }} 26 | 27 | {{- .Data | nindent 4 }} 28 | ` 29 | ) 30 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/app/options/configfile.go: -------------------------------------------------------------------------------- 1 | package options 2 | 3 | import ( 4 | "os" 5 | 6 | "gopkg.in/yaml.v2" 7 | 8 | "go-learning/practise/cobra-practise/demo-server/app/config" 9 | ) 10 | 11 | func loadConfigFromFile(file string) (*config.Config, error) { 12 | data, err := os.ReadFile(file) 13 | if err != nil { 14 | return nil, err 15 | } 16 | 17 | return loadConfig(data) 18 | } 19 | 20 | func loadConfig(data []byte) (*config.Config, error) { 21 | var c config.Config 22 | if err := yaml.Unmarshal(data, &c); err != nil { 23 | return nil, err 24 | } 25 | 26 | return &c, nil 27 | } 28 | -------------------------------------------------------------------------------- /practise/grpc-practise/pixiu/pixiu.proto: -------------------------------------------------------------------------------- 1 | 2 | syntax="proto3"; 3 | 4 | option go_package = "go-learning/practise/grpc-practise/pixiu/pixiu"; 5 | 6 | // protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative pixiu/pixiu.proto 7 | 8 | package pixiu; 9 | 10 | service Pixiu { 11 | rpc GetPixiu (PixiuRequest) returns (PixiuReply) {} 12 | } 13 | 14 | // The request message containing the user's name. 15 | message PixiuRequest { 16 | int64 id = 1; 17 | string name = 2; 18 | } 19 | 20 | // The response message containing the greetings 21 | message PixiuReply { 22 | string message = 1; 23 | } 24 | -------------------------------------------------------------------------------- /practise/http-practise/http-learning.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "flag" 5 | "log" 6 | "net/http" 7 | ) 8 | 9 | func main() { 10 | flag.Parse() 11 | 12 | http.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) { 13 | w.WriteHeader(200) 14 | w.Write([]byte("http " + httpVersion + " ok\n")) 15 | }) 16 | 17 | log.Fatal(http.ListenAndServe(":"+httpPort, nil)) 18 | } 19 | 20 | var ( 21 | httpVersion string 22 | httpPort string 23 | ) 24 | 25 | func init() { 26 | flag.StringVar(&httpVersion, "httpVersion", "", "httpVersion for test.") 27 | flag.StringVar(&httpPort, "httpPort", "80", "httpPort for test.") 28 | } 29 | -------------------------------------------------------------------------------- /practise/loadconfig-practise/config/config.go: -------------------------------------------------------------------------------- 1 | package config 2 | 3 | type PixiuConfiguration struct { 4 | Mysql struct { 5 | User string `yaml:"user"` 6 | Host string `yaml:"host"` 7 | Password string `yaml:"password"` 8 | Port string `yaml:"port"` 9 | Name string `yaml:"name"` 10 | } 11 | Cache struct { 12 | Enable bool `yaml:"enable"` 13 | List []string `yaml:"list,flow"` 14 | } 15 | Student Student `yaml:"student"` 16 | Name string `yaml:"name"` 17 | } 18 | 19 | type Student map[string]Item 20 | 21 | type Item struct { 22 | School string `yaml:"school" json:"school"` 23 | Age int `yaml:"age" json:"age,omitempty"` 24 | } 25 | -------------------------------------------------------------------------------- /practise/gin-practise/endpoint/optimise.go: -------------------------------------------------------------------------------- 1 | package endpoint 2 | 3 | import ( 4 | "github.com/gin-gonic/gin" 5 | 6 | "go-learning/practise/gin-practise/optimise" 7 | ) 8 | 9 | type PreInterface interface { 10 | optimise.OptimiseGetter 11 | } 12 | 13 | // practise is used to interact with features provided by the resource 14 | type practise struct{} 15 | 16 | func (p *practise) Optimise(c *gin.Context) optimise.OptimiseInterface { 17 | return optimise.NewOptimise(c) 18 | } 19 | 20 | // New creates a new practise 21 | func New() PreInterface { 22 | return &practise{} 23 | } 24 | 25 | var Practise PreInterface 26 | 27 | func Register(p PreInterface) { 28 | Practise = p 29 | } 30 | -------------------------------------------------------------------------------- /practise/error-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | ) 7 | 8 | const ( 9 | newOperatorError = "Cannot create new Operator: %s" 10 | ) 11 | 12 | type T1 struct { 13 | Name string `json:"name"` 14 | } 15 | 16 | func (t *T1) SetName(name string) { t.Name = name } 17 | 18 | type T2 struct { 19 | T1 `json:",inline"` 20 | 21 | Age string `json:"age"` 22 | } 23 | 24 | func (t *T2) SetAge(age string) { t.Age = age } 25 | 26 | func main() { 27 | in := &T2{} 28 | in.SetName("caoyingjunz") 29 | in.SetAge("19") 30 | 31 | d, _ := json.Marshal(in) 32 | fmt.Println(string(d)) 33 | 34 | fmt.Println(fmt.Errorf(newOperatorError, "cannot create operator with nil external type")) 35 | } 36 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/dao/db.go: -------------------------------------------------------------------------------- 1 | package dao 2 | 3 | import ( 4 | "fmt" 5 | "sync" 6 | 7 | "github.com/jinzhu/gorm" 8 | ) 9 | 10 | var DB *gorm.DB 11 | var once sync.Once 12 | 13 | func Register(db *gorm.DB) { 14 | once.Do(func() { // 如果是多次调用，执行 do once 15 | DB = db 16 | }) 17 | } 18 | 19 | type TestInterface interface { 20 | Get(name string) string 21 | } 22 | 23 | type test struct { 24 | factory ShareDBFactory 25 | db *gorm.DB 26 | } 27 | 28 | func (c *test) Get(name string) string { 29 | fmt.Println("get name", name) 30 | return fmt.Sprintf("get %s", name) 31 | } 32 | 33 | func NewTest(f ShareDBFactory, db *gorm.DB) TestInterface { 34 | return &test{ 35 | factory: f, 36 | db: db, 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /practise/gorm-practise/models/user.go: -------------------------------------------------------------------------------- 1 | package models 2 | 3 | import "time" 4 | 5 | //CREATE TABLE `user` ( 6 | // `id` INT NOT NULL AUTO_INCREMENT, 7 | // `name` VARCHAR(16) NOT NULL, 8 | // `age` TINYINT NOT NULL, 9 | // PRIMARY KEY (`id`) 10 | //) ENGINE=InnoDB CHARSET=utf8 AUTO_INCREMENT=20110000; 11 | 12 | type User struct { 13 | ID int64 `gorm:"column:id;primary_key;AUTO_INCREMENT;not null" json:"id"` 14 | GmtCreate time.Time `json:"gmt_create"` 15 | GmtModified time.Time `json:"gmt_modified"` 16 | ResourceVersion int64 `json:"resource_version"` 17 | Name string `json:"name"` 18 | Age int `json:"age"` 19 | } 20 | 21 | func (u *User) TableName() string { 22 | return "users" 23 | } 24 | -------------------------------------------------------------------------------- /practise/k8s-practise/node-metrics.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | 7 | "go-learning/practise/k8s-practise/app" 8 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 9 | resourceclient "k8s.io/metrics/pkg/client/clientset/versioned/typed/metrics/v1beta1" 10 | ) 11 | 12 | func main() { 13 | config, err := app.BuildClientConfig("") 14 | if err != nil { 15 | panic(err) 16 | } 17 | 18 | metricClient, err := resourceclient.NewForConfig(config) 19 | if err != nil { 20 | panic(err) 21 | } 22 | nodeMetrics, err := metricClient.NodeMetricses().List(context.TODO(), metav1.ListOptions{}) 23 | if err != nil { 24 | panic(err) 25 | } 26 | 27 | for _, nodeMetric := range nodeMetrics.Items { 28 | fmt.Println(nodeMetric) 29 | } 30 | } 31 | -------------------------------------------------------------------------------- /practise/interface-practise/v1/pixiu.go: -------------------------------------------------------------------------------- 1 | package v1 2 | 3 | import "fmt" 4 | 5 | // PixiuGetter has a method to return a PixiuInterface. 6 | // A group's client should implement this interface. 7 | type PixiuGetter interface { 8 | Pixiu(namespace string) PixiuInterface 9 | } 10 | 11 | // PixiuInterface has methods to work with Pixiu resources. 12 | type PixiuInterface interface { 13 | Create(name string) error 14 | } 15 | 16 | // pixiu implements PixiuInterface 17 | type pixiu struct { 18 | ns string 19 | svc string 20 | } 21 | 22 | func (p *pixiu) Create(name string) error { 23 | fmt.Println("pixiu create", p.ns, p.svc, name) 24 | return nil 25 | } 26 | 27 | func NewPixiu(svc, namespace string) PixiuInterface { 28 | return &pixiu{ 29 | ns: namespace, 30 | svc: svc, 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /practise/k8s-practise/annotation.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | 7 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 8 | "k8s.io/client-go/kubernetes" 9 | 10 | "go-learning/practise/k8s-practise/app" 11 | ) 12 | 13 | func main() { 14 | config, err := app.BuildClientConfig("") 15 | if err != nil { 16 | panic(err) 17 | } 18 | clientSet, err := kubernetes.NewForConfig(config) 19 | if err != nil { 20 | panic(err) 21 | } 22 | 23 | obj, err := clientSet.CoreV1().Nodes().Get(context.TODO(), "kirin", metav1.GetOptions{}) 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | obj.Annotations["testkey"] = "testvalue" 29 | _, _ = clientSet.CoreV1().Nodes().Update(context.TODO(), obj, metav1.UpdateOptions{}) 30 | 31 | fmt.Println(obj.Annotations) 32 | } 33 | -------------------------------------------------------------------------------- /hack/update-gofmt.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright 2021 The Pixiu Authors. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | set -o nounset 18 | set -o errexit 19 | set -o pipefail 20 | 21 | find . -name "*.go" | grep -v vendor | xargs gofmt -s -w 22 | -------------------------------------------------------------------------------- /practise/interface-practise/v2/scaler.go: -------------------------------------------------------------------------------- 1 | package v2 2 | 3 | import "fmt" 4 | 5 | // ScalerGetter has a method to return a ScalerInterface. 6 | // A group's client should implement this interface. 7 | type ScalerGetter interface { 8 | Scaler(namespace string) ScalerInterface 9 | } 10 | 11 | // ScalerInterface has methods to work with Scaler resources. 12 | type ScalerInterface interface { 13 | Create(name string) error 14 | } 15 | 16 | // scaler implements ScalerInterface 17 | type scaler struct { 18 | ns string 19 | svc string 20 | } 21 | 22 | func (p *scaler) Create(name string) error { 23 | fmt.Println("create scaler", p.ns, p.svc, name) 24 | return nil 25 | } 26 | 27 | func NewScaler(svc, namespace string) ScalerInterface { 28 | return &scaler{ 29 | ns: namespace, 30 | svc: svc, 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /practise/loadconfig-practise/loadfile.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | 7 | "gopkg.in/yaml.v2" 8 | 9 | "go-learning/practise/loadconfig-practise/config" 10 | ) 11 | 12 | func loadConfigFromFile(file string) (*config.PixiuConfiguration, error) { 13 | data, err := os.ReadFile(file) 14 | if err != nil { 15 | return nil, err 16 | } 17 | 18 | return loadConfig(data) 19 | } 20 | 21 | func loadConfig(data []byte) (*config.PixiuConfiguration, error) { 22 | var pc config.PixiuConfiguration 23 | if err := yaml.Unmarshal(data, &pc); err != nil { 24 | return nil, err 25 | } 26 | 27 | return &pc, nil 28 | } 29 | 30 | func main() { 31 | pixiuConfiguration, err := loadConfigFromFile("test.yaml") 32 | if err != nil { 33 | panic(err) 34 | } 35 | 36 | fmt.Println(pixiuConfiguration) 37 | } 38 | -------------------------------------------------------------------------------- /practise/k8s-practise/localstorage.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "path/filepath" 7 | 8 | "github.com/caoyingjunz/csi-driver-localstorage/pkg/client/clientset/versioned" 9 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 10 | "k8s.io/client-go/tools/clientcmd" 11 | "k8s.io/client-go/util/homedir" 12 | ) 13 | 14 | func main() { 15 | kubeConfig, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config")) 16 | if err != nil { 17 | panic(err) 18 | } 19 | lsClientSet, err := versioned.NewForConfig(kubeConfig) 20 | if err != nil { 21 | panic(err) 22 | } 23 | 24 | object, err := lsClientSet.StorageV1().LocalStorages().List(context.TODO(), metav1.ListOptions{}) 25 | if err != nil { 26 | panic(err) 27 | } 28 | 29 | fmt.Println(object.Items) 30 | } 31 | -------------------------------------------------------------------------------- /practise/grpc-practise/client/client.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "log" 6 | "time" 7 | 8 | "google.golang.org/grpc" 9 | "google.golang.org/grpc/credentials/insecure" 10 | 11 | pd "go-learning/practise/grpc-practise/pixiu" 12 | ) 13 | 14 | var ( 15 | addr = "127.0.0.1:30000" 16 | ) 17 | 18 | func main() { 19 | conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(insecure.NewCredentials())) 20 | if err != nil { 21 | log.Fatalf("failed to connect rpc server %v", err) 22 | } 23 | defer conn.Close() 24 | 25 | c := pd.NewPixiuClient(conn) 26 | 27 | ctx, cancel := context.WithTimeout(context.Background(), time.Second) 28 | defer cancel() 29 | 30 | r, err := c.GetPixiu(ctx, &pd.PixiuRequest{Id: 12345, Name: "caoyingjun"}) 31 | if err != nil { 32 | log.Fatalf("failed to sayhello %v", err) 33 | } 34 | log.Printf("say hello %v", r.Message) 35 | } 36 | -------------------------------------------------------------------------------- /practise/interface-practise/test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | type TestInterface interface { 6 | RunTest1() 7 | RunTest2() 8 | RunTest3() 9 | } 10 | 11 | type test struct { 12 | *Test1 13 | *Test2 14 | *Test3 15 | } 16 | 17 | func NewTest() TestInterface { 18 | return test{ 19 | &Test1{Name: "test1"}, 20 | &Test2{Name: "test2"}, 21 | &Test3{Name: "test3"}, 22 | } 23 | } 24 | 25 | type Test1 struct { 26 | Name string 27 | } 28 | 29 | type Test2 struct { 30 | Name string 31 | } 32 | 33 | type Test3 struct { 34 | Name string 35 | } 36 | 37 | func (t *Test1) RunTest1() { 38 | fmt.Println(t.Name) 39 | } 40 | 41 | func (t *Test2) RunTest2() { 42 | fmt.Println(t.Name) 43 | } 44 | 45 | func (t *Test3) RunTest3() { 46 | fmt.Println(t.Name) 47 | } 48 | 49 | func main() { 50 | t := NewTest() 51 | 52 | t.RunTest1() 53 | t.RunTest2() 54 | t.RunTest3() 55 | } 56 | -------------------------------------------------------------------------------- /practise/sync-practise/cond.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "log" 5 | "sync" 6 | "time" 7 | ) 8 | 9 | // 参考 https://www.jianshu.com/p/664c48bbaf6c 10 | // https://www.jb51.net/article/221684.htm 11 | 12 | var done = false 13 | 14 | func read(name string, c *sync.Cond) { 15 | c.L.Lock() 16 | for !done { 17 | c.Wait() 18 | } 19 | log.Println(name, "starts reading") 20 | c.L.Unlock() 21 | } 22 | 23 | func write(name string, c *sync.Cond) { 24 | log.Println(name, "starts writing") 25 | time.Sleep(time.Second) 26 | c.L.Lock() 27 | done = true 28 | c.L.Unlock() 29 | log.Println(name, "wakes") 30 | c.Broadcast() 31 | //c.Signal() 32 | } 33 | 34 | func main() { 35 | cond := sync.NewCond(&sync.Mutex{}) 36 | 37 | go read("reader1", cond) 38 | go read("reader2", cond) 39 | go read("reader3", cond) 40 | write("writer", cond) 41 | 42 | time.Sleep(time.Second * 3) 43 | } 44 | -------------------------------------------------------------------------------- /practise/interface-practise/people.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | type People interface { 6 | GetName() string 7 | GetAge() int 8 | 9 | GetData() people 10 | } 11 | 12 | type people struct { 13 | name string 14 | } 15 | 16 | func (p people) GetName() string { 17 | return p.name 18 | } 19 | 20 | func (p people) GetData() people { 21 | return p 22 | } 23 | 24 | func NewPeople(name string) people { 25 | return people{ 26 | name: name, 27 | } 28 | } 29 | 30 | type Ming struct { 31 | people 32 | age int 33 | } 34 | 35 | func (x Ming) GetAge() int { 36 | return x.age 37 | } 38 | 39 | //func (x Ming) GetName() string { 40 | // return "daming" 41 | //} 42 | 43 | func NewMing() People { 44 | return Ming{ 45 | people: NewPeople("xiaoming"), 46 | age: 18, 47 | } 48 | } 49 | 50 | func main() { 51 | m := NewMing() 52 | 53 | fmt.Println(m.GetAge()) 54 | fmt.Println(m.GetName()) 55 | fmt.Println(m.GetData()) 56 | } 57 | -------------------------------------------------------------------------------- /practise/grpc-practise/server/server.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // https://grpc.io/docs/languages/go/quickstart/ 4 | 5 | import ( 6 | "context" 7 | "fmt" 8 | "log" 9 | "net" 10 | 11 | "google.golang.org/grpc" 12 | 13 | pd "go-learning/practise/grpc-practise/pixiu" 14 | ) 15 | 16 | type server struct { 17 | pd.UnimplementedPixiuServer 18 | } 19 | 20 | func (s *server) GetPixiu(ctx context.Context, in *pd.PixiuRequest) (*pd.PixiuReply, error) { 21 | log.Printf("Received %s %d", in.Name, in.Id) 22 | return &pd.PixiuReply{Message: fmt.Sprintf("%s %d", in.GetName(), in.GetId())}, nil 23 | } 24 | 25 | func main() { 26 | l, err := net.Listen("tcp", ":30000") 27 | if err != nil { 28 | log.Fatalf("failed to listen %v", err) 29 | } 30 | 31 | s := grpc.NewServer() 32 | pd.RegisterPixiuServer(s, &server{}) 33 | 34 | log.Printf("listening at %v", l.Addr()) 35 | if err = s.Serve(l); err != nil { 36 | log.Fatalf("failed to serve %v", err) 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /examples/services/incluster/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "time" 7 | 8 | apiv1 "k8s.io/api/core/v1" 9 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 10 | "k8s.io/client-go/kubernetes" 11 | "k8s.io/client-go/rest" 12 | ) 13 | 14 | func main() { 15 | // creates the in-cluster config 16 | config, err := rest.InClusterConfig() 17 | if err != nil { 18 | fmt.Println(err) 19 | } 20 | // creates the clientset 21 | clientset, err := kubernetes.NewForConfig(config) 22 | if err != nil { 23 | fmt.Println(err) 24 | } 25 | for { 26 | services, err := clientset.CoreV1().Services(apiv1.NamespaceDefault).List(context.TODO(), metav1.ListOptions{}) 27 | if err != nil { 28 | fmt.Println(err) 29 | } else { 30 | for _, d := range services.Items { 31 | if err != nil { 32 | fmt.Println(err) 33 | } else { 34 | fmt.Printf("%s\n", d.Name) 35 | } 36 | } 37 | } 38 | } 39 | time.Sleep(10 * time.Second) 40 | } 41 | -------------------------------------------------------------------------------- /doc/kubernetes/code-generator.md: -------------------------------------------------------------------------------- 1 | # code-generator 2 | 3 | - deepcopy-gen: 生成深度拷贝方法，为每个 T 类型生成 func (t* T) DeepCopy() *T 方法，API 类型都需要实现深拷贝 4 | - client-gen: 为资源生成标准的 clientset 5 | - informer-gen: 生成 informer，提供事件机制来响应资源的事件 6 | - lister-gen: 生成 Lister，为 get 和 list 请求提供只读缓存层（通过 indexer 获取） 7 | 8 | ```shell 9 | git clone https://github.com/kubernetes/code-generator.git 10 | git checkout v0.24.9 11 | ``` 12 | 13 | ```shell 14 | # 进行安装 15 | go install ./cmd/{client-gen,deepcopy-gen,informer-gen,lister-gen} 16 | ``` 17 | 18 | ```shell 19 | # 验证安装 20 | client-gen -h 21 | ``` 22 | 23 | ```shell 24 | 代码结构可以参考 25 | https://github.com/kubernetes/sample-controller 26 | ``` 27 | 28 | ```shell 29 | $ mkdir test && cd test 30 | $ go mod init test 31 | $ mkdir -p pkg/apis/example.com/v1 32 | ➜ test tree 33 | ├── go.mod 34 | ├── go.sum 35 | └── pkg 36 | └── apis 37 | └── example.com 38 | └── v1 39 | ├── doc.go 40 | ├── register.go 41 | └── types.go 42 | ``` 43 | -------------------------------------------------------------------------------- /practise/configflags-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | 5 | // ConfigFlags composes the set of values necessary 6 | // for obtaining a REST client config 7 | type ConfigFlags struct { 8 | // config flags 9 | ClusterName *string 10 | Namespace *string 11 | 12 | discoveryBurst int 13 | discoveryQPS float32 14 | } 15 | 16 | func stringptr(val string) *string { 17 | return &val 18 | } 19 | 20 | func NewConfigFlags() *ConfigFlags { 21 | return &ConfigFlags{ 22 | ClusterName: stringptr(""), 23 | Namespace: stringptr(""), 24 | } 25 | } 26 | 27 | func (f *ConfigFlags) WithDiscoveryBurst(discoveryBurst int) *ConfigFlags { 28 | f.discoveryBurst = discoveryBurst 29 | return f 30 | } 31 | 32 | func (f *ConfigFlags) WithDiscoveryQPS(discoveryQPS float32) *ConfigFlags { 33 | f.discoveryQPS = discoveryQPS 34 | return f 35 | } 36 | 37 | func main() { 38 | defaultConfigFlags := NewConfigFlags().WithDiscoveryBurst(300).WithDiscoveryQPS(50.0) 39 | fmt.Println(fmt.Sprintf("%+v", defaultConfigFlags)) 40 | } 41 | -------------------------------------------------------------------------------- /practise/interface-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "go-learning/practise/interface-practise/v1" 5 | "go-learning/practise/interface-practise/v2" 6 | ) 7 | 8 | // moremorehttps://github.com/kubernetes/client-go/blob/4841142cdc4ba44627994f8c25da76c2be2cdb5d/kubernetes/typed/apps/v1/apps_client.go#L27 9 | 10 | type AppsInterface interface { 11 | v1.PixiuGetter 12 | v2.ScalerGetter 13 | } 14 | 15 | type apps struct { 16 | svc string 17 | } 18 | 19 | func (p *apps) Pixiu(namespace string) v1.PixiuInterface { 20 | return v1.NewPixiu(p.svc, namespace) 21 | } 22 | 23 | func (p *apps) Scaler(namespace string) v2.ScalerInterface { 24 | return v2.NewScaler(p.svc, namespace) 25 | } 26 | 27 | func New(svc string) AppsInterface { 28 | return &apps{svc} 29 | } 30 | 31 | func main() { 32 | p := New("service") 33 | 34 | if err := p.Pixiu("pixiu-ns").Create("pixiu"); err != nil { 35 | panic(err) 36 | } 37 | 38 | if err := p.Scaler("scaler-ns").Create("scaler"); err != nil { 39 | panic(err) 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /practise/klog-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "flag" 5 | "fmt" 6 | 7 | klog "k8s.io/klog/v2" 8 | 9 | "go-learning/practise/klog-practise/test" 10 | ) 11 | 12 | // https://github.com/kubernetes/klog/blob/master/examples/coexist_klog_v1_and_v2/coexist_klog_v1_and_v2.go 13 | 14 | func main() { 15 | // initialize klog/v2, can also bind to a local flagset if desired 16 | klog.InitFlags(nil) 17 | 18 | //flag.Set("logtostderr", "false") // By default klog logs to stderr, switch that off 19 | //flag.Set("alsologtostderr", "false") // false is default, but this is informative 20 | //flag.Set("stderrthreshold", "FATAL") // stderrthreshold defaults to ERROR, we don't want anything in stderr 21 | //flag.Set("log_file", "test.log") // log to a file 22 | 23 | // parse klog/v2 flags 24 | flag.Parse() 25 | // make sure we flush before exiting 26 | defer klog.Flush() 27 | 28 | test.Test1() 29 | klog.Warningf("aaa %v", "dddd") 30 | klog.InfoS("hello from klog (v2)!", "aaa", "bbb") 31 | klog.ErrorS(fmt.Errorf("error info"), "a", "aaaa", "bbb") 32 | } 33 | -------------------------------------------------------------------------------- /practise/gin-practise/hander/hanlder.go: -------------------------------------------------------------------------------- 1 | package hander 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | ) 8 | 9 | type ginMaps map[string]interface{} 10 | 11 | type Person struct { 12 | Id int64 `json:"id"` 13 | Age int `json:"age"` 14 | Sex string `json:"sex"` 15 | Max *map[string]string `json:"max"` 16 | Lis []string `json:"lis"` 17 | } 18 | 19 | type TP struct { 20 | Name string `json:"name"` 21 | } 22 | 23 | type GinRequest struct { 24 | Obj ginMaps `json:"obj"` 25 | Pers Person `json:"pers"` 26 | TP 27 | } 28 | 29 | func Dohandler(ctx context.Context, gp GinRequest) error { 30 | // 取值 31 | fmt.Println("Name:", gp.Name) 32 | fmt.Println("Obj:", gp.Obj) 33 | 34 | // 序列化，可进行外部存储 35 | px, err := json.Marshal(gp.Pers) 36 | if err != nil { 37 | return err 38 | } 39 | fmt.Println("Marshal:", string(px)) 40 | 41 | // 反序列化，继续使用 42 | var pp Person 43 | err = json.Unmarshal(px, &pp) 44 | if err != nil { 45 | return err 46 | } 47 | 48 | fmt.Println("Unmarshal:", pp) 49 | return nil 50 | } 51 | -------------------------------------------------------------------------------- /practise/gin-practise/log/logging.go: -------------------------------------------------------------------------------- 1 | package log 2 | 3 | type Configuration struct { 4 | LogFile string 5 | LogLevel string 6 | 7 | RotateMaxSize int 8 | RotateMaxAge int 9 | RotateMaxBackups int 10 | Compress bool 11 | } 12 | 13 | type Logger interface { 14 | Info(args ...interface{}) 15 | Infof(f string, args ...interface{}) 16 | } 17 | 18 | var Glog Logger 19 | var AccessLog Logger 20 | 21 | // 注册 AccessLog 22 | func Register(logDir string) { 23 | afileLog := "/Users/caoyuan/workstation/go-learning/practise/gin-practise/practise-access.log" 24 | AccessLog, _ = NewZapLogger(Configuration{ 25 | LogFile: afileLog, 26 | LogLevel: "INFO", 27 | RotateMaxSize: 500, 28 | RotateMaxAge: 7, 29 | RotateMaxBackups: 3, 30 | }) 31 | } 32 | 33 | // 或者 import 的时候初始化 34 | func init() { 35 | fileLog := "/Users/caoyuan/workstation/go-learning/practise/gin-practise/practise-log.log" 36 | Glog, _ = NewZapLogger(Configuration{ 37 | LogFile: fileLog, 38 | LogLevel: "INFO", 39 | RotateMaxSize: 500, 40 | RotateMaxAge: 7, 41 | RotateMaxBackups: 3, 42 | }) 43 | } 44 | -------------------------------------------------------------------------------- /examples/services/oncluster/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "flag" 6 | "fmt" 7 | "path/filepath" 8 | 9 | apiv1 "k8s.io/api/core/v1" 10 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 11 | "k8s.io/client-go/kubernetes" 12 | "k8s.io/client-go/tools/clientcmd" 13 | "k8s.io/client-go/util/homedir" 14 | ) 15 | 16 | func main() { 17 | var kubeconfig *string 18 | if home := homedir.HomeDir(); home != "" { 19 | kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file") 20 | } else { 21 | kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file") 22 | } 23 | flag.Parse() 24 | 25 | config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig) 26 | if err != nil { 27 | panic(err) 28 | } 29 | clientset, err := kubernetes.NewForConfig(config) 30 | if err != nil { 31 | panic(err) 32 | } 33 | 34 | serviceClient := clientset.CoreV1().Services(apiv1.NamespaceDefault) 35 | 36 | list, err := serviceClient.List(context.TODO(), metav1.ListOptions{}) 37 | if err != nil { 38 | panic(err) 39 | } 40 | for _, d := range list.Items { 41 | fmt.Printf("%s\n", d.Name) 42 | } 43 | } 44 | -------------------------------------------------------------------------------- /practise/k8s-practise/events.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "go-learning/practise/k8s-practise/app" 7 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 8 | "k8s.io/client-go/kubernetes" 9 | "strings" 10 | ) 11 | 12 | func main() { 13 | config, err := app.BuildClientConfig("") 14 | if err != nil { 15 | panic(err) 16 | } 17 | clientSet, err := kubernetes.NewForConfig(config) 18 | if err != nil { 19 | panic(err) 20 | } 21 | 22 | name := "nginx" 23 | namespace := "default" 24 | deployment, err := clientSet.AppsV1().Deployments(namespace).Get(context.TODO(), name, metav1.GetOptions{}) 25 | if err != nil { 26 | panic(err) 27 | } 28 | 29 | fieldSelector := []string{ 30 | "involvedObject.uid=" + string(deployment.UID), 31 | "involvedObject.name=" + deployment.Name, 32 | "involvedObject.namespace=" + deployment.Namespace, 33 | "involvedObject.kind=Deployment", 34 | } 35 | 36 | events, err := clientSet.CoreV1().Events(namespace).List(context.TODO(), metav1.ListOptions{ 37 | FieldSelector: strings.Join(fieldSelector, ","), 38 | Limit: 500, 39 | }) 40 | if err != nil { 41 | panic(err) 42 | } 43 | 44 | fmt.Println("events.Items", events.Items) 45 | } 46 | -------------------------------------------------------------------------------- /hack/verify-gofmt.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright 2021 The Pixiu Authors. 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | set -o errexit 18 | set -o nounset 19 | set -o pipefail 20 | 21 | HELM_ROOT=$(dirname "${BASH_SOURCE}")/.. 22 | cd "${HELM_ROOT}" 23 | 24 | find_files() { 25 | find . -not \( \ 26 | \( \ 27 | -wholename './output' \ 28 | -o -wholename '*/vendor/*' \ 29 | \) -prune \ 30 | \) -name '*.go' 31 | } 32 | 33 | GOFMT="gofmt -s" 34 | 35 | bad_files=$(find_files | xargs $GOFMT -l) 36 | if [[ -n "${bad_files}" ]]; then 37 | echo "Please run hack/update-gofmt.sh to fix the following files:" 38 | echo "${bad_files}" 39 | exit 1 40 | fi 41 | -------------------------------------------------------------------------------- /practise/k8s-practise/app/helper.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "net/http" 7 | "time" 8 | 9 | "k8s.io/apimachinery/pkg/util/wait" 10 | clientset "k8s.io/client-go/kubernetes" 11 | "k8s.io/klog/v2" 12 | ) 13 | 14 | // WaitForAPIServer waits for the API Server's /healthz endpoint to report "ok" with timeout. 15 | func WaitForAPIServer(client clientset.Interface, timeout time.Duration) error { 16 | var lastErr error 17 | 18 | err := wait.PollImmediate(time.Second, timeout, func() (bool, error) { 19 | healthStatus := 0 20 | result := client.Discovery().RESTClient().Get().AbsPath("/healthz").Do(context.TODO()).StatusCode(&healthStatus) 21 | if result.Error() != nil { 22 | lastErr = fmt.Errorf("failed to get apiserver /healthz status: %v", result.Error()) 23 | return false, nil 24 | } 25 | if healthStatus != http.StatusOK { 26 | content, _ := result.Raw() 27 | lastErr = fmt.Errorf("APIServer isn't healthy: %v", string(content)) 28 | klog.Warningf("APIServer isn't healthy yet: %v. Waiting a little while.", string(content)) 29 | return false, nil 30 | } 31 | 32 | return true, nil 33 | }) 34 | 35 | if err != nil { 36 | return fmt.Errorf("%v: %v", err, lastErr) 37 | } 38 | 39 | return nil 40 | } 41 | -------------------------------------------------------------------------------- /practise/gin-practise/worker/worker.go: -------------------------------------------------------------------------------- 1 | package worker 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "time" 7 | 8 | "k8s.io/apimachinery/pkg/util/wait" 9 | 10 | "go-learning/practise/gin-practise/workqueue" 11 | ) 12 | 13 | type WorkerInterface interface { 14 | Run(workers int, stopCh <-chan struct{}) 15 | 16 | DoTest(ctx context.Context, s string) error 17 | 18 | DoAfterTest(ctx context.Context, s string) error 19 | } 20 | 21 | type Worker struct { 22 | queue workqueue.DelayingInterface 23 | } 24 | 25 | func NewWorker() WorkerInterface { 26 | return &Worker{ 27 | queue: workqueue.NewQueue(), 28 | } 29 | } 30 | 31 | func (w *Worker) Run(workers int, stopCh <-chan struct{}) { 32 | for i := 0; i < workers; i++ { 33 | // 可启动多个协程 34 | go wait.Until(w.worker, 2*time.Second, stopCh) 35 | } 36 | } 37 | 38 | func (w *Worker) worker() { 39 | item := w.queue.Get() 40 | if len(item) == 0 { 41 | return 42 | } 43 | 44 | fmt.Println("item:", item, "length:", w.queue.Len()) 45 | } 46 | 47 | func (w *Worker) DoTest(ctx context.Context, s string) error { 48 | // TODO: do somethings 49 | w.queue.Add(s) 50 | return nil 51 | } 52 | 53 | // 延迟5秒 54 | func (w *Worker) DoAfterTest(ctx context.Context, s string) error { 55 | 56 | w.queue.AddAfter(s, 5) 57 | return nil 58 | } 59 | -------------------------------------------------------------------------------- /practise/grpc/client.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | 6 | "log" 7 | "time" 8 | 9 | "google.golang.org/grpc" 10 | "google.golang.org/grpc/credentials/insecure" 11 | 12 | pd "go-learning/practise/grpc/tunnel" 13 | ) 14 | 15 | var ( 16 | addr = "127.0.0.1:8092" 17 | ) 18 | 19 | func main() { 20 | conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(insecure.NewCredentials())) 21 | if err != nil { 22 | log.Fatalf("failed to connect rpc server %v", err) 23 | } 24 | defer conn.Close() 25 | 26 | c := pd.NewTunnelClient(conn) 27 | 28 | stream, err := c.Connect(context.Background()) 29 | if err != nil { 30 | log.Fatalf("%v", err) 31 | } 32 | 33 | clientId := "node2" 34 | 35 | // 启动协程，接受服务段回调 client 的请求 36 | go func() { 37 | for { 38 | msg, err := stream.Recv() 39 | if err != nil { 40 | log.Printf("Receive error: %v", err) 41 | return 42 | } 43 | log.Printf("Received from server: %s", msg.Result) 44 | } 45 | }() 46 | 47 | // 启动客户端定时测试DEMO 48 | ticker := time.NewTicker(10 * time.Second) 49 | defer ticker.Stop() 50 | 51 | for range ticker.C { 52 | ts := time.Now().String() 53 | if err = stream.Send(&pd.Request{ 54 | Type: clientId, 55 | Payload: []byte(ts), 56 | }); err != nil { 57 | log.Println("调用服务端失败", err) 58 | } 59 | } 60 | } 61 | -------------------------------------------------------------------------------- /practise/gorm-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "time" 7 | 8 | "go-learning/practise/gorm-practise/dbstone" 9 | "go-learning/practise/gorm-practise/models" 10 | ) 11 | 12 | const ( 13 | name = "caoyingjun" 14 | age = 18 15 | ) 16 | 17 | var udb = dbstone.NewUserDB() 18 | 19 | func main() { 20 | var err error 21 | 22 | // 创建 user 23 | now := time.Now() 24 | u1 := models.User{ 25 | GmtCreate: now, 26 | GmtModified: now, 27 | Name: name, 28 | Age: age, 29 | } 30 | if err = udb.Create(context.TODO(), &u1); err != nil { 31 | // handler err 32 | panic(err) 33 | } 34 | 35 | // 获取 user 36 | u2, err := udb.Get(context.TODO(), name, age) 37 | if err != nil { 38 | panic(err) 39 | } 40 | fmt.Println("get user: ", u2) 41 | 42 | // 更新 user 43 | // 业务层面的数据更新 44 | // 数据库层面的数据在 update 中实现 45 | updates := make(map[string]interface{}) 46 | updates["age"] = 19 47 | if err = udb.Update(context.TODO(), name, u2.ResourceVersion, updates); err != nil { 48 | panic(err) 49 | } 50 | 51 | users, err := udb.ListByPage(context.TODO(), name, 2, 2) 52 | if err != nil { 53 | panic(err) 54 | } 55 | fmt.Println("list page users:", users) 56 | 57 | // 删除 user 58 | //if err = udb.Delete(context.TODO(), u2.ID); err != nil { 59 | // panic(err) 60 | //} 61 | } 62 | -------------------------------------------------------------------------------- /practise/logrus-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // 参考文档，logrus 官方源码 https://github.com/sirupsen/logrus 4 | // 打印行数和文件 https://github.com/sirupsen/logrus/issues/63 5 | // 暂时够用，用空的话，可以测试下钩子，目前看起来用不到 6 | 7 | import ( 8 | "os" 9 | 10 | "github.com/sirupsen/logrus" 11 | ) 12 | 13 | var log = logrus.New() 14 | 15 | func init() { 16 | // Log as JSON instead of the default ASCII formatter. 17 | // log.SetFormatter(&logrus.JSONFormatter{}) 18 | log.SetFormatter(&logrus.TextFormatter{}) 19 | 20 | // Output to stdout instead of the default stderr 21 | // Can be any io.Writer, see below for File example 22 | log.SetOutput(os.Stdout) 23 | 24 | //用日志实例的方式使用日志 25 | //log.Out = os.Stdout //日志标准输出 26 | 27 | // 也可以把log 写入日志文件 28 | file, err := os.OpenFile("logrus.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666) 29 | if err != nil { 30 | panic("打开日志文件失败") 31 | } 32 | 33 | //log.Out = file 34 | log.SetOutput(file) 35 | 36 | // Only log the warning severity or above. 37 | log.SetLevel(logrus.WarnLevel) 38 | } 39 | 40 | func main() { 41 | 42 | // 是否需要打印行数和文件名称（影响性能），也可以对比较长的文件名进行切割，去除不必要的目录 43 | //log.SetReportCaller(true) 44 | 45 | log.WithFields(logrus.Fields{"request_id": "666"}).Info("测试 info ") 46 | log.WithFields(logrus.Fields{"request_id": "888"}).Warn("测试 warn ") 47 | log.WithFields(logrus.Fields{"request_id": "888"}).Error("测试 error ") 48 | } 49 | -------------------------------------------------------------------------------- /practise/gorm-practise/dbstone/base.go: -------------------------------------------------------------------------------- 1 | package dbstone 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/jinzhu/gorm" 7 | _ "github.com/jinzhu/gorm/dialects/mysql" 8 | 9 | "go-learning/practise/gorm-practise/models" 10 | ) 11 | 12 | var DB *gorm.DB 13 | 14 | func init() { 15 | user := "root" 16 | password := "oxN1bEcFML" 17 | ip := "103.39.211.122" 18 | port := 30692 19 | database := "gorm" 20 | dbConnection := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?charset=utf8&parseTime=True&loc=Local", 21 | user, 22 | password, 23 | ip, 24 | port, 25 | database) 26 | // must declare the err to aviod panic: runtime error: invalid memory address or nil pointer dereferences 27 | var err error 28 | DB, err = gorm.Open("mysql", dbConnection) 29 | if err != nil { 30 | panic(err) 31 | } 32 | 33 | // set the connect pools 34 | DB.DB().SetMaxIdleConns(10) 35 | DB.DB().SetMaxOpenConns(100) 36 | // 设置非复数表名 37 | //DB.SingularTable(true) 38 | 39 | // 检查表是否存在 40 | if !DB.HasTable(&models.User{}) { 41 | if err := DB.Set("gorm:table_options", "ENGINE=InnoDB DEFAULT CHARSET=utf8"). 42 | CreateTable(&models.User{}). 43 | Error; err != nil { 44 | panic(err) 45 | } 46 | } 47 | 48 | // 增加索引 49 | //DB.Model(&User{}).AddIndex("idx_user_name", "name") 50 | // 为`name`, `age`列添加索引`idx_user_name_age` 51 | DB.Model(&models.User{}).AddIndex("idx_user_name_age", "name", "age") 52 | } 53 | -------------------------------------------------------------------------------- /practise/excelize-practise/excel/excel.go: -------------------------------------------------------------------------------- 1 | package excel 2 | 3 | import ( 4 | "fmt" 5 | "sync" 6 | 7 | "github.com/xuri/excelize/v2" 8 | ) 9 | 10 | type ExcelFile struct { 11 | lock sync.RWMutex 12 | 13 | sheetName string 14 | f *excelize.File 15 | cursor int // 游标，用户标记当前所处位置 16 | } 17 | 18 | func NewExcelFile() *ExcelFile { 19 | return &ExcelFile{ 20 | f: excelize.NewFile(), 21 | cursor: 1, 22 | } 23 | } 24 | 25 | func (file *ExcelFile) SetSheetName(oldName, NewName string) { 26 | file.sheetName = NewName 27 | _ = file.f.SetSheetName(oldName, NewName) 28 | } 29 | 30 | // SetCellSlice 以行为单位 31 | func (file *ExcelFile) SetCellSlice(cells []string) error { 32 | file.lock.Lock() 33 | defer file.lock.Unlock() 34 | 35 | if len(cells) == 0 { 36 | return nil 37 | } 38 | for index, cell := range cells { 39 | excelIndex := fmt.Sprintf("%s%d", file.parseExcelIndex(index), file.cursor) 40 | if err := file.f.SetCellValue(file.sheetName, excelIndex, cell); err != nil { 41 | return err 42 | } 43 | } 44 | 45 | file.cursor += 1 46 | return nil 47 | } 48 | 49 | func (file *ExcelFile) parseExcelIndex(i int) string { 50 | index := 65 + i 51 | indexRune := rune(index) 52 | 53 | return string(indexRune) 54 | } 55 | 56 | func (file *ExcelFile) SaveAs(f string) error { 57 | return file.f.SaveAs(f) 58 | } 59 | 60 | func (file *ExcelFile) Close() error { 61 | return file.f.Close() 62 | } 63 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/README.md: -------------------------------------------------------------------------------- 1 | # Pixiuctl Usage Overview 2 | 3 | ## Global Help 4 | ```go 5 | # go run pixiuctl.go help 6 | pixiuctl controls the Pixiu cluster manager. 7 | 8 | Find more information at: https://github.com/caoyingjunz/go-learning 9 | 10 | Basic Commands (Beginner): 11 | create Create a pixiu resource from a file or stdin 12 | 13 | Advanced Commands: 14 | apply Apply a configuration to a pixiu resource by file name or stdin 15 | 16 | Other Commands: 17 | completion generate the autocompletion script for the specified shell 18 | 19 | Usage: 20 | pixiuctl [flags] [options] 21 | 22 | Use "pixiuctl --help" for more information about a given command. 23 | ``` 24 | 25 | ## Subcommand Help 26 | ```go 27 | # go run pixiuctl.go create --help 28 | Create a resource from a file or from stdin. 29 | 30 | JSON and YAML formats are accepted. 31 | 32 | Examples: 33 | pixiuctl create -f ./create.json 34 | 35 | Available Commands: 36 | service Create a ClusterIP service 37 | 38 | Options: 39 | --edit=false: Edit the API resource before creating 40 | --raw='': Raw URI to POST to the server. Uses the transport specified by the kubeconfig file. 41 | 42 | Usage: 43 | pixiuctl create -f FILENAME [options] 44 | 45 | Use "pixiuctl --help" for more information about a given command. 46 | Use "pixiuctl create options" for a list of global command-line options (applies to all commands). 47 | ``` -------------------------------------------------------------------------------- /practise/containerd-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "time" 7 | 8 | "github.com/containerd/containerd/integration/remote/util" 9 | "google.golang.org/grpc" 10 | "google.golang.org/grpc/credentials/insecure" 11 | runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1alpha2" 12 | ) 13 | 14 | const ( 15 | timeout = 1 * time.Minute 16 | endpoint = "unix:///run/containerd/containerd.sock" 17 | // maxMsgSize use 16MB as the default message size limit. 18 | // grpc library default is 4MB 19 | maxMsgSize = 1024 * 1024 * 16 20 | ) 21 | 22 | func main() { 23 | addr, dialer, err := util.GetAddressAndDialer(endpoint) 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | ctx, cancel := context.WithTimeout(context.Background(), timeout) 29 | defer cancel() 30 | 31 | conn, err := grpc.DialContext(ctx, addr, 32 | grpc.WithTransportCredentials(insecure.NewCredentials()), 33 | grpc.WithContextDialer(dialer), 34 | grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxMsgSize)), 35 | ) 36 | if err != nil { 37 | panic(err) 38 | } 39 | 40 | imc := runtimeapi.NewImageServiceClient(conn) 41 | info, err := imc.ImageFsInfo(ctx, &runtimeapi.ImageFsInfoRequest{}) 42 | if err != nil { 43 | panic(err) 44 | } 45 | fmt.Println(info.GetImageFilesystems()) 46 | 47 | images, err := imc.ListImages(ctx, &runtimeapi.ListImagesRequest{}) 48 | if err != nil { 49 | panic(err) 50 | } 51 | fmt.Println(images.Images) 52 | } 53 | -------------------------------------------------------------------------------- /examples/informer/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "log" 5 | "time" 6 | 7 | v1 "k8s.io/apimachinery/pkg/apis/meta/v1" 8 | "k8s.io/client-go/informers" 9 | "k8s.io/client-go/kubernetes" 10 | "k8s.io/client-go/tools/cache" 11 | "k8s.io/client-go/tools/clientcmd" 12 | ) 13 | 14 | func main() { 15 | 16 | // kubeconfigPath 根据实际情况调整 17 | kubeconfigPath := "/Users/caoyuan/.kube/config" 18 | config, err := clientcmd.BuildConfigFromFlags("", kubeconfigPath) 19 | if err != nil { 20 | panic(err) 21 | } 22 | 23 | clientset, err := kubernetes.NewForConfig(config) 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | stopCh := make(chan struct{}) 29 | defer close(stopCh) 30 | sharedInformers := informers.NewSharedInformerFactory(clientset, time.Minute) 31 | 32 | informer := sharedInformers.Core().V1().Services().Informer() 33 | 34 | informer.AddEventHandler(cache.ResourceEventHandlerFuncs{ 35 | AddFunc: func(obj interface{}) { 36 | mObj := obj.(v1.Object) 37 | log.Printf("New Services Added to Store: %s", mObj.GetName()) 38 | }, 39 | UpdateFunc: func(oldObj, newObj interface{}) { 40 | oObj := oldObj.(v1.Object) 41 | nObj := newObj.(v1.Object) 42 | log.Printf("%s Services Updated to %s", oObj.GetName(), nObj.GetName()) 43 | }, 44 | DeleteFunc: func(obj interface{}) { 45 | mObj := obj.(v1.Object) 46 | log.Printf("Services Deleted from Store: %s", mObj.GetName()) 47 | }, 48 | }) 49 | 50 | informer.Run(stopCh) 51 | } 52 | -------------------------------------------------------------------------------- /practise/k8s-practise/proxy.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "net/url" 5 | "path/filepath" 6 | 7 | "github.com/gin-gonic/gin" 8 | "k8s.io/apimachinery/pkg/util/proxy" 9 | "k8s.io/client-go/rest" 10 | "k8s.io/client-go/tools/clientcmd" 11 | "k8s.io/client-go/util/homedir" 12 | ) 13 | 14 | // 请求验证 curl 127.0.0.1:8090/apis/apps/v1/namespaces/default/deployments/toolbox 15 | 16 | func main() { 17 | route := gin.Default() 18 | 19 | // gin 指定代理，apis 原始请求转发到 k8s APIServer 20 | route.Any("/*proxy", proxyHandler) 21 | 22 | _ = route.Run(":8090") 23 | } 24 | 25 | func proxyHandler(c *gin.Context) { 26 | config, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config")) 27 | if err != nil { 28 | panic(err) 29 | } 30 | 31 | transport, err := rest.TransportFor(config) 32 | if err != nil { 33 | panic(err) 34 | } 35 | target, err := parseTarget(*c.Request.URL, config.Host) 36 | if err != nil { 37 | panic(err) 38 | } 39 | 40 | httpProxy := proxy.NewUpgradeAwareHandler(target, transport, false, false, nil) 41 | httpProxy.UpgradeTransport = proxy.NewUpgradeRequestRoundTripper(transport, transport) 42 | httpProxy.ServeHTTP(c.Writer, c.Request) 43 | } 44 | 45 | func parseTarget(target url.URL, host string) (*url.URL, error) { 46 | kubeURL, err := url.Parse(host) 47 | if err != nil { 48 | return nil, err 49 | } 50 | 51 | target.Host = kubeURL.Host 52 | target.Scheme = kubeURL.Scheme 53 | return &target, nil 54 | } 55 | -------------------------------------------------------------------------------- /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | # This is a basic workflow to help you get started with Actions 2 | name: CI 3 | 4 | # Controls when the workflow will run 5 | on: 6 | # Triggers the workflow on push or pull request events but only for the master branch 7 | push: 8 | branches: [ master ] 9 | pull_request: 10 | branches: [ master ] 11 | 12 | # Allows you to run this workflow manually from the Actions tab 13 | workflow_dispatch: 14 | 15 | env: 16 | GO_VERSION: '1.17.3' 17 | 18 | # A workflow run is made up of one or more jobs that can run sequentially or in parallel 19 | jobs: 20 | # misspell-check: 21 | # runs-on: ubuntu-latest 22 | # container: pouchcontainer/pouchlinter:v0.1.2 23 | # steps: 24 | # - name: Checkout 25 | # uses: actions/checkout@v2 26 | 27 | # - name: Lint markdown files 28 | # run: find ./ -name "*.md" | xargs mdl -r ~MD010,~MD013,~MD022,~MD024,~MD029,~MD031,~MD032,~MD033,~MD034,~MD036 29 | 30 | golang-lint: 31 | runs-on: ubuntu-latest 32 | steps: 33 | - name: Checkout 34 | uses: actions/checkout@v2 35 | with: 36 | submodules: true 37 | 38 | - name: Set up Golang 39 | uses: actions/setup-go@v2 40 | with: 41 | go-version: ${{ env.GO_VERSION }} 42 | 43 | - name: Run go fmt test 44 | run: hack/verify-gofmt.sh 45 | env: 46 | GO111MODULE: auto 47 | 48 | - name: Run golang unit test 49 | run: cd practise/test-practise && go test -v 50 | -------------------------------------------------------------------------------- /practise/checker.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | 7 | "github.com/pkg/errors" 8 | ) 9 | 10 | type Checker interface { 11 | Check() (warnings, errorList []error) 12 | Name() string 13 | } 14 | 15 | type FileAvailableCheck struct { 16 | Path string 17 | Label string 18 | } 19 | 20 | func (fac FileAvailableCheck) Name() string { 21 | if len(fac.Label) != 0 { 22 | return fac.Label 23 | } 24 | return "TestDir" 25 | } 26 | 27 | func (fac FileAvailableCheck) Check() (warnings, errorList []error) { 28 | if _, err := os.Stat(fac.Path); err == nil { 29 | return nil, []error{errors.Errorf("%s already exists", fac.Path)} 30 | } 31 | return nil, nil 32 | } 33 | 34 | type TestCheck struct { 35 | Test string 36 | } 37 | 38 | func (t TestCheck) Name() string { 39 | return "Test" 40 | } 41 | 42 | func (t TestCheck) Check() (warnings, errorList []error) { 43 | // TODO 44 | return nil, nil 45 | } 46 | 47 | // RunChecks runs each check, displays it's warnings/errors, and once all 48 | // are processed will exit if any errors occurred. 49 | func RunChecks(checks []Checker) error { 50 | for _, c := range checks { 51 | name := c.Name() 52 | warnings, errs := c.Check() 53 | 54 | fmt.Println("name:", name) 55 | fmt.Println("warnings:", warnings) 56 | fmt.Println("errs:", errs) 57 | } 58 | 59 | return errors.Wrap(fmt.Errorf("wrap raw error"), "check failed") 60 | } 61 | 62 | // 接口实现检查 demo 63 | func main() { 64 | checks := []Checker{ 65 | FileAvailableCheck{Path: "/root/test1"}, 66 | TestCheck{Test: "Test"}, 67 | } 68 | 69 | err := RunChecks(checks) 70 | fmt.Println(err) 71 | } 72 | -------------------------------------------------------------------------------- /practise/gin-practise/optimise/optimise.go: -------------------------------------------------------------------------------- 1 | package optimise 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/gin-gonic/gin" 7 | ) 8 | 9 | // OptimiseGetter has a method to return a OptimiseInterface. 10 | // A group's client should implement this interface. 11 | type OptimiseGetter interface { 12 | Optimise(c *gin.Context) OptimiseInterface 13 | } 14 | 15 | // OptimiseInterface has methods to work with Optimise resources. 16 | type OptimiseInterface interface { 17 | Create(ns string) error 18 | Get() (string, error) 19 | 20 | OptimiseExpansion 21 | } 22 | 23 | // optimise implements OptimiseInterface 24 | type optimise struct { 25 | c *gin.Context 26 | 27 | // 根据业务设置字段 28 | ns string 29 | } 30 | 31 | func (o *optimise) setNamespace(ns string) error { 32 | n := o.c.Query("namespace") 33 | if len(n) != 0 { 34 | return fmt.Errorf("empty") 35 | } 36 | o.ns = n 37 | return nil 38 | } 39 | 40 | // 处理post请求的参数 41 | func (o *optimise) shouldBindJSON() error { 42 | 43 | return nil 44 | } 45 | 46 | type Svars struct{} 47 | 48 | func (o *optimise) Create(ns string) error { 49 | var s Svars 50 | if err := o.c.ShouldBindJSON(&s); err != nil { 51 | return err 52 | } 53 | 54 | if err := o.setNamespace(ns); err != nil { 55 | return err 56 | } 57 | 58 | fmt.Print("optimise create", o.ns, s) 59 | return nil 60 | } 61 | 62 | func (o *optimise) Get() (string, error) { 63 | if err := o.setNamespace("s"); err != nil { 64 | return "", err 65 | } 66 | 67 | return o.ns, nil 68 | } 69 | 70 | func NewOptimise(c *gin.Context) *optimise { 71 | return &optimise{ 72 | c: c, 73 | } 74 | } 75 | 76 | type OptimiseExpansion interface{} 77 | -------------------------------------------------------------------------------- /doc/kubernetes/controller.md: -------------------------------------------------------------------------------- 1 | ### Kubernetes Defined Controllers 2 | 3 | 1. CRD 的定义 参考 `sample-controller`, 创建 `CRD` 和 其对应的 `Foo`. 4 | 5 | [crd.yaml](https://github.com/kubernetes/sample-controller/blob/master/artifacts/examples/crd.yaml) 6 | ```bash 7 | apiVersion: apiextensions.k8s.io/v1beta1 8 | kind: CustomResourceDefinition 9 | metadata: 10 | name: foos.samplecontroller.k8s.io 11 | spec: 12 | group: samplecontroller.k8s.io 13 | version: v1alpha1 14 | names: 15 | kind: Foo 16 | # 复数名 17 | plural: foos 18 | # 单数名 19 | singular: foo 20 | # 简称，类似 deployment 的简称是 deploy 21 | shortNames： 22 | - fo 23 | scope: Namespaced 24 | ``` 25 | 26 | [example-foo.yaml](https://github.com/kubernetes/sample-controller/blob/master/artifacts/examples/example-foo.yaml) 27 | 28 | ```bash 29 | apiVersion: samplecontroller.k8s.io/v1alpha1 30 | kind: Foo 31 | metadata: 32 | name: example-foo 33 | spec: 34 | deploymentName: example-foo 35 | replicas: 1 36 | ``` 37 | 38 | `controller 流程` 39 | ![流程图](./../pictures/crd.jpeg) 40 | 41 | `ectd 命令 example` 42 | ```bash 43 | ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key get /registry/apiextensions.k8s.io/customresourcedefinitions/ --prefix 44 | ``` 45 | 46 | ```bash 47 | ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key get /registry/bolingcavalry.k8s.io/students/default/object-student --print-value-only 48 | ``` 49 | 50 | 51 | 52 | -------------------------------------------------------------------------------- /practise/gin-practise/workqueue/queue.go: -------------------------------------------------------------------------------- 1 | package workqueue 2 | 3 | import ( 4 | "sync" 5 | "time" 6 | ) 7 | 8 | type DelayingInterface interface { 9 | Interface 10 | // AddAfter adds an item to the workqueue after the indicated duration has passed 11 | AddAfter(item string, duration time.Duration) 12 | } 13 | 14 | type Interface interface { 15 | Add(item string) 16 | Len() int 17 | Get() string 18 | } 19 | 20 | type Type struct { 21 | // queue defines the order in which we will work on items. 22 | queue []string 23 | 24 | lock sync.RWMutex 25 | } 26 | 27 | // Add marks item as needing processing. 28 | func (q *Type) Add(item string) { 29 | q.lock.Lock() 30 | defer q.lock.Unlock() 31 | 32 | q.queue = append(q.queue, item) 33 | } 34 | 35 | // Len returns the current queue length, for informational purposes only. You 36 | // shouldn't e.g. gate a call to Add() or Get() on Len() being a particular 37 | // value, that can't be synchronized properly. 38 | func (q *Type) Len() int { 39 | q.lock.Lock() 40 | defer q.lock.Unlock() 41 | 42 | return len(q.queue) 43 | } 44 | 45 | func (q *Type) Get() string { 46 | q.lock.Lock() 47 | defer q.lock.Unlock() 48 | 49 | if len(q.queue) == 0 { 50 | return "" 51 | } 52 | 53 | var item string 54 | item, q.queue = q.queue[0], q.queue[1:] 55 | 56 | return item 57 | } 58 | 59 | func (q *Type) AddAfter(item string, duration time.Duration) { 60 | // immediately add things with no delay 61 | if duration <= 0 { 62 | q.Add(item) 63 | return 64 | } 65 | 66 | go func() { 67 | time.Sleep(duration * time.Second) 68 | q.Add(item) 69 | return 70 | }() 71 | } 72 | 73 | func NewQueue() DelayingInterface { 74 | return &Type{ 75 | queue: []string{}, 76 | } 77 | } 78 | -------------------------------------------------------------------------------- /practise/sync-practise/waitgroup.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "sync" 6 | "testing" 7 | "time" 8 | ) 9 | 10 | // 封装在结构体中使用 https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/test/e2e/framework/log_size_monitoring.go#L173 11 | // 单独使用 12 | // 参考1 https://github.com/kubernetes/kubernetes/blob/9c147baa70c31afc966329df73302e9b52d8e432/test/e2e/apimachinery/flowcontrol.go#L157 13 | // 参考2 https://github.com/kubernetes/kubernetes/blob/9c147baa70c31afc966329df73302e9b52d8e432/pkg/controller/replicaset/replica_set.go#L616 14 | 15 | func Get(i string) error { 16 | time.Sleep(2 * time.Second) 17 | fmt.Println("i", i) 18 | return fmt.Errorf("test error") 19 | } 20 | 21 | //https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/pkg/scheduler/metrics/metric_recorder_test.go#L81 22 | 23 | func TestClear(t *testing.T) { 24 | var wg sync.WaitGroup 25 | incLoops, decLoops := 100, 80 26 | wg.Add(incLoops + decLoops) 27 | for i := 0; i < incLoops; i++ { 28 | go func() { 29 | wg.Done() 30 | }() 31 | } 32 | for i := 0; i < decLoops; i++ { 33 | go func() { 34 | wg.Done() 35 | }() 36 | } 37 | wg.Wait() 38 | } 39 | 40 | func main() { 41 | Items := []string{"1", "2", "3", "4", "5"} 42 | 43 | diff := len(Items) 44 | 45 | errCh := make(chan error, diff) 46 | var wg sync.WaitGroup 47 | wg.Add(diff) 48 | for _, i := range Items { 49 | // 参数避免使用指针，否则会出现同一个值问题 50 | go func(i string) { 51 | defer wg.Done() 52 | if err := Get(i); err != nil { 53 | errCh <- err 54 | } 55 | }(i) 56 | } 57 | wg.Wait() 58 | 59 | select { 60 | case err := <-errCh: 61 | if err != nil { 62 | fmt.Println(err.Error()) 63 | //return err 64 | } 65 | default: 66 | } 67 | } 68 | -------------------------------------------------------------------------------- /practise/image-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "flag" 5 | 6 | "github.com/caoyingjunz/pixiulib/config" 7 | "k8s.io/klog/v2" 8 | 9 | "go-learning/practise/image-practise/image" 10 | ) 11 | 12 | const ( 13 | defaultUser = "pixiu" 14 | defaultPassword = "123456" 15 | ) 16 | 17 | var ( 18 | harbor = flag.String("harbor", "harbor.cloud.pixiuio.com", "Choose a harbor to push (default harbor.cloud.pixiuio.com") 19 | imageRepository = flag.String("image-repository", "pixiuio", "Choose a container registry to push (default pixiuio") 20 | 21 | user = flag.String("user", "", "docker register user") 22 | password = flag.String("password", "", "docker register password") 23 | 24 | filePath = flag.String("configFile", "./config.yaml", "config file") 25 | ) 26 | 27 | func main() { 28 | klog.InitFlags(nil) 29 | flag.Parse() 30 | 31 | c := config.New() 32 | c.SetConfigFile(*filePath) 33 | c.SetConfigType("yaml") 34 | 35 | var cfg image.Config 36 | if err := c.Binding(&cfg); err != nil { 37 | klog.Fatal(err) 38 | } 39 | 40 | loginUser := *user 41 | if len(loginUser) == 0 || loginUser == "--password" { 42 | loginUser = defaultUser 43 | } 44 | loginPassword := *password 45 | if len(loginPassword) == 0 { 46 | loginPassword = defaultPassword 47 | } 48 | 49 | img := image.Image{ 50 | Harbor: *harbor, 51 | ImageRepository: *imageRepository, 52 | User: loginUser, 53 | Password: loginPassword, 54 | Cfg: cfg, 55 | } 56 | 57 | if err := img.Complete(); err != nil { 58 | klog.Fatal(err) 59 | } 60 | defer img.Close() 61 | 62 | if err := img.Validate(); err != nil { 63 | klog.Fatal(err) 64 | } 65 | 66 | if err := img.PushImages(); err != nil { 67 | klog.Fatal(err) 68 | } 69 | } 70 | -------------------------------------------------------------------------------- /practise/etcd-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "time" 7 | 8 | clientv3 "go.etcd.io/etcd/client/v3" 9 | ) 10 | 11 | func main() { 12 | etcd, err := clientv3.New(clientv3.Config{ 13 | Endpoints: []string{"http://127.0.0.1:12379"}, 14 | DialTimeout: 5 * time.Second, 15 | }) 16 | if err != nil { 17 | panic(err) 18 | } 19 | defer etcd.Close() 20 | 21 | // 写入 etcd 的值 22 | ctx, _ := context.WithTimeout(context.Background(), 5*time.Second) 23 | _, err = etcd.Put(ctx, "/config/name", "caoyingjun") 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | // 查询 etcd 的值 29 | resp, err := etcd.Get(ctx, "/config/name") 30 | if err != nil { 31 | panic(err) 32 | } 33 | for _, ev := range resp.Kvs { 34 | fmt.Printf("%s : %s\n", ev.Key, ev.Value) 35 | } 36 | // etcd支持key前缀匹配，添加 clientv3.WithPrefix()参数即可 37 | // 读取key前缀等于"/config"的所有值 38 | resp, err = etcd.Get(ctx, "/config", clientv3.WithPrefix()) 39 | if err != nil { 40 | panic(err) 41 | } 42 | // 遍历查询结果 43 | for _, ev := range resp.Kvs { 44 | fmt.Printf("%s : %s\n", ev.Key, ev.Value) 45 | } 46 | 47 | // 删除操作 48 | _, err = etcd.Delete(ctx, "/config/name") 49 | // 前缀删除 50 | _, err = etcd.Delete(ctx, "/config", clientv3.WithPrefix()) 51 | if err != nil { 52 | panic(err) 53 | } 54 | 55 | // 监听效果 56 | // 测试写入 57 | go func() { 58 | for { 59 | _, _ = etcd.Put(context.Background(), "/config/name", time.Now().String()) 60 | time.Sleep(2 * time.Second) 61 | } 62 | }() 63 | wChan := etcd.Watch(context.Background(), "/config/name", clientv3.WithPrefix()) // 监听key前缀的一组key的值 64 | for watchResp := range wChan { 65 | for _, event := range watchResp.Events { 66 | fmt.Printf("Event received! %s executed on %q with value %q\n", event.Type, event.Kv.Key, event.Kv.Value) 67 | } 68 | } 69 | } 70 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/app/server.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "os" 7 | "time" 8 | 9 | "github.com/spf13/cobra" 10 | "k8s.io/klog/v2" 11 | 12 | "go-learning/practise/cobra-practise/demo-server/app/options" 13 | ) 14 | 15 | func NewDemoCommand() *cobra.Command { 16 | o, err := options.NewOptions() 17 | if err != nil { 18 | klog.Fatalf("unable to initialize command options: %v", err) 19 | } 20 | 21 | cmd := &cobra.Command{ 22 | Use: "demo-server", 23 | Long: `The demo server controller is a daemon than embeds the core control loops shipped with demo.`, 24 | Run: func(cmd *cobra.Command, args []string) { 25 | if err = o.Complete(); err != nil { 26 | fmt.Fprintf(os.Stderr, "%v\n", err) 27 | os.Exit(1) 28 | } 29 | if err := Run(o); err != nil { 30 | fmt.Fprintf(os.Stderr, "%v\n", err) 31 | os.Exit(1) 32 | } 33 | }, 34 | Args: func(cmd *cobra.Command, args []string) error { 35 | for _, arg := range args { 36 | if len(arg) > 0 { 37 | return fmt.Errorf("%q does not take any arguments, got %q", cmd.CommandPath(), args) 38 | } 39 | } 40 | return nil 41 | }, 42 | } 43 | 44 | o.BindFlags(cmd) 45 | 46 | return cmd 47 | } 48 | 49 | func Run(c *options.Options) error { 50 | // 打印测试 51 | fmt.Println(c.ComponentConfig.Mysql) 52 | // 测试工厂函数 53 | fmt.Println(c.DBFactory.Test().Get("dd")) 54 | 55 | ctx, cancel := context.WithCancel(context.Background()) 56 | defer cancel() 57 | 58 | StartDemoServer(ctx) 59 | 60 | select {} 61 | } 62 | 63 | func StartDemoServer(ctx context.Context) { 64 | go func(ctx context.Context) { 65 | t := time.NewTicker(2 * time.Second) 66 | for { 67 | select { 68 | case <-ctx.Done(): 69 | fmt.Println("接到中断信号，退出!") 70 | return 71 | case <-t.C: 72 | fmt.Println("demo") 73 | } 74 | } 75 | }(ctx) 76 | } 77 | -------------------------------------------------------------------------------- /practise/grpc/README.md: -------------------------------------------------------------------------------- 1 | # gRPC Usage 2 | 3 | ### 目标 4 | 基于grpc的双向通信demo 5 | 6 | ### protoc (Protocol buffer compiler) 安装 7 | 8 | Install `protoc` from [protobuf](https://github.com/protocolbuffers/protobuf/releases) 9 | ```shell 10 | protoc --version # Ensure compiler version is 3+ 11 | ``` 12 | 13 | Install `Go plugins` for the `protoc` 14 | 15 | ```shell 16 | # Install the protocol compiler plugins for Go using the following commands: 17 | go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26 18 | go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1 19 | 20 | # Update your PATH so that the protoc compiler can find the plugins: 21 | export PATH="$PATH:$(go env GOPATH)/bin" 22 | ``` 23 | 24 | ### 构建 gRPC 服务 25 | 26 | 定义 `.proto` 文件，本文以 `pixiu.proto` 为例 27 | 28 | ```protobuf 29 | syntax="proto3"; 30 | 31 | option go_package = "go-learning/practise/grpc-practise/tunnel/tunnel"; 32 | 33 | package tunnel; 34 | 35 | service Tunnel { 36 | // Client 调用此方法建立连接 37 | rpc Connect(stream Request) returns (stream Response); 38 | } 39 | 40 | message Request { 41 | string type = 1; // "client_call" 或 "server_call" 42 | bytes payload = 2; 43 | } 44 | 45 | message Response { 46 | bytes result = 1; 47 | } 48 | ``` 49 | 50 | ### 生成 `gRPC` 代码 51 | 52 | ```shell 53 | protoc --go_out=. --go_opt=paths=source_relative \ 54 | --go-grpc_out=. --go-grpc_opt=paths=source_relative \ 55 | tunnel/tunnel.proto 56 | ``` 57 | 58 | 执行命令之后，会在 `tunnel` 目录生成 `tunnel.pb.go` 和 `tunnel_grpc.pb.go` 代码文件 59 | - `tunnel.pb.go` 结构体 60 | - `tunnel_grpc.pb.go`: 客户端和服务端代码 61 | 62 | ### 实现 gRPC 服务端 63 | ``` 64 | ``` 65 | 66 | ### 实现 gRPC 客户端 67 | ``` 68 | ``` 69 | 70 | ### 执行 71 | 72 | 启动 `gRPC server` 73 | ``` shell 74 | go run server.go 75 | ``` 76 | 77 | 执行 `gRPC client` 78 | ``` shell 79 | go run client.go 80 | 81 | # 回显 82 | 2022/03/20 19:43:13 say hello caoyingjun 12345 83 | ``` 84 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Go-Learning Overview 2 | 3 | Go-Learning's mission statement is: 4 | 5 | To provide a learning and practise map for OpenStack, kubernetes, golang, rust and the others. 6 | 7 | go-learning 适用于有一定 `kubernetes` 经验，且想更进一步的同学。 8 | 9 | - kubernetes 功能增强 [Pixiu(貔貅)](https://github.com/caoyingjunz/pixiu) 10 | - 快速部署 [kubez-ansible](https://github.com/caoyingjunz/kubez-ansible) 11 | - workload 自动扩缩容 [piuxiu-autoscaler](https://github.com/caoyingjunz/pixiu-autoscaler) 12 | 13 | ## Kubernetes 14 | - [clients 用法讲解](./practise/k8s-practise/clients.go) 15 | - [kubectl plugin 源码分析](./doc/kubernetes/kubectl-plugin.md) 16 | - [scheduler 源码分析一 - 启动](./doc/kubernetes/scheduler-start.md) 17 | - [scheduler 源码分析二 - 调度](./doc/kubernetes/scheduler-schedule.md) 18 | - [kubernetes 网络分析](./doc/kubernetes/network.md) 19 | - [kube-proxy 源码分析](./doc/kubernetes/kube-proxy.md) 20 | - [operator 用法详解](./doc/kubernetes/operator.md) 21 | - [code-generator](./doc/kubernetes/code-generator.md) 22 | - [CSI 注册机制源码分析](./doc/kubernetes/csi.md) 23 | - [cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack) 24 | 25 | ## Examples 26 | - [Examples](./examples/README.md) 提供丰富的 `kubernetes` 用法举例. 27 | - [pixiuctl](https://github.com/caoyingjunz/go-learning/tree/master/practise/cobra-practise/pixiuctl) 基于 [cobra](https://github.com/spf13/cobra) 实现命令行 28 | - subcommand 29 | - plugin 30 | - [gRPC Usage](./practise/grpc-practise/README.md) 31 | - [gin&informer](./practise/k8s-practise/gin-informer.go) 提供 `gin` 调用 `informer` 的用法 32 | - [webShell](https://github.com/caoyingjunz/kube-webshell) 提供 `webshell` 调用 `kubernetes` `pod` 的用法展示 33 | - [unit test](./practise/test-practise/README.md) 34 | 35 | ## Rust 36 | - TODO 37 | 38 | ## TODO 39 | - scheduler 代码分析(WIP) 40 | - kubelet 代码分析 41 | - 微服务学习（istio） 42 | - gc 机制分析 43 | - pod 驱逐代码分析 44 | - deployment 缩容 pod 的逻辑分析 45 | 46 | Copyright 2019 caoyingjun (cao.yingjunz@gmail.com) Apache License 2.0 47 | -------------------------------------------------------------------------------- /practise/exec-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | osexec "os/exec" 6 | ) 7 | 8 | // ErrExecutableNotFound is returned if the executable is not found. 9 | var ErrExecutableNotFound = osexec.ErrNotFound 10 | 11 | type Interface interface { 12 | Command(cmd string, args ...string) Cmd 13 | CommandInDir(cmd string, dir string, args ...string) Cmd 14 | 15 | // LookPath wraps os/exec.LookPath 16 | LookPath(file string) (string, error) 17 | } 18 | 19 | type Cmd interface { 20 | Run() error 21 | CombinedOutput() ([]byte, error) 22 | } 23 | 24 | func NewExec() (Interface, error) { 25 | execer := New() 26 | if _, err := execer.LookPath("ls"); err != nil { 27 | return nil, fmt.Errorf("%s is required for sail runtime", "python3") 28 | } 29 | 30 | return execer, nil 31 | } 32 | 33 | // Implements Interface in terms of really exec()ing. 34 | type executor struct{} 35 | 36 | // New returns a new Interface which will os/exec to run commands. 37 | func New() Interface { 38 | return &executor{} 39 | } 40 | 41 | // Command is part of the Interface interface. 42 | func (executor *executor) Command(cmd string, args ...string) Cmd { 43 | return osexec.Command(cmd, args...) 44 | } 45 | 46 | // Command is part of the Interface interface. 47 | func (executor *executor) CommandInDir(cmd string, dir string, args ...string) Cmd { 48 | cd := osexec.Command(cmd, args...) 49 | cd.Dir = dir 50 | 51 | return cd 52 | } 53 | 54 | // LookPath is part of the Interface interface 55 | func (executor *executor) LookPath(file string) (string, error) { 56 | return osexec.LookPath(file) 57 | } 58 | 59 | func main() { 60 | exec, err := NewExec() 61 | if err != nil { 62 | panic(err) 63 | } 64 | 65 | out, err := exec.Command("ls", "-al").CombinedOutput() 66 | if err != nil { 67 | panic(err) 68 | } 69 | 70 | // 切换 cmd 的执行目录 71 | out, err = exec.CommandInDir("ls", "/home", "-al").CombinedOutput() 72 | if err != nil { 73 | panic(err) 74 | } 75 | fmt.Println(string(out)) 76 | } 77 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module go-learning 2 | 3 | go 1.16 4 | 5 | require ( 6 | github.com/Masterminds/sprig/v3 v3.2.2 7 | github.com/caoyingjunz/csi-driver-localstorage v1.0.2 8 | github.com/caoyingjunz/pixiu v0.0.0-20230517113833-eab001ce6f5e 9 | github.com/caoyingjunz/pixiulib v1.0.0 10 | github.com/casbin/casbin/v2 v2.56.0 11 | github.com/casbin/gorm-adapter/v3 v3.12.1 12 | github.com/container-storage-interface/spec v1.7.0 13 | github.com/containerd/containerd v1.6.1 14 | github.com/docker/docker v17.12.0-ce-rc1.0.20200618181300-9dc6525e6118+incompatible 15 | github.com/emicklei/go-restful v2.9.5+incompatible 16 | github.com/fsnotify/fsnotify v1.5.1 17 | github.com/gin-gonic/gin v1.8.1 18 | github.com/gorilla/websocket v1.4.2 19 | github.com/hpcloud/tail v1.0.0 20 | github.com/jinzhu/gorm v1.9.12 21 | github.com/kubernetes-csi/csi-lib-utils v0.11.0 22 | github.com/pkg/errors v0.9.1 23 | github.com/pkg/sftp v1.13.4 24 | github.com/sirupsen/logrus v1.8.1 25 | github.com/spf13/cobra v1.6.0 26 | github.com/spf13/pflag v1.0.5 27 | github.com/xuri/excelize/v2 v2.7.0 28 | go.etcd.io/etcd/client/v3 v3.5.1 29 | go.uber.org/zap v1.22.0 30 | golang.org/x/crypto v0.5.0 31 | google.golang.org/grpc v1.54.0 32 | google.golang.org/protobuf v1.30.0 33 | gopkg.in/natefinch/lumberjack.v2 v2.0.0 34 | gopkg.in/yaml.v2 v2.4.0 35 | gorm.io/driver/mysql v1.4.1 36 | gorm.io/gorm v1.24.0 37 | k8s.io/api v0.24.8 38 | k8s.io/apimachinery v0.24.8 39 | k8s.io/cli-runtime v0.24.8 40 | k8s.io/client-go v0.24.8 41 | k8s.io/component-base v0.24.8 42 | k8s.io/controller-manager v0.24.8 43 | k8s.io/cri-api v0.23.1 44 | k8s.io/klog/v2 v2.90.1 45 | k8s.io/kubectl v0.24.8 46 | k8s.io/metrics v0.24.8 47 | ) 48 | 49 | require ( 50 | github.com/matttproud/golang_protobuf_extensions v1.0.2 // indirect 51 | github.com/moby/term v0.0.0-20220808134915-39b0c02b01ae // indirect 52 | github.com/onsi/gomega v1.23.0 // indirect 53 | github.com/prometheus/client_golang v1.14.0 // indirect 54 | k8s.io/utils v0.0.0-20221107191617-1a15be271d1d // indirect 55 | ) 56 | -------------------------------------------------------------------------------- /practise/http-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "errors" 5 | "fmt" 6 | "io/ioutil" 7 | "net" 8 | "net/http" 9 | "net/url" 10 | "strings" 11 | "time" 12 | ) 13 | 14 | // refer to https://blog.csdn.net/cheyo809775692/article/details/100924529 15 | 16 | const ( 17 | Url = "http://www.baidu.com" 18 | ) 19 | 20 | func main() { 21 | proxy := func(_ *http.Request) (*url.URL, error) { 22 | return url.Parse("http://127.0.0.1:1080") 23 | } 24 | 25 | noRedirect := func(req *http.Request, via []*http.Request) error { 26 | return errors.New("no Redirect") 27 | } 28 | 29 | transport := &http.Transport{ 30 | Proxy: proxy, 31 | IdleConnTimeout: time.Second * 2048, 32 | ResponseHeaderTimeout: time.Second * 5, 33 | DialContext: (&net.Dialer{ 34 | Timeout: 30 * time.Second, 35 | KeepAlive: 30 * time.Second, 36 | }).DialContext, 37 | } 38 | 39 | client := &http.Client{ 40 | // TODO: 可以在此处设置超时或者其他 41 | Transport: transport, 42 | CheckRedirect: noRedirect, 43 | } 44 | // 发送一个 POST 请求 45 | req, err := http.NewRequest("POST", Url, strings.NewReader("key=value")) 46 | if err != nil { 47 | fmt.Printf("NewRequest error: %v", err) 48 | return 49 | } 50 | 51 | // 2. 除了 timeout设置，也可以通过 context 52 | //ctx, _ := context.WithTimeout(context.Background(), time.Second) 53 | //req.WithContext(ctx) 54 | 55 | // 增加header 或者 cookes (可选) 56 | req.Header.Add("Content-Type", "application/json") 57 | 58 | // 增加 cookes (可选) 59 | cookie1 := &http.Cookie{Name: "name", Value: "caoyingjun", HttpOnly: true} 60 | req.AddCookie(cookie1) 61 | 62 | resp, err := client.Do(req) 63 | // 通过判断resp来准备是否close 64 | //if resp != nil { 65 | // defer resp.Body.Close() 66 | //} 67 | if err != nil { 68 | fmt.Printf("response error: %v", err) 69 | return 70 | } 71 | // 必须加 close 去关闭 连接 72 | // 需要在 err 处理后面，有的时候错误返回的时候，resp 为 nil 73 | defer resp.Body.Close() 74 | 75 | data, err := ioutil.ReadAll(resp.Body) 76 | if err != nil { 77 | fmt.Printf("%v", err) 78 | return 79 | } 80 | fmt.Print(string(data)) 81 | } 82 | -------------------------------------------------------------------------------- /practise/k8s-practise/metrics.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | 7 | "k8s.io/api/core/v1" 8 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 9 | cacheddiscovery "k8s.io/client-go/discovery/cached" 10 | "k8s.io/client-go/kubernetes" 11 | "k8s.io/client-go/restmapper" 12 | "k8s.io/controller-manager/pkg/clientbuilder" 13 | resourceclient "k8s.io/metrics/pkg/client/clientset/versioned/typed/metrics/v1beta1" 14 | 15 | "k8s.io/metrics/pkg/client/custom_metrics" 16 | "k8s.io/metrics/pkg/client/external_metrics" 17 | 18 | "go-learning/practise/k8s-practise/app" 19 | "go-learning/practise/k8s-practise/metrics" 20 | ) 21 | 22 | func main() { 23 | config, err := app.BuildClientConfig("") 24 | if err != nil { 25 | panic(err) 26 | } 27 | clientSet, err := kubernetes.NewForConfig(config) 28 | if err != nil { 29 | panic(err) 30 | } 31 | 32 | rootClientBuilder := clientbuilder.SimpleControllerClientBuilder{ 33 | ClientConfig: config, 34 | } 35 | discoveryClient := rootClientBuilder.DiscoveryClientOrDie("controller-discovery") 36 | cachedClient := cacheddiscovery.NewMemCacheClient(discoveryClient) 37 | mapper := restmapper.NewDeferredDiscoveryRESTMapper(cachedClient) 38 | 39 | apiVersionsGetter := custom_metrics.NewAvailableAPIsGetter(clientSet.Discovery()) 40 | 41 | metricsClient := metrics.NewRESTMetricsClient( 42 | resourceclient.NewForConfigOrDie(config), 43 | custom_metrics.NewForConfig(config, mapper, apiVersionsGetter), 44 | external_metrics.NewForConfigOrDie(config), 45 | ) 46 | 47 | deploy, err := clientSet.AppsV1().Deployments("default").Get(context.TODO(), "nginx", metav1.GetOptions{}) 48 | if err != nil { 49 | panic(err) 50 | } 51 | selector, _ := metav1.LabelSelectorAsSelector(deploy.Spec.Selector) 52 | // 获取 metrice cpu 53 | podMetricsInfo, time, err := metricsClient.GetResourceMetric(v1.ResourceCPU, "default", selector) 54 | if err != nil { 55 | panic(err) 56 | } 57 | 58 | fmt.Println("time", time) 59 | for name, metric := range podMetricsInfo { 60 | fmt.Println("name", name, "metric", metric) 61 | } 62 | } 63 | -------------------------------------------------------------------------------- /practise/grpc/server.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "io" 6 | "log" 7 | "net" 8 | "sync" 9 | 10 | "github.com/gin-gonic/gin" 11 | "google.golang.org/grpc" 12 | 13 | pd "go-learning/practise/grpc/tunnel" 14 | ) 15 | 16 | type server struct { 17 | pd.UnimplementedTunnelServer 18 | 19 | clients map[string]pd.Tunnel_ConnectServer 20 | lock sync.RWMutex 21 | } 22 | 23 | func (s *server) Connect(stream pd.Tunnel_ConnectServer) error { 24 | for { 25 | req, err := stream.Recv() 26 | if err == io.EOF { 27 | return nil 28 | } 29 | if err != nil { 30 | log.Printf("tream.Recv %v", err) 31 | return err 32 | } 33 | 34 | s.lock.Lock() 35 | _, ok := s.clients[req.Type] 36 | if !ok { 37 | s.clients[req.Type] = stream 38 | } 39 | s.lock.Unlock() 40 | 41 | // TODO 目前是DEMO 42 | log.Printf("Received from %s %s", req.Type, string(req.Payload)) 43 | } 44 | } 45 | func (s *server) Call(c *gin.Context) { 46 | _, _ = s.CallClient(c.Query("clientId"), nil) 47 | } 48 | 49 | func (s *server) CallClient(clientId string, data []byte) ([]byte, error) { 50 | stream, ok := s.clients[clientId] 51 | if !ok { 52 | return nil, fmt.Errorf("client not connected") 53 | } 54 | 55 | // 发送调用请求 56 | err := stream.Send(&pd.Response{ 57 | Result: []byte(clientId + " server callback"), 58 | }) 59 | if err != nil { 60 | return nil, err 61 | } 62 | 63 | return nil, err 64 | } 65 | 66 | func main() { 67 | listener, err := net.Listen("tcp", ":8092") 68 | if err != nil { 69 | log.Fatalf("failed to listen %v", err) 70 | } 71 | 72 | cs := &server{clients: make(map[string]pd.Tunnel_ConnectServer)} 73 | 74 | s := grpc.NewServer() 75 | pd.RegisterTunnelServer(s, cs) 76 | 77 | go func() { 78 | log.Printf("grpc listening at %v", listener.Addr()) 79 | if err = s.Serve(listener); err != nil { 80 | log.Fatalf("failed to serve %v", err) 81 | } 82 | }() 83 | 84 | r := gin.Default() 85 | r.GET("/ping", cs.Call) 86 | log.Printf("http listening at %v", ":8093") 87 | if err = r.Run(":8093"); err != nil { 88 | log.Fatalf("failed to start http: %v", err) 89 | } 90 | } 91 | -------------------------------------------------------------------------------- /practise/k8s-practise/gin-informer.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "log" 6 | "net/http" 7 | "path/filepath" 8 | 9 | "github.com/gin-gonic/gin" 10 | "k8s.io/apimachinery/pkg/labels" 11 | "k8s.io/apimachinery/pkg/runtime/schema" 12 | "k8s.io/client-go/informers" 13 | "k8s.io/client-go/kubernetes" 14 | "k8s.io/client-go/tools/clientcmd" 15 | "k8s.io/client-go/util/homedir" 16 | ) 17 | 18 | func main() { 19 | config, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config")) 20 | if err != nil { 21 | panic(err) 22 | } 23 | clientSet, err := kubernetes.NewForConfig(config) 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | ctx, cancel := context.WithCancel(context.Background()) 29 | defer cancel() 30 | 31 | sharedInformers := informers.NewSharedInformerFactory(clientSet, 0) 32 | // refer to https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/staging/src/k8s.io/client-go/dynamic/dynamicinformer/informer_test.go#L107 33 | 34 | // TODO: 可以追加更多的 gvr 35 | gvrs := []schema.GroupVersionResource{ 36 | {Group: "apps", Version: "v1", Resource: "deployments"}, 37 | {Group: "", Version: "v1", Resource: "pods"}, 38 | } 39 | for _, gvr := range gvrs { 40 | if _, err = sharedInformers.ForResource(gvr); err != nil { 41 | panic(err) 42 | } 43 | } 44 | 45 | // Start all informers. 46 | sharedInformers.Start(ctx.Done()) 47 | // Wait for all caches to sync. 48 | sharedInformers.WaitForCacheSync(ctx.Done()) 49 | 50 | log.Printf("all informers has been started") 51 | 52 | // 构造 pod podLister，用于 gin 的查询 53 | podLister := sharedInformers.Core().V1().Pods().Lister() 54 | // 启动 gin router 55 | // 仅做演示，无封装，无异常处理 56 | // 启动之后， curl 127.0.0.1:8088/pods 测试效果 57 | r := gin.Default() 58 | r.GET("/pods", func(c *gin.Context) { 59 | pods, err := podLister.List(labels.Everything()) 60 | if err != nil { 61 | panic(err) 62 | c.JSON(http.StatusBadRequest, gin.H{"message": err, "code": 400}) 63 | return 64 | } 65 | c.JSON(http.StatusOK, gin.H{"message": "pong", "code": 200, "result": pods}) 66 | }) 67 | 68 | _ = r.Run(":8088") 69 | } 70 | -------------------------------------------------------------------------------- /practise/gin-practise/log/zaplogging.go: -------------------------------------------------------------------------------- 1 | package log 2 | 3 | import ( 4 | "fmt" 5 | "io" 6 | "os" 7 | "time" 8 | 9 | "go.uber.org/zap" 10 | "go.uber.org/zap/zapcore" 11 | "gopkg.in/natefinch/lumberjack.v2" 12 | ) 13 | 14 | func TimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder) { 15 | enc.AppendString(t.Format("2006-01-02 15:04:05")) 16 | } 17 | 18 | func NewZapLogger(c Configuration) (Logger, error) { 19 | var w io.Writer 20 | switch c.LogFile { 21 | case "stdout": 22 | w = os.Stdout 23 | case "stderr": 24 | w = os.Stderr 25 | default: 26 | w = &lumberjack.Logger{ 27 | Filename: c.LogFile, 28 | MaxSize: c.RotateMaxSize, 29 | MaxAge: c.RotateMaxAge, 30 | MaxBackups: c.RotateMaxBackups, 31 | Compress: c.Compress, 32 | } 33 | } 34 | 35 | cfg := zapcore.EncoderConfig{ 36 | TimeKey: "time", 37 | LevelKey: "level", 38 | NameKey: "logger", 39 | CallerKey: "caller", 40 | MessageKey: "msg", 41 | StacktraceKey: "stacktrace", 42 | LineEnding: zapcore.DefaultLineEnding, 43 | EncodeLevel: zapcore.LowercaseLevelEncoder, 44 | EncodeTime: TimeEncoder, 45 | EncodeDuration: zapcore.SecondsDurationEncoder, 46 | EncodeCaller: zapcore.ShortCallerEncoder, 47 | EncodeName: zapcore.FullNameEncoder, 48 | } 49 | // 设置日志级别 50 | var level zapcore.Level 51 | if err := level.UnmarshalText([]byte(c.LogLevel)); err != nil { 52 | return nil, err 53 | } 54 | 55 | core := zapcore.NewCore(zapcore.NewConsoleEncoder(cfg), zapcore.NewMultiWriteSyncer(zapcore.AddSync(w)), level) 56 | 57 | var cores []zapcore.Core 58 | cores = append(cores, core) 59 | Tee := zapcore.NewTee(cores...) 60 | logger := zap.New(Tee, zap.AddCaller(), zap.AddCallerSkip(1)) 61 | return &zapLogger{ 62 | logger: logger, 63 | writer: w, 64 | verbosity: 0, 65 | }, nil 66 | } 67 | 68 | type zapLogger struct { 69 | logger *zap.Logger 70 | writer io.Writer 71 | verbosity int 72 | } 73 | 74 | func (l *zapLogger) Info(args ...interface{}) { 75 | l.logger.Info(fmt.Sprint(args...)) 76 | } 77 | 78 | func (l *zapLogger) Infof(f string, args ...interface{}) { 79 | l.logger.Info(fmt.Sprintf(f, args...)) 80 | } 81 | -------------------------------------------------------------------------------- /practise/gin-practise/middleware/middleware.go: -------------------------------------------------------------------------------- 1 | package middleware 2 | 3 | import ( 4 | "bytes" 5 | "encoding/json" 6 | "fmt" 7 | "io/ioutil" 8 | "time" 9 | 10 | "github.com/gin-gonic/gin" 11 | 12 | "go-learning/practise/gin-practise/log" 13 | ) 14 | 15 | const ( 16 | testName = "test" 17 | ) 18 | 19 | func Auth(c *gin.Context) { 20 | // TODO: 用于进行 auth 校验 21 | test := c.Query("test") 22 | if test == "err" { 23 | c.AbortWithStatusJSON(400, map[string]string{"error": "log"}) 24 | return 25 | } 26 | } 27 | 28 | type requestObject struct { 29 | Name string `json:"name"` 30 | } 31 | 32 | func (o *requestObject) validate() error { 33 | return nil 34 | } 35 | 36 | func parseObjectFromRequest(c *gin.Context) (*requestObject, error) { 37 | data, err := c.GetRawData() 38 | if err != nil { 39 | return nil, err 40 | } 41 | 42 | var obj requestObject 43 | if err = json.Unmarshal(data, &obj); err != nil { 44 | return nil, err 45 | } 46 | if err = obj.validate(); err != nil { 47 | return nil, err 48 | } 49 | 50 | c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(data)) // 回写字节流 51 | return &obj, nil 52 | } 53 | 54 | // AllowAccess 从 gin.Context 获取指定字段 55 | func AllowAccess() gin.HandlerFunc { 56 | return func(c *gin.Context) { 57 | fmt.Println("allow access start") 58 | o, err := parseObjectFromRequest(c) 59 | if err != nil { 60 | c.AbortWithStatusJSON(500, map[string]string{"error": err.Error()}) 61 | return 62 | } 63 | 64 | if o.Name == testName { 65 | c.AbortWithStatusJSON(400, map[string]string{"error": "test name"}) 66 | return 67 | } 68 | 69 | fmt.Println("access allow done") 70 | } 71 | } 72 | 73 | func LoggerToFile() gin.HandlerFunc { 74 | handlerFunc := func(c *gin.Context) { 75 | startTime := time.Now() 76 | 77 | // 处理请求操作 78 | c.Next() 79 | 80 | endTime := time.Now() 81 | 82 | latencyTime := endTime.Sub(startTime) 83 | 84 | reqMethod := c.Request.Method 85 | reqUri := c.Request.RequestURI 86 | statusCode := c.Writer.Status() 87 | clientIp := c.ClientIP() 88 | 89 | log.AccessLog.Infof("| %3d | %13v | %15s | %s | %s |", statusCode, latencyTime, clientIp, reqMethod, reqUri) 90 | } 91 | return handlerFunc 92 | } 93 | -------------------------------------------------------------------------------- /practise/cobra-practise/demo-server/app/options/options.go: -------------------------------------------------------------------------------- 1 | package options 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/jinzhu/gorm" 7 | _ "github.com/jinzhu/gorm/dialects/mysql" 8 | "github.com/spf13/cobra" 9 | 10 | "go-learning/practise/cobra-practise/demo-server/app/config" 11 | "go-learning/practise/cobra-practise/demo-server/dao" 12 | ) 13 | 14 | // Options has all the params needed to run a Autoscaler 15 | type Options struct { 16 | ComponentConfig *config.Config 17 | 18 | // ConfigFile is the location of the autoscaler's configuration file. 19 | ConfigFile string 20 | 21 | Master string 22 | 23 | DBFactory dao.ShareDBFactory 24 | 25 | DB *gorm.DB 26 | } 27 | 28 | func NewOptions() (*Options, error) { 29 | return &Options{ 30 | Master: "demo-master", 31 | }, nil 32 | } 33 | 34 | // BindFlags binds the demo Configuration struct fields 35 | func (o *Options) BindFlags(cmd *cobra.Command) { 36 | cmd.Flags().StringVar(&o.ConfigFile, "configfile", "", "ConfigFile is the location of the demo's configuration file.") 37 | } 38 | 39 | const ( 40 | defaultConfigFile = "democonfig.yaml" 41 | ) 42 | 43 | func (o *Options) registerDatabase() error { 44 | sqlConfig := o.ComponentConfig.Mysql 45 | dbConnection := fmt.Sprintf("%s:%s@tcp(%s:%s)/%s?charset=utf8&parseTime=True&loc=Local", 46 | sqlConfig.User, 47 | sqlConfig.Password, 48 | sqlConfig.Host, 49 | sqlConfig.Port, 50 | sqlConfig.Name) 51 | DB, err := gorm.Open("mysql", dbConnection) 52 | if err != nil { 53 | return err 54 | } 55 | 56 | // set the connect pools 57 | DB.DB().SetMaxIdleConns(10) 58 | DB.DB().SetMaxOpenConns(100) 59 | o.DB = DB 60 | 61 | // 注册或者通过 factory 的方式参数调用 62 | dao.Register(o.DB) 63 | return nil 64 | } 65 | 66 | func (o *Options) Complete() error { 67 | configFile := defaultConfigFile 68 | if len(o.ConfigFile) != 0 { 69 | configFile = o.ConfigFile 70 | } 71 | 72 | cfg, err := loadConfigFromFile(configFile) 73 | if err != nil { 74 | return err 75 | } 76 | o.ComponentConfig = cfg 77 | 78 | // 注册数据库链接池 79 | if err = o.registerDatabase(); err != nil { 80 | return err 81 | } 82 | 83 | // 构造数据库工厂函数 84 | o.DBFactory = dao.NewDBFactory(o.DB) 85 | 86 | // TODO 其他化客户端初始化 87 | 88 | return nil 89 | } 90 | -------------------------------------------------------------------------------- /practise/k8s-practise/scale.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 7 | "k8s.io/apimachinery/pkg/runtime/schema" 8 | "k8s.io/apimachinery/pkg/types" 9 | cacheddiscovery "k8s.io/client-go/discovery/cached" 10 | 11 | "k8s.io/client-go/dynamic" 12 | "k8s.io/client-go/kubernetes" 13 | "k8s.io/client-go/restmapper" 14 | "k8s.io/client-go/scale" 15 | 16 | "go-learning/practise/k8s-practise/app" 17 | ) 18 | 19 | func main() { 20 | config, err := app.BuildClientConfig("") 21 | if err != nil { 22 | panic(err) 23 | } 24 | clientSet, err := kubernetes.NewForConfig(config) 25 | if err != nil { 26 | panic(err) 27 | } 28 | cachedClient := cacheddiscovery.NewMemCacheClient(clientSet.Discovery()) 29 | 30 | scaleKindResolver := scale.NewDiscoveryScaleKindResolver(clientSet.Discovery()) 31 | scaleClient, err := scale.NewForConfig(config, restmapper.NewDeferredDiscoveryRESTMapper(cachedClient), dynamic.LegacyAPIPathResolverFunc, scaleKindResolver) 32 | 33 | gr := schema.GroupResource{ 34 | Group: "apps", 35 | Resource: "deployments", 36 | } 37 | 38 | // 1. 获取 scale, update scale 39 | sc, err := scaleClient.Scales("default").Get(context.TODO(), gr, "nginx", metav1.GetOptions{}) 40 | if err != nil { 41 | panic(err) 42 | } 43 | newSC := sc.DeepCopy() 44 | newSC.Spec.Replicas = newSC.Spec.Replicas + 1 45 | _, err = scaleClient.Scales("default").Update(context.TODO(), gr, newSC, metav1.UpdateOptions{}) 46 | if err != nil { 47 | panic(err) 48 | } 49 | 50 | //[ 51 | //{ "op": "test", "path": "/a/b/c", "value": "foo" }, 52 | //{ "op": "remove", "path": "/a/b/c" }, 53 | //{ "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] }, 54 | //{ "op": "replace", "path": "/a/b/c", "value": 42 }, 55 | //{ "op": "move", "from": "/a/b/c", "path": "/a/b/d" }, 56 | //{ "op": "copy", "from": "/a/b/d", "path": "/a/b/e" } 57 | //] 58 | 59 | // 2. scale patch 60 | payloadTemplate := `[{ "op": "%s", "path": "/spec/replicas", "value": %s }]` 61 | patchPayload := fmt.Sprintf(payloadTemplate, "replace", "1") 62 | if _, err = scaleClient.Scales("default").Patch(context.TODO(), gr.WithVersion("v1"), "nginx", types.JSONPatchType, []byte(patchPayload), metav1.PatchOptions{}); err != nil { 63 | panic(err) 64 | } 65 | } 66 | -------------------------------------------------------------------------------- /practise/fsnotify-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "os" 7 | "path/filepath" 8 | 9 | "github.com/fsnotify/fsnotify" 10 | ) 11 | 12 | // more info https://github.com/fsnotify/fsnotify 13 | // https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/pluginmanager/pluginwatcher/plugin_watcher.go#L60 14 | 15 | func handleCreateEvent(event fsnotify.Event) error { 16 | log.Println("Add file", event) 17 | fi, err := os.Stat(event.Name) 18 | if err != nil { 19 | return err 20 | } 21 | 22 | fmt.Println(fi.Name()) 23 | return nil 24 | } 25 | 26 | func handleDeleteEvent(event fsnotify.Event) error { 27 | log.Println("Delete file", event) 28 | 29 | _, name := filepath.Split(event.Name) 30 | fmt.Println(name) 31 | return nil 32 | } 33 | 34 | func Start(stopCh <-chan struct{}) error { 35 | fsWatcher, err := fsnotify.NewWatcher() 36 | if err != nil { 37 | log.Fatal(err) 38 | } 39 | 40 | // Traverse plugin dir and add filesystem watchers before starting the plugin processing goroutine. 41 | //if err := w.traversePluginDir(w.path); err != nil { 42 | // klog.ErrorS(err, "Failed to traverse plugin socket path", "path", w.path) 43 | //} 44 | 45 | // 监听两个文件夹 46 | if err = fsWatcher.Add("/Users/caoyuan/workstation/go-learning/practise/fsnotify-practise"); err != nil { 47 | log.Fatal(err) 48 | } 49 | if err = fsWatcher.Add("/Users/caoyuan/workstation/go-learning"); err != nil { 50 | log.Fatal(err) 51 | } 52 | 53 | go func(fsWatcher *fsnotify.Watcher) { 54 | for { 55 | select { 56 | case event := <-fsWatcher.Events: 57 | if event.Op&fsnotify.Create == fsnotify.Create { 58 | err = handleCreateEvent(event) 59 | if err != nil { 60 | fmt.Println("create event failed", err) 61 | } 62 | } else if event.Op&fsnotify.Remove == fsnotify.Remove { 63 | err = handleDeleteEvent(event) 64 | if err != nil { 65 | fmt.Println("remove event failed", err) 66 | } 67 | } 68 | case err = <-fsWatcher.Errors: 69 | log.Println("error:", err) 70 | case <-stopCh: 71 | fsWatcher.Close() 72 | } 73 | } 74 | }(fsWatcher) 75 | 76 | return nil 77 | } 78 | 79 | func main() { 80 | stopCh := make(chan struct{}) 81 | if err := Start(stopCh); err != nil { 82 | log.Fatal(err) 83 | } 84 | 85 | select {} 86 | } 87 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/create/create_service.go: -------------------------------------------------------------------------------- 1 | package create 2 | 3 | import ( 4 | "fmt" 5 | "k8s.io/kubectl/pkg/util/templates" 6 | 7 | "github.com/spf13/cobra" 8 | "k8s.io/cli-runtime/pkg/genericclioptions" 9 | cmdutil "k8s.io/kubectl/pkg/cmd/util" 10 | "k8s.io/kubectl/pkg/util/i18n" 11 | ) 12 | 13 | var ( 14 | serviceClusterIPLong = templates.LongDesc(i18n.T(` 15 | Create a ClusterIP service with the specified name.`)) 16 | 17 | serviceClusterIPExample = templates.Examples(i18n.T(` 18 | # Create a new ClusterIP service named my-cs (in headless mode) 19 | pixiuctl create service clusterip my-cs --clusterip="None"`)) 20 | ) 21 | 22 | // NewCmdCreateService is a macro command to create a new service 23 | func NewCmdCreateService(ioStreams genericclioptions.IOStreams) *cobra.Command { 24 | o := NewServiceOptions(ioStreams) 25 | 26 | cmd := &cobra.Command{ 27 | Use: "service", 28 | Aliases: []string{"svc"}, 29 | Short: i18n.T("Create a ClusterIP service"), 30 | Long: serviceClusterIPLong, 31 | Example: serviceClusterIPExample, 32 | Run: func(cmd *cobra.Command, args []string) { 33 | cmdutil.CheckErr(o.Complete(cmd, args)) 34 | cmdutil.CheckErr(o.Validate()) 35 | cmdutil.CheckErr(o.Run()) 36 | }, 37 | } 38 | 39 | cmd.Flags().StringVar(&o.Name, "name", o.Name, "usage name cluster ip") 40 | cmd.Flags().StringVar(&o.ClusterIP, "clusterip", o.ClusterIP, "usage cluster ip") 41 | 42 | return cmd 43 | } 44 | 45 | type ServiceOptions struct { 46 | Name string 47 | Namespace string 48 | 49 | ClusterIP string 50 | 51 | genericclioptions.IOStreams 52 | } 53 | 54 | func (o *ServiceOptions) Complete(cmd *cobra.Command, args []string) error { 55 | return nil 56 | } 57 | 58 | // Validate validates required fields are set to support structured generation 59 | func (o *ServiceOptions) Validate() error { 60 | if len(o.Name) == 0 { 61 | return fmt.Errorf("name must be specified") 62 | } 63 | 64 | return nil 65 | } 66 | 67 | func (o *ServiceOptions) Run() error { 68 | fmt.Println(fmt.Sprintf("create service: namespace %s, name %s, clusterip: %s", o.Namespace, o.Name, o.ClusterIP)) 69 | return nil 70 | } 71 | 72 | func NewServiceOptions(ioStreams genericclioptions.IOStreams) *ServiceOptions { 73 | return &ServiceOptions{ 74 | IOStreams: ioStreams, 75 | } 76 | } 77 | -------------------------------------------------------------------------------- /practise/k8s-practise/gin-informer2.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "log" 6 | "net/http" 7 | "path/filepath" 8 | 9 | "github.com/gin-gonic/gin" 10 | "k8s.io/apimachinery/pkg/labels" 11 | "k8s.io/client-go/informers" 12 | "k8s.io/client-go/kubernetes" 13 | "k8s.io/client-go/tools/cache" 14 | "k8s.io/client-go/tools/clientcmd" 15 | "k8s.io/client-go/util/homedir" 16 | ) 17 | 18 | func main() { 19 | config, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config")) 20 | if err != nil { 21 | panic(err) 22 | } 23 | clientSet, err := kubernetes.NewForConfig(config) 24 | if err != nil { 25 | panic(err) 26 | } 27 | 28 | ctx, cancel := context.WithCancel(context.Background()) 29 | defer cancel() 30 | 31 | sharedInformers := informers.NewSharedInformerFactory(clientSet, 0) 32 | // refer to https://github.com/kubernetes/kubernetes/blob/ea0764452222146c47ec826977f49d7001b0ea8c/staging/src/k8s.io/client-go/dynamic/dynamicinformer/informer_test.go#L107 33 | 34 | // TODO: 可以追加更多的 gvr 35 | //gvrs := []schema.GroupVersionResource{ 36 | // {Group: "apps", Version: "v1", Resource: "deployments"}, 37 | // {Group: "", Version: "v1", Resource: "pods"}, 38 | //} 39 | //for _, gvr := range gvrs { 40 | // if _, err = sharedInformers.ForResource(gvr); err != nil { 41 | // panic(err) 42 | // } 43 | //} 44 | 45 | lsListerSynced := sharedInformers.Core().V1().Pods().Informer().HasSynced 46 | 47 | // Start all informers. 48 | sharedInformers.Start(ctx.Done()) 49 | // Wait for all caches to sync. 50 | sharedInformers.WaitForCacheSync(ctx.Done()) 51 | 52 | if !cache.WaitForNamedCacheSync("pods", ctx.Done(), lsListerSynced) { 53 | return 54 | } 55 | 56 | log.Printf("all informers has been started") 57 | 58 | // 构造 pod podLister，用于 gin 的查询 59 | podLister := sharedInformers.Core().V1().Pods().Lister() 60 | // 启动 gin router 61 | // 仅做演示，无封装，无异常处理 62 | // 启动之后， curl 127.0.0.1:8088/pods 测试效果 63 | r := gin.Default() 64 | r.GET("/pods", func(c *gin.Context) { 65 | pods, err := podLister.Pods("default").List(labels.Everything()) 66 | if err != nil { 67 | panic(err) 68 | c.JSON(http.StatusBadRequest, gin.H{"message": err, "code": 400}) 69 | return 70 | } 71 | c.JSON(http.StatusOK, gin.H{"message": "pong", "code": 200, "result": pods}) 72 | }) 73 | 74 | _ = r.Run(":8088") 75 | } 76 | -------------------------------------------------------------------------------- /shares/20220825/README.md: -------------------------------------------------------------------------------- 1 | # Gin Web 框架的使用 2 | 3 | ## Gin 是什么 4 | [Gin](https://github.com/gin-gonic/gin) is a web framework written in Go (Golang) 5 | 6 | ## Installation 7 | ```shell 8 | go get -u github.com/gin-gonic/gin 9 | ``` 10 | 11 | ## 最简单的 demo 12 | 13 | ```shell 14 | package main 15 | 16 | import ( 17 | "net/http" 18 | 19 | "github.com/gin-gonic/gin" 20 | ) 21 | 22 | func main() { 23 | r := gin.Default() 24 | r.GET("/ping", func(c *gin.Context) { 25 | c.JSON(http.StatusOK, gin.H{"message": "pong"}) 26 | }) 27 | _ = r.Run() 28 | } 29 | ``` 30 | [demo](./demo.go) 31 | 32 | ## Gin 的参数处理 33 | 34 | - 从 URL 中获取参数 (GET DELETE) 35 | - query 参数 36 | ```shell 37 | GET http://127.0.0.1:8080/v1/detail?name=caoyingjun&age=18 38 | ``` 39 | 从 c.Query 中获取参数 40 | 41 | ``` shell 42 | name := c.Query("name") 43 | age := c.Query("age") 44 | // do somethings 45 | ``` 46 | 47 | - path参数 48 | ```shell 49 | GET http://127.0.0.1:8080/v1/name/caoyingjun/age/18 50 | ``` 51 | 从 c.Param 中获取参数 52 | ```shell 53 | name := c.Param("name") 54 | // do somethings 55 | ``` 56 | 57 | - 从 body 中获取参数 (POST PUT) 58 | 59 | - 通常是构造和参数匹配的结构体，然后进行 bind 60 | 61 | ```shell 62 | p := struct { 63 | Name string `json:"name,omitempty"` 64 | Age int `json:"age,omitempty"` 65 | }{} 66 | _ = c.ShouldBindJSON(&p) 67 | // do somethings 68 | ``` 69 | [demo](./parameter.go) 70 | 71 | ## Gin 的返回值处理 72 | 构造期望访问的结构体 73 | 74 | ```shell 75 | type Response struct { 76 | Code int `json:"code"` // 业务 code 77 | Result interface{} `json:"result,omitempty"` 78 | Message string `json:"message,omitempty"` 79 | } 80 | ``` 81 | 82 | 然后根据请求的处理结果进行赋值即可 83 | ```shell 84 | r := httputils.NewResponse() 85 | p := struct { 86 | Name string `json:"name,omitempty"` 87 | Age int `json:"age,omitempty"` 88 | }{} 89 | 90 | _ = c.ShouldBindJSON(&p) 91 | r.Result = p 92 | 93 | httputils.SetSuccess(c, r) 94 | ``` 95 | 96 | ## Gin 的中间件 97 | 通过在完成 gin 的初始化后，使用 Use 方法 98 | 99 | ```shell 100 | r := gin.Default() 101 | 102 | r.Use(LoggerToFile(), Auth) // 中间件 103 | ``` 104 | [demo](./parameter.go) 105 | 106 | ## 更多使用场景 107 | [more](https://github.com/gin-gonic/gin#gin-web-framework) -------------------------------------------------------------------------------- /practise/grpc-practise/csiclient/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "flag" 6 | "fmt" 7 | "os" 8 | "time" 9 | 10 | "github.com/container-storage-interface/spec/lib/go/csi" 11 | "github.com/kubernetes-csi/csi-lib-utils/connection" 12 | "github.com/kubernetes-csi/csi-lib-utils/metrics" 13 | csirpc "github.com/kubernetes-csi/csi-lib-utils/rpc" 14 | "k8s.io/klog/v2" 15 | ) 16 | 17 | var ( 18 | csiAddress = flag.String("csi-address", "/tmp/csi.sock", "Path of the CSI driver socket that the node-driver-registrar will connect to.") 19 | operationTimeout = flag.Duration("timeout", time.Second, "Timeout for waiting for communication with driver") 20 | ) 21 | 22 | func main() { 23 | flag.Parse() 24 | 25 | // Unused metrics manager, necessary for connection.Connect below 26 | cmm := metrics.NewCSIMetricsManagerForSidecar("") 27 | 28 | klog.V(0).Infof("Attempting to open a gRPC connection with: %q", *csiAddress) 29 | csiConn, err := connection.Connect(*csiAddress, cmm) 30 | if err != nil { 31 | klog.Errorf("error connecting to CSI driver: %v", err) 32 | os.Exit(1) 33 | } 34 | 35 | klog.V(0).Infof("Calling CSI driver to discover driver name") 36 | ctx, cancel := context.WithTimeout(context.Background(), *operationTimeout) 37 | defer cancel() 38 | 39 | // identityserver rpc 40 | csiDriverName, err := csirpc.GetDriverName(ctx, csiConn) 41 | if err != nil { 42 | klog.Errorf("error retreiving CSI driver name: %v", err) 43 | os.Exit(1) 44 | } 45 | fmt.Println("csiDriverName", csiDriverName) 46 | 47 | ready, err := csirpc.Probe(ctx, csiConn) 48 | if err != nil { 49 | klog.Errorf("error retreiving CSI Probe: %v", err) 50 | os.Exit(1) 51 | } 52 | fmt.Println("ready", ready) 53 | 54 | // nodeserver 55 | nodeClient := csi.NewNodeClient(csiConn) 56 | nodeInfo, err := nodeClient.NodeGetInfo(ctx, &csi.NodeGetInfoRequest{}) 57 | if err != nil { 58 | klog.Errorf("error NodeGetInfo: %v", err) 59 | os.Exit(1) 60 | } 61 | fmt.Println("NodeGetInfo", nodeInfo) 62 | 63 | // controllerserver rpc 64 | csiClient := csi.NewControllerClient(csiConn) 65 | resp, err := csiClient.CreateVolume(ctx, &csi.CreateVolumeRequest{ 66 | Name: "test-volume", 67 | VolumeCapabilities: []*csi.VolumeCapability{}, 68 | }) 69 | if err != nil { 70 | klog.Errorf("error CreateVolume: %v", err) 71 | os.Exit(1) 72 | } 73 | fmt.Println("create volume", resp) 74 | } 75 | -------------------------------------------------------------------------------- /practise/rbac-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "net/http" 6 | 7 | "github.com/casbin/casbin/v2" 8 | "github.com/casbin/gorm-adapter/v3" 9 | "github.com/gin-gonic/gin" 10 | _ "github.com/jinzhu/gorm/dialects/mysql" 11 | "gorm.io/driver/mysql" 12 | "gorm.io/gorm" 13 | ) 14 | 15 | func main() { 16 | user := "root" 17 | password := "password123456" 18 | ip := "pixiu01" 19 | port := 3306 20 | database := "rbacs" 21 | dbConnection := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?charset=utf8&parseTime=True&loc=Local", user, password, ip, port, database) 22 | // must declare the err to aviod panic: runtime error: invalid memory address or nil pointer dereferences 23 | var err error 24 | db, err := gorm.Open(mysql.Open(dbConnection), &gorm.Config{}) 25 | if err != nil { 26 | panic(err) 27 | } 28 | 29 | adapter, err := gormadapter.NewAdapterByDB(db) 30 | if err != nil { 31 | panic(err) 32 | } 33 | enforcer, err := casbin.NewEnforcer("./model.conf", adapter) 34 | if err != nil { 35 | panic(err) 36 | } 37 | 38 | // 从DB加载策略 39 | if err = enforcer.LoadPolicy(); err != nil { 40 | panic(err) 41 | } 42 | r := gin.Default() 43 | 44 | r.POST("/api/v1", func(c *gin.Context) { 45 | enforcer.AddPolicy("admin", "/api/v1/test", "GET") 46 | fmt.Println("Add policy") 47 | }) 48 | 49 | r.DELETE("/api/v1", func(c *gin.Context) { 50 | enforcer.RemovePolicy("admin", "/api/v1/test", "GET") 51 | fmt.Println("delete Policy") 52 | }) 53 | 54 | r.GET("/api/v1", func(c *gin.Context) { 55 | policies := enforcer.GetPolicy() 56 | for _, p := range policies { 57 | fmt.Println("policy", p) 58 | } 59 | }) 60 | 61 | //使用自定义拦截器中间件 62 | r.Use(Authorization(enforcer)) 63 | //创建请求 64 | r.GET("/api/v1/test", func(c *gin.Context) { 65 | fmt.Println("get v1 请求通过") 66 | }) 67 | r.POST("/api/v1/test", func(c *gin.Context) { 68 | fmt.Println("post v1 请求通过") 69 | }) 70 | 71 | _ = r.Run() 72 | } 73 | 74 | func Authorization(e *casbin.Enforcer) gin.HandlerFunc { 75 | return func(c *gin.Context) { 76 | //获取请求的URI 77 | obj := c.Request.URL.RequestURI() 78 | //获取请求方法 79 | act := c.Request.Method 80 | //获取用户的角色 81 | sub := "admin" 82 | 83 | if ok, err := e.Enforce(sub, obj, act); err == nil && ok { 84 | fmt.Println("验证通过") 85 | c.Next() 86 | } else { 87 | fmt.Println("无权访问") 88 | c.Abort() 89 | c.String(http.StatusUnauthorized, "无权访问") 90 | } 91 | } 92 | } 93 | -------------------------------------------------------------------------------- /doc/kubernetes/operator.md: -------------------------------------------------------------------------------- 1 | # Operator 用法详解 2 | 3 | ### Mac 安装 operator-sdk 4 | ``` bash 5 | curl -LO https://github.com/operator-framework/operator-sdk/releases/download/v1.21.0/operator-sdk_darwin_amd64 6 | chmod +x operator-sdk_darwin_amd64 7 | sudo cp operator-sdk_darwin_amd64 /usr/local/go/bin/operator-sdk 8 | 9 | or 10 | 11 | brew install operator-sdk 12 | ``` 13 | 14 | ### 文档 15 | * [官方文档](https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/) 16 | * [Example](http://www.dockone.io/article/8733) 17 | 18 | ### 版本要求 19 | - golang 1.17 20 | - operator-sdk v1.21.0 21 | 22 | ### 初始化 operator 项目 23 | ``` bash 24 | mkdir podset-operator 25 | cd podset-operator 26 | operator-sdk init --domain pixiu.io --repo github.com/caoyingjunz/podset-operator 27 | ``` 28 | 29 | ### 创建 API 和 controller 30 | ``` bash 31 | # Create a PodSet API with Group: pixiu, Version: v1beta1 and Kind: PodSet 32 | operator-sdk create api --group pixiu --version v1alpha1 --kind PodSet --resource --controller 33 | ``` 34 | 35 | ### 创建 Webhook 36 | ``` bash 37 | operator-sdk create webhook --group pixiu --version v1alpha1 --kind PodSet --defaulting --programmatic-validation 38 | ``` 39 | 40 | ### 生成 CRD, 生成 yaml 存放在 config/crd/bases 41 | ``` bash 42 | make manifests 43 | ``` 44 | 45 | ### 自定义控制器代码 46 | ```go 47 | // Reconcile is part of the main kubernetes reconciliation loop which aims to 48 | // move the current state of the cluster closer to the desired state. 49 | func (r *PodSetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { 50 | log := r.Log.WithValues("request", req) 51 | log.Info("reconciling operator") 52 | 53 | podSet := &pixiuv1alpha1.PodSet{} 54 | if err := r.Get(ctx, req.NamespacedName, podSet); err != nil { 55 | if apierrors.IsNotFound(err) { 56 | // Req object not found, Created objects are automatically garbage collected. 57 | // For additional cleanup logic use finalizers. 58 | // Return and don't requeue 59 | return reconcile.Result{}, nil 60 | } else { 61 | log.Error(err, "error requesting pod set operator") 62 | // Error reading the object - requeue the request. 63 | return reconcile.Result{Requeue: true}, nil 64 | } 65 | } 66 | ... 67 | ``` 68 | 69 | ### Build and push image 70 | ``` bash 71 | docker build -f Dockerfile . -t jacky06/podset-operator:v0.0.1 72 | docker push jacky06/podset-operator:v0.0.1 73 | ``` 74 | 75 | ### 部署 PodSet CRD 76 | CRD 来自 [podset-operator](https://github.com/caoyingjunz/podset-operator/tree/master/config/crd) 77 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/create/create.go: -------------------------------------------------------------------------------- 1 | package create 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/spf13/cobra" 7 | "k8s.io/cli-runtime/pkg/genericclioptions" 8 | cmdutil "k8s.io/kubectl/pkg/cmd/util" 9 | "k8s.io/kubectl/pkg/util/i18n" 10 | "k8s.io/kubectl/pkg/util/templates" 11 | 12 | pcmdutil "go-learning/practise/cobra-practise/pixiuctl/util" 13 | ) 14 | 15 | var ( 16 | createLong = templates.LongDesc(i18n.T(` 17 | Create a resource from a file or from stdin. 18 | 19 | JSON and YAML formats are accepted.`)) 20 | 21 | createExample = templates.Examples(i18n.T(` 22 | # Create a pod using the data in pod.json 23 | pixiuctl create -f ./create.json`)) 24 | ) 25 | 26 | type CreateOptions struct { 27 | Raw string 28 | EditBeforeCreate bool 29 | 30 | genericclioptions.IOStreams 31 | } 32 | 33 | // ValidateArgs makes sure there is no discrepency in command options 34 | func (o *CreateOptions) ValidateArgs(cmd *cobra.Command, args []string) error { 35 | 36 | return nil 37 | } 38 | 39 | // Complete completes all the required options 40 | func (o *CreateOptions) Complete(cmd *cobra.Command) error { 41 | fmt.Println("test create complete raw:", o.Raw) 42 | 43 | return nil 44 | } 45 | 46 | func (o *CreateOptions) RunCreate(cmd *cobra.Command) error { 47 | fmt.Println("run create edit", o.EditBeforeCreate, "raw", o.Raw) 48 | return nil 49 | } 50 | 51 | func NewCreateOptions(ioStreams genericclioptions.IOStreams) *CreateOptions { 52 | return &CreateOptions{ 53 | IOStreams: ioStreams, 54 | } 55 | } 56 | 57 | func NewCmdCreate(f pcmdutil.Factory, ioStreams genericclioptions.IOStreams) *cobra.Command { 58 | o := NewCreateOptions(ioStreams) 59 | 60 | cmd := &cobra.Command{ 61 | Use: "create -f FILENAME", 62 | DisableFlagsInUseLine: true, 63 | Short: i18n.T("Create a pixiu resource from a file or stdin"), 64 | Long: createLong, 65 | Example: createExample, 66 | Run: func(cmd *cobra.Command, args []string) { 67 | cmdutil.CheckErr(o.Complete(cmd)) 68 | cmdutil.CheckErr(o.ValidateArgs(cmd, args)) 69 | cmdutil.CheckErr(o.RunCreate(cmd)) 70 | }, 71 | } 72 | 73 | // 绑定参数 74 | cmd.Flags().BoolVar(&o.EditBeforeCreate, "edit", o.EditBeforeCreate, "Edit the API resource before creating") 75 | cmd.Flags().StringVar(&o.Raw, "raw", o.Raw, "Raw URI to POST to the server. Uses the transport specified by the kubeconfig file.") 76 | 77 | // create subcommands 78 | cmd.AddCommand(NewCmdCreateService(ioStreams)) 79 | return cmd 80 | } 81 | -------------------------------------------------------------------------------- /practise/gin-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "github.com/gin-gonic/gin" 5 | 6 | "go-learning/practise/gin-practise/endpoint" 7 | "go-learning/practise/gin-practise/log" 8 | "go-learning/practise/gin-practise/middleware" 9 | "go-learning/practise/gin-practise/worker" 10 | ) 11 | 12 | var https = ` 13 | ### 14 | GET http://127.0.0.1:8000/practise/get 15 | 16 | ### 17 | POST http://127.0.0.1:8000/practise/post 18 | 19 | { 20 | "name": "caoyingjun", 21 | "obj": {"k1": "v1", "k2": {"subk2": "subv2"}, "k3": "v3"}, 22 | "pers": {"id": 123, "age": 19, "sex": "boy", "max": {"k1": "v1"}, "lis": ["1", "2", "3"]} 23 | } 24 | 25 | ### 26 | POST http://127.0.0.1:8000/practise/queue?queue=test 27 | 28 | ### 29 | POST http://127.0.0.1:8000/practise/queue/after?after=after 30 | 31 | ### 32 | GET http://127.0.0.1:8000/practise/pod?name=mariadb-0&namespace=kubez-sysns&key=config 33 | ` 34 | 35 | type options struct { 36 | addr string 37 | engine *gin.Engine 38 | ws worker.WorkerInterface 39 | logDir string 40 | } 41 | 42 | func (c *options) registerHttpRoute() { 43 | c.engine.Use(middleware.LoggerToFile(), middleware.Auth) 44 | 45 | m := c.engine.Group("middleware", middleware.AllowAccess()) 46 | { 47 | m.POST("", endpoint.PostMid) 48 | } 49 | 50 | p := c.engine.Group("/practise") 51 | { 52 | p.GET("/get", endpoint.GetPractise) 53 | p.POST("/post", endpoint.PostPractise) 54 | p.POST("/queue", endpoint.TestQueue) 55 | p.POST("/queue/after", endpoint.TestAfterQueue) 56 | p.GET("/pod", endpoint.TestPod) 57 | p.POST("/optimise", endpoint.TestOptimise) 58 | p.GET("/download", endpoint.Download) 59 | } 60 | } 61 | 62 | func (c *options) run() { 63 | go func() { 64 | if err := c.engine.Run(c.addr); err != nil { 65 | panic(err) 66 | } 67 | }() 68 | 69 | go func() { 70 | stopCh := make(chan struct{}) 71 | defer close(stopCh) 72 | c.ws.Run(2, stopCh) 73 | }() 74 | } 75 | 76 | func (c *options) registerLog() { 77 | log.Register(c.logDir) 78 | } 79 | 80 | func (c *options) registerController() { 81 | p := endpoint.New() 82 | endpoint.Register(p) 83 | } 84 | 85 | func (c *options) registerWorker() { 86 | c.ws = worker.NewWorker() 87 | endpoint.RegisterWorker(c.ws) 88 | } 89 | 90 | func NewHttpServer(addr string) *options { 91 | o := &options{ 92 | addr: addr, 93 | engine: gin.Default(), 94 | } 95 | 96 | o.registerWorker() 97 | 98 | o.registerLog() 99 | 100 | o.registerHttpRoute() 101 | return o 102 | } 103 | 104 | func main() { 105 | gin.SetMode(gin.ReleaseMode) 106 | 107 | s := NewHttpServer(":8080") 108 | s.run() 109 | 110 | select {} 111 | } 112 | -------------------------------------------------------------------------------- /practise/sftp-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "os" 7 | "path" 8 | "time" 9 | 10 | "github.com/pkg/sftp" 11 | "golang.org/x/crypto/ssh" 12 | "k8s.io/client-go/util/homedir" 13 | ) 14 | 15 | func SftpConnect(user, passwd, host string, port int) (*sftp.Client, error) { 16 | // 1. 使用密码 17 | //clientConfig := &ssh.ClientConfig{ 18 | // User: user, 19 | // Auth: []ssh.AuthMethod{ 20 | // ssh.Password(passwd), 21 | // }, 22 | // Timeout: 30 * time.Second, 23 | // HostKeyCallback: func(hostname string, remote net.Addr, key ssh.PublicKey) error { 24 | // return nil 25 | // }, 26 | //} 27 | 28 | // 2. 使用公钥 29 | key, err := ioutil.ReadFile(path.Join(homedir.HomeDir(), ".ssh", "id_rsa")) 30 | if err != nil { 31 | return nil, err 32 | } 33 | signer, err := ssh.ParsePrivateKey(key) 34 | if err != nil { 35 | return nil, err 36 | } 37 | clientConfig := &ssh.ClientConfig{ 38 | User: user, 39 | Auth: []ssh.AuthMethod{ 40 | ssh.PublicKeys(signer), 41 | }, 42 | Timeout: 30 * time.Second, 43 | HostKeyCallback: ssh.InsecureIgnoreHostKey(), 44 | } 45 | 46 | addr := fmt.Sprintf("%s:%d", host, port) 47 | sshClient, err := ssh.Dial("tcp", addr, clientConfig) 48 | if err != nil { 49 | return nil, fmt.Errorf("ssh dial failed %v", err) 50 | } 51 | 52 | sftpClient, err := sftp.NewClient(sshClient) 53 | if err != nil { 54 | return nil, fmt.Errorf("new sftp client failed %v", err) 55 | } 56 | 57 | return sftpClient, nil 58 | } 59 | 60 | func CopyFromRemote(remoteFile, localFile, user, passwd, host string, port int) error { 61 | sftpClient, err := SftpConnect(user, passwd, host, port) 62 | if err != nil { 63 | return err 64 | } 65 | defer sftpClient.Close() 66 | 67 | srcFile, err := sftpClient.Open(remoteFile) 68 | if err != nil { 69 | return fmt.Errorf("open remote file %s failed %v", remoteFile, err) 70 | } 71 | defer srcFile.Close() 72 | 73 | // 直接读到内存 74 | //buf, err := io.ReadAll(srcFile) 75 | //if err != nil { 76 | // return nil, err 77 | //} 78 | 79 | f, err := os.Create(localFile) 80 | if err != nil { 81 | return fmt.Errorf("create file %s failed %v", localFile, err) 82 | } 83 | defer f.Close() 84 | 85 | if _, err = srcFile.WriteTo(f); err != nil { 86 | return fmt.Errorf("write to dest file %s failed %v", localFile, err) 87 | } 88 | 89 | return nil 90 | } 91 | 92 | func main() { 93 | remoteFile := "/root/test.txt" 94 | localFile := "/Users/caoyuan/test.txt" 95 | 96 | if err := CopyFromRemote(remoteFile, localFile, "root", "passwd", "host", 22); err != nil { 97 | fmt.Println("copy from remote file failed", err) 98 | } 99 | } 100 | -------------------------------------------------------------------------------- /practise/grpc-practise/README.md: -------------------------------------------------------------------------------- 1 | # gRPC Usage 2 | 3 | ### protoc (Protocol buffer compiler) 安装 4 | 5 | Install `protoc` from [protobuf](https://github.com/protocolbuffers/protobuf/releases) 6 | ```shell 7 | protoc --version # Ensure compiler version is 3+ 8 | ``` 9 | 10 | Install `Go plugins` for the `protoc` 11 | 12 | ```shell 13 | # Install the protocol compiler plugins for Go using the following commands: 14 | go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26 15 | go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1 16 | 17 | # Update your PATH so that the protoc compiler can find the plugins: 18 | export PATH="$PATH:$(go env GOPATH)/bin" 19 | ``` 20 | 21 | ### 构建 gRPC 服务 22 | 23 | 定义 `.proto` 文件，本文以 `pixiu.proto` 为例 24 | 25 | ```protobuf 26 | syntax="proto3"; 27 | 28 | option go_package = "go-learning/practise/grpc-practise/pixiu/pixiu"; 29 | 30 | package pixiu; 31 | 32 | service Pixiu { 33 | rpc GetPixiu (PixiuRequest) returns (PixiuReply) {} 34 | // 其他接口 35 | } 36 | 37 | // The request message. 38 | message PixiuRequest { 39 | int64 id = 1; 40 | string name = 2; 41 | } 42 | 43 | // The response message. 44 | message PixiuReply { 45 | string message = 1; 46 | } 47 | ``` 48 | 49 | ### 生成 `gRPC` 代码 50 | 51 | ```shell 52 | protoc --go_out=. --go_opt=paths=source_relative \ 53 | --go-grpc_out=. --go-grpc_opt=paths=source_relative \ 54 | pixiu/pixiu.proto 55 | ``` 56 | 57 | 执行命令之后，会在 `pixiu` 目录生成 `pixiu.pb.go` 和 `pixiu_grpc.pb.go` 代码文件 58 | - `pixiu.pb.go` 结构体 59 | - `pixiu_grpc.pb.go`: 客户端和服务端代码 60 | 61 | ### 实现 gRPC 服务端 62 | ``` go 63 | ... 64 | type server struct { 65 | pd.UnimplementedPixiuServer 66 | } 67 | 68 | func (s *server) GetPixiu(ctx context.Context, in *pd.PixiuRequest) (*pd.PixiuReply, error) { 69 | log.Printf("Received %s %d", in.Name, in.Id) 70 | return &pd.PixiuReply{Message: fmt.Sprintf("%s %d", in.GetName(), in.GetId())}, nil 71 | } 72 | 73 | func main() { 74 | l, _ := net.Listen("tcp", ":30000") 75 | 76 | s := grpc.NewServer() 77 | pd.RegisterPixiuServer(s, &server{}) 78 | 79 | if err = s.Serve(l); err != nil { 80 | ... 81 | } 82 | } 83 | ``` 84 | 85 | ### 实现 gRPC 客户端 86 | ``` go 87 | func main() { 88 | conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(insecure.NewCredentials())) 89 | if err != nil { 90 | log.Fatalf("failed to connect rpc server %v", err) 91 | } 92 | defer conn.Close() 93 | 94 | c := pd.NewPixiuClient(conn) 95 | 96 | ctx, cancel := context.WithTimeout(context.Background(), time.Second) 97 | defer cancel() 98 | 99 | r, err := c.GetPixiu(ctx, &pd.PixiuRequest{Id: 12345, Name: "caoyingjun"}) 100 | ... 101 | } 102 | ``` 103 | 104 | ### 执行 105 | 106 | 启动 `gRPC server` 107 | ``` shell 108 | go run server/server.go 109 | ``` 110 | 111 | 执行 `gRPC client` 112 | ``` shell 113 | go run client/client.go 114 | 115 | # 回显 116 | 2022/03/20 19:43:13 say hello caoyingjun 12345 117 | ``` 118 | -------------------------------------------------------------------------------- /practise/template-practise/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "io/ioutil" 7 | "os" 8 | "strings" 9 | "text/template" 10 | 11 | "github.com/Masterminds/sprig/v3" 12 | "gopkg.in/yaml.v2" 13 | 14 | pt "go-learning/practise/template-practise/template" 15 | ) 16 | 17 | // 内置一些用于比较的函数： 18 | // eq arg1 arg2： 19 | // arg1 == arg2时为true 20 | // ne arg1 arg2： 21 | // arg1 != arg2时为true 22 | // lt arg1 arg2： 23 | // arg1 < arg2时为true 24 | // le arg1 arg2： 25 | // arg1 <= arg2时为true 26 | // gt arg1 arg2： 27 | // arg1 > arg2时为true 28 | // ge arg1 arg2： 29 | // arg1 >= arg2时为true 30 | // 更多可以 参考 https://www.cnblogs.com/f-ck-need-u/p/10053124.html 31 | // 渲染 + 通用模板合并 32 | 33 | const ( 34 | // YAMLDocumentSeparator is the separator for YAML documents 35 | YAMLDocumentSeparator = "---\n" 36 | ) 37 | 38 | type Friend struct { 39 | Name string `json:"name"` 40 | } 41 | 42 | type Person struct { 43 | UserName string `json:"user_name"` 44 | Emails []string `json:"emails"` 45 | Friends []Friend `json:"friends"` 46 | Mods map[string]string `json:"mods"` 47 | Data string 48 | } 49 | 50 | func MergeBytes(data ...[]byte) []byte { 51 | return bytes.Join(data, []byte(YAMLDocumentSeparator)) 52 | } 53 | 54 | func toYAML(in interface{}) (string, error) { 55 | data, err := yaml.Marshal(in) 56 | if err != nil { 57 | return "", err 58 | } 59 | 60 | return strings.Trim(string(data), "\n"), nil 61 | } 62 | 63 | type Test struct { 64 | Name string `yaml:"name"` 65 | Age string `yaml:"age"` 66 | } 67 | 68 | func main() { 69 | fri := Friend{ 70 | Name: "name", 71 | } 72 | 73 | data, _ := toYAML(Test{Name: "caoyingjunz", Age: "18"}) 74 | 75 | emails := make([]string, 0) 76 | emails = append(emails, "test1@gmail.com") 77 | emails = append(emails, "test2@gmail.com") 78 | 79 | tpl := template.New("test").Funcs(sprig.TxtFuncMap()) 80 | tpl = template.Must(tpl.Parse(pt.ServiceTemplate)) 81 | 82 | m := make(map[string]string) 83 | m["m1"] = "v1" 84 | m["m2"] = "v2" 85 | 86 | p := Person{ 87 | UserName: "caoyingjun", 88 | Emails: emails, 89 | Friends: []Friend{fri}, 90 | Mods: m, 91 | Data: data, 92 | } 93 | 94 | f, err := os.Create("./service.yaml") 95 | if err != nil { 96 | panic(err) 97 | } 98 | defer f.Close() 99 | 100 | // 写入指定文件 101 | tpl.Execute(f, p) 102 | 103 | // 写入标准输出 104 | //tpl.Execute(os.Stdout, p) 105 | 106 | // 写入变量1 107 | var buf bytes.Buffer 108 | tpl.Execute(&buf, p) 109 | fmt.Println(string(buf.Bytes())) 110 | 111 | // 写入变量2 112 | //bs := strings.Builder{} 113 | //tpl.Execute(&bs, p) 114 | //fmt.Println(bs.String()) 115 | 116 | b, err := ioutil.ReadFile("./service.yaml") 117 | if err != nil { 118 | panic(err) 119 | } 120 | 121 | nb := MergeBytes([]byte(pt.CommonService), b) 122 | if err = ioutil.WriteFile("./merge-service.yaml", nb, 0640); err != nil { 123 | panic(err) 124 | } 125 | } 126 | -------------------------------------------------------------------------------- /practise/k8s-practise/clients.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | 7 | "go-learning/practise/k8s-practise/app" 8 | appsv1 "k8s.io/api/apps/v1" 9 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 10 | "k8s.io/apimachinery/pkg/runtime" 11 | "k8s.io/apimachinery/pkg/runtime/schema" 12 | 13 | "k8s.io/client-go/discovery" 14 | "k8s.io/client-go/dynamic" 15 | "k8s.io/client-go/kubernetes" 16 | "k8s.io/client-go/metadata" 17 | ) 18 | 19 | const ( 20 | defaultNamespace = "default" 21 | defaultObject = "nginx" 22 | ) 23 | 24 | var ( 25 | gvr = schema.GroupVersionResource{Group: "apps", Version: "v1", Resource: "deployments"} 26 | ) 27 | 28 | func main() { 29 | config, err := app.BuildClientConfig("") 30 | if err != nil { 31 | panic(err) 32 | } 33 | 34 | // 1. ClientSet 35 | // 代码地址: k8s.io/client-go/kubernetes 36 | // 调用 k8s 的内置资源，不能访问自定义资源 37 | clientSet, err := kubernetes.NewForConfig(config) 38 | if err != nil { 39 | panic(err) 40 | } 41 | ds, err := clientSet.AppsV1().Deployments(defaultNamespace).Get(context.TODO(), defaultObject, metav1.GetOptions{}) 42 | if err != nil { 43 | panic(err) 44 | } 45 | fmt.Println("clientSet", ds.Namespace, ds.Name) 46 | 47 | // 2. DynamicClient 48 | // 代码地址: k8s.io/client-go/dynamic 49 | // 通过 gvr 调用任意 k8s 资源，包括自定义资源 50 | dynamicClient, err := dynamic.NewForConfig(config) 51 | if err != nil { 52 | panic(err) 53 | } 54 | unstructured, err := dynamicClient.Resource(gvr).Namespace(defaultNamespace).Get(context.TODO(), defaultObject, metav1.GetOptions{}) 55 | if err != nil { 56 | panic(err) 57 | } 58 | var deployment appsv1.Deployment 59 | if err = runtime.DefaultUnstructuredConverter.FromUnstructured(unstructured.Object, &deployment); err != nil { 60 | panic(err) 61 | } 62 | fmt.Println("dynamic client", deployment.Namespace, deployment.Name) 63 | 64 | // 3. MetadataClient 65 | // 代码地址: k8s.io/client-go/metadata 66 | // 仅获取 k8s 对象的元数据 67 | // get access to a particular ObjectMeta schema without knowing the details of the version 68 | metadataClient, err := metadata.NewForConfig(config) 69 | if err != nil { 70 | panic(err) 71 | } 72 | obj, err := metadataClient.Resource(gvr).Namespace(defaultNamespace).Get(context.TODO(), defaultObject, metav1.GetOptions{}) 73 | if err != nil { 74 | panic(err) 75 | } 76 | fmt.Println("metadata Client", obj.Namespace, obj.Name) 77 | 78 | // 4. DiscoveryClient 79 | // 代码地址: k8s.io/client-go/discovery 80 | // To discover supported resources in the API server 81 | discoveryClient, err := discovery.NewDiscoveryClientForConfig(config) 82 | if err != nil { 83 | panic(err) 84 | } 85 | // get the supported resources for all groups and versions. 86 | _, APIResources, err := discoveryClient.ServerGroupsAndResources() 87 | if err != nil { 88 | panic(err) 89 | } 90 | _ = APIResources // 忽略打印 91 | fmt.Println("discovery Client") 92 | 93 | // 5. scaleClient 94 | // 直接调整副本数 95 | // https://github.com/caoyingjunz/go-learning/blob/master/practise/k8s-practise/scale.go 96 | 97 | // 6. metricClient 98 | // 获取资源的 metric 99 | // https://github.com/caoyingjunz/go-learning/blob/master/practise/k8s-practise/metrics.go 100 | } 101 | -------------------------------------------------------------------------------- /practise/cobra-practise/pixiuctl/apply/apply.go: -------------------------------------------------------------------------------- 1 | package apply 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/spf13/cobra" 7 | "k8s.io/cli-runtime/pkg/genericclioptions" 8 | cmdutil "k8s.io/kubectl/pkg/cmd/util" 9 | "k8s.io/kubectl/pkg/util/i18n" 10 | "k8s.io/kubectl/pkg/util/templates" 11 | 12 | pcmdutil "go-learning/practise/cobra-practise/pixiuctl/util" 13 | ) 14 | 15 | var ( 16 | applyLong = templates.LongDesc(i18n.T(` 17 | Apply a configuration to a resource by file name or stdin. 18 | The resource name must be specified. This resource will be created if it doesn't exist yet. 19 | To use 'apply', always create the resource initially with either 'apply' or 'create --save-config'. 20 | 21 | JSON and YAML formats are accepted. 22 | 23 | Alpha Disclaimer: the --prune functionality is not yet complete. Do not use unless you are aware of what the current state is. See https://issues.k8s.io/34274.`)) 24 | 25 | applyExample = templates.Examples(i18n.T(` 26 | # Apply the configuration in pod.json to a pod 27 | pixiuctl apply -f ./test.json`)) 28 | ) 29 | 30 | // NewCmdApply create the `apply` command 31 | func NewCmdApply(f pcmdutil.Factory, ioStreams genericclioptions.IOStreams) *cobra.Command { 32 | o := NewApplyOptions(ioStreams) 33 | 34 | cmd := &cobra.Command{ 35 | Use: "apply (-f FILENAME | -k DIRECTORY)", 36 | DisableFlagsInUseLine: true, 37 | Short: i18n.T("Apply a configuration to a pixiu resource by file name or stdin"), 38 | Long: applyLong, 39 | Example: applyExample, 40 | Run: func(cmd *cobra.Command, args []string) { 41 | cmdutil.CheckErr(o.Complete(cmd, args)) 42 | cmdutil.CheckErr(o.Validate(cmd)) 43 | cmdutil.CheckErr(o.Run()) 44 | }, 45 | } 46 | 47 | // 绑定 48 | o.AddFlags(cmd) 49 | 50 | return cmd 51 | } 52 | 53 | // ApplyOptions defines flags and other configuration parameters for the `apply` command 54 | type ApplyOptions struct { 55 | Kubeconfig string 56 | 57 | Name string 58 | Namespace string 59 | 60 | genericclioptions.IOStreams 61 | } 62 | 63 | func NewApplyOptions(ioStreams genericclioptions.IOStreams) *ApplyOptions { 64 | return &ApplyOptions{ 65 | IOStreams: ioStreams, 66 | } 67 | } 68 | 69 | func (o *ApplyOptions) AddFlags(cmd *cobra.Command) { 70 | cmd.Flags().StringVar(&o.Kubeconfig, "kubeconfig", o.Kubeconfig, "Path to the kubeconfig file to use for CLI requests.") 71 | cmd.Flags().StringVar(&o.Name, "name", o.Name, "Name to impersonate for the operation") 72 | cmd.Flags().StringVar(&o.Namespace, "namespace", o.Namespace, "Namespace to impersonate for the operation") 73 | } 74 | 75 | func (o *ApplyOptions) Complete(cmd *cobra.Command, args []string) error { 76 | fmt.Println("test apply complete name:", o.Name) 77 | 78 | return nil 79 | } 80 | 81 | // Just a demo 82 | func (o *ApplyOptions) Validate(cmd *cobra.Command) error { 83 | //if o.Namespace == "" { 84 | // return fmt.Errorf("invalied namespace") 85 | //} 86 | 87 | return nil 88 | } 89 | 90 | // Run executes the `apply` command. 91 | func (o *ApplyOptions) Run() error { 92 | // TODO: run with options 93 | fmt.Println(fmt.Sprintf("run apply command with Kubeconfig: %s, Namespace: %s, Name: %s", o.Kubeconfig, o.Namespace, o.Name)) 94 | return nil 95 | } 96 | -------------------------------------------------------------------------------- /shares/20220825/parameter.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "encoding/json" 6 | "fmt" 7 | "io/ioutil" 8 | "net/http" 9 | 10 | "github.com/gin-gonic/gin" 11 | 12 | "github.com/caoyingjunz/pixiu/api/server/httputils" 13 | ) 14 | 15 | // Auth 认证 16 | func Auth(c *gin.Context) { 17 | fmt.Println("auth") 18 | } 19 | 20 | // Limiter 限速 21 | func Limiter(c *gin.Context) { 22 | var limit bool 23 | if limit { 24 | c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"code": http.StatusForbidden, "message": "服务器繁忙，请稍后再试"}) 25 | return 26 | } 27 | 28 | fmt.Println("limiter") 29 | } 30 | 31 | func Audit() gin.HandlerFunc { 32 | return func(c *gin.Context) { 33 | obj := struct { 34 | Metadata struct { 35 | Name string `json:"name"` 36 | } `json:"metadata"` 37 | }{} 38 | 39 | _ = ShouldBindWith(c, &obj) 40 | 41 | c.Next() 42 | fmt.Println(c.Value("name")) 43 | } 44 | } 45 | 46 | // LoggerToFile 日志 47 | func LoggerToFile() gin.HandlerFunc { 48 | return func(c *gin.Context) { 49 | fmt.Println("log") 50 | } 51 | } 52 | 53 | func ShouldBindWith(c *gin.Context, v interface{}) error { 54 | rawData, err := c.GetRawData() 55 | if err != nil { 56 | return err 57 | } 58 | 59 | c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(rawData)) 60 | if err = json.Unmarshal(rawData, v); err != nil { 61 | return err 62 | } 63 | 64 | return nil 65 | } 66 | 67 | type Params struct { 68 | Name string `json:"name,omitempty" uri:"name" binding:"required" form:"name"` 69 | Age int `json:"age,omitempty" uri:"age" binding:"required" form:"age"` 70 | } 71 | 72 | type Objects struct { 73 | Metadata Metadata `json:"metadata"` 74 | } 75 | 76 | type Metadata struct { 77 | Name string `json:"name"` 78 | } 79 | 80 | func main() { 81 | r := gin.Default() 82 | 83 | r.Use(LoggerToFile(), Limiter, Auth, Audit()) // 中间件 84 | g1 := r.Group("/v1") 85 | { 86 | g1.GET("/detail", getParametersFromQuery) 87 | g1.GET("/name/:name/age/:age", getParametersFromPath) 88 | g1.POST("/create", getParametersFromBody) 89 | g1.PUT("/update", updateObject) 90 | } 91 | 92 | // TODO: the other groups 93 | _ = r.Run() 94 | } 95 | 96 | func updateObject(c *gin.Context) { 97 | r := httputils.NewResponse() 98 | var p Objects 99 | if err := c.ShouldBindJSON(&p); err != nil { 100 | httputils.SetFailed(c, r, err) 101 | return 102 | } 103 | 104 | r.Result = p 105 | httputils.SetSuccess(c, r) 106 | } 107 | 108 | func getParametersFromQuery(c *gin.Context) { 109 | r := httputils.NewResponse() 110 | var p Params 111 | _ = c.ShouldBindQuery(&p) 112 | 113 | r.Result = map[string]interface{}{"name": c.Query("name"), "age": c.Query("age")} 114 | httputils.SetSuccess(c, r) 115 | } 116 | 117 | func getParametersFromPath(c *gin.Context) { 118 | r := httputils.NewResponse() 119 | 120 | // do something 121 | var p Params 122 | c.ShouldBindQuery(&p) 123 | // do something 124 | 125 | r.Result = map[string]interface{}{"name": c.Param("name"), "age": c.Param("age")} 126 | httputils.SetSuccess(c, r) 127 | } 128 | 129 | func getParametersFromBody(c *gin.Context) { 130 | r := httputils.NewResponse() 131 | var p Params 132 | if err := c.ShouldBindJSON(&p); err != nil { 133 | httputils.SetFailed(c, r, err) 134 | return 135 | } 136 | 137 | // do something 138 | r.Result = p 139 | httputils.SetSuccess(c, r) 140 | 141 | c.Set("name", "caoyingjunz") 142 | } 143 | -------------------------------------------------------------------------------- /practise/grpc-practise/pixiu/pixiu_grpc.pb.go: -------------------------------------------------------------------------------- 1 | // Code generated by protoc-gen-go-grpc. DO NOT EDIT. 2 | 3 | package pixiu 4 | 5 | import ( 6 | context "context" 7 | grpc "google.golang.org/grpc" 8 | codes "google.golang.org/grpc/codes" 9 | status "google.golang.org/grpc/status" 10 | ) 11 | 12 | // This is a compile-time assertion to ensure that this generated file 13 | // is compatible with the grpc package it is being compiled against. 14 | // Requires gRPC-Go v1.32.0 or later. 15 | const _ = grpc.SupportPackageIsVersion7 16 | 17 | // PixiuClient is the client API for Pixiu service. 18 | // 19 | // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. 20 | type PixiuClient interface { 21 | GetPixiu(ctx context.Context, in *PixiuRequest, opts ...grpc.CallOption) (*PixiuReply, error) 22 | } 23 | 24 | type pixiuClient struct { 25 | cc grpc.ClientConnInterface 26 | } 27 | 28 | func NewPixiuClient(cc grpc.ClientConnInterface) PixiuClient { 29 | return &pixiuClient{cc} 30 | } 31 | 32 | func (c *pixiuClient) GetPixiu(ctx context.Context, in *PixiuRequest, opts ...grpc.CallOption) (*PixiuReply, error) { 33 | out := new(PixiuReply) 34 | err := c.cc.Invoke(ctx, "/pixiu.Pixiu/GetPixiu", in, out, opts...) 35 | if err != nil { 36 | return nil, err 37 | } 38 | return out, nil 39 | } 40 | 41 | // PixiuServer is the server API for Pixiu service. 42 | // All implementations must embed UnimplementedPixiuServer 43 | // for forward compatibility 44 | type PixiuServer interface { 45 | GetPixiu(context.Context, *PixiuRequest) (*PixiuReply, error) 46 | mustEmbedUnimplementedPixiuServer() 47 | } 48 | 49 | // UnimplementedPixiuServer must be embedded to have forward compatible implementations. 50 | type UnimplementedPixiuServer struct { 51 | } 52 | 53 | func (UnimplementedPixiuServer) GetPixiu(context.Context, *PixiuRequest) (*PixiuReply, error) { 54 | return nil, status.Errorf(codes.Unimplemented, "method GetPixiu not implemented") 55 | } 56 | func (UnimplementedPixiuServer) mustEmbedUnimplementedPixiuServer() {} 57 | 58 | // UnsafePixiuServer may be embedded to opt out of forward compatibility for this service. 59 | // Use of this interface is not recommended, as added methods to PixiuServer will 60 | // result in compilation errors. 61 | type UnsafePixiuServer interface { 62 | mustEmbedUnimplementedPixiuServer() 63 | } 64 | 65 | func RegisterPixiuServer(s grpc.ServiceRegistrar, srv PixiuServer) { 66 | s.RegisterService(&Pixiu_ServiceDesc, srv) 67 | } 68 | 69 | func _Pixiu_GetPixiu_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { 70 | in := new(PixiuRequest) 71 | if err := dec(in); err != nil { 72 | return nil, err 73 | } 74 | if interceptor == nil { 75 | return srv.(PixiuServer).GetPixiu(ctx, in) 76 | } 77 | info := &grpc.UnaryServerInfo{ 78 | Server: srv, 79 | FullMethod: "/pixiu.Pixiu/GetPixiu", 80 | } 81 | handler := func(ctx context.Context, req interface{}) (interface{}, error) { 82 | return srv.(PixiuServer).GetPixiu(ctx, req.(*PixiuRequest)) 83 | } 84 | return interceptor(ctx, in, info, handler) 85 | } 86 | 87 | // Pixiu_ServiceDesc is the grpc.ServiceDesc for Pixiu service. 88 | // It's only intended for direct use with grpc.RegisterService, 89 | // and not to be introspected or modified (even as a copy) 90 | var Pixiu_ServiceDesc = grpc.ServiceDesc{ 91 | ServiceName: "pixiu.Pixiu", 92 | HandlerType: (*PixiuServer)(nil), 93 | Methods: []grpc.MethodDesc{ 94 | { 95 | MethodName: "GetPixiu", 96 | Handler: _Pixiu_GetPixiu_Handler, 97 | }, 98 | }, 99 | Streams: []grpc.StreamDesc{}, 100 | Metadata: "pixiu/pixiu.proto", 101 | } 102 | -------------------------------------------------------------------------------- /practise/k8s-practise/infomer-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "log" 6 | 7 | corev1 "k8s.io/api/core/v1" 8 | v1 "k8s.io/apimachinery/pkg/apis/meta/v1" 9 | "k8s.io/client-go/informers" 10 | "k8s.io/client-go/kubernetes" 11 | "k8s.io/client-go/tools/cache" 12 | "k8s.io/client-go/tools/clientcmd" 13 | ) 14 | 15 | // assignedPod selects pods that are assigned (scheduled and running). 16 | func assignedPod(pod *corev1.Pod) bool { 17 | return pod.Spec.NodeName == "test" 18 | } 19 | 20 | func main() { 21 | config, err := clientcmd.BuildConfigFromFlags("", "/Users/caoyuan/.kube/config") 22 | if err != nil { 23 | panic(err) 24 | } 25 | clientset, err := kubernetes.NewForConfig(config) 26 | if err != nil { 27 | panic(err) 28 | } 29 | 30 | ctx, cancel := context.WithCancel(context.Background()) 31 | defer cancel() 32 | 33 | sharedInformers := informers.NewSharedInformerFactory(clientset, 0) 34 | 35 | // 用法 1 36 | //informer := sharedInformers.Core().V1().Services().Informer() 37 | //informer.AddEventHandler(cache.ResourceEventHandlerFuncs{ 38 | // AddFunc: func(obj interface{}) { 39 | // mObj := obj.(v1.Object) 40 | // log.Printf("New Services Added to Store: %s", mObj.GetName()) 41 | // }, 42 | // UpdateFunc: func(oldObj, newObj interface{}) { 43 | // oObj := oldObj.(v1.Object) 44 | // nObj := newObj.(v1.Object) 45 | // log.Printf("%s Services Updated to %s", oObj.GetName(), nObj.GetName()) 46 | // }, 47 | // DeleteFunc: func(obj interface{}) { 48 | // mObj := obj.(v1.Object) 49 | // log.Printf("Services Deleted from Store: %s", mObj.GetName()) 50 | // }, 51 | //}) 52 | //informer.Run(ctx.Done()) 53 | 54 | sharedInformers.Core().V1().Services().Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{ 55 | AddFunc: func(obj interface{}) { 56 | mObj := obj.(v1.Object) 57 | log.Printf("New Service Added to Store: %s", mObj.GetName()) 58 | }, 59 | UpdateFunc: func(oldObj, newObj interface{}) { 60 | oObj := oldObj.(v1.Object) 61 | nObj := newObj.(v1.Object) 62 | log.Printf("%s Service Updated to %s", oObj.GetName(), nObj.GetName()) 63 | }, 64 | DeleteFunc: func(obj interface{}) { 65 | mObj := obj.(v1.Object) 66 | log.Printf("Service Deleted from Store: %s", mObj.GetName()) 67 | }, 68 | }) 69 | 70 | sharedInformers.Core().V1().Pods().Informer().AddEventHandler(cache.FilteringResourceEventHandler{ 71 | Handler: cache.ResourceEventHandlerFuncs{ 72 | AddFunc: func(obj interface{}) { 73 | mObj := obj.(v1.Object) 74 | log.Printf("New Pod Added to Store: %s", mObj.GetName()) 75 | }, 76 | UpdateFunc: func(oldObj, newObj interface{}) { 77 | oObj := oldObj.(v1.Object) 78 | nObj := newObj.(v1.Object) 79 | log.Printf("%s Pod Updated to %s", oObj.GetName(), nObj.GetName()) 80 | }, 81 | DeleteFunc: func(obj interface{}) { 82 | mObj := obj.(v1.Object) 83 | log.Printf("Pod Deleted from Store: %s", mObj.GetName()) 84 | }, 85 | }, 86 | FilterFunc: func(obj interface{}) bool { 87 | switch t := obj.(type) { 88 | case *corev1.Pod: 89 | return assignedPod(t) 90 | case cache.DeletedFinalStateUnknown: 91 | if _, ok := t.Obj.(*corev1.Pod); ok { 92 | // The carried object may be stale, so we don't use it to check if 93 | // it's assigned or not. Attempting to cleanup anyways. 94 | return true 95 | } 96 | log.Printf("handle DeletedFinalStateUnknown error") 97 | return false 98 | default: 99 | log.Printf("handle object error") 100 | return false 101 | } 102 | }, 103 | }) 104 | 105 | // Start all informers. 106 | sharedInformers.Start(ctx.Done()) 107 | // Wait for all caches to sync. 108 | sharedInformers.WaitForCacheSync(ctx.Done()) 109 | 110 | log.Printf("informers has been started") 111 | select {} 112 | } 113 | -------------------------------------------------------------------------------- /practise/k8s-practise/monitor.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "path/filepath" 6 | "time" 7 | 8 | "k8s.io/apimachinery/pkg/apis/meta/v1" 9 | "k8s.io/apimachinery/pkg/runtime/schema" 10 | "k8s.io/apimachinery/pkg/util/wait" 11 | cacheddiscovery "k8s.io/client-go/discovery/cached" 12 | "k8s.io/client-go/informers" 13 | "k8s.io/client-go/restmapper" 14 | "k8s.io/client-go/tools/cache" 15 | "k8s.io/client-go/tools/clientcmd" 16 | "k8s.io/client-go/util/homedir" 17 | "k8s.io/controller-manager/pkg/clientbuilder" 18 | 19 | "go-learning/practise/k8s-practise/app" 20 | ) 21 | 22 | const ResourceResyncTime time.Duration = 0 23 | 24 | // monitor runs a Controller with a local stop channel. 25 | type monitor struct { 26 | controller cache.Controller 27 | store cache.Store 28 | 29 | stopCh chan struct{} 30 | } 31 | 32 | func (m *monitor) Run() { 33 | // 启动监控 34 | m.controller.Run(m.stopCh) 35 | } 36 | 37 | var ( 38 | sharedInformers informers.SharedInformerFactory 39 | restMapper *restmapper.DeferredDiscoveryRESTMapper 40 | ) 41 | 42 | func main() { 43 | config, err := clientcmd.BuildConfigFromFlags("", filepath.Join(homedir.HomeDir(), ".kube", "config")) 44 | if err != nil { 45 | panic(err) 46 | } 47 | rootClientBuilder := clientbuilder.SimpleControllerClientBuilder{ClientConfig: config} 48 | 49 | versionedClient := rootClientBuilder.ClientOrDie("shared-informers") 50 | sharedInformers = informers.NewSharedInformerFactory(versionedClient, ResourceResyncTime) 51 | 52 | if err := app.WaitForAPIServer(versionedClient, 10*time.Second); err != nil { 53 | panic(fmt.Errorf("failed to wait for apiserver being healthy: %v", err)) 54 | } 55 | 56 | stopCh := make(chan struct{}) 57 | // Use a discovery client capable of being refreshed. 58 | discoveryClient := rootClientBuilder.DiscoveryClientOrDie("controller-discovery") 59 | cachedClient := cacheddiscovery.NewMemCacheClient(discoveryClient) 60 | restMapper = restmapper.NewDeferredDiscoveryRESTMapper(cachedClient) 61 | go wait.Until(func() { 62 | restMapper.Reset() 63 | }, 30*time.Second, stopCh) 64 | 65 | gvrs := []schema.GroupVersionResource{ 66 | {Group: "apps", Version: "v1", Resource: "deployments"}, 67 | {Group: "", Version: "v1", Resource: "pods"}, 68 | } 69 | 70 | monitors := make(map[schema.GroupVersionResource]*monitor) 71 | for _, gvr := range gvrs { 72 | kind, err := restMapper.KindFor(gvr) 73 | if err != nil { 74 | panic(err) 75 | } 76 | 77 | c, s, err := controllerFor(gvr, kind) 78 | if err != nil { 79 | panic(err) 80 | } 81 | monitors[gvr] = &monitor{store: s, controller: c} 82 | } 83 | 84 | for kind, monitor := range monitors { 85 | if monitor.stopCh == nil { 86 | monitor.stopCh = make(chan struct{}) 87 | sharedInformers.Start(stopCh) 88 | fmt.Println("start monitor for ", kind) 89 | go monitor.Run() 90 | } 91 | } 92 | 93 | select {} 94 | } 95 | 96 | func controllerFor(resource schema.GroupVersionResource, kind schema.GroupVersionKind) (cache.Controller, cache.Store, error) { 97 | handlers := cache.ResourceEventHandlerFuncs{ 98 | // add the event to the dependencyGraphBuilder's graphChanges. 99 | AddFunc: func(obj interface{}) { 100 | mObj := obj.(v1.Object) 101 | fmt.Println("add", kind.String(), mObj.GetName(), mObj.GetFinalizers()) 102 | }, 103 | UpdateFunc: func(oldObj, newObj interface{}) { 104 | mObj := newObj.(v1.Object) 105 | fmt.Println("update", kind.String(), mObj.GetName(), mObj.GetFinalizers()) 106 | }, 107 | DeleteFunc: func(obj interface{}) { 108 | mObj := obj.(v1.Object) 109 | fmt.Println("delete", kind.String(), mObj.GetName(), mObj.GetFinalizers()) 110 | }, 111 | } 112 | shared, err := sharedInformers.ForResource(resource) 113 | if err != nil { 114 | panic(err) 115 | } 116 | shared.Informer().AddEventHandlerWithResyncPeriod(handlers, ResourceResyncTime) 117 | 118 | return shared.Informer().GetController(), shared.Informer().GetStore(), nil 119 | } 120 | -------------------------------------------------------------------------------- /practise/gin-practise/endpoint/endpoint.go: -------------------------------------------------------------------------------- 1 | package endpoint 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | 7 | "github.com/gin-gonic/gin" 8 | 9 | "go-learning/practise/gin-practise/hander" 10 | "go-learning/practise/gin-practise/k8s" 11 | "go-learning/practise/gin-practise/log" 12 | "go-learning/practise/gin-practise/worker" 13 | ) 14 | 15 | type GinResp struct { 16 | Code int `json:"code"` 17 | Resp interface{} `json:"resp,omitempty"` 18 | Message string `json:"message,omitempty"` 19 | } 20 | 21 | func (g *GinResp) SetCode(c int) { 22 | g.Code = c 23 | } 24 | func (g *GinResp) SetMessage(msg string) { 25 | g.Message = msg 26 | } 27 | 28 | type practiseDB struct { 29 | db interface{} 30 | s string 31 | } 32 | 33 | func (c *practiseDB) getPractise(db interface{}) *practiseDB { 34 | c.db = db 35 | return c 36 | } 37 | 38 | func (c *practiseDB) get() string { 39 | return c.s 40 | } 41 | 42 | var pdb = practiseDB{s: "ceshi db orm"} 43 | 44 | func GetPractise(c *gin.Context) { 45 | r := GinResp{} 46 | 47 | desc := pdb.getPractise("db-driver").get() 48 | log.Glog.Info(desc) 49 | 50 | log.Glog.Info("get GetPractise log") 51 | 52 | r.SetMessage("Get Practise Message") 53 | r.SetCode(200) 54 | 55 | c.JSON(200, r) 56 | } 57 | 58 | func PostMid(c *gin.Context) { 59 | fmt.Println("start mid test") 60 | n := struct { 61 | Name string `json:"name"` 62 | Age string `json:"age"` 63 | }{} 64 | 65 | if err := c.ShouldBindJSON(&n); err != nil { 66 | c.AbortWithStatusJSON(400, "bad request") 67 | return 68 | } 69 | 70 | c.JSON(200, "ok") 71 | } 72 | 73 | func PostPractise(c *gin.Context) { 74 | r := GinResp{} 75 | 76 | var gr hander.GinRequest 77 | var err error 78 | err = c.ShouldBindJSON(&gr) 79 | if err != nil { 80 | c.AbortWithStatusJSON(500, "bad request") 81 | return 82 | } 83 | 84 | if err = hander.Dohandler(context.TODO(), gr); err != nil { 85 | r.SetMessage(err.Error()) 86 | c.AbortWithStatusJSON(500, "bad request") 87 | return 88 | } 89 | 90 | r.SetMessage("Get Practise Message") 91 | r.SetCode(200) 92 | 93 | log.Glog.Info("Post Practise log") 94 | 95 | r.Resp = gr 96 | c.JSON(200, r) 97 | } 98 | 99 | var ( 100 | WorkerSet worker.WorkerInterface 101 | KubeEngine k8s.EngineInterface 102 | ) 103 | 104 | func RegisterWorker(w worker.WorkerInterface) { 105 | WorkerSet = w 106 | } 107 | 108 | func RegisterEngine(e k8s.EngineInterface) { 109 | KubeEngine = e 110 | } 111 | 112 | func TestQueue(c *gin.Context) { 113 | r := GinResp{} 114 | 115 | q := c.Query("queue") 116 | if err := WorkerSet.DoTest(context.TODO(), q); err != nil { 117 | r.SetMessage("test error queue") 118 | c.JSON(400, r) 119 | return 120 | } 121 | 122 | r.Resp = "test ok queue" 123 | c.JSON(200, r) 124 | } 125 | 126 | func TestAfterQueue(c *gin.Context) { 127 | r := GinResp{} 128 | 129 | q := c.Query("after") 130 | if err := WorkerSet.DoAfterTest(context.TODO(), q); err != nil { 131 | r.SetMessage("test error after queue") 132 | c.JSON(400, r) 133 | return 134 | } 135 | 136 | r.Resp = "test ok after queue" 137 | c.JSON(200, r) 138 | } 139 | 140 | func TestPod(c *gin.Context) { 141 | r := GinResp{} 142 | 143 | name := c.Query("name") 144 | namespace := c.Query("namespace") 145 | key := c.Query("key") 146 | 147 | pod, err := KubeEngine.GetPod(context.TODO(), key, name, namespace) 148 | if err != nil { 149 | r.SetMessage(err.Error()) 150 | c.JSON(400, r) 151 | return 152 | } 153 | 154 | r.Resp = pod 155 | c.JSON(200, r) 156 | } 157 | 158 | func TestOptimise(c *gin.Context) { 159 | r := GinResp{} 160 | // 在 Create 里面进行参数的解析和异常处理 161 | if err := Practise.Optimise(c).Create("dd"); err != nil { 162 | return 163 | } 164 | 165 | c.JSON(200, r) 166 | } 167 | 168 | func GetOptimise(c *gin.Context) { 169 | r := GinResp{} 170 | var err error 171 | // 在 get 里设置参数 172 | if r.Resp, err = Practise.Optimise(c).Get(); err != nil { 173 | return 174 | } 175 | } 176 | 177 | func Download(c *gin.Context) { 178 | filename := c.Query("filename") 179 | 180 | c.Header("Content-Type", "application/octet-stream") 181 | c.Header("Content-Disposition", "attachment; filename="+filename+".tar") 182 | c.File("/Users/xxx/yyy.tar") 183 | } 184 | -------------------------------------------------------------------------------- /practise/gorm-practise/dbstone/user.go: -------------------------------------------------------------------------------- 1 | package dbstone 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "fmt" 7 | "time" 8 | 9 | "github.com/jinzhu/gorm" 10 | 11 | "go-learning/practise/gorm-practise/models" 12 | ) 13 | 14 | var ( 15 | RecordNotUpdated = errors.New("record not updated") 16 | ) 17 | 18 | type UserInterface interface { 19 | Get(ctx context.Context, name string, age int) (*models.User, error) 20 | Create(ctx context.Context, user *models.User) error 21 | Update(ctx context.Context, name string, resourceVersion int64, updates map[string]interface{}) error 22 | Delete(ctx context.Context, uid int64) error 23 | 24 | List(ctx context.Context, name string) ([]models.User, error) 25 | 26 | ListByPage(ctx context.Context, name string, page int, pageSize int) ([]models.User, error) 27 | OptimisticUpdate(ctx context.Context, name string, resourceVersion int64, updates map[string]interface{}) error 28 | } 29 | 30 | func NewUserDB() UserInterface { 31 | return &UserDB{ 32 | dbstone: DB, 33 | } 34 | } 35 | 36 | type UserDB struct { 37 | dbstone *gorm.DB 38 | } 39 | 40 | func (u *UserDB) Get(ctx context.Context, name string, age int) (*models.User, error) { 41 | var user models.User 42 | // Frist 获取第一个 43 | // Find 获取满足条件，如果只有一个返回，返回最后一个 44 | if err := u.dbstone.Where("name = ? and age = ?", name, age).First(&user).Error; err != nil { 45 | return nil, err 46 | } 47 | return &user, nil 48 | } 49 | 50 | func (u *UserDB) List(ctx context.Context, name string) ([]models.User, error) { 51 | var users []models.User 52 | if err := u.dbstone.Where("name = ?", name).Find(&users).Error; err != nil { 53 | return nil, err 54 | } 55 | return users, nil 56 | } 57 | 58 | func (u *UserDB) Create(ctx context.Context, user *models.User) error { 59 | return u.dbstone.Create(&user).Error 60 | } 61 | 62 | func (u *UserDB) Update(ctx context.Context, name string, resourceVersion int64, updates map[string]interface{}) error { 63 | updates["resource_version"] = resourceVersion + 1 64 | updates["gmt_modified"] = time.Now() 65 | 66 | return u.dbstone.Model(&models.User{}). 67 | Where("name = ? and resource_version = ?", name, resourceVersion). 68 | Updates(updates).Error 69 | } 70 | 71 | func (u *UserDB) Delete(ctx context.Context, uid int64) error { 72 | return u.dbstone.Where("id = ?", uid).Delete(&models.User{}).Error 73 | } 74 | 75 | func (u *UserDB) GetRawUsers(names []string) (user []models.User, err error) { 76 | // 如果需要，强制使用索引 77 | if err = u.dbstone.Raw("select * from user force index(idx_user_name_age) where name in (?)", names).Scan(&user).Error; err != nil { 78 | return 79 | } 80 | return 81 | } 82 | 83 | // 分页查找 84 | func (u *UserDB) ListByPage(ctx context.Context, name string, page int, pageSize int) ([]models.User, error) { 85 | // 校验参数合理性 86 | if pageSize <= 0 || page <= 0 { 87 | return nil, fmt.Errorf("invalid page: %d or pageSize: %d", page, pageSize) 88 | } 89 | 90 | var total int 91 | if err := u.dbstone.Model(&models.User{}).Count(&total).Error; err != nil { 92 | return nil, err 93 | } 94 | pageNum := total / pageSize 95 | if total%pageSize != 0 { 96 | pageNum++ 97 | } 98 | // 限制下分页大小和范围 99 | if page > pageNum { 100 | page = pageNum 101 | } 102 | 103 | var users []models.User 104 | if err := u.dbstone.Limit(pageSize).Offset((page-1)*pageSize). 105 | Where("name = ?", name).Order("gmt_create desc"). 106 | Find(&users).Error; err != nil { 107 | return nil, err 108 | } 109 | 110 | // DEBUG 111 | fmt.Println("total:", total) 112 | fmt.Println("pageNum:", pageNum) 113 | return users, nil 114 | } 115 | 116 | // OptimisticUpdate 自定义乐观锁 117 | func (u *UserDB) OptimisticUpdate(ctx context.Context, name string, resourceVersion int64, updates map[string]interface{}) error { 118 | updates["resource_version"] = resourceVersion + 1 119 | updates["gmt_modified"] = time.Now() 120 | 121 | uc := u.dbstone.Model(&models.User{}). 122 | Where("name = ? and resource_version = ?", name, resourceVersion). 123 | Update(updates) 124 | 125 | if err := uc.Error; err != nil { 126 | return err 127 | } 128 | // RowsAffected 为 0 的时候，说明未匹配到更新 129 | if uc.RowsAffected == 0 { 130 | // 没有更新说明数据已经被其他进程修改 131 | return RecordNotUpdated 132 | } 133 | 134 | return nil 135 | } 136 | -------------------------------------------------------------------------------- /practise/gorestful-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | // repo: https://github.com/emicklei/go-restful 4 | // Refer to https://www.kubernetes.org.cn/1788.html 5 | // https://github.com/emicklei/go-restful/tree/master/examples 6 | import ( 7 | "fmt" 8 | "net/http" 9 | 10 | "github.com/emicklei/go-restful" 11 | ) 12 | 13 | type Userg struct { 14 | Id, Name, Age string 15 | } 16 | 17 | type UserResource struct { 18 | // normally one would use DAO (data access object) 19 | users map[string]Userg 20 | } 21 | 22 | type RequestBody struct { 23 | Name string 24 | Age string 25 | } 26 | 27 | func (u UserResource) Register(container *restful.Container) { 28 | // 创建新的WebService 29 | ws := new(restful.WebService) 30 | 31 | // 设定WebService对应的路径("/users")和支持的MIME类型(restful.MIME_XML/ restful.MIME_JSON) 32 | ws. 33 | Path("/users"). 34 | Consumes(restful.MIME_XML, restful.MIME_JSON). 35 | Produces(restful.MIME_JSON, restful.MIME_XML) // you can specify this per route as well 36 | 37 | // 添加路由： GET /{user-id} --> u.findUser 38 | // 带括号的表示变量，不带的是固定路径，如果不写 则404 39 | //ws.Route(ws.GET("/{user-id}/test/{user-name}").To(u.findUser)) 40 | ws.Route(ws.GET("/{user-id}").To(u.findUser)) 41 | 42 | // 添加路由： POST / --> u.updateUser 43 | ws.Route(ws.POST("").To(u.createUser)) 44 | 45 | // 添加路由： PUT /{user-id} --> u.createUser 46 | ws.Route(ws.PUT("/{user-id}").To(u.updateUser)) 47 | 48 | // 添加路由： DELETE /{user-id} --> u.removeUser 49 | ws.Route(ws.DELETE("/{user-id}").To(u.removeUser)) 50 | 51 | // 将初始化好的WebService添加到Container中 52 | container.Add(ws) 53 | } 54 | 55 | // GET http://127.0.0.1:8080/users/test?name=caoyingjun 56 | // GET http://127.0.0.1:8080/users/id2/test/user-name?k1=111&k2=222 57 | func (u UserResource) findUser(request *restful.Request, response *restful.Response) { 58 | // 获取user-id 和 url 中的变量方法 59 | // 获取指定 path 的路径变量 60 | userId := request.PathParameter("user-id") 61 | 62 | // 获取 k v 变量 63 | query := request.Request.URL.Query() 64 | name := query.Get("name") 65 | 66 | usr := u.users[userId] 67 | if len(usr.Id) == 0 { 68 | response.AddHeader("Content-Type", "text/plain") 69 | response.WriteErrorString(http.StatusNotFound, "User could not be found.") 70 | } else { 71 | response.WriteEntity(name) 72 | } 73 | } 74 | 75 | //POST http://127.0.0.1:8080/users 76 | //Content-Type: application/json 77 | // 78 | //{ 79 | //"name": "caoyingjun", 80 | //"age": "18" 81 | //} 82 | func (u *UserResource) createUser(request *restful.Request, response *restful.Response) { 83 | usr := Userg{ 84 | Id: request.PathParameter("user-id"), 85 | } 86 | 87 | err := request.ReadEntity(&usr) 88 | if err != nil { 89 | response.WriteEntity(usr) 90 | return 91 | } 92 | 93 | u.users[usr.Id] = usr 94 | // superfluous 多余的设置 95 | //response.WriteHeader(200) 96 | response.WriteEntity(usr) 97 | } 98 | 99 | //PUT http://127.0.0.1:8080/users/test 100 | //Content-Type: application/json 101 | // 102 | //{ 103 | //"name": "caoyingjun", 104 | //"age": "18" 105 | //} 106 | func (u *UserResource) updateUser(request *restful.Request, response *restful.Response) { 107 | usr := new(Userg) 108 | userId := request.PathParameter("user-id") 109 | // 将请求中的body获取 110 | err := request.ReadEntity(&usr) 111 | if err != nil { 112 | response.WriteEntity(usr) 113 | return 114 | } 115 | 116 | id := usr.Id 117 | name := usr.Name 118 | age := usr.Age 119 | fmt.Println(userId, id, name, age) 120 | u.users[usr.Id] = *usr 121 | response.WriteEntity(usr) 122 | } 123 | 124 | //DELETE http://127.0.0.1:8080/users/test 125 | func (u *UserResource) removeUser(request *restful.Request, response *restful.Response) { 126 | id := request.PathParameter("user-id") 127 | delete(u.users, id) 128 | // 设置返回的格式 129 | response.WriteEntity(map[string]string{"delete": "true"}) 130 | } 131 | 132 | func main() { 133 | // 创建一个空的Container 134 | wsContainer := restful.NewContainer() 135 | 136 | // 设定路由为CurlyRouter(快速路由) 137 | wsContainer.Router(restful.CurlyRouter{}) 138 | 139 | // 创建自定义的Resource Handle(此处为UserResource) 140 | u := UserResource{map[string]Userg{}} 141 | 142 | // 创建WebService，并将WebService加入到Container中 143 | u.Register(wsContainer) 144 | 145 | log.Printf("start listening on localhost:8080") 146 | server := &http.Server{Addr: ":8080", Handler: wsContainer} 147 | 148 | // 启动服务 149 | log.Fatal(server.ListenAndServe()) 150 | } 151 | -------------------------------------------------------------------------------- /practise/grpc/tunnel/tunnel_grpc.pb.go: -------------------------------------------------------------------------------- 1 | // Code generated by protoc-gen-go-grpc. DO NOT EDIT. 2 | 3 | package tunnel 4 | 5 | import ( 6 | context "context" 7 | grpc "google.golang.org/grpc" 8 | codes "google.golang.org/grpc/codes" 9 | status "google.golang.org/grpc/status" 10 | ) 11 | 12 | // This is a compile-time assertion to ensure that this generated file 13 | // is compatible with the grpc package it is being compiled against. 14 | // Requires gRPC-Go v1.32.0 or later. 15 | const _ = grpc.SupportPackageIsVersion7 16 | 17 | // TunnelClient is the client API for Tunnel service. 18 | // 19 | // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. 20 | type TunnelClient interface { 21 | // Client 调用此方法建立连接 22 | Connect(ctx context.Context, opts ...grpc.CallOption) (Tunnel_ConnectClient, error) 23 | } 24 | 25 | type tunnelClient struct { 26 | cc grpc.ClientConnInterface 27 | } 28 | 29 | func NewTunnelClient(cc grpc.ClientConnInterface) TunnelClient { 30 | return &tunnelClient{cc} 31 | } 32 | 33 | func (c *tunnelClient) Connect(ctx context.Context, opts ...grpc.CallOption) (Tunnel_ConnectClient, error) { 34 | stream, err := c.cc.NewStream(ctx, &Tunnel_ServiceDesc.Streams[0], "/tunnel.Tunnel/Connect", opts...) 35 | if err != nil { 36 | return nil, err 37 | } 38 | x := &tunnelConnectClient{stream} 39 | return x, nil 40 | } 41 | 42 | type Tunnel_ConnectClient interface { 43 | Send(*Request) error 44 | Recv() (*Response, error) 45 | grpc.ClientStream 46 | } 47 | 48 | type tunnelConnectClient struct { 49 | grpc.ClientStream 50 | } 51 | 52 | func (x *tunnelConnectClient) Send(m *Request) error { 53 | return x.ClientStream.SendMsg(m) 54 | } 55 | 56 | func (x *tunnelConnectClient) Recv() (*Response, error) { 57 | m := new(Response) 58 | if err := x.ClientStream.RecvMsg(m); err != nil { 59 | return nil, err 60 | } 61 | return m, nil 62 | } 63 | 64 | // TunnelServer is the server API for Tunnel service. 65 | // All implementations must embed UnimplementedTunnelServer 66 | // for forward compatibility 67 | type TunnelServer interface { 68 | // Client 调用此方法建立连接 69 | Connect(Tunnel_ConnectServer) error 70 | mustEmbedUnimplementedTunnelServer() 71 | } 72 | 73 | // UnimplementedTunnelServer must be embedded to have forward compatible implementations. 74 | type UnimplementedTunnelServer struct { 75 | } 76 | 77 | func (UnimplementedTunnelServer) Connect(Tunnel_ConnectServer) error { 78 | return status.Errorf(codes.Unimplemented, "method Connect not implemented") 79 | } 80 | func (UnimplementedTunnelServer) mustEmbedUnimplementedTunnelServer() {} 81 | 82 | // UnsafeTunnelServer may be embedded to opt out of forward compatibility for this service. 83 | // Use of this interface is not recommended, as added methods to TunnelServer will 84 | // result in compilation errors. 85 | type UnsafeTunnelServer interface { 86 | mustEmbedUnimplementedTunnelServer() 87 | } 88 | 89 | func RegisterTunnelServer(s grpc.ServiceRegistrar, srv TunnelServer) { 90 | s.RegisterService(&Tunnel_ServiceDesc, srv) 91 | } 92 | 93 | func _Tunnel_Connect_Handler(srv interface{}, stream grpc.ServerStream) error { 94 | return srv.(TunnelServer).Connect(&tunnelConnectServer{stream}) 95 | } 96 | 97 | type Tunnel_ConnectServer interface { 98 | Send(*Response) error 99 | Recv() (*Request, error) 100 | grpc.ServerStream 101 | } 102 | 103 | type tunnelConnectServer struct { 104 | grpc.ServerStream 105 | } 106 | 107 | func (x *tunnelConnectServer) Send(m *Response) error { 108 | return x.ServerStream.SendMsg(m) 109 | } 110 | 111 | func (x *tunnelConnectServer) Recv() (*Request, error) { 112 | m := new(Request) 113 | if err := x.ServerStream.RecvMsg(m); err != nil { 114 | return nil, err 115 | } 116 | return m, nil 117 | } 118 | 119 | // Tunnel_ServiceDesc is the grpc.ServiceDesc for Tunnel service. 120 | // It's only intended for direct use with grpc.RegisterService, 121 | // and not to be introspected or modified (even as a copy) 122 | var Tunnel_ServiceDesc = grpc.ServiceDesc{ 123 | ServiceName: "tunnel.Tunnel", 124 | HandlerType: (*TunnelServer)(nil), 125 | Methods: []grpc.MethodDesc{}, 126 | Streams: []grpc.StreamDesc{ 127 | { 128 | StreamName: "Connect", 129 | Handler: _Tunnel_Connect_Handler, 130 | ServerStreams: true, 131 | ClientStreams: true, 132 | }, 133 | }, 134 | Metadata: "tunnel/tunnel.proto", 135 | } 136 | -------------------------------------------------------------------------------- /practise/gin-practise/k8s/k8s.go: -------------------------------------------------------------------------------- 1 | package k8s 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "log" 7 | "os" 8 | "path/filepath" 9 | "strings" 10 | "sync" 11 | 12 | "github.com/fsnotify/fsnotify" 13 | "k8s.io/api/core/v1" 14 | metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" 15 | "k8s.io/client-go/kubernetes" 16 | "k8s.io/client-go/tools/clientcmd" 17 | ) 18 | 19 | type ClientInterface interface { 20 | Add(key string, obj *kubernetes.Clientset) 21 | Update(key string, obj *kubernetes.Clientset) 22 | Delete(key string) 23 | Get(key string) *kubernetes.Clientset 24 | } 25 | 26 | type ClientStore struct { 27 | lock sync.RWMutex 28 | 29 | items map[string]*kubernetes.Clientset 30 | } 31 | 32 | func (cs *ClientStore) Add(key string, obj *kubernetes.Clientset) { 33 | cs.lock.Lock() 34 | defer cs.lock.Unlock() 35 | 36 | cs.items[key] = obj 37 | } 38 | 39 | func (cs *ClientStore) Update(key string, obj *kubernetes.Clientset) { 40 | cs.lock.Lock() 41 | defer cs.lock.Unlock() 42 | 43 | cs.items[key] = obj 44 | } 45 | 46 | func (cs *ClientStore) Delete(key string) { 47 | cs.lock.Lock() 48 | defer cs.lock.Unlock() 49 | 50 | _, exits := cs.items[key] 51 | if exits { 52 | delete(cs.items, key) 53 | } 54 | } 55 | 56 | func (cs *ClientStore) Get(key string) *kubernetes.Clientset { 57 | cs.lock.Lock() 58 | defer cs.lock.Unlock() 59 | 60 | item, exits := cs.items[key] 61 | if !exits { 62 | return nil 63 | } 64 | 65 | return item 66 | } 67 | 68 | func NewClientStore() ClientInterface { 69 | return &ClientStore{ 70 | items: map[string]*kubernetes.Clientset{}, 71 | } 72 | } 73 | 74 | type EngineInterface interface { 75 | Start(stopCh chan struct{}) error 76 | 77 | GetPod(ctx context.Context, key string, name string, namespace string) (*v1.Pod, error) 78 | } 79 | 80 | type KubeEngine struct { 81 | client ClientInterface 82 | 83 | path string 84 | } 85 | 86 | // Start watches for the creation and deletion of plugin sockets at the path 87 | func (k *KubeEngine) Start(stopCh chan struct{}) error { 88 | fmt.Println("Plugin Watcher Start at", k.path) 89 | 90 | fsWatcher, err := fsnotify.NewWatcher() 91 | if err != nil { 92 | return err 93 | } 94 | 95 | // Traverse plugin dir and add filesystem watchers before starting the plugin processing goroutine. 96 | //if err := w.traversePluginDir(w.path); err != nil { 97 | // klog.ErrorS(err, "Failed to traverse plugin socket path", "path", w.path) 98 | //} 99 | 100 | err = fsWatcher.Add(k.path) 101 | if err != nil { 102 | return err 103 | } 104 | 105 | go func(fsWatcher *fsnotify.Watcher) { 106 | for { 107 | select { 108 | case event := <-fsWatcher.Events: 109 | if event.Op&fsnotify.Create == fsnotify.Create { 110 | if err = k.handleCreateEvent(event); err != nil { 111 | fmt.Println("create event failed", err) 112 | } 113 | } else if event.Op&fsnotify.Remove == fsnotify.Remove { 114 | if err = k.handleDeleteEvent(event); err != nil { 115 | fmt.Println("delete event failed", err) 116 | } 117 | } 118 | case err = <-fsWatcher.Errors: 119 | log.Println("error:", err) 120 | case <-stopCh: 121 | fsWatcher.Close() 122 | } 123 | } 124 | }(fsWatcher) 125 | 126 | return nil 127 | } 128 | 129 | func (k *KubeEngine) GetPod(cxt context.Context, key string, name string, namespace string) (*v1.Pod, error) { 130 | clientSet := k.client.Get(key) 131 | if clientSet == nil { 132 | return nil, fmt.Errorf("%s not register", key) 133 | } 134 | 135 | pod, err := clientSet.CoreV1().Pods(namespace).Get(context.TODO(), name, metav1.GetOptions{}) 136 | if err != nil { 137 | return nil, err 138 | } 139 | 140 | return pod, nil 141 | } 142 | 143 | func (k *KubeEngine) handleCreateEvent(event fsnotify.Event) error { 144 | client, err := k.newClient(event.Name) 145 | if err != nil { 146 | //打印log 147 | return err 148 | } 149 | 150 | fi, err := os.Stat(event.Name) 151 | if err != nil { 152 | return err 153 | } 154 | 155 | if strings.HasPrefix(fi.Name(), ".") || fi.IsDir() { 156 | fmt.Println("Ignoring file (starts with '.') or dir", "path", fi.Name()) 157 | return nil 158 | } 159 | 160 | name := fi.Name() 161 | 162 | k.client.Add(name, client) 163 | fmt.Println(name, "register") 164 | return nil 165 | } 166 | 167 | func (k *KubeEngine) handleDeleteEvent(event fsnotify.Event) error { 168 | _, name := filepath.Split(event.Name) 169 | 170 | k.client.Delete(name) 171 | fmt.Println(name, "unregister") 172 | return nil 173 | } 174 | 175 | func (k *KubeEngine) newClient(kubeConfig string) (*kubernetes.Clientset, error) { 176 | config, err := clientcmd.BuildConfigFromFlags("", kubeConfig) 177 | if err != nil { 178 | return nil, err 179 | } 180 | 181 | return kubernetes.NewForConfig(config) 182 | } 183 | 184 | func NewKubeEngine() EngineInterface { 185 | return &KubeEngine{ 186 | client: NewClientStore(), 187 | path: "/Users/caoyuan/workstation/go-learning/practise/gin-practise", 188 | } 189 | } 190 | -------------------------------------------------------------------------------- /doc/learning.md: -------------------------------------------------------------------------------- 1 | https://www.nowcoder.com/ta/huawei 2 | 3 | ### +++ 4 | 5 | 1. 了解底层原理才能有效地学习 +++ 6 | 2. 需要了解论点背后的逻辑 +++ 7 | 3. 信守自己的承诺 +++ 8 | 4. 能解决问题 +++ 9 | 5. 必须找到解决问题的办法 +++ 10 | 6. 守时是重要原则 +++ 11 | 7. 我一向准时到达 +++ 12 | 8. 能想出很多注意 +++ 13 | 9. 善于照章行事 +++ 14 | 10. 倾向于通过实践来学习 +++ 15 | 11. 喜欢在有压力的环境中工作 +++ 16 | 12. 有强烈的好奇心 +++ 17 | 13. 喜欢提出很多主意 +++ 18 | 14. 乐意去适应新的挑战 +++ 19 | 15. 工作繁忙时，仍能把事情处理的很好 +++ 20 | 16. 倾向于根据客观事实做决定 +++ 21 | 17. 做事的时候需要有章可循 +++ 22 | 18. 能制定有效的策略 +++ 23 | 19. 擅长倾听他人 +++ 24 | 20. 渴望能迅速从挫折中恢复 +++ 25 | 21. 喜欢抽象思想 +++ 26 | 22. 善于同时处理多项任务 +++ 27 | 23. 对变化秉持积极的态度 +++ 28 | 24. 仅根据客观事实做决策 +++ 29 | 25. 在重大活动期间，我很少紧张 +++ 30 | 26. 能够很快从挫折中恢复过来 +++ 31 | 27. 擅长遵守规则 +++ 32 | 28. 我是一个乐观的人 +++ 33 | 29. 我觉得人们信守承诺相当重要 +++ 34 | 30. 我善于完成任务 +++ 35 | 31. 我非常肯定自己的价值 +++ 36 | 32. 我能够了解论点背后的逻辑 +++ 37 | 33. 我认为无论在任何情况下，我都必须持之以恒 +++ 38 | 34. 我觉得在重大活动前没必要担心 +++ 39 | 35. 我善于快节奏的工作 +++ 40 | 36. 我喜欢工作繁忙 +++ 41 | 37. 我倾向于保持乐观的态度 +++ 42 | 38. 我善于提出新概念 +++ 43 | 39. 在重大活动前，我能保持冷静 +++ 44 | 40. 我非常喜欢充满活力的感觉 +++ 45 | 41. 我是一个开心的人 +++ 46 | 42. 我能够确保高质量水平 +++ 47 | 43. 我善于在团队中工作 +++ 48 | 44. 我能在有压力的环境下很好的工作 +++ 49 | 50 | ### --- 51 | 52 | 1. 掌握着自己的未来 --- 53 | 2. 相信我能决定自己的未来 --- 54 | 3. 善于解决朋友 --- 55 | 4. 从发现商业良机中获得真正的满足感 --- 56 | 5. 善于辨别商机 --- 57 | 6. 喜欢解释陌生人 --- 58 | 7. 觉得必须参考他人的观点 --- 59 | 8. 从指定策略中获得真正的满足感 --- 60 | 9. 通过阅读学习很多东西 --- 61 | 10. 我喜欢通过阅读来学习 --- 62 | 11. 很快和别人建立融洽的关系 --- 63 | 12. 善于处理不确定的事 --- 64 | 13. 善于发现如何对事情进行改进 — ? 65 | 14. 善于理解人们行为背后的动机 --- 66 | 15. 人们都说我能给人留下好的第一印象 --- 67 | 16. 善于谈判 --- 68 | 17. 已经准备好随时做出重要决策 --- 69 | 18. 渴望达成交易 --- 70 | 19. 喜欢与人争辩 --- 71 | 20. 善于进行掌控 --- 72 | 21. 希望能负责重大决策 --- 73 | 22. 善于挑战他人的观点 --- 74 | 23. 我善于体会他人的感受 --- 75 | 24. 我希望对事物进行掌控 --- 76 | 25. 我是一个求胜心很强的人 --- 77 | 78 | ### + 79 | 80 | 1. 能组合出有效的机会 + 81 | 2. 善于着手开展工作 + 82 | 3. 良好的书面沟通能力 + 83 | 4. 喜欢写作 + 84 | 5. 擅长把问题解释清楚 + 85 | 6. 喜欢把问题解释清楚 + 86 | 7. 人们说我活泼 + 87 | 8. 更喜欢低风险的选择 + 88 | 9. 避免风险较大的决策 + 89 | 10. 需要对任务的优先级做出明确的安排 + 90 | 11. 能够很好的分出工作的轻重缓急 + 91 | 12. 保持秘密是我的最大长处之一 + 92 | 13. 觉得保守秘密很重要 + 93 | 14. 非常注意细节 + 94 | 15. 信任他人 + 95 | 16. 我认为信任他人很重要 + 96 | 17. 希望大家能够一起参加最终的决策 + 97 | 18. 觉得认可自己很重要 + 98 | 19. 鼓励其他人对我的方法提出批评 + 99 | 20. 建立融洽的关系对我来说相当重要 + 100 | 21. 我觉得建立人际网络很重要 + 101 | 22. 需要肯定自己的价值 + 102 | 23. 对自己非常认可 + 103 | 24. 细化分析信息 + 104 | 25. 善于发现相关事实信息 + 105 | 26. 我善于分析信息 + 106 | 27. 利用信息科技是我的一大优势 + 107 | 28. 喜欢运用理论 + 108 | 29. 善于将理论应用于实践 + 109 | 30. 能够立刻意识到某些事的可行性 + 110 | 31. 寻找学习新事物的机会 + 111 | 32. 善于参考他人的观点 + 112 | 33. 我做事不喜欢半途而废 + 113 | 34. 我喜欢实操性的工作 + 114 | 35. 在结识陌生人时，我充满自信 + 115 | 36. 我喜欢处理数字资料 + 116 | 37. 面对他人的反馈意见，我能做出积极反馈 + 117 | 38. 我想要确保细节准确 + 118 | 39. 我觉得完全了解事物的底层原理很有趣 + 119 | 40. 我善于让大家一起参与最终的决策 + 120 | 41. 我喜欢倾听他人 + 121 | 42. 我需要有章可循 + 122 | 43. 我喜欢提出独到的见解 + 123 | 44. 我能很好的应对变化 + 124 | 45. 我的行为符合伦理道德 + 125 | 46. 人们说我有丰富的常识 + 126 | 47. 人们认为我是一个做事周密彻底的人 + 127 | 48. 我喜欢做事情井井有条 + 128 | 49. 我善于洞察问题的实质 + ？ 129 | 50. 学习新事物的机会能给我带来动力 + 130 | 51. 我善于提出探索性的问题 + ？ 131 | 52. 我善于做实操性的任务 + ？ 132 | 53. 我善于相处不同寻常的主意 + ？ 133 | 134 | ### ++ 135 | 136 | 1. 我通过实践来学习 ++ 137 | 2. 善于长远思考 ++ 138 | 3. 善于处理数字资料 ++ 139 | 4. 喜欢快速学习新鲜事物 ++ 140 | 5. 我很享受新事物带来的挑战 ++ 141 | 6. 喜欢同时处理多件事情 ++ 142 | 7. 我觉得待人接物符合伦理道德很重要 ++ 143 | 8. 我善于自我管理 ++ 144 | 9. 我需要有明确的远景规划 ++ 145 | 10. 我喜欢思考未来 ++ 146 | 147 | ### - 148 | 149 | 1. 宽容的人 - 150 | 2. 我很希望给人留下良好的第一印象 - 151 | 3. 我擅长做演讲陈述 - 152 | 4. 喜欢做演讲陈述 - 153 | 5. 善于启发别人 - 154 | 6. 喜欢协调各方人员 - 155 | 7. 人们都说我善于协调各方人员 - 156 | 8. 想让别人倾听我的观点 - 157 | 9. 认为能激励别人很重要 - 158 | 10. 我希望我能给人们带来启发 - 159 | 11. 我善于寻找方法来激励他人 - 160 | 12. 我希望能真正地激励他人 - 161 | 13. 认为取得突出的成绩很重要 - 162 | 14. 希望人们为他们的观点提出论据 - 163 | 15. 我通常是人们关注的焦点 - 164 | 16. 我很少改变主意 - 165 | 17. 善于让生气的人冷静下来- 166 | 18. 认为自己可以从容自若的应对生气的人 - 167 | 19. 我觉得我自己可以从容的应对情绪不安的人 - 168 | 20. 我善于应对情绪不好的人 - 169 | 21. 容易和人争辩 - 170 | 22. 善于推销 - 171 | 23. 我让人民注意到我的成就 - 172 | 24. 我想知道我的方法出了什么差错 - 173 | 25. 我急于采取行动 - 174 | 26. 我想立刻抓住主要问题 - 175 | 27. 我善于快速决策 - 176 | 28. 我总觉得有采取行动的必要 - 177 | 29. 我倾向于快速决策 - 178 | 30. 领导能力是我的一大优势 - 179 | 31. 当我不同意别人的意见时我会告诉他 - 180 | 32. 我觉得理解他人的很瘦很重要 - 181 | 33. 我是一个完美主义者 - 182 | 34. 我能有力的表达自己的观点 - 183 | 35. 我很爱说话 - 184 | 36. 我很健谈 - 185 | 37. 我能够为未来规划一个令人振奋的远景 - 186 | 38. 我会公开表达对他人的反对意见 - 187 | 39. 当我工作出色时会寻求赞赏 - 188 | 40. 我希望人们知道我的成功- ？ 189 | 41. 我觉得为他人着想是相当重要的 - 190 | 42. 愿意为人着想的人 - 191 | 43. 我认为宽容待人很重要 - ？ 192 | 44. 我很有雄心 - 193 | 45. 我对大多数问题持强烈的观点 - 194 | 46. 我是一个有说服力的人 - 195 | 47. 我想要说服他人接受我的观点 - 196 | 48. 我善于促使事情发生 - 197 | 49. 我需要胜利 - 198 | 50. 我喜欢结交新朋友 - 199 | 51. 对于事情是否可行，我相信我的自觉判断 - 200 | 201 | ### -- 202 | 203 | 1. 采取激进的方法解决问题 -- 204 | 2. 喜欢激进的解决方案 -- 205 | 206 | 207 | 208 | ### 最合适的性格 ### 209 | 210 | 我信守自己做出的承诺 211 | 212 | 我觉得人们信守承诺相当重要 213 | 214 | 我坚持按时完成任务 215 | 216 | 我做事广泛听取别人的意见 217 | 218 | 做事前我会征询大家的意见 219 | 220 | 我愿意花时间去帮助别人 221 | 222 | 我做事不喜欢半途而废 223 | 224 | 我必须了解底层原理才能更有效的学习 225 | 226 | 我倾向于根据客观事实做决定 227 | 228 | 我必须找到解决问题的办法 229 | 230 | 我需要了解论点背后的逻辑 231 | 232 | 我喜欢创新 233 | 234 | 做事时我喜欢有新方法、新点子 235 | 236 | 我能想出很多主意 237 | 238 | 我喜欢提出很多主意 239 | 240 | 我喜欢提出独到的见解 241 | 242 | 243 | ### 适中的性格 ### 244 | 245 | 我善于和别人建立融洽关系 246 | 247 | 我喜欢有压力的环境下工作 248 | 249 | 我喜欢忙碌的生活 250 | 251 | 我需要有明确的远景计划 252 | 253 | 我喜欢思考未来 254 | 255 | 我给自己设定了长远目标 256 | 257 | 我经常展望未来 258 | 259 | 我对未来比较乐观 260 | 261 | 我相信未来的事情会是好的 262 | 263 | 我做事很有远见 264 | 265 | 做事的时候我需要有章可循 266 | 267 | 我做事井井有条 268 | 269 | 我非常注重细节 270 | 271 | 我善于同时处理多项任务 272 | 273 | 我循规蹈矩地去做事情 274 | 275 | 我喜欢对数据进行统计与分析 276 | 277 | 我喜欢讨论抽象性的问题 278 | 279 | 我善于处理数字资料 280 | 281 | 我不容易被别人激怒 282 | 283 | 不信任他人 284 | 285 | 我在陌生朋友交流时会轻松自在 286 | 287 | 我大部分时间是快乐的 288 | 289 | 我有强烈的好奇心 290 | 291 | 我喜欢有竞争的工作 292 | 293 | 我是一个竞争心很强的人 294 | 295 | 296 | ### 不适合的性格 ### 297 | 298 | 我掌控着自己的未来 299 | 300 | 我在重大事情发生前会紧张 301 | 302 | 我想有人陪在我身边 303 | 304 | 我坚持自己的做事方式 305 | 306 | 我善于结交朋友 307 | 308 | 我喜欢结识陌生人 309 | 310 | 我善于谈判 311 | 312 | 我善于推销 313 | 314 | 我是个很健谈的人 315 | 316 | 我关注别人的做事目的与做事冬季 317 | 318 | 我善于理解人们背后的的动机 319 | 320 | 我善于体会他人的感受 321 | 322 | 我要超越别人 323 | 324 | 我有较大的野心 325 | 326 | 我想成为团队中的领导 327 | 328 | 我更愿意领导别人 329 | 330 | 我喜欢挑战别人的想法 331 | 332 | 我享受独自工作 333 | -------------------------------------------------------------------------------- /practise/ws-practise.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "html/template" 6 | "io/ioutil" 7 | "net/http" 8 | "os" 9 | "strconv" 10 | "time" 11 | 12 | "github.com/gorilla/websocket" 13 | "github.com/hpcloud/tail" 14 | ) 15 | 16 | const ( 17 | // Time allowed to write the file to the client. 18 | //writeWait = 1 * time.Second 19 | writeWait = 100 * time.Millisecond 20 | 21 | // Time allowed to read the next pong message from the client. 22 | //pongWait = 24 * time.Hour 23 | pongWait = 60 * time.Second 24 | 25 | // Send pings to client with this period. Must be less than pongWait. 26 | pingPeriod = (pongWait * 9) / 10 27 | 28 | // Poll file for changes with this period. 29 | filePeriod = 1 * time.Second 30 | ) 31 | 32 | var ( 33 | homeTempl = template.Must(template.New("").Parse(homeHTML)) 34 | filename = "/Users/caoyuan/test.log" 35 | addr = ":8090" 36 | start bool 37 | upgrader = websocket.Upgrader{ 38 | ReadBufferSize: 1024, 39 | WriteBufferSize: 1024, 40 | } 41 | ) 42 | 43 | func readFileIfModified(lastMod time.Time) ([]byte, time.Time, error) { 44 | fi, err := os.Stat(filename) 45 | if err != nil { 46 | return nil, lastMod, err 47 | } 48 | if !fi.ModTime().After(lastMod) { 49 | return nil, lastMod, nil 50 | } 51 | p, err := ioutil.ReadFile(filename) 52 | if err != nil { 53 | return nil, fi.ModTime(), err 54 | } 55 | return p, fi.ModTime(), nil 56 | } 57 | 58 | func reader(ws *websocket.Conn) { 59 | defer ws.Close() 60 | ws.SetReadLimit(512) 61 | ws.SetReadDeadline(time.Now().Add(pongWait)) 62 | ws.SetPongHandler(func(string) error { ws.SetReadDeadline(time.Now().Add(pongWait)); return nil }) 63 | for { 64 | _, _, err := ws.ReadMessage() 65 | if err != nil { 66 | break 67 | } 68 | } 69 | } 70 | func tailFile() *tail.Tail { 71 | tailfs, err := tail.TailFile(filename, tail.Config{ 72 | ReOpen: true, // 文件被移除或被打包，需要重新打开 73 | Follow: true, // 实时跟踪 74 | Location: &tail.SeekInfo{Offset: 0, Whence: 2}, // 如果程序出现异常，保存上次读取的位置，避免重新读取。 75 | MustExist: false, // 如果文件不存在，是否推出程序，false是不退出 76 | Poll: true, 77 | }) 78 | 79 | if err != nil { 80 | fmt.Println("tailf failed, err:", err) 81 | return nil 82 | } 83 | return tailfs 84 | } 85 | func writer(ws *websocket.Conn, lastMod time.Time) { 86 | tailfs := tailFile() 87 | pingTicker := time.NewTicker(pingPeriod) 88 | fileTicker := time.NewTicker(filePeriod) 89 | defer func() { 90 | pingTicker.Stop() 91 | fileTicker.Stop() 92 | ws.Close() 93 | }() 94 | 95 | for { 96 | select { 97 | case msg, ok := <-tailfs.Lines: 98 | if ok { 99 | ws.SetWriteDeadline(time.Now().Add(writeWait)) 100 | fmt.Printf("read file content： %s\n", msg) 101 | if err := ws.WriteMessage(websocket.TextMessage, []byte(msg.Text)); err != nil { 102 | return 103 | } 104 | } 105 | case <-pingTicker.C: 106 | ws.SetWriteDeadline(time.Now().Add(writeWait)) 107 | if err := ws.WriteMessage(websocket.PingMessage, []byte{}); err != nil { 108 | return 109 | } 110 | } 111 | } 112 | } 113 | 114 | func serveWs(w http.ResponseWriter, r *http.Request) { 115 | ws, err := upgrader.Upgrade(w, r, nil) 116 | if err != nil { 117 | if _, ok := err.(websocket.HandshakeError); !ok { 118 | fmt.Println(err) 119 | } 120 | return 121 | } 122 | 123 | var lastMod time.Time 124 | if n, err := strconv.ParseInt(r.FormValue("lastMod"), 16, 64); err == nil { 125 | lastMod = time.Unix(0, n) 126 | } 127 | 128 | go writer(ws, lastMod) 129 | reader(ws) 130 | } 131 | 132 | func serveHome(w http.ResponseWriter, r *http.Request) { 133 | if r.URL.Path != "/" { 134 | http.Error(w, "Not found", http.StatusNotFound) 135 | return 136 | } 137 | if r.Method != "GET" { 138 | http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) 139 | return 140 | } 141 | w.Header().Set("Content-Type", "text/html; charset=utf-8") 142 | p, lastMod, err := readFileIfModified(time.Time{}) 143 | if err != nil { 144 | p = []byte(err.Error()) 145 | lastMod = time.Unix(0, 0) 146 | } 147 | var v = struct { 148 | Host string 149 | Data string 150 | LastMod string 151 | }{ 152 | r.Host, 153 | string(p), 154 | strconv.FormatInt(lastMod.UnixNano(), 16), 155 | } 156 | homeTempl.Execute(w, &v) 157 | } 158 | 159 | func main() { 160 | http.HandleFunc("/", serveHome) 161 | http.HandleFunc("/ws", serveWs) 162 | 163 | if err := http.ListenAndServe(addr, nil); err != nil { 164 | panic(err) 165 | } 166 | } 167 | 168 | const homeHTML = ` 169 | 170 | 171 | 172 | WebSocket Example 173 | 194 | 195 | 196 |
197 | 198 | 实时日志 199 | 200 | 201 |
202 | 203 |
204 |
    205 | 
    
206 | 		

207 | 	

208 |         
230 |     
231 | 
232 | `
233 | 


--------------------------------------------------------------------------------
/doc/kubernetes/csi.md:
--------------------------------------------------------------------------------
  1 | # CSI Plugin 注册机制源码分析
  2 | 
  3 | ### node-driver-registrar 流程解析
  4 | 
  5 | 1. rpc 调用 nfs-csi-plugin 中的 identity server 中的 GetPluginInfo 方法，返回我们自研plugin相关的基本信息， 获取 csi driver name
  6 | 
  7 | 2. 启动 grpc server，并监听在宿主机的 /var/lib/kubelet/plugins_registry/${csiDriverName}-reg.sock （对应容器的/registration/${driver-name}-reg.sock）
  8 | 
  9 | 3. 该 grpc server 提供 GetInfo 和 NotifyRegistrationStatus 方法供 kubelet plugin manager 调用，
 10 | 
 11 | ### kubelet plugin manager 流程解析
 12 | 
 13 | 1. 当 ds 部署的 node-driver-registrar sidecar 启动时，/var/lib/kubelet/plugins_registry 会新增一个 socket file nfs.csi.k8s.io-reg.sock
 14 | 
 15 | 2. 这个 nfs.csi.k8s.io-reg.sock 会被 plugin watcher 监听并写入desiredStateOfWorld缓存中（通过github.com/fsnotify/fsnotify实现）
 16 | 
 17 | 3. reconciler 会对比缓存，找出需要新增或者删除的 csi plugin。当新增时，reconciler 会通过 grpc client（/var/lib/kubelet/plugins_registry）调用 node-driver-registrar sidecar container中rpc server提供的的GetInfo，然后根据返回字段的type，找到对应的 plugin hander，
 18 | 
 19 | ```go
 20 | // socketPath 为 /registration/${driver-name}-reg.sock
 21 | client, conn, err := dial(socketPath, dialTimeoutDuration)
 22 |         // 调用node-driver-registrar sidecar container中rpc server提供的的GetInfo
 23 |         infoResp, err := client.GetInfo(ctx, ®isterapi.InfoRequest{})
 24 |         // 这里handler就是上文说的CSIPlugin type的csi.RegistrationHandler{}对象
 25 |         handler, ok := pluginHandlers[infoResp.Type]
 26 |         // 调用handler.ValidatePlugin
 27 |         if err := handler.ValidatePlugin(infoResp.Name, infoResp.Endpoint, infoResp.SupportedVersions); err != nil {
 28 |         }
 29 |         ...
 30 |         // 加入actualStateOfWorldUpdater缓存
 31 |         err = actualStateOfWorldUpdater.AddPlugin(cache.PluginInfo{
 32 |             SocketPath: socketPath,
 33 |             Timestamp:  timestamp,
 34 |             Handler:    handler,
 35 |             Name:       infoResp.Name,
 36 |         })
 37 | // infoResp.Endpoint 就是 nfs-csi 的 socket path
 38 | if err := handler.RegisterPlugin(infoResp.Name, infoResp.Endpoint, infoResp.SupportedVersions); err != nil {
 39 |             return og.notifyPlugin(client, false, fmt.Sprintf("RegisterPlugin error -- plugin registration failed with err: %v", err))
 40 |         }
 41 |     ...
 42 |     return registerPluginFunc
 43 | ```
 44 | 
 45 | 4. plugin handler 会根据传入的 csi-plugin 监听的 socket path，直接和我们 nfs-csi-plugin 通信，并调用该对象的 ValidatePlugin 和 RegisterPlugin 来注册插件，这里的注册插件其实就是设置 node annotation 和 创建/更新CSINode对象。
 46 | 
 47 | ```bash
 48 | metadata:
 49 |  annotations:
 50 |    csi.volume.kubernetes.io/nodeid: '{"nfs.csi.k8s.io":"kube-master"}'
 51 | 
 52 | # kubectl get csinode
 53 | NAME           DRIVERS   AGE
 54 | kubez-master   1         101d
 55 | kubez-node1    1         99d
 56 | kubez-node2    1         101d
 57 | ```
 58 | 
 59 | 5. kubelet plugin-magner 通过 rpc 调用 NotifyRegistrationStatus 告知 node-driver-registrar 注册结果。
 60 | 
 61 | ### external-provisioner 流程解析
 62 | 
 63 | 1. external-provisioner 是运行在 k8s 集群内部的 controller，它作为 sidecar container 和 csi-nfs 运行在同一个 pod 中.
 64 | 
 65 | ```bash
 66 | containers:
 67 | - args:
 68 |  - -v=2
 69 |  - --csi-address=$(ADDRESS)
 70 |  - --leader-election
 71 |  env:
 72 |  - name: ADDRESS
 73 |    value: /csi/csi.sock
 74 |  image: jacky06/csi-provisioner:v2.1.0
 75 |  name: csi-provisioner
 76 |  volumeMounts:
 77 |  - mountPath: /csi
 78 |    name: socket-dir
 79 | - args:
 80 |  - -v=5
 81 |  - --nodeid=$(NODE_ID)
 82 |  - --endpoint=$(CSI_ENDPOINT)
 83 |  env:
 84 |  - name: NODE_ID
 85 |    valueFrom:
 86 |      fieldRef:
 87 |        apiVersion: v1
 88 |        fieldPath: spec.nodeName
 89 |  - name: CSI_ENDPOINT
 90 |    value: unix:///csi/csi.sock
 91 |  image: mcr.microsoft.com/k8s/csi/nfs-csi:latest
 92 | ```
 93 | 
 94 | 2. external-provisioner 通过 informer 机制去监听 pvc/pv，当新建 pv 或者 pvc 的时候，external-provisioner 会通过 grpc (/csi/csi.sock) 调用 nfs 的 CreateVolume(DeleteVolume)方法来实际创建一个外部存储 volume
 95 | 
 96 | ```go
 97 | // NewCSIProvisioner creates new CSI provisioner.
 98 | //
 99 | // vaLister is optional and only needed when VolumeAttachments are
100 | // meant to be checked before deleting a volume.
101 | func NewCSIProvisioner(client kubernetes.Interface,
102 | 	connectionTimeout time.Duration,
103 | 	...
104 | ) controller.Provisioner {
105 |     ...
106 | 	csiClient := csi.NewControllerClient(grpcClient)
107 | 
108 | 	provisioner := &csiProvisioner{
109 | 		client:                                client,
110 | 		grpcClient:                            grpcClient,
111 | 		csiClient:                             csiClient,
112 | 		snapshotClient:                        snapshotClient,
113 | 		...
114 | 		driverName:                            driverName,
115 | 		pluginCapabilities:                    pluginCapabilities,
116 | 		controllerCapabilities:                controllerCapabilities,
117 |         ...
118 | 		scLister:                              scLister,
119 | 		csiNodeLister:                         csiNodeLister,
120 | 		nodeLister:                            nodeLister,
121 | 		claimLister:                           claimLister,
122 | 		vaLister:                              vaLister,
123 | 		eventRecorder:                         eventRecorder,
124 | 	}
125 |     ...
126 | 	return provisioner
127 | }
128 | 
129 | // 创建
130 | func (p *csiProvisioner) Provision(ctx context.Context, options controller.ProvisionOptions) (*v1.PersistentVolume, controller.ProvisioningState, error) {
131 |     ...
132 | 	createCtx := markAsMigrated(ctx, result.migratedVolume)
133 | 	createCtx, cancel := context.WithTimeout(createCtx, p.timeout)
134 | 	defer cancel()
135 | 	rep, err := p.csiClient.CreateVolume(createCtx, req)
136 |     ...
137 | }
138 | 
139 | // 删除
140 | func (p *csiProvisioner) Delete(ctx context.Context, volume *v1.PersistentVolume) error {
141 |     ...
142 | 	_, err = p.csiClient.DeleteVolume(deleteCtx, &req)
143 |     ...
144 | ```
145 | 
146 | ### kubelet 通过调用 csiClient 去完成 pvc 的挂载
147 | 
148 | ```go
149 | type csiClient interface {
150 |         NodeGetInfo(ctx context.Context) (
151 |                 nodeID string,
152 |                 maxVolumePerNode int64,
153 |                 accessibleTopology map[string]string,
154 |                 err error)
155 |         NodePublishVolume(
156 |                 ctx context.Context,
157 |                 volumeid string,
158 |                 readOnly bool,
159 |                 stagingTargetPath string,
160 |                 targetPath string,
161 |                 accessMode api.PersistentVolumeAccessMode,
162 |                 publishContext map[string]string,
163 |                 volumeContext map[string]string,
164 |                 secrets map[string]string,
165 |                 fsType string,
166 |                 mountOptions []string,
167 |         ) error
168 |         NodeExpandVolume(ctx context.Context, volumeid, volumePath string, newSize resource.Quantity) (resource.Quantity, error)
169 |         NodeUnpublishVolume(
170 |                 ctx context.Context,
171 |                 volID string,
172 |                 targetPath string,
173 |         ) error
174 |         NodeStageVolume(ctx context.Context,
175 |                 volID string,
176 |                 publishVolumeInfo map[string]string,
177 |                 stagingTargetPath string,
178 |                 fsType string,
179 |                 accessMode api.PersistentVolumeAccessMode,
180 |                 secrets map[string]string,
181 |                 volumeContext map[string]string,
182 |                 mountOptions []string,
183 |         ) error
184 | ...
185 | ```
186 | 


--------------------------------------------------------------------------------
/practise/cobra-practise/pixiuctl/plugin/plugin.go:
--------------------------------------------------------------------------------
  1 | package plugin
  2 | 
  3 | import (
  4 | 	"bytes"
  5 | 	"fmt"
  6 | 	"io/ioutil"
  7 | 	"os"
  8 | 	"path/filepath"
  9 | 	"runtime"
 10 | 	"strings"
 11 | 
 12 | 	"github.com/spf13/cobra"
 13 | 	"k8s.io/cli-runtime/pkg/genericclioptions"
 14 | 	cmdutil "k8s.io/kubectl/pkg/cmd/util"
 15 | 	"k8s.io/kubectl/pkg/util/i18n"
 16 | 	"k8s.io/kubectl/pkg/util/templates"
 17 | )
 18 | 
 19 | var (
 20 | 	pluginLong = templates.LongDesc(i18n.T(`
 21 | 		Provides utilities for interacting with plugins.
 22 | 
 23 | 		Plugins provide extended functionality that is not part of the major command-line distribution.
 24 | 		Please refer to the documentation and examples for more information about how write your own plugins.`))
 25 | 
 26 | 	pluginListLong = templates.LongDesc(i18n.T(`
 27 | 		List all available plugin files on a user's PATH.
 28 | 
 29 | 		Available plugin files are those that are:
 30 | 		- executable
 31 | 		- anywhere on the user's PATH
 32 | 		- begin with "pixiuctl-"
 33 | 	`))
 34 | 
 35 | 	ValidPluginFilenamePrefixes = []string{"pixiuctl"}
 36 | )
 37 | 
 38 | func NewCmdPlugin(streams genericclioptions.IOStreams) *cobra.Command {
 39 | 	cmd := &cobra.Command{
 40 | 		Use:                   "plugin [flags]",
 41 | 		DisableFlagsInUseLine: true,
 42 | 		Short:                 i18n.T("Provides utilities for interacting with plugins"),
 43 | 		Long:                  pluginLong,
 44 | 		Run: func(cmd *cobra.Command, args []string) {
 45 | 			cmdutil.DefaultSubCommandRun(streams.ErrOut)(cmd, args)
 46 | 		},
 47 | 	}
 48 | 
 49 | 	cmd.AddCommand(NewCmdPluginList(streams))
 50 | 	return cmd
 51 | }
 52 | 
 53 | type PluginListOptions struct {
 54 | 	Verifier PathVerifier
 55 | 	NameOnly bool
 56 | 
 57 | 	PluginPaths []string
 58 | 
 59 | 	genericclioptions.IOStreams
 60 | }
 61 | 
 62 | type PathVerifier interface {
 63 | 	// Verify determines if a given path is valid
 64 | 	Verify(path string) []error
 65 | }
 66 | 
 67 | // NewCmdPluginList provides a way to list all plugin executables visible to pixiuctl
 68 | func NewCmdPluginList(streams genericclioptions.IOStreams) *cobra.Command {
 69 | 	o := &PluginListOptions{
 70 | 		IOStreams: streams,
 71 | 	}
 72 | 
 73 | 	cmd := &cobra.Command{
 74 | 		Use:   "list",
 75 | 		Short: i18n.T("List all visible plugin executables on a user's PATH"),
 76 | 		Long:  pluginListLong,
 77 | 		Run: func(cmd *cobra.Command, args []string) {
 78 | 			cmdutil.CheckErr(o.Complete(cmd))
 79 | 			cmdutil.CheckErr(o.Run())
 80 | 		},
 81 | 	}
 82 | 
 83 | 	cmd.Flags().BoolVar(&o.NameOnly, "name-only", o.NameOnly, "If true, display only the binary name of each plugin, rather than its full path")
 84 | 	return cmd
 85 | }
 86 | 
 87 | func (o *PluginListOptions) Complete(cmd *cobra.Command) error {
 88 | 	o.Verifier = &CommandOverrideVerifier{
 89 | 		root:        cmd.Root(),
 90 | 		seenPlugins: make(map[string]string),
 91 | 	}
 92 | 
 93 | 	o.PluginPaths = filepath.SplitList(os.Getenv("PATH"))
 94 | 	return nil
 95 | }
 96 | 
 97 | func (o *PluginListOptions) Run() error {
 98 | 	pluginsFound := false
 99 | 	isFirstFile := true
100 | 	pluginErrors := []error{}
101 | 	pluginWarnings := 0
102 | 
103 | 	for _, dir := range uniquePathsList(o.PluginPaths) {
104 | 		if len(strings.TrimSpace(dir)) == 0 {
105 | 			continue
106 | 		}
107 | 		files, err := ioutil.ReadDir(dir)
108 | 		if err != nil {
109 | 			if _, ok := err.(*os.PathError); ok {
110 | 				fmt.Fprintf(o.ErrOut, "Unable to read directory %q from your PATH: %v. Skipping...\n", dir, err)
111 | 				continue
112 | 			}
113 | 
114 | 			pluginErrors = append(pluginErrors, fmt.Errorf("error: unable to read directory %q in your PATH: %v", dir, err))
115 | 			continue
116 | 		}
117 | 
118 | 		for _, f := range files {
119 | 			if f.IsDir() {
120 | 				continue
121 | 			}
122 | 			if !hasValidPrefix(f.Name(), ValidPluginFilenamePrefixes) {
123 | 				continue
124 | 			}
125 | 
126 | 			if isFirstFile {
127 | 				fmt.Fprintf(o.Out, "The following compatible plugins are available:\n\n")
128 | 				pluginsFound = true
129 | 				isFirstFile = false
130 | 			}
131 | 
132 | 			pluginPath := f.Name()
133 | 			if !o.NameOnly {
134 | 				pluginPath = filepath.Join(dir, pluginPath)
135 | 			}
136 | 
137 | 			fmt.Fprintf(o.Out, "%s\n", pluginPath)
138 | 			if errs := o.Verifier.Verify(filepath.Join(dir, f.Name())); len(errs) != 0 {
139 | 				for _, err := range errs {
140 | 					fmt.Fprintf(o.ErrOut, "  - %s\n", err)
141 | 					pluginWarnings++
142 | 				}
143 | 			}
144 | 		}
145 | 	}
146 | 
147 | 	if !pluginsFound {
148 | 		pluginErrors = append(pluginErrors, fmt.Errorf("error: unable to find any pixiuctl plugins in your PATH"))
149 | 	}
150 | 
151 | 	if pluginWarnings > 0 {
152 | 		if pluginWarnings == 1 {
153 | 			pluginErrors = append(pluginErrors, fmt.Errorf("error: one plugin warning was found"))
154 | 		} else {
155 | 			pluginErrors = append(pluginErrors, fmt.Errorf("error: %v plugin warnings were found", pluginWarnings))
156 | 		}
157 | 	}
158 | 	if len(pluginErrors) > 0 {
159 | 		errs := bytes.NewBuffer(nil)
160 | 		for _, e := range pluginErrors {
161 | 			fmt.Fprintln(errs, e)
162 | 		}
163 | 		return fmt.Errorf("%s", errs.String())
164 | 	}
165 | 
166 | 	return nil
167 | }
168 | 
169 | type CommandOverrideVerifier struct {
170 | 	root        *cobra.Command
171 | 	seenPlugins map[string]string
172 | }
173 | 
174 | func (v *CommandOverrideVerifier) Verify(path string) []error {
175 | 	if v.root == nil {
176 | 		return []error{fmt.Errorf("unable to verify path with nil root")}
177 | 	}
178 | 
179 | 	errors := []error{}
180 | 	if isExec, err := isExecutable(path); err == nil && !isExec {
181 | 		errors = append(errors, fmt.Errorf("warning: %s identified as a pixiuctl plugin, but it is not executable", path))
182 | 	} else if err != nil {
183 | 		errors = append(errors, fmt.Errorf("error: unable to identify %s as an executable file: %v", path, err))
184 | 	}
185 | 
186 | 	// extract the plugin binary name
187 | 	segs := strings.Split(path, "/")
188 | 	binName := segs[len(segs)-1]
189 | 
190 | 	cmdPath := strings.Split(binName, "-")
191 | 	if len(cmdPath) > 1 {
192 | 		// the first argument is always "pixiuctl" for a plugin binary
193 | 		cmdPath = cmdPath[1:]
194 | 	}
195 | 	if cmd, _, err := v.root.Find(cmdPath); err == nil {
196 | 		errors = append(errors, fmt.Errorf("warning: %s overwrites existing command: %q", binName, cmd.CommandPath()))
197 | 	}
198 | 
199 | 	return errors
200 | }
201 | 
202 | // 去重
203 | func uniquePathsList(paths []string) []string {
204 | 	seen := map[string]bool{}
205 | 	newPaths := []string{}
206 | 	for _, p := range paths {
207 | 		if seen[p] {
208 | 			continue
209 | 		}
210 | 		seen[p] = true
211 | 		newPaths = append(newPaths, p)
212 | 	}
213 | 	return newPaths
214 | }
215 | 
216 | func hasValidPrefix(filepath string, validPrefixes []string) bool {
217 | 	for _, prefix := range validPrefixes {
218 | 		if !strings.HasPrefix(filepath, prefix+"-") {
219 | 			continue
220 | 		}
221 | 		return true
222 | 	}
223 | 	return false
224 | }
225 | 
226 | func isExecutable(fullPath string) (bool, error) {
227 | 	info, err := os.Stat(fullPath)
228 | 	if err != nil {
229 | 		return false, err
230 | 	}
231 | 
232 | 	if runtime.GOOS == "windows" {
233 | 		fileExt := strings.ToLower(filepath.Ext(fullPath))
234 | 
235 | 		switch fileExt {
236 | 		case ".bat", ".cmd", ".com", ".exe", ".ps1":
237 | 			return true, nil
238 | 		}
239 | 		return false, nil
240 | 	}
241 | 
242 | 	if m := info.Mode(); !m.IsDir() && m&0111 != 0 {
243 | 		return true, nil
244 | 	}
245 | 
246 | 	return false, nil
247 | }
248 | 


--------------------------------------------------------------------------------
/doc/kubernetes/network.md:
--------------------------------------------------------------------------------
  1 | # Service 实现原理分析
  2 | 
  3 | - 阅读本文档时，可能因为代理问题，无法正常查看截图
  4 | 
  5 | ## 环境信息
  6 | - kubernetes: v1.18.2
  7 | - cri: docker
  8 | - cni: flannel
  9 | - proxyMode: iptables
 10 | 
 11 | ## iptables
 12 | 数据包在linux系统中，是绕不开 iptables 的，因为篇幅问题，本文不对 **iptables** 展开讨论，有兴趣的同学请移步 [Iptables](https://en.wikipedia.org/wiki/Iptables)
 13 | 
 14 | ### iptables 的四表五链
 15 | - **tables**: raw, mangle, filter, nat
 16 | - **chains**: PREROUTING, FORWARD, INPUT, OUTPUT, POSTROUTING
 17 | 
 18 |     ![iptables](./pictures/iptablesflow.png)
 19 | 
 20 | ### kubernetes 支持的 serviceType
 21 |   - `ClusterIP`
 22 |   - `NodePort`
 23 |   - `LoadBalancer`
 24 | 
 25 | **ClusterIP**
 26 | 
 27 | 1. 在 k8s 集群创建一个 `ClusterIP` 类型的 `service`, 其 `ip` 为 `10.254.122.75`, `port` 为 `80`
 28 | 
 29 | 2. 在集群内的任一节点执行 `curl 10.254.122.75:80` ，可以正常访问服务，通过 `kubectl get ep` 获取对应的后端有若干个 `pod`
 30 | ![endpoint](./pictures/endpoints.png)
 31 | 
 32 | ***实现流程如下***
 33 | 
 34 | 3. 数据包首先进入 iptable 的 `PREROUTING` 链，依次进入 `raw` 表、 `mangle` 表、 和 `nat` 表，根据上文总结，k8s 只操作 `nat` 表，直接查看  `PREROUTING` 链的 `nat` 表：
 35 | ![PREROUTING](./pictures/clusterprerouting.png)
 36 | 
 37 | 4. 数据包命中，随后数据包进入 `KUBE-SERVICES` 链，先后命中 `KUBE-MARK-MASQ` 和 `KUBE-SVC-4N57TFCL4MD7ZTDA` 链
 38 | ![KUBE-SERVICES](./pictures/kube-services.png)
 39 | 
 40 | 4. 数据包随后进入 `KUBE-MARK-MASQ` 链，k8s 为该报文打上 `0x4000` 的标记 (具体用途后续介绍)
 41 | ![KUBE-MARK-MASQ](./pictures/KUBE-MARK-MASQ.png)
 42 | 
 43 | 6. 然后报文进入 `KUBE-SVC-4N57TFCL4MD7ZTDA` 链，并在其规则中完成 `loadbalance`, 本例中，报文随机命中 `KUBE-SEP-EZ6FKUNEIONYFN4Z` 链
 44 | ![KUBE-SVC-4N57TFCL4MD7ZTDA](./pictures/KUBE-SVC-4N57TFCL4MD7ZTDA.png)
 45 | 
 46 | 7. 随后报文进入 `KUBE-SEP-EZ6FKUNEIONYFN4Z` 链，并在该链中完成 `dnat`，实现从 `service ip` 到实际 endpoint `pod id` 的转换，对应 `kubectl get ep` 获取到的后端
 47 | ![KUBE-SEP-EZ6FKUNEIONYFN4Z](./pictures/KUBE-SEP-EZ6FKUNEIONYFN4Z.png)
 48 | 
 49 | 8. 完成 `dnat` 之后，报文完成所有 `PREROUTING` 链，进入 `routering decision` 阶段，
 50 | 数据报文的 `源ip` 为本地ip，目的ip 为 dnat 之后的 ip ，查看本地路由, 此时有两种情况，会在步骤 **9** 和 **10** 中分别讨论
 51 | ![router](./pictures/router.png)
 52 |     - 命中本节点，报文进入 cni0 设备
 53 |     - 命中其他节点，报文进入 flannel.1 设备
 54 | 
 55 | 9. 路由命中本节点场景分析 (endpoint 为 172.30.0.35:80)
 56 |     - 命中本地的数据包， 依次进入 `INPUT` 链的 `mangle` 和 `filter` 表，未命中任何规则，直接通过
 57 |     - 数据包到达 `cni0` 设备， `cni0` 设备是一个 `linux bridge`，作为宿主机上所有 `pod` 的网关 通过 `arp寻址` 数据包顺利到达 `pod`
 58 |     - `pod` 完成对数据包的处理
 59 |     - 数据包在处理完之后，进入 `routering decision` 阶段，路由后，依次进入 `OUTPUT` 链的 `raw`, `mangle`, `nat`, `filter` 表，未命中任何规则，直接通过
 60 |     - 数据包通过 `OUTPUT` 链后, 进行 `routering decision` ，随后数据包进入 `POSTROUTING` 链的 `mangle` 和 `nat` 表, `mangle` 表直接通过，数据包进入 `nat` 表会命中 `KUBE-POSTROUTING` 链，根据 `KUBE-MARK-MASQ` 的 `MARK` 完成 `MASQUERADE` . 至此，整个流程结束.
 61 |     ![KUBE-POSTROUTING](./pictures/kube-postrouting.png)
 62 | 
 63 | 10.  路由命中其他节点场景分析 (endpoint 为 172.30.4.43:80)
 64 |     - 命中其他节点的数据包，将依次进入 `FORWARD` 链的 `mangle` 和 `filter` 表，命中 `filter` 表的 `KUBE-FORWARD` 链，完成 `FORWARD`.
 65 |     ![kube-forward](./pictures/kube-forward.png)
 66 |     - 数据包进入 `routering decision` 阶段的, 其 `目的ip` 为 `172.30.1.43`, 根据路由，出口为 `flannel.1` 下一条为 `172.30.4.0`
 67 |     ![post-router](./pictures/postrouting.png)
 68 |     - 随后数据包进入 `POSTROUTING` 链的 `mangle` 和 `nat` 表，命中 `nat` 表的 `KUBE-POSTROUTING`,  完成 `MASQUERADE` 后，数据包离开本机，到达目的主机
 69 |     ![masq](./pictures/masq.png)
 70 | 
 71 | - 无头服务(headless)在服务&外传区讨论，此处不做赘述
 72 | 
 73 | **NodePort**
 74 | 
 75 | 1. 创建一个 `NodePort` 类型的 `service`，访问集群的任一台节点的 ip + port（本例为31980）均可以正常连接
 76 | ![nodeport](./pictures/nodeport.png)
 77 | 
 78 | 2. 查看被访问的节点（或者是任意节点）的 `iptables` 规则，数据包先后进入 `PREROUTING` 链的 `raw` 表，`mangle` 表， 和 `nat` 表（前文已介绍，raw 和 mangle 表均没有规则，默认通过），直接查看 `nat` 表规则
 79 | 
 80 | 3. 和 `ClusterIP` 场景相关，数据包会命中 `PREROUTING` 的 `KUBE-SERVICES` 链，然后命中其子链 `KUBE-NODEPORTS`
 81 | ![kube-nodeport](./pictures/kube-nodeport.png)
 82 | 
 83 | 4. 数据包进入 `KUBE-NODEPORTS` 后，根据 `dpt` 先后命中 `KUBE-MARK-MASQ`(0x4000) 和 `KUBE-SVC-37ROJ3MK6RKFMQ2B` 并在 `KUBE-SVC-37ROJ3MK6RKFMQ2B` 链中完成负载均衡 (statistic mode random)
 84 | ![lb](./pictures/lb.png)
 85 | 
 86 | 5. 随后数据包进入负载均衡之后的链中，本例是 `KUBE-SEP-4XK7BREWKZE733EB` 链，查看 `KUBE-SEP-4XK7BREWKZE733EB` 规则，发现数据包命中 `dnat` 规则，在此完成 `nodeport:port` 到 `pod:port` 的转换，数据包会根据 `dnat` 后的 `目的ip`，匹配路由，到达后端 `pod`, 此时分两种情况，在 **6** 和 **7** 中分别讨论
 87 | ![nodednat](./pictures/nodednat.png)
 88 | 
 89 | 6. `pod` 不在本节点，以 `172.30.1.53:80` 为例，此时数据包已通过 `PREROUTING`链, 根据 `路由`，将从 `flannel.1` 设备到达下一跳 `172.30.1.0`, 然后数据包进入 `FORWARD` 链
 90 | 
 91 |     ![node-route3](./pictures/node-route3.png)
 92 |     - 数据包依次通过 `FORWARD` 的 `mangle` 和 `filter` 表，命中 `filter` 的 `KUBE-FORWARD` 链的两条规则 (0x4000/0x4000)， 直接通过 `FORWARD` 链
 93 |      ![filter](./pictures/node-port-filter.png)
 94 |     - 数据包在通过 `FORWARD` 之后，未对数据包做修改，直接进入 `POSTROUTING` 链
 95 |     - 数据包进入 `POSTROUTING` 链的 `mangle` 和 `nat` 表, 命中 `nat` 表的 `KUBE-POSTROUTING` 链，完成 `MASQUERADE`, 进行 `snat`
 96 |         ![node-masquerage](./pictures/nodeport-masqerade.png)
 97 |     - 然后数据包命中 `POSTROUTING` 的 `RETURN` 规则，数据包完成全部规则，即将离开本节点，去往下一跳, 此时数据包的源ip为本节点 `flannel.1` 的地址（伪装）, 目的地址为后端 `pod` 地址
 98 |         ![return](./pictures/return.png)
 99 |     - 然后数据包被送到目的 `pod` 所在的节点，进入其 `PREROUTING` 链, 依次通过 `raw`, `mangle`, `nat` 表, 未命中任何规则，然后数据包进行路由，根据路由表，数据包会被送到 `cni0` 设备，
100 |        ![cni0](./pictures/cni0.png)
101 |     - `cni0` 是本地设备，意味着数据包会进入 `INPUT` 链，依次通过 `mangle` 和 `filter` 表, 未命中规则，然后进行 `Local Process` （处理过程忽略）
102 |     - `Local Process` 之后，数据包进入 `OUTPUT` 链， 依次通过 `raw`, `mangle`, `nat` 表，未命中规则
103 |     - 然后数据包进入 `route decsion`，目前数据包的 `源ip` 是 pod 的 ip ，`目的ip` 是之前 node 节点的 `flannelf.1` 的 ip, 根据路由，数据包通过 `flannelf.1` 设备前往对端的 `flannelf.1` 网关是 `172.30.0.0`
104 |        ![node-port-back](./pictures/node-port-back.png)
105 |     - 然后数据包进入 `POSTROUTING` 链, 依次通过 `mangle` 和 `nat` 表， 命中 `nat` 表的 `RETURN`, 完成 `目的节点` 的所有操作, 根据路由，数据包被送会 `源节点`.
106 |        ![node-port-back2](./pictures/nodeport-back2.png)
107 |     - 数据包达到 `源节点` (`源ip` 为 `pod ip`，`目的ip` 为 `172.30.0.0/32`), 依次通过 `PREROUTING` 的 `raw`, `mangle` 和 `nat` 表，未命中规则，进入 `route decsion` 阶段，命中默认路由
108 |        ![default-route](./pictures/defualt-router.png)
109 |     - 随后数据包进入 `FORWARD` 链的 `mangle` 和 `filter` 表，命中 `KUBE-FORWARD` 的规则，快速通过 `FORWARD` 链
110 |        ![kube-forward2](./pictures/kube-forward2.png)
111 |     - 随后数据包进入 `POSTROUTING` 的 `mangle` 和 `nat` 表，命中 `nat` 表的 `KUBE-POSTROUTING` 链, 完成 `snat` (MASQUERADE)
112 |        ![kube-post2](./pictures/kube-post2.png)
113 |     - 完成 `POSTROUTING` 链之后，数据包正式离开 `node` 节点，返回给外端的请求端. 至此，整个流程结束.
114 | 
115 | 7. `pod` 在本节点, 以 `172.30.3.2:80` 为例, 此时数据包已通过 `PREROUTING` 链
116 | 
117 |     - 根据路由，命中本地，数据包将被送到容器网关 `cni0` 设备
118 |   ![nodeportcni](./pictures/nodeportcni.png)
119 |     - 数据包依次进入 `INPUT` 链的 `mangle` 和 `filter` 表，未命中任何规则，直接通过
120 |     - 数据包在处理完之后，进入 routering decision 阶段，路由后，依次进入 OUTPUT 链的 raw, mangle, nat, filter 表，未命中任何规则，直接通过
121 |     - 数据包通过 `OUTPUT` 链后, 进行 `routering decision` ，随后数据包进入 `POSTROUTING` 链的 `mangle` 和 `nat` 表, `mangle` 表直接通过，数据包进入 `nat` 表会命中 `KUBE-POSTROUTING` 链，根据 `KUBE-MARK-MASQ` 的 MARK 完成 `MASQUERADE`. 然后 `数据包` 根据 `conntrack` 离开宿主机. 至此，整个流程结束.
122 | 
123 | **LoadBalancer**
124 | 
125 |    - 参考 **NodePort** 章节，**LoadBalancer** 和 **NodePort** 集群内实现原理大致相同，又因云厂商 `provider` 有所差异
126 | 
127 | ### 服务&外传区
128 | 
129 | **ClusterIP(headless)**
130 | 
131 | 1. 什么是无头服务？[Headless Service](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services)
132 | 
133 | 2. 创建一个无头服务 —— 指定服务类型为 `ClusterIP`, 且设置 `Spec.ClusterIP` 为 `None`， 对于无头服务，`kubernetes` 不会为其分配 `ClusterIP`, `kube-proxy` 也不会处理(无规则下发)，由 `dns` 根据 `selector` 自动配置转发：
134 | 
135 |     - 如果配置 `selector`, `DNS` 直接指向后端 `pods`
136 |     - 如果未配置 `selector`, `DNS` 指向和 `service` 名称相同的任意后端, 或 externalName 类型的服务
137 | 
138 | 3. `headless service` 对应的 `endpoints`
139 | ![headless](./pictures/headless-service.png)
140 | 
141 | 4. 通过 `nslookup` 解析无头服务域名对应的后端ip
142 | ![nslookup](./pictures/headless2.png)
143 | 
144 | 5. 综上所述，无头服务通过 `dns` 的能力实现 `loadbalance` ，不经过 `kubernetes` 转发实现机制，直接将请求转发到后端的 `pod` 上
145 | 
146 | **ExternalName**
147 | 
148 | 1. 直接作用于域名（可以用作 `kubernetes` 的跨 `namespaces` 服务访问）
149 |    - `服务名`是 `namespaces` 隔离的 (test-svc.default.svc.cluster.local)
150 |    - `ClusterIP` 是非 `namespaces` 隔离的
151 | 


--------------------------------------------------------------------------------