├── pkg ├── tests │ ├── scripts │ │ ├── run_exit.sh │ │ ├── text_ok.sh │ │ ├── media_len_zero.sh │ │ ├── show_args.sh │ │ ├── longrunning.sh │ │ ├── media_mp3.sh │ │ ├── media_pdf.sh │ │ ├── preformatted_ok.sh │ │ ├── run_immortal.sh │ │ ├── media_ogg.sh │ │ ├── media_jpeg.sh │ │ ├── preformatted_complex_ok.sh │ │ ├── media_bin.sh │ │ ├── media_mp4.sh │ │ ├── media_png.sh │ │ ├── run_show_args.sh │ │ ├── text_long.sh │ │ ├── text_too_long.sh │ │ ├── preformatted_long.sh │ │ ├── preformatted_too_long.sh │ │ └── run_slow.sh │ ├── check_scripts_test.go │ ├── apiserver │ │ └── apiserver.go │ └── integraion_test.go ├── ctxlog │ ├── README.md │ ├── err_test.go │ ├── ctx.go │ ├── err.go │ ├── handler.go │ └── examples_test.go ├── app │ ├── version.go │ ├── log.go │ ├── app.go │ └── cfg.go ├── xlog │ ├── log.go │ ├── log_test.go │ └── log_fields.go ├── xbot │ ├── bot.go │ └── request.go ├── xjson │ ├── json_test.go │ └── json.go ├── xproc │ └── proc.go ├── xctrl │ └── ctrl.go └── xloop │ └── loop.go ├── .codecov.yml ├── go.mod ├── cmd └── cnbot │ └── main.go ├── demo ├── debugging_wrapper.sh ├── Dockerfile ├── bot_long.sh ├── README.md └── bot.sh ├── LICENSE ├── go.sum ├── .github └── workflows │ └── ci.yaml ├── .golangci.yaml └── README.md /pkg/tests/scripts/run_exit.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | exit 28 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/text_ok.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo ok 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_len_zero.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # do nothing 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/show_args.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "$@ [n=$#]" 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/longrunning.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo args [$tg_x_to]: "$@" 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_mp3.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "ID3" # mp3 signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_pdf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "%%PDF-" # pdf signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/preformatted_ok.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo '%!PRE' 4 | echo ok 5 | -------------------------------------------------------------------------------- /pkg/tests/scripts/run_immortal.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | trap '' SIGINT 4 | sleep 1 5 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_ogg.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "OggS\x00" # ogg signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_jpeg.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "\xFF\xD8\xFF" # jpeg signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/preformatted_complex_ok.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo '%!PRE' 4 | echo ⚒️ 5 | -------------------------------------------------------------------------------- /pkg/ctxlog/README.md: -------------------------------------------------------------------------------- 1 | It seems a good idea to move this logging tooling to the dedicated module. 2 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_bin.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "\x00\x00\x00\x00" # unkown bianry data 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_mp4.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "\x00\x00\x00\x0cftypmp4_" # mp4 signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/media_png.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf "\x89PNG\x0d\x0a\x1a\x0a" # png signature 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/run_show_args.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo "arg1=$1 arg2=$2 test1=$test1 test2=$test2 TEST=$TEST" 4 | -------------------------------------------------------------------------------- /.codecov.yml: -------------------------------------------------------------------------------- 1 | # validate it before commit: curl --data-binary @codecov.yml https://codecov.io/validate 2 | coverage: 3 | range: 60..75 4 | -------------------------------------------------------------------------------- /pkg/tests/scripts/text_long.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # (12+1)*315 = 4095 chars 4 | for i in {1..315} 5 | do 6 | echo "⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘" # 12+1 chars (including "\n") 7 | done 8 | echo 1 # exactly 4096 chars after trimming spaces 9 | -------------------------------------------------------------------------------- /pkg/tests/scripts/text_too_long.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # (12+1)*315 = 4095 chars 4 | for i in {1..315} 5 | do 6 | echo "⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘" # 12+1 chars (including "\n") 7 | done 8 | echo 12 # exactly 4097 chars after trimming spaces 9 | -------------------------------------------------------------------------------- /pkg/tests/scripts/preformatted_long.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo '%!PRE' 4 | 5 | # (12+1)*315 = 4095 chars 6 | for i in {1..315} 7 | do 8 | echo "⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘" # 12+1 chars (including "\n") 9 | done 10 | echo 1 # exactly 4096 chars after trimming spaces 11 | -------------------------------------------------------------------------------- /pkg/tests/scripts/preformatted_too_long.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | echo '%!PRE' 4 | 5 | # (12+1)*315 = 4095 chars 6 | for i in {1..315} 7 | do 8 | echo "⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘" # 12+1 chars (including "\n") 9 | done 10 | echo 12 # exactly 4097 chars after trimming spaces 11 | -------------------------------------------------------------------------------- /pkg/tests/scripts/run_slow.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | trap 'echo trap SIGINT' SIGINT 4 | trap 'echo trap SIGTERM' SIGTERM 5 | trap 'echo trap SIGHUP' SIGHUP 6 | trap 'echo trap SIGQUIT' SIGQUIT 7 | trap 'echo trap EXIT' EXIT 8 | trap 'echo trap ERR' ERR 9 | 10 | echo 'start' 11 | sleep 1 12 | echo 'end' 13 | -------------------------------------------------------------------------------- /pkg/ctxlog/err_test.go: -------------------------------------------------------------------------------- 1 | package ctxlog_test 2 | 3 | import ( 4 | "errors" 5 | "testing" 6 | 7 | "github.com/michurin/cnbot/pkg/ctxlog" 8 | ) 9 | 10 | func TestErrWrap(t *testing.T) { 11 | specificErr := errors.New("x") 12 | err := ctxlog.Errorf("err: %w", specificErr) 13 | t.Log(errors.Is(err, specificErr)) 14 | } 15 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/michurin/cnbot 2 | 3 | go 1.24.0 4 | 5 | toolchain go1.24.2 6 | 7 | require ( 8 | github.com/michurin/systemd-env-file v0.0.0-20251116092800-59c2928ec73e 9 | github.com/stretchr/testify v1.11.1 10 | golang.org/x/sync v0.19.0 11 | ) 12 | 13 | require ( 14 | github.com/davecgh/go-spew v1.1.1 // indirect 15 | github.com/pmezard/go-difflib v1.0.0 // indirect 16 | gopkg.in/yaml.v3 v3.0.1 // indirect 17 | ) 18 | -------------------------------------------------------------------------------- /pkg/app/version.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "fmt" 5 | "runtime/debug" 6 | ) 7 | 8 | func ShowVersionInfo() { 9 | info, ok := debug.ReadBuildInfo() 10 | if !ok { 11 | fmt.Println("No build info") 12 | return 13 | } 14 | fmt.Println(info.Main.Version) 15 | fmt.Println(info.String()) 16 | } 17 | 18 | func MainVersion() string { 19 | info, ok := debug.ReadBuildInfo() 20 | if !ok { 21 | return "no version" 22 | } 23 | return info.Main.Version 24 | } 25 | -------------------------------------------------------------------------------- /cmd/cnbot/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "flag" 6 | "os" 7 | "os/signal" 8 | 9 | "github.com/michurin/cnbot/pkg/app" 10 | "github.com/michurin/cnbot/pkg/xlog" 11 | ) 12 | 13 | var Build = "development" 14 | 15 | func main() { 16 | vFlag := flag.Bool("v", false, "show version") 17 | flag.Parse() 18 | if vFlag != nil && *vFlag { 19 | app.ShowVersionInfo() 20 | return 21 | } 22 | 23 | ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) 24 | defer stop() 25 | 26 | app.SetupLogging() 27 | cfg, tgAPIOrigin, err := app.LoadConfigs(flag.Args()...) 28 | if err != nil { 29 | xlog.L(ctx, err) 30 | return 31 | } 32 | err = app.Application(ctx, cfg, tgAPIOrigin, Build+" "+app.MainVersion()) 33 | if err != nil { 34 | xlog.L(ctx, err) 35 | return 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /pkg/ctxlog/ctx.go: -------------------------------------------------------------------------------- 1 | package ctxlog 2 | 3 | import ( 4 | "context" 5 | ) 6 | 7 | type ctxKeyT int 8 | 9 | const ctxKey = ctxKeyT(0) 10 | 11 | func Add(ctx context.Context, x ...any) context.Context { 12 | if ox, ok := ctx.Value(ctxKey).([][]any); ok { 13 | return context.WithValue(ctx, ctxKey, append(ox, x)) 14 | } 15 | return context.WithValue(ctx, ctxKey, [][]any{x}) 16 | } 17 | 18 | type PatchAttrs struct { 19 | attrs [][]any 20 | } 21 | 22 | func Patch(ctx context.Context) PatchAttrs { 23 | if ox, ok := ctx.Value(ctxKey).([][]any); ok { 24 | return PatchAttrs{attrs: ox} 25 | } 26 | return PatchAttrs{attrs: nil} 27 | } 28 | 29 | func ApplyPatch(ctx context.Context, p PatchAttrs) context.Context { 30 | if len(p.attrs) == 0 { 31 | return ctx 32 | } 33 | if ox, ok := ctx.Value(ctxKey).([][]any); ok { 34 | return context.WithValue(ctx, ctxKey, append(ox, p.attrs...)) 35 | } 36 | return ctx 37 | } 38 | -------------------------------------------------------------------------------- /demo/debugging_wrapper.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # just check correctness of call 4 | if test "$0" = "${0%_debug.sh}" 5 | then 6 | echo "To use wrapper you are to create symlink to your original script or other executable." 7 | echo "To wrap demo_bot.sh, please do" 8 | echo "$ ln -s debugging_wrapper.sh demo_bot_debug.sh" 9 | echo "It will tell debugging_wrapper.sh what executable is wrapped" 10 | exit 1 11 | fi 12 | 13 | # put your command here 14 | cmd="${0%_debug.sh}.sh" 15 | 16 | # tune naming for your taste 17 | t="${cmd##*/}" 18 | t="${t%%.*}" 19 | base="${0%/*}/logs/$(date +%s)_${t}_${$}_" 20 | ext='.log' 21 | 22 | # do not forget to create all necessary directories 23 | mkdir -p "$(dirname "$base")" 24 | 25 | # store command line arguments 26 | n=0 27 | for a in "$@" 28 | do 29 | echo "$a" >"${base}arg_${n}${ext}" 30 | n="$(($n+1))" 31 | done 32 | 33 | # store environment variables 34 | env | sort >"${base}env${ext}" 35 | 36 | # run and store standard streams 37 | "$cmd" "$@" 2>"${base}err${ext}" | tee "${base}out${ext}" 38 | 39 | # store final exit code 40 | echo "$?" >"${base}status${ext}" 41 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018-2025 Alexey Michurin . 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 2 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 3 | github.com/michurin/systemd-env-file v0.0.0-20251116092800-59c2928ec73e h1:2zi9WjLE4vUcRehAFD8Eq5cFHYvWOhkGuHBN783Lqko= 4 | github.com/michurin/systemd-env-file v0.0.0-20251116092800-59c2928ec73e/go.mod h1:qStrSfOCe40VR/pUu0Hl4C6mRoKHumMrET2XslSJzIs= 5 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 6 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 7 | github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= 8 | github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= 9 | golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= 10 | golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= 11 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= 12 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 13 | gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= 14 | gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 15 | -------------------------------------------------------------------------------- /pkg/xlog/log.go: -------------------------------------------------------------------------------- 1 | package xlog 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "log/slog" 7 | "runtime" 8 | "sync/atomic" 9 | "time" 10 | ) 11 | 12 | var defaultLogger atomic.Pointer[slog.Logger] 13 | 14 | func init() { //nolint:gochecknoinits 15 | defaultLogger.Store(slog.Default()) 16 | } 17 | 18 | // SetDefault mimics slog.SetDefault 19 | func SetDefault(l *slog.Logger) { 20 | defaultLogger.Store(l) 21 | } 22 | 23 | // L is botwide logging function, it could be private or internal 24 | // it is best with ctxlog.Handler 25 | func L(ctx context.Context, a any) { 26 | var pcs [1]uintptr 27 | runtime.Callers(2, pcs[:]) // skip 28 | r := slog.Record{} 29 | switch v := a.(type) { 30 | case error: 31 | r = slog.NewRecord(time.Now(), slog.LevelError, v.Error(), pcs[0]) 32 | r.Add("raw_error", v) // if v is wrapped error, the key will be skipped in ctxlog.Handler, value will be interpreted and split to several key-value pairs 33 | case string: 34 | r = slog.NewRecord(time.Now(), slog.LevelInfo, v, pcs[0]) 35 | case []byte: 36 | r = slog.NewRecord(time.Now(), slog.LevelInfo, safeString(v), pcs[0]) 37 | case nil: 38 | r = slog.NewRecord(time.Now(), slog.LevelInfo, "", pcs[0]) 39 | default: 40 | r = slog.NewRecord(time.Now(), slog.LevelWarn, fmt.Sprintf("%[1]T: %#[1]v", a), pcs[0]) 41 | } 42 | _ = defaultLogger.Load().Handler().Handle(ctx, r) 43 | } 44 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: build # this string appears on badge 3 | on: 4 | - push 5 | - pull_request 6 | jobs: 7 | build: 8 | runs-on: ubuntu-latest 9 | timeout-minutes: 10 10 | name: "Go ${{ matrix.go }} build and behavioral tests" 11 | strategy: 12 | matrix: 13 | go: 14 | - "1.24" 15 | - "1.25" 16 | steps: 17 | - uses: actions/checkout@v4 18 | - uses: actions/setup-go@v5 19 | with: 20 | go-version: "${{ matrix.go }}" 21 | - run: "go version" 22 | - run: "go build ./cmd/..." 23 | test: 24 | runs-on: ubuntu-latest 25 | timeout-minutes: 10 26 | name: "Unit tests and linting" 27 | steps: 28 | - uses: actions/checkout@v4 29 | - uses: actions/setup-go@v5 30 | with: 31 | go-version: "1.25" 32 | - uses: golangci/golangci-lint-action@v7 33 | with: 34 | version: "v2.7.2" 35 | - run: "go test -v -coverprofile=coverage.tmp -covermode=atomic -coverpkg=./pkg/... ./pkg/..." 36 | - run: "grep -v /pkg/app/ coverage.tmp >coverage.txt" 37 | - run: "diff coverage.tmp coverage.txt || true" # just to see what has been excluded 38 | - run: "rm coverage.tmp" # otherwise it will be taken into account 39 | - uses: codecov/codecov-action@v5 40 | with: 41 | files: ./coverage.txt 42 | verbose: true 43 | token: ${{ secrets.CODECOV_TOKEN }} # required 44 | -------------------------------------------------------------------------------- /pkg/tests/check_scripts_test.go: -------------------------------------------------------------------------------- 1 | package tests_test 2 | 3 | import ( 4 | "os" 5 | "path" 6 | "testing" 7 | 8 | "github.com/stretchr/testify/assert" 9 | "github.com/stretchr/testify/require" 10 | ) 11 | 12 | func TestCheckScripts(t *testing.T) { 13 | knownEntitiesToSkip := map[string]struct{}{ 14 | "../../demo/logs": {}, 15 | "../../demo/bot_debug.sh": {}, 16 | "../../demo/bot_long_debug.sh": {}, 17 | "../../demo/README.md": {}, // TODO: check it too 18 | "../../demo/Dockerfile": {}, // TODO: check it too 19 | } 20 | for _, scriptsDir := range []string{"scripts", "../../demo"} { 21 | ee, err := os.ReadDir(scriptsDir) 22 | require.NoError(t, err) 23 | require.NotNil(t, len(ee)) 24 | for _, e := range ee { 25 | t.Run(e.Name(), func(t *testing.T) { 26 | scriptName := path.Join(scriptsDir, e.Name()) 27 | if _, skip := knownEntitiesToSkip[scriptName]; skip { 28 | t.Skip("Skipping known entity") 29 | } 30 | t.Log("Script", scriptName) 31 | require.True(t, e.Type().IsRegular()) 32 | c, err := os.ReadFile(scriptName) 33 | require.NoError(t, err) 34 | content := string(c) 35 | assert.Regexp(t, `^#!/bin/bash\n\n[^\n]`, content) 36 | assert.NotRegexp(t, `[\t\r\v]`, content) // no tabs etc 37 | assert.NotRegexp(t, `\x20+\n`, content) // no leading spaces (except EOF case) 38 | assert.Regexp(t, `\S\n$`, content) // strictly one EOL at EOF 39 | }) 40 | } 41 | } 42 | } 43 | -------------------------------------------------------------------------------- /pkg/ctxlog/err.go: -------------------------------------------------------------------------------- 1 | package ctxlog 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "fmt" 7 | "runtime" 8 | ) 9 | 10 | type wrapError struct { 11 | err error 12 | attrs [][]any 13 | pc uintptr 14 | } 15 | 16 | func Errorf(format string, a ...any) error { 17 | err := fmt.Errorf(format, a...) //nolint:goerr113 18 | x := new(wrapError) 19 | if errors.As(err, &x) { // do not wrap twice 20 | return &wrapError{err: err, attrs: x.attrs, pc: x.pc} // new message (Error()), but attrs and pc from wrapped error 21 | } 22 | var pcs [1]uintptr 23 | runtime.Callers(2, pcs[:]) 24 | return &wrapError{err: err, attrs: nil, pc: pcs[0]} 25 | } 26 | 27 | func Errorfx(ctx context.Context, format string, a ...any) error { 28 | err := fmt.Errorf(format, a...) //nolint:goerr113 29 | x := new(wrapError) 30 | if errors.As(err, &x) { // already wrapped 31 | attrs := x.attrs // however it is possible we do not have any attrs yet 32 | if attrs == nil { // in this case we are fill attrs 33 | if a, ok := ctx.Value(ctxKey).([][]any); ok { 34 | attrs = a 35 | } 36 | } 37 | return &wrapError{err: err, attrs: attrs, pc: x.pc} // new message (Error()), but attrs and pc from wrapped error 38 | } 39 | var pcs [1]uintptr 40 | runtime.Callers(2, pcs[:]) 41 | attrs := [][]any(nil) 42 | if a, ok := ctx.Value(ctxKey).([][]any); ok { 43 | attrs = a 44 | } 45 | return &wrapError{err: err, attrs: attrs, pc: pcs[0]} 46 | } 47 | 48 | func (e *wrapError) Error() string { 49 | return e.err.Error() 50 | } 51 | 52 | func (e *wrapError) Unwrap() error { 53 | return e.err 54 | } 55 | -------------------------------------------------------------------------------- /demo/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM debian:trixie-slim 2 | 3 | RUN apt-get update \ 4 | && echo 'Installing packages...' \ 5 | && apt-get install -y \ 6 | git golang \ 7 | shared-mime-info ca-certificates \ 8 | curl libcap2-bin imagemagick jq procps \ 9 | less mc vim ne \ 10 | pipx \ 11 | && echo 'Installing mitmproxy...' \ 12 | && pipx install --global mitmproxy \ 13 | && echo 'Adding ordinary user...' \ 14 | && groupadd -r cnbotgroup \ 15 | && useradd -r -g cnbotgroup cnbot \ 16 | && echo 'Adding directories...' \ 17 | && mkdir \ 18 | /app \ 19 | /app/src \ 20 | /app/logs \ 21 | && echo 'Building cnbot...' \ 22 | && cd /app/src \ 23 | && git clone --depth 1 --branch master https://github.com/michurin/cnbot.git \ 24 | && cd cnbot \ 25 | && go mod download \ 26 | && go mod verify \ 27 | && go build ./cmd/... \ 28 | && mv cnbot /app \ 29 | && cp \ 30 | demo/bot.sh \ 31 | demo/bot_long.sh \ 32 | demo/debugging_wrapper.sh \ 33 | /app \ 34 | && cd /app \ 35 | && ln -s debugging_wrapper.sh bot_debug.sh \ 36 | && ln -s debugging_wrapper.sh bot_long_debug.sh \ 37 | && chown cnbot:cnbotgroup \ 38 | logs \ 39 | bot.sh \ 40 | bot_long.sh \ 41 | debugging_wrapper.sh \ 42 | && chown -h cnbot:cnbotgroup \ 43 | bot_debug.sh \ 44 | bot_long_debug.sh \ 45 | && echo 'RUN successfully finished.' 46 | 47 | USER cnbot 48 | WORKDIR /app 49 | 50 | # You must specify TB_TOKEN variable 51 | ENV TB_CTRL_ADDR=:9999 \ 52 | TB_SCRIPT=/app/bot.sh \ 53 | TB_LONG_RUNNING_SCRIPT=/app/bot_long.sh 54 | 55 | CMD ["/app/cnbot"] 56 | -------------------------------------------------------------------------------- /pkg/xlog/log_test.go: -------------------------------------------------------------------------------- 1 | package xlog_test 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "log/slog" 7 | "os" 8 | 9 | "github.com/michurin/cnbot/pkg/ctxlog" 10 | "github.com/michurin/cnbot/pkg/xlog" 11 | ) 12 | 13 | // WARNING: L is global and this tests can ruin other tests 14 | // -------------------------------------------------------- 15 | 16 | var optToBeReproducible = &slog.HandlerOptions{ 17 | ReplaceAttr: func(_ []string, a slog.Attr) slog.Attr { 18 | if a.Key == slog.TimeKey { 19 | return slog.Attr{} 20 | } 21 | return a 22 | }, 23 | } 24 | 25 | func ExampleL() { 26 | xlog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, optToBeReproducible))) 27 | ctx := context.Background() 28 | xlog.L(ctx, "ok") 29 | xlog.L(ctx, errors.New("err")) 30 | // Output: 31 | // level=INFO msg=ok 32 | // level=ERROR msg=err raw_error=err 33 | } 34 | 35 | func ExampleL_withCtxLogHandler() { 36 | xlog.SetDefault(slog.New(ctxlog.Handler(slog.NewTextHandler(os.Stdout, optToBeReproducible), "log_test.go"))) 37 | ctx := context.Background() 38 | ctx = ctxlog.Add(ctx, "label", "A") 39 | xlog.L(ctx, "ok") 40 | err := func(ctx context.Context) error { // some random function 41 | ctx = ctxlog.Add(ctx, "scope", "S") 42 | err := errors.New("err") 43 | return ctxlog.Errorfx(ctx, "error: %w", err) 44 | }(ctx) 45 | xlog.L(ctx, err) // we do not have scope=S in this ctx, however, we can see it in logs 46 | // Output: 47 | // level=INFO msg=ok source=log_test.go:39 label=A 48 | // level=ERROR msg="error: err" source=log_test.go:45 err_source=log_test.go:43 err_msg="error: err" label=A scope=S 49 | } 50 | -------------------------------------------------------------------------------- /pkg/xlog/log_fields.go: -------------------------------------------------------------------------------- 1 | package xlog 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "fmt" 7 | "log/slog" 8 | "unicode/utf8" 9 | 10 | "github.com/michurin/cnbot/pkg/ctxlog" 11 | ) 12 | 13 | func Bot(ctx context.Context, name string) context.Context { 14 | return ctxlog.Add(ctx, slog.String("bot", name)) 15 | } 16 | 17 | func Comp(ctx context.Context, name string) context.Context { 18 | return ctxlog.Add(ctx, slog.String("comp", name)) 19 | } 20 | 21 | func API(ctx context.Context, method, contentType string) context.Context { 22 | return ctxlog.Add(ctx, slog.String("api", method), slog.String("content_type", contentType)) 23 | } 24 | 25 | func Status(ctx context.Context, status int) context.Context { 26 | return ctxlog.Add(ctx, slog.Int("status", status)) 27 | } 28 | 29 | func Request(ctx context.Context, request []byte) context.Context { 30 | return ctxlog.Add(ctx, slog.String("request", safeString(request))) 31 | } 32 | 33 | func Response(ctx context.Context, response []byte) context.Context { 34 | return ctxlog.Add(ctx, slog.String("response", safeString(response))) 35 | } 36 | 37 | func Path(ctx context.Context, path string) context.Context { 38 | return ctxlog.Add(ctx, slog.String("path", path)) 39 | } 40 | 41 | func User(ctx context.Context, user int64) context.Context { 42 | return ctxlog.Add(ctx, slog.Int64("user", user)) 43 | } 44 | 45 | func Pid(ctx context.Context, pid int) context.Context { 46 | return ctxlog.Add(ctx, slog.Int("pid", pid)) 47 | } 48 | 49 | func safeString(x []byte) string { // TODO move to field-specific place (does not exist yet) 50 | idx := bytes.IndexByte(x, '\n') 51 | if idx >= 0 { 52 | x = x[:idx] // side effect prone 53 | } 54 | if utf8.Valid(x) { 55 | return string(x) 56 | } 57 | return fmt.Sprintf("%q", x) 58 | } 59 | -------------------------------------------------------------------------------- /pkg/app/log.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "log/slog" 7 | "sort" 8 | 9 | "github.com/michurin/cnbot/pkg/ctxlog" 10 | "github.com/michurin/cnbot/pkg/xlog" 11 | ) 12 | 13 | // logHandler implements interface slog.Handler 14 | // it is drop-in replacement for slog.NewTextHandler, but more human friendly 15 | type logHandler struct{} 16 | 17 | func (logHandler) Enabled(context.Context, slog.Level) bool { 18 | return true 19 | } 20 | 21 | func (logHandler) Handle(_ context.Context, r slog.Record) error { 22 | kv := map[string]any{} // not thread safe, however r.Attrs works consequently 23 | r.Attrs(func(a slog.Attr) bool { 24 | kv[a.Key] = a.Value.Any() 25 | return true 26 | }) 27 | std := "" // std attributes 28 | for _, a := range []string{"bot", "comp", "api", "source"} { // order significant 29 | if v, ok := kv[a]; ok { 30 | std = std + " [" + v.(string) + "]" //nolint:forcetypeassert // we use typed helples to enrich context with all this values 31 | delete(kv, a) 32 | } 33 | } 34 | ekeys := []string(nil) // extra keys 35 | for k := range kv { 36 | ekeys = append(ekeys, k) 37 | } 38 | sort.Strings(ekeys) 39 | nstd := "" 40 | for _, a := range ekeys { 41 | nstd += fmt.Sprintf(" %s=%v", a, kv[a]) 42 | } 43 | fmt.Printf("%s [%s]%s%s %s\n", r.Time.Format("2006-01-02 15:04:05"), r.Level.String(), std, nstd, r.Message) 44 | return nil 45 | } 46 | 47 | func (h logHandler) WithAttrs(_ []slog.Attr) slog.Handler { 48 | panic("NOT IMPLEMENTED") 49 | } 50 | 51 | func (logHandler) WithGroup(_ string) slog.Handler { 52 | panic("NOT IMPLEMENTED") 53 | } 54 | 55 | func SetupLogging() { 56 | l := slog.New(ctxlog.Handler(logHandler{}, "app/log.go")) 57 | xlog.SetDefault(l) 58 | } 59 | -------------------------------------------------------------------------------- /.golangci.yaml: -------------------------------------------------------------------------------- 1 | version: "2" 2 | linters: 3 | default: all 4 | disable: 5 | - err113 6 | - exhaustruct 7 | - gochecknoglobals 8 | - godot 9 | - godox 10 | - mnd 11 | - modernize # nice to be turned on in future 12 | - nlreturn 13 | - paralleltest 14 | - perfsprint # nice to be turned on 15 | - wrapcheck # nice to be turned on 16 | - wsl 17 | - wsl_v5 18 | settings: 19 | cyclop: 20 | max-complexity: 20 21 | funlen: 22 | lines: 100 23 | statements: 60 24 | depguard: 25 | rules: 26 | main: 27 | allow: 28 | - $gostd 29 | - github.com/michurin/cnbot 30 | - github.com/michurin/systemd-env-file 31 | - github.com/stretchr/testify 32 | - golang.org/x/sync/errgroup 33 | lll: 34 | line-length: 160 35 | testifylint: 36 | disable: 37 | - require-error 38 | revive: 39 | rules: 40 | - name: exported 41 | disabled: true 42 | varnamelen: 43 | max-distance: 50 44 | exclusions: 45 | generated: lax 46 | warn-unused: true 47 | rules: 48 | - path: "pkg/app/(log|version)\\.go$" 49 | linters: 50 | - forbidigo 51 | - source: "^\\s*defer\\s+" 52 | linters: 53 | - errcheck 54 | - source: "\\s(os\\.ReadFile|exec\\.Command)\\(" 55 | linters: 56 | - gosec 57 | - path: "_test\\.go$" 58 | linters: 59 | - lll 60 | - varnamelen 61 | - source: "\\*errgroup.Group" 62 | linters: 63 | - varnamelen 64 | - source: "w http.ResponseWriter" 65 | linters: 66 | - varnamelen 67 | - source: "\\(nil\\)$" 68 | linters: 69 | - wastedassign 70 | formatters: 71 | enable: 72 | - gci 73 | - gofmt 74 | - gofumpt 75 | - goimports 76 | settings: 77 | gci: 78 | sections: 79 | - standard 80 | - default 81 | - prefix(github.com/michurin/cnbot) 82 | -------------------------------------------------------------------------------- /pkg/tests/apiserver/apiserver.go: -------------------------------------------------------------------------------- 1 | package apiserver 2 | 3 | import ( 4 | "context" 5 | "io" 6 | "net/http" 7 | "net/http/httptest" 8 | "strings" 9 | "testing" 10 | 11 | "github.com/stretchr/testify/assert" 12 | ) 13 | 14 | type APIAct struct { 15 | IsJSON bool // TODO use content type? 16 | Stream bool // IsJSON must be false if Stream is true 17 | Request string 18 | Response []byte 19 | } 20 | 21 | func APIServer(t *testing.T, cancel context.CancelFunc, api map[string][]APIAct) (string, func()) { 22 | t.Helper() 23 | testDone := make(chan struct{}) 24 | steps := map[string]int{} // it looks ugly, however we can use it without locks 25 | tg := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { 26 | // DO NOT user require.* in this handler. 27 | // require.* is based on t.FailNow() it won't work in goroutines 28 | // assert.* is founded on t.Fatal() 29 | // assert.Equal(t, http.MethodPost, r.Method) // TODO! Assert method 30 | bodyBytes, err := io.ReadAll(r.Body) 31 | assert.NoError(t, err) 32 | body := string(bodyBytes) 33 | 34 | url := r.URL.String() 35 | t.Logf("Mock server: url=%q", url) 36 | n := steps[url] 37 | ax, ok := api[url] 38 | assert.True(t, ok, "URL not found: "+url) 39 | a := ax[n] 40 | steps[url] = n + 1 41 | if a.Stream { 42 | _, err = w.Write(a.Response) 43 | assert.NoError(t, err) 44 | assert.False(t, a.IsJSON, "IsJSON must be false if Stream is true") 45 | return 46 | } 47 | if a.IsJSON { 48 | assert.Equal(t, "application/json", r.Header.Get("Content-Type")) 49 | assert.JSONEq(t, a.Request, body) 50 | } else { 51 | ctype := r.Header.Get("Content-Type") 52 | assert.Contains(t, ctype, "multipart/form-data") 53 | idx := strings.Index(ctype, "boundary=") 54 | assert.Greater(t, idx, -1, "ctype="+ctype) 55 | universal := strings.ReplaceAll(body, ctype[idx+9:], "BOUND") 56 | assert.Equal(t, a.Request, universal) 57 | } 58 | if a.Response == nil { 59 | cancel() 60 | <-testDone 61 | } 62 | _, err = w.Write(a.Response) 63 | assert.NoError(t, err) 64 | })) 65 | return tg.URL, func() { 66 | close(testDone) 67 | tg.Close() 68 | eSteps := map[string]int{} // expected steps 69 | for k, v := range api { 70 | eSteps[k] = len(v) 71 | } 72 | assert.Equal(t, steps, eSteps, "all calls are really happened") 73 | } 74 | } 75 | -------------------------------------------------------------------------------- /pkg/xbot/bot.go: -------------------------------------------------------------------------------- 1 | package xbot 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "io" 7 | "net/http" 8 | 9 | "github.com/michurin/cnbot/pkg/ctxlog" 10 | "github.com/michurin/cnbot/pkg/xlog" 11 | ) 12 | 13 | type Client interface { 14 | Do(r *http.Request) (*http.Response, error) 15 | } 16 | 17 | type Bot struct { 18 | APIOrigin string // injection to be testable 19 | Token string 20 | Client Client // injection to be observable // TODO move all logging into client middleware? 21 | } 22 | 23 | func (b *Bot) API(ctx context.Context, request *Request) ([]byte, error) { 24 | ctx = xlog.API(ctx, request.Method, request.ContentType) 25 | err := error(nil) 26 | req := (*http.Request)(nil) 27 | resp := (*http.Response)(nil) 28 | respCode := 0 29 | data := []byte(nil) 30 | defer func() { 31 | msg := any(nil) 32 | if err != nil { 33 | msg = err 34 | } else { 35 | msg = "ok" 36 | } 37 | xlog.L(xlog.Status(xlog.Request(xlog.Response(ctx, data), request.Body), respCode), msg) 38 | }() 39 | reqURL := b.APIOrigin + "/bot" + b.Token + "/" + request.Method 40 | req, err = http.NewRequestWithContext(ctx, http.MethodPost, reqURL, bytes.NewReader(request.Body)) 41 | if err != nil { 42 | return nil, ctxlog.Errorfx(ctx, "request constructor: %w", err) 43 | } 44 | req.Header.Set("Content-Type", request.ContentType) 45 | resp, err = b.Client.Do(req) 46 | if err != nil { 47 | return nil, ctxlog.Errorfx(ctx, "client: %w", err) 48 | } 49 | respCode = resp.StatusCode 50 | defer resp.Body.Close() // we are skipping error here 51 | data, err = io.ReadAll(resp.Body) // we have to read and close Body even for non-200 responses 52 | if err != nil { 53 | return nil, ctxlog.Errorfx(ctx, "reading: %w", err) 54 | } 55 | return data, nil 56 | } 57 | 58 | func (b *Bot) Download(ctx context.Context, path string, stream io.Writer) error { 59 | ctx = xlog.API(ctx, "x-download", "x") 60 | err := error(nil) 61 | defer func() { 62 | xlog.L(xlog.Path(ctx, path), err) 63 | }() 64 | reqURL := b.APIOrigin + "/file/bot" + b.Token + "/" + path 65 | req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil) 66 | if err != nil { 67 | return ctxlog.Errorfx(ctx, "request constructor: %w", err) 68 | } 69 | resp, err := b.Client.Do(req) 70 | if err != nil { 71 | return ctxlog.Errorfx(ctx, "client: %w", err) 72 | } 73 | defer resp.Body.Close() // we are skipping error here 74 | _, err = io.Copy(stream, resp.Body) 75 | if err != nil { 76 | return ctxlog.Errorfx(ctx, "coping: %w", err) 77 | } 78 | return nil 79 | } 80 | -------------------------------------------------------------------------------- /demo/bot_long.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | LOG=logs/log_long.log # /dev/null to disable logging 4 | mkdir -p "$(dirname "$LOG")" # do not forget to create all necessary directories 5 | 6 | FROM="$tg_x_to" 7 | 8 | API() { 9 | API_STDOUT "$@" >>"$LOG" 10 | } 11 | 12 | API_STDOUT() { 13 | url="http://localhost$tg_x_ctrl_addr/$1" 14 | shift 15 | echo "====== curl $url $@" >>"$LOG" 16 | curl -qs "$url" "$@" 2>>"$LOG" 17 | echo >>"$LOG" 18 | echo >>"$LOG" 19 | } 20 | 21 | case "$1" in 22 | reactions) 23 | MESSAGE_ID="$2" 24 | for e in "👾" "🤔" "😎" 25 | do 26 | API setMessageReaction -F chat_id=$FROM -F message_id=$MESSAGE_ID -F reaction='[{"type":"emoji","emoji":"'"$e"'"}]' 27 | sleep 1 28 | done 29 | API setMessageReaction -F chat_id=$FROM -F message_id=$MESSAGE_ID -F reaction='[]' 30 | ;; 31 | editing) 32 | MESSAGE_ID="$(API_STDOUT sendMessage -F chat_id=$FROM -F text='Starting...' | jq .result.message_id)" 33 | if test -n "$MESSAGE_ID" 34 | then 35 | for i in 2 4 6 8 36 | do 37 | sleep 1 38 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text="Doing... ${i}0% complete..." 39 | done 40 | sleep 1 41 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text='Done.' 42 | else 43 | echo "cannot obtain message id" 44 | fi 45 | ;; 46 | progress) 47 | MESSAGE_ID="$(API_STDOUT sendMessage -F chat_id=$FROM -F text='Starting...' | jq .result.message_id)" 48 | if test -n "$MESSAGE_ID" 49 | then 50 | for i in \ 51 | '[..........]' \ 52 | '[#.........]' \ 53 | '[##........]' \ 54 | '[###.......]' \ 55 | '[####......]' \ 56 | '[#####.....]' \ 57 | '[######....]' \ 58 | '[#######...]' \ 59 | '[########..]' \ 60 | '[#########.]' 61 | do 62 | sleep 1 63 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text="\`${i}\` Doing..." -F parse_mode=Markdown 64 | done 65 | sleep 1 66 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text='Done.' 67 | else 68 | echo "cannot obtain message id" 69 | fi 70 | ;; 71 | *) 72 | echo 'invalid mode' 73 | ;; 74 | esac 75 | -------------------------------------------------------------------------------- /pkg/xjson/json_test.go: -------------------------------------------------------------------------------- 1 | package xjson_test 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/stretchr/testify/assert" 7 | "github.com/stretchr/testify/require" 8 | 9 | "github.com/michurin/cnbot/pkg/xjson" 10 | ) 11 | 12 | func TestJSONToEnv(t *testing.T) { 13 | t.Run("ok", func(t *testing.T) { 14 | x := map[string]any{ 15 | // basic types 16 | "a": nil, 17 | "b": false, 18 | "c": true, 19 | "d": float64(1), 20 | "e": "text", 21 | "f": []any{"element"}, 22 | "g": map[string]any{"h": "sub"}, 23 | // complex nested structure 24 | "i": []any{ 25 | map[string]any{ 26 | "a": float64(1), 27 | "b": float64(2), 28 | }, 29 | map[string]any{ 30 | "c": []any{float64(3), float64(4)}, 31 | }, 32 | }, 33 | // corner cases 34 | "j": "", 35 | "k": float64(.3), 36 | "l": float64(.2) + float64(.1), 37 | "m": float64(1<<53 - 1), 38 | // corner cases: partially skipping 39 | "n": []any{float64(1), nil, float64(3)}, // m[1] won't appear 40 | "o": map[string]any{"a": float64(1), "b": nil, "c": float64(3)}, // n["b"] won't appear 41 | // not appears: empty structures 42 | "p": map[string]any{}, 43 | "q": []any{}, 44 | // not appears: empty nested structures 45 | "r": map[string]any{"a": nil, "b": []any{}, "c": map[string]any{}}, 46 | "s": []any{nil, []any{}, map[string]any{}}, 47 | } 48 | env, err := xjson.JSONToEnv(x) 49 | require.NoError(t, err) 50 | assert.Equal(t, []string{ 51 | "tg_b=false", 52 | "tg_c=true", 53 | "tg_d=1", 54 | "tg_e=text", 55 | "tg_f=tg_f_0", 56 | "tg_f_0=element", 57 | "tg_g_h=sub", 58 | "tg_i=tg_i_0 tg_i_1", 59 | "tg_i_0_a=1", 60 | "tg_i_0_b=2", 61 | "tg_i_1_c=tg_i_1_c_0 tg_i_1_c_1", 62 | "tg_i_1_c_0=3", 63 | "tg_i_1_c_1=4", 64 | "tg_j=", 65 | "tg_k=0.3", 66 | "tg_l=0.30000000000000004", 67 | "tg_m=9007199254740991", // 2**53-1 (all bits are '1') 68 | "tg_n=tg_n_0 tg_n_2", 69 | "tg_n_0=1", 70 | "tg_n_2=3", 71 | "tg_o_a=1", 72 | "tg_o_c=3", 73 | }, env) 74 | }) 75 | t.Run("invalidType", func(t *testing.T) { 76 | x := float32(1) 77 | env, err := xjson.JSONToEnv(x) 78 | assert.EqualError(t, err, "invalid type [pfx=tg]: float32") 79 | require.Nil(t, env) 80 | }) 81 | t.Run("invalidTypeInSlice", func(t *testing.T) { 82 | x := []any{float32(1)} 83 | env, err := xjson.JSONToEnv(x) 84 | assert.EqualError(t, err, "invalid type [pfx=tg_0]: float32") 85 | require.Nil(t, env) 86 | }) 87 | t.Run("invalidTypeInMap", func(t *testing.T) { 88 | x := map[string]any{"k": float32(1)} 89 | env, err := xjson.JSONToEnv(x) 90 | assert.EqualError(t, err, "invalid type [pfx=tg_k]: float32") 91 | require.Nil(t, env) 92 | }) 93 | } 94 | -------------------------------------------------------------------------------- /demo/README.md: -------------------------------------------------------------------------------- 1 | # Docker cheat sheet 2 | 3 | ## About container 4 | 5 | This `Dockerfile` provides container for development, testing and playing around. Not for production. 6 | It contains tools for editing and viewing, for sniffing network traffic, for debugging scripts and verbose logging. 7 | 8 | ## Prerequisites 9 | 10 | All you need to run demo `cnbot` is Telegram bot token. You can [obtain it for free](https://core.telegram.org/bots/tutorial#obtain-your-bot-token). 11 | 12 | ## First run and first glance 13 | 14 | Build image 15 | 16 | ```sh 17 | sudo docker build -t cnbot:v2 . 18 | ``` 19 | 20 | Run bot (use your own token) 21 | 22 | ```sh 23 | sudo docker run -it --rm --name cnbot -e 'TB_TOKEN=4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc' cnbot:v2 24 | ``` 25 | 26 | Now bot is ready, you can to talk to it in your Telegram client. 27 | 28 | ## Enter to container, viewing logs and playing with scripts 29 | 30 | You are free to enter to container 31 | 32 | ```sh 33 | sudo docker exec -it cnbot /bin/bash 34 | ``` 35 | 36 | If you want to be `root` in the container, you can: 37 | 38 | ```sh 39 | sudo docker exec -it -u root cnbot /bin/bash 40 | ``` 41 | 42 | Now you are in container. You can run `mc` for navigation, `vim` or `ne` for editing. 43 | 44 | You can find logs in `/app/logs`, you can modify `/app/bot.sh` and `/app/bot_logn.sh`. You do not need to restart `cnbot` to see your changes. 45 | 46 | You can turn on super verbose logging 47 | 48 | ```sh 49 | sudo docker run -it --rm --name cnbot -e 'TB_TOKEN=4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc' -e 'TB_SCRIPT=/app/bot_debug.sh' -e 'TB_LONG_RUNNING_SCRIPT=/app/bot_long_debug.sh' cnbot:v2 50 | ``` 51 | 52 | ## Calling bot's control API in container 53 | 54 | For example to call `getMe` just say 55 | 56 | ```sh 57 | sudo docker exec cnbot curl -qs http://localhost:9999/getMe | jq 58 | ``` 59 | 60 | ## Examination interaction with Telegram API 61 | 62 | It is `mitmproxy` available in the container. You can run `mitmproxy` tool like this 63 | 64 | ```sh 65 | sudo docker run -it --rm --name cnbot -e 'TB_TOKEN=4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc' -e 'TB_SCRIPT=/app/bot_debug.sh' -e 'TB_LONG_RUNNING_SCRIPT=/app/bot_long_debug.sh' -e 'TB_API_ORIGIN=http://localhost:9001' cnbot:v2 /usr/local/bin/mitmdump --set confdir=/tmp --flow-detail 4 -p 9001 --mode reverse:https://api.telegram.org 66 | ``` 67 | 68 | And than execute `cnbot` in this container: 69 | 70 | ```sh 71 | sudo docker exec -it cnbot ./cnbot 72 | ``` 73 | 74 | Now try to talk with bot and enjoy verbose output. 75 | 76 | ## Just docker memo 77 | 78 | ```sh 79 | sudo docker images # list images 80 | sudo docker image rm -f cnbot:v2 # remove image 81 | 82 | sudo docker image prune # cleanup everything 83 | sudo docker container prune 84 | ``` 85 | 86 | You are free to install what you need in container 87 | 88 | ```sh 89 | apt-get update 90 | apt-get install -y zsh 91 | ``` 92 | -------------------------------------------------------------------------------- /pkg/ctxlog/handler.go: -------------------------------------------------------------------------------- 1 | package ctxlog 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "fmt" 7 | "log/slog" 8 | "runtime" 9 | "strings" 10 | ) 11 | 12 | type handler struct { 13 | pfx string 14 | next slog.Handler 15 | } 16 | 17 | func Handler(next slog.Handler, sfx string) slog.Handler { 18 | pfx := "" 19 | _, file, _, ok := runtime.Caller(1) 20 | if ok { 21 | p, ok := strings.CutSuffix(file, sfx) 22 | if ok { 23 | pfx = p 24 | } 25 | } 26 | return &handler{pfx: pfx, next: next} 27 | } 28 | 29 | func (h *handler) Enabled(ctx context.Context, level slog.Level) bool { 30 | return h.next.Enabled(ctx, level) 31 | } 32 | 33 | func (h *handler) Handle(ctx context.Context, ro slog.Record) error { 34 | // Slightly overcomplicated approach. So we want to: 35 | // - save attrs from handler as is 36 | // - find and skip wrapped errors 37 | // - deduplicate keys, that come from ctx and err 38 | // We want to manage !BADKEY naturally, preserve order of keys and 39 | // certain priority. 40 | // Beware: we does not see here attrs of next handler, so we do not consider and process them. 41 | r := slog.NewRecord(ro.Time, ro.Level, ro.Message, ro.PC) // result record; we are enable duplicates here 42 | if ro.PC != 0 { 43 | r.AddAttrs(slog.Any(slog.SourceKey, h.fname(ro.PC))) 44 | } 45 | lastErr := (*wrapError)(nil) // not thread safe code; h.Attrs doing consequently 46 | ro.Attrs(func(a slog.Attr) bool { 47 | v := a.Value.Any() 48 | if err, ok := v.(error); ok { 49 | e := new(wrapError) 50 | if errors.As(err, &e) { 51 | lastErr = e 52 | return true 53 | } 54 | } 55 | r.AddAttrs(a) 56 | return true 57 | }) 58 | ri := slog.NewRecord(ro.Time, ro.Level, ro.Message, ro.PC) // interim record; to manage !BADKEY etc in native way 59 | if aa, ok := ctx.Value(ctxKey).([][]any); ok { 60 | for _, a := range aa { 61 | ri.Add(a...) 62 | } 63 | } 64 | if lastErr != nil { 65 | r.AddAttrs(slog.Any("err_source", h.fname(lastErr.pc)), slog.Any("err_msg", lastErr.Error())) 66 | for _, a := range lastErr.attrs { 67 | ri.Add(a...) 68 | } 69 | } 70 | idx := map[string]int{} 71 | i := 0 72 | ri.Attrs(func(a slog.Attr) bool { 73 | idx[a.Key] = i 74 | i++ 75 | return true 76 | }) 77 | i = 0 78 | ri.Attrs(func(a slog.Attr) bool { 79 | if idx[a.Key] == i { 80 | r.AddAttrs(a) 81 | } 82 | i++ 83 | return true 84 | }) 85 | return h.next.Handle(ctx, r) //nolint:wrapcheck // this error will be ignored at log/slog/logger.log() 86 | } 87 | 88 | func (h *handler) WithAttrs(attrs []slog.Attr) slog.Handler { 89 | return &handler{ 90 | pfx: h.pfx, 91 | next: h.next.WithAttrs(attrs), 92 | } 93 | } 94 | 95 | func (h *handler) WithGroup(name string) slog.Handler { 96 | return &handler{ 97 | pfx: h.pfx, 98 | next: h.next.WithGroup(name), 99 | } 100 | } 101 | 102 | func (h *handler) fname(pc uintptr) string { 103 | f, _ := runtime.CallersFrames([]uintptr{pc}).Next() 104 | file, _ := strings.CutPrefix(f.File, h.pfx) 105 | return fmt.Sprintf("%s:%d", file, f.Line) 106 | } 107 | -------------------------------------------------------------------------------- /pkg/app/app.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "context" 5 | "net/http" 6 | "os" 7 | "os/exec" 8 | "path" 9 | "path/filepath" 10 | "strings" 11 | "time" 12 | 13 | "golang.org/x/sync/errgroup" 14 | 15 | "github.com/michurin/cnbot/pkg/ctxlog" 16 | "github.com/michurin/cnbot/pkg/xbot" 17 | "github.com/michurin/cnbot/pkg/xctrl" 18 | "github.com/michurin/cnbot/pkg/xlog" 19 | "github.com/michurin/cnbot/pkg/xloop" 20 | "github.com/michurin/cnbot/pkg/xproc" 21 | ) 22 | 23 | func bot(ctx context.Context, eg *errgroup.Group, cfg BotConfig, tgAPIOrigin, build string) error { 24 | bot := &xbot.Bot{ 25 | APIOrigin: tgAPIOrigin, 26 | Token: cfg.Token, 27 | Client: http.DefaultClient, 28 | } 29 | 30 | envCommon := []string{"tg_x_ctrl_addr=" + cfg.ControlAddr, "tg_x_build=" + build} 31 | 32 | cfgDir, err := filepath.Abs(cfg.ConfigFileDir) // if dir is "", it uses CWD 33 | if err != nil { 34 | return ctxlog.Errorfx(ctx, "invalid config dir: %w", err) 35 | } 36 | 37 | // caution: 38 | // do not return errors and do not interrupt flow after goroutines running 39 | // it will lead to goroutine leaking 40 | 41 | command := &xproc.Cmd{ 42 | InterruptDelay: 10 * time.Second, 43 | KillDelay: 10 * time.Second, 44 | Env: envCommon, 45 | Command: absPathToBinary(cfgDir, cfg.Script), 46 | } 47 | 48 | commandLong := &xproc.Cmd{ 49 | InterruptDelay: 10 * time.Minute, 50 | KillDelay: 10 * time.Minute, 51 | Env: envCommon, 52 | Command: absPathToBinary(cfgDir, cfg.LongRunningScript), 53 | } 54 | 55 | eg.Go(func() error { 56 | err := xloop.Loop(xlog.Comp(ctx, "loop"), bot, command) 57 | if err != nil { 58 | return ctxlog.Errorfx(ctx, "polling loop: %w", err) 59 | } 60 | return nil 61 | }) 62 | 63 | server := &http.Server{ 64 | Addr: cfg.ControlAddr, 65 | Handler: xctrl.Handler( 66 | bot, 67 | commandLong, 68 | ctxlog.Patch(xlog.Comp(ctx, "ctrl")), 69 | ), 70 | ReadHeaderTimeout: time.Minute, // slowloris attack protection 71 | } 72 | eg.Go(func() error { 73 | <-ctx.Done() 74 | cx, stop := context.WithTimeout(context.Background(), time.Second) 75 | defer stop() 76 | return server.Shutdown(cx) //nolint:contextcheck 77 | }) 78 | 79 | eg.Go(func() error { 80 | err := server.ListenAndServe() 81 | if err != nil { 82 | return ctxlog.Errorfx(ctx, "control server listener: %w", err) 83 | } 84 | return nil 85 | }) 86 | 87 | return nil 88 | } 89 | 90 | func absPathToBinary(cwd, b string) string { 91 | // This function assumes that cwd is absolute. It returns absolute path to executable file. 92 | // case: is it absolute 93 | if path.IsAbs(b) { 94 | return b 95 | } 96 | // case: is it regular binary, accessible via $path 97 | if !strings.Contains(b, string(os.PathSeparator)) { 98 | x, err := exec.LookPath(b) 99 | if err == nil { 100 | return x 101 | } 102 | } 103 | return path.Join(cwd, b) 104 | } 105 | 106 | type BotConfig struct { 107 | ControlAddr string 108 | Token string 109 | Script string 110 | LongRunningScript string 111 | ConfigFileDir string 112 | } 113 | 114 | func Application(rootCtx context.Context, bots map[string]BotConfig, tgAPIOrigin, build string) error { 115 | if len(bots) == 0 { 116 | return ctxlog.Errorfx(rootCtx, "there is no configuration") 117 | } 118 | eg, ctx := errgroup.WithContext(rootCtx) 119 | for name, cfg := range bots { 120 | err := bot(xlog.Bot(ctx, name), eg, cfg, tgAPIOrigin, build) 121 | if err != nil { 122 | return err 123 | } 124 | } 125 | xlog.L(ctx, "Run. Build="+build) 126 | return eg.Wait() 127 | } 128 | -------------------------------------------------------------------------------- /pkg/xjson/json.go: -------------------------------------------------------------------------------- 1 | package xjson 2 | 3 | import ( 4 | "fmt" 5 | "sort" 6 | "strconv" 7 | "strings" 8 | ) 9 | 10 | func jsonToEnv(pfx string, x any) ([]string, error) { 11 | switch e := x.(type) { 12 | case nil: 13 | return nil, nil 14 | case bool: 15 | return []string{pfx + "=" + strconv.FormatBool(e)}, nil 16 | case float64: 17 | return []string{pfx + "=" + strconv.FormatFloat(e, 'f', -1, 64)}, nil 18 | case string: 19 | return []string{pfx + "=" + e}, nil 20 | case []any: 21 | l := len(e) 22 | res := make([]string, 0, l+1) // we cannot predict final length, however it's not less than l+1 23 | lst := make([]string, 0, l) 24 | for i, v := range e { 25 | p := pfx + "_" + strconv.Itoa(i) 26 | t, err := jsonToEnv(p, v) 27 | if err != nil { 28 | return nil, err 29 | } 30 | if len(t) != 0 { 31 | lst = append(lst, p) 32 | res = append(res, t...) 33 | } 34 | } 35 | if len(res) > 0 { 36 | res = append(res, pfx+"="+strings.Join(lst, " ")) // append indexes 37 | } 38 | return res, nil 39 | case map[string]any: 40 | res := make([]string, 0, len(e)) // the same case as above 41 | for k, v := range e { 42 | t, err := jsonToEnv(pfx+"_"+k, v) 43 | if err != nil { 44 | return nil, err 45 | } 46 | res = append(res, t...) 47 | } 48 | return res, nil 49 | } 50 | return nil, fmt.Errorf("invalid type [pfx=%s]: %T", pfx, x) 51 | } 52 | 53 | func JSONToEnv(x any) ([]string, error) { 54 | a, err := jsonToEnv("tg", x) 55 | if err != nil { 56 | return nil, err 57 | } 58 | sort.Strings(a) // to be reproducible 59 | return a, nil 60 | } 61 | 62 | func Slice(x any, k ...string) ([]any, error) { 63 | v, ok, err := Any(x, k...) 64 | if err != nil { 65 | return nil, err 66 | } 67 | if !ok { 68 | return nil, fmt.Errorf("key not found: %#v", k) 69 | } 70 | b, ok := v.([]any) 71 | if !ok { 72 | return nil, fmt.Errorf("invalid type of value: %T", v) 73 | } 74 | return b, nil 75 | } 76 | 77 | func String(x any, k ...string) (string, error) { 78 | v, ok, err := Any(x, k...) 79 | if err != nil { 80 | return "", err 81 | } 82 | if !ok { 83 | return "", fmt.Errorf("key not found: %#v", k) 84 | } 85 | s, ok := v.(string) 86 | if !ok { 87 | return "", fmt.Errorf("invalid type of value: %T", v) 88 | } 89 | return s, nil 90 | } 91 | 92 | func Int(x any, k ...string) (int64, error) { 93 | v, err := float(x, k...) 94 | if err != nil { 95 | return 0, err 96 | } 97 | return int64(v), nil 98 | } 99 | 100 | func float(x any, k ...string) (float64, error) { 101 | v, ok, err := Any(x, k...) 102 | if err != nil { 103 | return 0, err 104 | } 105 | if !ok { 106 | return 0, fmt.Errorf("key not found: %#v", k) 107 | } 108 | b, ok := v.(float64) 109 | if !ok { 110 | return 0, fmt.Errorf("invalid type of value: %T", v) 111 | } 112 | return b, nil 113 | } 114 | 115 | func Bool(x any, k ...string) (bool, error) { 116 | v, ok, err := Any(x, k...) 117 | if err != nil { 118 | return false, err 119 | } 120 | if !ok { 121 | return false, fmt.Errorf("key not found: %#v", k) 122 | } 123 | b, ok := v.(bool) 124 | if !ok { 125 | return false, fmt.Errorf("invalid type of value: %T", v) 126 | } 127 | return b, nil 128 | } 129 | 130 | func Any(x any, k ...string) (any, bool, error) { 131 | if len(k) == 0 { 132 | panic("no key") // invalid function usage; panic is reasonable 133 | } 134 | kv, ok := x.(map[string]any) 135 | if !ok { 136 | return nil, false, fmt.Errorf("invalid type of x: %T", x) 137 | } 138 | v, ok := kv[k[0]] 139 | if !ok { 140 | return nil, false, nil 141 | } 142 | if len(k) == 1 { 143 | return v, true, nil 144 | } 145 | return Any(v, k[1:]...) 146 | } 147 | -------------------------------------------------------------------------------- /pkg/xproc/proc.go: -------------------------------------------------------------------------------- 1 | package xproc 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "fmt" 7 | "os" 8 | "os/exec" 9 | "path/filepath" 10 | "strings" 11 | "syscall" 12 | "time" 13 | 14 | "github.com/michurin/cnbot/pkg/ctxlog" 15 | "github.com/michurin/cnbot/pkg/xlog" 16 | ) 17 | 18 | type Cmd struct { 19 | InterruptDelay time.Duration 20 | KillDelay time.Duration 21 | Env []string 22 | Command string // must be absolute path to executable 23 | } 24 | 25 | func killGrp(ctx context.Context, pid int, sig syscall.Signal) { 26 | // We do not consider error as critical because the process could 27 | // disappear by its own. It is not easy to identify error in this case. 28 | // For example you can get ESRCH (0x3) that doesn't support by syscall.Errno.Is(). 29 | pgid, err := syscall.Getpgid(pid) // not cmd.SysProcAttr.Pgid 30 | if err != nil { 31 | xlog.L(ctx, fmt.Errorf("kill: getpgid: %w", err)) 32 | return 33 | } 34 | err = syscall.Kill(-pgid, sig) // minus 35 | if err != nil { 36 | xlog.L(ctx, fmt.Errorf("kill: kill %d: %w", -pgid, err)) 37 | return 38 | } 39 | } 40 | 41 | func (c *Cmd) Run( 42 | ctx context.Context, // Note: don't use ctx for timeouts 43 | args []string, 44 | env []string, 45 | ) ( 46 | []byte, 47 | error, 48 | ) { 49 | // setup cmd 50 | command := c.Command 51 | cmd := exec.Command(command, args...) //nolint:noctx // we don't use CommandContext here because it kills only process, not group 52 | cmd.SysProcAttr = &syscall.SysProcAttr{ 53 | Setpgid: true, 54 | } 55 | cmd.Dir = filepath.Dir(command) 56 | cmd.Env = append(env, c.Env...) //nolint:gocritic // looks like false positive 57 | var outBuffer bytes.Buffer 58 | cmd.Stdout = &outBuffer 59 | var errBuffer bytes.Buffer 60 | cmd.Stderr = &errBuffer 61 | 62 | xlog.L(ctx, fmt.Sprintf("starting %s %v", cmd.Path, args)) // TODO put command to context? 63 | 64 | err := cmd.Start() // start command synchronously 65 | if err != nil { 66 | wd, e := os.Getwd() 67 | if e != nil { 68 | wd = e.Error() 69 | } 70 | return nil, ctxlog.Errorfx(ctx, "start (PATH=%s; CWD=%s; cmd.Dir=%s): %w", os.Getenv("PATH"), wd, cmd.Dir, err) 71 | } 72 | ctx = xlog.Pid(ctx, cmd.Process.Pid) 73 | 74 | done := make(chan struct{}) 75 | intBound := time.NewTimer(c.InterruptDelay) 76 | killBound := time.NewTimer(c.InterruptDelay + c.KillDelay) 77 | defer func() { 78 | intBound.Stop() 79 | killBound.Stop() 80 | close(done) 81 | }() 82 | go func() { 83 | for { 84 | select { 85 | case <-done: // it has to appear before kill sections to catch stat errors 86 | return 87 | case <-ctx.Done(): // urgent exit, we doesn't even wait for process finalization 88 | xlog.L(ctx, "Exec terminated by context") 89 | killGrp(ctx, cmd.Process.Pid, syscall.SIGKILL) 90 | return 91 | case <-intBound.C: 92 | killGrp(ctx, cmd.Process.Pid, syscall.SIGINT) // Not all OS support SIGTERM 93 | case <-killBound.C: 94 | killGrp(ctx, cmd.Process.Pid, syscall.SIGKILL) 95 | } 96 | } 97 | }() 98 | err = cmd.Wait() 99 | if err != nil { 100 | return nil, ctxlog.Errorfx(ctx, "wait: %w", err) 101 | } 102 | 103 | errMsg := []string(nil) 104 | exitCode := cmd.ProcessState.ExitCode() 105 | if exitCode != 0 { 106 | errMsg = append(errMsg, fmt.Sprintf("exit code: %d", exitCode)) 107 | } 108 | errStr := errBuffer.String() 109 | if errStr != "" { 110 | xlog.L(ctx, fmt.Errorf("stderr: %s", errStr)) // TODO consider as error? 111 | // errMsg = append(errMsg, fmt.Sprintf("stderr: %q", errStr)) 112 | } 113 | outBytes := outBuffer.Bytes() 114 | if errMsg == nil { 115 | return outBytes, nil 116 | } 117 | errMsg = append(errMsg, fmt.Sprintf("stdout: %q", string(outBytes))) // TODO trim? 118 | return nil, ctxlog.Errorfx(ctx, "%s", strings.Join(errMsg, "; ")) 119 | } 120 | -------------------------------------------------------------------------------- /pkg/ctxlog/examples_test.go: -------------------------------------------------------------------------------- 1 | package ctxlog_test 2 | 3 | import ( 4 | "context" 5 | "log/slog" 6 | "os" 7 | 8 | "github.com/michurin/cnbot/pkg/ctxlog" 9 | ) 10 | 11 | const thisFileName = "ctxlog/examples_test.go" 12 | 13 | var optsNoTimeNoSourceNoLevel = slog.HandlerOptions{ 14 | AddSource: false, 15 | Level: nil, 16 | ReplaceAttr: func(_ []string, a slog.Attr) slog.Attr { // remove time; just to be reproducible 17 | if a.Key == slog.TimeKey { 18 | return slog.Attr{} 19 | } 20 | return a 21 | }, 22 | } 23 | 24 | func ExampleHandler_usualUsecase() { 25 | // Somewhere you create handler. 26 | 27 | baseHandler := slog.Handler(slog.NewTextHandler(os.Stdout, &optsNoTimeNoSourceNoLevel)) 28 | 29 | // You can setup custom attrs for handler. Our wrapper won't manage that attrs. 30 | 31 | baseHandler = baseHandler.WithAttrs([]slog.Attr{slog.Any("app", "one")}) 32 | 33 | // Now you are able to setup global logger. You can setup lib-wide or application-wide logger using slog.SetDefault() 34 | 35 | log := slog.New(ctxlog.Handler(baseHandler, thisFileName)) 36 | 37 | // You may have a chain of calls in you apps, let's say next two funcs. 38 | 39 | funcContextFreeLogic := func() error { 40 | return ctxlog.Errorf("initial error") 41 | } 42 | 43 | funcClient := func(ctx context.Context, arg int) error { 44 | ctx = ctxlog.Add(ctx, "client", "clientLabel", "arg", arg) 45 | err := funcContextFreeLogic() 46 | if err != nil { 47 | return ctxlog.Errorfx(ctx, "client error: %w", err) 48 | } 49 | return nil 50 | } 51 | 52 | funcHandler := func(ctx context.Context, input int) error { 53 | ctx = ctxlog.Add(ctx, "component", "handlerLabel") 54 | err := funcClient(ctx, input) 55 | if err != nil { 56 | return ctxlog.Errorfx(ctx, "handler failure: %w", err) 57 | } 58 | return nil 59 | } 60 | 61 | // You instrumentation is able to setup context and call the chain 62 | 63 | ctx := context.Background() 64 | 65 | ctx = ctxlog.Add(ctx, "request_id", "deadbeef") 66 | 67 | err := funcHandler(ctx, -1) // -1 will cause error 68 | if err != nil { 69 | log.Error("Error", "key_does_not_matter", err) // key doesn't matter as long as error is wrapped; handler does magic with err 70 | } 71 | 72 | // output: 73 | // level=ERROR msg=Error app=one source=ctxlog/examples_test.go:69 err_source=ctxlog/examples_test.go:40 err_msg="handler failure: client error: initial error" request_id=deadbeef component=handlerLabel client=clientLabel arg=-1 74 | } 75 | 76 | func ExampleHandler_howGroupsAndAttrsDoing() { 77 | baseHandler := slog.Handler(slog.NewTextHandler(os.Stdout, &optsNoTimeNoSourceNoLevel)) 78 | 79 | log := slog.New(ctxlog.Handler(baseHandler, thisFileName)) 80 | log.Info("Message") 81 | log.Info("Message-inline-attrs", "P", "Q") 82 | log.InfoContext(ctxlog.Add(context.Background(), "V", "W"), "Message-1-ctx-attrs") 83 | log = log.With("X", "Y") 84 | log.Info("Message-with-attrs") 85 | log = log.WithGroup("G") 86 | log.Info("Message-with-group") 87 | 88 | // output: 89 | // level=INFO msg=Message source=ctxlog/examples_test.go:80 90 | // level=INFO msg=Message-inline-attrs source=ctxlog/examples_test.go:81 P=Q 91 | // level=INFO msg=Message-1-ctx-attrs source=ctxlog/examples_test.go:82 V=W 92 | // level=INFO msg=Message-with-attrs X=Y source=ctxlog/examples_test.go:84 93 | // level=INFO msg=Message-with-group X=Y G.source=ctxlog/examples_test.go:86 94 | } 95 | 96 | func ExampleHandler_indirectContextEnrichment() { 97 | baseHandler := slog.Handler(slog.NewTextHandler(os.Stdout, &optsNoTimeNoSourceNoLevel)) 98 | 99 | log := slog.New(ctxlog.Handler(baseHandler, thisFileName)) 100 | 101 | // we populate context somewhere, for example in main.go; we are setting up high level context of logging 102 | ctx := ctxlog.Add(context.Background(), "component", "A") 103 | 104 | // now we want to create handler or factory, that will be used somewhere else and has to use attrs from this base context 105 | handler := func(patch ctxlog.PatchAttrs) func(ctx context.Context) { // return handler func 106 | return func(ctx context.Context) { // ctx here is a handle time context 107 | ctx = ctxlog.ApplyPatch(ctx, patch) // populate ctx 108 | log.InfoContext(ctx, "OK") // here both handle time attr and factory initialization time attrs show up 109 | } 110 | }(ctxlog.Patch(ctx)) 111 | 112 | // somewhere our handler is called with some low level context 113 | handlerContext := ctxlog.Add(context.Background(), "request_id", 99) 114 | handler(handlerContext) 115 | 116 | // output: 117 | // level=INFO msg=OK source=ctxlog/examples_test.go:108 request_id=99 component=A 118 | } 119 | -------------------------------------------------------------------------------- /pkg/app/cfg.go: -------------------------------------------------------------------------------- 1 | package app 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "os" 7 | "path" 8 | "sort" 9 | "strings" 10 | 11 | "github.com/michurin/systemd-env-file/sdenv" 12 | 13 | "github.com/michurin/cnbot/pkg/ctxlog" 14 | "github.com/michurin/cnbot/pkg/xlog" 15 | ) 16 | 17 | const ( 18 | varPrefix = "tb_" 19 | varPrefixLen = len(varPrefix) 20 | ) 21 | 22 | var varSfxs = []string{ 23 | "_ctrl_addr", 24 | "_token", 25 | "_config_dir", 26 | "_long_running_script", // order matters; to be greedy longer has to be first 27 | "_script", 28 | } 29 | 30 | func LoadConfigs(files ...string) (map[string]BotConfig, string, error) { 31 | ctx := xlog.Comp(context.Background(), "cfg") 32 | 33 | // init collection of variables by system environment 34 | 35 | envs := sdenv.NewCollectsion() 36 | envs.PushStd(os.Environ()) // so environment variables has maximum priority 37 | 38 | // enrich variables from files 39 | 40 | for _, file := range files { 41 | data, err := os.ReadFile(file) 42 | if err != nil { 43 | return nil, "", ctxlog.Errorfx(ctx, "reading: %s: %w", file, err) 44 | } 45 | pairs, err := sdenv.Parser(data) 46 | if err != nil { 47 | return nil, "", ctxlog.Errorfx(ctx, "parser: %s: %w", file, err) 48 | } 49 | envs.Push(pairs) 50 | envs.Push(configFileDirForEachSetup(file, pairs)) // add configuration file information with minimum priority 51 | } 52 | 53 | // sort to be reproducible 54 | 55 | ev := envs.CollectionStd() 56 | sort.Strings(ev) // we have to sort in original case, before tolowering 57 | 58 | // consider all 59 | 60 | c, err := buildConfigs(ev) 61 | if err != nil { 62 | return nil, "", ctxlog.Errorfx(ctx, "building configs: %w", err) 63 | } 64 | 65 | return c, buildOrigin(ev), nil 66 | } 67 | 68 | func buildOrigin(pairs []string) string { 69 | for _, pair := range pairs { 70 | ek, ev, ok := strings.Cut(pair, "=") 71 | if !ok { 72 | continue 73 | } 74 | if len(ek) <= varPrefixLen { // skip: it cannot be configuration variable 75 | continue 76 | } 77 | if strings.ToLower(ek[varPrefixLen:]) == "api_origin" { 78 | return ev 79 | } 80 | } 81 | return "https://api.telegram.org" 82 | } 83 | 84 | func buildConfigs(pairs []string) (map[string]BotConfig, error) { 85 | configMaps := map[string][5]string{} // TODO: 5 hardcoded 86 | // TODO minor code duplication in this loop 87 | for _, pair := range pairs { 88 | ek, ev, ok := strings.Cut(pair, "=") 89 | if !ok { 90 | continue 91 | } 92 | if len(ek) <= varPrefixLen { // skip: it cannot be configuration variable 93 | continue 94 | } 95 | ekLower := strings.ToLower(ek) 96 | if !strings.HasPrefix(ekLower, varPrefix) { // skip: is not a configuration variable due to prefix 97 | continue 98 | } 99 | for i, sfx := range varSfxs { 100 | if strings.HasSuffix(ekLower, sfx) { 101 | k := "default" 102 | if len(ek) > varPrefixLen+len(sfx) { 103 | k = ekLower[varPrefixLen : len(ek)-len(sfx)] 104 | } 105 | v := configMaps[k] 106 | v[i] = ev 107 | configMaps[k] = v 108 | break 109 | } 110 | } 111 | } 112 | res := map[string]BotConfig{} 113 | for k, v := range configMaps { 114 | token := v[1] 115 | if token == "" { 116 | return nil, fmt.Errorf("bot %[1]s: token is empty, you must specify tb_%[1]s_token", k) 117 | } 118 | if strings.HasPrefix(token, "@") { 119 | x, err := os.ReadFile(token[1:]) // TODO consider relative paths; it has to share logic with runner 120 | if err != nil { 121 | xlog.L(context.TODO(), fmt.Errorf("skipping bot name %q: cannot get token from file: %q: %w", k, token, err)) 122 | continue 123 | } 124 | token = strings.TrimSpace(string(x)) 125 | } 126 | res[k] = BotConfig{ 127 | ControlAddr: v[0], 128 | Token: token, 129 | Script: v[4], 130 | LongRunningScript: v[3], 131 | ConfigFileDir: v[2], 132 | } 133 | } 134 | return res, nil 135 | } 136 | 137 | func configFileDirForEachSetup(cfgFile string, pairs [][2]string) [][2]string { 138 | dir := path.Dir(cfgFile) 139 | res := [][2]string(nil) 140 | for _, pair := range pairs { 141 | ek, ev := pair[0], pair[1] 142 | if len(ev) <= varPrefixLen { // skip: it cannot be configuration variable 143 | continue 144 | } 145 | ekLower := strings.ToLower(ek) 146 | if strings.HasPrefix(ekLower, varPrefix) { // skip: is not a configuration variable due to prefix 147 | continue 148 | } 149 | for _, sfx := range varSfxs { 150 | if strings.HasSuffix(ekLower, sfx) { 151 | k := "default" 152 | if len(ek) > varPrefixLen+len(sfx) { 153 | k = ekLower[varPrefixLen : len(ek)-len(sfx)] 154 | } 155 | res = append(res, [2]string{varPrefix + k + "_config_dir", dir}) 156 | break // end on first matching 157 | } 158 | } 159 | } 160 | return res 161 | } 162 | -------------------------------------------------------------------------------- /pkg/xctrl/ctrl.go: -------------------------------------------------------------------------------- 1 | package xctrl 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | "io" 8 | "mime" 9 | "net/http" 10 | "strconv" 11 | "strings" 12 | 13 | "github.com/michurin/cnbot/pkg/ctxlog" 14 | "github.com/michurin/cnbot/pkg/xbot" 15 | "github.com/michurin/cnbot/pkg/xjson" 16 | "github.com/michurin/cnbot/pkg/xlog" 17 | "github.com/michurin/cnbot/pkg/xproc" 18 | ) 19 | 20 | //nolint:nestif // reason to refactor 21 | func Handler(bot *xbot.Bot, cmd *xproc.Cmd, loggingPatch ctxlog.PatchAttrs) http.HandlerFunc { //nolint:gocognit,cyclop,funlen // reason to refactor 22 | return func(w http.ResponseWriter, r *http.Request) { 23 | ctx := ctxlog.ApplyPatch(r.Context(), loggingPatch) 24 | // TODO mark ctx for logging? 25 | // TODO put http method to ctx 26 | // TODO put http content-type to ctx 27 | body, err := io.ReadAll(r.Body) 28 | if err != nil { 29 | xlog.L(ctx, fmt.Errorf("body reading: %w", err)) 30 | } 31 | method := r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:] 32 | data := []byte(nil) 33 | switch r.Method { 34 | case http.MethodGet: 35 | fileID := r.URL.Query().Get("file_id") 36 | if fileID == "" { 37 | data, err = bot.API(ctx, &xbot.Request{Method: method}) 38 | } else { 39 | req, err := xbot.RequestStruct("getFile", map[string]string{"file_id": fileID}) 40 | if err != nil { 41 | xlog.L(ctx, err) // TODO response! 42 | return 43 | } 44 | x, err := bot.API(ctx, req) 45 | if err != nil { 46 | xlog.L(ctx, err) // TODO response! 47 | return 48 | } 49 | xlog.L(ctx, x) // TODO!!!!!! 50 | s := any(nil) 51 | err = json.Unmarshal(x, &s) 52 | if err != nil { 53 | xlog.L(ctx, err) // TODO response! 54 | return 55 | } 56 | ok, err := xjson.Bool(s, "ok") 57 | if err != nil { 58 | xlog.L(ctx, err) // TODO response! 59 | return 60 | } 61 | filePath, err := xjson.String(s, "result", "file_path") 62 | if err != nil { 63 | xlog.L(ctx, err) // TODO response! 64 | return 65 | } 66 | xlog.L(ctx, fmt.Sprintf("%s %t %s", "ok/filePath", ok, filePath)) // TODO remove 67 | w.WriteHeader(http.StatusOK) 68 | err = bot.Download(ctx, filePath, w) 69 | if err != nil { 70 | xlog.L(ctx, err) // TODO response! 71 | return 72 | } 73 | return 74 | } 75 | case http.MethodPost: 76 | ct := r.Header.Get("Content-Type") 77 | sct, _, err := mime.ParseMediaType(ct) 78 | if err != nil { 79 | xlog.L(ctx, err) // TODO response! 80 | return 81 | } 82 | if sct == "application/json" || sct == "multipart/form-data" { 83 | data, err = bot.API(ctx, &xbot.Request{ 84 | Method: method, 85 | ContentType: ct, 86 | Body: body, 87 | }) 88 | if err != nil { 89 | xlog.L(ctx, err) // TODO response! 90 | return 91 | } 92 | } else { 93 | var to int64 // TODO refactor 94 | var req *xbot.Request // TODO refactor 95 | to, err = strconv.ParseInt(r.URL.Query().Get("to"), 10, 64) 96 | if err != nil { 97 | xlog.L(ctx, err) // TODO response! 98 | return 99 | } 100 | // TODO add `to` to log context 101 | req, err = xbot.RequestFromBinary(body, to) //nolint:contextcheck 102 | if err != nil { 103 | xlog.L(ctx, err) // TODO response! 104 | return 105 | } 106 | data, err = bot.API(ctx, req) 107 | if err != nil { 108 | xlog.L(ctx, err) // TODO response! 109 | return 110 | } 111 | } 112 | case "RUN": 113 | q := r.URL.Query() 114 | to, err := strconv.ParseInt(r.URL.Query().Get("to"), 10, 64) 115 | if err != nil { 116 | xlog.L(ctx, err) // TODO response! 117 | return 118 | } 119 | ctx := xlog.User(ctx, to) 120 | logCtxPatch := ctxlog.Patch(ctx) 121 | go func() { //nolint:contextcheck // TODO: limit concurrency 122 | ctx := ctxlog.ApplyPatch(context.Background(), logCtxPatch) 123 | // TODO refactor. it is similar to processMessage 124 | body, err := cmd.Run(ctx, q["a"], []string{"tg_x_to=" + strconv.FormatInt(to, 10)}) 125 | if err != nil { 126 | xlog.L(ctx, err) 127 | return 128 | } 129 | req, err := xbot.RequestFromBinary(body, to) 130 | if err != nil { 131 | xlog.L(ctx, err) 132 | return 133 | } 134 | if req == nil { // TODO hmm... it happens? 135 | xlog.L(ctx, "Script response skipped") 136 | return 137 | } 138 | _, err = bot.API(ctx, req) // TODO check body? 139 | if err != nil { 140 | xlog.L(ctx, err) 141 | return 142 | } 143 | }() 144 | return 145 | default: 146 | xlog.L(ctx, fmt.Errorf("method not allowed: %q", r.Method)) 147 | w.WriteHeader(http.StatusMethodNotAllowed) 148 | return 149 | } 150 | if err != nil { 151 | xlog.L(ctx, err) 152 | return 153 | } 154 | w.WriteHeader(http.StatusOK) 155 | // TODO consider `silent=true` parameter and skip writing if present 156 | _, err = w.Write(data) // TODO consider error 157 | if err != nil { 158 | xlog.L(ctx, err) 159 | return 160 | } 161 | } 162 | } 163 | -------------------------------------------------------------------------------- /pkg/xbot/request.go: -------------------------------------------------------------------------------- 1 | package xbot 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "encoding/json" 7 | "fmt" 8 | "mime" 9 | "mime/multipart" 10 | "net/http" 11 | "net/textproto" 12 | "strconv" 13 | "strings" 14 | "unicode/utf16" 15 | "unicode/utf8" 16 | 17 | "github.com/michurin/cnbot/pkg/ctxlog" 18 | "github.com/michurin/cnbot/pkg/xlog" 19 | ) 20 | 21 | type Request struct { 22 | Method string 23 | ContentType string 24 | Body []byte 25 | } 26 | 27 | func RequestStruct(method string, x any) (*Request, error) { 28 | d, err := json.Marshal(x) 29 | if err != nil { 30 | return nil, err 31 | } 32 | return &Request{ 33 | Method: method, 34 | ContentType: "application/json", 35 | Body: d, 36 | }, nil 37 | } 38 | 39 | var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"") 40 | 41 | func RequestFromBinary(data []byte, userID int64) (*Request, error) { 42 | contentType := http.DetectContentType(data) 43 | switch { 44 | case strings.HasPrefix(contentType, "text/"): 45 | if !utf8.Valid(data) { 46 | return nil, ctxlog.Errorf("invalid utf8") 47 | } 48 | if bytes.HasPrefix(data, []byte("%!PRE\n")) { 49 | croped := bytes.TrimSpace(data[6:]) // 6 is len of prefix 50 | if len(croped) == 0 { 51 | return RequestStruct("sendMessage", map[string]any{ 52 | "chat_id": userID, 53 | "text": "-", // Forcing empty preformated text to "-" 54 | "entities": []any{map[string]any{"type": "pre", "offset": 0, "length": 1}}, 55 | }) 56 | } 57 | str := string(croped) 58 | r := []rune(str) 59 | if len(r) > 4096 { // In this place length is in terms of Unicode chars 60 | return reqMultipart("sendDocument", userID, "document", croped, "formatted_text", "text/plain") // TODO nice to add caption... with caption_entities? 61 | } 62 | return RequestStruct("sendMessage", map[string]any{ 63 | "chat_id": userID, 64 | "text": str, 65 | "entities": []any{ 66 | map[string]any{ 67 | "type": "pre", 68 | "offset": 0, 69 | "length": len(utf16.Encode(r)), // According Telegram's API, in this place the length is in terms of UTF-16 70 | }, 71 | }, 72 | }) 73 | } 74 | data = bytes.TrimSpace(data) 75 | if len(data) == 0 { 76 | return nil, ctxlog.Errorf("zero length data: skipping response") 77 | } 78 | str := string(data) 79 | r := []rune(str) 80 | if len(r) > 4096 { // In this place length is in terms of Unicode chars 81 | return reqMultipart("sendDocument", userID, "document", data, "message", "text/plain") // TODO nice to add caption 82 | } 83 | return RequestStruct("sendMessage", map[string]any{"chat_id": userID, "text": str}) 84 | case strings.HasPrefix(contentType, "image/"): // TODO to limit image formats 85 | return reqMultipart("sendPhoto", userID, "photo", data, "image", contentType) 86 | case strings.HasPrefix(contentType, "video/"): // TODO to limit video formats 87 | return reqMultipart("sendVideo", userID, "video", data, "video", contentType) 88 | case strings.HasPrefix(contentType, "audio/"): // it seems application/ogg is not fully supported; it requires OPUS encoding 89 | return reqMultipart("sendAudio", userID, "audio", data, "audio", contentType) 90 | default: // TODO hmm... application/* and font/* 91 | xlog.L(context.TODO(), fmt.Sprintf("Fallback to multipart from %q", contentType)) // TODO no context here 92 | return reqMultipart("sendDocument", userID, "document", data, "document", contentType) 93 | } 94 | } 95 | 96 | func fext(ctype string) string { 97 | // mime.ExtensionsByType can return several extension sorted alphabetical. 98 | // We are trying to find most common extension, by comparing with last 99 | // part of mime type of data. 100 | prefExt, _, _ := mime.ParseMediaType(ctype) 101 | idx := strings.LastIndex(prefExt, "/") 102 | if idx >= 0 { 103 | prefExt = "." + prefExt[idx+1:] 104 | } 105 | exts, _ := mime.ExtensionsByType(ctype) 106 | if len(exts) == 0 { 107 | return ".dat" 108 | } 109 | for _, e := range exts { // find preferable extension, if any 110 | if e == prefExt { 111 | return e 112 | } 113 | } 114 | e := exts[0] 115 | if e == ".asc" { 116 | return ".txt" // looks more reasonable for text/plain 117 | } 118 | return e 119 | } 120 | 121 | func reqMultipart(method string, to int64, fieldname string, data []byte, filename string, ctype string) (*Request, error) { // TODO legacy 122 | var body bytes.Buffer 123 | w := multipart.NewWriter(&body) 124 | err := w.WriteField("chat_id", strconv.FormatInt(to, 10)) 125 | if err != nil { 126 | return nil, err 127 | } 128 | fw, err := w.CreatePart(textproto.MIMEHeader{ // see implementation of CreateFormFile 129 | "Content-Disposition": {fmt.Sprintf( 130 | `form-data; name="%s"; filename="%s"`, 131 | quoteEscaper.Replace(fieldname), 132 | quoteEscaper.Replace(filename+fext(ctype)), 133 | )}, 134 | "Content-Type": {ctype}, 135 | }) 136 | if err != nil { 137 | return nil, err 138 | } 139 | _, err = fw.Write(data) 140 | if err != nil { 141 | return nil, err 142 | } 143 | err = w.Close() 144 | if err != nil { 145 | return nil, err 146 | } 147 | return &Request{ 148 | Method: method, 149 | ContentType: w.FormDataContentType(), 150 | Body: body.Bytes(), 151 | }, nil 152 | } 153 | -------------------------------------------------------------------------------- /pkg/xloop/loop.go: -------------------------------------------------------------------------------- 1 | package xloop 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "strings" 7 | "unicode/utf8" 8 | 9 | "github.com/michurin/cnbot/pkg/ctxlog" 10 | "github.com/michurin/cnbot/pkg/xbot" 11 | "github.com/michurin/cnbot/pkg/xjson" 12 | "github.com/michurin/cnbot/pkg/xlog" 13 | "github.com/michurin/cnbot/pkg/xproc" 14 | ) 15 | 16 | var consideringMessageTypes = []string{ // TODO it has to be tunable 17 | "callback_query", // this strings are using in two places: 18 | "inline_query", // in getUpdate and 19 | "message", // in parsing function 20 | "message_reaction", // we assume we can get userID from any types using the same way. I'm not sure it works 21 | "poll", 22 | "poll_answer", 23 | } 24 | 25 | func Loop(ctx context.Context, bot *xbot.Bot, command *xproc.Cmd) error { 26 | offset := int64(0) 27 | for { 28 | result, err := getUpdates(ctx, bot, offset) 29 | if err != nil { 30 | return err 31 | } 32 | if len(result) == 0 { // we won't change offset if there is no new messages 33 | continue 34 | } 35 | offset = 0 // offset can be dropped 36 | for _, m := range result { 37 | u, err := xjson.Int(m, "update_id") 38 | if err != nil { 39 | return err 40 | } 41 | if u > offset { 42 | offset = u 43 | } 44 | // TODO refactor: get env, get args, run command 45 | req, err := processMessage(ctx, m, command) 46 | if err != nil { 47 | xlog.L(ctx, ctxlog.Errorf("skip message: %w", err)) 48 | continue 49 | } 50 | if req == nil { 51 | continue 52 | } 53 | _, err = bot.API(ctx, req) 54 | if err != nil { 55 | xlog.L(ctx, err) // TODO process error 56 | } 57 | } 58 | offset++ 59 | } 60 | } 61 | 62 | func getUpdates(ctx context.Context, bot *xbot.Bot, offset int64) ([]any, error) { 63 | req, err := xbot.RequestStruct("getUpdates", map[string]any{ 64 | "offset": offset, 65 | "timeout": 30, 66 | "allowed_updates": consideringMessageTypes, 67 | }) 68 | if err != nil { 69 | return nil, ctxlog.Errorfx(ctx, "cannot build request") 70 | } 71 | bytes, err := bot.API(ctx, req) 72 | if err != nil { 73 | return nil, ctxlog.Errorfx(ctx, "api: %w", err) // TODO all returns are too hard? 74 | } 75 | data := any(nil) 76 | err = json.Unmarshal(bytes, &data) 77 | if err != nil { 78 | return nil, ctxlog.Errorfx(ctx, "unmarshal: %w", err) 79 | } 80 | ok, err := xjson.Bool(data, "ok") // TODO xjson.True()? 81 | if err != nil { 82 | return nil, err 83 | } 84 | if !ok { 85 | return nil, ctxlog.Errorf("response is not ok: %#v", data) 86 | } 87 | result, err := xjson.Slice(data, "result") 88 | if err != nil { 89 | return nil, err 90 | } 91 | return result, nil 92 | } 93 | 94 | func userID(m any) (int64, error) { // TODO consider all types 95 | for _, bodyKey := range consideringMessageTypes { // TODO we are thinking all messages has the same structure related userID 96 | body, ok, err := xjson.Any(m, bodyKey) 97 | if err != nil { 98 | return 0, err // TODO wrap, mention k in err message 99 | } 100 | if ok { 101 | path := []string{"from", "id"} 102 | if bodyKey == "message_reaction" { // hakish 103 | path = []string{"user", "id"} 104 | } 105 | userID, err := xjson.Int(body, path...) 106 | if err != nil { 107 | return 0, ctxlog.Errorf("user not found: key=%s, path=%v: %w", bodyKey, path, err) 108 | } 109 | return userID, nil 110 | } 111 | } 112 | return 0, ctxlog.Errorf("body not found: consider %v", consideringMessageTypes) 113 | } 114 | 115 | func userText(m any) (string, error) { // TODO consider all types 116 | for _, bodyKey := range consideringMessageTypes { // TODO we are thinking all messages has the same structure related userID 117 | body, ok, err := xjson.Any(m, bodyKey) 118 | if err != nil { 119 | return "", err // TODO wrap, mention k in err message 120 | } 121 | if ok { 122 | if bodyKey == "message" { // TODO very hakish 123 | x, e := xjson.String(body, "text") 124 | if e != nil { 125 | x, e = xjson.String(body, "caption") 126 | } 127 | return x, e 128 | } 129 | if bodyKey == "callback_query" { 130 | return xjson.String(body, "data") 131 | } 132 | return bodyKey, nil 133 | } 134 | } 135 | return "", ctxlog.Errorf("body not found: consider %v", consideringMessageTypes) 136 | } 137 | 138 | func processMessage(ctx context.Context, m any, command *xproc.Cmd) (*xbot.Request, error) { 139 | userID, err := userID(m) 140 | if err != nil { 141 | return nil, ctxlog.Errorfx(ctx, "no user id: %w", err) 142 | } 143 | ctx = xlog.User(ctx, userID) 144 | env, err := xjson.JSONToEnv(m) 145 | if err != nil { 146 | return nil, ctxlog.Errorfx(ctx, "cannot create env: %w", err) 147 | } 148 | text, err := userText(m) 149 | if err != nil { 150 | xlog.L(ctx, err) // return nil, err // TODO callback_query... 151 | } 152 | args := textToArgs(text) 153 | data, err := command.Run(ctx, args, env) 154 | if err != nil { 155 | return nil, err // already wrapped with context 156 | } 157 | req, err := xbot.RequestFromBinary(data, userID) //nolint:contextcheck 158 | if err != nil { 159 | return nil, ctxlog.Errorfx(ctx, "invalid data: %w", err) 160 | } 161 | if req == nil { // TODO hmm... it happens? 162 | return nil, ctxlog.Errorfx(ctx, "cannot prepare request (nil): %w", err) 163 | } 164 | return req, nil 165 | } 166 | 167 | // valid chars: 168 | // - %+,-.^_{}~ 169 | // - digits: 0123456789 170 | // - letters: a-zA-Z will be converted to lower case 171 | var asciiSpaceAndUnsafe = [256]uint8{ 172 | '\x00': 1, '\x01': 1, '\x02': 1, '\x03': 1, '\x04': 1, '\x05': 1, '\x06': 1, '\x07': 1, 173 | '\x08': 1, '\x09': 1, '\x0a': 1, '\x0b': 1, '\x0c': 1, '\x0d': 1, '\x0e': 1, '\x0f': 1, 174 | '\x10': 1, '\x11': 1, '\x12': 1, '\x13': 1, '\x14': 1, '\x15': 1, '\x16': 1, '\x17': 1, 175 | '\x18': 1, '\x19': 1, '\x1a': 1, '\x1b': 1, '\x1c': 1, '\x1d': 1, '\x1e': 1, '\x1f': 1, '\x20': 1, 176 | '"': 1, '\\': 1, '`': 1, '$': 1, // systemd SHELL_NEED_ESCAPE 177 | '*': 1, '?': 1, '[': 1, ']': 1, // systemd GLOB_CHARS 178 | '\'': 1, '(': 1, ')': 1, '<': 1, '>': 1, '|': 1, '&': 1, ';': 1, '!': 1, // systemd SHELL_NEED_QUOTES 179 | '/': 1, ':': 1, // path separators 180 | '=': 1, '#': 1, '@': 1, '+': 1, '-': 1, '.', // extra 181 | } 182 | 183 | func textToArgs(text string) []string { // TODO tests; move to package or file before? 184 | if !utf8.ValidString(text) { // just drop invalid strings 185 | return nil 186 | } 187 | a := strings.Fields(strings.ToLower(text)) 188 | b := make([]string, len(a)) 189 | for i, v := range a { 190 | r := make([]byte, 0, len(v)) 191 | for _, q := range []byte(v) { 192 | if asciiSpaceAndUnsafe[q] == 0 { 193 | r = append(r, q) 194 | } 195 | } 196 | b[i] = string(r) 197 | } 198 | return b 199 | } 200 | -------------------------------------------------------------------------------- /demo/bot.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | LOG=logs/log.log # /dev/null 4 | mkdir -p "$(dirname "$LOG")" # do not forget to create all necessary directories 5 | 6 | FROM="$tg_message_from_id" 7 | 8 | API() { 9 | API_STDOUT "$@" >>"$LOG" 10 | } 11 | 12 | API_STDOUT() { 13 | url="http://localhost$tg_x_ctrl_addr/$1" 14 | shift 15 | echo "====== curl $url $@" >>"$LOG" 16 | curl -qs "$url" "$@" 2>>"$LOG" 17 | echo >>"$LOG" 18 | echo >>"$LOG" 19 | } 20 | 21 | ( 22 | echo '===================' 23 | echo "Args: $@" 24 | echo "Environment:" 25 | env | grep tg_ | sort 26 | echo '...................' 27 | ) >>"$LOG" 28 | 29 | case "$1" in 30 | debug) 31 | echo '%!PRE' 32 | echo '' 33 | echo "📌 Arguments:" 34 | i=0 35 | for a in "$@" 36 | do 37 | i=$(($i+1)) 38 | echo "$i: $a" 39 | done 40 | echo '' 41 | echo "📌 Environment:" 42 | env | grep tg_ | sort 43 | echo '' 44 | echo "📌 Configuration:" 45 | echo "LOG=$LOG" 46 | echo 'working directory:' 47 | echo " $PWD" 48 | echo 'binary:' 49 | echo " $0" 50 | ;; 51 | about) 52 | echo '%!PRE' 53 | API_STDOUT getMe | jq 54 | ;; 55 | two) 56 | API "?to=$FROM" -d 'OK ONE!' 57 | API "?to=$FROM" -d 'OK TWO!!' 58 | echo 'OK (native response)' 59 | ;; 60 | buttons) 61 | bGoogle='{"text":"Google","url":"https://www.google.com/"}' 62 | bDuck='{"text":"DuckDuckGo","url":"https://duckduckgo.com/"}' 63 | API sendMessage \ 64 | -F chat_id=$FROM \ 65 | -F text='Select search engine' \ 66 | -F reply_markup='{"inline_keyboard":[['"$bGoogle,$bDuck"']]}' 67 | ;; 68 | image) 69 | curl -qs https://github.com/fluidicon.png 70 | ;; 71 | invert) 72 | wm=0 73 | fid='' 74 | for x in $tg_message_photo # finding the biggest image but ignoring too big ones 75 | do 76 | v=${x}_file_size 77 | s=${!v} # trick: getting variable name from variable; we need bash for it 78 | if test $s -gt 102400; then continue; fi # skipping too big files 79 | v=${x}_width 80 | w=${!v} 81 | v=${x}_file_id 82 | f=${!v} 83 | if test $w -gt $wm; then wm=$w; fid=$f; fi 84 | done 85 | if test -n "$fid" 86 | then 87 | API_STDOUT '' -G --data-urlencode "file_id=$fid" -o - | mogrify -flip -flop -format png - 88 | else 89 | echo "attache not found (maybe it was skipped due to enormous size)" 90 | fi 91 | ;; 92 | reaction) 93 | API setMessageReaction \ 94 | -F chat_id=$FROM \ 95 | -F message_id=$tg_message_message_id \ 96 | -F reaction='[{"type":"emoji","emoji":"👾"}]' 97 | echo 'Bot reacted to your message☝️' 98 | ;; 99 | madrid) 100 | API sendLocation \ 101 | -F chat_id="$FROM" \ 102 | -F latitude='40.423467' \ 103 | -F longitude='-3.712184' 104 | ;; 105 | menu) 106 | mShowEnv='{"text":"show environment","callback_data":"ment_debug"}' 107 | mShowNotification='{"text":"show notification","callback_data":"ment_notification"}' 108 | mShowAlert='{"text":"show alert","callback_data":"ment_alert"}' 109 | mLikeIt='{"text":"like it","callback_data":"ment_like"}' 110 | mUnlikeIt='{"text":"unlike it","callback_data":"ment_unlike"}' 111 | mCopy='{"text":"copy \"MAGIC\" to clipboard","copy_text":{"text":"MAGIC"}}' 112 | mDelete='{"text":"delete this message","callback_data":"ment_delete"}' 113 | mLayout="[[$mShowEnv],[$mShowAlert,$mShowNotification],[$mLikeIt,$mUnlikeIt],[$mCopy],[$mDelete]]" 114 | API sendMessage \ 115 | -F chat_id=$FROM \ 116 | -F text='Actions' \ 117 | -F reply_markup='{"inline_keyboard":'"$mLayout"'}' 118 | ;; 119 | run) 120 | API "?to=$FROM&a=reactions&a=$tg_message_message_id" -X RUN 121 | echo "Let me show you long run ☝️" 122 | ;; 123 | edit) 124 | API "?to=$FROM&a=editing" -X RUN 125 | ;; 126 | progress) 127 | API "?to=$FROM&a=progress" -X RUN 128 | ;; 129 | id) 130 | echo '%!PRE' 131 | id 2>&1 132 | ;; 133 | caps) 134 | echo '%!PRE' 135 | getpcaps --verbose --iab $$ 136 | ;; 137 | hostname) 138 | echo '%!PRE' 139 | hostname 2>&1 140 | ;; 141 | help) 142 | API sendMessage -F chat_id=$FROM -F parse_mode=Markdown -F text=' 143 | Known commands: 144 | 145 | - `debug` — show args, environment and vars 146 | - `about` — about bot (reslut of `/getMe` API call) 147 | - `two` — one request, two additional responses 148 | - `buttons` — message with buttons 149 | - `image` — show image 150 | - `invert` (as capture to image) — returns flipped flopped image 151 | - `reaction` — show reaction 152 | - `madrid` — show location 153 | - `menu` — scripted buttons 154 | - `run` — long-run example (long sequence of reactions) 155 | - `edit` — long-run example (editing) 156 | - `progress` — one more long-run example (editing) 157 | - `id` — show current user 158 | - `caps` — show current capabilities (`getpcaps $$`) 159 | - `hostname` — show current hostname 160 | - `help` — show this message 161 | - `privacy` — mandatory privacy information 162 | - `start` — just very first greeting message 163 | ' 164 | ;; 165 | start) 166 | API sendMessage -F chat_id=$FROM -F parse_mode=Markdown -F text=' 167 | Hi there!👋 168 | It is demo bot to show an example of usage [cnbot](https://github.com/michurin/cnbot) bot engine. 169 | You can use `help` command to see all available commands.' 170 | ;; 171 | privacy) # https://telegram.org/tos/bot-developers#4-privacy 172 | echo "This bot does not collect or share any personal information." 173 | ;; 174 | *) 175 | if test -n "$tg_callback_query_data" 176 | then 177 | case "$1" in 178 | ment_debug) 179 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" 180 | echo '%!PRE' 181 | echo "Environment:" 182 | env | grep tg_ | sort 183 | ;; 184 | ment_like) 185 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F "text=Like it" 186 | API setMessageReaction -F chat_id=$tg_callback_query_message_chat_id \ 187 | -F message_id=$tg_callback_query_message_message_id \ 188 | -F reaction='[{"type":"emoji","emoji":"👾"}]' 189 | ;; 190 | ment_unlike) 191 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F "text=Don't like it" 192 | API setMessageReaction -F chat_id=$tg_callback_query_message_chat_id \ 193 | -F message_id=$tg_callback_query_message_message_id \ 194 | -F reaction='[]' 195 | ;; 196 | ment_delete) 197 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" 198 | API deleteMessage -F chat_id=$tg_callback_query_message_chat_id \ 199 | -F message_id=$tg_callback_query_message_message_id 200 | ;; 201 | ment_notification) 202 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F text="Notification text (200 chars maximum)" 203 | ;; 204 | ment_alert) 205 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F text="Notification text shown as alert" -F show_alert=true 206 | ;; 207 | esac 208 | else 209 | API sendMessage -F chat_id=$FROM -F text='Invalid command. Say `help`.' -F parse_mode=Markdown 210 | fi 211 | ;; 212 | esac 213 | -------------------------------------------------------------------------------- /pkg/tests/integraion_test.go: -------------------------------------------------------------------------------- 1 | package tests_test 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "net/http" 7 | "net/http/httptest" 8 | "os/exec" 9 | "path/filepath" 10 | "strings" 11 | "testing" 12 | "time" 13 | 14 | "github.com/stretchr/testify/assert" 15 | "github.com/stretchr/testify/require" 16 | 17 | "github.com/michurin/cnbot/pkg/ctxlog" 18 | "github.com/michurin/cnbot/pkg/tests/apiserver" 19 | "github.com/michurin/cnbot/pkg/xbot" 20 | "github.com/michurin/cnbot/pkg/xctrl" 21 | "github.com/michurin/cnbot/pkg/xloop" 22 | "github.com/michurin/cnbot/pkg/xproc" 23 | ) 24 | 25 | // ***************************************************************************** 26 | // Integration tests use real curl utility. It must be installed in your system. 27 | // Motivation: to cover real behavior of curls command line options. 28 | // ***************************************************************************** 29 | 30 | // TODO setup xlog 31 | 32 | func TestAPI_justCall(t *testing.T) { 33 | /* case 34 | tg bot 35 | | | 36 | |<--req---| 37 | |---resp->| 38 | */ 39 | tgURL, tgClose := apiserver.APIServer(t, nil, map[string][]apiserver.APIAct{ 40 | "/botMORN/xMorn": {{ 41 | IsJSON: true, 42 | Request: `{"ok":1}`, 43 | Response: []byte(`{"response":1}`), 44 | }}, 45 | }) 46 | defer tgClose() 47 | 48 | ctx := context.Background() 49 | 50 | bot := buildBot(tgURL) 51 | 52 | body, err := bot.API(ctx, &xbot.Request{ 53 | Method: "xMorn", 54 | ContentType: "application/json", 55 | Body: []byte(`{"ok":1}`), 56 | }) 57 | 58 | require.NoError(t, err) 59 | assert.JSONEq(t, `{"response":1}`, string(body)) 60 | } 61 | 62 | func TestMethods(t *testing.T) { 63 | /* cases 64 | tg bots loop 65 | | | 66 | |<--req---| (call for update) 67 | |---resp->| 68 | | | 69 | | |--exec-->| script 70 | | |<-stdout-| 71 | | | 72 | |<--req---| (call for update) 73 | |---resp->| 74 | |<--req---| (and send response from script) 75 | |---resp->| (the order of update and send doesn't meter) 76 | */ 77 | for _, cs := range []struct { 78 | name string 79 | updateResponse string 80 | sendRequest string 81 | }{ 82 | { 83 | name: "message", 84 | updateResponse: `{"ok": true, "result": [{"update_id": 500, "message": { 85 | "message_id": 100, 86 | "from": {"id": 1500, "is_bot": false, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "language_code": "en"}, 87 | "chat": {"id": 1501, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "type": "private"}, 88 | "date": 1682222222, 89 | "text": "word"}}]}`, 90 | sendRequest: `{"chat_id": 1500, "text": "word [n=1]"}`, 91 | }, 92 | { 93 | name: "message_reaction", 94 | updateResponse: `{"ok": true, "result": [{"update_id": 500, "message_reaction": { 95 | "message_id": 100, 96 | "user": {"id": 1500, "is_bot": false, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "language_code": "en"}, 97 | "chat": {"id": 1501, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "type": "private"}, 98 | "date": 1682222222, 99 | "old_reaction": [], 100 | "new_reaction": [{"type":"emoji","emoji":"\ud83e\udd1d"}]}}]}`, 101 | sendRequest: `{"chat_id": 1500, "text": "message_reaction [n=1]"}`, 102 | }, 103 | { 104 | name: "callback_query", 105 | updateResponse: `{"ok": true, "result": [{"update_id": 500, "callback_query": { 106 | "id": "333333333333333333", 107 | "from": {"id": 1500, "is_bot": false, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "language_code": "en"}, 108 | "chat": {"id": 1501, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "type": "private"}, 109 | "message": { 110 | "message_id": 90, 111 | "from": {"id": 1600, "is_bot": true, "first_name": "BOT", "username":"BOT_bot"}, 112 | "date": 1682222222, 113 | "text": "OK", 114 | "reply_markup": {"inline_keyboard": [[{"text": "button_text", "callback_data": "button_data (in message)"}]]}}, 115 | "chat_instance": "4444444444444444444", 116 | "data": "button_data"}}]}`, 117 | sendRequest: `{"chat_id": 1500, "text": "button_data [n=1]"}`, 118 | }, 119 | } { 120 | t.Run(cs.name, func(t *testing.T) { 121 | ctx, cancel := context.WithCancel(context.Background()) 122 | defer cancel() 123 | 124 | tgURL, tgClose := apiserver.APIServer(t, cancel, map[string][]apiserver.APIAct{ 125 | "/botMORN/getUpdates": { 126 | { 127 | IsJSON: true, 128 | Request: `{"offset":0,"timeout":30,"allowed_updates":["callback_query","inline_query","message","message_reaction","poll","poll_answer"]}`, 129 | Response: []byte(cs.updateResponse), 130 | }, 131 | { 132 | IsJSON: true, 133 | Request: `{"offset":501,"timeout":30,"allowed_updates":["callback_query","inline_query","message","message_reaction","poll","poll_answer"]}`, 134 | Response: nil, 135 | }, 136 | }, 137 | "/botMORN/sendMessage": { 138 | { 139 | IsJSON: true, 140 | Request: cs.sendRequest, 141 | Response: []byte(`{"ok": true, "result": {}}`), 142 | }, 143 | }, 144 | }) 145 | defer tgClose() 146 | 147 | bot := buildBot(tgURL) 148 | 149 | command := buildCommand(t, "scripts/show_args.sh") 150 | 151 | err := xloop.Loop(ctx, bot, command) 152 | require.Error(t, err) 153 | require.Contains(t, err.Error(), "context canceled") // like "api: client: Post \"http://127.0.0.1:34241/botMORN/getUpdates\": context canceled" 154 | }) 155 | } 156 | } 157 | 158 | func TestScriptOutputTypes(t *testing.T) { //nolint:funlen 159 | /* cases 160 | tg bots loop 161 | | | 162 | |<--req---| (call for update) 163 | |---resp->| 164 | | | 165 | | |--exec-->| script 166 | | |<-stdout-| 167 | | | 168 | |<--req---| (call for update) 169 | |---resp->| 170 | |<--req---| (and send response from script) 171 | |---resp->| (the order of update and send doesn't meter) 172 | */ 173 | simpleUpdates := []apiserver.APIAct{ 174 | { 175 | IsJSON: true, 176 | Request: `{"offset":0,"timeout":30,"allowed_updates":["callback_query","inline_query","message","message_reaction","poll","poll_answer"]}`, 177 | Response: []byte(`{"ok": true, "result": [{"update_id": 500, "message": { 178 | "message_id": 100, 179 | "from": {"id": 1500, "is_bot": false, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "language_code": "en"}, 180 | "chat": {"id": 1501, "first_name": "Alex", "last_name": "Morn", "username": "AlexMorn", "type": "private"}, 181 | "date": 1682222222, 182 | "text": "word"}}]}`), 183 | }, 184 | { 185 | IsJSON: true, 186 | Request: `{"offset":501,"timeout":30,"allowed_updates":["callback_query","inline_query","message","message_reaction","poll","poll_answer"]}`, 187 | Response: nil, // the second update call will stop Mock API server by this nil 188 | }, 189 | } 190 | sendMessageResponseJSON := []byte(`{"ok": true, "result": {}}`) 191 | for _, cs := range []struct { 192 | script string 193 | api map[string][]apiserver.APIAct 194 | }{ 195 | { 196 | script: "scripts/text_ok.sh", 197 | api: map[string][]apiserver.APIAct{ 198 | "/botMORN/getUpdates": simpleUpdates, 199 | "/botMORN/sendMessage": { 200 | { 201 | IsJSON: true, 202 | Request: `{"chat_id": 1500, "text": "ok"}`, 203 | Response: sendMessageResponseJSON, 204 | }, 205 | }, 206 | }, 207 | }, 208 | { 209 | script: "scripts/text_long.sh", 210 | api: map[string][]apiserver.APIAct{ 211 | "/botMORN/getUpdates": simpleUpdates, 212 | "/botMORN/sendMessage": { 213 | { 214 | IsJSON: true, 215 | Request: `{"chat_id": 1500, "text": "` + strings.Repeat(`⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘\n`, 315) + `1` + `"}`, 216 | Response: sendMessageResponseJSON, 217 | }, 218 | }, 219 | }, 220 | }, 221 | { 222 | script: "scripts/text_too_long.sh", 223 | api: map[string][]apiserver.APIAct{ 224 | "/botMORN/getUpdates": simpleUpdates, 225 | "/botMORN/sendDocument": { 226 | { 227 | IsJSON: false, 228 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"document\"; filename=\"message.txt\"\r\nContent-Type: text/plain\r\n\r\n" + strings.Repeat("⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘\n", 315) + `12` + "\r\n--BOUND--\r\n", 229 | Response: sendMessageResponseJSON, 230 | }, 231 | }, 232 | }, 233 | }, 234 | { 235 | script: "scripts/preformatted_ok.sh", 236 | api: map[string][]apiserver.APIAct{ 237 | "/botMORN/getUpdates": simpleUpdates, 238 | "/botMORN/sendMessage": { 239 | { 240 | IsJSON: true, 241 | Request: `{"chat_id": 1500, "text": "ok", "entities": [{"type": "pre", "offset": 0, "length": 2}]}`, 242 | Response: sendMessageResponseJSON, 243 | }, 244 | }, 245 | }, 246 | }, 247 | { 248 | script: "scripts/preformatted_complex_ok.sh", // one unicode char, however it is two utf16 words and length=2 249 | api: map[string][]apiserver.APIAct{ 250 | "/botMORN/getUpdates": simpleUpdates, 251 | "/botMORN/sendMessage": { 252 | { 253 | IsJSON: true, 254 | Request: `{"chat_id": 1500, "text": "⚒️", "entities": [{"type": "pre", "offset": 0, "length": 2}]}`, 255 | Response: sendMessageResponseJSON, 256 | }, 257 | }, 258 | }, 259 | }, 260 | { 261 | script: "scripts/preformatted_long.sh", 262 | api: map[string][]apiserver.APIAct{ 263 | "/botMORN/getUpdates": simpleUpdates, 264 | "/botMORN/sendMessage": { 265 | { 266 | IsJSON: true, 267 | Request: `{"chat_id": 1500, "entities": [{"length":4096, "offset":0, "type":"pre"}], "text": "` + strings.Repeat(`⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘\n`, 315) + `1` + `"}`, 268 | Response: sendMessageResponseJSON, 269 | }, 270 | }, 271 | }, 272 | }, 273 | { 274 | script: "scripts/preformatted_too_long.sh", 275 | api: map[string][]apiserver.APIAct{ 276 | "/botMORN/getUpdates": simpleUpdates, 277 | "/botMORN/sendDocument": { 278 | { 279 | IsJSON: false, 280 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"document\"; filename=\"formatted_text.txt\"\r\nContent-Type: text/plain\r\n\r\n" + strings.Repeat("⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘⌘\n", 315) + `12` + "\r\n--BOUND--\r\n", 281 | Response: sendMessageResponseJSON, 282 | }, 283 | }, 284 | }, 285 | }, 286 | { 287 | script: "scripts/media_jpeg.sh", 288 | api: map[string][]apiserver.APIAct{ 289 | "/botMORN/getUpdates": simpleUpdates, 290 | "/botMORN/sendPhoto": { 291 | { 292 | IsJSON: false, 293 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"photo\"; filename=\"image.jpeg\"\r\nContent-Type: image/jpeg\r\n\r\n\xff\xd8\xff\r\n--BOUND--\r\n", 294 | Response: sendMessageResponseJSON, 295 | }, 296 | }, 297 | }, 298 | }, 299 | { 300 | script: "scripts/media_png.sh", 301 | api: map[string][]apiserver.APIAct{ 302 | "/botMORN/getUpdates": simpleUpdates, 303 | "/botMORN/sendPhoto": { 304 | { 305 | IsJSON: false, 306 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"photo\"; filename=\"image.png\"\r\nContent-Type: image/png\r\n\r\n\x89PNG\r\n\x1a\n\r\n--BOUND--\r\n", 307 | Response: sendMessageResponseJSON, 308 | }, 309 | }, 310 | }, 311 | }, 312 | { 313 | script: "scripts/media_mp3.sh", 314 | api: map[string][]apiserver.APIAct{ 315 | "/botMORN/getUpdates": simpleUpdates, 316 | "/botMORN/sendAudio": { 317 | { 318 | IsJSON: false, 319 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"audio\"; filename=\"audio.mp3\"\r\nContent-Type: audio/mpeg\r\n\r\nID3\r\n--BOUND--\r\n", 320 | Response: sendMessageResponseJSON, 321 | }, 322 | }, 323 | }, 324 | }, 325 | { 326 | script: "scripts/media_ogg.sh", 327 | api: map[string][]apiserver.APIAct{ 328 | "/botMORN/getUpdates": simpleUpdates, 329 | "/botMORN/sendDocument": { // consider ogg as document, it seems it's not fully supported 330 | { 331 | IsJSON: false, 332 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"document\"; filename=\"document.ogx\"\r\nContent-Type: application/ogg\r\n\r\nOggS\x00\r\n--BOUND--\r\n", 333 | Response: sendMessageResponseJSON, 334 | }, 335 | }, 336 | }, 337 | }, 338 | { 339 | script: "scripts/media_mp4.sh", 340 | api: map[string][]apiserver.APIAct{ 341 | "/botMORN/getUpdates": simpleUpdates, 342 | "/botMORN/sendVideo": { 343 | { 344 | IsJSON: false, 345 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"video\"; filename=\"video.mp4\"\r\nContent-Type: video/mp4\r\n\r\n\x00\x00\x00\fftypmp4_\r\n--BOUND--\r\n", 346 | Response: sendMessageResponseJSON, 347 | }, 348 | }, 349 | }, 350 | }, 351 | { 352 | script: "scripts/media_pdf.sh", 353 | api: map[string][]apiserver.APIAct{ 354 | "/botMORN/getUpdates": simpleUpdates, 355 | "/botMORN/sendDocument": { 356 | { 357 | IsJSON: false, 358 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"document\"; filename=\"document.pdf\"\r\nContent-Type: application/pdf\r\n\r\n%PDF-\r\n--BOUND--\r\n", 359 | Response: sendMessageResponseJSON, 360 | }, 361 | }, 362 | }, 363 | }, 364 | { 365 | script: "scripts/media_bin.sh", 366 | api: map[string][]apiserver.APIAct{ 367 | "/botMORN/getUpdates": simpleUpdates, 368 | "/botMORN/sendDocument": { 369 | { 370 | IsJSON: false, 371 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"chat_id\"\r\n\r\n1500\r\n--BOUND\r\nContent-Disposition: form-data; name=\"document\"; filename=\"document.dat\"\r\nContent-Type: application/octet-stream\r\n\r\n\x00\x00\x00\x00\r\n--BOUND--\r\n", 372 | Response: sendMessageResponseJSON, 373 | }, 374 | }, 375 | }, 376 | }, 377 | { 378 | script: "scripts/media_len_zero.sh", 379 | api: map[string][]apiserver.APIAct{ 380 | "/botMORN/getUpdates": simpleUpdates, 381 | }, 382 | }, 383 | } { 384 | t.Run(cs.script[8:], func(t *testing.T) { 385 | ctx, cancel := context.WithCancel(context.Background()) 386 | defer cancel() 387 | 388 | tgURL, tgClose := apiserver.APIServer(t, cancel, cs.api) 389 | defer tgClose() 390 | 391 | bot := buildBot(tgURL) 392 | 393 | command := buildCommand(t, cs.script) 394 | 395 | err := xloop.Loop(ctx, bot, command) 396 | require.Error(t, err) 397 | require.ErrorContains(t, err, "context canceled") // like "api: client: Post \"http://127.0.0.1:34241/botMORN/getUpdates\": context canceled" 398 | }) 399 | } 400 | } 401 | 402 | func TestHttp(t *testing.T) { 403 | /* cases 404 | tg bots ctrl 405 | | | 406 | | |<-- someone external calls bot over http 407 | |<--req---| (request to send) 408 | |---resp->| 409 | | |--> reply to external client 410 | */ 411 | for _, cs := range []struct { 412 | name string 413 | curl []string 414 | qs string 415 | api map[string][]apiserver.APIAct 416 | }{ 417 | { 418 | name: "curl_F", // curl -F works transparently as is 419 | curl: []string{"-q", "-s", "-F", "user_id=10", "-F", "text=ok"}, 420 | qs: "", 421 | api: map[string][]apiserver.APIAct{ 422 | "/botMORN/someMethod": {{ 423 | IsJSON: false, 424 | Request: "--BOUND\r\nContent-Disposition: form-data; name=\"user_id\"\r\n\r\n10\r\n--BOUND\r\nContent-Disposition: form-data; name=\"text\"\r\n\r\nok\r\n--BOUND--\r\n", 425 | Response: []byte("done."), 426 | }}, 427 | }, 428 | }, 429 | { 430 | name: "curl_d", 431 | curl: []string{"-q", "-s", "-d", "ok"}, 432 | qs: "?to=111", 433 | api: map[string][]apiserver.APIAct{ 434 | "/botMORN/sendMessage": {{ 435 | IsJSON: true, 436 | Request: `{"chat_id":111, "text":"ok"}`, 437 | Response: []byte("done."), 438 | }}, 439 | }, 440 | }, 441 | } { 442 | t.Run(cs.name, func(t *testing.T) { 443 | tgURL, tgClose := apiserver.APIServer(t, nil, cs.api) 444 | defer tgClose() 445 | 446 | bot := buildBot(tgURL) 447 | 448 | h := xctrl.Handler(bot, nil, ctxlog.Patch(context.Background())) // we won't use second argument in this test 449 | 450 | s := httptest.NewServer(h) 451 | 452 | ou, er := runCurl(t, append(cs.curl, s.URL+"/x/someMethod"+cs.qs)...) 453 | assert.Equal(t, "done.", ou) 454 | assert.Empty(t, er) 455 | }) 456 | } 457 | } 458 | 459 | func TestDownload(t *testing.T) { 460 | /* cases 461 | tg bots ctrl 462 | | | 463 | | |<-- someone external calls bot over http with file_id 464 | |<--req---| (getFile) 465 | |---resp->| 466 | |<--req---| (download data) 467 | |---resp->| 468 | | |--> reply to external client 469 | */ 470 | tgURL, tgClose := apiserver.APIServer(t, nil, map[string][]apiserver.APIAct{ 471 | "/botMORN/getFile": {{ 472 | IsJSON: true, 473 | Request: `{"file_id":"FILE"}`, 474 | Response: []byte(`{"ok":true, "result":{"file_path":"file/path.jpeg"}}`), 475 | }}, 476 | "/file/botMORN/file/path.jpeg": {{ 477 | IsJSON: false, 478 | Stream: true, 479 | Request: "", 480 | Response: []byte("DATA"), 481 | }}, 482 | }) 483 | defer tgClose() 484 | 485 | bot := buildBot(tgURL) 486 | 487 | h := xctrl.Handler(bot, nil, ctxlog.Patch(context.Background())) // we won't use second argument in this test 488 | 489 | s := httptest.NewServer(h) 490 | 491 | ou, er := runCurl(t, "-q", "-s", s.URL+"?file_id=FILE") 492 | assert.Equal(t, "DATA", ou) 493 | assert.Empty(t, er) 494 | } 495 | 496 | func TestHttp_long(t *testing.T) { // CAUTION: test has sleep 497 | /* cases 498 | tg bots loop 499 | | | 500 | | |<-- someone external calls bot over http (method=RUN) 501 | | | 502 | | |--exec-->| long-running external script 503 | | |<-stdout-| 504 | | | 505 | |<--req---| (request to send) 506 | |---resp->| (response will be skipped; and test tries cover it by making small sleep) 507 | */ 508 | 509 | ctx, cancel := context.WithCancel(context.Background()) 510 | defer cancel() 511 | 512 | tgURL, tgClose := apiserver.APIServer(t, cancel, map[string][]apiserver.APIAct{ 513 | "/botMORN/sendMessage": {{ 514 | IsJSON: true, 515 | Request: `{"chat_id":222, "text":"args [222]: a1 a2"}`, 516 | Response: nil, // response will be skipped, but in fact, we do not test this fact 517 | }}, 518 | }) 519 | defer tgClose() 520 | 521 | bot := buildBot(tgURL) 522 | command := buildCommand(t, "scripts/longrunning.sh") 523 | 524 | h := xctrl.Handler(bot, command, ctxlog.Patch(context.Background())) 525 | 526 | s := httptest.NewServer(h) 527 | 528 | ou, er := runCurl(t, "-q", "-s", "-X", "RUN", s.URL+"/?to=222&a=a1&a=a2") 529 | assert.Empty(t, ou) 530 | assert.Empty(t, er) 531 | <-ctx.Done() 532 | time.Sleep(time.Millisecond * 100) // we give small amount of time to let Bot.API method finishing after receiving response; it is not necessary 533 | } 534 | 535 | func TestProc(t *testing.T) { // CAUTION: test has sleep indirectly 536 | ctx := context.Background() 537 | t.Run("show_args", func(t *testing.T) { 538 | data, err := buildCommand(t, "scripts/run_show_args.sh").Run(ctx, []string{"ARG1", "ARG2"}, []string{"test1=TEST1", "test2=TEST2"}) 539 | require.NoError(t, err, "data="+string(data)) 540 | assert.Equal(t, "arg1=ARG1 arg2=ARG2 test1=TEST1 test2=TEST2 TEST=test\n", string(data)) 541 | }) 542 | t.Run("exit", func(t *testing.T) { 543 | data, err := buildCommand(t, "scripts/run_exit.sh").Run(ctx, nil, nil) 544 | require.Error(t, err) 545 | assert.Contains(t, err.Error(), "wait: exit status 28") 546 | assert.Nil(t, data) 547 | }) 548 | t.Run("slow", func(t *testing.T) { 549 | data, err := buildCommand(t, "scripts/run_slow.sh").Run(ctx, nil, nil) 550 | require.NoError(t, err) 551 | assert.Equal(t, 552 | `start 553 | trap SIGINT 554 | trap ERR 555 | end 556 | trap EXIT 557 | `, string(data)) 558 | }) 559 | t.Run("immortal", func(t *testing.T) { 560 | data, err := buildCommand(t, "scripts/run_immortal.sh").Run(ctx, nil, nil) 561 | assert.Error(t, err) 562 | assert.Contains(t, err.Error(), "wait: signal: killed") 563 | assert.Nil(t, data) 564 | }) 565 | t.Run("notfound", func(t *testing.T) { 566 | data, err := buildCommand(t, "scripts/NOTFOUND").Run(ctx, nil, nil) 567 | assert.Error(t, err) 568 | assert.Contains(t, err.Error(), "/NOTFOUND: no such file or directory") // it is absolute path appears in error message 569 | assert.Nil(t, data) 570 | }) 571 | } 572 | 573 | func buildCommand(t *testing.T, cmd string) *xproc.Cmd { 574 | t.Helper() 575 | absCmd, err := filepath.Abs(cmd) // app does it 576 | require.NoError(t, err) 577 | return &xproc.Cmd{ 578 | InterruptDelay: 200 * time.Millisecond, // timeouts important for TestProc 579 | KillDelay: 200 * time.Millisecond, 580 | Env: []string{"TEST=test"}, 581 | Command: absCmd, 582 | } 583 | } 584 | 585 | func buildBot(origin string) *xbot.Bot { 586 | return &xbot.Bot{ 587 | APIOrigin: origin, 588 | Token: "MORN", 589 | Client: http.DefaultClient, 590 | } 591 | } 592 | 593 | func runCurl(t *testing.T, args ...string) (string, string) { 594 | t.Helper() 595 | t.Logf("Run curl %s", strings.Join(args, " ")) 596 | cmd := exec.CommandContext(t.Context(), "curl", args...) 597 | var stdOut, stdErr bytes.Buffer 598 | cmd.Stdout = &stdOut 599 | cmd.Stderr = &stdErr 600 | err := cmd.Run() 601 | require.NoError(t, err) 602 | return stdOut.String(), stdErr.String() 603 | } 604 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # cnbot 2 | 3 | [![build](https://github.com/michurin/cnbot/actions/workflows/ci.yaml/badge.svg)](https://github.com/michurin/cnbot/actions/workflows/ci.yaml) 4 | [![codecov](https://codecov.io/gh/michurin/cnbot/graph/badge.svg?token=3GdCf3TqZC)](https://codecov.io/gh/michurin/cnbot) 5 | [![Go Report Card](https://goreportcard.com/badge/github.com/michurin/cnbot)](https://goreportcard.com/report/github.com/michurin/cnbot) 6 | 7 | The goal of this project is to provide a way 8 | to alive Telegram bots by scripting that 9 | even simpler than CGI scripts. 10 | All you need to write is a script (on any language) 11 | that is complying with extremely simple contract. 12 | 13 | ![Telegram bot demo screenshot](https://raw.githubusercontent.com/michurin/cnbot/static/screenshot-2024.gif) 14 | 15 | ## What is it for 16 | 17 | This bot engine has proven itself in alerting, system monitoring and managing tasks. 18 | 19 | It also good for prototyping and fast proofing ideas. 20 | 21 | ## How mature is it 22 | 23 | The engine is not perfect. Some error messages could be more informative. 24 | Somewhere you can face a lug of documentation and the need to appeal to source code. 25 | 26 | However, the engine has already proven itself in production and prototyping. 27 | 28 | It served bots for huge conferences, meetings and events. It has helped customers 29 | and provided control functionality for crew. 30 | 31 | The engine successfully drives several monitoring and alerting bots. 32 | 33 | It seems, API of this bot engines is quite stable and won't change dramatically in the near future. 34 | 35 | ## Basic ideas 36 | 37 | You implement all your business logic in your scripts. You are totally free to use all Telegram API abilities. 38 | 39 | `cnbot` interact with scripts using (i) `stdout` stream, (ii) arguments and (iii) environment variables. 40 | 41 | The engine automatically recognize multimedia and images. It cares about concurrency and races. 42 | 43 | It also provides simple API for asynchronous messaging from `cron`s and such things. 44 | 45 | It manages tasks (subprocesses), controls timeouts, sends signals and provides abilities to 46 | run long-running tasks like long image/video conversions and/or downloading. 47 | 48 | One instance of engine is able to manage several different bots. 49 | 50 | ## Quick start 51 | 52 | ### Zero-effort Docker-way to run full-featured bot 53 | 54 | All you need is bot token ([instructions](https://core.telegram.org/bots#how-do-i-create-a-bot)). 55 | 56 | ```sh 57 | docker build -t cnbot:latest https://raw.githubusercontent.com/michurin/cnbot/master/demo/Dockerfile 58 | docker run -it --rm --name cnbot -e TB_TOKEN=4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc cnbot:latest 59 | ``` 60 | 61 | [More details](https://github.com/michurin/cnbot/tree/master/demo) 62 | 63 | ### Run simplest one-line bot 64 | 65 | #### Prepare 66 | 67 | First things first, you need to create bot and get it's token. 68 | It is free, just follow [instructions](https://core.telegram.org/bots#how-do-i-create-a-bot). 69 | 70 | #### Build and run 71 | 72 | You need Telegram API token, `golang` and standard system commands `echo` and `true`. 73 | 74 | ```sh 75 | go install github.com/michurin/cnbot/cmd/...@latest 76 | tb_token='4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc' tb_script=echo tb_long_running_script=true tb_ctrl_addr=:9999 cnbot 77 | ``` 78 | 79 | or without installation: 80 | 81 | ```sh 82 | git clone https://github.com/michurin/cnbot 83 | cd cnbot 84 | tb_token='4839574812:AAFD39kkdpWt3ywyRZergyOLMaJhac60qc' tb_script=echo tb_long_running_script=true tb_ctrl_addr=:9999 go run ./cmd/... 85 | ``` 86 | 87 | You are free to keep your token in file and use syntax like this to refer to file: `tb_token=@filename` 88 | 89 | Don't worry, we will use configuration file further. The engine is able to use both files and direct environment variables. 90 | 91 | - `tb_YOURBOTNAME_token` is a token your are given: `digits:long_string` 92 | - `tb_YOURBOTNAME_script` is a command to run. We use the standard system command `echo`. I can be located elsewhere in your system. Try to say `whereis echo` to fine it 93 | - `tb_YOURBOTNAME_long_running_script` let it be the same command. We consider it later 94 | - `tb_YOURBOTNAME_ctrl_addr` we consider it soon 95 | 96 | Run this command with correct variables and try to say something to you bot. You will be echoed by it. 97 | 98 | ### Put your configuration into file 99 | 100 | You may as well put your configuration into env-file. The format of file is literally the same as `systemd` use. 101 | So you are able to load it in `systemd` files as well. For example: 102 | 103 | ```sh 104 | # let's name it config.env 105 | tb_token='TOKEN' 106 | tb_script=/usr/bin/echo 107 | tb_long_running_script=/usr/bin/echo 108 | tb_ctrl_addr=:9999 109 | ``` 110 | 111 | Now just start bot like this: 112 | 113 | ```sh 114 | cnbot config.env 115 | ``` 116 | 117 | ## Playing with random features 118 | 119 | ### Your first script (finding out your UserID) 120 | 121 | Let's look at the script, that shows its arguments and environment variables: 122 | 123 | ```sh 124 | #!/bin/sh 125 | 126 | echo "Args: $@" 127 | echo "Environment:" 128 | env | grep tg_ | sort 129 | ``` 130 | 131 | Name it `mybot.sh` and mention it in configuration variable `tb_script=./mybot.sh`. Restart the bot and say to it `Hello bot!`. 132 | It will reply to you something like that: 133 | 134 | ``` 135 | ╭─────────────────────────────────────────╮ 136 | │ Args: hello bot! │ 137 | │ Environment: │ 138 | │ tg_message_chat_first_name=Alexey │ 139 | │ tg_message_chat_id=153333328 │ 140 | │ tg_message_chat_last_name=Michurin │ 141 | │ tg_message_chat_type=private │ 142 | │ tg_message_chat_username=AlexeyMichurin │ 143 | │ tg_message_date=1717171717 │ 144 | │ tg_message_from_first_name=Alexey │ 145 | │ tg_message_from_id=153333328 │ 146 | │ tg_message_from_is_bot=false │ 147 | │ tg_message_from_language_code=en │ 148 | │ tg_message_from_last_name=Michurin │ 149 | │ tg_message_from_username=AlexeyMichurin │ 150 | │ tg_message_message_id=4554 │ 151 | │ tg_message_text=Hello bot! │ 152 | │ tg_update_id=513333387 │ 153 | │ tg_x_build=development (devel) │ 154 | │ tg_x_ctrl_addr=:9999 │ 155 | ╰─────────────────────────────────────────╯ 156 | ``` 157 | 158 | You can see that your message has been put to arguments in convenient normalized form, and you have a bunch of useful variables 159 | with additional information. We will consider them further. At this point we just figure out that our user id is `tg_message_from_id=153333328`. 160 | We will use this information very soon. 161 | 162 | ### Asynchronous messaging 163 | 164 | You are free to send messages from anywhere: from cron jobs, from init scripts... Try it just from command line: 165 | 166 | ```sh 167 | curl -qs http://localhost:9999/?to=153333328 -d 'OK!' 168 | ``` 169 | 170 | If you bot is running, you will obtain the message `OK!` in you Telegram client. 171 | 172 | ``` 173 | ╭──────────╮ 174 | │ OK! │ 175 | ╰──────────╯ 176 | ``` 177 | 178 | Do not forget to use *your* user id from previous section. 179 | 180 | It makes sense what variable `tb_ctrl_addr=:9999` is for. It defines a control interface for external interactions with bot engine. 181 | 182 | ### Call arbitrary Telegram API methods 183 | 184 | You can call whatever method you want. Full list of methods can be found in the 185 | [official Telegram bot API documentation](https://core.telegram.org/bots/api). 186 | 187 | For example, you can obtain information about your bot 188 | (using method [getMe](https://core.telegram.org/bots/api#getme)): 189 | 190 | ```sh 191 | curl -qs http://localhost:9999/method/getMe | jq 192 | ``` 193 | 194 | The response will look like this: 195 | 196 | ```json 197 | { 198 | "ok": true, 199 | "result": { 200 | "id": 223333386, 201 | "is_bot": true, 202 | "first_name": "Your Bot", 203 | "username": "your_bot", 204 | "can_join_groups": true, 205 | "can_read_all_group_messages": false, 206 | "supports_inline_queries": false, 207 | "can_connect_to_business": false 208 | } 209 | } 210 | ``` 211 | 212 | It enables you to send extended messages. For example, you can send a message with buttons 213 | (method [sendMessage](https://core.telegram.org/bots/api#sendmessage)): 214 | 215 | ```sh 216 | curl -qs http://localhost:9999/sendMessage -F chat_id=153333328 -F text='Select search engine' -F reply_markup='{"inline_keyboard":[[{"text":"Google","url":"https://www.google.com/"}, {"text":"DuckDuckGo","url":"https://duckduckgo.com/"}]]}' 217 | ``` 218 | 219 | You will receive message with two clickable buttons: 220 | 221 | ``` 222 | ╭───────────────────────────╮ 223 | │ Select search engine │ 224 | ├─────────────┬─────────────┤ 225 | │ Google ↗│ DuckDuckGo ↗│ 226 | ╰─────────────┴─────────────╯ 227 | ``` 228 | 229 | Do not forget to change `user_id`. 230 | 231 | > [!NOTE] 232 | > You can use any prefixes in URLs. 233 | > URLs `http://localhost:9999/sendMessage` and `http://localhost:9999/ANITHING/sendMessage` are equal. 234 | > It allows you to put engine's API behind prefix. 235 | 236 | ### Sending images 237 | 238 | Bot recognizes media type of input. It will send text: 239 | 240 | ```sh 241 | echo 'Hello!' | curl -qs http://localhost:9999/?to=153333328 --data-binary '@-' 242 | ``` 243 | 244 | However, it will send you image: 245 | 246 | ```sh 247 | curl -qs https://github.githubassets.com/favicons/favicon.png | curl -qs http://localhost:9999/?to=153333328 --data-binary '@-' 248 | ``` 249 | 250 | > [!IMPORTANT] 251 | > Please use the `--data-binary` option for binary data. Option `-d` corrupts EOLs. 252 | 253 | ### Formatted text 254 | 255 | ```sh 256 | (echo '%!PRE'; echo 'Hello!') | curl -qs http://localhost:9999/?to=153333328 --data-binary '@-' 257 | ``` 258 | 259 | ## Big picture 260 | 261 | ### Prepare playground 262 | 263 | Let's extend our `mybot.sh` like that (it is literally [demo script](https://github.com/michurin/cnbot/tree/master/demo/bot.sh) 264 | you can [run it in docker](https://github.com/michurin/cnbot/tree/master/demo)): 265 | 266 | ```sh 267 | #!/bin/bash 268 | 269 | LOG=logs/log.log # /dev/null 270 | 271 | FROM="$tg_message_from_id" 272 | 273 | API() { 274 | API_STDOUT "$@" >>"$LOG" 275 | } 276 | 277 | API_STDOUT() { 278 | url="http://localhost$tg_x_ctrl_addr/$1" 279 | shift 280 | echo "====== curl $url $@" >>"$LOG" 281 | curl -qs "$url" "$@" 2>>"$LOG" 282 | echo >>"$LOG" 283 | echo >>"$LOG" 284 | } 285 | 286 | ( 287 | echo '===================' 288 | echo "Args: $@" 289 | echo "Environment:" 290 | env | grep tg_ | sort 291 | echo '...................' 292 | ) >>"$LOG" 293 | 294 | case "$1" in 295 | debug) 296 | echo '%!PRE' 297 | echo "Args: $@" 298 | echo "Environment:" 299 | env | grep tg_ | sort 300 | echo "FROM=$FROM" 301 | echo "LOG=$LOG" 302 | ;; 303 | about) 304 | echo '%!PRE' 305 | API_STDOUT getMe | jq 306 | ;; 307 | two) 308 | API "?to=$FROM" -d 'OK ONE!' 309 | API "?to=$FROM" -d 'OK TWO!!' 310 | echo 'OK NATIVE' 311 | ;; 312 | buttons) 313 | bGoogle='{"text":"Google","url":"https://www.google.com/"}' 314 | bDuck='{"text":"DuckDuckGo","url":"https://duckduckgo.com/"}' 315 | API sendMessage \ 316 | -F chat_id=$FROM \ 317 | -F text='Select search engine' \ 318 | -F reply_markup='{"inline_keyboard":[['"$bGoogle,$bDuck"']]}' 319 | ;; 320 | image) 321 | curl -qs https://github.com/fluidicon.png 322 | ;; 323 | invert) 324 | wm=0 325 | fid='' 326 | for x in $tg_message_photo # finding the biggest image but ignoring too big ones 327 | do 328 | v=${x}_file_size 329 | s=${!v} # trick: getting variable name from variable; we need bash for it 330 | if test $s -gt 102400; then continue; fi # skipping too big files 331 | v=${x}_width 332 | w=${!v} 333 | v=${x}_file_id 334 | f=${!v} 335 | if test $w -gt $wm; then wm=$w; fid=$f; fi 336 | done 337 | if test -n "$fid" 338 | then 339 | API_STDOUT '' -G --data-urlencode "file_id=$fid" -o - | mogrify -flip -flop -format png - 340 | else 341 | echo "attache not found (maybe it was skipped due to enormous size)" 342 | fi 343 | ;; 344 | reaction) 345 | API setMessageReaction \ 346 | -F chat_id=$FROM \ 347 | -F message_id=$tg_message_message_id \ 348 | -F reaction='[{"type":"emoji","emoji":"👾"}]' 349 | echo 'Bot reacted to your message☝️' 350 | ;; 351 | madrid) 352 | API sendLocation \ 353 | -F chat_id="$FROM" \ 354 | -F latitude='40.423467' \ 355 | -F longitude='-3.712184' 356 | ;; 357 | menu) 358 | mShowEnv='{"text":"show environment","callback_data":"menu-debug"}' 359 | mShowNotification='{"text":"show notification","callback_data":"menu-notification"}' 360 | mShowAlert='{"text":"show alert","callback_data":"menu-alert"}' 361 | mLikeIt='{"text":"like it","callback_data":"menu-like"}' 362 | mUnlikeIt='{"text":"unlike it","callback_data":"menu-unlike"}' 363 | mDelete='{"text":"delete this message","callback_data":"menu-delete"}' 364 | mLayout="[[$mShowEnv],[$mShowAlert,$mShowNotification],[$mLikeIt,$mUnlikeIt],[$mDelete]]" 365 | API sendMessage \ 366 | -F chat_id=$FROM \ 367 | -F text='Actions' \ 368 | -F reply_markup='{"inline_keyboard":'"$mLayout"'}' 369 | ;; 370 | run) 371 | API "?to=$FROM&a=reactions&a=$tg_message_message_id" -X RUN 372 | echo "I'll show you long run" 373 | ;; 374 | edit) 375 | API "?to=$FROM&a=editing" -X RUN 376 | ;; 377 | id) 378 | echo '%!PRE' 379 | id 2>&1 380 | ;; 381 | caps) 382 | echo '%!PRE' 383 | getpcaps --verbose --iab $$ 384 | ;; 385 | hostname) 386 | echo '%!PRE' 387 | hostname 2>&1 388 | ;; 389 | help) 390 | API sendMessage -F chat_id=$FROM -F parse_mode=Markdown -F text=' 391 | Known commands: 392 | 393 | - `debug` — show args, environment and vars 394 | - `about` — reslut of getMe 395 | - `two` — one request, two responses 396 | - `buttons` — message with buttons 397 | - `image` — show image 398 | - `invert` (as capture to image) — returns flipped flopped image 399 | - `reaction` — show reaction 400 | - `madrid` — show location 401 | - `menu` — scripted buttons 402 | - `run` — long-run example (long sequence of reactions) 403 | - `edit` — long-run example (editing) 404 | - `id` — check user who script runs from 405 | - `caps` — check current capabilities (`getpcaps $$`) 406 | - `hostname` — check hostname where script runs 407 | - `help` — show this message 408 | - `privacy` — mandatory privacy information 409 | - `start` — just very first greeting message 410 | ' 411 | ;; 412 | start) 413 | API sendMessage -F chat_id=$FROM -F parse_mode=Markdown -F text=' 414 | Hi there!👋 415 | It is demo bot to show an example of usage [cnbot](https://github.com/michurin/cnbot) bot engine. 416 | You can use `help` command to see all available commands.' 417 | ;; 418 | privacy) # https://telegram.org/tos/bot-developers#4-privacy 419 | echo "This bot does not collect or share any personal information." 420 | ;; 421 | *) 422 | if test -n "$tg_callback_query_data" 423 | then 424 | case "$1" in 425 | menu-debug) 426 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" 427 | echo '%!PRE' 428 | echo "Environment:" 429 | env | grep tg_ | sort 430 | ;; 431 | menu-like) 432 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F "text=Like it" 433 | API setMessageReaction -F chat_id=$tg_callback_query_message_chat_id \ 434 | -F message_id=$tg_callback_query_message_message_id \ 435 | -F reaction='[{"type":"emoji","emoji":"👾"}]' 436 | ;; 437 | menu-unlike) 438 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F "text=Don't like it" 439 | API setMessageReaction -F chat_id=$tg_callback_query_message_chat_id \ 440 | -F message_id=$tg_callback_query_message_message_id \ 441 | -F reaction='[]' 442 | ;; 443 | menu-delete) 444 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" 445 | API deleteMessage -F chat_id=$tg_callback_query_message_chat_id \ 446 | -F message_id=$tg_callback_query_message_message_id 447 | ;; 448 | menu-notification) 449 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F text="Notification text (200 chars maximum)" 450 | ;; 451 | menu-alert) 452 | API answerCallbackQuery -F callback_query_id="$tg_callback_query_id" -F text="Notification text shown as alert" -F show_alert=true 453 | ;; 454 | esac 455 | else 456 | API sendMessage -F chat_id=$FROM -F text='Invalid command. Say `help`.' -F parse_mode=Markdown 457 | fi 458 | ;; 459 | esac 460 | ``` 461 | 462 | Let's add script for long-running tasks `mybot_long.sh` (it's [demo script](https://github.com/michurin/cnbot/tree/master/demo/bot_long.sh)): 463 | 464 | ```sh 465 | #!/bin/sh 466 | 467 | LOG=logs/log_long.log # /dev/null 468 | 469 | FROM="$tg_x_to" 470 | 471 | API() { 472 | API_STDOUT "$@" >>"$LOG" 473 | } 474 | 475 | API_STDOUT() { 476 | url="http://localhost$tg_x_ctrl_addr/$1" 477 | shift 478 | echo "====== curl $url $@" >>"$LOG" 479 | curl -qs "$url" "$@" 2>>"$LOG" 480 | echo >>"$LOG" 481 | echo >>"$LOG" 482 | } 483 | 484 | case "$1" in 485 | reactions) 486 | MESSAGE_ID="$2" 487 | for e in "👾" "🤔" "😎" 488 | do 489 | API setMessageReaction -F chat_id=$FROM -F message_id=$MESSAGE_ID -F reaction='[{"type":"emoji","emoji":"'"$e"'"}]' 490 | sleep 1 491 | done 492 | API setMessageReaction -F chat_id=$FROM -F message_id=$MESSAGE_ID -F reaction='[]' 493 | ;; 494 | editing) 495 | MESSAGE_ID="$(API_STDOUT sendMessage -F chat_id=$FROM -F text='Starting...' | jq .result.message_id)" 496 | if test -n "$MESSAGE_ID" 497 | then 498 | for i in 2 4 6 8 499 | do 500 | sleep 1 501 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text="Doing... ${i}0% complete..." 502 | done 503 | sleep 1 504 | API editMessageText -F chat_id=$FROM -F message_id="$MESSAGE_ID" -F text='Done.' 505 | else 506 | echo "cannot obtain message id" 507 | fi 508 | ;; 509 | *) 510 | echo 'invalid mode' 511 | ;; 512 | esac 513 | ``` 514 | 515 | Restart bot with this configuration (`mybot.env`): 516 | 517 | ```ini 518 | tb_token = 'TOKEN' 519 | tb_script = ./mybot.sh 520 | tb_long_running_script = ./mybot_long.sh 521 | tb_ctrl_addr = :9999 522 | ``` 523 | 524 | Like that: 525 | 526 | ```sh 527 | # if you install it 528 | cnbot mybot.env 529 | # if you start it without installing, just from sources 530 | go run ./cmd/cnbot/... mybot.env 531 | ``` 532 | 533 | > [!NOTE] 534 | > Please note when you are modifying script, all changes takes effect immediately. You don't need to restart the bot engine. 535 | > You have to restart the bot engine if you want to change its environment variables only. 536 | 537 | Try to talk to your bot. Now it recognizes commands and shows you many different possibilities. 538 | 539 | Let me explain what is happening in this examples step by step. 540 | 541 | ### Script structure 542 | 543 | You wouldn't be mistaken for thinking that this script is slightly awkward. It is written that way 544 | to be more splittable. We will consider better structure further. 545 | 546 | ### Helpers overview 547 | 548 | Let's briefly touch on two helpers functions we are using in this scripts. 549 | 550 | Both of them helps you to call bot engine API (not Telegram API, but bot engine). 551 | 552 | `API_STDOUT()` takes it's first argument as a tail of API URL and consider all the rest of arguments 553 | as `curl`'s arguments. For example, `API_STDOUT getMe` means literally 554 | `curl -qs "http://localhost$tg_x_ctrl_addr/getMe"`. 555 | 556 | `API_STDOUT()` throws it's output to `stdout`, `API()` doesn't though. 557 | `API "?to=$FROM" -d 'OK'` means `curl -qs "http://localhost$tg_x_ctrl_addr/?to=$FROM -d 'OK'` 558 | 559 | Both of them logs their output to `$LOG` file. 560 | 561 | ### Commands 562 | 563 | This script recognizes several commands. We already consider the following commands: 564 | 565 | - `debug` — it's our first script 566 | - `about` — just call `getMe` API method. You can also see how we use `API_STDOUT` helper 567 | - `two` — shows how to send asynchronous message from script. We saw how to do it from command line before. You can also see how we use `API` helper 568 | - `buttons` — message with buttons as we saw before 569 | - `image` — shows how to send image. Just throw it to `stdout` and bot engine will recognize that it is image and send it in proper way 570 | 571 | All the rest commands we will consider further. 572 | 573 | ## Advanced topics 574 | 575 | ### Configuration details and driving multiple bots 576 | 577 | You are already seeing the bot can be configured by configuration file and directory by environment variable. 578 | 579 | Environment has higher priority. 580 | 581 | All variables have the same structure: `tb_{MEANING}` or `tb_{BOTNAME}_{MEANING}` if you need to start several bots. 582 | 583 | To configure bot `x` and `y`, you need to pass this variable to `cnbot`: 584 | 585 | ```sh 586 | tb_x_token='TOKEN_X' 587 | tb_x_script=/usr/bin/echo 588 | tb_x_long_running_script=/usr/bin/echo 589 | tb_x_ctrl_addr=:9999 590 | 591 | tb_y_token='TOKEN_Y' 592 | tb_y_script=/usr/bin/echo 593 | tb_y_long_running_script=/usr/bin/echo 594 | tb_y_ctrl_addr=:9998 595 | ``` 596 | 597 | ### Arguments processing 598 | 599 | Bot engine runs your scripts with command line arguments. It can be useful for small bots. 600 | 601 | Arguments prepared from messages, captions and callback's data. Strings are cast to lower-case, cleaned of control characters and split by white spaces. 602 | 603 | For example the message `$Hello world!` will be represented as two arguments `hello` and `world`. 604 | 605 | Following characters will be removed from the arguments: ``!"#$&'()*+-./:;<=>?@[\]`|``. 606 | 607 | ### Environment details 608 | 609 | #### Turning telegram payload to environment variables 610 | 611 | Bot engine converts every [JSON-update](https://core.telegram.org/bots/api#update) to flat set of environment variables this way: 612 | 613 | ```json 614 | { 615 | "ok": true, 616 | "result": [ 617 | { 618 | "message": { 619 | "caption": "Hi!", 620 | "chat": { 621 | "first_name": "Alexey", 622 | "id": 150000000, 623 | "last_name": "Michurin", 624 | "type": "private", 625 | "username": "AlexeyMichurin" 626 | }, 627 | "date": 1600000000, 628 | "from": { 629 | "first_name": "Alexey", 630 | "id": 150000000, 631 | "is_bot": false, 632 | "language_code": "en", 633 | "last_name": "Michurin", 634 | "username": "AlexeyMichurin" 635 | }, 636 | "message_id": 2222, 637 | "photo": [ 638 | { 639 | "file_id": "aaa0", 640 | "file_size": 2444, 641 | "file_unique_id": "id0", 642 | "height": 90, 643 | "width": 90 644 | }, 645 | { 646 | "file_id": "aaa1", 647 | "file_size": 4888, 648 | "file_unique_id": "id1", 649 | "height": 128, 650 | "width": 128 651 | } 652 | ] 653 | }, 654 | "update_id": 500000000 655 | } 656 | ] 657 | } 658 | ``` 659 | 660 | turns to the following environment variables: 661 | 662 | ```ini 663 | tg_message_caption=Hi! 664 | tg_message_chat_first_name=Alexey 665 | tg_message_chat_id=150000000 666 | tg_message_chat_last_name=Michurin 667 | tg_message_chat_type=private 668 | tg_message_chat_username=AlexeyMichurin 669 | tg_message_date=1600000000 670 | tg_message_from_first_name=Alexey 671 | tg_message_from_id=150000000 672 | tg_message_from_is_bot=false 673 | tg_message_from_language_code=en 674 | tg_message_from_last_name=Michurin 675 | tg_message_from_username=AlexeyMichurin 676 | tg_message_message_id=2222 677 | tg_message_photo=tg_message_photo_0 tg_message_photo_1 678 | tg_message_photo_0_file_id=aaa0 679 | tg_message_photo_0_file_size=2444 680 | tg_message_photo_0_file_unique_id=id0 681 | tg_message_photo_0_height=90 682 | tg_message_photo_0_width=90 683 | tg_message_photo_1_file_id=aaa1 684 | tg_message_photo_1_file_size=4888 685 | tg_message_photo_1_file_unique_id=id1 686 | tg_message_photo_1_height=128 687 | tg_message_photo_1_width=128 688 | tg_update_id=500000000 689 | ``` 690 | 691 | #### Build-in variables (`x`-variables) 692 | 693 | Engine provides the following additional variables: 694 | 695 | - `tg_x_build` 696 | - `tg_x_ctrl_addr` 697 | - `tg_x_to` (long-running scripts only) 698 | 699 | #### System variables 700 | 701 | > [!NOTE] 702 | > Beware. Bot engine does *NOT* convey its environment to child scripts. 703 | 704 | Bot engine does not transfer environment to child scripts. It is conscious decision cause it helps to 705 | make script's behavior more predictable and reproducible. Variables like `$PATH`, `$LANG`, `$LS_ALL` can 706 | change behavior of many commands and functions. It can lead to hard to debug behavior. 707 | 708 | If you need to have some environment variables, just set them in you script explicitly. 709 | 710 | ### Working directory 711 | 712 | Current working directory is directory, where the script is located in. 713 | 714 | ### Process management: concurrency, timeouts, signals, long-running tasks 715 | 716 | #### Ordinary tasks 717 | 718 | Bot engine generates all tasks of the same bot run strictly concurrently. It means you can use 719 | shared resources like files without any doubts. And your tasks have to finish in short time. 720 | 721 | Bot engine will send `SIGTERM` to task after 10 seconds, and `SIGKILL` after next 10 seconds. 722 | 723 | #### Long-running tasks 724 | 725 | Long-running tasks can be executed simultaneously though. 726 | 727 | They also have timeouts: 10 minutes. 728 | 729 | ### Uploading and downloading 730 | 731 | To upload something (image, video, audio, etc) you can just throw it stdout of your script. 732 | If you need to add capture or group multimedia files in one message, you need to call 733 | Telegram API. As usual, you don't need to care about secrets etc just use `cnbot` control handler as we did above. 734 | 735 | To download attachments (file, video, audio, photos, etc) you have to use `file_id` from message and 736 | just perform `GET` request to control handler with `file_id=...` in query string. See action `invert` 737 | in example above. 738 | 739 | ## Tips and tricks 740 | 741 | ### Improved script structure and security aspects 742 | 743 | ```sh 744 | # --- global variables 745 | ... 746 | # --- helper variables 747 | ... 748 | # --- must have commands 749 | case $1 in 750 | start) 751 | echo "Hello message" 752 | exit 753 | ;; 754 | privacy) # https://telegram.org/tos/bot-developers#4-privacy 755 | echo "This bot does not collect or share any personal information." 756 | exit 757 | esac 758 | # --- whitelist checks for user_id 759 | # it is just example: 760 | # - allows.list have contains strings line "_${ID}_" (it makes you able to write comments and things like that) 761 | # - we consider messages and callbacks 762 | if grep "_${tg_message_from_id}${tg_callback_query_from_id}_" allows.list 2>&1 >/dev/null 763 | then 764 | : # pass this user, you may want to log it 765 | else 766 | echo 'You are not allowd' 767 | exit 768 | fi 769 | # --- process text messages 770 | if [ -n "$tg_message_text" ] 771 | then 772 | case "$1" in 773 | ... 774 | esac 775 | exit 776 | fi 777 | # --- process images 778 | if [ -n "$tg_message_photo" ] 779 | then 780 | case "$1" in 781 | ... 782 | esac 783 | exit 784 | fi 785 | # --- process voices (for instance) 786 | if [ -n "$tg_message_voice_file_id" ] 787 | then 788 | ... 789 | exit # don't forget to exit 790 | fi 791 | # --- process callbacks 792 | if [ -n "$tg_callback_query_data" ] 793 | then 794 | ... 795 | exit 796 | fi 797 | # process... whatever you want 798 | if ... 799 | ... 800 | exit 801 | fi 802 | ``` 803 | 804 | Of course, it is good idea to split script, using `source file.sh` instruction. 805 | And you are still able to use other languages and approaches for sure. 806 | 807 | > [!CAUTION] 808 | > Just don't forget to be careful, keep in mind that anybody in internet can send anything to your bot. 809 | > 810 | > Keep reading. We will consider how to protect your bot. 811 | 812 | ### Debugging wrapper 813 | 814 | To debug your scripts, you can use this wrapper. Tune `$CMD`, and enjoy 815 | full logging: arguments, environment, out and err streams, exit code. 816 | 817 | ```sh 818 | #!/bin/sh 819 | 820 | # put your command here 821 | CMD=./mybot.py 822 | 823 | # tune naming for your taste 824 | base="logs/$(date +%s-)_${$}_" 825 | ext='.log' 826 | 827 | n=0 828 | for a in "$@" 829 | do 830 | echo "$a" >"${base}arg_${n}${ext}" 831 | n="$(($n+1))" 832 | done 833 | 834 | env | sort >"${base}env${ext}" 835 | 836 | set -o pipefail 837 | 838 | "$CMD" "$@" 2>"${base}err${ext}" | tee "${base}out${ext}" 839 | 840 | code="$?" 841 | 842 | echo "$code" >"${base}status${ext}" 843 | exit "$code" 844 | ``` 845 | 846 | ## System administration topics 847 | 848 | ### Installation 849 | 850 | ```sh 851 | ./build.sh 852 | sudo install ./cnbot /usr/bin 853 | ``` 854 | 855 | ### Running 856 | 857 | The process itself does not try to be immortal. It dies on fatal issues that can not be solved by process itself. Like network problems. 858 | It is believed that the process will be restart by `systemd` or stuff like that according the proper way with timeouts, logging, notifications, alerting. 859 | 860 | Systemd unit file example (`/etc/systemd/system/cnbot.service`): 861 | 862 | ```ini 863 | [Unit] 864 | Description=Telegram bot (cnbot) service 865 | Documentation=https://github.com/michurin/cnbot 866 | After=network.target nss-lookup.target 867 | 868 | [Service] 869 | Type=simple 870 | Restart=always 871 | RestartSec=1 872 | User=nobody 873 | ExecStart=/usr/bin/cnbot /etc/cnbot-config.env 874 | 875 | [Install] 876 | WantedBy=multi-user.target 877 | ``` 878 | 879 | ## Known issues 880 | 881 | - Some engine API methods are using both POST-body and query parameters. It's against standards. However I haven't invented something more convenient and standard yet. 882 | - Engine API uses non-standard method `RUN`. It allows by standards, however it doesn't seem inevitable. 883 | - Engine uses [`mime.ExtensionsByType()`](https://pkg.go.dev/mime#ExtensionsByType) to detect extensions for multimedia attachments. This function relies on the system configuration. It's highly recommended to install package like `shared-mime-info`. Pleas keep it in mind when you build production docker images and deploy the engine to remote servers. 884 | - Integration tests rely exclusively on `bash` rather than any other shell. Simple `sh` won't work in most cases. 885 | - Tests also rely on `curl`. 886 | - Engine doesn't retry any requests to Telegram API. Looks like issue. However, Telegram API doesn't provide any idempotency keys, and engine doesn't save state between restarts. It seems you have to solve this issue somehow else. 887 | - It hasn't been tested on MS Windows and FreeBSD. 888 | - The engine doesn't support persistent storage. You have to save state if you need by yourself. 889 | - Engine consider kill signals as errors. So it's final log message is error mostly. It is confusing. 890 | - Right now code has a lot of public types, methods and functions. I want this code to be able to be embedded and integrated. However, public API needs to be reviewed. 891 | 892 | ## Developing and contributing 893 | 894 | ### Main ideas 895 | 896 | - Contract must be simple and flexible 897 | - New features of [Telegram bot API](https://core.telegram.org/bots/api) has to be available instantly without changing of code of the bot 898 | - Bot has to manage subprocesses: timeouts, etc 899 | - Bot has to manage API call: [rate limits](https://core.telegram.org/bots/faq#my-bot-is-hitting-limits-how-do-i-avoid-this), etc 900 | - Configuration must be simple 901 | - Code must be testable and has to be covered 902 | - Functionality has to be observable and has to provide ability to add metrics and monitoring by adding middleware without code changing 903 | - The engine tries to be case insensitive considering environment variables. It can lead to false warnings 904 | 905 | ### Deep debugging 906 | 907 | Run proxy. For example [mitmproxy](https://mitmproxy.org/): 908 | 909 | ```sh 910 | mitmdump --flow-detail 4 -p 9001 --mode reverse:https://api.telegram.org 911 | ``` 912 | 913 | Instruct the bot to use proxy and run it: 914 | 915 | ```sh 916 | export tb_api_origin=http://localhost:9001 917 | ./cnbot ... # run bot, it will deal with Telegram API through the proxy and you will see everything 918 | ``` 919 | 920 | ### Application structure 921 | 922 | (horrible ASCII art warning) 923 | 924 | ``` 925 | Telegram infrastructure 926 | ^ ............. crons 927 | HTTP : HTTP : scripts 928 | : v any other 929 | .=BOT================================================. asynchronous 930 | | API | HTTP server for | 931 | |..........................| asynchronous messaging | 932 | | polling for : sending | | 933 | | updates : messages <-- send data from req | 934 | `====================================================' 935 | | ^ ^ send stdout | 936 | | | `---------. | request params 937 | | message | send | | as command line positional args 938 | v data | stdout | v 939 | ........................ ...................... 940 | : run script for every : : long-running : 941 | : message : : script : 942 | :......................: :....................: 943 | ``` 944 | --------------------------------------------------------------------------------