3 |
2 |
3 |
Tool managing Elasticsearch Index
5 | 6 | 7 |
[](https://godoc.org/github.com/po3rin/eskeeper) 
8 |
9 | eskeeper synchronizes index and alias with configuration files while ensuring idempotency. It still only supports WRITE. DELETE is not yet supported because the operation of deleting persistent data is dangerous and needs to be implemented carefully.
10 |
11 | :clipboard: [A Tour of eskeeper](https://github.com/po3rin/eskeeper/blob/main/example/README.md) explains more detail usage.
12 |
13 | ## :muscle: Currently supports
14 |
15 | ### mode
16 |
17 | - [x] CLI mode
18 | - [ ] Agent mode
19 |
20 | ### sync
21 |
22 | * index
23 | - [x] create
24 | - [x] status (open or close)
25 | - [x] reindex (only basic parameter)
26 | - [x] status(open/close only)
27 | - [ ] lifecycke
28 | - [ ] update mapping
29 | - [ ] delete
30 |
31 | * alias
32 | - [x] create
33 | - [x] update
34 | - [ ] delete
35 |
36 | ## :four_leaf_clover: How to use
37 |
38 | :clipboard: [A Tour of eskeeper](https://github.com/po3rin/eskeeper/blob/main/example/README.md) explains more detail usage.
39 |
40 | ### Quick Start
41 |
42 | eskeeper recieves yaml format data from stdin.
43 |
44 | ```bash
45 | $ go install github.com/po3rin/eskeeper/cmd/eskeeper
46 | $ eskeeper < es.yaml
47 | ```
48 |
49 | es.yaml is indices & aliases config file.
50 | Below is an example of es.yaml.
51 |
52 | ```yaml
53 | index:
54 | - name: test-v1 # index name
55 | mapping: testdata/test.json # index setting & mapping (json)
56 |
57 | - name: test-v2
58 | mapping: testdata/test.json
59 |
60 | - name: close-v1
61 | mapping: testdata/test.json
62 | status: close
63 |
64 | # reindex test-v1 -> reindex-v1
65 | - name: reindex-v1
66 | mapping: testdata/test.json
67 | reindex:
68 | source: test-v1
69 | slices: 3 # default=1
70 | waitForCompletion: true
71 |
72 | # 'on' field supports 2 hooks.
73 | # 'firstCreated': only when index is created for the first time.
74 | # 'always': always exec reindex.
75 | on: firstCreated
76 |
77 |
78 | alias:
79 | - name: alias1
80 | index:
81 | - test-v1
82 |
83 | # multi indicies
84 | - name: alias2
85 | index:
86 | - test-v1
87 | - test-v2
88 | ```
89 |
90 | results
91 |
92 | ```bach
93 | curl localhost:9200/_cat/indices
94 | yellow open test-v1 ... 1 1 0 0 208b 208b
95 | yellow open test-v2 ... 1 1 0 0 208b 208b
96 | yellow close close-v1 xxxxxxxxxxxx 1 1
97 | yellow open reindex-v1 ... 1 1 0 0 208b 208b
98 |
99 | curl localhost:9200/_cat/aliases
100 | alias2 test-v2 - - - -
101 | alias1 test-v1 - - - -
102 | alias2 test-v1 - - - -
103 | ```
104 |
105 |
106 | ### :triangular_ruler: CLI Options
107 |
108 | eskeeper supports flag & environment value.
109 |
110 | ```bash
111 | # use flags
112 | eskeeper -u user -p pass -e=http://localhost:9200,http://localhost9300 < testdata/es.yaml
113 |
114 | # use env
115 | ESKEEPER_ES_USER=user ESKEEPER_ES_PASS=pass ESKEEPER_ES_URLS=http://localhost:9200 eskeeper < testdata/es.yaml
116 | ```
117 |
118 | eskeeper can also execute validation only with validate subcommand.
119 |
120 | ```bash
121 | eskeeper validate < testdata/es.yaml
122 | ```
123 |
124 | pre-check stage is slow processing. you can skip pre-check stage using -s flag.
125 |
126 | ```bash
127 | eskeeper -s < testdata/es.yaml
128 | ```
129 |
130 | ## :mag_right: Internals
131 |
132 | eskeeper process is divided into four stages. verbose option lets you know eskeeper details.
133 |
134 | ```
135 | $ eskeeper < es.yaml
136 | loading config ...
137 |
138 | === validation stage ===
139 | [pass] index: test-v1
140 | [pass] index: test-v2
141 | [pass] index: close-v1
142 | [pass] alias: alias1
143 | [pass] alias: alias2
144 |
145 | === pre-check stage ===
146 | [pass] index: test-v1
147 | [pass] index: test-v2
148 | [pass] index: close-v1
149 | [pass] alias: alias1
150 | [pass] alias: alias2
151 |
152 | === sync stage ===
153 | [synced] index: test-v1
154 | [synced] index: test-v2
155 | [synced] index: close-v1
156 | [synced] alias: alias1
157 | [synced] alias: alias2
158 |
159 | === post-check stage ===
160 | [pass] index: test-v1
161 | [pass] index: test-v2
162 | [pass] index: close-v1
163 | [pass] alias: alias1
164 | [pass] alias: alias2
165 |
166 | succeeded
167 | ```
168 |
169 | #### validation stage
170 | * Validates config yaml format
171 |
172 | #### pre-check stage
173 |
174 | * Check if mapping file is valid format
175 | * Check if there is an index for alias
176 |
177 | #### sync stage
178 | * Sync indices and aliases with config
179 |
180 | The order of synchronization is as follows.
181 |
182 | ```
183 | create index
184 | ↓
185 | open index
186 | ↓
187 | update alias
188 | ↓
189 | close index
190 | ```
191 |
192 | Index close operation should be done after switching the alias.
193 | Because there can be downtime before switching aliases.
194 |
195 | #### post-check stage
196 | * Check if indices & aliases has been created
197 |
198 |
199 | ## :triangular_flag_on_post: Contributing
200 |
201 | Did you find something technically wrong, something to fix, or something? Please give me Issue or Pull Request !!
202 |
203 | ### Test
204 |
205 | eskeeper's test uses [github.com/ory/dockertest](github.com/ory/dockertest). So you need docker to test.
206 |
--------------------------------------------------------------------------------
/alias.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "bytes"
5 | "context"
6 | "encoding/json"
7 | "fmt"
8 | "io/ioutil"
9 | )
10 |
11 | func (c *esclient) existAlias(ctx context.Context, alias string) (bool, error) {
12 | exists := c.client.Indices.ExistsAlias
13 |
14 | res, err := exists(
15 | []string{alias},
16 | exists.WithContext(ctx),
17 | )
18 | if err != nil {
19 | return false, fmt.Errorf("check alias exists: %w", err)
20 | }
21 | if res.StatusCode == 200 {
22 | return true, nil
23 | }
24 | return false, nil
25 | }
26 | func (c *esclient) syncAlias(ctx context.Context, alias alias) error {
27 | i := c.client.Indices
28 |
29 | query := aliasQuery(alias.Name, alias.Indices)
30 |
31 | var buf bytes.Buffer
32 | if err := json.NewEncoder(&buf).Encode(query); err != nil {
33 | return fmt.Errorf("build query: %w", err)
34 | }
35 |
36 | res, err := i.UpdateAliases(&buf, i.UpdateAliases.WithContext(ctx))
37 | if err != nil {
38 | return fmt.Errorf("upsert aliases: %w", err)
39 | }
40 | if res.StatusCode != 200 {
41 | body, err := ioutil.ReadAll(res.Body)
42 | if err != nil {
43 | return fmt.Errorf("failed to sync alias [alias=%v, statusCode=%v]", alias.Name, res.StatusCode)
44 | }
45 | return fmt.Errorf("failed to sync alias [alias=%v, statusCode=%v, res=%v]", alias.Name, res.StatusCode, string(body))
46 | }
47 | return nil
48 | }
49 |
50 | func (c *esclient) syncAliases(ctx context.Context, conf config) error {
51 | for _, alias := range conf.Aliases {
52 | err := c.syncAlias(ctx, alias)
53 | if err != nil {
54 | c.logf("[fail] alias: %v\n", alias.Name)
55 | return fmt.Errorf("sync alias: %w", err)
56 | }
57 | c.logf("[synced] alias: %v\n", alias.Name)
58 | }
59 | return nil
60 | }
61 |
62 | func aliasQuery(aliasName string, indices []string) map[string]interface{} {
63 | Actions := make([]map[string]interface{}, 0, len(indices)+1)
64 |
65 | Actions = append(Actions, map[string]interface{}{
66 | "remove": map[string]interface{}{
67 | "index": "*",
68 | "alias": aliasName,
69 | },
70 | })
71 |
72 | for _, index := range indices {
73 | Actions = append(Actions, map[string]interface{}{
74 | "add": map[string]interface{}{
75 | "index": index,
76 | "alias": aliasName,
77 | },
78 | })
79 | }
80 |
81 | return map[string]interface{}{
82 | "actions": Actions,
83 | }
84 | }
85 |
--------------------------------------------------------------------------------
/alias_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "testing"
6 | "time"
7 | )
8 |
9 | func TestExistAlias(t *testing.T) {
10 | tests := []struct {
11 | name string
12 | alias string
13 | setup func(tb testing.TB)
14 | want bool
15 | cleanup func(tb testing.TB)
16 | }{
17 | {
18 | name: "simple",
19 | alias: "alias-exist-v1",
20 | setup: func(tb testing.TB) {
21 | createTmpIndexHelper(tb, "alias-exist-index-v1")
22 | createTmpIndexHelper(tb, "alias-exist-index-v2")
23 | time.Sleep(10 * time.Second)
24 | createTmpAliasHelper(tb, "alias-exist-v1", "alias-exist-index-v1")
25 | createTmpAliasHelper(tb, "alias-exist-v1", "alias-exist-index-v2")
26 | },
27 | want: true,
28 | },
29 | {
30 | name: "not-found",
31 | alias: "not-found",
32 | setup: func(tb testing.TB) {
33 | },
34 | want: false,
35 | },
36 | }
37 |
38 | es, err := newEsClient([]string{url}, "", "")
39 | if err != nil {
40 | t.Fatal(err)
41 | }
42 |
43 | for _, tt := range tests {
44 | t.Run(tt.name, func(t *testing.T) {
45 | ctx := context.Background()
46 | tt.setup(t)
47 | ok, err := es.existAlias(ctx, tt.alias)
48 | if err != nil {
49 | t.Error(err)
50 | }
51 | if ok != tt.want {
52 | t.Errorf("want: %+v, got: %+v\n", tt.want, ok)
53 | }
54 | })
55 | }
56 | }
57 |
58 | func TestSyncAliases(t *testing.T) {
59 | tests := []struct {
60 | name string
61 | conf config
62 | setup func(tb testing.TB)
63 | cleanup func(tb testing.TB)
64 | }{
65 | {
66 | name: "simple",
67 | conf: config{
68 | Aliases: []alias{
69 | {
70 | Name: "test-sync-alias",
71 | Indices: []string{"test-v1", "test-v2"},
72 | },
73 | },
74 | },
75 | setup: func(tb testing.TB) {
76 | createTmpIndexHelper(tb, "test-v1")
77 | createTmpIndexHelper(tb, "test-v2")
78 | },
79 | },
80 | {
81 | name: "switch",
82 | conf: config{
83 | Aliases: []alias{
84 | {
85 | Name: "test-sync-alias",
86 | Indices: []string{"test-v3"},
87 | },
88 | },
89 | },
90 | setup: func(tb testing.TB) {
91 | createTmpIndexHelper(tb, "test-v3")
92 | },
93 | },
94 | }
95 |
96 | es, err := newEsClient([]string{url}, "", "")
97 | if err != nil {
98 | t.Fatal(err)
99 | }
100 |
101 | for _, tt := range tests {
102 | t.Run(tt.name, func(t *testing.T) {
103 | ctx := context.Background()
104 | tt.setup(t)
105 | err := es.syncAliases(ctx, tt.conf)
106 | if err != nil {
107 | t.Error(err)
108 | }
109 | })
110 | }
111 | }
112 |
--------------------------------------------------------------------------------
/beta.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "bytes"
5 | "context"
6 | "encoding/json"
7 | "errors"
8 | "fmt"
9 | "io/ioutil"
10 |
11 | "github.com/Cside/jsondiff"
12 | )
13 |
14 | type indexConfigWithName map[string]indexConfig
15 |
16 | type indexConfig struct {
17 | Settings map[string]interface{} `json:"settings"`
18 | Mappings map[string]interface{} `json:"mappings"`
19 | }
20 |
21 | func (c *esclient) index(ctx context.Context, index index) ([]byte, error) {
22 | get := c.client.Indices.Get
23 | res, err := get(
24 | []string{index.Name},
25 | get.WithContext(ctx),
26 | get.WithFlatSettings(true),
27 | )
28 | if err != nil {
29 | return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
30 | }
31 | if res.StatusCode != 200 {
32 | body, err := ioutil.ReadAll(res.Body)
33 | if err != nil {
34 | return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
35 | }
36 | return nil, fmt.Errorf("get %v settings: %v", index.Name, string(body))
37 | }
38 |
39 | b, err := ioutil.ReadAll(res.Body)
40 | if err != nil {
41 | return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
42 | }
43 | return b, nil
44 | }
45 |
46 | // func (c *esclient) settings(ctx context.Context, index index) ([]byte, error) {
47 | // getSettings := c.client.Indices.GetSettings
48 | // res, err := getSettings(
49 | // getSettings.WithIndex(index.Name),
50 | // getSettings.WithContext(ctx),
51 | // )
52 | // if err != nil {
53 | // return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
54 | // }
55 | // if res.StatusCode != 200 {
56 | // body, err := ioutil.ReadAll(res.Body)
57 | // if err != nil {
58 | // return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
59 | // }
60 | // return nil, fmt.Errorf("get %v settings: %v", index.Name, string(body))
61 | // }
62 |
63 | // b, err := ioutil.ReadAll(res.Body)
64 | // if err != nil {
65 | // return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
66 | // }
67 | // return b, nil
68 | // }
69 |
70 | // func (c *esclient) mapping(ctx context.Context, index index) ([]byte, error) {
71 | // getMapping := c.client.Indices.GetMapping
72 | // res, err := getMapping(
73 | // getMapping.WithIndex(index.Name),
74 | // getMapping.WithContext(ctx),
75 | // )
76 | // if err != nil {
77 | // return nil, fmt.Errorf("get %v mapping: %w", index.Name, err)
78 | // }
79 | // if res.StatusCode != 200 {
80 | // body, err := ioutil.ReadAll(res.Body)
81 | // if err != nil {
82 | // return nil, fmt.Errorf("get %v mapping: %w", index.Name, err)
83 | // }
84 | // return nil, fmt.Errorf("get %v mapping: %v", index.Name, string(body))
85 | // }
86 |
87 | // b, err := ioutil.ReadAll(res.Body)
88 | // if err != nil {
89 | // return nil, fmt.Errorf("get %v settings: %w", index.Name, err)
90 | // }
91 | // fmt.Println(string(b))
92 | // return b, nil
93 | // }
94 |
95 | // func (c *esclient) equalSettings(ctx context.Context, index index, settingJSON []byte) (bool, error) {
96 | // m, err := c.settings(ctx, index)
97 | // if err != nil {
98 | // return false, fmt.Errorf("get setting: %w", err)
99 | // }
100 |
101 | // if diff := jsondiff.Diff(m, settingJSON); diff != "" {
102 | // // fmt.Printf("detect settings diff:\n%s", diff)
103 | // return false, nil
104 | // }
105 | // return true, nil
106 | // }
107 |
108 | func (c *esclient) equalMappingsProperties(ctx context.Context, index index, mappingsJSON []byte) (bool, error) {
109 | res, err := c.index(ctx, index)
110 | if err != nil {
111 | return false, fmt.Errorf("get mappings: %w", err)
112 | }
113 |
114 | gotConf := make(indexConfigWithName, 0)
115 |
116 | err = json.Unmarshal(res, &gotConf)
117 | if err != nil {
118 | return false, fmt.Errorf("unmarshal mapping json: %w", err)
119 | }
120 |
121 | v, ok := gotConf[index.Name]
122 | if !ok {
123 | return false, errors.New("get index response dose not contain index name field")
124 | }
125 |
126 | b, err := json.Marshal(v.Mappings)
127 | if err != nil {
128 | return false, fmt.Errorf("marshal properties json: %w", err)
129 | }
130 |
131 | if diff := jsondiff.Diff(b, mappingsJSON); diff != "" {
132 | // fmt.Printf("detect mapping diff:\n%s", diff)
133 | return false, nil
134 |
135 | }
136 | return true, nil
137 | }
138 |
139 | func (c *esclient) updateIndex(ctx context.Context, index index) error {
140 | putMapping := c.client.Indices.PutMapping
141 | // putSetting := c.client.Indices.PutSettings
142 |
143 | b, err := ioutil.ReadFile(index.Mapping)
144 | if err != nil {
145 | return fmt.Errorf("open mapping file: %w", err)
146 | }
147 |
148 | config := &indexConfig{}
149 | err = json.Unmarshal(b, config)
150 | if err != nil {
151 | return fmt.Errorf("unmarshal mapping json: %w", err)
152 | }
153 |
154 | // mapping --------
155 | if config.Mappings == nil {
156 | // dose not contain
157 | // return fmt.Errorf("get mappings from file: %w", err)
158 | } else {
159 | j, err := json.Marshal(config.Mappings)
160 | if err != nil {
161 | return fmt.Errorf("marshal mappings json: %w", err)
162 | }
163 |
164 | ok, err := c.equalMappingsProperties(ctx, index, j)
165 | if err != nil {
166 | return err
167 | }
168 |
169 | if !ok {
170 | res, err := putMapping(
171 | bytes.NewReader(j),
172 | putMapping.WithIndex(index.Name),
173 | putMapping.WithContext(ctx),
174 | )
175 | if err != nil {
176 | return fmt.Errorf("get %v mapping: %w", index, err)
177 | }
178 | if res.StatusCode != 200 {
179 | body, err := ioutil.ReadAll(res.Body)
180 | if err != nil {
181 | return fmt.Errorf("update %v mapping: %w", index.Name, err)
182 | }
183 | return fmt.Errorf("update %v mapping: %v", index.Name, string(body))
184 | }
185 | }
186 | }
187 |
188 | // // setting -------
189 | // if config.Settings == nil {
190 | // // dose not contain
191 | // // return fmt.Errorf("get settings from file: %w", err)
192 | // return nil
193 | // }
194 |
195 | // j, err := json.Marshal(config.Settings)
196 | // if err != nil {
197 | // return fmt.Errorf("marshal mappings json: %w", err)
198 | // }
199 |
200 | // ok, err := c.equalSettings(ctx, index, j)
201 | // if err != nil {
202 | // return err
203 | // }
204 | // if ok {
205 | // return nil
206 | // }
207 |
208 | // res, err := putSetting(
209 | // bytes.NewReader(j),
210 | // putSetting.WithIndex(index.Name),
211 | // putSetting.WithContext(ctx),
212 | // )
213 | // if err != nil {
214 | // return fmt.Errorf("update %v setting: %w", index, err)
215 | // }
216 | // if res.StatusCode != 200 {
217 | // body, err := ioutil.ReadAll(res.Body)
218 | // if err != nil {
219 | // return fmt.Errorf("update %v setting: %w", index.Name, err)
220 | // }
221 | // return fmt.Errorf("update %v setting: %v", index.Name, string(body))
222 | // }
223 |
224 | return nil
225 | }
226 |
--------------------------------------------------------------------------------
/beta_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "testing"
6 | )
7 |
8 | func TestUpdateIndex(t *testing.T) {
9 | tests := []struct {
10 | name string
11 | index index
12 | setup func(tb testing.TB)
13 | wantErr bool
14 | }{
15 | {
16 | name: "simple",
17 | index: index{
18 | Name: "update-test-v1",
19 | Mapping: "testdata/updateIndex.json",
20 | },
21 | setup: func(tb testing.TB) {
22 | createTmpIndexHelper(tb, "update-test-v1")
23 | },
24 | wantErr: false,
25 | },
26 | // {
27 | // name: "same",
28 | // index: index{
29 | // Name: "update-test-v2",
30 | // Mapping: "testdata/test.json",
31 | // },
32 | // setup: func(tb testing.TB) {
33 | // createTmpIndexHelper(tb, "update-test-v2")
34 | // },
35 | // wantErr: false,
36 | // },
37 | // {
38 | // name: "invalid-update",
39 | // index: index{
40 | // Name: "update-test-v3",
41 | // Mapping: "testdata/invalidUpdateIndex.json",
42 | // },
43 | // setup: func(tb testing.TB) {
44 | // createTmpIndexHelper(tb, "update-test-v3")
45 | // },
46 | // wantErr: true,
47 | // },
48 | }
49 |
50 | es, err := newEsClient([]string{url}, "", "")
51 | if err != nil {
52 | t.Fatal(err)
53 | }
54 |
55 | for _, tt := range tests {
56 | t.Run(tt.name, func(t *testing.T) {
57 | ctx := context.Background()
58 | tt.setup(t)
59 | err := es.updateIndex(ctx, tt.index)
60 | if tt.wantErr && err == nil {
61 | t.Error("expect error")
62 | }
63 | if !tt.wantErr && err != nil {
64 | t.Error(err)
65 | }
66 | })
67 | }
68 | }
69 |
--------------------------------------------------------------------------------
/check.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "fmt"
6 |
7 | "github.com/gofrs/uuid"
8 | )
9 |
10 | func (c *esclient) preCheckIndex(ctx context.Context, ix index) error {
11 | // generate uuid for pre-check create index
12 | u2, err := uuid.NewV4()
13 | if err != nil {
14 | return fmt.Errorf("generate UUID for pre-check: %w", err)
15 | }
16 |
17 | preIndex := index{
18 | Name: fmt.Sprintf("eskeeper-%s", u2.String()),
19 | Mapping: ix.Mapping,
20 | }
21 |
22 | err = c.syncIndex(ctx, preIndex)
23 | if err != nil {
24 | return fmt.Errorf("pre-check: pre create using random name index: %w", err)
25 | }
26 |
27 | err = c.deleteIndex(ctx, preIndex.Name)
28 | if err != nil {
29 | return fmt.Errorf("pre-check: delete pre-created index: %w", err)
30 | }
31 | return nil
32 | }
33 |
34 | func (c *esclient) preCheckAlias(ctx context.Context, alias alias, createIndices map[string]struct{}) error {
35 | // TODO: check duplicated name
36 | // ok, err := c.existIndex(ctx, alias.Name)
37 | // if err != nil {
38 | // return fmt.Errorf("pre-check: checks for duplicate index and alias names: %w", err)
39 | // }
40 | // if ok {
41 | // return fmt.Errorf("pre-check: detects duplicate index and alias names %v", alias.Name)
42 | // }
43 |
44 | for _, index := range alias.Indices {
45 | if _, ok := createIndices[index]; ok {
46 | continue
47 | }
48 | ok, err := c.existIndex(ctx, index)
49 | if err != nil {
50 | return fmt.Errorf("pre-check: check index %v exists for alias %v: %w", index, alias.Name, err)
51 | }
52 | if !ok {
53 | c.logf("[fail] alias: %v\n", alias.Name)
54 | return fmt.Errorf("pre-check: index %v for alias %v is not found", index, alias.Name)
55 | }
56 | }
57 |
58 | return nil
59 | }
60 |
61 | func (c *esclient) preCheck(ctx context.Context, conf config) error {
62 | // use alias pre-check
63 | createIndices := make(map[string]struct{}, 0)
64 |
65 | for _, ix := range conf.Indices {
66 | ok, err := c.existIndex(ctx, ix.Name)
67 | if err != nil {
68 | return fmt.Errorf("pre-check: check index %v exists: %w", ix.Name, err)
69 | }
70 | if ok {
71 | createIndices[ix.Name] = struct{}{}
72 | c.logf("[skip] index %v already exists\n", ix.Name)
73 | continue
74 | }
75 |
76 | createIndices[ix.Name] = struct{}{}
77 | err = c.preCheckIndex(ctx, ix)
78 | if err != nil {
79 | c.logf("[fail] index: %v\n", ix.Name)
80 | return err
81 | }
82 | c.logf("[pass] index: %v\n", ix.Name)
83 | }
84 |
85 | // check target index exists
86 | for _, alias := range conf.Aliases {
87 | err := c.preCheckAlias(ctx, alias, createIndices)
88 | if err != nil {
89 | c.logf("[fail] alias: %v\n", alias.Name)
90 | return err
91 | }
92 | c.logf("[pass] alias: %v\n", alias.Name)
93 | }
94 |
95 | return nil
96 | }
97 |
98 | // postCheck checks created index & alias by name only.
99 | func (c *esclient) postCheck(ctx context.Context, conf config) error {
100 | for _, index := range conf.Indices {
101 | ok, err := c.existIndex(ctx, index.Name)
102 | if err != nil {
103 | c.logf("[fail] index: %v\n", index.Name)
104 | return fmt.Errorf("post-check: check created index %v exist: %w", index.Name, err)
105 | }
106 | if !ok {
107 | c.logf("[fail] index: %v\n", index.Name)
108 | return fmt.Errorf("post-check: created index %v is not found", index.Name)
109 | }
110 | c.logf("[pass] index: %v\n", index.Name)
111 | }
112 |
113 | for _, alias := range conf.Aliases {
114 | ok, err := c.existAlias(ctx, alias.Name)
115 | if err != nil {
116 | c.logf("[fail] alias: %v\n", alias.Name)
117 | return fmt.Errorf("post-check: check created alias %v exist: %w", alias.Name, err)
118 | }
119 | if !ok {
120 | c.logf("[fail] alias: %v\n", alias.Name)
121 | return fmt.Errorf("post-check: created alias %v is not found", alias.Name)
122 | }
123 | c.logf("[pass] alias: %v\n", alias.Name)
124 | }
125 | return nil
126 | }
127 |
--------------------------------------------------------------------------------
/check_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "testing"
6 | )
7 |
8 | func TestPreCheck(t *testing.T) {
9 | tests := []struct {
10 | name string
11 | conf config
12 | wantErr bool
13 | }{
14 | {
15 | name: "simple",
16 | conf: config{
17 | Indices: []index{
18 | {
19 | Name: "precheck1",
20 | Mapping: "testdata/test.json",
21 | },
22 | },
23 | },
24 | wantErr: false,
25 | },
26 | {
27 | name: "invalid-field",
28 | conf: config{
29 | Indices: []index{
30 | {
31 | Name: "precheck2",
32 | Mapping: "testdata/invalid.json",
33 | },
34 | },
35 | },
36 | wantErr: true,
37 | },
38 | {
39 | name: "not-found-index",
40 | conf: config{
41 | Indices: []index{
42 | {
43 | Name: "precheck3",
44 | Mapping: "testdata/invalid.json",
45 | },
46 | },
47 | Aliases: []alias{
48 | {
49 | Name: "precheck-alias",
50 | Indices: []string{
51 | "precheck3",
52 | "precheck4", // not found
53 | },
54 | },
55 | },
56 | },
57 | wantErr: true,
58 | },
59 | // {
60 | // name: "duplicated name",
61 | // conf: config{
62 | // Indices: []index{
63 | // {
64 | // Name: "duplicated-name",
65 | // Mapping: "testdata/test.json",
66 | // },
67 | // },
68 | // Aliases: []alias{
69 | // {
70 | // Name: "duplicated-name",
71 | // Indices: []string{
72 | // "duplicated-name",
73 | // },
74 | // },
75 | // },
76 | // },
77 | // wantErr: true,
78 | // },
79 | }
80 |
81 | es, err := newEsClient([]string{url}, "", "")
82 | if err != nil {
83 | t.Fatal(err)
84 | }
85 |
86 | for _, tt := range tests {
87 | t.Run(tt.name, func(t *testing.T) {
88 | ctx := context.Background()
89 | err := es.preCheck(ctx, tt.conf)
90 | if tt.wantErr && err == nil {
91 | t.Error("expect error")
92 | }
93 | if !tt.wantErr && err != nil {
94 | t.Error(err)
95 | }
96 | })
97 | }
98 | }
99 |
--------------------------------------------------------------------------------
/cmd/eskeeper/main.go:
--------------------------------------------------------------------------------
1 | package main
2 |
3 | import (
4 | "context"
5 | "fmt"
6 | "os"
7 |
8 | "github.com/po3rin/eskeeper"
9 | "github.com/spf13/cobra"
10 | "github.com/spf13/pflag"
11 | "github.com/spf13/viper"
12 | "golang.org/x/crypto/ssh/terminal"
13 | )
14 |
15 | var rootCmd = &cobra.Command{
16 | Use: "eskeeper",
17 | Short: "eskeeper synchronizes index and alias with configuration files while ensuring idempotency.",
18 | Run: func(cmd *cobra.Command, args []string) {
19 | k, err := eskeeper.New(
20 | viper.GetStringSlice("es_urls"),
21 | eskeeper.UserName(viper.GetString("es_user")),
22 | eskeeper.Pass(viper.GetString("es_pass")),
23 | eskeeper.Verbose(viper.GetBool("verbose")),
24 | eskeeper.SkipPreCheck(viper.GetBool("skip_precheck")),
25 | )
26 | if err != nil {
27 | fmt.Fprintln(os.Stdout, err)
28 | os.Exit(1)
29 | }
30 |
31 | if terminal.IsTerminal(int(os.Stdin.Fd())) {
32 | fmt.Fprintln(os.Stdout, "Currently does not support interactive mode")
33 | os.Exit(1)
34 | }
35 |
36 | ctx := context.Background()
37 | err = k.Sync(ctx, os.Stdin)
38 | if err != nil {
39 | fmt.Fprintln(os.Stdout, err)
40 | os.Exit(1)
41 | }
42 | },
43 | }
44 |
45 | var validate = &cobra.Command{
46 | Use: "validate",
47 | Short: "Validates config",
48 | Run: func(cmd *cobra.Command, args []string) {
49 | k, err := eskeeper.New(
50 | []string{},
51 | eskeeper.Verbose(true),
52 | )
53 | if err != nil {
54 | fmt.Fprintln(os.Stdout, err)
55 | os.Exit(1)
56 | }
57 |
58 | if terminal.IsTerminal(int(os.Stdin.Fd())) {
59 | fmt.Fprintln(os.Stdout, "Currently does not support interactive mode")
60 | os.Exit(1)
61 | }
62 |
63 | ctx := context.Background()
64 | err = k.Validate(ctx, os.Stdin)
65 | if err != nil {
66 | fmt.Fprintln(os.Stdout, err)
67 | os.Exit(1)
68 | }
69 | fmt.Println("pass")
70 | },
71 | }
72 |
73 | func init() {
74 | rootCmd.AddCommand(validate)
75 | viper.SetEnvPrefix("eskeeper")
76 | viper.AutomaticEnv()
77 |
78 | pflag.StringP("es_user", "u", "", "Elasticsearch user name")
79 | pflag.StringP("es_pass", "p", "", "Elasticsearch password")
80 | pflag.StringSliceP("es_urls", "e", []string{"http://localhost:9200"}, "Elasticserch endpoint URLs (comma delimited)")
81 | pflag.BoolP("verbose", "v", false, "Make the operation more talkative")
82 | pflag.BoolP("skip_precheck", "s", false, "Skip pre-check stage")
83 |
84 | viper.BindPFlags(pflag.CommandLine)
85 | }
86 |
87 | func main() {
88 | if err := rootCmd.Execute(); err != nil {
89 | fmt.Fprintln(os.Stdout, err)
90 | os.Exit(1)
91 | }
92 | }
93 |
--------------------------------------------------------------------------------
/conf.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "encoding/json"
5 | "errors"
6 | "fmt"
7 | "io"
8 | "io/ioutil"
9 |
10 | "github.com/goccy/go-yaml"
11 | )
12 |
13 | var status = map[string]struct{}{
14 | "open": struct{}{},
15 | "close": struct{}{},
16 | "": struct{}{}, // default
17 | }
18 |
19 | var reindexOn = map[string]struct{}{
20 | "always": struct{}{},
21 | "firstCreated": struct{}{},
22 | "": struct{}{}, // default
23 | }
24 |
25 | type config struct {
26 | Indices []index `json:"index"`
27 | Aliases []alias `json:"alias"` // supports close only
28 | }
29 |
30 | type index struct {
31 | Name string `json:"name"`
32 | Mapping string `json:"mapping"`
33 | Status string `json:"status"`
34 | Reindex reindex `json:"reindex"`
35 | }
36 |
37 | type reindex struct {
38 | Source string `json:"source"`
39 | Slices int `json:"slices"`
40 | WaitForCompletion bool `json:"waitForCompletion"`
41 | On string `json:"on"`
42 | }
43 |
44 | type alias struct {
45 | Name string `json:"name"`
46 | Indices []string `json:"index"`
47 | }
48 |
49 | func yaml2Conf(reader io.Reader) (config, error) {
50 | conf := config{}
51 |
52 | b, err := ioutil.ReadAll(reader)
53 | if err != nil {
54 | return conf, err
55 | }
56 |
57 | if err := yaml.Unmarshal(b, &conf); err != nil {
58 | return conf, err
59 | }
60 |
61 | return conf, nil
62 | }
63 |
64 | func validateIndex(index index) error {
65 | if index.Name == "" {
66 | return errors.New("index name is empty")
67 | }
68 | if index.Mapping != "" {
69 | m, err := ioutil.ReadFile(index.Mapping)
70 | if err != nil {
71 | return fmt.Errorf("read file %v: %w", index.Mapping, err)
72 | }
73 | // validate json format
74 | var jsonStr map[string]interface{}
75 | if err := json.Unmarshal(m, &jsonStr); err != nil {
76 | return fmt.Errorf("mapping json is invalid: %w", err)
77 | }
78 | }
79 | _, ok := status[index.Status]
80 | if !ok {
81 | return fmt.Errorf("unsupported status %v", index.Status)
82 | }
83 |
84 | if index.Reindex.Source != "" {
85 | if index.Status == "close" {
86 | return errors.New("unsupported close status and reindex cannot be used together")
87 | }
88 | _, ok := reindexOn[index.Reindex.On]
89 | if !ok {
90 | return fmt.Errorf("unsupported reindex hook %v. [always or firstCreated]", index.Reindex.On)
91 | }
92 | }
93 |
94 | return nil
95 | }
96 |
97 | func validateAlias(alias alias) error {
98 | if alias.Name == "" {
99 | return errors.New("alias name is empty")
100 | }
101 |
102 | if len(alias.Indices) == 0 {
103 | return fmt.Errorf("no indices in %v alias", alias.Name)
104 | }
105 |
106 | for _, index := range alias.Indices {
107 | if index == "" {
108 | return errors.New("index name is empty")
109 | }
110 | }
111 | return nil
112 | }
113 |
114 | func (e *Eskeeper) validateConfigFormat(c config) error {
115 | createIndices := make(map[string]struct{}, 0)
116 |
117 | for _, index := range c.Indices {
118 | _, exist := createIndices[index.Name]
119 | if exist {
120 | e.logf("[fail] index: %v\n", index.Name)
121 | return fmt.Errorf("duplicated index name %v", index.Name)
122 | }
123 |
124 | createIndices[index.Name] = struct{}{}
125 |
126 | err := validateIndex(index)
127 | if err != nil {
128 | e.logf("[fail] index: %v\n", index.Name)
129 | return fmt.Errorf("validate index: %w", err)
130 | }
131 |
132 | e.logf("[pass] index: %v\n", index.Name)
133 | }
134 |
135 | for _, alias := range c.Aliases {
136 | _, ok := createIndices[alias.Name]
137 | if ok {
138 | e.logf("[fail] alias: %v\n", alias.Name)
139 | return fmt.Errorf("alias name %v is a duplicate of an index name that already exists", alias.Name)
140 | }
141 |
142 | err := validateAlias(alias)
143 | if err != nil {
144 | e.logf("[fail] alias: %v\n", alias.Name)
145 | return fmt.Errorf("validate alias: %w", err)
146 | }
147 |
148 | e.logf("[pass] alias: %v\n", alias.Name)
149 | }
150 | return nil
151 | }
152 |
--------------------------------------------------------------------------------
/conf_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "os"
5 | "reflect"
6 |
7 | "testing"
8 | )
9 |
10 | func TestYaml2Conf(t *testing.T) {
11 | tests := []struct {
12 | name string
13 | yaml string
14 | want config
15 | }{
16 | {
17 | name: "simple",
18 | yaml: "testdata/es.yaml",
19 | want: config{
20 | Indices: []index{
21 | {
22 | Name: "test-v1",
23 | Mapping: "testdata/test.json",
24 | },
25 | {
26 | Name: "test-v2",
27 | Mapping: "testdata/test.json",
28 | },
29 | {
30 | Name: "close-v1",
31 | Mapping: "testdata/test.json",
32 | Status: "close",
33 | },
34 | },
35 | Aliases: []alias{
36 | {
37 | Name: "alias1",
38 | Indices: []string{"test-v1"},
39 | },
40 | {
41 | Name: "alias2",
42 | Indices: []string{"test-v1", "test-v2"},
43 | },
44 | },
45 | },
46 | },
47 | {
48 | name: "reindex",
49 | yaml: "testdata/es.reindex.yaml",
50 | want: config{
51 | Indices: []index{
52 | {
53 | Name: "test-v1",
54 | Mapping: "testdata/test.json",
55 | },
56 | {
57 | Name: "reindex-v1",
58 | Mapping: "testdata/test.json",
59 | Reindex: reindex{
60 | Source: "test-v1",
61 | Slices: 3,
62 | WaitForCompletion: true,
63 | On: "firstCreated",
64 | },
65 | },
66 | },
67 | Aliases: []alias{
68 | {
69 | Name: "alias1",
70 | Indices: []string{"test-v2"},
71 | },
72 | },
73 | },
74 | },
75 | }
76 |
77 | for _, tt := range tests {
78 | t.Run(tt.name, func(t *testing.T) {
79 | r, err := os.Open(tt.yaml)
80 | if err != nil {
81 | t.Fatal(err)
82 | }
83 | got, err := yaml2Conf(r)
84 | if err != nil {
85 | t.Fatal(err)
86 | }
87 | if !reflect.DeepEqual(got, tt.want) {
88 | t.Errorf("\nwant: %+v\ngot : %+v\n", tt.want, got)
89 | }
90 | })
91 | }
92 | }
93 |
--------------------------------------------------------------------------------
/docker-compose.yaml:
--------------------------------------------------------------------------------
1 | version: '3'
2 | services:
3 | elasticsearch:
4 | image: docker.elastic.co/elasticsearch/elasticsearch:7.15.1
5 | ports:
6 | - "9200:9200"
7 | - "9300:9300"
8 | environment:
9 | - discovery.type=single-node
10 |
--------------------------------------------------------------------------------
/es.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "fmt"
5 | "time"
6 |
7 | "github.com/cenkalti/backoff/v4"
8 | "github.com/elastic/go-elasticsearch/v7"
9 | )
10 |
11 | type esclient struct {
12 | client *elasticsearch.Client
13 | verbose bool
14 | }
15 |
16 | func newEsClient(urls []string, user, pass string) (*esclient, error) {
17 | retryBackoff := backoff.NewExponentialBackOff()
18 | retryBackoff.InitialInterval = time.Second
19 |
20 | conf := elasticsearch.Config{
21 | Addresses: urls,
22 | Username: user,
23 | Password: pass,
24 | RetryOnStatus: []int{408, 429, 502, 503, 504},
25 | RetryBackoff: func(i int) time.Duration {
26 | if i == 1 {
27 | retryBackoff.Reset()
28 | }
29 | d := retryBackoff.NextBackOff()
30 | return d
31 | },
32 | }
33 | es, err := elasticsearch.NewClient(conf)
34 | if err != nil {
35 | return nil, err
36 | }
37 | return &esclient{
38 | client: es,
39 | }, nil
40 | }
41 |
42 | func (e *esclient) log(msg string) {
43 | if e.verbose {
44 | fmt.Println(msg)
45 | }
46 | }
47 |
48 | func (e *esclient) logf(format string, a ...interface{}) {
49 | if e.verbose {
50 | fmt.Printf(format, a...)
51 | }
52 | }
53 |
--------------------------------------------------------------------------------
/eskeeper.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "fmt"
6 | "io"
7 | )
8 |
9 | // Eskeeper manages indices & aliases.
10 | type Eskeeper struct {
11 | client *esclient
12 |
13 | // options
14 | user string
15 | pass string
16 | verbose bool
17 | skipPreCheck bool
18 | }
19 |
20 | // NewOption is optional func for eskeeper.New
21 | type NewOption func(*Eskeeper)
22 |
23 | // UserName is optional func for Elasticsearch user name.
24 | func UserName(name string) NewOption {
25 | return func(e *Eskeeper) {
26 | e.user = name
27 | }
28 | }
29 |
30 | // Pass is optional func for Elasticsearch password.
31 | func Pass(pass string) NewOption {
32 | return func(e *Eskeeper) {
33 | e.pass = pass
34 | }
35 | }
36 |
37 | // Verbose is optional func for verbose option.
38 | func Verbose(v bool) NewOption {
39 | return func(e *Eskeeper) {
40 | e.verbose = v
41 | }
42 | }
43 |
44 | // SkipPreCheck is optional func for skipping pre-check-stage..
45 | func SkipPreCheck(v bool) NewOption {
46 | return func(e *Eskeeper) {
47 | e.skipPreCheck = v
48 | }
49 | }
50 |
51 | // New inits Eskeeper.
52 | func New(urls []string, opts ...NewOption) (*Eskeeper, error) {
53 | eskeeper := &Eskeeper{}
54 |
55 | for _, opt := range opts {
56 | opt(eskeeper)
57 | }
58 |
59 | es, err := newEsClient(urls, eskeeper.user, eskeeper.pass)
60 | if err != nil {
61 | return nil, err
62 | }
63 |
64 | es.verbose = eskeeper.verbose
65 | eskeeper.client = es
66 |
67 | return eskeeper, nil
68 | }
69 |
70 | // Sync synchronizes config & Elasticsearch State.
71 | func (e *Eskeeper) Sync(ctx context.Context, reader io.Reader) error {
72 | e.log("loading config ...")
73 | conf, err := yaml2Conf(reader)
74 | if err != nil {
75 | return err
76 | }
77 |
78 | e.log("\n=== validation stage ===")
79 | err = e.validateConfigFormat(conf)
80 | if err != nil {
81 | return err
82 | }
83 |
84 | if !e.skipPreCheck {
85 | e.log("\n=== pre-check stage ===")
86 | err = e.client.preCheck(ctx, conf)
87 | if err != nil {
88 | return err
89 | }
90 | }
91 |
92 | e.log("\n=== sync stage ===")
93 | err = e.client.syncIndices(ctx, conf)
94 | if err != nil {
95 | return err
96 | }
97 |
98 | err = e.client.syncAliases(ctx, conf)
99 | if err != nil {
100 | return err
101 | }
102 |
103 | err = e.client.syncCloseStatus(ctx, conf)
104 | if err != nil {
105 | return err
106 | }
107 |
108 | e.log("\n=== post-check stage ===")
109 | err = e.client.postCheck(ctx, conf)
110 | if err != nil {
111 | return err
112 | }
113 |
114 | e.log("\nsucceeded")
115 | return nil
116 | }
117 |
118 | // Validate validates cofig.
119 | func (e *Eskeeper) Validate(ctx context.Context, reader io.Reader) error {
120 | conf, err := yaml2Conf(reader)
121 | if err != nil {
122 | return err
123 | }
124 | err = e.validateConfigFormat(conf)
125 | if err != nil {
126 | return err
127 | }
128 | return nil
129 | }
130 |
131 | func (e *Eskeeper) log(msg string) {
132 | if e.verbose {
133 | fmt.Printf("\x1b[34m%s\x1b[0m\n", msg)
134 | }
135 | }
136 |
137 | func (e *Eskeeper) logf(format string, a ...interface{}) {
138 | if e.verbose {
139 | fmt.Printf(format, a...)
140 | }
141 | }
142 |
--------------------------------------------------------------------------------
/example/README.md:
--------------------------------------------------------------------------------
1 | # A Tour of eskeeper
2 |
3 | A Tour of eskeeper explains the basic usage of eskeeper.
4 |
5 | The files to be used are prepared under the `example` directory.
6 |
7 | ```tree
8 | example
9 | ├── README.md
10 | ├── docker-compose.yaml
11 | ├── es.yaml
12 | ├── README.md
13 | └── test.json
14 | ```
15 |
16 | ## start Elasticsearch
17 |
18 | First, start Elasticserarch.
19 |
20 | ```bash
21 | $ docker-compose up -d --build
22 | ```
23 |
24 | ## create indices & aliases
25 |
26 | Next, prepare index setting JSON file `test.json`. The content of this file is the same as the request body of the [Create index API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html)
27 |
28 | ```json
29 | {
30 | "settings": {
31 | "number_of_shards": 1,
32 | "number_of_replicas": 1,
33 | "analysis": {
34 | "analyzer": {
35 | "test_analyzer": {
36 | "type": "custom",
37 | "tokenizer": "standard",
38 | "filter": ["lowercase"]
39 | }
40 | }
41 | }
42 | },
43 | "mappings": {
44 | "properties": {
45 | "id": {
46 | "type": "long"
47 | },
48 | "title": {
49 | "type": "text"
50 | },
51 | "body": {
52 | "type": "text",
53 | "analyzer": "test_analyzer"
54 | }
55 | }
56 | }
57 | }
58 | ```
59 |
60 | create `es.yaml` to manage indices.
61 |
62 | ```yaml
63 | index:
64 | - name: test-v1 # index name
65 | mapping: test.json # index setting & mapping (json)
66 |
67 | - name: test-v2
68 | mapping: test.json
69 |
70 | - name: close-v1
71 | mapping: test.json
72 | status: close
73 |
74 | alias:
75 | - name: alias1
76 | index:
77 | - test-v1
78 |
79 | # multi indicies
80 | - name: alias2
81 | index:
82 | - test-v1
83 | - test-v2
84 | ```
85 |
86 | You can see in this file that you create 3 indices and 2 aliases. You can create an index with the same settings as the test.json created earlier. `close-v1` index use `status: close`. This allows closed indices to not have to maintain internal data structures for indexing or searching documents, resulting in a smaller overhead on the cluster.
87 |
88 | `alias` field declares alias settings. You can redirection the index without downtime.
89 |
90 | Now that we're ready, we can use eskeeper to create the index and alias.
91 |
92 | ```bash
93 | eskeeper < es.yaml
94 |
95 | curl localhost:9200/_cat/indices
96 | yellow open test-v1 ... 1 1 0 0 208b 208b
97 | yellow open test-v2 ... 1 1 0 0 208b 208b
98 | yellow close close-v1 ... 1 1
99 |
100 | curl localhost:9200/_cat/aliases
101 | alias2 test-v2 - - - -
102 | alias1 test-v1 - - - -
103 | alias2 test-v1 - - - -
104 | ```
105 |
106 | ## change index setting and switch alias
107 |
108 | Next, change mapping & config file.
109 |
110 | ```diff
111 | {
112 | "settings": {
113 | "number_of_shards": 1,
114 | "number_of_replicas": 1,
115 | "analysis": {
116 | "analyzer": {
117 | "test_analyzer": {
118 | "type": "custom",
119 | "tokenizer": "standard",
120 | "filter": ["lowercase"]
121 | }
122 | }
123 | }
124 | },
125 | "mappings": {
126 | "properties": {
127 | "id": {
128 | "type": "long"
129 | },
130 | "title": {
131 | "type": "text"
132 | },
133 | "body": {
134 | "type": "text",
135 | "analyzer": "test_analyzer"
136 | },
137 | + "category": {
138 | + "type": "text"
139 | + }
140 | }
141 | }
142 | ```
143 |
144 | mapping update is not supported. so you should create new index.
145 |
146 | ```diff
147 | index:
148 | - name: test-v1 # index name
149 | mapping: test.json # index setting & mapping (json)
150 |
151 | - name: test-v2
152 | mapping: test.json
153 |
154 | - name: close-v1
155 | mapping: test.json
156 | status: close
157 |
158 | + - name: test-v3
159 | + mapping: test.json
160 |
161 | alias:
162 | - name: alias1
163 | index:
164 | - test-v1
165 |
166 | # multi indicies
167 | - name: alias2
168 | index:
169 | - test-v1
170 | - - test-v2
171 | + - test-v3
172 | ```
173 |
174 | eskeeper creates new index ```test-v3``` with new mapping has `category` field. But, ```test-v1``` and ```test-v2``` setting & mapping is not changed (these index done not have `category` field).
175 |
176 | ```bash
177 | eskeeper < es.yaml
178 |
179 | curl localhost:9200/_cat/indices
180 | yellow open test-v1 ... 1 1 0 0 208b 208b
181 | yellow open test-v2 ... 1 1 0 0 208b 208b
182 | yellow open test-v3 ... 1 1 0 0 208b 208b
183 | yellow close close-v1 ... 1 1
184 |
185 | curl localhost:9200/_cat/aliases
186 | alias2 test-v3 - - - -
187 | alias1 test-v1 - - - -
188 | alias2 test-v1 - - - -
189 | ```
190 |
191 | ## reindex
192 |
193 | Finally, let's try reindex. put data to `test-v3` index.
194 |
195 | ```
196 | curl -X POST localhost:9200/test-v3/_doc -H "Content-Type: application/json" -d '{
197 | "id": 1,
198 | "title": "how to use eskeeper",
199 | "body": "this is A Tour of eskeeper",
200 | "category": "Elasticsearch"
201 | }'
202 | ```
203 |
204 | checks `test-v3` has one doc.
205 |
206 | ```
207 | curl localhost:9200/_cat/indices
208 | yellow open test-v1 ... 1 1 0 0 208b 208b
209 | yellow open test-v2 ... 1 1 0 0 208b 208b
210 | yellow open test-v3 ... 1 1 1 0 4.1kb 4.1kb
211 | yellow close close-v1 xxxxxxxxxxxx 1 1
212 | ```
213 |
214 | To reidnex `test-v3` to new index `test-v4`, use reindex option.
215 |
216 | ```diff
217 | index:
218 | - name: test-v1 # index name
219 | mapping: test.json # index setting & mapping (json)
220 |
221 | - name: test-v2
222 | mapping: test.json
223 |
224 | - name: close-v1
225 | mapping: test.json
226 | status: close
227 |
228 | - name: test-v3
229 | mapping: test.json
230 |
231 | + - name: test-v4
232 | + mapping: test.json
233 | + reindex:
234 | + source: test-v3
235 | + waitForCompletion: false
236 | + on: firstCreated
237 |
238 | alias:
239 | - name: alias1
240 | index:
241 | - test-v1
242 |
243 | # multi indicies
244 | - name: alias2
245 | index:
246 | - test-v1
247 | - - test-v3
248 | + - test-v4
249 | ```
250 |
251 | reindex field supports `source`, `slices`, `waitForComletion`, `on` options. `source`, `slices`, `waitForComletion` is same as [Reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) request body options.
252 |
253 | `on` field is eskeeper original option. If `on: firstCreated` is specified, reindex will only be executed the first time an index is created. If `on: always` is specified, a reindex will be run each time eskeeper is run.
254 |
255 | ```bash
256 | eskeeper < es.yaml
257 |
258 | curl localhost:9200/_cat/indices
259 | yellow open test-v1 ... 1 1 0 0 208b 208b
260 | yellow open test-v2 ... 1 1 0 0 208b 208b
261 | yellow open test-v3 ... 1 1 1 0 4.1kb 4.1kb
262 | yellow open test-v4 ... 1 1 1 0 4.1kb 4.1kb
263 | yellow close close-v1 xxxxxxxxxxxx 1 1
264 |
265 | curl localhost:9200/_cat/aliases
266 | alias2 test-v4 - - - -
267 | alias1 test-v1 - - - -
268 | alias2 test-v1 - - - -
269 | ```
270 |
271 | reindex is completed. check that `test-v4` has one doc.
272 |
--------------------------------------------------------------------------------
/example/docker-compose.yaml:
--------------------------------------------------------------------------------
1 | version: '3'
2 | services:
3 | elasticsearch:
4 | image: docker.elastic.co/elasticsearch/elasticsearch:7.15.1
5 | ports:
6 | - "9200:9200"
7 | - "9300:9300"
8 | environment:
9 | - discovery.type=single-node
10 |
--------------------------------------------------------------------------------
/example/es.yaml:
--------------------------------------------------------------------------------
1 | index:
2 | - name: test-v1 # index name
3 | mapping: test.json # index setting & mapping (json)
4 |
5 | - name: test-v2
6 | mapping: test.json
7 |
8 | - name: close-v1
9 | mapping: test.json
10 | status: close
11 |
12 | # - name: test-v3
13 | # mapping: test.json
14 |
15 | # - name: test-v4
16 | # mapping: test.json
17 | # reindex:
18 | # source: test-v3
19 | # waitForCompletion: false
20 | # on: firstCreated
21 |
22 | alias:
23 | - name: alias1
24 | index:
25 | - test-v1
26 |
27 | # multi indicies
28 | - name: alias2
29 | index:
30 | - test-v1
31 | - test-v2
32 |
--------------------------------------------------------------------------------
/example/test.json:
--------------------------------------------------------------------------------
1 | {
2 | "settings": {
3 | "number_of_shards": 1,
4 | "number_of_replicas": 1,
5 | "analysis": {
6 | "analyzer": {
7 | "test_analyzer": {
8 | "type": "custom",
9 | "tokenizer": "standard",
10 | "filter": ["lowercase"]
11 | }
12 | }
13 | }
14 | },
15 | "mappings": {
16 | "properties": {
17 | "id": {
18 | "type": "long"
19 | },
20 | "title": {
21 | "type": "text"
22 | },
23 | "body": {
24 | "type": "text",
25 | "analyzer": "test_analyzer"
26 | }
27 | }
28 | }
29 | }
30 |
--------------------------------------------------------------------------------
/go.mod:
--------------------------------------------------------------------------------
1 | module github.com/po3rin/eskeeper
2 |
3 | go 1.17
4 |
5 | require (
6 | github.com/Cside/jsondiff v0.0.0-20180209072652-0e50d980b458
7 | github.com/Masterminds/semver/v3 v3.1.1 // indirect
8 | github.com/avast/retry-go v3.0.0+incompatible // indirect
9 | github.com/cenkalti/backoff v2.2.1+incompatible // indirect
10 | github.com/cenkalti/backoff/v3 v3.2.2 // indirect
11 | github.com/cenkalti/backoff/v4 v4.1.1 // indirect
12 | github.com/containerd/continuity v0.0.0-20200928162600-f2cc35102c2a // indirect
13 | github.com/creack/pty v1.1.9 // indirect
14 | github.com/elastic/go-elasticsearch v0.0.0
15 | github.com/elastic/go-elasticsearch/v7 v7.11.0
16 | github.com/evanphx/json-patch v4.9.0+incompatible // indirect
17 | github.com/fatih/color v1.10.0 // indirect
18 | github.com/goccy/go-yaml v1.8.9
19 | github.com/gofrs/uuid v4.0.0+incompatible
20 | github.com/hokaccha/go-prettyjson v0.0.0-20190818114111-108c894c2c0e // indirect
21 | github.com/itchyny/astgen-go v0.0.0-20200815150004-12a293722290 // indirect
22 | github.com/itchyny/gojq v0.12.2
23 | github.com/kataras/pio v0.0.10
24 | github.com/mattn/go-colorable v0.1.8 // indirect
25 | github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
26 | github.com/oligot/go-mod-upgrade v0.4.1 // indirect
27 | github.com/ory/dockertest v3.3.5+incompatible
28 | github.com/ory/dockertest/v3 v3.6.0 // indirect
29 | github.com/pkg/errors v0.9.1
30 | github.com/po3rin/bmfzf v0.0.2
31 | github.com/sergi/go-diff v1.1.0 // indirect
32 | github.com/sirupsen/logrus v1.7.0 // indirect
33 | github.com/spf13/cobra v1.1.3
34 | github.com/spf13/pflag v1.0.5
35 | github.com/spf13/viper v1.7.1
36 | github.com/stretchr/testify v1.6.1 // indirect
37 | golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
38 | golang.org/x/sys v0.0.0-20210313110737-8e9fff1a3a18 // indirect
39 | golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d // indirect
40 | golang.org/x/text v0.3.5 // indirect
41 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
42 | gopkg.in/go-playground/assert.v1 v1.2.1 // indirect
43 | gopkg.in/go-playground/validator.v9 v9.30.0 // indirect
44 | gopkg.in/yaml.v2 v2.4.0
45 | )
46 |
--------------------------------------------------------------------------------
/go.sum:
--------------------------------------------------------------------------------
1 | bazil.org/fuse v0.0.0-20160811212531-371fbbdaa898/go.mod h1:Xbm+BRKSBEpa4q4hTSxohYNQpsxXPbPry4JJWOB3LB8=
2 | cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
3 | cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
4 | cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
5 | cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
6 | cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
7 | cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
8 | cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
9 | cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
10 | cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
11 | cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=
12 | cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
13 | cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
14 | dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
15 | github.com/AlecAivazis/survey/v2 v2.2.9 h1:LWvJtUswz/W9/zVVXELrmlvdwWcKE60ZAw0FWV9vssk=
16 | github.com/AlecAivazis/survey/v2 v2.2.9/go.mod h1:9DYvHgXtiXm6nCn+jXnOXLKbH+Yo9u8fAS/SduGdoPk=
17 | github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 h1:w+iIsaOQNcT7OZ575w+acHgRric5iCyQh+xv+KJ4HB8=
18 | github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
19 | github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
20 | github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
21 | github.com/Cside/jsondiff v0.0.0-20180209072652-0e50d980b458 h1:qmUvmwHdHBWSICS5F1lWf2A5B20QkueQXluuGp/5gnY=
22 | github.com/Cside/jsondiff v0.0.0-20180209072652-0e50d980b458/go.mod h1:FSe4ejBYCH1Cwo9T60GhzjKATA8f/qvmfEn6N3W985s=
23 | github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
24 | github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
25 | github.com/Masterminds/semver/v3 v3.0.3/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
26 | github.com/Masterminds/semver/v3 v3.1.1 h1:hLg3sBzpNErnxhQtUy/mmLR2I9foDujNK030IGemrRc=
27 | github.com/Masterminds/semver/v3 v3.1.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
28 | github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
29 | github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
30 | github.com/Netflix/go-expect v0.0.0-20180615182759-c93bf25de8e8/go.mod h1:oX5x61PbNXchhh0oikYAH+4Pcfw5LKv21+Jnpr6r6Pc=
31 | github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEVMRuU21PR1EtLVZJmdB18Gu3Rw=
32 | github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk=
33 | github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
34 | github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
35 | github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
36 | github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
37 | github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
38 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
39 | github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
40 | github.com/avast/retry-go v3.0.0+incompatible h1:4SOWQ7Qs+oroOTQOYnAHqelpCO0biHSxpiH9JdtuBj0=
41 | github.com/avast/retry-go v3.0.0+incompatible/go.mod h1:XtSnn+n/sHqQIpZ10K1qAevBhOOCWBLXXy3hyiqqBrY=
42 | github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
43 | github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
44 | github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
45 | github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84=
46 | github.com/cenkalti/backoff v1.1.0 h1:QnvVp8ikKCDWOsFheytRCoYWYPO/ObCTBGxT19Hc+yE=
47 | github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4=
48 | github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
49 | github.com/cenkalti/backoff/v3 v3.0.0 h1:ske+9nBpD9qZsTBoF41nW5L+AIuFBKMeze18XQ3eG1c=
50 | github.com/cenkalti/backoff/v3 v3.0.0/go.mod h1:cIeZDE3IrqwwJl6VUwCN6trj1oXrTS4rc0ij+ULvLYs=
51 | github.com/cenkalti/backoff/v3 v3.2.2 h1:cfUAAO3yvKMYKPrvhDuHSwQnhZNk/RMHKdZqKTxfm6M=
52 | github.com/cenkalti/backoff/v3 v3.2.2/go.mod h1:cIeZDE3IrqwwJl6VUwCN6trj1oXrTS4rc0ij+ULvLYs=
53 | github.com/cenkalti/backoff/v4 v4.1.1 h1:G2HAfAmvm/GcKan2oOQpBXOd2tT2G57ZnZGWa1PxPBQ=
54 | github.com/cenkalti/backoff/v4 v4.1.1/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw=
55 | github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
56 | github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
57 | github.com/containerd/continuity v0.0.0-20190827140505-75bee3e2ccb6 h1:NmTXa/uVnDyp0TY5MKi197+3HWcnYWfnHGyaFthlnGw=
58 | github.com/containerd/continuity v0.0.0-20190827140505-75bee3e2ccb6/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
59 | github.com/containerd/continuity v0.0.0-20200928162600-f2cc35102c2a h1:jEIoR0aA5GogXZ8pP3DUzE+zrhaF6/1rYZy+7KkYEWM=
60 | github.com/containerd/continuity v0.0.0-20200928162600-f2cc35102c2a/go.mod h1:W0qIOTD7mp2He++YVq+kgfXezRYqzP1uDuMVH1bITDY=
61 | github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
62 | github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
63 | github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
64 | github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
65 | github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
66 | github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
67 | github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
68 | github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
69 | github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
70 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
71 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
72 | github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
73 | github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
74 | github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
75 | github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
76 | github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
77 | github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
78 | github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
79 | github.com/elastic/go-elasticsearch v0.0.0 h1:Pd5fqOuBxKxv83b0+xOAJDAkziWYwFinWnBO0y+TZaA=
80 | github.com/elastic/go-elasticsearch v0.0.0/go.mod h1:TkBSJBuTyFdBnrNqoPc54FN0vKf5c04IdM4zuStJ7xg=
81 | github.com/elastic/go-elasticsearch/v7 v7.9.0 h1:UEau+a1MiiE/F+UrDj60kqIHFWdzU1M2y/YtBU2NC2M=
82 | github.com/elastic/go-elasticsearch/v7 v7.9.0/go.mod h1:OJ4wdbtDNk5g503kvlHLyErCgQwwzmDtaFC4XyOxXA4=
83 | github.com/elastic/go-elasticsearch/v7 v7.11.0 h1:bv+2GqsVrPdX/ChJqAHAFtWgtGvVJ0icN/WdBGAdNuw=
84 | github.com/elastic/go-elasticsearch/v7 v7.11.0/go.mod h1:OJ4wdbtDNk5g503kvlHLyErCgQwwzmDtaFC4XyOxXA4=
85 | github.com/evanphx/json-patch v4.9.0+incompatible h1:kLcOMZeuLAJvL2BPWLMIj5oaZQobrkAqrL+WFZwQses=
86 | github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
87 | github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
88 | github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
89 | github.com/fatih/color v1.9.0 h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s=
90 | github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
91 | github.com/fatih/color v1.10.0 h1:s36xzo75JdqLaaWoiEHk767eHiwo0598uUxyfiPkDsg=
92 | github.com/fatih/color v1.10.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM=
93 | github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
94 | github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
95 | github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
96 | github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
97 | github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
98 | github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
99 | github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
100 | github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
101 | github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
102 | github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
103 | github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
104 | github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
105 | github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
106 | github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
107 | github.com/goccy/go-yaml v1.8.2 h1:gDYrSN12XK/wQTFjxWIgcIqjNCV/Zb5V09M7cq+dbCs=
108 | github.com/goccy/go-yaml v1.8.2/go.mod h1:wS4gNoLalDSJxo/SpngzPQ2BN4uuZVLCmbM4S3vd4+Y=
109 | github.com/goccy/go-yaml v1.8.9 h1:4AEXg2qx+/w29jXnXpMY6mTckmYu1TMoHteKuMf0HFg=
110 | github.com/goccy/go-yaml v1.8.9/go.mod h1:U/jl18uSupI5rdI2jmuCswEA2htH9eXfferR3KfscvA=
111 | github.com/gofrs/uuid v1.2.0 h1:coDhrjgyJaglxSjxuJdqQSSdUpG3w6p1OwN2od6frBU=
112 | github.com/gofrs/uuid v3.3.0+incompatible h1:8K4tyRfvU1CYPgJsveYFQMhpFd/wXNM7iK6rR7UHz84=
113 | github.com/gofrs/uuid v3.3.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
114 | github.com/gofrs/uuid v4.0.0+incompatible h1:1SD/1F5pU8p29ybwgQSwpQk+mwdRrXCYuPhW6m+TnJw=
115 | github.com/gofrs/uuid v4.0.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
116 | github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
117 | github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
118 | github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
119 | github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
120 | github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
121 | github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
122 | github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
123 | github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
124 | github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
125 | github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
126 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
127 | github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
128 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
129 | github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
130 | github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
131 | github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
132 | github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
133 | github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
134 | github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
135 | github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
136 | github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
137 | github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
138 | github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
139 | github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
140 | github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
141 | github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
142 | github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
143 | github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
144 | github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
145 | github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=
146 | github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
147 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
148 | github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
149 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
150 | github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
151 | github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
152 | github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=
153 | github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
154 | github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=
155 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
156 | github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
157 | github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=
158 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
159 | github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
160 | github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
161 | github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
162 | github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
163 | github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
164 | github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
165 | github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
166 | github.com/hinshun/vt10x v0.0.0-20180616224451-1954e6464174/go.mod h1:DqJ97dSdRW1W22yXSB90986pcOyQ7r45iio1KN2ez1A=
167 | github.com/hokaccha/go-prettyjson v0.0.0-20190818114111-108c894c2c0e/go.mod h1:pFlLw2CfqZiIBOx6BuCeRLCrfxBJipTY0nIOF/VbGcI=
168 | github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
169 | github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
170 | github.com/itchyny/astgen-go v0.0.0-20200815150004-12a293722290/go.mod h1:296z3W7Xsrp2mlIY88ruDKscuvrkL6zXCNRtaYVshzw=
171 | github.com/itchyny/go-flags v1.5.0/go.mod h1:lenkYuCobuxLBAd/HGFE4LRoW8D3B6iXRQfWYJ+MNbA=
172 | github.com/itchyny/gojq v0.11.1 h1:k54XkzWCGDfRJSZFRW4rXowTVzPlSjU2xUErkaFjfdo=
173 | github.com/itchyny/gojq v0.11.1/go.mod h1:8MKtgvJwkmRduSuzN25byPdNHfvv6y+/hmOVXei9e7k=
174 | github.com/itchyny/gojq v0.11.2 h1:lKhMKfH7fTKMWj2Zr8az/9TliCn0TTXVc/BXfQ8Jhfc=
175 | github.com/itchyny/gojq v0.11.2/go.mod h1:XtmtF1PxeDpwLC1jyz/xAmV78ANlP0S9LVEPsKweK0A=
176 | github.com/itchyny/gojq v0.12.2 h1:TxhFjk1w7Vnb0SwQPeG4FxTC98O4Es+x/mPaD5Azgfs=
177 | github.com/itchyny/gojq v0.12.2/go.mod h1:mi4PdXSlFllHyByM68JKUrbiArtEdEnNEmjbwxcQKAg=
178 | github.com/itchyny/timefmt-go v0.1.0/go.mod h1:0osSSCQSASBJMsIZnhAaF1C2fCBTJZXrnj37mG8/c+A=
179 | github.com/itchyny/timefmt-go v0.1.1 h1:rLpnm9xxb39PEEVzO0n4IRp0q6/RmBc7Dy/rE4HrA0U=
180 | github.com/itchyny/timefmt-go v0.1.1/go.mod h1:0osSSCQSASBJMsIZnhAaF1C2fCBTJZXrnj37mG8/c+A=
181 | github.com/itchyny/timefmt-go v0.1.2 h1:q0Xa4P5it6K6D7ISsbLAMwx1PnWlixDcJL6/sFs93Hs=
182 | github.com/itchyny/timefmt-go v0.1.2/go.mod h1:0osSSCQSASBJMsIZnhAaF1C2fCBTJZXrnj37mG8/c+A=
183 | github.com/jmoiron/jsonq v0.0.0-20150511023944-e874b168d07e h1:ZZCvgaRDZg1gC9/1xrsgaJzQUCQgniKtw0xjWywWAOE=
184 | github.com/jmoiron/jsonq v0.0.0-20150511023944-e874b168d07e/go.mod h1:+rHyWac2R9oAZwFe1wGY2HBzFJJy++RHBg1cU23NkD8=
185 | github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
186 | github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
187 | github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
188 | github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
189 | github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
190 | github.com/kataras/pio v0.0.10 h1:b0qtPUqOpM2O+bqa5wr2O6dN4cQNwSmFd6HQqgVae0g=
191 | github.com/kataras/pio v0.0.10/go.mod h1:gS3ui9xSD+lAUpbYnjOGiQyY7sUMJO+EHpiRzhtZ5no=
192 | github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
193 | github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
194 | github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
195 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
196 | github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
197 | github.com/konsorten/go-windows-terminal-sequences v1.0.2 h1:DB17ag19krx9CFsz4o3enTrPXyIXCl+2iCXH/aMAp9s=
198 | github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
199 | github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
200 | github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
201 | github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
202 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
203 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
204 | github.com/kr/pty v1.1.4/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
205 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
206 | github.com/ktr0731/go-fuzzyfinder v0.2.1 h1:YR9LXobzd9N+RVU9j4ASc0kWktTyJnkTex8Y6TW99f0=
207 | github.com/ktr0731/go-fuzzyfinder v0.2.1/go.mod h1:1BUWoT8siOp5n8ns8S6rtfTVigx/dvPPUJuu79nixgo=
208 | github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
209 | github.com/lib/pq v0.0.0-20180327071824-d34b9ff171c2/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
210 | github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
211 | github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4=
212 | github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
213 | github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
214 | github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
215 | github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
216 | github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
217 | github.com/mattn/go-colorable v0.1.7/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
218 | github.com/mattn/go-colorable v0.1.8 h1:c1ghPdyEDarC70ftn0y+A/Ee++9zz8ljHG1b13eJ0s8=
219 | github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
220 | github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
221 | github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
222 | github.com/mattn/go-isatty v0.0.10 h1:qxFzApOv4WsAL965uUPIsXzAKCZxN2p9UqdhFS4ZW10=
223 | github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
224 | github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
225 | github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
226 | github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
227 | github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
228 | github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
229 | github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
230 | github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
231 | github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=
232 | github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
233 | github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
234 | github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
235 | github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
236 | github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
237 | github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
238 | github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
239 | github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
240 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
241 | github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE=
242 | github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
243 | github.com/mitchellh/mapstructure v1.3.2 h1:mRS76wmkOn3KkKAyXDu42V+6ebnXWIztFSYGN7GeoRg=
244 | github.com/mitchellh/mapstructure v1.3.2/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
245 | github.com/moby/term v0.0.0-20200915141129-7f0af18e79f2 h1:SPoLlS9qUUnXcIY4pvA4CTwYjk0Is5f4UPEkeESr53k=
246 | github.com/moby/term v0.0.0-20200915141129-7f0af18e79f2/go.mod h1:TjQg8pa4iejrUrjiz0MCtMV38jdMNW4doKSiBrEvCQQ=
247 | github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
248 | github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
249 | github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
250 | github.com/nsf/termbox-go v0.0.0-20200418040025-38ba6e5628f1 h1:lh3PyZvY+B9nFliSGTn5uFuqQQJGuNrD0MLCokv09ag=
251 | github.com/nsf/termbox-go v0.0.0-20200418040025-38ba6e5628f1/go.mod h1:IuKpRQcYE1Tfu+oAQqaLisqDeXgjyyltCfsaoYN18NQ=
252 | github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
253 | github.com/oligot/go-mod-upgrade v0.4.1 h1:dc7TeRDPclyVKnEh4O1VzMzRkoer5cTwZIFpNIwfbOA=
254 | github.com/oligot/go-mod-upgrade v0.4.1/go.mod h1:sKrcfERdiA1WOfY5S24RteSgxfk3wnfHTQxuHU/u81E=
255 | github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ=
256 | github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
257 | github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
258 | github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
259 | github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
260 | github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
261 | github.com/opencontainers/runc v0.1.1 h1:GlxAyO6x8rfZYN9Tt0Kti5a/cP41iuiO2yYT0IJGY8Y=
262 | github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
263 | github.com/opencontainers/runc v1.0.0-rc9 h1:/k06BMULKF5hidyoZymkoDCzdJzltZpz/UU4LguQVtc=
264 | github.com/opencontainers/runc v1.0.0-rc9/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
265 | github.com/ory/dockertest v3.3.5+incompatible h1:iLLK6SQwIhcbrG783Dghaaa3WPzGc+4Emza6EbVUUGA=
266 | github.com/ory/dockertest v3.3.5+incompatible/go.mod h1:1vX4m9wsvi00u5bseYwXaSnhNrne+V0E6LAcBILJdPs=
267 | github.com/ory/dockertest/v3 v3.6.0/go.mod h1:4ZOpj8qBUmh8fcBSVzkH2bws2s91JdGvHUqan4GHEuQ=
268 | github.com/ory/dockertest/v3 v3.6.1 h1:gxheDizSurXBKRj1Lu5Ws/xzQx35VgzkLFr1woC6Hoc=
269 | github.com/ory/dockertest/v3 v3.6.1/go.mod h1:EFLcVUOl8qCwp9NyDAcCDtq/QviLtYswW/VbWzUnTNE=
270 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
271 | github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
272 | github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
273 | github.com/pelletier/go-toml v1.8.0 h1:Keo9qb7iRJs2voHvunFtuuYFsbWeOBh8/P9v/kVMFtw=
274 | github.com/pelletier/go-toml v1.8.0/go.mod h1:D6yutnOGMveHEPV7VQOuvI/gXY61bv+9bAOTRnLElKs=
275 | github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
276 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
277 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
278 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
279 | github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=
280 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
281 | github.com/po3rin/bmfzf v0.0.1 h1:O8hP4rTevvk/jl9i5yaDQHbTOKOAaAy1b++oqOPOzo4=
282 | github.com/po3rin/bmfzf v0.0.1/go.mod h1:eAoLgvjpJiJU7OPP+O0LDM1o4Uyv/bcEST7+rWROXKw=
283 | github.com/po3rin/bmfzf v0.0.2 h1:jEeTaIOdvewWNqsyoeMDHiCvexAPMJIoX8/yFemfwLI=
284 | github.com/po3rin/bmfzf v0.0.2/go.mod h1:z5yDQ+AXqRkQ9zFblk/QbSn/sldI2HAouDSsBsc1YOQ=
285 | github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
286 | github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
287 | github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
288 | github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
289 | github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
290 | github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
291 | github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
292 | github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
293 | github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
294 | github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
295 | github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
296 | github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
297 | github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
298 | github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
299 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
300 | github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
301 | github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
302 | github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
303 | github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
304 | github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
305 | github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
306 | github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
307 | github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
308 | github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM=
309 | github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
310 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
311 | github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
312 | github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
313 | github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
314 | github.com/spf13/afero v1.1.2 h1:m8/z1t7/fwjysjQRYbP0RD+bUIF/8tJwPdEZsI83ACI=
315 | github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
316 | github.com/spf13/afero v1.3.1 h1:GPTpEAuNr98px18yNQ66JllNil98wfRZ/5Ukny8FeQA=
317 | github.com/spf13/afero v1.3.1/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
318 | github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
319 | github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
320 | github.com/spf13/cast v1.3.1 h1:nFm6S0SMdyzrzcmThSipiEubIDy8WEXKNZ0UOgiRpng=
321 | github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
322 | github.com/spf13/cobra v0.0.2-0.20171109065643-2da4a54c5cee/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
323 | github.com/spf13/cobra v1.0.0 h1:6m/oheQuQ13N9ks4hubMG6BnvwOeaJrqSPLahSnczz8=
324 | github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=
325 | github.com/spf13/cobra v1.1.3 h1:xghbfqPkxzxP3C/f3n5DdpAbdKLj4ZE4BWQI362l53M=
326 | github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo=
327 | github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk=
328 | github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
329 | github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
330 | github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
331 | github.com/spf13/pflag v1.0.1-0.20171106142849-4c012f6dcd95/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
332 | github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
333 | github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
334 | github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
335 | github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
336 | github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
337 | github.com/spf13/viper v1.7.0 h1:xVKxvI7ouOI5I+U9s2eeiUfMaWBVoXA3AWskkrqK0VM=
338 | github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg=
339 | github.com/spf13/viper v1.7.1 h1:pM5oEahlgWv/WnHXpgbKz7iLIxRf65tye2Ci+XFK5sk=
340 | github.com/spf13/viper v1.7.1/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg=
341 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
342 | github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
343 | github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
344 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
345 | github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
346 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
347 | github.com/stretchr/testify v1.5.0/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
348 | github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
349 | github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s=
350 | github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
351 | github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
352 | github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
353 | github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
354 | github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
355 | go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
356 | go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
357 | go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
358 | go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
359 | go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
360 | go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
361 | golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
362 | golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
363 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
364 | golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
365 | golang.org/x/crypto v0.0.0-20190530122614-20be4c3c3ed5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
366 | golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
367 | golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
368 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
369 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
370 | golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g=
371 | golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
372 | golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
373 | golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
374 | golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
375 | golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
376 | golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
377 | golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
378 | golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
379 | golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
380 | golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
381 | golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
382 | golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
383 | golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
384 | golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
385 | golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
386 | golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
387 | golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
388 | golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
389 | golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
390 | golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
391 | golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
392 | golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
393 | golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
394 | golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
395 | golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
396 | golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
397 | golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
398 | golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
399 | golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
400 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
401 | golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
402 | golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
403 | golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
404 | golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
405 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
406 | golang.org/x/net v0.0.0-20191003171128-d98b1b443823 h1:Ypyv6BNJh07T1pUSrehkLemqPKXhus2MkfktJ91kRh4=
407 | golang.org/x/net v0.0.0-20191003171128-d98b1b443823/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
408 | golang.org/x/net v0.0.0-20201010224723-4f7140c49acb h1:mUVeFHoDKis5nxCAzoAi7E8Ghb86EXh/RK6wtvJIqRY=
409 | golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
410 | golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
411 | golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
412 | golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
413 | golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
414 | golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
415 | golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
416 | golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
417 | golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
418 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
419 | golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208 h1:qwRHBd0NqMbJxfbotnDhm2ByMI1Shq4Y6oRJo21SGJA=
420 | golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
421 | golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
422 | golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
423 | golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
424 | golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
425 | golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
426 | golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
427 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
428 | golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
429 | golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
430 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
431 | golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
432 | golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
433 | golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
434 | golang.org/x/sys v0.0.0-20190530182044-ad28b68e88f1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
435 | golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
436 | golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
437 | golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
438 | golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
439 | golang.org/x/sys v0.0.0-20191010194322-b09406accb47 h1:/XfQ9z7ib8eEJX2hdgFTZJ/ntt0swNk5oYBziWeTCvY=
440 | golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
441 | golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
442 | golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
443 | golang.org/x/sys v0.0.0-20200121082415-34d275377bf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
444 | golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
445 | golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
446 | golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
447 | golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
448 | golang.org/x/sys v0.0.0-20200821140526-fda516888d29/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
449 | golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6 h1:DvY3Zkh7KabQE/kfzMvYvKirSiguP9Q/veMtkYyf0o8=
450 | golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
451 | golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
452 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f h1:+Nyd8tzPX9R7BWHguqsrbFdRx3WQ/1ib8I44HXV5yTA=
453 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
454 | golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211 h1:9UQO31fZ+0aKQOFldThf7BKPMJTiBfWycGh/u3UoO88=
455 | golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
456 | golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
457 | golang.org/x/sys v0.0.0-20210301091718-77cc2087c03b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
458 | golang.org/x/sys v0.0.0-20210313110737-8e9fff1a3a18 h1:jxr7/dEo+rR29uEBoLSWJ1tRHCFAMwFbGUU9nRqzpds=
459 | golang.org/x/sys v0.0.0-20210313110737-8e9fff1a3a18/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
460 | golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
461 | golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d h1:SZxvLBoTP5yHO3Frd4z4vrF+DBX9vMVanchswa69toE=
462 | golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
463 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
464 | golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
465 | golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
466 | golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=
467 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
468 | golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ=
469 | golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
470 | golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
471 | golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
472 | golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
473 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
474 | golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
475 | golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
476 | golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
477 | golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
478 | golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
479 | golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
480 | golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
481 | golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
482 | golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
483 | golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
484 | golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
485 | golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
486 | golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
487 | golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
488 | golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
489 | golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
490 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
491 | golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
492 | golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
493 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
494 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
495 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
496 | google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
497 | google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
498 | google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
499 | google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
500 | google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
501 | google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
502 | google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
503 | google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
504 | google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
505 | google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
506 | google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
507 | google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
508 | google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
509 | google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
510 | google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
511 | google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
512 | google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
513 | google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
514 | google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
515 | google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
516 | google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
517 | google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
518 | gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
519 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
520 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
521 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
522 | gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
523 | gopkg.in/go-playground/assert.v1 v1.2.1/go.mod h1:9RXL0bg/zibRAgZUYszZSwO/z8Y/a8bDuhia5mkpMnE=
524 | gopkg.in/go-playground/validator.v9 v9.30.0/go.mod h1:+c9/zcJMFNgbLvly1L1V+PpxWdVbfP1avr/N00E2vyQ=
525 | gopkg.in/ini.v1 v1.51.0 h1:AQvPpx3LzTDM0AjnIRlVFwFFGC+npRopjZxLJj6gdno=
526 | gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
527 | gopkg.in/ini.v1 v1.57.0 h1:9unxIsFcTt4I55uWluz+UmL95q4kdJ0buvQ1ZIqVQww=
528 | gopkg.in/ini.v1 v1.57.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
529 | gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
530 | gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
531 | gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
532 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
533 | gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
534 | gopkg.in/yaml.v2 v2.2.7 h1:VUgggvou5XRW9mHwD/yXxIYSMtY0zoKQf/v226p2nyo=
535 | gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
536 | gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
537 | gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
538 | gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
539 | gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
540 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
541 | gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
542 | gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
543 | gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk=
544 | honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
545 | honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
546 | honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
547 | honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
548 | rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
549 |
--------------------------------------------------------------------------------
/index.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "fmt"
6 | "io/ioutil"
7 | "os"
8 | )
9 |
10 | func (c *esclient) existIndex(ctx context.Context, index string) (bool, error) {
11 | exists := c.client.Indices.Exists
12 |
13 | res, err := exists(
14 | []string{index},
15 | exists.WithContext(ctx),
16 | )
17 | if err != nil {
18 | return false, fmt.Errorf("check index exists: %w", err)
19 | }
20 | if res.StatusCode == 200 {
21 | return true, nil
22 | }
23 | return false, nil
24 | }
25 |
26 | func (c *esclient) syncIndex(ctx context.Context, index index) error {
27 | create := c.client.Indices.Create
28 | ok, err := c.existIndex(ctx, index.Name)
29 | if err != nil {
30 | return err
31 | }
32 |
33 | // index dose not exist.
34 | if !ok {
35 | f, err := os.Open(index.Mapping)
36 | if err != nil {
37 | return fmt.Errorf("open mapping file: %w", err)
38 | }
39 |
40 | res, err := create(
41 | index.Name,
42 | create.WithContext(ctx),
43 | create.WithBody(f),
44 | )
45 | if err != nil {
46 | return fmt.Errorf("create index: %w", err)
47 | }
48 | if res.StatusCode != 200 {
49 | body, err := ioutil.ReadAll(res.Body)
50 | if err != nil {
51 | return fmt.Errorf("failed to create index [index=%v, statusCode=%v]", index.Name, res.StatusCode)
52 | }
53 | return fmt.Errorf("failed to create index [index=%v, statusCode=%v, res=%v]", index.Name, res.StatusCode, string(body))
54 | }
55 |
56 | // reindex --------
57 |
58 | if index.Reindex.Source == "" {
59 | return nil
60 | }
61 |
62 | ok, err := c.existIndex(ctx, index.Reindex.Source)
63 | if err != nil {
64 | return fmt.Errorf("check index exists for reindex process: %w", err)
65 | }
66 | if !ok {
67 | return fmt.Errorf("reindex (%s -> %s) conf is invalid. Make sure %s index exists", index.Reindex.Source, index.Reindex.Source, index.Name)
68 | }
69 |
70 | err = c.reindex(ctx, index.Name, index.Reindex)
71 | if err != nil {
72 | return fmt.Errorf("reindex (%s -> %s)", index.Reindex.Source, index.Name)
73 | }
74 | return nil
75 | }
76 |
77 | // index already exists.
78 |
79 | // reindex -------
80 | if index.Reindex.Source != "" && index.Reindex.On == "always" {
81 | ok, err = c.existIndex(ctx, index.Reindex.Source)
82 | if err != nil {
83 | return fmt.Errorf("check index exists for reindex process: %w", err)
84 | }
85 | if !ok {
86 | return fmt.Errorf("reindex (%s -> %s) conf is invalid. Make sure %s index exists", index.Reindex.Source, index.Reindex.Source, index.Name)
87 | }
88 | err = c.reindex(ctx, index.Name, index.Reindex)
89 | if err != nil {
90 | return fmt.Errorf("reindex (%s -> %s)", index.Reindex.Source, index.Name)
91 | }
92 | return nil
93 | }
94 |
95 | // Since downtime may occur when switching aliases, only open is processed before switching aliases.
96 | // TODO: refactoring.
97 | if index.Status == "close" {
98 | return nil
99 | }
100 | err = c.openIndex(ctx, index)
101 | if err != nil {
102 | return fmt.Errorf("open index: %w", err)
103 | }
104 | return nil
105 | }
106 |
107 | func (c *esclient) deleteIndex(ctx context.Context, index string) error {
108 | delete := c.client.Indices.Delete
109 | res, err := delete([]string{index}, delete.WithContext(ctx))
110 | if err != nil {
111 | return fmt.Errorf("delete index: %w", err)
112 | }
113 | if res.StatusCode != 200 {
114 | body, err := ioutil.ReadAll(res.Body)
115 | if err != nil {
116 | return fmt.Errorf("failed to delete index [index= %v, statusCode=%v]", index, res.StatusCode)
117 | }
118 | return fmt.Errorf("failed to delete index [index= %v, statusCode=%v, res=%v]", index, res.StatusCode, string(body))
119 | }
120 | return nil
121 | }
122 |
123 | func (c *esclient) syncCloseStatus(ctx context.Context, conf config) error {
124 | for _, index := range conf.Indices {
125 | if index.Status == "close" {
126 | err := c.closeIndex(ctx, index)
127 | if err != nil {
128 | return fmt.Errorf("crearted index status action: %w", err)
129 | }
130 | }
131 | }
132 | return nil
133 | }
134 |
135 | // func (c *esclient) indexStatusAction(ctx context.Context, index index) error {
136 | // switch index.Status {
137 | // case "close":
138 | // err := c.closeIndex(ctx, index)
139 | // if err != nil {
140 | // return fmt.Errorf("close index: %w", err)
141 | // }
142 | // case "open":
143 | // err := c.openIndex(ctx, index)
144 | // if err != nil {
145 | // return fmt.Errorf("open index: %w", err)
146 | // }
147 | // default:
148 | // err := c.openIndex(ctx, index)
149 | // if err != nil {
150 | // return fmt.Errorf("open index: %w", err)
151 | // }
152 | // }
153 | // return nil
154 | // }
155 |
156 | func (c *esclient) closeIndex(ctx context.Context, index index) error {
157 | close := c.client.Indices.Close
158 | res, err := close([]string{index.Name}, close.WithContext(ctx))
159 | if err != nil {
160 | return fmt.Errorf("close index: %w", err)
161 | }
162 | if res.StatusCode != 200 {
163 | body, err := ioutil.ReadAll(res.Body)
164 | if err != nil {
165 | return fmt.Errorf("failed to close index [index= %v, statusCode=%v]", index, res.StatusCode)
166 | }
167 | return fmt.Errorf("failed to close index [index= %v, statusCode=%v, res=%v]", index, res.StatusCode, string(body))
168 | }
169 | return nil
170 | }
171 |
172 | func (c *esclient) openIndex(ctx context.Context, index index) error {
173 | open := c.client.Indices.Open
174 | res, err := open([]string{index.Name}, open.WithContext(ctx))
175 | if err != nil {
176 | return fmt.Errorf("open index: %w", err)
177 | }
178 | if res.StatusCode != 200 {
179 | body, err := ioutil.ReadAll(res.Body)
180 | if err != nil {
181 | return fmt.Errorf("failed to open index [index= %v, statusCode=%v]: %w", index, res.StatusCode, err)
182 | }
183 | return fmt.Errorf("failed to open index [index= %v, statusCode=%v, res=%v]: %w", index, res.StatusCode, string(body), err)
184 | }
185 | return nil
186 | }
187 |
188 | func (c *esclient) syncIndices(ctx context.Context, conf config) error {
189 | for _, index := range conf.Indices {
190 | err := c.syncIndex(ctx, index)
191 | if err != nil {
192 | c.logf("[fail] index: %v\n", index.Name)
193 | return fmt.Errorf("sync index: %w", err)
194 | }
195 | c.logf("[synced] index: %v\n", index.Name)
196 | }
197 | return nil
198 | }
199 |
--------------------------------------------------------------------------------
/index_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "testing"
6 | )
7 |
8 | func TestSyncIndices(t *testing.T) {
9 | tests := []struct {
10 | name string
11 | conf config
12 | setup func(tb testing.TB)
13 | cleanup func(tb testing.TB)
14 | }{
15 | {
16 | name: "simple",
17 | conf: config{
18 | Indices: []index{
19 | {
20 | Name: "create1",
21 | Mapping: "testdata/test.json",
22 | },
23 | },
24 | },
25 | cleanup: func(tb testing.TB) {
26 | deleteIndexHelper(tb, []string{"create1"})
27 | },
28 | },
29 | {
30 | name: "multi",
31 | conf: config{
32 | Indices: []index{
33 | {
34 | Name: "create2",
35 | Mapping: "testdata/test.json",
36 | },
37 | {
38 | Name: "create3",
39 | Mapping: "testdata/test.json",
40 | },
41 | },
42 | },
43 | cleanup: func(tb testing.TB) {
44 | deleteIndexHelper(tb, []string{"create2", "create3"})
45 | },
46 | },
47 | {
48 | name: "idempotence",
49 | conf: config{
50 | Indices: []index{
51 | {
52 | Name: "idempotence",
53 | Mapping: "testdata/test.json",
54 | },
55 | {
56 | Name: "idempotence",
57 | Mapping: "testdata/test.json",
58 | },
59 | },
60 | },
61 | cleanup: func(tb testing.TB) {
62 | deleteIndexHelper(tb, []string{"idempotence"})
63 | },
64 | },
65 | {
66 | name: "close",
67 | conf: config{
68 | Indices: []index{
69 | {
70 | Name: "create-with-close-v1",
71 | Mapping: "testdata/test.json",
72 | Status: "close",
73 | },
74 | },
75 | },
76 | cleanup: func(tb testing.TB) {
77 | deleteIndexHelper(tb, []string{"create-with-close-v1"})
78 | },
79 | },
80 | {
81 | name: "exists close",
82 | conf: config{
83 | Indices: []index{
84 | {
85 | Name: "create-with-close-v2",
86 | Mapping: "testdata/test.json",
87 | Status: "close",
88 | },
89 | },
90 | },
91 | setup: func(tb testing.TB) {
92 | createTmpIndexHelper(tb, "create-with-close-v2")
93 | },
94 | cleanup: func(tb testing.TB) {
95 | deleteIndexHelper(tb, []string{"create-with-close-v2"})
96 | },
97 | },
98 | {
99 | name: "open",
100 | conf: config{
101 | Indices: []index{
102 | {
103 | Name: "open-v1",
104 | Mapping: "testdata/test.json",
105 | },
106 | },
107 | },
108 | setup: func(tb testing.TB) {
109 | createTmpIndexHelper(tb, "open-v1")
110 | closeIndexHelper(tb, "open-v1")
111 | },
112 | cleanup: func(tb testing.TB) {
113 | deleteIndexHelper(tb, []string{"open-v1"})
114 | },
115 | },
116 | {
117 | name: "open-already-open",
118 | conf: config{
119 | Indices: []index{
120 | {
121 | Name: "open-already-open-v1",
122 | Mapping: "testdata/test.json",
123 | },
124 | },
125 | },
126 | cleanup: func(tb testing.TB) {
127 | deleteIndexHelper(tb, []string{"open-already-open-v1"})
128 | },
129 | },
130 | {
131 | name: "reindex",
132 | conf: config{
133 | Indices: []index{
134 | {
135 | Name: "reindex-v1",
136 | Mapping: "testdata/test.json",
137 | Reindex: reindex{
138 | Source: "reindex-v0",
139 | Slices: 3,
140 | WaitForCompletion: true,
141 | On: "firstCreated",
142 | },
143 | },
144 | },
145 | },
146 | setup: func(tb testing.TB) {
147 | createTmpIndexHelper(tb, "reindex-v0")
148 | postDocHelper(tb, "reindex-v0")
149 | },
150 | cleanup: func(tb testing.TB) {
151 | deleteIndexHelper(tb, []string{"reindex-v0", "reindex-v1"})
152 | },
153 | },
154 | {
155 | name: "reindex-already-exists",
156 | conf: config{
157 | Indices: []index{
158 | {
159 | Name: "reindex-exists",
160 | Mapping: "testdata/test.json",
161 | Reindex: reindex{
162 | Source: "reindex-v0",
163 | Slices: 3,
164 | WaitForCompletion: true,
165 | On: "firstCreated",
166 | },
167 | },
168 | },
169 | },
170 | setup: func(tb testing.TB) {
171 | createTmpIndexHelper(tb, "reindex-already-exists")
172 | createTmpIndexHelper(tb, "reindex-v0")
173 | postDocHelper(tb, "reindex-v0")
174 | },
175 | cleanup: func(tb testing.TB) {
176 | deleteIndexHelper(tb, []string{"reindex-already-exists", "reindex-v0"})
177 | },
178 | },
179 | }
180 |
181 | es, err := newEsClient([]string{url}, "", "")
182 | if err != nil {
183 | t.Fatal(err)
184 | }
185 |
186 | for _, tt := range tests {
187 | t.Run(tt.name, func(t *testing.T) {
188 | ctx := context.Background()
189 | if tt.setup != nil {
190 | tt.setup(t)
191 | }
192 | err := es.syncIndices(ctx, tt.conf)
193 | if err != nil {
194 | t.Error(err)
195 | }
196 | if tt.cleanup != nil {
197 | tt.cleanup(t)
198 | }
199 | })
200 | }
201 | }
202 |
203 | func TestExistIndex(t *testing.T) {
204 | tests := []struct {
205 | name string
206 | index string
207 | setup func(tb testing.TB)
208 | want bool
209 | cleanup func(tb testing.TB)
210 | }{
211 | {
212 | name: "simple",
213 | index: "exist-v1",
214 | setup: func(tb testing.TB) {
215 | createTmpIndexHelper(tb, "exist-v1")
216 | },
217 | want: true,
218 | },
219 | {
220 | name: "not-found",
221 | index: "exist-v2",
222 | setup: func(tb testing.TB) {
223 | },
224 | want: false,
225 | },
226 | }
227 |
228 | es, err := newEsClient([]string{url}, "", "")
229 | if err != nil {
230 | t.Fatal(err)
231 | }
232 |
233 | for _, tt := range tests {
234 | t.Run(tt.name, func(t *testing.T) {
235 | ctx := context.Background()
236 | tt.setup(t)
237 | ok, err := es.existIndex(ctx, tt.index)
238 | if err != nil {
239 | t.Error(err)
240 | }
241 | if ok != tt.want {
242 | t.Errorf("want: %+v, got: %+v\n", tt.want, ok)
243 | }
244 | })
245 | }
246 | }
247 |
248 | func TestDeleteIndex(t *testing.T) {
249 | tests := []struct {
250 | name string
251 | index string
252 | setup func(tb testing.TB)
253 | cleanup func(tb testing.TB)
254 | }{
255 | {
256 | name: "simple",
257 | index: "delete-v1",
258 | setup: func(tb testing.TB) {
259 | createTmpIndexHelper(tb, "delete-v1")
260 | },
261 | },
262 | }
263 |
264 | es, err := newEsClient([]string{url}, "", "")
265 | if err != nil {
266 | t.Fatal(err)
267 | }
268 |
269 | for _, tt := range tests {
270 | t.Run(tt.name, func(t *testing.T) {
271 | ctx := context.Background()
272 | tt.setup(t)
273 | err := es.deleteIndex(ctx, tt.index)
274 | if err != nil {
275 | t.Error(err)
276 | }
277 | })
278 | }
279 | }
280 |
281 | func TestSyncCloseStatus(t *testing.T) {
282 | tests := []struct {
283 | name string
284 | conf config
285 | setup func(tb testing.TB)
286 | }{
287 | {
288 | name: "exists close",
289 | conf: config{
290 | Indices: []index{
291 | {
292 | Name: "sync-close-v1",
293 | Mapping: "testdata/test.json",
294 | Status: "close",
295 | },
296 | },
297 | },
298 | setup: func(tb testing.TB) {
299 | createTmpIndexHelper(tb, "sync-close-v1")
300 | },
301 | },
302 | }
303 |
304 | es, err := newEsClient([]string{url}, "", "")
305 | if err != nil {
306 | t.Fatal(err)
307 | }
308 |
309 | for _, tt := range tests {
310 | t.Run(tt.name, func(t *testing.T) {
311 | ctx := context.Background()
312 | if tt.setup != nil {
313 | tt.setup(t)
314 | }
315 | err := es.syncCloseStatus(ctx, tt.conf)
316 | if err != nil {
317 | t.Error(err)
318 | }
319 | })
320 | }
321 | }
322 |
--------------------------------------------------------------------------------
/logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/po3rin/eskeeper/6a6db07f7147afba9689bd18c67d024bdad9143b/logo.png
--------------------------------------------------------------------------------
/main_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "fmt"
5 | "io/ioutil"
6 | "log"
7 | "os"
8 | "strings"
9 | "testing"
10 |
11 | "github.com/elastic/go-elasticsearch"
12 | "github.com/ory/dockertest"
13 | )
14 |
15 | var url string
16 |
17 | func TestMain(m *testing.M) {
18 | pool, err := dockertest.NewPool("")
19 | if err != nil {
20 | log.Fatalf("Could not connect to docker: %s", err)
21 | }
22 |
23 | resource, err := pool.Run(
24 | "docker.elastic.co/elasticsearch/elasticsearch",
25 | "7.11.1",
26 | []string{
27 | "ES_JAVA_OPTS=-Xms512m -Xmx512m",
28 | "discovery.type=single-node",
29 | "node.name=es01",
30 | },
31 | )
32 | if err != nil {
33 | log.Fatalf("Could not start resource: %s", err)
34 | }
35 |
36 | port := resource.GetPort("9200/tcp")
37 | url = fmt.Sprintf("http://localhost:%s", port)
38 |
39 | cfg := elasticsearch.Config{
40 | Addresses: []string{url},
41 | }
42 | es, err := elasticsearch.NewClient(cfg)
43 | if err != nil {
44 | log.Fatalf("Error creating the client: %s", err)
45 | }
46 |
47 | // exponential backoff-retry, because the application in the container might not be ready to accept connections yet
48 | if err := pool.Retry(func() error {
49 | res, err := es.Info()
50 | if err != nil {
51 | log.Println("waiting to be ready...")
52 | return err
53 | }
54 | defer res.Body.Close()
55 | return nil
56 | }); err != nil {
57 | log.Fatalf("could not retry to connect : %s\n", err)
58 | }
59 |
60 | code := m.Run()
61 |
62 | // You can't defer this because os.Exit doesn't care for defer
63 | if err := pool.Purge(resource); err != nil {
64 | log.Fatalf("Could not purge resource: %s", err)
65 | }
66 |
67 | os.Exit(code)
68 | }
69 |
70 | func createTmpIndexHelper(tb testing.TB, name string) {
71 | tb.Helper()
72 | conf := elasticsearch.Config{
73 | Addresses: []string{url},
74 | }
75 | es, err := elasticsearch.NewClient(conf)
76 | if err != nil {
77 | tb.Fatal(err)
78 | }
79 |
80 | f, err := os.Open("testdata/test.json")
81 | if err != nil {
82 | tb.Fatal(err)
83 | }
84 |
85 | res, err := es.Indices.Create(
86 | name,
87 | es.Indices.Create.WithBody(f),
88 | )
89 | if err != nil {
90 | tb.Fatal(err)
91 | }
92 |
93 | if res.StatusCode != 200 {
94 | body, err := ioutil.ReadAll(res.Body)
95 | if err != nil {
96 | tb.Fatal(err)
97 | }
98 | tb.Fatalf("failed to create index [index=%v, statusCode=%v, res=%v]", name, res.StatusCode, string(body))
99 | }
100 | }
101 |
102 | var testAliasQuery = `
103 | {
104 | "actions" : [
105 | { "add" : { "index" : "%v", "alias" : "%v" } }
106 | ]
107 | }`
108 |
109 | func createTmpAliasHelper(tb testing.TB, name string, index string) {
110 | tb.Helper()
111 | conf := elasticsearch.Config{
112 | Addresses: []string{url},
113 | }
114 | es, err := elasticsearch.NewClient(conf)
115 | if err != nil {
116 | tb.Fatal(err)
117 | }
118 |
119 | res, err := es.Indices.UpdateAliases(
120 | strings.NewReader(fmt.Sprintf(testAliasQuery, index, name)),
121 | )
122 | if err != nil {
123 | tb.Fatal(err)
124 | }
125 |
126 | if res.StatusCode != 200 {
127 | body, err := ioutil.ReadAll(res.Body)
128 | if err != nil {
129 | tb.Fatal(err)
130 | }
131 | tb.Fatalf("failed to create alias [index= %v, statusCode=%v, res=%v]", name, res.StatusCode, string(body))
132 | }
133 | }
134 | func deleteIndexHelper(tb testing.TB, indices []string) {
135 | conf := elasticsearch.Config{
136 | Addresses: []string{url},
137 | }
138 | es, err := elasticsearch.NewClient(conf)
139 | if err != nil {
140 | tb.Fatal(err)
141 | }
142 | d := es.Indices.Delete
143 | res, err := d(indices)
144 | if err != nil {
145 | tb.Fatalf("close index: %v", err)
146 | }
147 | if res.StatusCode != 200 {
148 | body, err := ioutil.ReadAll(res.Body)
149 | if err != nil {
150 | tb.Fatalf("failed to close index [statusCode=%v]", res.StatusCode)
151 | }
152 | tb.Fatalf("failed to close index [statusCode=%v, res=%v]", res.StatusCode, string(body))
153 | }
154 | }
155 |
156 | func closeIndexHelper(tb testing.TB, index string) {
157 | conf := elasticsearch.Config{
158 | Addresses: []string{url},
159 | }
160 | es, err := elasticsearch.NewClient(conf)
161 | if err != nil {
162 | tb.Fatal(err)
163 | }
164 | close := es.Indices.Close
165 | res, err := close([]string{index})
166 | if err != nil {
167 | tb.Fatalf("close index: %v", err)
168 | }
169 | if res.StatusCode != 200 {
170 | body, err := ioutil.ReadAll(res.Body)
171 | if err != nil {
172 | tb.Fatalf("failed to close index [index= %v, statusCode=%v]", index, res.StatusCode)
173 | }
174 | tb.Fatalf("failed to close index [index= %v, statusCode=%v, res=%v]", index, res.StatusCode, string(body))
175 | }
176 | }
177 |
178 | func postDocHelper(tb testing.TB, index string) {
179 | tb.Helper()
180 | conf := elasticsearch.Config{
181 | Addresses: []string{url},
182 | }
183 | es, err := elasticsearch.NewClient(conf)
184 | if err != nil {
185 | tb.Fatal(err)
186 | }
187 |
188 | body := strings.NewReader(`{"title":"this is title","body":"this is body"}`)
189 |
190 | res, err := es.Index(
191 | index,
192 | body,
193 | es.Index.WithRefresh("true"),
194 | )
195 | if err != nil {
196 | tb.Fatal(err)
197 | }
198 |
199 | if res.StatusCode != 201 {
200 | body, err := ioutil.ReadAll(res.Body)
201 | if err != nil {
202 | tb.Fatal(err)
203 | }
204 | tb.Fatalf("failed to post document [index=%v, statusCode=%v, res=%v]", index, res.StatusCode, string(body))
205 | }
206 | }
207 |
--------------------------------------------------------------------------------
/reindex.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "fmt"
6 | "io/ioutil"
7 | "strings"
8 | )
9 |
10 | func (c *esclient) reindex(ctx context.Context, dest string, reindex reindex) error {
11 | ri := c.client.Reindex
12 | body := strings.NewReader(
13 | fmt.Sprintf(`
14 | {
15 | "source": {
16 | "index": "%s"
17 | },
18 | "dest": {
19 | "index": "%s"
20 | }
21 | }`,
22 | reindex.Source, dest,
23 | ),
24 | )
25 |
26 | slices := reindex.Slices
27 | if slices == 0 {
28 | slices = 1
29 | }
30 |
31 | res, err := ri(
32 | body,
33 | ri.WithContext(ctx),
34 | ri.WithSlices(slices),
35 | ri.WithWaitForCompletion(reindex.WaitForCompletion),
36 | )
37 | if err != nil {
38 | return fmt.Errorf("reindex: %w", err)
39 | }
40 | if res.StatusCode != 200 {
41 | body, err := ioutil.ReadAll(res.Body)
42 | if err != nil {
43 | return err
44 | }
45 | return fmt.Errorf("failed to reindex [index=%v, statusCode=%v, res=%v]", reindex.Source, res.StatusCode, string(body))
46 | }
47 | return nil
48 | }
49 |
--------------------------------------------------------------------------------
/reindex_test.go:
--------------------------------------------------------------------------------
1 | package eskeeper
2 |
3 | import (
4 | "context"
5 | "testing"
6 | )
7 |
8 | func TestReindex(t *testing.T) {
9 | tests := []struct {
10 | name string
11 | dest string
12 | reindex reindex
13 | setup func(tb testing.TB)
14 | cleanup func(tb testing.TB)
15 | }{
16 | {
17 | name: "reindex",
18 | dest: "reindex-dest",
19 | reindex: reindex{
20 | Source: "reindex-src",
21 | Slices: 3,
22 | WaitForCompletion: true,
23 | On: "firstCreated",
24 | },
25 | setup: func(tb testing.TB) {
26 | createTmpIndexHelper(tb, "reindex-src")
27 | createTmpIndexHelper(tb, "reindex-dest")
28 | postDocHelper(tb, "reindex-src")
29 | },
30 | cleanup: func(tb testing.TB) {
31 | deleteIndexHelper(tb, []string{"reindex-src", "reindex-dest"})
32 | },
33 | },
34 | }
35 |
36 | es, err := newEsClient([]string{url}, "", "")
37 | if err != nil {
38 | t.Fatal(err)
39 | }
40 |
41 | for _, tt := range tests {
42 | t.Run(tt.name, func(t *testing.T) {
43 | ctx := context.Background()
44 | if tt.setup != nil {
45 | tt.setup(t)
46 | }
47 | err := es.reindex(ctx, tt.dest, tt.reindex)
48 | if err != nil {
49 | t.Error(err)
50 | }
51 | if tt.cleanup != nil {
52 | tt.cleanup(t)
53 | }
54 | })
55 | }
56 | }
57 |
--------------------------------------------------------------------------------
/testdata/es.reindex.yaml:
--------------------------------------------------------------------------------
1 | index:
2 | - name: test-v1
3 | mapping: testdata/test.json
4 |
5 | # reindex test-v1 -> reindex-v1
6 | - name: reindex-v1
7 | mapping: testdata/test.json
8 | reindex:
9 | source: test-v1
10 | slices: 3 # default=1
11 | waitForCompletion: true
12 | # 'on' field supports 2 hooks.
13 | # 'reindex': only when index is created for the first time.
14 | # 'always': always exec reindex.
15 | on: firstCreated
16 |
17 |
18 | alias:
19 | - name: alias1
20 | index:
21 | - test-v2
22 |
23 |
--------------------------------------------------------------------------------
/testdata/es.yaml:
--------------------------------------------------------------------------------
1 | index:
2 | - name: test-v1 # index name
3 | mapping: testdata/test.json # index setting & mapping (json)
4 |
5 | - name: test-v2
6 | mapping: testdata/test.json
7 |
8 | - name: close-v1
9 | mapping: testdata/test.json
10 | status: close
11 |
12 | alias:
13 | - name: alias1
14 | index:
15 | - test-v1
16 |
17 | # multi indicies
18 | - name: alias2
19 | index:
20 | - test-v1
21 | - test-v2
22 |
--------------------------------------------------------------------------------
/testdata/invalid.json:
--------------------------------------------------------------------------------
1 | {
2 | "mappings": {
3 | "roperties": {
4 | "id": {
5 | "type": "long",
6 | "index": true
7 | },
8 | "title": {
9 | "type": "text"
10 | },
11 | "body": {
12 | "type": "text"
13 | }
14 | }
15 | }
16 | }
17 |
18 |
--------------------------------------------------------------------------------
/testdata/invalidUpdateIndex.json:
--------------------------------------------------------------------------------
1 | {
2 | "settings":{
3 | "number_of_shards": 2,
4 | "number_of_replicas" : 3,
5 | "analysis":{
6 | "analyzer":{
7 | "my_analyzer":{
8 | "type":"custom",
9 | "tokenizer":"standard",
10 | "filter":[
11 | "lowercase"
12 | ]
13 | },
14 | "my_stop_analyzer":{
15 | "type":"custom",
16 | "tokenizer":"standard",
17 | "filter":[
18 | "lowercase",
19 | "english_stop"
20 | ]
21 | }
22 | },
23 | "filter":{
24 | "english_stop":{
25 | "type":"stop",
26 | "stopwords":"_english_"
27 | }
28 | }
29 | }
30 | },
31 | "mappings": {
32 | "properties": {
33 | "id": {
34 | "type": "long",
35 | "index": true
36 | },
37 | "title": {
38 | "type": "text"
39 | },
40 | "body": {
41 | "type": "text"
42 | }
43 | }
44 | }
45 | }
46 |
47 |
--------------------------------------------------------------------------------
/testdata/test.json:
--------------------------------------------------------------------------------
1 | {
2 | "settings":{
3 | "number_of_shards": 1,
4 | "number_of_replicas" : 1,
5 | "analysis":{
6 | "analyzer":{
7 | "test_analyzer":{
8 | "type":"custom",
9 | "tokenizer":"standard",
10 | "filter":[
11 | "lowercase"
12 | ]
13 | },
14 | "my_stop_analyzer":{
15 | "type":"custom",
16 | "tokenizer":"standard",
17 | "filter":[
18 | "lowercase",
19 | "english_stop"
20 | ]
21 | }
22 | },
23 | "filter":{
24 | "english_stop":{
25 | "type":"stop",
26 | "stopwords":"_english_"
27 | }
28 | }
29 | }
30 | },
31 | "mappings": {
32 | "properties": {
33 | "id": {
34 | "type": "long",
35 | "index": true
36 | },
37 | "title": {
38 | "type": "text"
39 | },
40 | "body": {
41 | "type": "text"
42 | }
43 | }
44 | }
45 | }
46 |
47 |
--------------------------------------------------------------------------------
/testdata/updateIndex.json:
--------------------------------------------------------------------------------
1 | {
2 | "settings":{
3 | "number_of_shards": 1,
4 | "number_of_replicas" : 1,
5 | "analysis":{
6 | "analyzer":{
7 | "my_analyzer":{
8 | "type":"custom",
9 | "tokenizer":"standard",
10 | "filter":[
11 | "lowercase"
12 | ]
13 | },
14 | "my_stop_analyzer":{
15 | "type":"custom",
16 | "tokenizer":"standard",
17 | "filter":[
18 | "lowercase",
19 | "english_stop"
20 | ]
21 | }
22 | },
23 | "filter":{
24 | "english_stop":{
25 | "type":"stop",
26 | "stopwords":"_english_"
27 | }
28 | }
29 | }
30 | },
31 | "mappings": {
32 | "properties": {
33 | "id": {
34 | "type": "long",
35 | "index": true
36 | },
37 | "title": {
38 | "type": "text"
39 | },
40 | "body": {
41 | "type": "text"
42 | },
43 | "append": {
44 | "type": "text"
45 | }
46 | }
47 | }
48 | }
49 |
50 |
--------------------------------------------------------------------------------