├── .github ├── ISSUE_TEMPLATE.md └── PULL_REQUEST_TEMPLATE.md ├── .gitignore ├── .travis.yml ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── Makefile ├── README.md ├── api ├── entry_data.go ├── server.go └── settings.go ├── cmd ├── api_cmd.go ├── build_cmd.go ├── root_cmd.go ├── serve_cmd.go └── version_cmd.go ├── comments ├── builder.go ├── comments.go └── server.go ├── conf ├── configuration.go └── reflect.go ├── go.mod ├── go.sum └── main.go /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 21 | 22 | **- Do you want to request a *feature* or report a *bug*?** 23 | 24 | **- What is the current behavior?** 25 | 26 | **- If the current behavior is a bug, please provide the steps to reproduce.** 27 | 28 | **- What is the expected behavior?** 29 | 30 | **- Please mention your Go version, and operating system version.** 31 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 13 | 14 | **- Summary** 15 | 16 | 20 | 21 | **- Test plan** 22 | 23 | 27 | 28 | **- Description for the changelog** 29 | 30 | 34 | 35 | **- A picture of a cute animal (not mandatory but encouraged)** 36 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | vendor 2 | netlify-comments 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | language: go 3 | 4 | go: 5 | - 1.13 6 | 7 | install: make deps 8 | script: make all 9 | notifications: 10 | email: false 11 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of experience, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at david@netlify.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at [http://contributor-covenant.org/version/1/4][version] 72 | 73 | [homepage]: http://contributor-covenant.org 74 | [version]: http://contributor-covenant.org/version/1/4/ 75 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # CONTRIBUTING 2 | 3 | Contributions are always welcome, no matter how large or small. Before contributing, 4 | please read the [code of conduct](CODE_OF_CONDUCT.md). 5 | 6 | ## Setup 7 | 8 | > Install Go and Glide https://github.com/Masterminds/glide 9 | 10 | ```sh 11 | $ git clone https://github.com/netlify/gotell 12 | $ cd gotell 13 | $ make deps 14 | ``` 15 | 16 | ## Building 17 | 18 | ```sh 19 | $ make build 20 | ``` 21 | 22 | ## Testing 23 | 24 | ```sh 25 | $ make test 26 | ``` 27 | 28 | ## Pull Requests 29 | 30 | We actively welcome your pull requests. 31 | 32 | 1. Fork the repo and create your branch from `master`. 33 | 2. If you've added code that should be tested, add tests. 34 | 3. If you've changed APIs, update the documentation. 35 | 4. Ensure the test suite passes. 36 | 5. Make sure your code lints. 37 | 38 | ## License 39 | 40 | By contributing to Netlify CMS, you agree that your contributions will be licensed 41 | under its [MIT license](LICENSE). 42 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang:1.13 2 | 3 | ADD . /go/src/github.com/netlify/gotell 4 | 5 | RUN useradd -m netlify && cd /go/src/github.com/netlify/gotell && make deps build && mv gotell /usr/local/bin/ 6 | 7 | USER netlify 8 | CMD ["gotell"] 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016 Netlify 2 | 3 | MIT License 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining 6 | a copy of this software and associated documentation files (the 7 | "Software"), to deal in the Software without restriction, including 8 | without limitation the rights to use, copy, modify, merge, publish, 9 | distribute, sublicense, and/or sell copies of the Software, and to 10 | permit persons to whom the Software is furnished to do so, subject to 11 | the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be 14 | included in all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 19 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 20 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 22 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 23 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PONY: all build deps image lint test 2 | 3 | help: ## Show this help. 4 | @awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {sub("\\\\n",sprintf("\n%22c"," "), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST) 5 | 6 | all: test build ## Run the tests and build the binary. 7 | 8 | build: ## Build the binary. 9 | go build -ldflags "-X github.com/netlify/gotell/cmd.Version=`git rev-parse HEAD`" 10 | 11 | deps: ## Install dependencies. 12 | @go get -u golang.org/x/lint/golint 13 | @go mod download 14 | 15 | image: ## Build the Docker image. 16 | docker build . 17 | 18 | lint: ## Lint the code 19 | golint `go list ./... | grep -v /vendor/` 20 | 21 | test: ## Run tests. 22 | go test -v `go list ./... | grep -v /vendor/` 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GoTell 2 | 3 | A commenting system for [JAMstack sites](https://jamstack.org). 4 | 5 | ## How it works 6 | 7 | GoTell is both a build tool and a small API. 8 | 9 | The API accepts HTTP POST requests to a thread with a JSON body like: 10 | 11 | * **POST** /2016/09/this-is-a-thread-on-my-site 12 | 13 | ```json 14 | { 15 | "author": "Matt Biilmann", 16 | "email": "joe@example.com", 17 | "www": "https://www.example.com", 18 | "body": "Hi there - this is a fantastic comment!" 19 | } 20 | ``` 21 | 22 | GoTell will check to see that the thread exists and verify that it is 23 | still open, run some checks on the comment to classify obvious spam, and then push 24 | the comment to a Github repository as a JSON document. 25 | 26 | That will trigger a build through Netlify with GoTell and a new version 27 | of the thread will be pushed as a JSON file to a static endpoint. 28 | 29 | From your site, you can fetch comments and comment metadata from the static endpoint 30 | and let users POST new comments via the API. 31 | 32 | GoTell is not a ready made comment system like Disqus or Facebook Comments, 33 | but a buildingblock for making your own custom styled comments on your site. 34 | 35 | ## Getting Started 36 | 37 | ### Setting up the static Comments 38 | 39 | First clone our [Netlify Comments starter template](https://github.com/netlify/netlify-comments-starter) and push it to your own GitHub account. 40 | 41 | Then visit [Netlify](https://app.netlify.com/signup) and pick your new repository. Click **Save** and Netlify will start building your comment threads 42 | 43 | ### Setting up the API 44 | 45 | You'll need to run the API on a server. On the server, we recommend settings these environment variables: 46 | 47 | ```bash 48 | GOTELL_SITE_URL=https://mysite.example.com # URL to your static site 49 | GOTELL_REPOSITORY=user/repo # Username/repo of the GitHub repository created from netliy-comments-starter 50 | GOTELL_ACCESS_TOKEN=1253523421313 # A Personal GitHub Access Token with write permissions to the repository 51 | ``` 52 | 53 | With these environment variables in place, run: 54 | 55 | ```bash 56 | gotell api 57 | ``` 58 | 59 | ### Integrating with your site 60 | 61 | Each post on your static site that should have comments, needs to add a metadata tag to it's page like this: 62 | 63 | ```html 64 | 65 | ``` 66 | 67 | To configure GoTell add a file called `/gotell/settings.json` to your site (this is optional). It should look like this: 68 | 69 | ```json 70 | { 71 | "banned_ips": [], 72 | "banned_keywords": [], 73 | "banned_emails": [], 74 | "timelimit": 604800 75 | } 76 | ``` 77 | 78 | These settings controls the rudimentary spam filter and the time limit from a post is created and until 79 | commenting is closed for the thread. 80 | 81 | To allow people to comment you'll need your comment form to send a request to the comment API. Here's an 82 | example using the modern `fetch` API: 83 | 84 | ```js 85 | const thread = document.location.pathname; 86 | fetch(API_URL + thread, { 87 | method: 'POST', 88 | headers: {'Content-Type': 'application/json'}, 89 | body: JSON.stringify({ 90 | author: data.name, 91 | email: data.email, 92 | body: data.message, 93 | parent: data.parent 94 | }) 95 | }).then((response) => { 96 | console.log("Comment posted!"); 97 | }); 98 | ``` 99 | 100 | To display comments for a thread, fetch the JSON via: 101 | 102 | ```js 103 | const slug = document.location.pathname.replace(/\//g, '-').replace(/(^-|-$)/g, '') + '.json'; 104 | fetch(COMMENT_URL + '/' + slug).then((response) => { 105 | console.log("Got comments: %o", response); 106 | }); 107 | ``` 108 | 109 | GoTell also builds a file called `threadname.count.json` for each thread with a JSON 110 | object looking like: 111 | 112 | ```json 113 | {"count": 42} 114 | ``` 115 | 116 | As a lower bandwidth way to fetch comment counts for a thread. 117 | 118 | ## Licence 119 | 120 | GoTell is released under the [MIT License](LICENSE). 121 | Please make sure you understand its [implications and guarantees](https://writing.kemitchell.com/2016/09/21/MIT-License-Line-by-Line.html). 122 | -------------------------------------------------------------------------------- /api/entry_data.go: -------------------------------------------------------------------------------- 1 | package api 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "net/http" 7 | "regexp" 8 | "strings" 9 | "time" 10 | 11 | "github.com/PuerkitoBio/goquery" 12 | ) 13 | 14 | var cleanPathRE = regexp.MustCompile("(^-+|-+$)") 15 | 16 | type entryData struct { 17 | Thread string 18 | CreatedAt time.Time 19 | } 20 | 21 | func (s *Server) entryData(path string) (*entryData, error) { 22 | resp, err := http.Get(s.config.API.SiteURL + path) 23 | if err != nil { 24 | return nil, err 25 | } 26 | defer resp.Body.Close() 27 | 28 | doc, err := goquery.NewDocumentFromResponse(resp) 29 | if err != nil { 30 | return nil, err 31 | } 32 | 33 | metaTag := doc.Find("#gotell").First() 34 | if metaTag.Length() == 0 { 35 | return nil, fmt.Errorf("No script tag with id gotell found for '%v'", path) 36 | } 37 | entryData := &entryData{} 38 | if err := json.Unmarshal([]byte(metaTag.Text()), entryData); err != nil { 39 | return nil, err 40 | } 41 | 42 | if entryData.Thread == "" { 43 | entryData.Thread = strings.Replace(path, "/", "-", -1) 44 | entryData.Thread = cleanPathRE.ReplaceAllLiteralString(entryData.Thread, "") 45 | } 46 | 47 | return entryData, nil 48 | } 49 | -------------------------------------------------------------------------------- /api/server.go: -------------------------------------------------------------------------------- 1 | package api 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "fmt" 7 | "net/http" 8 | "path" 9 | "regexp" 10 | "strings" 11 | "sync" 12 | "time" 13 | 14 | "github.com/Sirupsen/logrus" 15 | jwt "github.com/dgrijalva/jwt-go" 16 | "github.com/google/go-github/github" 17 | "github.com/guregu/kami" 18 | "github.com/netlify/gotell/comments" 19 | "github.com/netlify/gotell/conf" 20 | "github.com/rs/cors" 21 | "github.com/zenazn/goji/web/mutil" 22 | ) 23 | 24 | const defaultVersion = "unknown version" 25 | 26 | var threadRegexp = regexp.MustCompile(`(\d+)-(\d+)-(.+)`) 27 | var slugify = regexp.MustCompile(`\W`) 28 | var squeeze = regexp.MustCompile(`-+`) 29 | var bearerRegexp = regexp.MustCompile(`^(?:B|b)earer (\S+$)`) 30 | 31 | type Server struct { 32 | handler http.Handler 33 | config *conf.Configuration 34 | client *github.Client 35 | settings *settings 36 | mutex sync.Mutex 37 | version string 38 | } 39 | 40 | func Min(x, y int) int { 41 | if x < y { 42 | return x 43 | } 44 | return y 45 | } 46 | 47 | func (s *Server) postComment(ctx context.Context, w http.ResponseWriter, req *http.Request) { 48 | entryPath := req.URL.Path 49 | 50 | w.Header().Set("Content-Type", "application/json") 51 | 52 | settings := s.getSettings() 53 | for _, ip := range settings.BannedIPs { 54 | if req.RemoteAddr == ip { 55 | w.Header().Add("X-Banned", "IP-Banned") 56 | fmt.Fprintln(w, "{}") 57 | return 58 | } 59 | } 60 | 61 | entryData, err := s.entryData(entryPath) 62 | if err != nil { 63 | jsonError(w, fmt.Sprintf("Unable to read entry data: %v", err), 400) 64 | return 65 | } 66 | if settings.TimeLimit != 0 && time.Now().Sub(entryData.CreatedAt) > time.Duration(settings.TimeLimit) { 67 | jsonError(w, "Thread is closed for new comments", 401) 68 | return 69 | } 70 | 71 | comment := &comments.RawComment{} 72 | decoder := json.NewDecoder(req.Body) 73 | if err := decoder.Decode(comment); err != nil { 74 | jsonError(w, fmt.Sprintf("Error decoding JSON body: %v", err), 422) 75 | return 76 | } 77 | 78 | for _, email := range settings.BannedEmails { 79 | if strings.Contains(comment.Email, email) || strings.Contains(comment.Body, email) || strings.Contains(comment.URL, email) { 80 | w.Header().Add("X-Banned", "Email-Banned") 81 | fmt.Fprintln(w, "{}") 82 | return 83 | } 84 | } 85 | 86 | for _, keyword := range settings.BannedKeywords { 87 | if strings.Contains(comment.Email, keyword) || strings.Contains(comment.Body, keyword) || strings.Contains(comment.URL, keyword) { 88 | w.Header().Add("X-Banned", "Keyword-Banned") 89 | fmt.Fprintln(w, "{}") 90 | return 91 | } 92 | } 93 | 94 | comment.IP = req.RemoteAddr 95 | comment.Date = time.Now().String() 96 | comment.ID = fmt.Sprintf("%v", time.Now().UnixNano()) 97 | comment.Verified = s.verify(comment.Email, req) 98 | 99 | parts := strings.Split(s.config.API.Repository, "/") 100 | matches := threadRegexp.FindStringSubmatch(entryData.Thread) 101 | dir := matches[1] + "/" + matches[2] + "/" + matches[3] 102 | firstParagraph := strings.SplitAfterN(strings.ToLower(strings.TrimSpace(comment.Body[0:len(comment.Body)])), "\n", 1)[0] 103 | name := squeeze.ReplaceAllString(strings.Trim(slugify.ReplaceAllString(firstParagraph[0:Min(50, len(firstParagraph))], "-"), "-"), "-") 104 | 105 | pathname := path.Join( 106 | s.config.Threads.Source, 107 | dir, 108 | fmt.Sprintf("%v-%v.json", (time.Now().UnixNano()/1000000), name), 109 | ) 110 | 111 | message := firstParagraph 112 | if len(message) > 255 { 113 | message = message[:255] 114 | } 115 | content, _ := json.Marshal(comment) 116 | branch := "master" 117 | 118 | if settings.RequireApproval || comment.IsSuspicious() { 119 | branch = "comment-" + comment.ID 120 | master, _, err := s.client.Repositories.GetBranch(ctx, parts[0], parts[1], "master") 121 | sha := master.Commit.GetSHA() 122 | refName := "refs/heads/" + branch 123 | if err != nil { 124 | jsonError(w, fmt.Sprintf("Failed to write comment: %v", err), 500) 125 | return 126 | } 127 | 128 | _, _, err = s.client.Git.CreateRef(ctx, parts[0], parts[1], &github.Reference{ 129 | Ref: &refName, 130 | Object: &github.GitObject{SHA: &sha}, 131 | }) 132 | if err != nil { 133 | jsonError(w, fmt.Sprintf("Failed to create comment branch: %v", err), 500) 134 | return 135 | } 136 | _, _, err = s.client.Repositories.CreateFile(ctx, parts[0], parts[1], pathname, &github.RepositoryContentFileOptions{ 137 | Message: &message, 138 | Content: content, 139 | Branch: &branch, 140 | }) 141 | 142 | if err != nil { 143 | jsonError(w, fmt.Sprintf("Failed to write comment: %v", err), 500) 144 | return 145 | } 146 | 147 | pr := &github.NewPullRequest{ 148 | Title: &message, 149 | Head: &branch, 150 | Base: master.Name, 151 | } 152 | _, _, err = s.client.PullRequests.Create(ctx, parts[0], parts[1], pr) 153 | if err != nil { 154 | jsonError(w, fmt.Sprintf("Failed to create PR: %v", err), 500) 155 | return 156 | } 157 | } else { 158 | _, _, err = s.client.Repositories.CreateFile(ctx, parts[0], parts[1], pathname, &github.RepositoryContentFileOptions{ 159 | Message: &message, 160 | Content: content, 161 | Branch: &branch, 162 | }) 163 | 164 | if err != nil { 165 | jsonError(w, fmt.Sprintf("Failed to write comment: %v", err), 500) 166 | return 167 | } 168 | } 169 | 170 | parsedComment := comments.ParseRaw(comment) 171 | response, _ := json.Marshal(parsedComment) 172 | w.Write(response) 173 | } 174 | 175 | func (s *Server) verify(email string, r *http.Request) bool { 176 | authHeader := r.Header.Get("Authorization") 177 | if authHeader == "" { 178 | logrus.Info("No auth header") 179 | return false 180 | } 181 | 182 | matches := bearerRegexp.FindStringSubmatch(authHeader) 183 | if len(matches) != 2 { 184 | logrus.Info("Not a bearer auth header") 185 | return false 186 | } 187 | 188 | token, err := jwt.Parse(matches[1], func(token *jwt.Token) (interface{}, error) { 189 | if token.Method.Alg() != jwt.SigningMethodHS256.Name { 190 | return nil, fmt.Errorf("Unexpected signing method: %v", token.Method.Alg()) 191 | } 192 | return []byte(s.config.JWT.Secret), nil 193 | }) 194 | if err != nil { 195 | logrus.Errorf("Error verifying JWT: %v", err) 196 | return false 197 | } 198 | 199 | if claims, ok := token.Claims.(jwt.MapClaims); ok && token.Valid { 200 | claimedEmail, ok := claims["email"] 201 | logrus.Infof("Checking email %v from claims %v against %v", claimedEmail, claims, email) 202 | return ok && claimedEmail == email 203 | } 204 | 205 | return false 206 | } 207 | 208 | // Index endpoint 209 | func (s *Server) index(ctx context.Context, w http.ResponseWriter, r *http.Request) { 210 | sendJSON(w, 200, map[string]string{ 211 | "version": s.version, 212 | "name": "GoTell", 213 | "description": "GoTell is an API and build tool for handling large amounts of comments for JAMstack products", 214 | }) 215 | } 216 | 217 | // ListenAndServe starts the Comments Server 218 | func (s *Server) ListenAndServe() error { 219 | l := fmt.Sprintf("%v:%v", s.config.API.Host, s.config.API.Port) 220 | logrus.Infof("GoTell API started on: %s", l) 221 | return http.ListenAndServe(l, s.handler) 222 | } 223 | 224 | func NewServer(config *conf.Configuration, githubClient *github.Client) *Server { 225 | return NewServerWithVersion(config, githubClient, defaultVersion) 226 | } 227 | 228 | func NewServerWithVersion(config *conf.Configuration, githubClient *github.Client, version string) *Server { 229 | s := &Server{ 230 | config: config, 231 | client: githubClient, 232 | version: version, 233 | } 234 | 235 | mux := kami.New() 236 | mux.LogHandler = logHandler 237 | mux.Use("/", timeRequest) 238 | mux.Use("/", jsonTypeRequired) 239 | mux.Get("/", s.index) 240 | mux.Post("/*path", s.postComment) 241 | 242 | corsHandler := cors.New(cors.Options{ 243 | AllowedMethods: []string{"GET", "POST", "PATCH", "PUT", "DELETE"}, 244 | AllowedHeaders: []string{"Accept", "Authorization", "Content-Type"}, 245 | ExposedHeaders: []string{"Link", "X-Total-Count"}, 246 | AllowCredentials: true, 247 | }) 248 | 249 | s.handler = corsHandler.Handler(mux) 250 | return s 251 | } 252 | 253 | func timeRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) context.Context { 254 | return context.WithValue(ctx, "_gotell_timing", time.Now()) 255 | } 256 | 257 | func logHandler(ctx context.Context, wp mutil.WriterProxy, req *http.Request) { 258 | start := ctx.Value("_gotell_timing").(time.Time) 259 | logrus.WithFields(logrus.Fields{ 260 | "method": req.Method, 261 | "path": req.URL.Path, 262 | "status": wp.Status(), 263 | "duration": time.Since(start), 264 | }).Info("") 265 | } 266 | 267 | func jsonTypeRequired(ctx context.Context, w http.ResponseWriter, r *http.Request) context.Context { 268 | if r.Method == "POST" && r.Header.Get("Content-Type") != "application/json" { 269 | http.Error(w, "Content-Type must be application/json", 422) 270 | return nil 271 | } 272 | return ctx 273 | } 274 | 275 | func sendJSON(w http.ResponseWriter, status int, obj interface{}) { 276 | w.Header().Set("Content-Type", "application/json") 277 | w.WriteHeader(status) 278 | encoder := json.NewEncoder(w) 279 | encoder.Encode(obj) 280 | } 281 | 282 | func jsonError(w http.ResponseWriter, message string, status int) { 283 | w.WriteHeader(status) 284 | encoder := json.NewEncoder(w) 285 | encoder.Encode(map[string]string{"msg": message}) 286 | } 287 | -------------------------------------------------------------------------------- /api/settings.go: -------------------------------------------------------------------------------- 1 | package api 2 | 3 | import ( 4 | "encoding/json" 5 | "net/http" 6 | "time" 7 | 8 | "github.com/Sirupsen/logrus" 9 | ) 10 | 11 | const settingsRefreshTime = 10 * time.Minute 12 | 13 | type settings struct { 14 | BannedIPs []string `json:"banned_ips"` 15 | BannedKeywords []string `json:"banned_keywords"` 16 | BannedEmails []string `json:"banned_emails"` 17 | RequireApproval bool `json:"require_approval"` 18 | TimeLimit int `json:"timelimit"` 19 | lastLoad time.Time 20 | } 21 | 22 | func (s *settings) fresh() bool { 23 | return time.Since(s.lastLoad) < settingsRefreshTime 24 | } 25 | 26 | func (s *Server) getSettings() *settings { 27 | if s.settings != nil && s.settings.fresh() { 28 | return s.settings 29 | } 30 | 31 | resp, err := http.Get(s.config.API.SiteURL + "/gotell/settings.json") 32 | if err != nil { 33 | return &settings{} 34 | } 35 | 36 | defer resp.Body.Close() 37 | st := &settings{} 38 | 39 | decoder := json.NewDecoder(resp.Body) 40 | if err := decoder.Decode(st); err != nil { 41 | logrus.Warnf("Error decoding settings: %v", err) 42 | return st 43 | } 44 | 45 | st.lastLoad = time.Now() 46 | s.mutex.Lock() 47 | s.settings = st 48 | s.mutex.Unlock() 49 | 50 | return st 51 | } 52 | -------------------------------------------------------------------------------- /cmd/api_cmd.go: -------------------------------------------------------------------------------- 1 | package cmd 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "net/http" 7 | "strings" 8 | 9 | "golang.org/x/oauth2" 10 | 11 | "github.com/Sirupsen/logrus" 12 | "github.com/google/go-github/github" 13 | "github.com/netlify/gotell/api" 14 | "github.com/netlify/gotell/conf" 15 | "github.com/spf13/cobra" 16 | ) 17 | 18 | func apiCommand() *cobra.Command { 19 | return &cobra.Command{ 20 | Use: "api", 21 | Short: "api", 22 | Run: func(cmd *cobra.Command, args []string) { 23 | execWithConfig(cmd, serveAPI) 24 | }, 25 | } 26 | } 27 | 28 | func serveAPI(config *conf.Configuration) { 29 | if err := verifyAPISettings(config); err != nil { 30 | logrus.Fatalf("Error verifying settings: %v", err) 31 | } 32 | 33 | if err := verifySite(config.API.SiteURL); err != nil { 34 | logrus.Fatalf("Error verifying site: %v", err) 35 | } 36 | 37 | githubClient := newGitHubClient(config) 38 | if err := verifyRepoAndToken(config.API.Repository, githubClient); err != nil { 39 | logrus.Fatalf("Error verifying repo: %v", err) 40 | } 41 | 42 | server := api.NewServerWithVersion(config, githubClient, Version) 43 | server.ListenAndServe() 44 | } 45 | 46 | func verifyAPISettings(config *conf.Configuration) error { 47 | if config.API.SiteURL == "" { 48 | return fmt.Errorf("API requires a site url") 49 | } 50 | 51 | if config.API.Repository == "" { 52 | return fmt.Errorf("API requires a GitHub repository path") 53 | } 54 | 55 | if config.API.AccessToken == "" { 56 | return fmt.Errorf("API requires a GitHub access token") 57 | } 58 | 59 | return nil 60 | } 61 | 62 | func verifySite(url string) error { 63 | resp, err := http.Get(url) 64 | if err != nil { 65 | return err 66 | } 67 | if resp.StatusCode != 200 { 68 | return fmt.Errorf("Expected 200 status code for %v, got %v", url, resp.StatusCode) 69 | } 70 | return nil 71 | } 72 | 73 | func verifyRepoAndToken(repository string, client *github.Client) error { 74 | parts := strings.Split(repository, "/") 75 | if len(parts) != 2 { 76 | return fmt.Errorf("Repo format must be owner/repo - %v", repository) 77 | } 78 | ctx := context.Background() 79 | 80 | _, _, err := client.Repositories.Get(ctx, parts[0], parts[1]) 81 | return err 82 | } 83 | 84 | func newGitHubClient(config *conf.Configuration) *github.Client { 85 | ts := oauth2.StaticTokenSource( 86 | &oauth2.Token{AccessToken: config.API.AccessToken}, 87 | ) 88 | tc := oauth2.NewClient(oauth2.NoContext, ts) 89 | return github.NewClient(tc) 90 | } 91 | -------------------------------------------------------------------------------- /cmd/build_cmd.go: -------------------------------------------------------------------------------- 1 | package cmd 2 | 3 | import ( 4 | "github.com/netlify/gotell/comments" 5 | "github.com/spf13/cobra" 6 | ) 7 | 8 | func buildCommand() *cobra.Command { 9 | return &cobra.Command{ 10 | Use: "build", 11 | Short: "build", 12 | Run: func(cmd *cobra.Command, args []string) { 13 | execWithConfig(cmd, comments.Build) 14 | }, 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /cmd/root_cmd.go: -------------------------------------------------------------------------------- 1 | package cmd 2 | 3 | import ( 4 | "github.com/Sirupsen/logrus" 5 | "github.com/netlify/gotell/conf" 6 | "github.com/spf13/cobra" 7 | ) 8 | 9 | func RootCommand() *cobra.Command { 10 | rootCmd := cobra.Command{ 11 | Use: "gotell", 12 | Run: run, 13 | } 14 | 15 | rootCmd.AddCommand(buildCommand(), serveCommand(), apiCommand(), &versionCmd) 16 | rootCmd.PersistentFlags().StringP("config", "c", "", "the config file to use") 17 | 18 | return &rootCmd 19 | } 20 | 21 | func run(cmd *cobra.Command, args []string) { 22 | execWithConfig(cmd, serveAPI) 23 | } 24 | 25 | func execWithConfig(cmd *cobra.Command, fn func(config *conf.Configuration)) { 26 | config, err := conf.LoadConfig(cmd) 27 | if err != nil { 28 | logrus.Fatalf("Failed to load configration: %+v", err) 29 | } 30 | 31 | fn(config) 32 | } 33 | -------------------------------------------------------------------------------- /cmd/serve_cmd.go: -------------------------------------------------------------------------------- 1 | package cmd 2 | 3 | import ( 4 | "github.com/netlify/gotell/comments" 5 | "github.com/netlify/gotell/conf" 6 | "github.com/spf13/cobra" 7 | ) 8 | 9 | func serveCommand() *cobra.Command { 10 | return &cobra.Command{ 11 | Use: "serve", 12 | Short: "serve", 13 | Run: func(cmd *cobra.Command, args []string) { 14 | execWithConfig(cmd, serveComments) 15 | }, 16 | } 17 | } 18 | 19 | func serveComments(config *conf.Configuration) { 20 | server := comments.NewServer(config) 21 | server.ListenAndServe() 22 | } 23 | -------------------------------------------------------------------------------- /cmd/version_cmd.go: -------------------------------------------------------------------------------- 1 | package cmd 2 | 3 | import ( 4 | "fmt" 5 | 6 | "github.com/spf13/cobra" 7 | ) 8 | 9 | var Version string 10 | 11 | var versionCmd = cobra.Command{ 12 | Run: showVersion, 13 | Use: "version", 14 | } 15 | 16 | func showVersion(cmd *cobra.Command, args []string) { 17 | fmt.Println(Version) 18 | } 19 | -------------------------------------------------------------------------------- /comments/builder.go: -------------------------------------------------------------------------------- 1 | package comments 2 | 3 | import ( 4 | "crypto/md5" 5 | "encoding/json" 6 | "fmt" 7 | "io/ioutil" 8 | "log" 9 | "os" 10 | "path" 11 | "path/filepath" 12 | "regexp" 13 | "strings" 14 | "sync" 15 | 16 | "github.com/Sirupsen/logrus" 17 | "github.com/netlify/gotell/conf" 18 | ) 19 | 20 | var ( 21 | filenameRegexp = regexp.MustCompile(`(?:[^/]+)/(\d+)/(\d+)/([^/]+)`) 22 | ) 23 | 24 | func Build(config *conf.Configuration) { 25 | if err := os.MkdirAll(config.Threads.Destination, 0755); err != nil { 26 | logrus.Fatalf("Failed to create output dir: %v", err) 27 | } 28 | 29 | threads, err := filepath.Glob(config.Threads.Source + "/*/*/*") 30 | if err != nil { 31 | logrus.Fatalf("Failed to list threads: %v", err) 32 | } 33 | 34 | var wg sync.WaitGroup 35 | sem := make(chan int, 100) 36 | 37 | for _, thread := range threads { 38 | sem <- 1 39 | wg.Add(1) 40 | go func(t string) { 41 | generate(t, config.Threads.Destination) 42 | <-sem 43 | wg.Done() 44 | }(thread) 45 | } 46 | 47 | wg.Wait() 48 | 49 | emptyPath := path.Join(config.Threads.Destination, "empty.json") 50 | empty, err := os.Create(emptyPath) 51 | if err != nil { 52 | log.Fatalf("Error opening the empty file %v: %v", emptyPath, err) 53 | } 54 | defer empty.Close() 55 | empty.WriteString("[]") 56 | 57 | countPath := path.Join(config.Threads.Destination, "empty.count.json") 58 | count, err := os.Create(countPath) 59 | if err != nil { 60 | log.Fatalf("Error opening the count file %v: %v", countPath, err) 61 | } 62 | defer count.Close() 63 | count.WriteString(fmt.Sprintf("{\"count\": %v}", 0)) 64 | 65 | redirectsPath := path.Join(config.Threads.Destination, "_redirects") 66 | redirects, err := os.Create(redirectsPath) 67 | if err != nil { 68 | log.Fatalf("Error opening the redirects file %v: %v", redirectsPath, err) 69 | } 70 | defer redirects.Close() 71 | redirects.WriteString(` 72 | /*.count.json /empty.count.json 200 73 | /*.json /empty.json 200 74 | `) 75 | } 76 | 77 | func generate(source, dest string) { 78 | comments, err := ioutil.ReadDir(source) 79 | if err != nil { 80 | log.Fatalf("Failed to read thread %v: %v", source, err) 81 | } 82 | 83 | output := []*ParsedComment{} 84 | for _, comment := range comments { 85 | if strings.HasSuffix(comment.Name(), ".json") { 86 | filePath := path.Join(source, comment.Name()) 87 | reader, err := os.Open(filePath) 88 | if err != nil { 89 | log.Fatalf("Failed to open comment %v: %v", filePath, err) 90 | } 91 | defer reader.Close() 92 | 93 | data := &RawComment{} 94 | decoder := json.NewDecoder(reader) 95 | if err := decoder.Decode(data); err != nil { 96 | log.Fatalf("Failed to decode json for %v: %v", filePath, err) 97 | } 98 | output = append(output, ParseRaw(data)) 99 | } 100 | } 101 | 102 | matches := filenameRegexp.FindStringSubmatch(source) 103 | name := matches[1] + "-" + matches[2] + "-" + matches[3] 104 | 105 | distPath := path.Join(dest, name+".json") 106 | dist, err := os.Create(distPath) 107 | if err != nil { 108 | log.Fatalf("Error opening output file %v: %v", distPath, err) 109 | } 110 | defer dist.Close() 111 | 112 | encoder := json.NewEncoder(dist) 113 | if err := encoder.Encode(output); err != nil { 114 | log.Fatalf("Failed to encode json for %v: %v", distPath, err) 115 | } 116 | 117 | countPath := path.Join(dest, name+".count.json") 118 | count, err := os.Create(countPath) 119 | if err != nil { 120 | log.Fatalf("Error opening the count file %v: %v", countPath, err) 121 | } 122 | defer count.Close() 123 | count.WriteString(fmt.Sprintf("{\"count\": %v}", len(comments))) 124 | } 125 | 126 | func ParseRaw(data *RawComment) *ParsedComment { 127 | return &ParsedComment{ 128 | ID: data.ID, 129 | ParentID: data.ParentID, 130 | Author: data.Author, 131 | Twitter: data.Twitter, 132 | URL: data.URL, 133 | Body: data.Body, 134 | Date: data.Date, 135 | MD5: fmt.Sprintf("%x", md5.Sum([]byte(data.Email))), 136 | } 137 | } 138 | -------------------------------------------------------------------------------- /comments/comments.go: -------------------------------------------------------------------------------- 1 | package comments 2 | 3 | import "regexp" 4 | 5 | type RawComment struct { 6 | ID string `json:"id"` 7 | ParentID string `json:"parent"` 8 | Author string `json:"author"` 9 | Twitter string `json:"twitter"` 10 | Email string `json:"email"` 11 | Verified bool `json:"verified"` 12 | URL string `json:"www"` 13 | IP string `json:"ip"` 14 | Body string `json:"body"` 15 | Date string `json:"date"` 16 | } 17 | 18 | type ParsedComment struct { 19 | ID string `json:"id"` 20 | ParentID string `json:"parent"` 21 | Author string `json:"author"` 22 | Verified bool `json:"verified"` 23 | Twitter string `json:"twitter"` 24 | MD5 string `json:"md5"` 25 | URL string `json:"www"` 26 | Body string `json:"body"` 27 | Date string `json:"date"` 28 | } 29 | 30 | var urlRegexp = regexp.MustCompile("(?i)https?://") 31 | 32 | func (r *RawComment) IsSuspicious() bool { 33 | return urlRegexp.MatchString(r.Author + r.Twitter + r.Email + r.Body) 34 | } 35 | -------------------------------------------------------------------------------- /comments/server.go: -------------------------------------------------------------------------------- 1 | package comments 2 | 3 | import ( 4 | "context" 5 | "fmt" 6 | "net/http" 7 | "path/filepath" 8 | 9 | "github.com/Sirupsen/logrus" 10 | "github.com/guregu/kami" 11 | "github.com/netlify/gotell/conf" 12 | ) 13 | 14 | type Server struct { 15 | handler http.Handler 16 | config *conf.Configuration 17 | } 18 | 19 | // ListenAndServe starts the Comments Server 20 | func (s *Server) ListenAndServe() error { 21 | l := fmt.Sprintf("%v:%v", s.config.Threads.Host, s.config.Threads.Port) 22 | logrus.Infof("GoTell Server started on: %s", l) 23 | return http.ListenAndServe(l, s.handler) 24 | } 25 | 26 | func (s *Server) serveFile(ctx context.Context, w http.ResponseWriter, r *http.Request) { 27 | path := ctx.Value("path").(string) 28 | 29 | fs := filepath.Join(s.config.Threads.Destination, path) 30 | http.ServeFile(w, r, fs) 31 | } 32 | 33 | func NewServer(config *conf.Configuration) *Server { 34 | s := &Server{ 35 | config: config, 36 | } 37 | 38 | mux := kami.New() 39 | mux.Get("/*path", s.serveFile) 40 | s.handler = mux 41 | return s 42 | } 43 | -------------------------------------------------------------------------------- /conf/configuration.go: -------------------------------------------------------------------------------- 1 | package conf 2 | 3 | import ( 4 | "bufio" 5 | "os" 6 | "strings" 7 | 8 | "github.com/Sirupsen/logrus" 9 | "github.com/spf13/cobra" 10 | "github.com/spf13/viper" 11 | ) 12 | 13 | type Configuration struct { 14 | Threads struct { 15 | Source string `mapstructure:"source" json:"source"` 16 | Destination string `mapstructure:"destination" json:"destination"` 17 | Host string `mapstructure:"host" json:"host"` 18 | Port int `mapstructure:"port" json:"port"` 19 | } 20 | 21 | API struct { 22 | SiteURL string `mapstructure:"site_url" json:"site_url"` 23 | Repository string `mapstructure:"repository" json:"repository"` 24 | AccessToken string `mapstructure:"access_token" json:"access_token"` 25 | Host string `mapstructure:"host" json:"host"` 26 | Port int `mapstructure:"port" json:"port"` 27 | } `mapstructure:"api" json:"api"` 28 | 29 | Logging struct { 30 | Level string `mapstructure:"level" json:"level"` 31 | File string `mapstructure:"file" json:"file"` 32 | } `mapstructure:"logging" json:"logging"` 33 | 34 | JWT struct { 35 | Secret string `mapstructure:"secret" json:"secret"` 36 | } `mapstructure:"jwt" json:"jwt"` 37 | } 38 | 39 | func LoadConfig(cmd *cobra.Command) (*Configuration, error) { 40 | err := viper.BindPFlags(cmd.Flags()) 41 | if err != nil { 42 | return nil, err 43 | } 44 | 45 | viper.SetEnvPrefix("GOTELL") 46 | viper.SetDefault("threads.source", "threads") 47 | viper.SetDefault("threads.destination", "dist") 48 | viper.SetDefault("threads.port", "9091") 49 | 50 | if os.Getenv("PORT") == "" { 51 | viper.SetDefault("api.port", "9090") 52 | } 53 | 54 | viper.SetConfigType("json") 55 | viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_")) 56 | viper.AutomaticEnv() 57 | 58 | if configFile, _ := cmd.Flags().GetString("config"); configFile != "" { 59 | viper.SetConfigFile(configFile) 60 | } else { 61 | viper.SetConfigName("config") 62 | viper.AddConfigPath("./") 63 | viper.AddConfigPath("$HOME/.netlify/gotell/") 64 | } 65 | 66 | if err := viper.ReadInConfig(); err != nil && !os.IsNotExist(err) { 67 | return nil, err 68 | } 69 | 70 | config := new(Configuration) 71 | if err := viper.Unmarshal(config); err != nil { 72 | return nil, err 73 | } 74 | 75 | if err := populateConfig(config); err != nil { 76 | return nil, err 77 | } 78 | 79 | if err := configureLogging(config); err != nil { 80 | return nil, err 81 | } 82 | 83 | return config, nil 84 | } 85 | 86 | // configureLogging will take the logging configuration and also adds 87 | // a few default parameters 88 | func configureLogging(config *Configuration) error { 89 | logConfig := config.Logging 90 | 91 | // use a file if you want 92 | if logConfig.File != "" { 93 | f, errOpen := os.OpenFile(logConfig.File, os.O_RDWR|os.O_APPEND, 0660) 94 | if errOpen != nil { 95 | return errOpen 96 | } 97 | logrus.SetOutput(bufio.NewWriter(f)) 98 | } 99 | 100 | if logConfig.Level != "" { 101 | level, err := logrus.ParseLevel(strings.ToUpper(logConfig.Level)) 102 | if err != nil { 103 | return err 104 | } 105 | logrus.SetLevel(level) 106 | } 107 | 108 | // always use the fulltimestamp 109 | logrus.SetFormatter(&logrus.TextFormatter{ 110 | FullTimestamp: true, 111 | DisableTimestamp: false, 112 | }) 113 | 114 | return nil 115 | } 116 | -------------------------------------------------------------------------------- /conf/reflect.go: -------------------------------------------------------------------------------- 1 | package conf 2 | 3 | import ( 4 | "fmt" 5 | "os" 6 | "reflect" 7 | "strconv" 8 | 9 | "github.com/pkg/errors" 10 | "github.com/spf13/viper" 11 | ) 12 | 13 | const tagPrefix = "viper" 14 | 15 | func populateConfig(config *Configuration) error { 16 | if err := recursivelySet(reflect.ValueOf(config), ""); err != nil { 17 | return errors.Wrap(err, "setting configuration values") 18 | } 19 | 20 | if config.API.Port == 0 && os.Getenv("PORT") != "" { 21 | port, err := strconv.Atoi(os.Getenv("PORT")) 22 | if err != nil { 23 | return errors.Wrap(err, "formatting PORT into int") 24 | } 25 | 26 | config.API.Port = port 27 | } 28 | 29 | return nil 30 | } 31 | 32 | func recursivelySet(val reflect.Value, prefix string) error { 33 | if val.Kind() != reflect.Ptr { 34 | return errors.Wrap(fmt.Errorf("unexpected value: %v", val), "expected pointer value") 35 | } 36 | 37 | // dereference 38 | val = reflect.Indirect(val) 39 | if val.Kind() != reflect.Struct { 40 | return errors.Wrap(fmt.Errorf("unexpected value: %v", val), "expected struct value") 41 | } 42 | 43 | // grab the type for this instance 44 | vType := reflect.TypeOf(val.Interface()) 45 | 46 | // go through child fields 47 | for i := 0; i < val.NumField(); i++ { 48 | thisField := val.Field(i) 49 | thisType := vType.Field(i) 50 | tag := prefix + getTag(thisType) 51 | 52 | switch thisField.Kind() { 53 | case reflect.Struct: 54 | recursivelySet(thisField.Addr(), tag+".") 55 | case reflect.Int: 56 | fallthrough 57 | case reflect.Int32: 58 | fallthrough 59 | case reflect.Int64: 60 | // you can only set with an int64 -> int 61 | configVal := int64(viper.GetInt(tag)) 62 | thisField.SetInt(configVal) 63 | case reflect.Bool: 64 | configVal := viper.GetBool(tag) 65 | thisField.SetBool(configVal) 66 | case reflect.String: 67 | configVal := viper.GetString(tag) 68 | thisField.SetString(configVal) 69 | default: 70 | return fmt.Errorf("unexpected type detected ~ aborting: %s", thisField.Kind()) 71 | } 72 | } 73 | 74 | return nil 75 | } 76 | 77 | func getTag(field reflect.StructField) string { 78 | // check if maybe we have a special magic tag 79 | tag := field.Tag 80 | if tag != "" { 81 | for _, prefix := range []string{tagPrefix, "mapstructure", "json"} { 82 | if v := tag.Get(prefix); v != "" { 83 | return v 84 | } 85 | } 86 | } 87 | 88 | return field.Name 89 | } 90 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/netlify/gotell 2 | 3 | go 1.13 4 | 5 | require ( 6 | github.com/PuerkitoBio/goquery v1.0.2-0.20160831162211-152b1a2c8f5d 7 | github.com/Sirupsen/logrus v0.10.1-0.20160829202321-3ec0642a7fb6 8 | github.com/andybalholm/cascadia v0.0.0-20160831153056-1c31af6f6c1a // indirect 9 | github.com/davecgh/go-spew v1.1.1 // indirect 10 | github.com/dgrijalva/jwt-go v3.0.0+incompatible 11 | github.com/dimfeld/httppath v0.0.0-20140515164737-c8e499c3ef3c // indirect 12 | github.com/dimfeld/httptreemux v3.3.0+incompatible // indirect 13 | github.com/fsnotify/fsnotify v1.3.2-0.20160816051541-f12c6236fe7b // indirect 14 | github.com/golang/protobuf v0.0.0-20160829194233-1f49d83d9aa0 // indirect 15 | github.com/google/go-github v8.0.1-0.20170725155112-35d38108ba83+incompatible 16 | github.com/google/go-querystring v0.0.0-20160401233042-9235644dd9e5 // indirect 17 | github.com/guregu/kami v2.2.1+incompatible 18 | github.com/hashicorp/hcl v0.0.0-20160902165219-99df0eb941dd // indirect 19 | github.com/inconshreveable/mousetrap v1.0.0 // indirect 20 | github.com/kr/fs v0.0.0-20131111012553-2788f0dbd169 // indirect 21 | github.com/kr/pretty v0.1.0 // indirect 22 | github.com/magiconair/properties v1.7.1-0.20160908093658-0723e352fa35 // indirect 23 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee // indirect 24 | github.com/pelletier/go-buffruneio v0.1.0 // indirect 25 | github.com/pelletier/go-toml v0.3.6-0.20160906202557-31055c2ff0bb // indirect 26 | github.com/pkg/errors v0.7.1 27 | github.com/pkg/sftp v0.0.0-20160908100035-8197a2e58073 // indirect 28 | github.com/rs/cors v0.0.0-20160617231935-a62a804a8a00 29 | github.com/rs/xhandler v0.0.0-20160618193221-ed27b6fd6521 // indirect 30 | github.com/spf13/afero v0.0.0-20160821083612-20500e2abd0d // indirect 31 | github.com/spf13/cast v0.0.0-20160730092037-e31f36ffc91a // indirect 32 | github.com/spf13/cobra v0.0.0-20160830174925-9c28e4bbd74e 33 | github.com/spf13/jwalterweatherman v0.0.0-20160311093646-33c24e77fb80 // indirect 34 | github.com/spf13/pflag v0.0.0-20160915153101-c7e63cf4530b // indirect 35 | github.com/spf13/viper v0.0.0-20160830143246-16990631d4aa 36 | github.com/stretchr/testify v1.4.0 // indirect 37 | github.com/zenazn/goji v0.9.1-0.20160823030549-4d7077956293 38 | golang.org/x/crypto v0.0.0-20160915071417-81372b2fc2f1 // indirect 39 | golang.org/x/net v0.0.0-20160914084325-de35ec43e7a9 // indirect 40 | golang.org/x/oauth2 v0.0.0-20160902055913-3c3a985cb79f 41 | golang.org/x/sys v0.0.0-20160907145946-30de6d19a3bd // indirect 42 | golang.org/x/text v0.0.0-20160915075942-04b8648d973c // indirect 43 | google.golang.org/appengine v0.0.0-20160914034556-78199dcb0669 // indirect 44 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect 45 | ) 46 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/PuerkitoBio/goquery v1.0.2-0.20160831162211-152b1a2c8f5d h1:vVDdsHBraJtpA60U3WSdVXQcCE8s5phzQv7sOkGggjk= 2 | github.com/PuerkitoBio/goquery v1.0.2-0.20160831162211-152b1a2c8f5d/go.mod h1:T9ezsOHcCrDCgA8aF1Cqr3sSYbO/xgdy8/R/XiIMAhA= 3 | github.com/Sirupsen/logrus v0.10.1-0.20160829202321-3ec0642a7fb6 h1:8UA7ycG8qUIDb/pPY863f2BKlm8JZQhttl48uruw8bE= 4 | github.com/Sirupsen/logrus v0.10.1-0.20160829202321-3ec0642a7fb6/go.mod h1:rmk17hk6i8ZSAJkSDa7nOxamrG+SP4P0mm+DAvExv4U= 5 | github.com/andybalholm/cascadia v0.0.0-20160831153056-1c31af6f6c1a h1:9snH35uH1IOcdF3Xde3KuD6lUVNIbZZuv6fMycyRpD4= 6 | github.com/andybalholm/cascadia v0.0.0-20160831153056-1c31af6f6c1a/go.mod h1:3I+3V7B6gTBYfdpYgIG2ymALS9H+5VDKUl3lHH7ToM4= 7 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 8 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 9 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 10 | github.com/dgrijalva/jwt-go v3.0.0+incompatible h1:nfVqwkkhaRUethVJaQf5TUFdFr3YUF4lJBTf/F2XwVI= 11 | github.com/dgrijalva/jwt-go v3.0.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= 12 | github.com/dimfeld/httppath v0.0.0-20140515164737-c8e499c3ef3c h1:ZUydtQ3zIrvM9JaBsb4vcVgueTwEDFLfo3pm7MWWH7k= 13 | github.com/dimfeld/httppath v0.0.0-20140515164737-c8e499c3ef3c/go.mod h1:0FpDmbrt36utu8jEmeU05dPC9AB5tsLYVVi+ZHfyuwI= 14 | github.com/dimfeld/httptreemux v3.3.0+incompatible h1:pSXFqMP/msQgAby8WScbShWIr0e2RLu8lq/GJtuzAbc= 15 | github.com/dimfeld/httptreemux v3.3.0+incompatible/go.mod h1:rbUlSV+CCpv/SuqUTP/8Bk2O3LyUV436/yaRGkhP6Z0= 16 | github.com/fsnotify/fsnotify v1.3.2-0.20160816051541-f12c6236fe7b h1:clQtr7BsnoijdumdhlbbOGglPb1lIAJ3yTPjYOHlKdQ= 17 | github.com/fsnotify/fsnotify v1.3.2-0.20160816051541-f12c6236fe7b/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= 18 | github.com/golang/protobuf v0.0.0-20160829194233-1f49d83d9aa0 h1:80TTswsNQ80smEffZo/oPOnqP1EA59ZP8C2GGfcL7Us= 19 | github.com/golang/protobuf v0.0.0-20160829194233-1f49d83d9aa0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= 20 | github.com/google/go-github v8.0.1-0.20170725155112-35d38108ba83+incompatible h1:DnE4WhPszV+a6FHJmF8wIGTSMlSzeCR0XfBpkAxEEN0= 21 | github.com/google/go-github v8.0.1-0.20170725155112-35d38108ba83+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ= 22 | github.com/google/go-querystring v0.0.0-20160401233042-9235644dd9e5 h1:oERTZ1buOUYlpmKaqlO5fYmz8cZ1rYu5DieJzF4ZVmU= 23 | github.com/google/go-querystring v0.0.0-20160401233042-9235644dd9e5/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= 24 | github.com/guregu/kami v2.2.1+incompatible h1:G8vzA3Bx2jnm+AQOHXtHW0vTSQ7tQqfxLc5nuHFtkgI= 25 | github.com/guregu/kami v2.2.1+incompatible/go.mod h1:EWmehSBHxCbnLPj3XytBR5sht/UebNLwwFYpGcfbptQ= 26 | github.com/hashicorp/hcl v0.0.0-20160902165219-99df0eb941dd h1:84QdurP28/GTPuAXatiouvNyhN9dflZY9KtMbatmupI= 27 | github.com/hashicorp/hcl v0.0.0-20160902165219-99df0eb941dd/go.mod h1:oZtUIOe8dh44I2q6ScRibXws4Ajl+d+nod3AaR9vL5w= 28 | github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= 29 | github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= 30 | github.com/kr/fs v0.0.0-20131111012553-2788f0dbd169 h1:YUrU1/jxRqnt0PSrKj1Uj/wEjk/fjnE80QFfi2Zlj7Q= 31 | github.com/kr/fs v0.0.0-20131111012553-2788f0dbd169/go.mod h1:glhvuHOU9Hy7/8PwwdtnarXqLagOX0b/TbZx2zLMqEg= 32 | github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= 33 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= 34 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 35 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 36 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 37 | github.com/magiconair/properties v1.7.1-0.20160908093658-0723e352fa35 h1:WtkHGe1cgg+lvDj9p5CvjXrfopsIss0vIAz+/zeYZyQ= 38 | github.com/magiconair/properties v1.7.1-0.20160908093658-0723e352fa35/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= 39 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee h1:kK7VuFVykgt0LfMSloWYjDOt4TnOcL0AxF0/rDq2VkM= 40 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 41 | github.com/pelletier/go-buffruneio v0.1.0 h1:ig6N9Cg71k/P+UUbhwdOFtJWz+qa8/3by7AzMprMWBM= 42 | github.com/pelletier/go-buffruneio v0.1.0/go.mod h1:JkE26KsDizTr40EUHkXVtNPvgGtbSNq5BcowyYOWdKo= 43 | github.com/pelletier/go-toml v0.3.6-0.20160906202557-31055c2ff0bb h1:itpNy1S158xYBrFLYeyGyGjlds546+4bSNENlR5kcU4= 44 | github.com/pelletier/go-toml v0.3.6-0.20160906202557-31055c2ff0bb/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= 45 | github.com/pkg/errors v0.7.1 h1:0XSZhzhcAUrs2vsv1y5jaxWejlCCgvxI/kBpbRFMZ+o= 46 | github.com/pkg/errors v0.7.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 47 | github.com/pkg/sftp v0.0.0-20160908100035-8197a2e58073 h1:9PqYQCzKEbilrPJl3LDO16HdbA25Yqc3I25aUfgFaCs= 48 | github.com/pkg/sftp v0.0.0-20160908100035-8197a2e58073/go.mod h1:NxmoDg/QLVWluQDUYG7XBZTLUpKeFa8e3aMf1BfjyHk= 49 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 50 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 51 | github.com/rs/cors v0.0.0-20160617231935-a62a804a8a00 h1:8DPul/X0IT/1TNMIxoKLwdemEOBBHDC/K4EB16Cw5WE= 52 | github.com/rs/cors v0.0.0-20160617231935-a62a804a8a00/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU= 53 | github.com/rs/xhandler v0.0.0-20160618193221-ed27b6fd6521 h1:3hxavr+IHMsQBrYUPQM5v0CgENFktkkbg1sfpgM3h20= 54 | github.com/rs/xhandler v0.0.0-20160618193221-ed27b6fd6521/go.mod h1:RvLn4FgxWubrpZHtQLnOf6EwhN2hEMusxZOhcW9H3UQ= 55 | github.com/spf13/afero v0.0.0-20160821083612-20500e2abd0d h1:sd1qRX4NJPdIHizg+E7cK0kFaJRjwfJFQyUJUguwIak= 56 | github.com/spf13/afero v0.0.0-20160821083612-20500e2abd0d/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= 57 | github.com/spf13/cast v0.0.0-20160730092037-e31f36ffc91a h1:tPI5RnYZJhcXj0LhJ9pi7PS7gqOhuFR+4HEKyDz3BnQ= 58 | github.com/spf13/cast v0.0.0-20160730092037-e31f36ffc91a/go.mod h1:r2rcYCSwa1IExKTDiTfzaxqT2FNHs8hODu4LnUfgKEg= 59 | github.com/spf13/cobra v0.0.0-20160830174925-9c28e4bbd74e h1:YdP6GKJS0Ls++kXc85WCCX2ArKToqixBwpBrWP/5J/k= 60 | github.com/spf13/cobra v0.0.0-20160830174925-9c28e4bbd74e/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= 61 | github.com/spf13/jwalterweatherman v0.0.0-20160311093646-33c24e77fb80 h1:evyGXhHMrxKBDkdlSPv9HMWV2o53o+Ibhm28BGc0450= 62 | github.com/spf13/jwalterweatherman v0.0.0-20160311093646-33c24e77fb80/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= 63 | github.com/spf13/pflag v0.0.0-20160915153101-c7e63cf4530b h1:wT0f1lvMzot+G0vEQQqBBJIHEj5l+fVx72f7BC9xU14= 64 | github.com/spf13/pflag v0.0.0-20160915153101-c7e63cf4530b/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= 65 | github.com/spf13/viper v0.0.0-20160830143246-16990631d4aa h1:+yZESdfzohzq0Jtw+Cqbc8DzjoodNv3CB+BOww04drs= 66 | github.com/spf13/viper v0.0.0-20160830143246-16990631d4aa/go.mod h1:A8kyI5cUJhb8N+3pkfONlcEcZbueH6nhAm0Fq7SrnBM= 67 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 68 | github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= 69 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 70 | github.com/zenazn/goji v0.9.1-0.20160823030549-4d7077956293 h1:d4Z2pghRnUeh1i1mo+eirsIo1UB1MDzKs9d6w7rI//o= 71 | github.com/zenazn/goji v0.9.1-0.20160823030549-4d7077956293/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q= 72 | golang.org/x/crypto v0.0.0-20160915071417-81372b2fc2f1 h1:sNqTn8YefEwa0w/VlBwT2oC87+vpmr8LapsQZHdR4CM= 73 | golang.org/x/crypto v0.0.0-20160915071417-81372b2fc2f1/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 74 | golang.org/x/net v0.0.0-20160914084325-de35ec43e7a9 h1:kx/Lof15CmxipDbbp6a+NuQHIUBL4ujnHPdEoj6Z/L4= 75 | golang.org/x/net v0.0.0-20160914084325-de35ec43e7a9/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 76 | golang.org/x/oauth2 v0.0.0-20160902055913-3c3a985cb79f h1:VWt05OS3Al9w09GSPgltoHP90whAHlpik/Bys7HVEDE= 77 | golang.org/x/oauth2 v0.0.0-20160902055913-3c3a985cb79f/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= 78 | golang.org/x/sys v0.0.0-20160907145946-30de6d19a3bd h1:623bV31oLOV7UxfAiittr+7N1SmYu2sKBBQnm0dFJp4= 79 | golang.org/x/sys v0.0.0-20160907145946-30de6d19a3bd/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 80 | golang.org/x/text v0.0.0-20160915075942-04b8648d973c h1:MZZXXxZX9mtQf2bHJmbdLO8VhC0ABAptXeSFNPugg3M= 81 | golang.org/x/text v0.0.0-20160915075942-04b8648d973c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 82 | google.golang.org/appengine v0.0.0-20160914034556-78199dcb0669 h1:Gn/iNp0aEQHNkLa3fPIB1MaMpzxY0MopwQfO1mHVjBM= 83 | google.golang.org/appengine v0.0.0-20160914034556-78199dcb0669/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= 84 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 85 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= 86 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 87 | gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= 88 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 89 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "log" 5 | 6 | "github.com/netlify/gotell/cmd" 7 | ) 8 | 9 | func main() { 10 | if err := cmd.RootCommand().Execute(); err != nil { 11 | log.Fatal(err) 12 | } 13 | } 14 | --------------------------------------------------------------------------------