Golang File Upload

├── .github ├── ISSUE_TEMPLATE.md ├── PULL_REQUEST_TEMPLATE.md └── workflows │ └── go.yml ├── .gitignore ├── .travis.yml ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── TODO.md ├── action.go ├── cmd └── fsagent │ ├── fs-example-config.json │ ├── main.go │ ├── web-example-config.json │ └── web │ └── index.html ├── config.go ├── config.json ├── modules ├── file.go ├── file.json ├── http.go ├── http.json ├── mail.go ├── mail.json ├── sleep.go └── sleep.json ├── package.json ├── program.go ├── util.go └── web.go /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | Thank you for creating the issue! 2 | 3 | Please include the following information: 4 | 1. Version of fsagent (git commit hash) 5 | 2. Go environment: `go version && go env` 6 | 3. As many Logs from the application as possible (of course be sure to remove all confidential informations like passwords) -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | Thank you for creating this pull request! -------------------------------------------------------------------------------- /.github/workflows/go.yml: -------------------------------------------------------------------------------- 1 | name: Go 2 | on: [push] 3 | jobs: 4 | 5 | build: 6 | name: Build 7 | runs-on: ubuntu-latest 8 | steps: 9 | 10 | - name: Set up Go 1.13 11 | uses: actions/setup-go@v1 12 | with: 13 | go-version: 1.13 14 | id: go 15 | 16 | - name: Check out code into the Go module directory 17 | uses: actions/checkout@v1 18 | 19 | - name: Get dependencies 20 | run: | 21 | go get -v -t -d ./... 22 | if [ -f Gopkg.toml ]; then 23 | curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh 24 | dep ensure 25 | fi 26 | 27 | - name: Build 28 | run: go build -v . 29 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | cmd/fsagent/fsagent 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: go 2 | 3 | go: 4 | - 1.13.x 5 | - tip 6 | 7 | script: 8 | - go get simonwaldherr.de/go/fsagent/cmd/fsagent 9 | - go fmt simonwaldherr.de/go/fsagent/cmd/fsagent 10 | - go build simonwaldherr.de/go/fsagent/cmd/fsagent 11 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | ## How do I... 4 | 5 | * [Use This Guide](#introduction)? 6 | * Ask or Say Something? 🤔🐛😱 7 | * [Request Support](#request-support) 8 | * [Report an Error or Bug](#report-an-error-or-bug) 9 | * [Request a Feature](#request-a-feature) 10 | * Make Something? 🤓👩🏽‍💻📜🍳 11 | * [Project Setup](#project-setup) 12 | * [Contribute Documentation](#contribute-documentation) 13 | * [Contribute Code](#contribute-code) 14 | * Manage Something ✅🙆🏼💃👔 15 | * [Provide Support on Issues](#provide-support-on-issues) 16 | * [Label Issues](#label-issues) 17 | * [Clean Up Issues and PRs](#clean-up-issues-and-prs) 18 | * [Review Pull Requests](#review-pull-requests) 19 | * [Merge Pull Requests](#merge-pull-requests) 20 | * [Tag a Release](#tag-a-release) 21 | * [Join the Project Team](#join-the-project-team) 22 | * Add a Guide Like This One [To My Project](#attribution)? 🤖😻👻 23 | 24 | ## Introduction 25 | 26 | Thank you so much for your interest in contributing!. All types of contributions are encouraged and valued. See the [table of contents](#toc) for different ways to help and details about how this project handles them!📝 27 | 28 | ## Request Support 29 | 30 | If you have a question about this project, how to use it, or just need clarification about something: 31 | 32 | * Open an Issue at https://github.com/SimonWaldherr/fsagent/issues 33 | * Provide as much context as you can about what you're running into. 34 | * Provide as many Logs from the application as possible (of course be sure to remove all confidential informations like passwords) 35 | * Provide project and platform versions (OS, Golang, ... etc), depending on what seems relevant. If not, please be ready to provide that information if maintainers ask for it. 36 | * Be patient - this is only a hobby project, I work more then 10 hours in my day job - after work I help in a volunteer fire department as deputy commander. 37 | * If your problem is resolved, be nice and helpful and inform us about the solution. This helps others who have possibly the same or similar problems. 38 | 39 | ## Report an Error or Bug 40 | 41 | If you run into an error or bug with the project: 42 | 43 | * Open an Issue at https://github.com/SimonWaldherr/fsagent/issues 44 | * Include *reproduction steps* that someone else can follow to recreate the bug or error on their own. 45 | * Provide as many Logs from the application as possible (of course be sure to remove all confidential informations like passwords) 46 | * Provide project and platform versions (OS, Golang, ... etc), depending on what seems relevant. If not, please be ready to provide that information if maintainers ask for it. 47 | 48 | ## Request a Feature 49 | 50 | If the project doesn't do something you need or want it to do: 51 | 52 | * Open an Issue at https://github.com/SimonWaldherr/fsagent/issues 53 | * Provide as much context as you can about what you're running into. 54 | * Please try and be clear about why existing features and alternatives would not work for you. 55 | 56 | Note: It is unlikely to be able to accept every single feature request that could filed. Please understand if I need to say no. 57 | 58 | ## Project Setup 59 | 60 | So you wanna contribute some code! That's great! This project uses GitHub Pull Requests to manage contributions, so [read up on how to fork a GitHub project and file a PR](https://guides.github.com/activities/forking) if you've never done it before. 61 | 62 | If this seems like a lot or you aren't able to do all this setup, you might also be able to [edit the files directly](https://help.github.com/articles/editing-files-in-another-user-s-repository/) without having to do any of this setup. Yes, [even code](#contribute-code). 63 | 64 | If you want to go the usual route and run the project locally, though: 65 | 66 | * [Install Go](https://golang.org/doc/install) 67 | * [Fork the project](https://guides.github.com/activities/forking/#fork) 68 | 69 | Then in your terminal: 70 | ```sh 71 | go get simonwaldherr.de/go/fsagent 72 | ``` 73 | 74 | And you should be ready to go! 75 | 76 | ## Contribute Documentation 77 | 78 | Documentation is a super important part of this project and the whole Free and Open Source Community. Docs are how we keep track of what we're doing, how, and (most of all) why. It's how we stay on the same page. And it's how we tell others everything they need in order to be able to use this project -- or contribute to it. So thank you in advance. 79 | 80 | Documentation contributions of any size are welcome! Feel free to file a PR even if you're just rewording a sentence to be more clear, or fixing a spelling mistake! 81 | 82 | To contribute documentation: 83 | 84 | * [Set up the project](#project-setup). 85 | * Edit or add any relevant documentation. 86 | * Make sure your changes are formatted correctly and consistently with the rest of the documentation. 87 | * Re-read what you wrote, and run a spellchecker on it to make sure you didn't miss anything. 88 | * In your commit message(s), begin the first line with `docs: `. For example: `docs: Adding a doc contrib section to CONTRIBUTING.md`. 89 | * Write clear, concise commit message(s) using [conventional-changelog format](https://github.com/conventional-changelog/conventional-changelog-angular/blob/master/convention.md). Documentation commits should use `docs(): `. 90 | * Go to https://github.com/SimonWaldherr/fsagent/pulls and open a new pull request with your changes. 91 | * If your PR is connected to an open issue, add a line in your PR's description that says `Fixes: #123`, where `#123` is the number of the issue you're fixing. 92 | 93 | Once you've filed the PR: 94 | 95 | * One or more maintainers will use GitHub's review feature to review your PR. 96 | * If the maintainer asks for any changes, edit your changes, push, and ask for another review. 97 | * If the maintainer decides to pass on your PR, they will thank you for the contribution and explain why they won't be accepting the changes. That's ok! We still really appreciate you taking the time to do it, and we don't take that lightly. 💚 98 | * If your PR gets accepted, it will be marked as such, and merged into the `latest` branch soon after. Your contribution will be distributed to the masses next time the maintainers [tag a release](#tag-a-release) 99 | 100 | ## Contribute Code 101 | 102 | We like code commits a lot! They're super handy, and they keep the project going and doing the work it needs to do to be useful to others. 103 | 104 | Code contributions of just about any size are welcome! 105 | 106 | The main difference between code contributions and documentation contributions is that contributing code can and should contain [relevant tests](https://en.wikipedia.org/wiki/Continuous_testing) or at least informations on how to test, for the code being added or changed. 107 | 108 | To contribute code: 109 | 110 | * [Set up the project](#project-setup). 111 | * Make any necessary changes to the source code. 112 | * Include any [additional documentation](#contribute-documentation) the changes might need. 113 | * Write tests that verify that your contribution works as expected. 114 | * Write clear, concise commit message(s) using [conventional-changelog format](https://github.com/conventional-changelog/conventional-changelog-angular/blob/master/convention.md). 115 | * Dependency updates, additions, or removals must be in individual commits, and the message must use the format: `(deps): PKG@VERSION`, where `` is any of the usual `conventional-changelog` prefixes, at your discretion. 116 | * Go to https://github.com/SimonWaldherr/fsagent/pulls and open a new pull request with your changes. 117 | * If your PR is connected to an open issue, add a line in your PR's description that says `Fixes: #123`, where `#123` is the number of the issue you're fixing. 118 | 119 | ## Provide Support on Issues 120 | 121 | Helping out other users with their questions is a really awesome way of contributing to any community. It's not uncommon for most of the issues on an open source projects being support-related questions by users trying to understand something they ran into, or find their way around a known bug. 122 | 123 | Sometimes, the `support` label will be added to things that turn out to actually be other things, like bugs or feature requests. In that case, suss out the details with the person who filed the original issue, add a comment explaining what the bug is, and change the label from `support` to `bug` or `feature`. If you can't do this yourself, @mention a maintainer so they can do it. 124 | 125 | In order to help other folks out with their questions: 126 | 127 | * Go to the issue tracker and [filter open issues by the `support` label](https://github.com/SimonWaldherr/fsagent/issues?q=is%3Aopen+is%3Aissue+label%3Asupport). 128 | * Read through the list until you find something that you're familiar enough with to give an answer to. 129 | * Respond to the issue with whatever details are needed to clarify the question, or get more details about what's going on. 130 | * Once the discussion wraps up and things are clarified, either close the issue, or ask the original issue filer (or a maintainer) to close it for you. 131 | 132 | Some notes on picking up support issues: 133 | 134 | * Avoid responding to issues you don't know you can answer accurately. 135 | * As much as possible, try to refer to past issues with accepted answers. Link to them from your replies with the `#123` format. 136 | * Be kind and patient with users -- often, folks who have run into confusing things might be upset or impatient. This is ok. Try to understand where they're coming from, and if you're too uncomfortable with the tone, feel free to stay away or withdraw from the issue. (note: if the user is outright hostile or is violating the CoC, [refer to the Code of Conduct](CODE_OF_CONDUCT.md) to resolve the conflict). 137 | 138 | ## Label Issues 139 | 140 | One of the most important tasks in handling issues is labeling them usefully and accurately. All other tasks involving issues ultimately rely on the issue being classified in such a way that relevant parties looking to do their own tasks can find them quickly and easily. 141 | 142 | In order to label issues, [open up the list of unlabeled issues](https://github.com/SimonWaldherr/fsagent/issues?q=is%3Aopen+is%3Aissue+no%3Alabel) and, **from newest to oldest**, read through each one and apply issue labels according to the table below. If you're unsure about what label to apply, skip the issue and try the next one: don't feel obligated to label each and every issue yourself! 143 | 144 | Label | Apply When | Notes 145 | --- | --- | --- 146 | `bug` | Cases where the code (or documentation) is behaving in a way it wasn't intended to. | If something is happening that surprises the *user* but does not go against the way the code is designed, it should use the `enhancement` label. 147 | `critical` | Added to `bug` issues if the problem described makes the code completely unusable in a common situation. | 148 | `documentation` | Added to issues or pull requests that affect any of the documentation for the project. | Can be combined with other labels, such as `bug` or `enhancement`. 149 | `duplicate` | Added to issues or PRs that refer to the exact same issue as another one that's been previously labeled. | Duplicate issues should be marked and closed right away, with a message referencing the issue it's a duplicate of (with `#123`) 150 | `enhancement` | Added to [feature requests](#request-a-feature), PRs, or documentation issues that are purely additive: the code or docs currently work as expected, but a change is being requested or suggested. | 151 | `help wanted` | Applied by [Committers](#join-the-project-team) to issues and PRs that they would like to get outside help for. Generally, this means it's lower priority for the maintainer team to itself implement, but that the community is encouraged to pick up if they so desire | Never applied on first-pass labeling. 152 | `in-progress` | Applied by [Committers](#join-the-project-team) to PRs that are pending some work before they're ready for review. | The original PR submitter should @mention the team member that applied the label once the PR is complete. 153 | `performance` | This issue or PR is directly related to improving performance. | 154 | `refactor` | Added to issues or PRs that deal with cleaning up or modifying the project for the betterment of it. | 155 | `starter` | Applied by [Committers](#join-the-project-team) to issues that they consider good introductions to the project for people who have not contributed before. These are not necessarily "easy", but rather focused around how much context is necessary in order to understand what needs to be done for this project in particular. | Existing project members are expected to stay away from these unless they increase in priority. 156 | `support` | This issue is either asking a question about how to use the project, clarifying the reason for unexpected behavior, or possibly reporting a `bug` but does not have enough detail yet to determine whether it would count as such. | The label should be switched to `bug` if reliable reproduction steps are provided. Issues primarily with unintended configurations of a user's environment are not considered bugs, even if they cause crashes. 157 | `tests` | This issue or PR either requests or adds primarily tests to the project. | If a PR is pending tests, that will be handled through the [PR review process](#review-pull-requests) 158 | `wontfix` | Labelers may apply this label to issues that clearly have nothing at all to do with the project or are otherwise entirely outside of its scope/sphere of influence. [Committers](#join-the-project-team) may apply this label and close an issue or PR if they decide to pass on an otherwise relevant issue. | The issue or PR should be closed as soon as the label is applied, and a clear explanation provided of why the label was used. Contributors are free to contest the labeling, but the decision ultimately falls on committers as to whether to accept something or not. 159 | 160 | ## Review Pull Requests 161 | 162 | While anyone can comment on a PR, add feedback, etc, PRs are only *approved* by team members with Issue Tracker or higher permissions. 163 | 164 | PR reviews use [GitHub's own review feature](https://help.github.com/articles/about-pull-request-reviews/), which manages comments, approval, and review iteration. 165 | 166 | Some notes: 167 | 168 | * You may ask for minor changes ("nitpicks"), but consider whether they are really blockers to merging: try to err on the side of "approve, with comments". 169 | * *ALL PULL REQUESTS* should be covered by a test: either by a previously-failing test, an existing test that covers the entire functionality of the submitted code, or new tests to verify any new/changed behavior. All tests must also pass and follow established conventions. Test coverage should not drop, unless the specific case is considered reasonable by maintainers. 170 | * Please make sure you're familiar with the code or documentation being updated, unless it's a minor change (spellchecking, minor formatting, etc). You may @mention another project member who you think is better suited for the review, but still provide a non-approving review of your own. 171 | * Be extra kind: people who submit code/doc contributions are putting themselves in a pretty vulnerable position, and have put time and care into what they've done (even if that's not obvious to you!) -- always respond with respect, be understanding, but don't feel like you need to sacrifice your standards for their sake, either. Just don't be a jerk about it? 172 | 173 | ## Attribution 174 | 175 | This guide was generated using the [WeAllJS](https://wealljs.org) `CONTRIBUTING.md` generator. 176 | If you need a `CONTRIBUTING.md` file for your Repo, use this document as inspiration or use the [weallcontribute](https://github.com/WeAllJS/weallcontribute)-cli-tool and adapt it to your needs. 177 | 178 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2022 Simon Waldherr 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # fsagent 2 | 3 | fsagent is a Golang Application to perform various standard actions triggered by various events. FSAgent is highly customizable. 4 | 5 | ## Name 6 | 7 | The name FSAgent was originally the shorthand for File System Agent. 8 | With the support of additional triggers and data sources outside of file system events in the monitored folders, a new meaning for the letters F and S must be found. 9 | Currently, I prefer the definition Free Service Agent. 10 | 11 | ## Why 12 | 13 | **WARNING:** *sad true reality* 14 | 15 | When planning and developing new (open source) applications / systems, you usually use the latest technologies (containerization (e.g. Docker), message queues (e.g. RabbitMQ, ZeroMQ, ActiveMQ, ...), databases (e.g. PostgreSQL, MariaDB, Redis, MongoDB, ...), ...), but if you're working for a non-startup Company, you often have to deal with old legacy enterprise applications. 16 | These applications do not have modern interfaces, many are decades old. The most modern communication channels of these applications are mostly FTP uploads and emails. 17 | I really mean FTP, not those new and fancy SFTP Servers. 18 | However, many of these applications do not even have the functionality to upload, but can only store files in directories and need other applications such as [Bat](https://en.wikipedia.org/wiki/The_Bat!), [Blat](http://www.blat.net) or [Outlook](https://en.wikipedia.org/wiki/Microsoft_Outlook) to upload. 19 | Bat or Blat is not fundamentally bad, but if your whole business depends on software like Blat, you have a big problem. 20 | In my free time, I have written this program for replacing such "interfaces". It is not just a replacement for Bat or Outlook, it monitors directories and executes predefined actions for new files. 21 | The configuration is kept as simple as possible (and will become even easier) to be done by anyone in IT departments, not just programmers. 22 | The first goal was the elimination of the biggest pain in the ass. 23 | Gradually, however, it is also planned to extend fsagent for the service composition/orchestration of other protocols such as [HTTP(S)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), [AMQP](https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol), [WebSub (PubSubHubbub)](https://en.wikipedia.org/wiki/WebSub). 24 | 25 | ## Install 26 | 27 | fsagent can easily installed by the ```go get```-command: 28 | 29 | ```go get simonwaldherr.de/go/fsagent``` 30 | 31 | ## Config 32 | 33 | fsagent can do many things, these can be defined and configured with json files. 34 | 35 | start the fsagent daemon with ```go run fsagent.go config.json``` or compile a binary (```go build```) and run it with ```./fsagent config.json```. 36 | 37 | the **config.json** file could look like: 38 | ```json 39 | [ 40 | { 41 | "verbose": true, 42 | "debounce": true, 43 | "folder": "/mnt/prod/Server/Transfer/701/%Y/%m/%d/", 44 | "trigger": "fsevent", 45 | "match": "^[0-9]+\\.[Tt][Xx][Tt]$", 46 | "action": [ 47 | { 48 | "do": "sleep", 49 | "config": { 50 | "time": 500 51 | }, 52 | "onSuccess": [ 53 | { 54 | "do": "mail", 55 | "config": { 56 | "name": "mail", 57 | "subject": "Lorem Ipsum", 58 | "body": "dolor sit amet", 59 | "from": "notification@company.tld", 60 | "to": ["example@domain.tld"], 61 | "cc": ["example2@domain.tld"], 62 | "bcc": ["example3@domain.tld"], 63 | "user": "notification", 64 | "pass": "spring2018", 65 | "server": "webmail.domain.tld", 66 | "port": 587 67 | }, 68 | "onSuccess": [ 69 | { 70 | "do": "move", 71 | "config": { 72 | "name": "success/$file_%Y%m%d%H%M%S" 73 | } 74 | } 75 | ], 76 | "onFailure": [ 77 | { 78 | "do": "move", 79 | "config": { 80 | "name": "error/$file_%Y%m%d%H%M%S" 81 | } 82 | } 83 | ] 84 | } 85 | ] 86 | } 87 | ] 88 | } 89 | ] 90 | ``` 91 | 92 | ### Trigger 93 | 94 | Currently there are two triggers available, the most important trigger is filesystem event trigger based on [fsnotify](github.com/fsnotify/fsnotify). 95 | If this is not possible (e.g. if you work on mounted drives and the fs event comes from a different system) you can use a ticker as trigger. 96 | 97 | Trigger | Info 98 | --------|------ 99 | fsevent | file system event based on [fsnotify](github.com/fsnotify/fsnotify) 100 | ticker | checks for new files at a customizable frequency 101 | http | files can be uploaded via a web form 102 | 103 | ### Actions 104 | 105 | There are some ready-made actions, but you can easily create others yourself. 106 | 107 | Action | Info 108 | -----------|------ 109 | Copy | creates a copy of a given file at the specified destination 110 | Delete | removes a file 111 | Move | moves a file to a new location 112 | Decompress | decompresses a file 113 | Compress | compresses a file 114 | HttpPostR. | sends the content of a file in a HTTP Post Request Body 115 | SendMail | sends the file as mail attachment 116 | Sleep | waits for a specified duration 117 | 118 | ## Todo / Contribute 119 | 120 | Informations about the [license](https://github.com/SimonWaldherr/fsagent/blob/master/LICENSE), [how to contribute](https://github.com/SimonWaldherr/fsagent/blob/master/CONTRIBUTING.md) and a [list of improvements to do](https://github.com/SimonWaldherr/fsagent/blob/master/TODO.md) are in separate files. 121 | -------------------------------------------------------------------------------- /TODO.md: -------------------------------------------------------------------------------- 1 | # Todo 2 | 3 | these features will come sometime 4 | 5 | * [ ] support for [YAML](https://en.wikipedia.org/wiki/YAML) 6 | * [ ] more verbose logging 7 | * [ ] search/extract strings from files 8 | * [ ] file-checks (e.g. on file size) 9 | * [ ] more comments in source code 10 | * [ ] database inserts (csv to db) 11 | * [ ] status website 12 | * [ ] add config file editor 13 | * [ ] use [Blockly](https://developers.google.com/blockly/) as graphical editor 14 | * [ ] add (S)FTP(S)-Support 15 | * [ ] as Action 16 | * [ ] add [AMQP](https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol)-Support 17 | * [ ] as Trigger 18 | * [ ] as Action 19 | * [ ] add [WebSub](https://en.wikipedia.org/wiki/WebSub)-Support 20 | * [ ] as Trigger 21 | * [ ] as Action 22 | -------------------------------------------------------------------------------- /action.go: -------------------------------------------------------------------------------- 1 | package fsagent 2 | 3 | import ( 4 | "encoding/json" 5 | "log" 6 | "time" 7 | 8 | "github.com/SimonWaldherr/golibs/cachedfile" 9 | gfile "github.com/SimonWaldherr/golibs/file" 10 | 11 | "simonwaldherr.de/go/fsagent/modules" 12 | ) 13 | 14 | func init() { 15 | cachedfile.Init(15*time.Minute, 1*time.Minute) 16 | } 17 | 18 | // Action is something that should be performed. 19 | type Action []struct { 20 | Do string `json:"do"` 21 | Config json.RawMessage `json:"config"` 22 | Onsuccess Action `json:"onSuccess"` 23 | Onfailure Action `json:"onFailure"` 24 | } 25 | 26 | // Actionable defines a common set of methods each action should have. 27 | type Actionable interface { 28 | Name() string 29 | EmptyConfig() interface{} 30 | Perform(config interface{}, fileName string) error 31 | } 32 | 33 | // Actions is a list of Actionable types that can be added to. 34 | var Actions = []Actionable{ 35 | &modules.Mail{}, 36 | &modules.HTTP{}, 37 | &modules.Sleep{}, 38 | &modules.Delete{}, 39 | &modules.Move{}, 40 | &modules.Decompress{}, 41 | &modules.Compress{}, 42 | &modules.IsFile{}, 43 | &modules.CheckSize{}, 44 | } 45 | 46 | func do(act Action, file string) { 47 | for _, a := range act { 48 | var err error 49 | log.Printf("Do \"%v\" on file \"%v\"\n", a.Do, file) 50 | 51 | for _, action := range Actions { 52 | if a.Do == action.Name() { 53 | config := action.EmptyConfig() 54 | if gfile.IsFile(string(a.Config)) { 55 | str, _ := cachedfile.Read(string(a.Config)) 56 | json.Unmarshal([]byte(str), &config) 57 | } else { 58 | json.Unmarshal(a.Config, &config) 59 | } 60 | 61 | err = action.Perform(config, file) 62 | if err != nil { 63 | log.Printf("Error \"%v\" for file \"%v\"", err, file) 64 | } 65 | break 66 | } 67 | } 68 | 69 | if err == nil { 70 | do(a.Onsuccess, file) 71 | } else { 72 | log.Printf("Error \"%v\" for file \"%v\"", err, file) 73 | do(a.Onfailure, file) 74 | } 75 | } 76 | } 77 | -------------------------------------------------------------------------------- /cmd/fsagent/fs-example-config.json: -------------------------------------------------------------------------------- 1 | [{ 2 | "verbose": true, 3 | "debounce": true, 4 | "folder": "/mnt/prod/Server/Transfer/701/%Y/%m/%d/", 5 | "trigger": "ticker", 6 | "ticker": 2000, 7 | "onlynew": false, 8 | "match": "^[0-9]+\\.[Tt][Xx][Tt]$", 9 | "action": [{ 10 | "do": "checksize", 11 | "config": { 12 | "minSize": "1MB", 13 | "maxSize": "5MB" 14 | }, 15 | "onSuccess": [{ 16 | "do": "sleep", 17 | "config": { 18 | "time": 500 19 | }, 20 | "onSuccess": [{ 21 | "do": "mail", 22 | "config": { 23 | "name": "mail", 24 | "subject": "Lorem Ipsum", 25 | "body": "dolor sit amet", 26 | "from": "notification@company.tld", 27 | "to": ["example@domain.tld"], 28 | "cc": ["example2@domain.tld"], 29 | "bcc": ["example3@domain.tld"], 30 | "user": "notification", 31 | "pass": "spring2018", 32 | "server": "webmail.domain.tld", 33 | "port": 587 34 | }, 35 | "onSuccess": [{ 36 | "do": "move", 37 | "config": { 38 | "name": "/mnt/prod/Server/Archive/701/$file_%Y%m%d%H%M%S" 39 | } 40 | }], 41 | "onFailure": [{ 42 | "do": "move", 43 | "config": { 44 | "name": "/mnt/prod/Server/Error/701/$file_%Y%m%d%H%M%S" 45 | } 46 | }] 47 | }] 48 | }], 49 | "onFailure": [{ 50 | "do": "move", 51 | "config": { 52 | "name": "/mnt/prod/Server/Error/701/$file_%Y%m%d%H%M%S" 53 | } 54 | }] 55 | }] 56 | }] -------------------------------------------------------------------------------- /cmd/fsagent/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "os" 7 | "os/signal" 8 | 9 | "github.com/kardianos/service" 10 | "simonwaldherr.de/go/fsagent" 11 | ) 12 | 13 | var logger service.Logger 14 | 15 | func main() { 16 | svcConfig := &service.Config{ 17 | Name: "fsagent", 18 | DisplayName: "FileSystem Agent", 19 | Description: "this service can monitor a folder and do configurable things on filesystem triggers.", 20 | } 21 | 22 | prg := &fsagent.Program{} 23 | s, err := service.New(prg, svcConfig) 24 | if err != nil { 25 | log.Fatal(err) 26 | } 27 | logger, err = s.Logger(nil) 28 | if err != nil { 29 | log.Fatal(err) 30 | } 31 | 32 | c := make(chan os.Signal, 1) 33 | signal.Notify(c, os.Interrupt) 34 | go func() { 35 | for sig := range c { 36 | fmt.Printf("Signal: %v\n", sig) 37 | prg.Stop(s) 38 | } 39 | }() 40 | 41 | fmt.Println("run ...") 42 | err = s.Run() 43 | if err != nil { 44 | logger.Error(err) 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /cmd/fsagent/web-example-config.json: -------------------------------------------------------------------------------- 1 | [{ 2 | "verbose": true, 3 | "port": ":8080", 4 | "folder": "/upload", 5 | "trigger": "http", 6 | "match": "^*$", 7 | "action": [{ 8 | "do": "checksize", 9 | "config": { 10 | "minSize": "1MB", 11 | "maxSize": "5MB" 12 | }, 13 | "onSuccess": [{ 14 | "do": "move", 15 | "config": { 16 | "name": "./ok_$file_%Y%m%d%H%M%S" 17 | } 18 | }], 19 | "onFailure": [{ 20 | "do": "move", 21 | "config": { 22 | "name": "./error_$file_%Y%m%d%H%M%S" 23 | } 24 | }] 25 | }] 26 | }] -------------------------------------------------------------------------------- /cmd/fsagent/web/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Golang File Upload 6 | 7 | 8 | 13 | 14 | -------------------------------------------------------------------------------- /config.go: -------------------------------------------------------------------------------- 1 | package fsagent 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "log" 7 | "path/filepath" 8 | "sync" 9 | "time" 10 | 11 | "github.com/fsnotify/fsnotify" 12 | "simonwaldherr.de/go/golibs/cache" 13 | "simonwaldherr.de/go/golibs/regex" 14 | "simonwaldherr.de/go/golibs/xtime" 15 | ) 16 | 17 | // Config represents an element of the application configuration. 18 | type Config struct { 19 | Folder string `json:"folder"` 20 | Port string `json:"port"` 21 | Trigger string `json:"trigger"` 22 | Ticker int `json:"ticker"` 23 | Match string `json:"match"` 24 | Action Action `json:"action"` 25 | OnlyNew bool `json:"onlynew"` 26 | Verbose bool `json:"verbose"` 27 | Debounce bool `json:"debounce"` 28 | } 29 | 30 | func runConfig(wg *sync.WaitGroup, conf Config, i int, stop chan struct{}, watcher map[string]*fsnotify.Watcher) { 31 | var timer *time.Ticker 32 | var eventCache *cache.Cache 33 | var filenameChannel map[string]chan string 34 | filenameChannel = make(map[string]chan string) 35 | 36 | fmt.Println("start loop ...") 37 | 38 | if conf.Debounce { 39 | eventCache = cache.New(3*time.Second, 1*time.Second) 40 | } 41 | 42 | folderidx := fmt.Sprintf("%s:%v", conf.Folder, i) 43 | 44 | switch conf.Trigger { 45 | case "fsevent": 46 | watcher[folderidx], _ = fsnotify.NewWatcher() 47 | defer watcher[folderidx].Close() 48 | case "http": 49 | filenameChannel[folderidx] = make(chan string) 50 | go serveHTTP(conf, filenameChannel[folderidx]) 51 | case "ticker": 52 | timer = time.NewTicker(time.Millisecond * time.Duration(conf.Ticker)) 53 | } 54 | 55 | wg.Add(1) 56 | defer wg.Done() 57 | 58 | for { 59 | fmt.Println("waiting for events") 60 | switch conf.Trigger { 61 | case "fsevent": 62 | handleFsEvent(conf, eventCache, i, watcher) 63 | case "http": 64 | handleFile(conf, <-filenameChannel[folderidx]) 65 | case "ticker": 66 | handleTicker(conf, eventCache, i, timer) 67 | default: 68 | fmt.Println("no or incompatible trigger configured") 69 | return 70 | } 71 | 72 | select { 73 | case <-stop: 74 | return 75 | default: 76 | } 77 | } 78 | 79 | } 80 | 81 | func handleTicker(conf Config, eventCache *cache.Cache, i int, timer *time.Ticker) { 82 | select { 83 | case _ = <-timer.C: 84 | Folder := xtime.Fmt(conf.Folder, time.Now()) 85 | TickerFiles, _ := ioutil.ReadDir(Folder) 86 | 87 | for _, f := range TickerFiles { 88 | match, _ := regex.MatchString(f.Name(), conf.Match) 89 | if match { 90 | if conf.OnlyNew { 91 | lastmod, _ := FileLastModified(f.Name()) 92 | if lastmod.Unix() < time.Now().Add(time.Millisecond*time.Duration(conf.Ticker)*-1).Unix() { 93 | continue 94 | } 95 | } 96 | 97 | fmt.Println(f.Name()) 98 | cv := fmt.Sprintf("%v:%v:%v", Folder, i, f.Name()) 99 | if conf.Debounce && eventCache.Get(cv) == nil { 100 | eventCache.Set(cv, true) 101 | go do(conf.Action, Folder+f.Name()) 102 | } 103 | } 104 | } 105 | } 106 | 107 | } 108 | 109 | func handleFsEvent(conf Config, eventCache *cache.Cache, i int, watcher map[string]*fsnotify.Watcher) { 110 | select { 111 | case event := <-watcher[conf.Folder+fmt.Sprintf(":%v", i)].Events: 112 | fmt.Println("Event detected!") 113 | _, filename := filepath.Split(event.Name) 114 | match, _ := regex.MatchString(filename, conf.Match) 115 | if event.Op != fsnotify.Remove && match { 116 | cv := fmt.Sprintf("%v:%v:%v", conf.Folder, i, event.Name) 117 | if conf.Debounce && eventCache.Get(cv) == nil { 118 | eventCache.Set(cv, true) 119 | do(conf.Action, event.Name) 120 | } 121 | } 122 | case err := <-watcher[conf.Folder+fmt.Sprintf(":%v", i)].Errors: 123 | log.Println("error:", err) 124 | } 125 | } 126 | 127 | func handleFile(conf Config, fileName string) { 128 | fmt.Println("Event detected!") 129 | match, _ := regex.MatchString(fileName, conf.Match) 130 | if match { 131 | do(conf.Action, fileName) 132 | } 133 | } 134 | -------------------------------------------------------------------------------- /config.json: -------------------------------------------------------------------------------- 1 | [{ 2 | "verbose": true, 3 | "debounce": true, 4 | "folder": "/mnt/prod/Server/Transfer/701/%Y/%m/%d/", 5 | "trigger": "ticker", 6 | "ticker": 2000, 7 | "onlynew": false, 8 | "match": "^[0-9]+\\.[Tt][Xx][Tt]$", 9 | "action": [{ 10 | "do": "checksize", 11 | "config": { 12 | "minSize": "1MB", 13 | "maxSize": "5MB" 14 | }, 15 | "onSuccess": [{ 16 | "do": "sleep", 17 | "config": { 18 | "time": 500 19 | }, 20 | "onSuccess": [{ 21 | "do": "mail", 22 | "config": { 23 | "name": "mail", 24 | "subject": "Lorem Ipsum", 25 | "body": "dolor sit amet", 26 | "from": "notification@company.tld", 27 | "to": ["example@domain.tld"], 28 | "cc": ["example2@domain.tld"], 29 | "bcc": ["example3@domain.tld"], 30 | "user": "notification", 31 | "pass": "spring2018", 32 | "server": "webmail.domain.tld", 33 | "port": 587 34 | }, 35 | "onSuccess": [{ 36 | "do": "move", 37 | "config": { 38 | "name": "/mnt/prod/Server/Archive/701/$file_%Y%m%d%H%M%S" 39 | } 40 | }], 41 | "onFailure": [{ 42 | "do": "move", 43 | "config": { 44 | "name": "/mnt/prod/Server/Error/701/$file_%Y%m%d%H%M%S" 45 | } 46 | }] 47 | }] 48 | }], 49 | "onFailure": [{ 50 | "do": "move", 51 | "config": { 52 | "name": "/mnt/prod/Server/Error/701/$file_%Y%m%d%H%M%S" 53 | } 54 | }] 55 | }] 56 | }] -------------------------------------------------------------------------------- /modules/file.go: -------------------------------------------------------------------------------- 1 | package modules 2 | 3 | import ( 4 | "compress/flate" 5 | "fmt" 6 | "io" 7 | "os" 8 | "path/filepath" 9 | "strings" 10 | "time" 11 | 12 | "simonwaldherr.de/go/golibs/file" 13 | "simonwaldherr.de/go/golibs/xtime" 14 | 15 | "github.com/c2h5oh/datasize" 16 | ) 17 | 18 | type fileConfig struct { 19 | Name string `json:"name"` 20 | MinSize string `json:"minSize"` 21 | MaxSize string `json:"maxSize"` 22 | } 23 | 24 | type baseFile struct{} 25 | 26 | func (baseFile) EmptyConfig() interface{} { 27 | return &fileConfig{} 28 | } 29 | 30 | func formatName(c fileConfig, fileName string) string { 31 | _, filename := filepath.Split(fileName) 32 | str := xtime.Fmt(c.Name, time.Now()) 33 | str = strings.Replace(str, "$file", "%v", -1) 34 | str = fmt.Sprintf(str, filename) 35 | return str 36 | } 37 | 38 | func Copy(config interface{}, fileName string) error { 39 | c := config.(*fileConfig) 40 | 41 | str := formatName(*c, fileName) 42 | 43 | return file.Copy(fileName, str) 44 | } 45 | 46 | type Delete struct { 47 | baseFile 48 | } 49 | 50 | func (Delete) Name() string { 51 | return "delete" 52 | } 53 | 54 | func (Delete) Perform(_ interface{}, fileName string) error { 55 | return file.Delete(fileName) 56 | } 57 | 58 | type Move struct { 59 | baseFile 60 | } 61 | 62 | func (Move) Name() string { 63 | return "move" 64 | } 65 | 66 | func (Move) Perform(config interface{}, fileName string) error { 67 | c := config.(*fileConfig) 68 | 69 | _, filename := filepath.Split(fileName) 70 | str := xtime.Fmt(c.Name, time.Now()) 71 | str = strings.Replace(str, "$file", "%v", -1) 72 | str = fmt.Sprintf(str, filename) 73 | return file.Rename(fileName, str) 74 | } 75 | 76 | type Decompress struct { 77 | baseFile 78 | } 79 | 80 | func (Decompress) Name() string { 81 | return "decompress" 82 | } 83 | 84 | func (Decompress) Perform(config interface{}, fileName string) error { 85 | c := config.(*fileConfig) 86 | 87 | str := formatName(*c, fileName) 88 | 89 | i, err := os.Open(fileName) 90 | if err != nil { 91 | return err 92 | } 93 | defer i.Close() 94 | 95 | f := flate.NewReader(i) 96 | defer f.Close() 97 | 98 | o, err := os.Create(str) 99 | if err != nil { 100 | return err 101 | } 102 | defer o.Close() 103 | 104 | if _, err = io.Copy(o, f); err != nil { 105 | return err 106 | } 107 | 108 | return nil 109 | } 110 | 111 | type Compress struct { 112 | baseFile 113 | } 114 | 115 | func (Compress) Name() string { 116 | return "compress" 117 | } 118 | 119 | func (Compress) Perform(config interface{}, fileName string) error { 120 | c := config.(*fileConfig) 121 | 122 | str := formatName(*c, fileName) 123 | 124 | i, err := os.Open(fileName) 125 | if err != nil { 126 | return err 127 | } 128 | defer i.Close() 129 | 130 | o, err := os.Create(str) 131 | if err != nil { 132 | return err 133 | } 134 | defer o.Close() 135 | 136 | f, err := flate.NewWriter(o, flate.BestCompression) 137 | if err != nil { 138 | return err 139 | } 140 | defer f.Close() 141 | 142 | if _, err = io.Copy(f, i); err != nil { 143 | return err 144 | } 145 | 146 | return nil 147 | } 148 | 149 | type IsFile struct { 150 | baseFile 151 | } 152 | 153 | func (IsFile) Name() string { 154 | return "isfile" 155 | } 156 | 157 | func (IsFile) Perform(config interface{}, fileName string) error { 158 | if file.IsFile(fileName) { 159 | return nil 160 | } 161 | return fmt.Errorf("\"%v\" does not exist anymore\n", fileName) 162 | } 163 | 164 | type CheckSize struct { 165 | baseFile 166 | } 167 | 168 | func (CheckSize) Name() string { 169 | return "checksize" 170 | } 171 | 172 | func (CheckSize) Perform(config interface{}, fileName string) error { 173 | c := config.(*fileConfig) 174 | 175 | if !file.IsFile(fileName) { 176 | return fmt.Errorf("\"%v\" does not exist anymore\n", fileName) 177 | } 178 | 179 | var minSize datasize.ByteSize 180 | var maxSize datasize.ByteSize 181 | 182 | size, _ := file.Size(fileName) 183 | usize := uint64(size) 184 | 185 | minSize.UnmarshalText([]byte(c.MinSize)) 186 | 187 | if c.MinSize != "" && minSize.Bytes() > usize { 188 | return fmt.Errorf("size of \"%v\" to small\n", fileName) 189 | } 190 | 191 | maxSize.UnmarshalText([]byte(c.MaxSize)) 192 | 193 | if c.MaxSize != "" && maxSize.Bytes() < usize { 194 | return fmt.Errorf("size of \"%v\" to big\n", fileName) 195 | } 196 | 197 | return nil 198 | } 199 | -------------------------------------------------------------------------------- /modules/file.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "new_$file_%Y%m%d%H%M%S", 3 | "minSize": "400B", 4 | "maxSize": "5MB" 5 | } -------------------------------------------------------------------------------- /modules/http.go: -------------------------------------------------------------------------------- 1 | package modules 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "net/http" 7 | "strconv" 8 | 9 | "simonwaldherr.de/go/golibs/file" 10 | ) 11 | 12 | type httpConfig struct { 13 | Path string `json:"path"` 14 | } 15 | 16 | type HTTP struct{} 17 | 18 | func (HTTP) Name() string { 19 | return "http" 20 | } 21 | 22 | func (HTTP) EmptyConfig() interface{} { 23 | return &httpConfig{} 24 | } 25 | 26 | func (HTTP) Perform(config interface{}, fileName string) error { 27 | c := config.(*httpConfig) 28 | 29 | client := &http.Client{} 30 | str, _ := file.Read(fileName) 31 | body := bytes.NewBufferString(str) 32 | clength := strconv.Itoa(len(str)) 33 | r, _ := http.NewRequest("POST", c.Path, body) 34 | r.Header.Add("User-Agent", "FS-Agent") 35 | r.Header.Add("Content-Type", "application/x-www-form-urlencoded") 36 | r.Header.Add("Content-Length", clength) 37 | 38 | rsp, err := client.Do(r) 39 | if err != nil { 40 | return err 41 | } 42 | defer rsp.Body.Close() 43 | 44 | if rsp.StatusCode == 200 { 45 | return nil 46 | } 47 | 48 | return fmt.Errorf("the remote did not return a HTTP 200 response: %#v", rsp) 49 | } 50 | -------------------------------------------------------------------------------- /modules/http.json: -------------------------------------------------------------------------------- 1 | { 2 | "path": "http://google.de/" 3 | } -------------------------------------------------------------------------------- /modules/mail.go: -------------------------------------------------------------------------------- 1 | package modules 2 | 3 | import ( 4 | "crypto/tls" 5 | 6 | "gopkg.in/gomail.v2" 7 | ) 8 | 9 | type mailConfig struct { 10 | Name string `json:"name"` 11 | Subject string `json:"subject"` 12 | Body string `json:"body"` 13 | From string `json:"from"` 14 | To []string `json:"to"` 15 | Cc []string `json:"cc"` 16 | Bcc []string `json:"bcc"` 17 | User string `json:"user"` 18 | Pass string `json:"pass"` 19 | Server string `json:"server"` 20 | Port int `json:"port"` 21 | } 22 | 23 | func addRecipients(rtype string, recipients []string, mail *gomail.Message) { 24 | addresses := make([]string, len(recipients)) 25 | for i, recipient := range recipients { 26 | addresses[i] = mail.FormatAddress(recipient, "") 27 | } 28 | mail.SetHeader(rtype, addresses...) 29 | } 30 | 31 | type Mail struct { 32 | } 33 | 34 | func (Mail) Name() string { 35 | return "mail" 36 | } 37 | 38 | func (Mail) EmptyConfig() interface{} { 39 | return &mailConfig{} 40 | } 41 | 42 | func (Mail) Perform(config interface{}, fileName string) error { 43 | c := config.(*mailConfig) 44 | 45 | m := gomail.NewMessage() 46 | m.SetHeader("From", c.From) 47 | 48 | addRecipients("To", c.To, m) 49 | addRecipients("Cc", c.Cc, m) 50 | addRecipients("Bcc", c.Bcc, m) 51 | 52 | m.SetHeader("Subject", c.Subject) 53 | m.SetBody("text/plain", c.Body) 54 | m.Attach(fileName) 55 | 56 | d := gomail.NewDialer(c.Server, c.Port, c.User, c.Pass) 57 | d.TLSConfig = &tls.Config{InsecureSkipVerify: true} 58 | 59 | return d.DialAndSend(m) 60 | } 61 | -------------------------------------------------------------------------------- /modules/mail.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "mail", 3 | "subject": "Lorem Ipsum", 4 | "body": "dolor sit amet", 5 | "from": "notification@company.tld", 6 | "to": ["example@domain.tld"], 7 | "cc": ["example2@domain.tld"], 8 | "bcc": ["example3@domain.tld"], 9 | "user": "notification", 10 | "pass": "spring2018", 11 | "server": "webmail.domain.tld", 12 | "port": 587 13 | } 14 | -------------------------------------------------------------------------------- /modules/sleep.go: -------------------------------------------------------------------------------- 1 | package modules 2 | 3 | import ( 4 | "time" 5 | ) 6 | 7 | type sleepConfig struct { 8 | Time int `json:"time"` 9 | } 10 | 11 | type Sleep struct{} 12 | 13 | func (Sleep) Name() string { 14 | return "sleep" 15 | } 16 | 17 | func (Sleep) EmptyConfig() interface{} { 18 | return &sleepConfig{} 19 | } 20 | 21 | func (Sleep) Perform(config interface{}, fileName string) error { 22 | c := config.(*sleepConfig) 23 | 24 | time.Sleep(time.Millisecond * time.Duration(c.Time)) 25 | 26 | return nil 27 | } 28 | -------------------------------------------------------------------------------- /modules/sleep.json: -------------------------------------------------------------------------------- 1 | { 2 | "time": 500 3 | } 4 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "fsagent", 3 | "version": "1.0.1", 4 | "description": "fsagent is a Golang Application to perform various standard actions triggered by various events", 5 | "main": "fsagent.go", 6 | "files": [ 7 | "fsagent.go" 8 | ], 9 | "repository": "https://github.com/SimonWaldherr/fsagent", 10 | "keywords": [ 11 | "filesystem", 12 | "events", 13 | "golang", 14 | "devops" 15 | ], 16 | "author": { 17 | "name": "Simon Waldherr", 18 | "email": "contact@simonwaldherr.de", 19 | "twitter": "@SimonWaldherr" 20 | }, 21 | "license": "MIT License" 22 | } -------------------------------------------------------------------------------- /program.go: -------------------------------------------------------------------------------- 1 | package fsagent 2 | 3 | import ( 4 | "encoding/json" 5 | "fmt" 6 | "log" 7 | "os" 8 | "sync" 9 | 10 | "github.com/fsnotify/fsnotify" 11 | "github.com/kardianos/service" 12 | 13 | "simonwaldherr.de/go/golibs/file" 14 | ) 15 | 16 | var Done chan bool 17 | var WG sync.WaitGroup 18 | 19 | type Program struct { 20 | stop chan struct{} 21 | } 22 | 23 | func (p *Program) Start(s service.Service) error { 24 | // done is a global channel 25 | Done = make(chan bool) 26 | p.stop = make(chan struct{}) 27 | go p.run() 28 | return nil 29 | } 30 | 31 | func (p *Program) Stop(s service.Service) error { 32 | close(p.stop) 33 | <-Done 34 | WG.Wait() 35 | return nil 36 | } 37 | 38 | func (p *Program) run() { 39 | var config []Config 40 | str, _ := file.Read(os.Args[1]) 41 | 42 | err := json.Unmarshal([]byte(str), &config) 43 | 44 | if err != nil { 45 | log.Println(err) 46 | } 47 | 48 | watcher := make(map[string]*fsnotify.Watcher) 49 | 50 | for i, conf := range config { 51 | fmt.Println("load config ...") 52 | 53 | go runConfig(&WG, conf, i, p.stop, watcher) 54 | } 55 | Done <- true 56 | } 57 | -------------------------------------------------------------------------------- /util.go: -------------------------------------------------------------------------------- 1 | package fsagent 2 | 3 | import ( 4 | "os" 5 | "time" 6 | ) 7 | 8 | // FileLastModified returns the Time a file was last modified. 9 | func FileLastModified(filename string) (*time.Time, error) { 10 | f, err := os.Open(filename) 11 | if err != nil { 12 | return nil, err 13 | } 14 | defer f.Close() 15 | statInfo, _ := f.Stat() 16 | modTime := statInfo.ModTime() 17 | return &modTime, nil 18 | } 19 | -------------------------------------------------------------------------------- /web.go: -------------------------------------------------------------------------------- 1 | package fsagent 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "net/http" 7 | ) 8 | 9 | var size int64 = 200 * 1024 * 1024 10 | 11 | func serveHTTP(conf Config, fnChannel chan<- string) { 12 | http.HandleFunc(conf.Folder, func(w http.ResponseWriter, r *http.Request) { 13 | var path string 14 | if err := r.ParseMultipartForm(size); err != nil { 15 | fmt.Println(err) 16 | http.Error(w, err.Error(), http.StatusForbidden) 17 | } 18 | 19 | for _, fileHeaders := range r.MultipartForm.File { 20 | for _, fileHeader := range fileHeaders { 21 | file, _ := fileHeader.Open() 22 | path = fmt.Sprintf("%s", fileHeader.Filename) 23 | buf, _ := ioutil.ReadAll(file) 24 | tempFile, err := ioutil.TempFile("", fileHeader.Filename) 25 | if err != nil { 26 | fmt.Println(err) 27 | } 28 | tempFile.Write(buf) 29 | fnChannel <- tempFile.Name() 30 | } 31 | } 32 | fmt.Printf("File \"%v\" uploaded\n", path) 33 | }) 34 | http.Handle("/", http.FileServer(http.Dir("./web/"))) 35 | fmt.Print(http.ListenAndServe(conf.Port, nil)) 36 | } 37 | --------------------------------------------------------------------------------