├── .gitignore ├── DEVDOCS.md ├── README.md ├── adapter ├── adapter.go ├── gnl2go_adapter.go └── ipvsadm_adapter.go ├── api ├── api.go └── http_api.go ├── api_clients ├── cfg_slb.py ├── client.py ├── gokeepalived.thrift └── server.py ├── cfgparser └── cfgparser.go ├── healthchecks ├── http_check.go ├── tcp_check.go └── zk_check.go ├── main ├── go_keepalived.go └── gokeepalived_test.cfg ├── notifier ├── bgp_notifier.go └── notifier.go ├── service ├── msgs2service_processing.go ├── service.go └── service_test.go └── structure.jpg /.gitignore: -------------------------------------------------------------------------------- 1 | test.cfg 2 | bingokeep 3 | -------------------------------------------------------------------------------- /DEVDOCS.md: -------------------------------------------------------------------------------- 1 | ###### Testing on local node w/o bgp and ipvsadm support: 2 | for testing purpose you can add "testing" as a first line in cfg file. this stanza will instruct adapter module to use dummy adapter instead 3 | of ipvsadm. also you need to change notifier's name from bgp to something else (in that case we will start dummy notifier instead of bgp2go) 4 | 5 | ##Project's architecture 6 | ![alt text](https://github.com/tehnerd/go_keepalived/blob/master/structure.jpg?raw=true "Project's architecture") 7 | 8 | #### Startup process 9 | During startup we are reading configure from the cfg file (example could be found @ main/gokeepalived_test.cfg) with 10 | cfgparser (cfgparser/cfgparser.go) using ReadCfg routine. 11 | 12 | This routine is responsible for the: 13 | 1. parse config and exit the program if something goes wrong 14 | 2. populate service.ServicesList, notifier.NotifierConfig and api.GenericAPI structs (more about this structs bellow) 15 | 3. Starts API's goroutine at the end of the ReadCfg 16 | 17 | after these steps control goes back to the main function. 18 | 19 | main routine is going to start ServicesList goroutine and will wait indefinitely 20 | 21 | #### Modules description 22 | High lvl overview of program's architecture could be found on the picture above. 23 | 24 | Description of each module (as well as it's configuration parts etc could be found in sections bellow) 25 | 26 | 27 | #####Api module (api/api.go) 28 | Api module is responsible for the communication with the outside world. For example we could add/remove 29 | services and reals with api cals from the external node. 30 | 31 | Depending on cfg file we could have any type of api endpoints. Right now only http is implemented (api/http_api.go). 32 | New endpoints could be easily added. The must communication with Api thru APIMsg chans. 33 | ```go 34 | type APIMsg struct { 35 | Cmnd string //not used right now, instead we are using Data["Command"]; prob will be removed 36 | Data *map[string]string 37 | } 38 | ``` 39 | API's endpoint must convert external request to struct map[string]string and must insure that there is exists "Command" 40 | key in this struct. 41 | 42 | 43 | API module will parse the request (ProcessApiRequest routine): make sanity check (that all requered fields for particular 44 | "Command" are there, if during the startup master key was provided, it's going to check the msg's digest. 45 | 46 | at any point, if there was error during the parsing, it will return the result to endpoint for further processing. 47 | 48 | if there wasnt any error, the parsed request is going to be send to the ToServiceList chan of type service.ServiceMsg 49 | ```go 50 | type ServiceMsg struct { 51 | Cmnd string 52 | Data string 53 | DataMap *map[string]string 54 | } 55 | ``` 56 | 57 | after that API will wait for the response from the FromServiceList chan and send it back to the endpoint. 58 | 59 | #####ServicesList module (service/service.go and service/msgs2service_processing.go; struct ServicesList) 60 | After startup this module is responsible for the processing of msgs from the API. (thru the DispatchSLMsgs routine in 61 | msgs2service_processing.go). 62 | 63 | On receiving of msg from api it's going to parse it and send it further (according to the command inside the msg) 64 | to either goroutine, responsible for the particular (described in msg) service, or notifier (if, for example we would like 65 | to stop advertise availability of the service to the world) 66 | 67 | In case of service, msg is going to be send to the ToService chan (type ServiceMsg) and in case of notifier - to ToNotifier chan (type notfier.NotifierMsg) 68 | ```go 69 | type NotifierMsg struct { 70 | Type string 71 | Data string 72 | } 73 | ``` 74 | 75 | #####Service module (service.go; Service struct) 76 | Service module is responsible for the bookkeeping of the service's related information 77 | (such as: how many reals alive; if service has enough alive reals; does it exceed quorum etc; for more info check Service struct) 78 | the code is pretty much selfexplained. we run one goroutine for each of the service (check func StartService() in service.go) 79 | and listens for events from: 80 | 1. ServicesList: thru ToService chan. such as add/remove real. or shutdown the service. 81 | 2. RealServers: thru FromReal chan. such as alive/dead or fatal error in real 82 | 83 | and notify adapter about this events (to add/remove alive/daed reals in forwarding path; or to send msg thru notifier) 84 | 85 | 86 | #####RealServer module (service.go; RealServer struct) 87 | RealServer module is responsible for bookkeeping information about healthcheks results toward this real 88 | it listens for: 89 | 1. Events from healthcheck (if it was successfull or not).and if server became alive -> send this info to the Service goroutine. 90 | 2. Events from Service (right now there is only one event: to shutdown/remove real from the pool. in future we could change checks and/or weight of the real etc) 91 | 92 | During startup this module starts healthcheck goroutine of particular (according to the configuration) type. 93 | right now we support tcp healthcheck and http/https (healtchecks/tcp_check.go and http_check.go) 94 | 95 | 96 | #####Adapter module (adapter/adapter.go adapter/ipvsadm_adapter.go) 97 | Adapter is responsible for the programming of forwarding path. 98 | 99 | right now it also acts as a proxy to notifier module 100 | 101 | currently we supports ipvsadm (and therefore linux's lvs; adapter/ipvsadm_adapter.go) and dummy (which just prints recved msgs. using it for the testing) 102 | as a way to programm the forwarding path. 103 | but it shouldn't be hard to rewrite this part if something else will be needed. 104 | 105 | Adapter listens for msgs on chan type AdapterMsg. 106 | 107 | Received msgs will be parsed with IPVSAdmExec routine and the will be proxied to the notifier (right now we dont destinct if msg is destined to adapter or notifier; 108 | if either of em doesnt support msg's type it just ignores it. for example adapter ignores "AdvertiseService" msg's type and notifier ignores "AddReal") 109 | 110 | 111 | #####Notifier module (notifier/notifier.go notifier/bgp_notifier.go) 112 | This module is responsible for notification of "outside world" about health of Service on local node. Right now we are using bgp for that purpose 113 | (thru github.com/tehnerd/bgp2go lib) (and dummy notifier if case of testing; which just prints all rcved msgs) 114 | 115 | this module is listening for the msgs on chan NotifierMsg, parse em, and send parsed commands to bgp2go lib. 116 | for example we could either advertise host-route to local bgp peer, if service became "alive" or "withdraw" it, if there is zero (or less than quorum) 117 | alive realservers. 118 | 119 | all suported commands could be found in notifier/bgp_notifier.go and in future will be described in user's guide. 120 | 121 | 122 | #####Misc: 123 | example of http's api client could be found @ api_clients/cfg_slb.py (TODO: add comments with example of all supported commands. 124 | i'm afraid right now you need to check api/api.go for supported commands) 125 | 126 | 127 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Golang based keepalive server 2 | [developer's documentation](https://github.com/tehnerd/go_keepalived/blob/master/DEVDOCS.md) 3 | ### Goals of the project 4 | The main goal of the project is to build keepalived-like server with easy way to decouple control plane 5 | and data path configuration (keepalived is tightly coupled with linux'es ipvs) and easy way to add new features 6 | (i think that it's much easier to add something in Go program, than into C-bassed). 7 | Other goals: 8 | try to implement other control plane features in main server (for example bgp-signaling to upstream peer that we 9 | have live service; i want to try to write my own go-based tiny bgp implementation instead of using external bgp 10 | signaling (thru exabgp/bird/quagga)) 11 | 12 | 13 | ### Current state: 14 | * basic configuration parser (it's working, not so many checks, but at least we can read cfg files and move on; 15 | it's not that hard to add new derectives into the parser; so rewriting it (or make it more "beautiful") isn't my main 16 | priority 17 | 18 | * so far we have implemented two adapters (adapter in our case is a way to program forwarding path): 19 | ipvsadm (limited features supported; we are using it thru os/exec) and generic netlink msgs (thru github.com/tehnerd/gnl2go lib) 20 | by default we are using ipvsadm (check cfg example inside main for more info about how to use netlink instead of ipvsadm) 21 | 22 | * tcp and http/https healtchecks 23 | 24 | * signaling that real server has failed it's check/or check was successful 25 | 26 | * bgp speaker/injector. now we can advertise/withdraw v4 routes into routing domain 27 | (work in progress; https://github.com/tehnerd/bgp2go) 28 | * initial IPv6 support (v6 vips + v6reals; TODO: check that v4 vips/v6 reals works with netlink). we can advertise VIPs to bgp peers as well. 29 | * initial support for http api (todo: httpS; more features) 30 | * testing support for zookeper's based healthcheck (actually just fooling around with zookeeper; IMO this is wrong way to implement ZK's support. 31 | it will make more sense to make separate, ZK bassed tool, which will pull all the configuration (IPs of VIPs, state of reals, other type of 32 | cfg from) from ZK, and wont have any cfg file at all (or just with few lines - IPs of ZK itself). If you are interesting in this type of tool - drop me a not 33 | and/or PR) 34 | 35 | basicly that gives us a minimal feature set to move on with other parts of the project. 36 | 37 | ### Future plans and TODOs: 38 | * draft of howto/tutorial 39 | * add more features into http api 40 | * add other external apis for services configuration (grpc,thrift(if i able to find proper go's 41 | thrift's compiler etc) 42 | * stability and features improvement (lots of things must be added) 43 | * ... 44 | -------------------------------------------------------------------------------- /adapter/adapter.go: -------------------------------------------------------------------------------- 1 | package adapter 2 | 3 | import ( 4 | "fmt" 5 | "go_keepalived/notifier" 6 | "regexp" 7 | "strings" 8 | ) 9 | 10 | /* 11 | this type of messages are using to communicate to/from 12 | adapter (right now we are using only ipvsadm as adapter for 13 | datapath; mb will add something else in the future (i hope there would be 14 | libnl for go; so we wont need external bin's). we are trying to be implementation 15 | agnostic (so it would be easy to add something else (for example l7 slb's cfg parser etc)) 16 | */ 17 | type AdapterMsg struct { 18 | /* 19 | Message Type. Could be: 20 | "AddX","ChangeX","DeleteX". msgs derection Service->Adapter; where X could be either 21 | "Service" or "Real" 22 | "Error", Adapter->Service 23 | "AdvertiseService","WithdrawService" when service goes up/down 24 | */ 25 | Type string 26 | 27 | /* 28 | We need to be able to distinguish one service from another. service could be 29 | uniquely identified by: VIP,Proto,Port 30 | Meta field: for misc info, such as scheduler's type etc 31 | */ 32 | ServiceVIP string 33 | ServicePort string 34 | ServiceProto string 35 | ServiceMeta string 36 | /* 37 | Same as for service, but for realServers. Could be identified by 38 | RIP,Port,Meta. Also we want to be able to change Reals weight (in future) 39 | So we are passing Real's weight as well 40 | */ 41 | RealServerRIP string 42 | RealServerPort string 43 | RealServerMeta string 44 | RealServerWeight string 45 | } 46 | 47 | func StartAdapter(msgChan, replyChan chan AdapterMsg, 48 | notifierChan chan notifier.NotifierMsg, adapterType string) { 49 | loop := 1 50 | numRe, _ := regexp.Compile(`^\d{1,}$`) 51 | for loop == 1 { 52 | select { 53 | case msg := <-msgChan: 54 | switch adapterType { 55 | case "ipvsadm": 56 | err := IPVSAdmExec(&msg) 57 | if err != nil { 58 | //TODO(tehnerd): proper handling 59 | continue 60 | } 61 | case "netlink", "gnl2go": 62 | err := GNLExec(&msg) 63 | if err != nil { 64 | fmt.Println(err) 65 | continue 66 | } 67 | case "testing": 68 | DummyAdapter(&msg) 69 | default: 70 | DummyAdapter(&msg) 71 | } 72 | /* 73 | The whole purpose of notifier is to tell about state of our services to 74 | outerworld (right now the only implemented notifier is bgp injector) 75 | */ 76 | if numRe.MatchString(msg.ServiceVIP) { 77 | /* 78 | this section is for fwmark ipvs service. for this type of service 79 | we asume that msg.ServiceMeta would be formated like 80 |
... 81 | */ 82 | fields := strings.Fields(msg.ServiceMeta) 83 | if len(fields) >= 3 { 84 | msg.ServiceVIP = fields[2] 85 | } 86 | } 87 | notifierChan <- notifier.NotifierMsg{Type: msg.Type, Data: msg.ServiceVIP} 88 | } 89 | } 90 | } 91 | 92 | func DummyAdapter(msg *AdapterMsg) { 93 | fmt.Printf("Dummy Adapter Msg: %#v\n", msg) 94 | } 95 | -------------------------------------------------------------------------------- /adapter/gnl2go_adapter.go: -------------------------------------------------------------------------------- 1 | package adapter 2 | 3 | import ( 4 | "regexp" 5 | "strconv" 6 | "strings" 7 | 8 | "github.com/tehnerd/gnl2go" 9 | ) 10 | 11 | const ( 12 | AF_INET6 = 10 13 | AF_INET = 2 14 | ) 15 | 16 | var ( 17 | supportedCommands = createMap([]string{ 18 | "AddService", 19 | "DeleteService", 20 | "ChangeService", 21 | "AddRealServer", 22 | "DeleteRealServer", 23 | "ChangeRealServer", 24 | }) 25 | ) 26 | 27 | func createMap(stringSlice []string) map[string]bool { 28 | m := make(map[string]bool) 29 | for _, v := range stringSlice { 30 | m[v] = true 31 | } 32 | return m 33 | } 34 | 35 | type NetlinkCommand struct { 36 | Command string 37 | Vip string 38 | Rip string 39 | Scheduler string 40 | VipPort uint16 41 | RipPort uint16 42 | Protocol uint16 43 | Weight int32 44 | Fwd uint32 45 | VAF uint16 46 | FWMark uint32 47 | } 48 | 49 | func parseAdapterMsg(msg *AdapterMsg) (NetlinkCommand, error) { 50 | v6re, _ := regexp.Compile(`^\[(((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8})\]$`) 51 | var nlcmnd NetlinkCommand 52 | nlcmnd.Command = msg.Type 53 | if v6re.MatchString(msg.ServiceVIP) { 54 | vip := v6re.FindStringSubmatch(msg.ServiceVIP) 55 | nlcmnd.Vip = vip[1] 56 | } else { 57 | nlcmnd.Vip = msg.ServiceVIP 58 | } 59 | if v6re.MatchString(msg.RealServerRIP) { 60 | rip := v6re.FindStringSubmatch(msg.RealServerRIP) 61 | nlcmnd.Rip = rip[1] 62 | } else { 63 | nlcmnd.Rip = msg.RealServerRIP 64 | } 65 | if len(msg.ServicePort) > 0 { 66 | vport, err := strconv.ParseUint(msg.ServicePort, 10, 16) 67 | if err != nil { 68 | return nlcmnd, err 69 | } 70 | nlcmnd.VipPort = uint16(vport) 71 | } 72 | if len(msg.RealServerPort) > 0 { 73 | rport, err := strconv.ParseUint(msg.RealServerPort, 10, 16) 74 | if err != nil { 75 | return nlcmnd, err 76 | } 77 | nlcmnd.RipPort = uint16(rport) 78 | } 79 | nlcmnd.Protocol = uint16(gnl2go.ToProtoNum(gnl2go.NulStringType(msg.ServiceProto))) 80 | /* this means that service isn't tcp or udp - theorefor fwmark */ 81 | if nlcmnd.Protocol == 0 { 82 | fwmark, err := strconv.ParseUint(msg.ServiceVIP, 10, 32) 83 | if err != nil { 84 | return nlcmnd, err 85 | } 86 | nlcmnd.FWMark = uint32(fwmark) 87 | nlcmnd.Vip = "fwmark" 88 | nlcmnd.VipPort = 0 89 | nlcmnd.RipPort = 0 90 | nlcmnd.VAF = AF_INET 91 | } 92 | 93 | if len(msg.RealServerWeight) > 0 { 94 | weight, err := strconv.ParseInt(msg.RealServerWeight, 10, 32) 95 | if err != nil { 96 | return nlcmnd, err 97 | } 98 | nlcmnd.Weight = int32(weight) 99 | /* srvces metainfo will contain: scheduler vip_address_family etc */ 100 | } 101 | srvcMetaFields := strings.Fields(msg.ServiceMeta) 102 | if len(srvcMetaFields) >= 1 { 103 | nlcmnd.Scheduler = srvcMetaFields[0] 104 | } else { 105 | nlcmnd.Scheduler = "wrr" 106 | } 107 | if len(srvcMetaFields) >= 2 { 108 | switch srvcMetaFields[1] { 109 | case "ipv6": 110 | nlcmnd.VAF = AF_INET6 111 | default: 112 | nlcmnd.VAF = AF_INET 113 | } 114 | } 115 | realMetaFields := strings.Fields(msg.RealServerMeta) 116 | if len(realMetaFields) >= 1 { 117 | switch msg.RealServerMeta { 118 | case "tunnel", "tun": 119 | nlcmnd.Fwd = gnl2go.IPVS_TUNNELING 120 | default: 121 | nlcmnd.Fwd = gnl2go.IPVS_MASQUERADING 122 | } 123 | } else { 124 | nlcmnd.Fwd = gnl2go.IPVS_MASQUERADING 125 | } 126 | return nlcmnd, nil 127 | } 128 | 129 | func ExecCmnd(nlcmnd NetlinkCommand) error { 130 | ipvs := new(gnl2go.IpvsClient) 131 | err := ipvs.Init() 132 | if err != nil { 133 | return err 134 | } 135 | defer ipvs.Exit() 136 | switch nlcmnd.Command { 137 | case "AddService": 138 | if nlcmnd.Vip == "fwmark" { 139 | return ipvs.AddFWMService(nlcmnd.FWMark, 140 | nlcmnd.Scheduler, nlcmnd.VAF) 141 | } else { 142 | return ipvs.AddService(nlcmnd.Vip, nlcmnd.VipPort, 143 | nlcmnd.Protocol, nlcmnd.Scheduler) 144 | } 145 | case "DeleteService": 146 | if nlcmnd.Vip == "fwmark" { 147 | return ipvs.DelFWMService(nlcmnd.FWMark, nlcmnd.VAF) 148 | } else { 149 | return ipvs.DelService(nlcmnd.Vip, nlcmnd.VipPort, nlcmnd.Protocol) 150 | } 151 | case "ChangeService": 152 | return nil 153 | case "AddRealServer": 154 | if nlcmnd.Vip == "fwmark" { 155 | return ipvs.AddFWMDestFWD(nlcmnd.FWMark, nlcmnd.Rip, nlcmnd.VAF, 156 | nlcmnd.VipPort, nlcmnd.Weight, nlcmnd.Fwd) 157 | } else { 158 | return ipvs.AddDestPort(nlcmnd.Vip, nlcmnd.VipPort, nlcmnd.Rip, 159 | nlcmnd.RipPort, nlcmnd.Protocol, nlcmnd.Weight, nlcmnd.Fwd) 160 | } 161 | case "DeleteRealServer": 162 | if nlcmnd.Vip == "fwmark" { 163 | return ipvs.DelFWMDest(nlcmnd.FWMark, nlcmnd.Rip, nlcmnd.VAF, 164 | nlcmnd.RipPort) 165 | } else { 166 | return ipvs.DelDestPort(nlcmnd.Vip, nlcmnd.VipPort, nlcmnd.Rip, 167 | nlcmnd.RipPort, nlcmnd.Protocol) 168 | } 169 | case "ChangeRealServer": 170 | if nlcmnd.Vip == "fwmark" { 171 | return ipvs.UpdateFWMDestFWD(nlcmnd.FWMark, nlcmnd.Rip, nlcmnd.VAF, 172 | nlcmnd.RipPort, nlcmnd.Weight, nlcmnd.Fwd) 173 | } else { 174 | return ipvs.UpdateDestPort(nlcmnd.Vip, nlcmnd.VipPort, nlcmnd.Rip, 175 | nlcmnd.RipPort, nlcmnd.Protocol, nlcmnd.Weight, nlcmnd.Fwd) 176 | } 177 | } 178 | return nil 179 | } 180 | 181 | func GNLExec(msg *AdapterMsg) error { 182 | if _, exists := supportedCommands[msg.Type]; !exists { 183 | return nil 184 | } 185 | nlcmnd, err := parseAdapterMsg(msg) 186 | if err != nil { 187 | return err 188 | } 189 | return ExecCmnd(nlcmnd) 190 | } 191 | -------------------------------------------------------------------------------- /adapter/ipvsadm_adapter.go: -------------------------------------------------------------------------------- 1 | package adapter 2 | 3 | import ( 4 | "fmt" 5 | "os/exec" 6 | "strings" 7 | ) 8 | 9 | const ( 10 | //TODO: mb add something else. like /usr/bin/ipvsadm etc 11 | IPVSADM_CMND = "ipvsadm" 12 | ) 13 | 14 | /* 15 | This is ipvsadm's view of AdapterMsg struct 16 | */ 17 | type IPVSAdmCmnds struct { 18 | VIP string 19 | RIP string 20 | // "-A","-D","-a","-d", "-E", "-e" 21 | ActionFlag string 22 | // "-s"; select schedulers type 23 | SchedFlag string 24 | // "-t", "-u", "-f" 25 | ProtoFlag string 26 | // "-w" 27 | WeightFlag string 28 | /* 29 | flag, which indicates that command must include/exclude particular fields 30 | 0 - means there is no realServer's info and we adding service 31 | 1 - delete service 32 | 128 - add or change real 33 | 129 - delete real 34 | */ 35 | CommandFlag uint8 36 | //method of lb; such as ipip or nat; -m or -i 37 | MetaInfo string 38 | } 39 | 40 | func IPVSParseAdapterMsg(msg *AdapterMsg) IPVSAdmCmnds { 41 | ipvsCmnds := IPVSAdmCmnds{} 42 | //TODO: check (prob regexp) for ipv6 43 | ipvsCmnds.VIP = strings.Join([]string{msg.ServiceVIP, msg.ServicePort}, ":") 44 | ipvsCmnds.RIP = strings.Join([]string{msg.RealServerRIP, msg.RealServerPort}, ":") 45 | switch msg.ServiceProto { 46 | case "tcp", "Tcp", "TCP": 47 | ipvsCmnds.ProtoFlag = "-t" 48 | case "udp", "UDP", "Udp": 49 | ipvsCmnds.ProtoFlag = "-u" 50 | case "fwmark", "FW": 51 | ipvsCmnds.ProtoFlag = "-f" 52 | } 53 | /* 54 | TODO: meta (masq or tun), weight, sched etc... now we are trying 55 | to implement the most basic functions for POC 56 | metainfo hardcoded now just for the sake of implementing minimum working 57 | adapter 58 | */ 59 | ipvsCmnds.MetaInfo = msg.RealServerMeta 60 | ipvsCmnds.SchedFlag = msg.ServiceMeta 61 | ipvsCmnds.WeightFlag = msg.RealServerWeight 62 | return ipvsCmnds 63 | } 64 | 65 | func IPVSCliArgs(ipvsCmnds *IPVSAdmCmnds) []string { 66 | cliCmnds := []string{} 67 | cliCmnds = append(cliCmnds, ipvsCmnds.ActionFlag, ipvsCmnds.ProtoFlag, 68 | ipvsCmnds.VIP) 69 | if ipvsCmnds.CommandFlag > 127 { 70 | cliCmnds = append(cliCmnds, "-r", ipvsCmnds.RIP) 71 | if ipvsCmnds.CommandFlag == 128 { 72 | switch ipvsCmnds.MetaInfo { 73 | case "tunnel": 74 | cliCmnds = append(cliCmnds, "-i") 75 | default: 76 | cliCmnds = append(cliCmnds, "-m") 77 | } 78 | // this section is for add/change real commands 79 | cliCmnds = append(cliCmnds, "-w", ipvsCmnds.WeightFlag) 80 | } 81 | } else { 82 | switch ipvsCmnds.CommandFlag { 83 | case 0: 84 | cliCmnds = append(cliCmnds, "-s", ipvsCmnds.SchedFlag) 85 | } 86 | } 87 | return cliCmnds 88 | } 89 | 90 | func IPVSAdmExec(msg *AdapterMsg) error { 91 | ipvsCmnds := IPVSParseAdapterMsg(msg) 92 | switch msg.Type { 93 | case "AdvertiseService", "WithdrawService": 94 | return nil 95 | case "AddService": 96 | ipvsCmnds.ActionFlag = "-A" 97 | case "DeleteService": 98 | ipvsCmnds.ActionFlag = "-D" 99 | ipvsCmnds.CommandFlag = 1 100 | case "ChangeService": 101 | ipvsCmnds.ActionFlag = "-E" 102 | case "AddRealServer": 103 | ipvsCmnds.ActionFlag = "-a" 104 | ipvsCmnds.CommandFlag = 128 105 | case "DeleteRealServer": 106 | ipvsCmnds.ActionFlag = "-d" 107 | ipvsCmnds.CommandFlag = 129 108 | case "ChangeRealServer": 109 | ipvsCmnds.ActionFlag = "-e" 110 | ipvsCmnds.CommandFlag = 128 111 | 112 | } 113 | cliArgs := IPVSCliArgs(&ipvsCmnds) 114 | fmt.Println(cliArgs) 115 | execCmnd := exec.Command(IPVSADM_CMND, cliArgs...) 116 | /* 117 | TODO: better check for error etc; sanitizing cliArgs (so we wont have "-A; rm -rf") 118 | */ 119 | output, err := execCmnd.CombinedOutput() 120 | if err != nil { 121 | return err 122 | } 123 | fmt.Println(string(output)) 124 | return nil 125 | } 126 | -------------------------------------------------------------------------------- /api/api.go: -------------------------------------------------------------------------------- 1 | package api 2 | 3 | import ( 4 | "crypto/hmac" 5 | "crypto/sha256" 6 | "encoding/hex" 7 | "fmt" 8 | "go_keepalived/service" 9 | "regexp" 10 | "sort" 11 | ) 12 | 13 | type APIMsg struct { 14 | Cmnd string 15 | Data *map[string]string 16 | } 17 | 18 | type GenericAPI struct { 19 | Enable bool 20 | ToServiceList chan service.ServiceMsg 21 | FromServiceList chan service.ServiceMsg 22 | ToApi chan APIMsg 23 | FromApi chan APIMsg 24 | HttpApi bool 25 | //TODO: 26 | GRPCApi bool 27 | ThriftApi bool 28 | //TODO: for authentication/authorization 29 | MasterPwd string 30 | } 31 | 32 | func InitAPI(api GenericAPI, ToServiceList, FromServiceList chan service.ServiceMsg) { 33 | api.ToApi = make(chan APIMsg) 34 | api.FromApi = make(chan APIMsg) 35 | api.ToServiceList = ToServiceList 36 | api.FromServiceList = FromServiceList 37 | if api.HttpApi { 38 | go StartHTTPApi(api.ToApi, api.FromApi) 39 | } 40 | for { 41 | select { 42 | case request := <-api.ToApi: 43 | resp := ProcessApiRequest(&api, request) 44 | if resp != nil { 45 | api.FromApi <- APIMsg{Data: &resp} 46 | } else { 47 | api.FromApi <- APIMsg{Data: &map[string]string{"result": "wrong command"}} 48 | } 49 | } 50 | } 51 | } 52 | 53 | /* 54 | Right now i want to implement this type of Command: 55 | "GetInfo" - get generic info about services 56 | "AddReal" - add real server to service 57 | "RemoveReal" - remove real server from service 58 | "AddService" - add new service 59 | "RemoveService" - remove service 60 | "ChangeReal" - change data in real server (for example weight) 61 | "ChangeService" - change data in service (for example quorum etc) 62 | "AddPeer" - add new peer (fro bgp based notifier) 63 | "RemovePeer" - remove peer (fro bgp based notifier) 64 | "StopNotification" - stop advertise that service is available to external world 65 | "StartNotification" - start to advertise "alive" service to external world 66 | */ 67 | 68 | func ProcessApiRequest(api *GenericAPI, request APIMsg) map[string]string { 69 | if api.MasterPwd != "" { 70 | err := checkRequestAuth(request, []byte(api.MasterPwd)) 71 | if err != nil { 72 | return map[string]string{"result": "false", "info": "incorrect msg digest"} 73 | } 74 | } 75 | cmnd := (*request.Data)["Command"] 76 | switch cmnd { 77 | case "GetInfo": 78 | api.ToServiceList <- service.ServiceMsg{Cmnd: "GetInfo"} 79 | resp := <-api.FromServiceList 80 | return (*resp.DataMap) 81 | case "AddService": 82 | err := serviceSanityCheck(request.Data) 83 | if err != nil { 84 | return nil 85 | } 86 | api.ToServiceList <- service.ServiceMsg{Cmnd: "AddService", DataMap: request.Data} 87 | resp := <-api.FromServiceList 88 | return (*resp.DataMap) 89 | case "RemoveService": 90 | err := serviceSanityCheck(request.Data) 91 | if err != nil { 92 | return nil 93 | } 94 | api.ToServiceList <- service.ServiceMsg{Cmnd: "RemoveService", DataMap: request.Data} 95 | resp := <-api.FromServiceList 96 | return (*resp.DataMap) 97 | case "ChangeService": 98 | err := serviceSanityCheck(request.Data) 99 | if err != nil { 100 | return nil 101 | } 102 | api.ToServiceList <- service.ServiceMsg{Cmnd: "ChangeService", DataMap: request.Data} 103 | resp := <-api.FromServiceList 104 | return (*resp.DataMap) 105 | case "AddReal": 106 | err := serviceSanityCheck(request.Data) 107 | if err != nil { 108 | return nil 109 | } 110 | err = realSrvSanityCheck(request.Data) 111 | if err != nil { 112 | return nil 113 | } 114 | api.ToServiceList <- service.ServiceMsg{Cmnd: "AddReal", DataMap: request.Data} 115 | resp := <-api.FromServiceList 116 | return (*resp.DataMap) 117 | case "RemoveReal": 118 | err := serviceSanityCheck(request.Data) 119 | if err != nil { 120 | return nil 121 | } 122 | err = realSrvSanityCheck(request.Data) 123 | if err != nil { 124 | return nil 125 | } 126 | api.ToServiceList <- service.ServiceMsg{Cmnd: "RemoveReal", DataMap: request.Data} 127 | resp := <-api.FromServiceList 128 | return (*resp.DataMap) 129 | case "ChangeReal": 130 | err := serviceSanityCheck(request.Data) 131 | if err != nil { 132 | return nil 133 | } 134 | err = realSrvSanityCheck(request.Data) 135 | if err != nil { 136 | return nil 137 | } 138 | api.ToServiceList <- service.ServiceMsg{Cmnd: "ChangeReal", DataMap: request.Data} 139 | resp := <-api.FromServiceList 140 | return (*resp.DataMap) 141 | case "AddPeer": 142 | err := peerSanityCheck(request.Data) 143 | if err != nil { 144 | return nil 145 | } 146 | api.ToServiceList <- service.ServiceMsg{Cmnd: "AddPeer", DataMap: request.Data} 147 | resp := <-api.FromServiceList 148 | return (*resp.DataMap) 149 | case "RemovePeer": 150 | err := peerSanityCheck(request.Data) 151 | if err != nil { 152 | return nil 153 | } 154 | api.ToServiceList <- service.ServiceMsg{Cmnd: "RemovePeer", DataMap: request.Data} 155 | resp := <-api.FromServiceList 156 | return (*resp.DataMap) 157 | case "StopNotification": 158 | err := addressSanityCheck(request.Data) 159 | if err != nil { 160 | return nil 161 | } 162 | api.ToServiceList <- service.ServiceMsg{Cmnd: "StopNotification", DataMap: request.Data} 163 | resp := <-api.FromServiceList 164 | return (*resp.DataMap) 165 | case "StartNotification": 166 | err := addressSanityCheck(request.Data) 167 | if err != nil { 168 | return nil 169 | } 170 | api.ToServiceList <- service.ServiceMsg{Cmnd: "StartNotification", DataMap: request.Data} 171 | resp := <-api.FromServiceList 172 | return (*resp.DataMap) 173 | case "StartAllNotification", "StopAllNotification": 174 | api.ToServiceList <- service.ServiceMsg{Cmnd: (*request.Data)["Command"]} 175 | resp := <-api.FromServiceList 176 | return (*resp.DataMap) 177 | } 178 | return nil 179 | } 180 | 181 | func serviceSanityCheck(request *map[string]string) error { 182 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 183 | v6re, _ := regexp.Compile(`^\[((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8}\]$`) 184 | numRe, _ := regexp.Compile(`^\d+$`) 185 | if vip, exists := (*request)["VIP"]; !exists { 186 | return fmt.Errorf("request doesnt has mandatory VIP field\n") 187 | } else { 188 | if !v4re.MatchString(vip) && !v6re.MatchString(vip) { 189 | return fmt.Errorf("VIP is not v4 or v6 address\n") 190 | } 191 | } 192 | if _, exists := (*request)["Proto"]; !exists { 193 | return fmt.Errorf("request doesnt has mandatory Proto field\n") 194 | } 195 | if port, exists := (*request)["Port"]; !exists { 196 | return fmt.Errorf("request doesnt has mandatory Port field\n") 197 | } else if !numRe.MatchString(port) { 198 | return fmt.Errorf("port must be a number\n") 199 | } 200 | return nil 201 | } 202 | 203 | func realSrvSanityCheck(request *map[string]string) error { 204 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 205 | v6re, _ := regexp.Compile(`^\[((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8}\]$`) 206 | numRe, _ := regexp.Compile(`^\d+$`) 207 | if rip, exists := (*request)["RIP"]; !exists { 208 | return fmt.Errorf("request doesnt has mandatory RIP field\n") 209 | } else { 210 | if !v4re.MatchString(rip) && !v6re.MatchString(rip) { 211 | return fmt.Errorf("RIP is not v4 or v6 address\n") 212 | } 213 | } 214 | if _, exists := (*request)["Check"]; !exists { 215 | return fmt.Errorf("request doesnt has mandatory Check field\n") 216 | } 217 | if port, exists := (*request)["RealPort"]; !exists { 218 | return fmt.Errorf("request doesnt has mandatory Port field\n") 219 | } else if !numRe.MatchString(port) { 220 | return fmt.Errorf("port must be a number\n") 221 | } 222 | return nil 223 | } 224 | 225 | func peerSanityCheck(request *map[string]string) error { 226 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 227 | v6re, _ := regexp.Compile(`^((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8}$`) 228 | if addr, exists := (*request)["Address"]; !exists { 229 | return fmt.Errorf("request doesnt has mandatory Address field\n") 230 | } else { 231 | if !v4re.MatchString(addr) && !v6re.MatchString(addr) { 232 | return fmt.Errorf("Address is not v4 or v6 address\n") 233 | } 234 | } 235 | return nil 236 | } 237 | 238 | func addressSanityCheck(request *map[string]string) error { 239 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 240 | v6re, _ := regexp.Compile(`^\[((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8}\]$`) 241 | if vip, exists := (*request)["VIP"]; !exists { 242 | return fmt.Errorf("request doesnt has mandatory VIP field\n") 243 | } else { 244 | if !v4re.MatchString(vip) && !v6re.MatchString(vip) { 245 | return fmt.Errorf("VIP is not v4 or v6 address\n") 246 | } 247 | } 248 | return nil 249 | } 250 | 251 | func checkRequestAuth(request APIMsg, hmacKey []byte) error { 252 | //TODO: add salt ? 253 | if _, exist := (*request.Data)["Digest"]; !exist { 254 | return fmt.Errorf("msg doesnt contain Digest field\n") 255 | } 256 | keyList := make([]string, 0) 257 | mac := hmac.New(sha256.New, hmacKey) 258 | msgMac := make([]byte, 0) 259 | //TODO: mb there is a better way to calc the HMAC of map 260 | for key, val := range *request.Data { 261 | if key == "Digest" { 262 | mac, err := hex.DecodeString(val) 263 | if err != nil { 264 | return fmt.Errorf("cant decode Digest field\n") 265 | } 266 | msgMac = mac 267 | continue 268 | } 269 | keyList = append(keyList, key) 270 | } 271 | sort.Strings(keyList) 272 | for _, key := range keyList { 273 | mac.Write([]byte((*request.Data)[key])) 274 | } 275 | expectedMAC := mac.Sum(nil) 276 | if !hmac.Equal(msgMac, expectedMAC) { 277 | return fmt.Errorf("msg digest is incorrect\n") 278 | } 279 | return nil 280 | } 281 | -------------------------------------------------------------------------------- /api/http_api.go: -------------------------------------------------------------------------------- 1 | package api 2 | 3 | import ( 4 | "encoding/json" 5 | "net/http" 6 | ) 7 | 8 | func makeChanHandler(fn func(w http.ResponseWriter, r *http.Request, a, b chan APIMsg), a, b chan APIMsg) http.HandlerFunc { 9 | return func(w http.ResponseWriter, r *http.Request) { 10 | fn(w, r, a, b) 11 | } 12 | } 13 | 14 | func handler(w http.ResponseWriter, r *http.Request, requestChan, 15 | responseChan chan APIMsg) { 16 | if r.Method != "POST" { 17 | w.WriteHeader(http.StatusBadRequest) 18 | return 19 | } 20 | requestStruct := make(map[string]string) 21 | decoder := json.NewDecoder(r.Body) 22 | err := decoder.Decode(&requestStruct) 23 | if err != nil { 24 | w.WriteHeader(http.StatusBadRequest) 25 | return 26 | } 27 | if _, exists := requestStruct["Command"]; !exists { 28 | w.WriteHeader(http.StatusBadRequest) 29 | w.Write([]byte("Command argument is required")) 30 | return 31 | } 32 | requestChan <- APIMsg{Data: &requestStruct} 33 | responseStruct := <-responseChan 34 | resp, err := json.Marshal(responseStruct) 35 | if err != nil { 36 | w.WriteHeader(http.StatusInternalServerError) 37 | w.Write([]byte("cant marshal json response")) 38 | return 39 | } 40 | w.Write(resp) 41 | } 42 | 43 | //TODO: https instead of http 44 | func StartHTTPApi(requestChan, responseChan chan APIMsg) { 45 | http.HandleFunc("/", makeChanHandler(handler, requestChan, responseChan)) 46 | http.ListenAndServe("localhost:62307", nil) 47 | } 48 | -------------------------------------------------------------------------------- /api_clients/cfg_slb.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | try: 4 | from urllib import request as urllib_request 5 | except ImportError: 6 | import urllib2 as urllib_request 7 | import json 8 | import sys 9 | import hmac 10 | import hashlib 11 | 12 | class slbAPI(object): 13 | 14 | def __init__(self, endpoint, password): 15 | self._endpoint = endpoint 16 | self._password = password 17 | 18 | def ExecCmnd(self,cmnd_data): 19 | line = str() 20 | keyList = list() 21 | for key in cmnd_data: 22 | keyList.append(key) 23 | for key in sorted(keyList): 24 | line = "".join((line,cmnd_data[key])) 25 | cmnd_data["Digest"] = hmac.new(self._password, line, digestmod=hashlib.sha256).hexdigest() 26 | data = json.dumps(cmnd_data) 27 | try: 28 | request = urllib_request.Request(self._endpoint,data) 29 | reader = urllib_request.urlopen(request) 30 | api_response = reader.read() 31 | resp = json.loads(api_response) 32 | return resp 33 | except urllib_request.HTTPError: 34 | #TODO: more meaningfull comment; more errors handling 35 | return {"result":"error during api urllib_request"} 36 | except: 37 | return {"result":"generic error"} 38 | 39 | 40 | 41 | 42 | def main(): 43 | ''' 44 | examples of supported commands(lots of copy paste right now while i'm writing new features 45 | and testing the output; gona remove it later) 46 | ''' 47 | url = sys.argv[1] 48 | slb = slbAPI(url,"123") 49 | data = {"Command":"GetInfo"} 50 | #data = {"Command":"AddService", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp"} 51 | #data = {"Command":"RemoveService", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp"} 52 | #data = {"Command":"ChangeService", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp"} 53 | #data = {"Command":"AddReal", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp", 54 | # "RIP":"[fc00::1]","RealPort":"22","Check":"tcp"} 55 | #data = {"Command":"RemoveReal", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp", 56 | # "RIP":"[fc00::1]","RealPort":"22","Check":"tcp"} 57 | #data = {"Command":"ChangeReal", "VIP":"[fc12:1::1]","Port":"22","Proto":"tcp", 58 | # "RIP":"[fc00::1]","RealPort":"22","Check":"tcp"} 59 | #data = {"Command":"AddPeer", "Address":"fc12:1::1"} 60 | #data = {"Command":"RemovePeer", "Address":"fc12:1::1"} 61 | #data = {"Command":"StartNotification", "VIP":"[fc12:1::1]"} 62 | #data = {"Command":"StopNotification", "VIP":"[fc12:1::1]"} 63 | #data = {"Command":"StopAllNotification"} 64 | #data = {"Command":"StartAllNotification"} 65 | resp = slb.ExecCmnd(data) 66 | print(resp) 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | if __name__ == "__main__": 78 | main() 79 | 80 | 81 | -------------------------------------------------------------------------------- /api_clients/client.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | gen_py_path = '/tmp/' 4 | thrift_pylib = '/tmp/' 5 | if 'GEN_PY' in os.environ: 6 | gen_py_path = os.environ['GEN_PY'] 7 | if 'THRIFT_PYLIB' in os.environ: 8 | thrift_pylib = os.environ['THRIFT_PYLIB'] 9 | sys.path.append(gen_py_path) 10 | sys.path.append(thrift_pylib) 11 | 12 | from gokeepalived import Gokeepalived 13 | from gokeepalived.ttypes import * 14 | 15 | from thrift import Thrift 16 | from thrift.transport import TSocket 17 | from thrift.transport import TTransport 18 | from thrift.protocol import TBinaryProtocol 19 | 20 | try: 21 | # Make socket 22 | transport = TSocket.TSocket('localhost', 9090) 23 | 24 | # Buffering is critical. Raw sockets are very slow 25 | transport = TTransport.TBufferedTransport(transport) 26 | 27 | # Wrap in a protocol 28 | protocol = TBinaryProtocol.TBinaryProtocol(transport) 29 | 30 | # Create a client to use the protocol encoder 31 | client = Gokeepalived.Client(protocol) 32 | 33 | # Connect! 34 | transport.open() 35 | data = {"Command":"GetInfo"} 36 | response = client.api_call(data) 37 | print(response) 38 | transport.close() 39 | 40 | except Thrift.TException, tx: 41 | print '%s' % (tx.message) 42 | -------------------------------------------------------------------------------- /api_clients/gokeepalived.thrift: -------------------------------------------------------------------------------- 1 | /** 2 | * The first thing to know about are types. The available types in Thrift are: 3 | * 4 | * bool Boolean, one byte 5 | * byte Signed byte 6 | * i16 Signed 16-bit integer 7 | * i32 Signed 32-bit integer 8 | * i64 Signed 64-bit integer 9 | * double 64-bit floating point value 10 | * string String 11 | * binary Blob (byte array) 12 | * map Map from one type to another 13 | * list Ordered list of one type 14 | * set Set of unique elements of one type 15 | * 16 | * Did you also notice that Thrift supports C style comments? 17 | */ 18 | 19 | namespace cpp gokeepalived 20 | 21 | service Gokeepalived { 22 | map api_call(1:map request) 23 | } 24 | 25 | -------------------------------------------------------------------------------- /api_clients/server.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | gen_py_path = '/tmp/' 4 | thrift_pylib = '/tmp/' 5 | if 'GEN_PY' in os.environ: 6 | gen_py_path = os.environ['GEN_PY'] 7 | if 'THRIFT_PYLIB' in os.environ: 8 | thrift_pylib = os.environ['THRIFT_PYLIB'] 9 | sys.path.append(gen_py_path) 10 | sys.path.append(thrift_pylib) 11 | 12 | 13 | #TODO: proper path to cfg_slb.py 14 | from cfg_slb import slbAPI 15 | 16 | from gokeepalived import Gokeepalived 17 | from gokeepalived.ttypes import * 18 | 19 | 20 | 21 | from thrift.transport import TSocket 22 | from thrift.transport import TTransport 23 | from thrift.protocol import TBinaryProtocol 24 | from thrift.server import TServer 25 | 26 | class GokeepalivedHandler: 27 | def __init__(self): 28 | # 1 - link to http api; 2 - master pwd 29 | self._api = slbAPI(sys.argv[1],sys.argv[2]) 30 | self.log = {} 31 | 32 | def api_call(self, request): 33 | response = self._api.ExecCmnd(request) 34 | return response["Data"] 35 | 36 | 37 | 38 | 39 | 40 | handler = GokeepalivedHandler() 41 | processor = Gokeepalived.Processor(handler) 42 | transport = TSocket.TServerSocket(port=9090) 43 | tfactory = TTransport.TBufferedTransportFactory() 44 | pfactory = TBinaryProtocol.TBinaryProtocolFactory() 45 | 46 | #server = TServer.TSimpleServer(processor, transport, tfactory, pfactory) 47 | 48 | # You could do one of these for a multithreaded server 49 | server = TServer.TThreadedServer(processor, transport, tfactory, pfactory) 50 | #server = TServer.TThreadPoolServer(processor, transport, tfactory, pfactory) 51 | 52 | print 'Starting the server...' 53 | server.serve() 54 | print 'done.' 55 | -------------------------------------------------------------------------------- /cfgparser/cfgparser.go: -------------------------------------------------------------------------------- 1 | package cfgparser 2 | 3 | import ( 4 | "bufio" 5 | "fmt" 6 | "go_keepalived/api" 7 | "go_keepalived/notifier" 8 | "go_keepalived/service" 9 | "os" 10 | "regexp" 11 | "strconv" 12 | "strings" 13 | ) 14 | 15 | //TODO: unittests 16 | func ReadCfg(cfgFile string) (*service.ServicesList, *notifier.NotifierConfig) { 17 | fd, err := os.Open(cfgFile) 18 | defer fd.Close() 19 | if err != nil { 20 | fmt.Println("cant open cfg file") 21 | os.Exit(-1) 22 | } 23 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 24 | v6re, _ := regexp.Compile(`^\[((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8}\]$`) 25 | numRe, _ := regexp.Compile(`^\d{1,}$`) 26 | sl := service.ServicesList{} 27 | nc := notifier.NotifierConfig{} 28 | adapterType := "ipvsadm" 29 | sl.Init() 30 | srvc := service.Service{} 31 | srvc.Init() 32 | rlSrv := service.RealServer{} 33 | API := api.GenericAPI{} 34 | /* 35 | counting "{"; so we will be able to see when old servicse defenition stops and new one starts 36 | */ 37 | sectionCntr := 0 38 | //Flags 39 | generalCfg := false 40 | serviceCfg := false 41 | notifierCfg := false 42 | apiCfg := false 43 | 44 | scanner := bufio.NewScanner(fd) 45 | /* main section of cfg parsing */ 46 | line := 0 47 | for scanner.Scan() { 48 | line++ 49 | fields := strings.Fields(scanner.Text()) 50 | if len(fields) == 0 { 51 | continue 52 | } 53 | if fields[0] == "service" { 54 | if len(fields) < 4 { 55 | fmt.Println("line: ", line) 56 | fmt.Println("error in parsing service's defenition\n must be service {") 57 | os.Exit(-1) 58 | } 59 | if sectionCntr != 0 { 60 | fmt.Println("line: ", line) 61 | fmt.Println("error in cfg file. it seems that you defined service inside other service") 62 | fmt.Println(fields) 63 | os.Exit(-1) 64 | } 65 | if len(v4re.FindString(fields[1])) == 0 && len(v6re.FindString(fields[1])) == 0 && 66 | len(numRe.FindString(fields[1])) == 0 { 67 | fmt.Println("line: ", line) 68 | fmt.Println("error in srvc address") 69 | fmt.Println(fields) 70 | os.Exit(-1) 71 | } 72 | if fields[3] != "{" { 73 | fmt.Println("line: ", line) 74 | fmt.Println("error in parsing service's defenition\n must be service {") 75 | os.Exit(-1) 76 | } 77 | serviceCfg = true 78 | sectionCntr++ 79 | srvc.VIP = fields[1] 80 | srvc.Port = fields[2] 81 | } else if fields[0] == "real" && serviceCfg == true { 82 | if len(fields) < 4 { 83 | fmt.Println("line: ", line) 84 | fmt.Println("error in parsing real's defenition\n must be real {") 85 | os.Exit(-1) 86 | } 87 | if sectionCntr != 1 { 88 | fmt.Println("line: ", line) 89 | fmt.Println("error in cfg file. it seems that you defined real inside other real or real outside of service") 90 | fmt.Println(fields) 91 | os.Exit(-1) 92 | } 93 | if len(v4re.FindString(fields[1])) == 0 && len(v6re.FindString(fields[1])) == 0 { 94 | fmt.Println("line: ", line) 95 | fmt.Println("error in real address") 96 | fmt.Println(fields) 97 | os.Exit(-1) 98 | } 99 | if fields[3] != "{" { 100 | fmt.Println("line: ", line) 101 | fmt.Println("error in parsing real's defenition\n must be real {") 102 | os.Exit(-1) 103 | } 104 | sectionCntr++ 105 | rlSrv.RIP = fields[1] 106 | rlSrv.Port = fields[2] 107 | } else if fields[0] == "general" { 108 | if sectionCntr != 0 || (len(fields) != 2 && fields[1] != "{") { 109 | fmt.Println("line: ", line) 110 | fmt.Println("error in parsing genaral's cfg defenition\n must be general {") 111 | os.Exit(-1) 112 | 113 | } 114 | generalCfg = true 115 | sectionCntr++ 116 | } else if fields[0] == "notifier" { 117 | if sectionCntr != 1 || (len(fields) != 3 && fields[2] != "{") || generalCfg != true { 118 | fmt.Println("line: ", line) 119 | fmt.Println("error in parsing notifiers's cfg defenition\n must be notifier {") 120 | os.Exit(-1) 121 | } 122 | nc.Type = fields[1] 123 | notifierCfg = true 124 | sectionCntr++ 125 | } else if fields[0] == "adapter" { 126 | if sectionCntr != 1 || len(fields) != 2 || generalCfg != true { 127 | fmt.Println("line: ", line) 128 | fmt.Println("error in parsing adapter's cfg defenition\n must be adapter ") 129 | os.Exit(-1) 130 | } 131 | adapterType = fields[1] 132 | } else if fields[0] == "api" { 133 | if sectionCntr != 1 || (len(fields) != 2 && fields[1] != "{") || generalCfg != true { 134 | fmt.Println("line: ", line) 135 | fmt.Println("error in parsing api's cfg defenition\n must be api {") 136 | os.Exit(-1) 137 | } 138 | apiCfg = true 139 | API.Enable = true 140 | sectionCntr++ 141 | } else if fields[0] == "testing" { 142 | sl.Testing = true 143 | } else if fields[0] == "}" { 144 | if sectionCntr == 2 { 145 | if serviceCfg == true { 146 | srvc.AddReal(rlSrv) 147 | rlSrv = service.RealServer{} 148 | } 149 | if notifierCfg == true { 150 | notifierCfg = false 151 | } 152 | if apiCfg == true { 153 | apiCfg = false 154 | } 155 | sectionCntr-- 156 | } else if sectionCntr == 1 { 157 | if serviceCfg == true { 158 | if service.IsServiceValid(srvc) { 159 | sl.Add(srvc) 160 | 161 | } else { 162 | fmt.Println("unvalid service defenition, fields VIP,PORT and Proto are mandatory") 163 | } 164 | srvc = service.Service{} 165 | srvc.Init() 166 | serviceCfg = false 167 | } 168 | if generalCfg == true { 169 | generalCfg = false 170 | } 171 | sectionCntr-- 172 | } else { 173 | fmt.Println("line: ", line) 174 | fmt.Println("error in section's count (not enough/too many {})") 175 | os.Exit(-1) 176 | } 177 | } else if len(fields) > 1 { 178 | if serviceCfg == true { 179 | if fields[0] == "proto" && sectionCntr == 1 { 180 | srvc.Proto = fields[1] 181 | } else if fields[0] == "scheduler" && sectionCntr == 1 { 182 | srvc.Scheduler = fields[1] 183 | } else if fields[0] == "meta" && sectionCntr == 1 { 184 | srvc.Meta = strings.Join(fields[1:], " ") 185 | } else if fields[0] == "quorum" && sectionCntr == 1 { 186 | qnum, err := strconv.Atoi(fields[1]) 187 | if err != nil { 188 | fmt.Println("line: ", line) 189 | fmt.Println("cant convert quorum to int") 190 | os.Exit(-1) 191 | } 192 | srvc.Quorum = qnum 193 | } else if fields[0] == "timeout" && sectionCntr == 1 { 194 | timeout, err := strconv.Atoi(fields[1]) 195 | if err != nil { 196 | fmt.Println("line: ", line) 197 | fmt.Println("cant convert timeout to int") 198 | os.Exit(-1) 199 | } 200 | srvc.Timeout = timeout 201 | } else if fields[0] == "hysteresis" && sectionCntr == 1 { 202 | hnum, err := strconv.Atoi(fields[1]) 203 | if err != nil { 204 | fmt.Println("line: ", line) 205 | fmt.Println("cant convert hysteresis to int") 206 | os.Exit(-1) 207 | } 208 | srvc.Hysteresis = hnum 209 | } else if fields[0] == "check" && sectionCntr == 2 { 210 | check := strings.Join(fields[1:], " ") 211 | rlSrv.Check = check 212 | } else if fields[0] == "meta" && sectionCntr == 2 { 213 | meta := strings.Join(fields[1:], " ") 214 | rlSrv.Meta = meta 215 | } else if fields[0] == "weight" && sectionCntr == 2 { 216 | weight := strings.Join(fields[1:], " ") 217 | rlSrv.Weight = weight 218 | } 219 | 220 | } else if notifierCfg == true { 221 | if fields[0] == "ASN" { 222 | val, err := strconv.ParseUint(fields[1], 10, 32) 223 | if err != nil { 224 | fmt.Println("line: ", line) 225 | fmt.Println("cant convert asn to uint32") 226 | os.Exit(-1) 227 | } 228 | nc.ASN = uint32(val) 229 | } else if fields[0] == "listen" { 230 | if fields[1] == "enable" { 231 | nc.ListenLocal = true 232 | } 233 | } else if fields[0] == "neighbour" { 234 | nc.NeighboursList = append(nc.NeighboursList, fields[1]) 235 | } 236 | } else if apiCfg == true { 237 | //TODO: custom ports etc 238 | if fields[0] == "http" { 239 | API.HttpApi = true 240 | } 241 | if fields[0] == "password" { 242 | API.MasterPwd = fields[1] 243 | } 244 | 245 | } 246 | } 247 | } 248 | sl.AddNotifier(nc) 249 | sl.StartAdapter(adapterType) 250 | if API.Enable { 251 | go api.InitAPI(API, sl.ToServiceList, sl.FromServiceList) 252 | } 253 | return &sl, &nc 254 | } 255 | -------------------------------------------------------------------------------- /healthchecks/http_check.go: -------------------------------------------------------------------------------- 1 | package healthchecks 2 | 3 | import ( 4 | "net/http" 5 | "time" 6 | ) 7 | 8 | func HTTPCheck(toCheck, fromCheck chan int, checkLine []string, timeOut int) { 9 | if len(checkLine) < 1 { 10 | fromCheck <- -1 11 | return 12 | } 13 | client := &http.Client{Timeout: time.Second * time.Duration(timeOut)} 14 | loop := 1 15 | for loop == 1 { 16 | select { 17 | case <-toCheck: 18 | loop = 0 19 | case <-time.After(time.Second * time.Duration(timeOut)): 20 | resp, err := client.Get(checkLine[0]) 21 | if err != nil { 22 | select { 23 | case fromCheck <- 0: 24 | case <-toCheck: 25 | loop = 0 26 | } 27 | } else if resp.StatusCode != http.StatusOK { 28 | select { 29 | case fromCheck <- 0: 30 | resp.Body.Close() 31 | case <-toCheck: 32 | loop = 0 33 | } 34 | } else { 35 | select { 36 | case fromCheck <- 1: 37 | resp.Body.Close() 38 | case <-toCheck: 39 | loop = 0 40 | } 41 | } 42 | } 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /healthchecks/tcp_check.go: -------------------------------------------------------------------------------- 1 | package healthchecks 2 | 3 | import ( 4 | "net" 5 | "strings" 6 | "time" 7 | ) 8 | 9 | func TCPCheck(toCheck, fromCheck chan int, checkLine []string, timeOut int) { 10 | Addr := strings.Join(checkLine, ":") 11 | loop := 1 12 | for loop == 1 { 13 | select { 14 | case <-toCheck: 15 | loop = 0 16 | case <-time.After(time.Second * time.Duration(timeOut)): 17 | tcpConn, err := net.DialTimeout("tcp", Addr, 18 | time.Second*time.Duration(timeOut)) 19 | if err != nil { 20 | select { 21 | case fromCheck <- 0: 22 | case <-toCheck: 23 | loop = 0 24 | } 25 | } else { 26 | select { 27 | case fromCheck <- 1: 28 | tcpConn.Close() 29 | case <-toCheck: 30 | loop = 0 31 | } 32 | } 33 | } 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /healthchecks/zk_check.go: -------------------------------------------------------------------------------- 1 | package healthchecks 2 | 3 | import ( 4 | "time" 5 | 6 | "github.com/samuel/go-zookeeper/zk" 7 | ) 8 | 9 | func ZKCheck(toCheck, fromCheck chan int, checkLine []string, timeOut int) { 10 | zkAddr := checkLine[0] 11 | zkPath := checkLine[1] 12 | conn, connEvent, err := zk.Connect([]string{zkAddr}, time.Duration(timeOut)*time.Second) 13 | if err != nil { 14 | CONNECT_LOOP: 15 | for { 16 | conn, connEvent, err = zk.Connect([]string{zkAddr}, time.Duration(timeOut)*time.Second) 17 | if err != nil { 18 | select { 19 | case <-time.After(time.Duration(timeOut) * time.Second): 20 | case <-toCheck: 21 | return 22 | } 23 | } else { 24 | break CONNECT_LOOP 25 | } 26 | } 27 | } 28 | CHECK_LOOP: 29 | for { 30 | exists, _, event, err := conn.ExistsW(zkPath) 31 | if err != nil { 32 | select { 33 | case fromCheck <- 0: 34 | case <-toCheck: 35 | break CHECK_LOOP 36 | } 37 | select { 38 | case <-time.After(time.Second * 10): 39 | continue CHECK_LOOP 40 | case <-toCheck: 41 | break CHECK_LOOP 42 | } 43 | } 44 | if exists { 45 | select { 46 | case fromCheck <- 1: 47 | case <-toCheck: 48 | break CHECK_LOOP 49 | } 50 | } else { 51 | select { 52 | case fromCheck <- 0: 53 | case <-toCheck: 54 | break CHECK_LOOP 55 | } 56 | } 57 | select { 58 | case <-connEvent: 59 | case <-toCheck: 60 | break CHECK_LOOP 61 | case <-event: 62 | } 63 | } 64 | } 65 | -------------------------------------------------------------------------------- /main/go_keepalived.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "go_keepalived/cfgparser" 6 | "os" 7 | "sync" 8 | ) 9 | 10 | func main() { 11 | if len(os.Args) < 2 { 12 | fmt.Println("usage: go_keepalived ") 13 | os.Exit(-1) 14 | } 15 | slList, notifCfg := cfgparser.ReadCfg(os.Args[1]) 16 | fmt.Println(*slList) 17 | fmt.Println(*notifCfg) 18 | slList.Start() 19 | wg := sync.WaitGroup{} 20 | (&wg).Add(1) 21 | (&wg).Wait() 22 | } 23 | -------------------------------------------------------------------------------- /main/gokeepalived_test.cfg: -------------------------------------------------------------------------------- 1 | general { 2 | notifier bgp { 3 | ASN 65111 4 | neighbour 192.168.140.11 5 | neighbour fc0::1 6 | } 7 | api { 8 | http enable 9 | password 123 10 | } 11 | adapter netlink 12 | } 13 | 14 | service 192.168.1.1 80 { 15 | proto tcp 16 | quorum 1 17 | hysteresis 1 18 | scheduler wlc 19 | real 172.16.1.1 8082 { 20 | check zk 192.168.132.10 /172.16.1.1 21 | weight 10 22 | } 23 | real 172.16.1.1 8080 { 24 | check http http://example.com/ 25 | weight 10 26 | } 27 | 28 | real 172.16.1.1 8081 { 29 | check tcp 30 | weight 1 31 | meta notyetimplemented 32 | } 33 | } 34 | 35 | 36 | service [fc00:2::1] 80 { 37 | proto tcp 38 | quorum 1 39 | hysteresis 1 40 | 41 | real [fc00:1::1] 8081 { 42 | check tcp 43 | meta notyetimplemented 44 | } 45 | } 46 | 47 | 48 | 49 | service 192.168.1.1 81 { 50 | proto udp 51 | quorum 2 52 | hysteresis 2 53 | real 172.16.1.1 8082 { 54 | check https https://example.com/ 55 | } 56 | 57 | real 172.16.1.1 8083 { 58 | check tcp 59 | } 60 | 61 | 62 | } 63 | -------------------------------------------------------------------------------- /notifier/bgp_notifier.go: -------------------------------------------------------------------------------- 1 | package notifier 2 | 3 | import ( 4 | "regexp" 5 | "strings" 6 | 7 | "github.com/tehnerd/bgp2go" 8 | ) 9 | 10 | func BGPNotifier(msgChan chan NotifierMsg, responseChan chan NotifierMsg, 11 | notifierConfig NotifierConfig) { 12 | v4re, _ := regexp.Compile(`^(\d{1,3}\.){3}\d{1,3}$`) 13 | v6re, _ := regexp.Compile(`^\[(((\d|a|b|c|d|e|f|A|B|C|D|E|F){0,4}\:?){1,8})\]$`) 14 | bgpMainContext := bgp2go.BGPContext{ASN: notifierConfig.ASN, ListenLocal: notifierConfig.ListenLocal} 15 | toBGPProcess := make(chan bgp2go.BGPProcessMsg) 16 | fromBGPProcess := make(chan bgp2go.BGPProcessMsg) 17 | go bgp2go.StartBGPProcess(toBGPProcess, fromBGPProcess, bgpMainContext) 18 | /* 19 | table of service's refcount. we could has two different services on the same vip (for example tcp/80; tcp/443) 20 | //TODO: overflow check 21 | */ 22 | serviceTable := make(map[string]uint32) 23 | silencedServices := make(map[string]bool) 24 | stopAllNotifications := false 25 | for { 26 | msg := <-msgChan 27 | switch msg.Type { 28 | case "AdvertiseService": 29 | if _, exists := serviceTable[msg.Data]; exists { 30 | serviceTable[msg.Data]++ 31 | //more that one service has same vip; we have already advertised it 32 | if serviceTable[msg.Data] > 1 { 33 | continue 34 | } 35 | } else { 36 | serviceTable[msg.Data] = 1 37 | } 38 | if _, exists := silencedServices[msg.Data]; exists || stopAllNotifications { 39 | continue 40 | } 41 | if v4re.MatchString(msg.Data) { 42 | Route := strings.Join([]string{msg.Data, "/32"}, "") 43 | toBGPProcess <- bgp2go.BGPProcessMsg{ 44 | Cmnd: "AddV4Route", 45 | Data: Route} 46 | } else { 47 | //TODO: mb use regexp findstring instead 48 | prefix := v6re.FindStringSubmatch(msg.Data) 49 | if len(prefix) >= 2 { 50 | Route := strings.Join([]string{prefix[1], "/128"}, "") 51 | toBGPProcess <- bgp2go.BGPProcessMsg{ 52 | Cmnd: "AddV6Route", 53 | Data: Route} 54 | } 55 | } 56 | case "WithdrawService": 57 | if _, exists := serviceTable[msg.Data]; exists { 58 | if serviceTable[msg.Data] == 0 { 59 | continue 60 | } 61 | serviceTable[msg.Data]-- 62 | //there are still alive services who uses this vip 63 | if serviceTable[msg.Data] > 0 { 64 | continue 65 | } 66 | } else { 67 | continue 68 | } 69 | if _, exists := silencedServices[msg.Data]; exists || stopAllNotifications { 70 | continue 71 | } 72 | if v4re.MatchString(msg.Data) { 73 | Route := strings.Join([]string{msg.Data, "/32"}, "") 74 | toBGPProcess <- bgp2go.BGPProcessMsg{ 75 | Cmnd: "WithdrawV4Route", 76 | Data: Route} 77 | } else { 78 | prefix := v6re.FindStringSubmatch(msg.Data) 79 | if len(prefix) >= 2 { 80 | 81 | Route := strings.Join([]string{prefix[1], "/128"}, "") 82 | toBGPProcess <- bgp2go.BGPProcessMsg{ 83 | Cmnd: "WithdrawV6Route", 84 | Data: Route} 85 | } 86 | } 87 | case "StopNotification": 88 | if _, exists := silencedServices[msg.Data]; exists || stopAllNotifications { 89 | continue 90 | } 91 | silencedServices[msg.Data] = true 92 | if _, exists := serviceTable[msg.Data]; !exists { 93 | continue 94 | } 95 | if v4re.MatchString(msg.Data) { 96 | Route := strings.Join([]string{msg.Data, "/32"}, "") 97 | toBGPProcess <- bgp2go.BGPProcessMsg{ 98 | Cmnd: "WithdrawV4Route", 99 | Data: Route} 100 | } else { 101 | prefix := v6re.FindStringSubmatch(msg.Data) 102 | if len(prefix) >= 2 { 103 | 104 | Route := strings.Join([]string{prefix[1], "/128"}, "") 105 | toBGPProcess <- bgp2go.BGPProcessMsg{ 106 | Cmnd: "WithdrawV6Route", 107 | Data: Route} 108 | } 109 | } 110 | case "StartNotification": 111 | if _, exists := silencedServices[msg.Data]; !exists || stopAllNotifications { 112 | continue 113 | } 114 | delete(silencedServices, msg.Data) 115 | if cntr, exists := serviceTable[msg.Data]; !exists { 116 | continue 117 | } else if cntr == 0 { 118 | continue 119 | } 120 | if v4re.MatchString(msg.Data) { 121 | Route := strings.Join([]string{msg.Data, "/32"}, "") 122 | toBGPProcess <- bgp2go.BGPProcessMsg{ 123 | Cmnd: "AddV4Route", 124 | Data: Route} 125 | } else { 126 | //TODO: mb use regexp findstring instead 127 | prefix := v6re.FindStringSubmatch(msg.Data) 128 | if len(prefix) >= 2 { 129 | Route := strings.Join([]string{prefix[1], "/128"}, "") 130 | toBGPProcess <- bgp2go.BGPProcessMsg{ 131 | Cmnd: "AddV6Route", 132 | Data: Route} 133 | } 134 | } 135 | case "StopAllNotification": 136 | stopAllNotifications = true 137 | for service, cntr := range serviceTable { 138 | if _, exists := silencedServices[service]; exists { 139 | delete(silencedServices, service) 140 | continue 141 | } 142 | if cntr != 0 { 143 | if v4re.MatchString(service) { 144 | Route := strings.Join([]string{service, "/32"}, "") 145 | toBGPProcess <- bgp2go.BGPProcessMsg{ 146 | Cmnd: "WithdrawV4Route", 147 | Data: Route} 148 | } else { 149 | prefix := v6re.FindStringSubmatch(service) 150 | if len(prefix) >= 2 { 151 | Route := strings.Join([]string{prefix[1], "/128"}, "") 152 | toBGPProcess <- bgp2go.BGPProcessMsg{ 153 | Cmnd: "WithdrawV6Route", 154 | Data: Route} 155 | } 156 | } 157 | 158 | } 159 | } 160 | case "StartAllNotification": 161 | stopAllNotifications = false 162 | for service, cntr := range serviceTable { 163 | if cntr > 0 { 164 | if v4re.MatchString(service) { 165 | Route := strings.Join([]string{service, "/32"}, "") 166 | toBGPProcess <- bgp2go.BGPProcessMsg{ 167 | Cmnd: "AddV4Route", 168 | Data: Route} 169 | } else { 170 | prefix := v6re.FindStringSubmatch(service) 171 | if len(prefix) >= 2 { 172 | Route := strings.Join([]string{prefix[1], "/128"}, "") 173 | toBGPProcess <- bgp2go.BGPProcessMsg{ 174 | Cmnd: "AddV6Route", 175 | Data: Route} 176 | } 177 | } 178 | 179 | } 180 | } 181 | case "AddPeer": 182 | if v4re.MatchString(msg.Data) { 183 | toBGPProcess <- bgp2go.BGPProcessMsg{ 184 | Cmnd: "AddNeighbour", 185 | Data: msg.Data} 186 | } else { 187 | //TODO: sanity check for v6 address 188 | //for ResolveTCPAddr to work ipv6 must be in format [] 189 | data := strings.Join([]string{"[", msg.Data, "]"}, "") 190 | data = strings.Join([]string{data, "inet6"}, " ") 191 | toBGPProcess <- bgp2go.BGPProcessMsg{ 192 | Cmnd: "AddNeighbour", 193 | Data: data} 194 | } 195 | case "RemovePeer": 196 | if v4re.MatchString(msg.Data) { 197 | toBGPProcess <- bgp2go.BGPProcessMsg{ 198 | Cmnd: "RemoveNeighbour", 199 | Data: msg.Data} 200 | } else { 201 | //TODO: sanity check for v6 address 202 | //for ResolveTCPAddr to work ipv6 must be in format [] 203 | data := strings.Join([]string{"[", msg.Data, "]"}, "") 204 | data = strings.Join([]string{data, "inet6"}, " ") 205 | toBGPProcess <- bgp2go.BGPProcessMsg{ 206 | Cmnd: "RemoveNeighbour", 207 | Data: data} 208 | } 209 | 210 | } 211 | 212 | } 213 | } 214 | -------------------------------------------------------------------------------- /notifier/notifier.go: -------------------------------------------------------------------------------- 1 | package notifier 2 | 3 | import "fmt" 4 | 5 | //TODO: logging 6 | 7 | /* 8 | Msg's struct to control notifer 9 | */ 10 | type NotifierMsg struct { 11 | 12 | /* 13 | Message Type. It copies most of the messages types from adapter Could be: 14 | "AddPeer", - add external point of notification (aka bgp peer in our cases; we dont have 15 | any other way to notify so far; mb going to add "exec external script" notifier) 16 | "RemovePeer", - remove external point of notificatio 17 | */ 18 | Type string 19 | 20 | /* 21 | data could be vip (v4/v6) address etc 22 | */ 23 | Data string 24 | } 25 | 26 | /* 27 | General notifier configuration 28 | */ 29 | type NotifierConfig struct { 30 | Type string 31 | //if notifier is BGP; bgp related configuration 32 | ASN uint32 33 | ListenLocal bool 34 | NeighboursList []string 35 | } 36 | 37 | func Notifier(msgChan chan NotifierMsg, responseChan chan NotifierMsg, 38 | notifierConfig NotifierConfig) { 39 | //TODO: ExecExcternal shell script; should be easy to implement 40 | switch notifierConfig.Type { 41 | case "bgp": 42 | go BGPNotifier(msgChan, responseChan, notifierConfig) 43 | case "dummy": 44 | go DummyNotifier(msgChan) 45 | default: 46 | go SilentNotifier(msgChan) 47 | } 48 | } 49 | 50 | func DummyNotifier(msgChan chan NotifierMsg) { 51 | for { 52 | msg := <-msgChan 53 | fmt.Printf("Dummy Notifier: %#v\n", msg) 54 | } 55 | } 56 | 57 | func SilentNotifier(msgChan chan NotifierMsg) { 58 | for { 59 | <-msgChan 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /service/msgs2service_processing.go: -------------------------------------------------------------------------------- 1 | package service 2 | 3 | import ( 4 | "go_keepalived/notifier" 5 | "strconv" 6 | "strings" 7 | ) 8 | 9 | func DispatchSLMsgs(sl *ServicesList) { 10 | for { 11 | select { 12 | case msg := <-sl.ToServiceList: 13 | responseStruct := make(map[string]string) 14 | switch msg.Cmnd { 15 | case "GetInfo": 16 | //TODO: info from bgp notifier about peers, their state etc 17 | for _, service := range sl.List { 18 | service.ToService <- ServiceMsg{Cmnd: "GetInfo"} 19 | serviceResponse := <-service.FromService 20 | vip := createVipName(service) 21 | responseStruct[vip] = serviceResponse.Data 22 | } 23 | case "AddService": 24 | srvc := serviceFromDefinition(msg.DataMap) 25 | err := sl.Add(srvc) 26 | if err != nil { 27 | break 28 | } 29 | srvc = sl.List[len(sl.List)-1] 30 | go srvc.StartService() 31 | vip := createVipName(srvc) 32 | responseStruct["result"] = "true" 33 | responseStruct["info"] = "Successfully added new service:" + vip 34 | case "RemoveService": 35 | srvc := serviceFromDefinition(msg.DataMap) 36 | err := sl.Remove(srvc) 37 | if err != nil { 38 | break 39 | } 40 | vip := createVipName(srvc) 41 | responseStruct["result"] = "true" 42 | responseStruct["info"] = "Successfully removed service:" + vip 43 | case "ChangeService": 44 | srvc := serviceFromDefinition(msg.DataMap) 45 | i := sl.FindService(&srvc) 46 | if i == -1 { 47 | break 48 | } 49 | sl.List[i].ToService <- ServiceMsg{Cmnd: "ChangeService", DataMap: msg.DataMap} 50 | vip := createVipName(srvc) 51 | responseStruct["result"] = "true" 52 | responseStruct["info"] = "Successfully changed service:" + vip 53 | 54 | case "AddReal": 55 | srvcName := serviceFromDefinition(msg.DataMap) 56 | rlSrv := realSrvFromDefinition(msg.DataMap) 57 | i := sl.FindService(&srvcName) 58 | if i == -1 { 59 | break 60 | } 61 | sl.List[i].ToService <- ServiceMsg{Cmnd: "AddReal", DataMap: msg.DataMap} 62 | result := <-sl.List[i].FromService 63 | if result.Data == "RealAdded" { 64 | vip := createVipName(sl.List[i]) 65 | rip := createRlSrvName(rlSrv) 66 | responseStruct["result"] = "true" 67 | responseStruct["info"] = "Successfully added new real " + rip + " for service:" + vip 68 | } else { 69 | responseStruct["result"] = "false" 70 | } 71 | case "RemoveReal": 72 | srvcName := serviceFromDefinition(msg.DataMap) 73 | rlSrv := realSrvFromDefinition(msg.DataMap) 74 | i := sl.FindService(&srvcName) 75 | if i == -1 { 76 | break 77 | } 78 | sl.List[i].ToService <- ServiceMsg{Cmnd: "RemoveReal", 79 | Data: strings.Join([]string{rlSrv.RIP, rlSrv.Port, rlSrv.Meta}, " ")} 80 | result := <-sl.List[i].FromService 81 | if result.Data == "RealRemoved" { 82 | vip := createVipName(sl.List[i]) 83 | rip := createRlSrvName(rlSrv) 84 | responseStruct["result"] = "true" 85 | responseStruct["info"] = "Successfully removed real " + rip + " for service:" + vip 86 | } else { 87 | responseStruct["result"] = "false" 88 | } 89 | case "ChangeReal": 90 | srvcName := serviceFromDefinition(msg.DataMap) 91 | rlSrv := realSrvFromDefinition(msg.DataMap) 92 | i := sl.FindService(&srvcName) 93 | if i == -1 { 94 | break 95 | } 96 | sl.List[i].ToService <- ServiceMsg{Cmnd: "ChangeReal", 97 | Data: strings.Join([]string{rlSrv.RIP, rlSrv.Port, rlSrv.Meta}, " "), 98 | DataMap: msg.DataMap} 99 | vip := createVipName(sl.List[i]) 100 | rip := createRlSrvName(rlSrv) 101 | responseStruct["result"] = "true" 102 | responseStruct["info"] = "Successfully changed real " + rip + " for service:" + vip 103 | case "AddPeer": 104 | sl.ToNotifier <- notifier.NotifierMsg{Type: "AddPeer", Data: (*msg.DataMap)["Address"]} 105 | //TODO: read sl.FromNotifier for actual result 106 | responseStruct["result"] = "true" 107 | responseStruct["info"] = "Successfully added peer: " + (*msg.DataMap)["Address"] 108 | case "RemovePeer": 109 | sl.ToNotifier <- notifier.NotifierMsg{Type: "RemovePeer", Data: (*msg.DataMap)["Address"]} 110 | //TODO: read sl.FromNotifier for actual result 111 | responseStruct["result"] = "true" 112 | responseStruct["info"] = "Successfully removed peer: " + (*msg.DataMap)["Address"] 113 | case "StopNotification": 114 | sl.ToNotifier <- notifier.NotifierMsg{Type: "StopNotification", Data: (*msg.DataMap)["VIP"]} 115 | //TODO: read sl.FromNotifier for actual result 116 | responseStruct["result"] = "true" 117 | responseStruct["info"] = "Successfully stopped notification for: " + (*msg.DataMap)["VIP"] 118 | case "StartNotification": 119 | sl.ToNotifier <- notifier.NotifierMsg{Type: "StartNotification", Data: (*msg.DataMap)["VIP"]} 120 | //TODO: read sl.FromNotifier for actual result 121 | responseStruct["result"] = "true" 122 | responseStruct["info"] = "Successfully started notification for: " + (*msg.DataMap)["VIP"] 123 | case "StopAllNotification": 124 | sl.ToNotifier <- notifier.NotifierMsg{Type: "StopAllNotification"} 125 | //TODO: read sl.FromNotifier for actual result 126 | responseStruct["result"] = "true" 127 | responseStruct["info"] = "Successfully stopped all notification" 128 | case "StartAllNotification": 129 | sl.ToNotifier <- notifier.NotifierMsg{Type: "StartAllNotification"} 130 | //TODO: read sl.FromNotifier for actual result 131 | responseStruct["result"] = "true" 132 | responseStruct["info"] = "Successfully started all notification" 133 | } 134 | sl.FromServiceList <- ServiceMsg{DataMap: (&responseStruct)} 135 | } 136 | } 137 | } 138 | 139 | //TODO: scheduler, lbmethod (nat or tun) , timeout, etc 140 | func serviceFromDefinition(definition *map[string]string) Service { 141 | srvc := Service{} 142 | srvc.Init() 143 | srvc.VIP = (*definition)["VIP"] 144 | srvc.Proto = (*definition)["Proto"] 145 | srvc.Port = (*definition)["Port"] 146 | if quorum, exists := (*definition)["Quorum"]; exists { 147 | q, err := strconv.Atoi(quorum) 148 | if err != nil { 149 | srvc.Quorum = 1 150 | } else { 151 | srvc.Quorum = q 152 | } 153 | 154 | } else { 155 | srvc.Quorum = 1 156 | } 157 | return srvc 158 | } 159 | 160 | //TODO: meta. weight etc 161 | func realSrvFromDefinition(definition *map[string]string) RealServer { 162 | rlSrv := RealServer{} 163 | rlSrv.RIP = (*definition)["RIP"] 164 | rlSrv.Port = (*definition)["RealPort"] 165 | rlSrv.Check = (*definition)["Check"] 166 | return rlSrv 167 | } 168 | 169 | func createVipName(srvc Service) string { 170 | return strings.Join([]string{srvc.VIP, srvc.Proto, srvc.Port}, ":") 171 | } 172 | 173 | func createRlSrvName(rlSrv RealServer) string { 174 | return strings.Join([]string{rlSrv.RIP, rlSrv.Proto, rlSrv.Port}, ":") 175 | } 176 | -------------------------------------------------------------------------------- /service/service.go: -------------------------------------------------------------------------------- 1 | package service 2 | 3 | import ( 4 | "fmt" 5 | "go_keepalived/adapter" 6 | "go_keepalived/healthchecks" 7 | "go_keepalived/notifier" 8 | "log/syslog" 9 | "strconv" 10 | "strings" 11 | ) 12 | 13 | type RealServer struct { 14 | RIP string // real ip address 15 | Port string 16 | Check string // type of healthcheck 17 | Meta string // meta info; for example: if need to be tunneled 18 | Proto string // inhereted from service 19 | State bool 20 | Timeout int 21 | Weight string 22 | ToService chan ServiceMsg // inhereted from service; chan to communicate events to service container 23 | ToReal chan ServiceMsg 24 | } 25 | 26 | type Service struct { 27 | VIP string //Virtual IP address 28 | Proto string 29 | Port string 30 | Scheduler string 31 | Meta string 32 | State bool 33 | AliveReals int 34 | Quorum int 35 | Hysteresis int 36 | Timeout int 37 | ToService chan ServiceMsg 38 | FromService chan ServiceMsg 39 | FromReal chan ServiceMsg 40 | ToAdapter chan adapter.AdapterMsg 41 | logWriter *syslog.Writer 42 | Reals []RealServer 43 | } 44 | 45 | /* 46 | service container <-> realServer communication (for example "kill realServer") 47 | or realServer failed healthcheck etc 48 | Type of cmnds: 49 | "Alive" (RS->Service) if healtcheck was successful and realServer is working 50 | Data in this case would be RIP+Port+Proto in case of lack of metainfo 51 | and RIP+Meta if meta exists 52 | "Dead" (RS->Service) if healthcheck was failed and realServer is not working 53 | Data will be the same as above 54 | "RSFatalError" (RS->Service) if there was a fatal error during setup of real server 55 | (for example we wasnt able to resolve real's ip+port etc) 56 | "Shutdown" (Service->RS) if we want to remove realServer from service context 57 | */ 58 | type ServiceMsg struct { 59 | Cmnd string 60 | Data string 61 | DataMap *map[string]string 62 | } 63 | 64 | type ServicesList struct { 65 | List []Service 66 | ToAdapter chan adapter.AdapterMsg 67 | FromAdapter chan adapter.AdapterMsg 68 | ToNotifier chan notifier.NotifierMsg 69 | FromNotifier chan notifier.NotifierMsg 70 | ToServiceList chan ServiceMsg 71 | FromServiceList chan ServiceMsg 72 | Testing bool 73 | logWriter *syslog.Writer 74 | } 75 | 76 | /* 77 | ServicesList is a container for all services, which must be monitored by 78 | keepalived. 79 | */ 80 | 81 | func (sl *ServicesList) Init() { 82 | writer, err := syslog.New(syslog.LOG_INFO, "go_keepalived:") 83 | if err != nil { 84 | panic("cant connect to syslog") 85 | } 86 | sl.ToAdapter = make(chan adapter.AdapterMsg) 87 | sl.FromAdapter = make(chan adapter.AdapterMsg) 88 | sl.ToNotifier = make(chan notifier.NotifierMsg) 89 | sl.FromNotifier = make(chan notifier.NotifierMsg) 90 | sl.ToServiceList = make(chan ServiceMsg) 91 | sl.FromServiceList = make(chan ServiceMsg) 92 | sl.logWriter = writer 93 | } 94 | 95 | func (sl *ServicesList) StartAdapter(adapterType string) { 96 | go adapter.StartAdapter(sl.ToAdapter, sl.FromAdapter, sl.ToNotifier, adapterType) 97 | } 98 | 99 | func (sl *ServicesList) Add(srvc Service) error { 100 | for cntr := 0; cntr < len(sl.List); cntr++ { 101 | if sl.List[cntr].isEqual(&srvc) { 102 | logMsg := strings.Join([]string{"service already exists", srvc.VIP, " ", 103 | srvc.Proto, ":", srvc.Port}, " ") 104 | sl.logWriter.Write([]byte(logMsg)) 105 | return fmt.Errorf("service already exists") 106 | } 107 | } 108 | srvc.logWriter = sl.logWriter 109 | srvc.ToAdapter = sl.ToAdapter 110 | //TODO(tehnerd): add check if in list of supported schedulers 111 | if srvc.Scheduler == "" { 112 | srvc.Scheduler = "wrr" 113 | } 114 | sl.List = append(sl.List, srvc) 115 | logMsg := strings.Join([]string{"added new service", srvc.VIP, " ", 116 | srvc.Proto, ":", srvc.Port}, " ") 117 | sl.logWriter.Write([]byte(logMsg)) 118 | return nil 119 | } 120 | 121 | func (sl *ServicesList) AddNotifier(notifierCfg notifier.NotifierConfig) { 122 | go notifier.Notifier(sl.ToNotifier, sl.FromNotifier, notifierCfg) 123 | for _, neighbour := range notifierCfg.NeighboursList { 124 | sl.ToNotifier <- notifier.NotifierMsg{Type: "AddPeer", Data: neighbour} 125 | } 126 | } 127 | 128 | func (srvc *Service) isEqual(otherSrvc *Service) bool { 129 | if srvc.VIP == otherSrvc.VIP && srvc.Proto == otherSrvc.Proto && 130 | srvc.Port == otherSrvc.Port { 131 | return true 132 | } else { 133 | return false 134 | } 135 | } 136 | 137 | func (sl *ServicesList) FindService(srvc *Service) int { 138 | for num, localSrvc := range sl.List { 139 | if localSrvc.isEqual(srvc) { 140 | return num 141 | } 142 | } 143 | return -1 144 | } 145 | 146 | func (sl *ServicesList) Remove(srvc Service) error { 147 | for cntr := 0; cntr < len(sl.List); cntr++ { 148 | if sl.List[cntr].isEqual(&srvc) { 149 | removedSrvc := sl.List[cntr] 150 | if cntr != len(sl.List)-1 { 151 | sl.List = append(sl.List[:cntr], sl.List[cntr+1:]...) 152 | } else { 153 | sl.List = sl.List[:cntr] 154 | } 155 | removedSrvc.ToService <- ServiceMsg{Cmnd: "RemoveService"} 156 | logMsg := strings.Join([]string{"removed service", srvc.VIP, " ", 157 | srvc.Proto, ":", srvc.Port, " from services list"}, " ") 158 | sl.logWriter.Write([]byte(logMsg)) 159 | return nil 160 | } 161 | } 162 | return fmt.Errorf("service doesnt exists\n") 163 | } 164 | 165 | func (sl *ServicesList) Start() { 166 | if sl.Testing { 167 | sl.ToAdapter <- adapter.AdapterMsg{Type: "StartTesting"} 168 | } 169 | for cntr := 0; cntr < len(sl.List); cntr++ { 170 | go sl.List[cntr].StartService() 171 | } 172 | go DispatchSLMsgs(sl) 173 | } 174 | 175 | /* 176 | Funcs for working with Service container and 177 | real servers inside Service container 178 | TODO: move to sep file? 179 | */ 180 | 181 | func (srvc *Service) Init() { 182 | srvc.Timeout = 1 183 | srvc.ToService = make(chan ServiceMsg) 184 | srvc.FromService = make(chan ServiceMsg) 185 | srvc.FromReal = make(chan ServiceMsg) 186 | } 187 | 188 | func (srvc *Service) FindRealFromMsgData(msgData string) (*RealServer, int) { 189 | //right now we always have msg data in format "rip port (meta)?" 190 | dataFields := strings.Fields(msgData) 191 | if len(dataFields) != 3 { 192 | dataFields = append(dataFields, "") 193 | } 194 | rlSrv, i := srvc.FindReal(dataFields[0], dataFields[1], dataFields[2]) 195 | return rlSrv, i 196 | } 197 | 198 | func (srvc *Service) FindReal(RIP, Port, Meta string) (*RealServer, int) { 199 | for cntr := 0; cntr < len(srvc.Reals); cntr++ { 200 | rlSrv := &(srvc.Reals[cntr]) 201 | if rlSrv.RIP == RIP && rlSrv.Port == Port && rlSrv.Meta == Meta { 202 | return rlSrv, cntr 203 | } 204 | } 205 | return nil, -1 206 | } 207 | 208 | func (srvc *Service) RemoveReal(rlSrv *RealServer, index int, notifyReal bool) { 209 | if notifyReal { 210 | rlSrv.ToReal <- ServiceMsg{Cmnd: "Shutdown"} 211 | } 212 | //TODO: add logic w/ adapter (adapter is responsible for adding and removing datapath to real) 213 | if index == len(srvc.Reals) { 214 | srvc.Reals = srvc.Reals[:index] 215 | } else { 216 | srvc.Reals = append(srvc.Reals[:index], srvc.Reals[index+1:]...) 217 | } 218 | srvc.ToAdapter <- GenerateAdapterMsg("DeleteRealServer", srvc, rlSrv) 219 | logMsg := strings.Join([]string{"removing real server", rlSrv.RIP, rlSrv.Port, 220 | rlSrv.Meta, "for service", srvc.VIP, srvc.Port, srvc.Proto}, " ") 221 | srvc.logWriter.Write([]byte(logMsg)) 222 | } 223 | 224 | func GenerateAdapterMsg(msgType string, srvc *Service, rlSrv *RealServer) adapter.AdapterMsg { 225 | adapterMsg := adapter.AdapterMsg{} 226 | adapterMsg.Type = msgType 227 | adapterMsg.ServiceVIP = srvc.VIP 228 | adapterMsg.ServicePort = srvc.Port 229 | adapterMsg.ServiceProto = srvc.Proto 230 | adapterMsg.ServiceMeta = strings.Join([]string{srvc.Scheduler, srvc.Meta}, " ") 231 | if rlSrv != nil { 232 | adapterMsg.RealServerRIP = rlSrv.RIP 233 | adapterMsg.RealServerPort = rlSrv.Port 234 | adapterMsg.RealServerWeight = rlSrv.Weight 235 | adapterMsg.RealServerMeta = rlSrv.Meta 236 | } 237 | return adapterMsg 238 | } 239 | 240 | func (srvc *Service) StartService() { 241 | for cntr := 0; cntr < len(srvc.Reals); cntr++ { 242 | go srvc.Reals[cntr].StartReal() 243 | } 244 | loop := 1 245 | srvc.ToAdapter <- GenerateAdapterMsg("AddService", srvc, nil) 246 | for loop == 1 { 247 | select { 248 | case msg := <-srvc.FromReal: 249 | switch msg.Cmnd { 250 | case "Dead": 251 | srvc.AliveReals-- 252 | logMsg := strings.Join([]string{"real server", msg.Data, "now dead"}, " ") 253 | srvc.logWriter.Write([]byte(logMsg)) 254 | rlSrv, _ := srvc.FindRealFromMsgData(msg.Data) 255 | if rlSrv != nil { 256 | srvc.ToAdapter <- GenerateAdapterMsg("DeleteRealServer", srvc, rlSrv) 257 | } 258 | if srvc.AliveReals < srvc.Quorum && srvc.State == true { 259 | srvc.State = false 260 | logMsg = strings.Join([]string{"turning down service", srvc.VIP, 261 | srvc.Port, srvc.Proto}, " ") 262 | srvc.logWriter.Write([]byte(logMsg)) 263 | srvc.ToAdapter <- GenerateAdapterMsg("WithdrawService", srvc, nil) 264 | } 265 | case "Alive": 266 | srvc.AliveReals++ 267 | logMsg := strings.Join([]string{"real server", msg.Data, "now alive"}, " ") 268 | srvc.logWriter.Write([]byte(logMsg)) 269 | rlSrv, _ := srvc.FindRealFromMsgData(msg.Data) 270 | if rlSrv != nil { 271 | srvc.ToAdapter <- GenerateAdapterMsg("AddRealServer", srvc, rlSrv) 272 | } 273 | //TODO: hysteresis 274 | if srvc.AliveReals >= srvc.Quorum && srvc.State == false { 275 | srvc.State = true 276 | logMsg = strings.Join([]string{"bringing up service", srvc.VIP, 277 | srvc.Port, srvc.Proto}, " ") 278 | srvc.logWriter.Write([]byte(logMsg)) 279 | srvc.ToAdapter <- GenerateAdapterMsg("AdvertiseService", srvc, nil) 280 | } 281 | case "RSFatalError": 282 | /* 283 | right now it seems that real could send this tipe of msg only 284 | at the beggining of his life (when it's not yet enbled and always counted 285 | as dead; so we dont need to check if it's alive and do something with 286 | AliveReals counter. Mb this would change in future 287 | */ 288 | DataFields := strings.Fields(msg.Data) 289 | if len(DataFields) < 3 { 290 | DataFields = append(DataFields, "") 291 | } 292 | rlSrv, index := srvc.FindReal(DataFields[0], DataFields[1], DataFields[2]) 293 | if rlSrv != nil { 294 | srvc.RemoveReal(rlSrv, index, false) 295 | } 296 | } 297 | case msgFromSL := <-srvc.ToService: 298 | switch msgFromSL.Cmnd { 299 | case "GetInfo": 300 | data := strings.Join([]string{ 301 | "State: ", strconv.FormatBool(srvc.State), 302 | "Alive Reals: ", strconv.Itoa(srvc.AliveReals), 303 | "Quorum: ", strconv.Itoa(srvc.Quorum)}, " ") 304 | srvc.FromService <- ServiceMsg{Data: data} 305 | case "RemoveService": 306 | if srvc.State { 307 | srvc.ToAdapter <- GenerateAdapterMsg("WithdrawService", srvc, nil) 308 | } 309 | for _, rlSrv := range srvc.Reals { 310 | rlSrv.ToReal <- ServiceMsg{Cmnd: "Shutdown"} 311 | //TODO: check if real was alive 312 | srvc.ToAdapter <- GenerateAdapterMsg("DeleteRealServer", srvc, &rlSrv) 313 | } 314 | srvc.ToAdapter <- GenerateAdapterMsg("DeleteService", srvc, nil) 315 | logMsg := strings.Join([]string{"service ", srvc.VIP, 316 | srvc.Port, srvc.Proto, " successfully shuted down"}, " ") 317 | srvc.logWriter.Write([]byte(logMsg)) 318 | loop = 0 319 | case "AddReal": 320 | rlSrv := realSrvFromDefinition(msgFromSL.DataMap) 321 | i := srvc.AddReal(rlSrv) 322 | if i == -1 { 323 | srvc.FromService <- ServiceMsg{Data: "RealAlreadyExists"} 324 | break 325 | } 326 | go srvc.Reals[i].StartReal() 327 | srvc.FromService <- ServiceMsg{Data: "RealAdded"} 328 | case "RemoveReal": 329 | rlSrv, i := srvc.FindRealFromMsgData(msgFromSL.Data) 330 | if rlSrv == nil { 331 | srvc.FromService <- ServiceMsg{Data: "RealDoesntExists"} 332 | break 333 | } 334 | if rlSrv.State { 335 | /* TODO: possible sync issue */ 336 | srvc.ToAdapter <- GenerateAdapterMsg("DeleteRealServer", srvc, rlSrv) 337 | srvc.AliveReals-- 338 | if srvc.AliveReals < srvc.Quorum && srvc.State == true { 339 | srvc.State = false 340 | logMsg := strings.Join([]string{"turning down service", srvc.VIP, 341 | srvc.Port, srvc.Proto}, " ") 342 | srvc.logWriter.Write([]byte(logMsg)) 343 | srvc.ToAdapter <- GenerateAdapterMsg("WithdrawService", srvc, nil) 344 | } 345 | } 346 | srvc.RemoveReal(rlSrv, i, true) 347 | srvc.FromService <- ServiceMsg{Data: "RealRemoved"} 348 | case "ChangeService": 349 | //TODO: actually implement something 350 | case "ChangeReal": 351 | //TODO: actually implement something 352 | 353 | } 354 | } 355 | } 356 | } 357 | 358 | func (rlSrv *RealServer) ServiceMsgDataString() string { 359 | DataString := "" 360 | DataString = strings.Join([]string{rlSrv.RIP, rlSrv.Port, rlSrv.Meta}, " ") 361 | return DataString 362 | } 363 | 364 | func (rlSrv *RealServer) StartReal() { 365 | fields := strings.Fields(rlSrv.Check) 366 | checkType := fields[0] 367 | /* 368 | we will send 1 to this chan when we are going to remove this real server 369 | and turn off this check 370 | */ 371 | toCheck := make(chan int) 372 | /* 373 | we are going to receive feedback from the check subsystem thru this chan; 374 | for example 0 if check failed and 1 otherwise 375 | */ 376 | fromCheck := make(chan int) 377 | switch checkType { 378 | case "tcp": 379 | checkLine := []string{rlSrv.RIP, rlSrv.Port} 380 | go healthchecks.TCPCheck(toCheck, fromCheck, checkLine, rlSrv.Timeout) 381 | case "http", "https": 382 | if len(fields) < 2 { 383 | DataString := rlSrv.ServiceMsgDataString() 384 | rlSrv.ToService <- ServiceMsg{Cmnd: "RSFatalError", Data: DataString} 385 | return 386 | } 387 | checkLine := fields[1:] 388 | go healthchecks.HTTPCheck(toCheck, fromCheck, checkLine, rlSrv.Timeout) 389 | case "zk": 390 | if len(fields) < 3 { 391 | DataString := rlSrv.ServiceMsgDataString() 392 | rlSrv.ToService <- ServiceMsg{Cmnd: "RSFatalError", Data: DataString} 393 | return 394 | } 395 | checkLine := fields[1:] 396 | go healthchecks.ZKCheck(toCheck, fromCheck, checkLine, rlSrv.Timeout) 397 | } 398 | loop := 1 399 | for loop == 1 { 400 | select { 401 | case result := <-fromCheck: 402 | switch result { 403 | case -1: 404 | //TODO: add logic to remove real coz of this error 405 | fmt.Println("cant resolve remote addr") 406 | DataString := rlSrv.ServiceMsgDataString() 407 | rlSrv.ToService <- ServiceMsg{Cmnd: "RSFatalError", Data: DataString} 408 | loop = 0 409 | case 1: 410 | if rlSrv.State == false { 411 | rlSrv.State = true 412 | DataString := rlSrv.ServiceMsgDataString() 413 | select { 414 | case rlSrv.ToService <- ServiceMsg{Cmnd: "Alive", Data: DataString}: 415 | case msgToReal := <-rlSrv.ToReal: 416 | ProccessCmndToReal(rlSrv, msgToReal, toCheck, &loop) 417 | } 418 | } 419 | case 0: 420 | if rlSrv.State == true { 421 | rlSrv.State = false 422 | DataString := rlSrv.ServiceMsgDataString() 423 | select { 424 | case rlSrv.ToService <- ServiceMsg{Cmnd: "Dead", Data: DataString}: 425 | case msgToReal := <-rlSrv.ToReal: 426 | ProccessCmndToReal(rlSrv, msgToReal, toCheck, &loop) 427 | } 428 | } 429 | } 430 | case msgToReal := <-rlSrv.ToReal: 431 | ProccessCmndToReal(rlSrv, msgToReal, toCheck, &loop) 432 | } 433 | } 434 | 435 | } 436 | 437 | func ProccessCmndToReal(rlSrv *RealServer, msg ServiceMsg, 438 | toCheck chan int, flag *int) { 439 | switch msg.Cmnd { 440 | case "Shutdown": 441 | toCheck <- 1 442 | *flag = 0 443 | } 444 | } 445 | 446 | func IsServiceValid(srvc Service) bool { 447 | if srvc.VIP != "" && srvc.Port != "" && srvc.Proto != "" { 448 | return true 449 | } else { 450 | return false 451 | } 452 | } 453 | 454 | func (rlsrv *RealServer) isEqualReal(otherRlsrv RealServer) bool { 455 | if rlsrv.RIP == otherRlsrv.RIP && rlsrv.Port == otherRlsrv.Port && 456 | rlsrv.Meta == otherRlsrv.Meta { 457 | return true 458 | } else { 459 | return false 460 | } 461 | } 462 | 463 | //TODO: add loging as well 464 | func (srvc *Service) AddReal(rlsrv RealServer) int { 465 | for cntr := 0; cntr < len(srvc.Reals); cntr++ { 466 | if srvc.Reals[cntr].isEqualReal(rlsrv) { 467 | return -1 468 | } 469 | } 470 | rlsrv.Proto = srvc.Proto 471 | rlsrv.Timeout = srvc.Timeout 472 | rlsrv.ToService = srvc.FromReal 473 | rlsrv.ToReal = make(chan ServiceMsg) 474 | if rlsrv.Weight == "" { 475 | rlsrv.Weight = "1" 476 | } 477 | srvc.Reals = append(srvc.Reals, rlsrv) 478 | return len(srvc.Reals) - 1 479 | } 480 | 481 | //TODO: change realServer logic 482 | -------------------------------------------------------------------------------- /service/service_test.go: -------------------------------------------------------------------------------- 1 | package service 2 | 3 | import ( 4 | "fmt" 5 | "go_keepalived/adapter" 6 | "go_keepalived/notifier" 7 | "log/syslog" 8 | "testing" 9 | "time" 10 | ) 11 | 12 | func DummyAdapterReader(msgChan chan adapter.AdapterMsg) { 13 | for { 14 | msg := <-msgChan 15 | fmt.Println(msg) 16 | } 17 | } 18 | 19 | func TestServicesListAdd(t *testing.T) { 20 | srvc1 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 21 | srvc2 := Service{VIP: "127.0.0.2", Proto: "tcp", Port: "22"} 22 | srvc3 := Service{VIP: "127.0.0.3", Proto: "tcp", Port: "22"} 23 | srvc4 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 24 | srvcList := ServicesList{} 25 | srvcList.Init() 26 | //HACK. we wont need to send anything to ipvsadm 27 | srvcList.ToAdapter = make(chan adapter.AdapterMsg) 28 | go DummyAdapterReader(srvcList.ToAdapter) 29 | srvcList.Add(srvc1) 30 | srvcList.Add(srvc2) 31 | srvcList.Add(srvc3) 32 | srvcList.Add(srvc4) 33 | if len(srvcList.List) != 3 { 34 | t.Errorf("error in adding services") 35 | } 36 | } 37 | 38 | func TestServicesListRemove(t *testing.T) { 39 | srvc1 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 40 | srvc2 := Service{VIP: "127.0.0.2", Proto: "tcp", Port: "22"} 41 | srvc3 := Service{VIP: "127.0.0.3", Proto: "tcp", Port: "22"} 42 | srvc4 := Service{VIP: "127.0.0.4", Proto: "tcp", Port: "22"} 43 | srvc1.Init() 44 | srvc2.Init() 45 | srvc3.Init() 46 | srvc4.Init() 47 | srvcList := ServicesList{} 48 | srvcList.Init() 49 | srvcList.StartAdapter("testing") 50 | nc := notifier.NotifierConfig{} 51 | nc.Type = "dummy" 52 | srvcList.AddNotifier(nc) 53 | srvcList.Add(srvc1) 54 | srvcList.Add(srvc2) 55 | srvcList.Add(srvc3) 56 | srvcList.Add(srvc4) 57 | srvcList.Start() 58 | srvcList.Remove(srvc4) 59 | if len(srvcList.List) != 3 { 60 | t.Errorf("error in adding services") 61 | } 62 | srvcList.Remove(srvc2) 63 | if len(srvcList.List) != 2 { 64 | t.Errorf("error in adding services") 65 | } 66 | 67 | } 68 | 69 | func TestServiceAddReal(t *testing.T) { 70 | srvc1 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 71 | rlSrv1 := RealServer{RIP: "127.0.0.2", Port: "22"} 72 | rlSrv2 := RealServer{RIP: "127.0.0.3", Port: "22"} 73 | rlSrv3 := RealServer{RIP: "127.0.0.4", Port: "22"} 74 | rlSrv4 := RealServer{RIP: "127.0.0.4", Port: "22"} 75 | srvc1.Init() 76 | srvc1.ToAdapter = make(chan adapter.AdapterMsg) 77 | go DummyAdapterReader(srvc1.ToAdapter) 78 | srvc1.Timeout = 33 79 | srvc1.AddReal(rlSrv1) 80 | srvc1.AddReal(rlSrv2) 81 | srvc1.AddReal(rlSrv3) 82 | srvc1.AddReal(rlSrv4) 83 | 84 | if len(srvc1.Reals) != 3 { 85 | t.Errorf("error in adding realServers (duplicate detection logic)") 86 | } 87 | if srvc1.Reals[0].Timeout != 33 { 88 | t.Errorf("error in Timeout's val copying") 89 | } 90 | if srvc1.Reals[0].Proto != "tcp" { 91 | t.Errorf("error in Proto's val copying") 92 | } 93 | 94 | } 95 | 96 | func TestServiceRemoveReal(t *testing.T) { 97 | srvc1 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 98 | rlSrv1 := RealServer{RIP: "127.0.0.2", Port: "22"} 99 | rlSrv2 := RealServer{RIP: "127.0.0.3", Port: "22"} 100 | rlSrv3 := RealServer{RIP: "127.0.0.4", Port: "22"} 101 | srvc1.Init() 102 | //because we didnt add srvc1 to services list we dont have syslog Writer 103 | srvc1.logWriter, _ = syslog.New(syslog.LOG_INFO, "go_keepalive_test") 104 | srvc1.Timeout = 33 105 | srvc1.ToAdapter = make(chan adapter.AdapterMsg) 106 | go DummyAdapterReader(srvc1.ToAdapter) 107 | 108 | srvc1.AddReal(rlSrv1) 109 | srvc1.AddReal(rlSrv2) 110 | srvc1.AddReal(rlSrv3) 111 | _, index := srvc1.FindReal(rlSrv2.RIP, rlSrv2.Port, rlSrv2.Meta) 112 | if index != 1 { 113 | t.Errorf("FindReal is not working") 114 | } 115 | r, index := srvc1.FindReal(rlSrv3.RIP, rlSrv3.Port, rlSrv3.Meta) 116 | srvc1.RemoveReal(r, index, false) 117 | if len(srvc1.Reals) != 2 { 118 | t.Errorf("RemoveReal is not working") 119 | } 120 | srvc1.AddReal(rlSrv3) 121 | r, index = srvc1.FindReal(rlSrv2.RIP, rlSrv2.Port, rlSrv2.Meta) 122 | srvc1.RemoveReal(r, index, false) 123 | if len(srvc1.Reals) != 2 { 124 | t.Errorf("RemoveReal is not working") 125 | } 126 | 127 | } 128 | 129 | func TestServiceStart(t *testing.T) { 130 | srvc1 := Service{VIP: "127.0.0.1", Proto: "tcp", Port: "22"} 131 | rlSrv1 := RealServer{RIP: "127.0.0.1", Port: "60021", Check: "tcp"} 132 | rlSrv2 := RealServer{RIP: "127.0.0.1", Port: "60022", Check: "tcp"} 133 | rlSrv3 := RealServer{RIP: "127.0.0.1", Port: "600023", Check: "tcp"} 134 | rlSrv4 := RealServer{RIP: "127.0.0.1", Port: "60024", Check: "https"} 135 | srvc1.Init() 136 | srvc1.logWriter, _ = syslog.New(syslog.LOG_INFO, "go_keepalive_test") 137 | srvc1.Timeout = 33 138 | srvc1.ToAdapter = make(chan adapter.AdapterMsg) 139 | go DummyAdapterReader(srvc1.ToAdapter) 140 | srvc1.AddReal(rlSrv1) 141 | srvc1.AddReal(rlSrv2) 142 | srvc1.AddReal(rlSrv3) 143 | srvc1.AddReal(rlSrv4) 144 | go srvc1.StartService() 145 | //stupid way for sync, mb add something better in future 146 | time.Sleep(3 * time.Second) 147 | if len(srvc1.Reals) != 3 { 148 | t.Errorf("something wrong with RSFatalError handling, there are %d reals", len(srvc1.Reals)) 149 | } 150 | } 151 | -------------------------------------------------------------------------------- /structure.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tehnerd/go_keepalived/130ff4fd60627c3ecb5dcc4f4b1ba557d028b0a8/structure.jpg --------------------------------------------------------------------------------