├── static ├── robots.txt ├── favicon.ico ├── stats.hbs └── main.css ├── .dockerignore ├── init.sql ├── ansible ├── playbook.yml ├── inventory └── roles │ └── radiobrowser │ ├── templates │ └── radio-browser.conf.j2 │ └── tasks │ └── main.yml ├── install_from_source.sh ├── src ├── check │ ├── mod.rs │ ├── diff_calc.rs │ └── favicon.rs ├── db │ ├── mod.rs │ ├── models │ │ ├── station_click_item_new.rs │ │ ├── station_click_item.rs │ │ ├── db_country.rs │ │ ├── station_check_step_item_new.rs │ │ ├── station_check_step_item.rs │ │ ├── streaming_server_new.rs │ │ ├── station_change_item_new.rs │ │ ├── streaming_server.rs │ │ ├── station_history_item.rs │ │ ├── mod.rs │ │ ├── station_check_item.rs │ │ ├── extra_info.rs │ │ ├── state.rs │ │ ├── station_check_item_new.rs │ │ └── station_item.rs │ ├── db_error.rs │ ├── db_mysql │ │ ├── simple_migrate.rs │ │ └── conversions.rs │ └── db.rs ├── api │ ├── api_response.rs │ ├── api_error.rs │ ├── data │ │ ├── result_message.rs │ │ ├── mod.rs │ │ ├── station_add_result.rs │ │ ├── status.rs │ │ ├── api_streaming_server.rs │ │ ├── api_language.rs │ │ ├── station_check_step.rs │ │ ├── api_country.rs │ │ ├── station_click.rs │ │ ├── api_config.rs │ │ ├── station_history.rs │ │ └── station_check.rs │ ├── cache │ │ ├── memcached.rs │ │ ├── redis.rs │ │ ├── builtin.rs │ │ └── mod.rs │ ├── all_params.rs │ ├── prometheus_exporter.rs │ └── parameters.rs ├── pull │ ├── uuid_with_time.rs │ └── pull_error.rs ├── config │ ├── config_error.rs │ ├── data_mapping_item.rs │ └── config.rs ├── cleanup │ └── mod.rs ├── checkserver │ └── mod.rs ├── cli.rs ├── refresh │ └── mod.rs ├── logger.rs └── main.rs ├── .gitignore ├── etc ├── tag-replace.csv ├── logrotate ├── language-replace.csv ├── language-to-code.csv └── config-example.toml ├── start.sh ├── uninstall.sh ├── debian ├── postinst └── radiobrowser.service ├── deployment ├── backup.sh └── HEADER.html ├── traefik-dyn-config.toml ├── .github └── workflows │ └── rust.yml ├── Dockerfile ├── docker-compose.yml ├── install_from_dist.sh ├── .gitlab-ci.yml ├── docker-compose-traefik.yml ├── Cargo.toml ├── radiobrowser-dev.toml ├── CODE_OF_CONDUCT.md └── CHANGELOG.md /static/robots.txt: -------------------------------------------------------------------------------- 1 | User-agent: * 2 | Disallow: / -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | target/ 2 | .git 3 | .gitignore 4 | Dockerfile 5 | dist/ -------------------------------------------------------------------------------- /init.sql: -------------------------------------------------------------------------------- 1 | CREATE DATABASE radio; 2 | GRANT all ON radio.* TO radiouser IDENTIFIED BY "password"; -------------------------------------------------------------------------------- /ansible/playbook.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - hosts: all 3 | remote_user: root 4 | roles: 5 | - radiobrowser -------------------------------------------------------------------------------- /install_from_source.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | ./builddist.sh 6 | cd dist 7 | ./install.sh -------------------------------------------------------------------------------- /src/check/mod.rs: -------------------------------------------------------------------------------- 1 | mod check; 2 | mod favicon; 3 | mod diff_calc; 4 | 5 | pub use check::dbcheck; 6 | -------------------------------------------------------------------------------- /static/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/segler-alex/radiobrowser-api-rust/HEAD/static/favicon.ico -------------------------------------------------------------------------------- /ansible/inventory: -------------------------------------------------------------------------------- 1 | [servers] 2 | test.api.radio-browser.info 3 | 4 | [servers:vars] 5 | ansible_python_interpreter=auto -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | /target/ 3 | **/*.rs.bk 4 | 5 | dbdata/ 6 | 7 | radio\.sql\.gz 8 | 9 | access\.log 10 | dist/ 11 | *.log 12 | -------------------------------------------------------------------------------- /etc/tag-replace.csv: -------------------------------------------------------------------------------- 1 | # replace tags in the database in the first columen 2 | # with tags in the second column 3 | # empty strings in the second column does remove the item 4 | from;to 5 | -------------------------------------------------------------------------------- /etc/logrotate: -------------------------------------------------------------------------------- 1 | /var/log/radiobrowser/*.log { 2 | daily 3 | rotate 10 4 | copytruncate 5 | compress 6 | delaycompress 7 | notifempty 8 | missingok 9 | } 10 | -------------------------------------------------------------------------------- /etc/language-replace.csv: -------------------------------------------------------------------------------- 1 | # replace languages in the database in the first columen 2 | # with languages in the second column 3 | # empty strings in the second column does remove the item 4 | from;to 5 | -------------------------------------------------------------------------------- /src/db/mod.rs: -------------------------------------------------------------------------------- 1 | mod db; 2 | mod db_mysql; 3 | mod db_error; 4 | 5 | pub mod models; 6 | 7 | pub use self::db::DbConnection; 8 | pub use self::db_mysql::MysqlConnection; 9 | pub use self::db_error::DbError; -------------------------------------------------------------------------------- /start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | rm radio.sql.gz 4 | wget http://www.radio-browser.info/backups/latest.sql.gz -O radio.sql.gz 5 | mkdir -p dbdata 6 | docker stack deploy -c docker-compose.yml radiobrowser 7 | -------------------------------------------------------------------------------- /src/db/models/station_click_item_new.rs: -------------------------------------------------------------------------------- 1 | #[derive(Clone,Debug)] 2 | pub struct StationClickItemNew { 3 | pub stationuuid: String, 4 | pub ip: String, 5 | pub clickuuid: String, 6 | pub clicktimestamp: String, 7 | } -------------------------------------------------------------------------------- /static/stats.hbs: -------------------------------------------------------------------------------- 1 | 2 | 3 | RadioBrowser Server Information 4 | 5 | 6 |

RadioBrowser Server Information

7 | Status: {{ status.status }} 8 | 9 | -------------------------------------------------------------------------------- /src/api/api_response.rs: -------------------------------------------------------------------------------- 1 | use std::fs::File; 2 | 3 | pub enum ApiResponse { 4 | Text(String), 5 | File(String, File), 6 | ServerError(String), 7 | NotFound, 8 | UnknownContentType, 9 | //ParameterError(String), 10 | Locked(String), 11 | } 12 | -------------------------------------------------------------------------------- /uninstall.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | sudo rm /usr/bin/radiobrowser 3 | sudo userdel radiobrowser 4 | sudo groupdel radiobrowser 5 | 6 | sudo rm /etc/systemd/system/radiobrowser.service 7 | sudo systemctl daemon-reload 8 | sudo rm -rf /usr/share/radiobrowser 9 | sudo rm -rf /var/log/radiobrowser 10 | 11 | sudo rm /etc/logrotate.d/radiobrowser -------------------------------------------------------------------------------- /src/pull/uuid_with_time.rs: -------------------------------------------------------------------------------- 1 | use std::time::{Instant}; 2 | pub struct UuidWithTime { 3 | pub uuid: String, 4 | pub instant: Instant, 5 | } 6 | 7 | impl UuidWithTime { 8 | pub fn new(uuid: &str) -> Self{ 9 | UuidWithTime { 10 | uuid: uuid.to_string(), 11 | instant: Instant::now(), 12 | } 13 | } 14 | } -------------------------------------------------------------------------------- /src/db/models/station_click_item.rs: -------------------------------------------------------------------------------- 1 | use chrono::DateTime; 2 | use chrono::Utc; 3 | 4 | #[derive(Clone, Debug)] 5 | pub struct StationClickItem { 6 | pub id: i32, 7 | pub stationuuid: String, 8 | pub ip: String, 9 | pub clickuuid: String, 10 | pub clicktimestamp_iso8601: Option>, 11 | pub clicktimestamp: String, 12 | } 13 | -------------------------------------------------------------------------------- /debian/postinst: -------------------------------------------------------------------------------- 1 | # Add user 2 | groupadd --system radiobrowser 3 | useradd --system --no-create-home --home-dir /var/lib/radiobrowser --gid radiobrowser radiobrowser 4 | 5 | # Create log dir 6 | mkdir -p /var/log/radiobrowser 7 | chown radiobrowser:radiobrowser /var/log/radiobrowser 8 | 9 | # Create home dir 10 | mkdir -p /var/lib/radiobrowser 11 | chown radiobrowser:radiobrowser /var/lib/radiobrowser 12 | 13 | #DEBHELPER# -------------------------------------------------------------------------------- /deployment/backup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | DBNAME="radio" 3 | NAME="/var/www/backups/backup_$(hostname -s)_$(date +"%Y-%m-%d").sql" 4 | mysqldump ${DBNAME} --ignore-table=${DBNAME}.StationCheckHistory --ignore-table=${DBNAME}.StationClick > ${NAME} 5 | mysqldump --no-data ${DBNAME} StationCheckHistory StationClick >> ${NAME} 6 | gzip -f ${NAME} 7 | rm /var/www/backups/latest.sql.gz 8 | ln -s "${NAME}.gz" /var/www/backups/latest.sql.gz 9 | -------------------------------------------------------------------------------- /src/check/diff_calc.rs: -------------------------------------------------------------------------------- 1 | #[derive(Clone, Debug)] 2 | pub struct DiffCalc { 3 | pub old: T, 4 | pub new: T, 5 | } 6 | 7 | impl DiffCalc { 8 | pub fn new(current: T) -> Self { 9 | DiffCalc { 10 | old: current.clone(), 11 | new: current, 12 | } 13 | } 14 | 15 | pub fn changed(&self) -> bool { 16 | self.new != self.old 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /src/db/models/db_country.rs: -------------------------------------------------------------------------------- 1 | use serde::{Deserialize, Serialize}; 2 | 3 | #[derive(PartialEq, Eq, Serialize, Deserialize)] 4 | pub struct DBCountry { 5 | pub iso_3166_1: String, 6 | pub stationcount: u32, 7 | } 8 | 9 | impl DBCountry { 10 | pub fn new(iso_3166_1: String, stationcount: u32) -> Self { 11 | DBCountry { 12 | iso_3166_1, 13 | stationcount, 14 | } 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /src/db/models/station_check_step_item_new.rs: -------------------------------------------------------------------------------- 1 | use serde::{Serialize,Deserialize}; 2 | 3 | #[derive(PartialEq, Eq, Serialize, Deserialize, Debug, Clone)] 4 | pub struct StationCheckStepItemNew { 5 | pub stepuuid: String, 6 | pub parent_stepuuid: Option, 7 | pub checkuuid: String, 8 | pub stationuuid: String, 9 | pub url: String, 10 | pub urltype: Option, 11 | pub error: Option, 12 | } -------------------------------------------------------------------------------- /traefik-dyn-config.toml: -------------------------------------------------------------------------------- 1 | [tls.options] 2 | [tls.options.default] 3 | minVersion = "VersionTLS12" 4 | cipherSuites = [ 5 | "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", 6 | "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", 7 | "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", 8 | "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", 9 | "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", 10 | "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305" 11 | ] -------------------------------------------------------------------------------- /src/api/api_error.rs: -------------------------------------------------------------------------------- 1 | use std::error::Error; 2 | use std::fmt::Display; 3 | use std::fmt::Formatter; 4 | use std::fmt::Result; 5 | 6 | #[derive(Debug, Clone)] 7 | pub enum ApiError { 8 | InternalError(String), 9 | } 10 | 11 | impl Display for ApiError { 12 | fn fmt(&self, f: &mut Formatter) -> Result { 13 | match *self { 14 | ApiError::InternalError(ref v) => write!(f, "InternalError '{}'", v), 15 | } 16 | } 17 | } 18 | 19 | impl Error for ApiError {} 20 | -------------------------------------------------------------------------------- /src/pull/pull_error.rs: -------------------------------------------------------------------------------- 1 | use std::error::Error; 2 | use std::fmt::Display; 3 | use std::fmt::Formatter; 4 | use std::fmt::Result; 5 | 6 | #[derive(Debug, Clone)] 7 | pub enum PullError { 8 | UnknownApiVersion(u32), 9 | } 10 | 11 | impl Display for PullError { 12 | fn fmt(&self, f: &mut Formatter) -> Result { 13 | match *self { 14 | PullError::UnknownApiVersion(ref v) => write!(f, "UnknownApiVersion {}", v), 15 | } 16 | } 17 | } 18 | 19 | impl Error for PullError {} 20 | -------------------------------------------------------------------------------- /static/main.css: -------------------------------------------------------------------------------- 1 | /* 2 | #558C89 dark 3 | #74AFAD light 4 | #D9853B orange 5 | #ECECEA grey 6 | j*/ 7 | 8 | body { 9 | background-color: #ede7df; 10 | font-family: "Segoe UI", Helvetica, Arial, sans-serif; 11 | } 12 | .syntax { 13 | line-height: 1.4; 14 | } 15 | .dropdown-item:hover{ 16 | background-color: #007bff; 17 | color: white; 18 | } 19 | .format, .searchTerm{ 20 | font-style: italic; 21 | font-weight: bold; 22 | color: rgb(116, 116, 116); 23 | } 24 | 25 | .result { 26 | margin: 20px; 27 | } -------------------------------------------------------------------------------- /src/db/models/station_check_step_item.rs: -------------------------------------------------------------------------------- 1 | use chrono::DateTime; 2 | use chrono::Utc; 3 | use serde::{Serialize,Deserialize}; 4 | 5 | #[derive(PartialEq, Eq, Serialize, Deserialize, Debug, Clone)] 6 | pub struct StationCheckStepItem { 7 | pub id: u32, 8 | pub stepuuid: String, 9 | pub parent_stepuuid: Option, 10 | pub checkuuid: String, 11 | pub stationuuid: String, 12 | pub url: String, 13 | pub urltype: Option, 14 | pub error: Option, 15 | pub inserttime: DateTime, 16 | } -------------------------------------------------------------------------------- /deployment/HEADER.html: -------------------------------------------------------------------------------- 1 |

Radio-browser database backups

2 | The backups are done with the following commands on a daily basis: 3 | 4 | 
 5 | #!/bin/sh
 6 | DBNAME="radio"
 7 | NAME="/var/www/backups/backup_$(hostname -s)_$(date +"%Y-%m-%d").sql"
 8 | mysqldump ${DBNAME} --ignore-table=${DBNAME}.StationCheckHistory  --ignore-table=${DBNAME}.StationClick > ${NAME}
 9 | mysqldump --no-data ${DBNAME} StationCheckHistory StationClick >> ${NAME}
10 | gzip -f ${NAME}
11 | rm /var/www/backups/latest.sql.gz
12 | ln -s "${NAME}.gz" /var/www/backups/latest.sql.gz
13 | 
14 | 
15 | 


--------------------------------------------------------------------------------
/src/db/models/streaming_server_new.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Deserialize, Serialize};
 2 | 
 3 | #[derive(PartialEq, Eq, Serialize, Deserialize, Debug)]
 4 | pub struct DbStreamingServerNew {
 5 |     pub url: String,
 6 |     pub statusurl: Option,
 7 |     pub status: Option,
 8 |     pub error: Option,
 9 | }
10 | 
11 | impl DbStreamingServerNew {
12 |     pub fn new(url: String, statusurl: Option, status: Option, error: Option) -> Self {
13 |         DbStreamingServerNew {
14 |             url,
15 |             statusurl,
16 |             status,
17 |             error,
18 |         }
19 |     }
20 | }
21 | 


--------------------------------------------------------------------------------
/src/db/models/station_change_item_new.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Serialize,Deserialize};
 2 | 
 3 | #[derive(PartialEq, Serialize, Deserialize, Debug, Clone)]
 4 | pub struct StationChangeItemNew {
 5 |     pub name: String,
 6 |     pub url: String,
 7 |     pub homepage: String,
 8 |     pub favicon: String,
 9 |     pub country: String,
10 |     pub state: String,
11 |     pub countrycode: String,
12 |     pub language: String,
13 |     pub languagecodes: String,
14 |     pub tags: String,
15 |     pub votes: i32,
16 | 
17 |     pub changeuuid: String,
18 |     pub stationuuid: String,
19 |     pub geo_lat: Option,
20 |     pub geo_long: Option,
21 | }


--------------------------------------------------------------------------------
/ansible/roles/radiobrowser/templates/radio-browser.conf.j2:
--------------------------------------------------------------------------------
 1 | 
 2 |     ServerName {{ inventory_hostname }}
 3 | 
 4 |     ServerAdmin {{ email }}
 5 |     DocumentRoot /var/www/radio
 6 | 
 7 |     #ErrorLog ${APACHE_LOG_DIR}/error.radio.log
 8 |     #CustomLog ${APACHE_LOG_DIR}/access.radio.log combined
 9 | 
10 |     ProxyPass "/"  "http://localhost:8080/"
11 |     ProxyPassReverse "/"  "http://localhost:8080/"
12 | 
13 |     AddOutputFilterByType DEFLATE application/xml text/xml
14 |     AddOutputFilterByType DEFLATE application/json
15 | 
16 |     
17 |         AllowOverride All
18 |         Order allow,deny
19 |         allow from all
20 |     
21 | 


--------------------------------------------------------------------------------
/src/config/config_error.rs:
--------------------------------------------------------------------------------
 1 | use std::error::Error;
 2 | use std::fmt::Display;
 3 | use std::fmt::Formatter;
 4 | use std::fmt::Result;
 5 | 
 6 | #[derive(Debug, Clone)]
 7 | pub enum ConfigError {
 8 |     TypeError(String, String),
 9 | }
10 | 
11 | impl Display for ConfigError {
12 |     fn fmt(&self, f: &mut Formatter) -> Result {
13 |         match *self {
14 |             ConfigError::TypeError(ref field_name, ref field_value) => write!(f, "Value {} for field {} has wrong type", field_name, field_value),
15 |         }
16 |     }
17 | }
18 | 
19 | impl Error for ConfigError {
20 |     fn description(&self) -> &str {
21 |         "NO DESCRIPTION"
22 |     }
23 | 
24 |     fn cause(&self) -> Option<&dyn Error> {
25 |         None
26 |     }
27 | }


--------------------------------------------------------------------------------
/src/db/models/streaming_server.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Deserialize, Serialize};
 2 | 
 3 | #[derive(PartialEq, Eq, Serialize, Deserialize, Debug)]
 4 | pub struct DbStreamingServer {
 5 |     pub id: u32,
 6 |     pub uuid: String,
 7 |     pub url: String,
 8 |     pub statusurl: Option,
 9 |     pub status: Option,
10 |     pub error: Option,
11 | }
12 | 
13 | impl DbStreamingServer {
14 |     pub fn new(id: u32, uuid: String, url: String, statusurl: Option, status: Option, error: Option) -> Self {
15 |         DbStreamingServer {
16 |             id,
17 |             uuid,
18 |             url,
19 |             statusurl,
20 |             status,
21 |             error,
22 |         }
23 |     }
24 | }
25 | 


--------------------------------------------------------------------------------
/src/db/models/station_history_item.rs:
--------------------------------------------------------------------------------
 1 | use chrono::Utc;
 2 | use chrono::DateTime;
 3 | use serde::{Serialize,Deserialize};
 4 | 
 5 | #[derive(PartialEq, Serialize, Deserialize, Debug)]
 6 | pub struct StationHistoryItem {
 7 |     pub id: i32,
 8 |     pub changeuuid: String,
 9 |     pub stationuuid: String,
10 |     pub name: String,
11 |     pub url: String,
12 |     pub homepage: String,
13 |     pub favicon: String,
14 |     pub tags: String,
15 |     pub countrycode: String,
16 |     pub state: String,
17 |     pub language: String,
18 |     pub languagecodes: String,
19 |     pub votes: i32,
20 |     pub lastchangetime: String,
21 |     pub lastchangetime_iso8601: Option>,
22 |     pub geo_lat: Option,
23 |     pub geo_long: Option,
24 | }
25 | 


--------------------------------------------------------------------------------
/debian/radiobrowser.service:
--------------------------------------------------------------------------------
 1 | [Unit]
 2 | Description=RadioBrowserAPI
 3 | After=network.target
 4 | 
 5 | [Install]
 6 | WantedBy=multi-user.target
 7 | 
 8 | [Service]
 9 | 
10 | ##############################################################################
11 | ## Core requirements
12 | ##
13 | 
14 | Type=simple
15 | 
16 | ##############################################################################
17 | ## Package maintainers
18 | ##
19 | 
20 | User=radiobrowser
21 | Group=radiobrowser
22 | 
23 | # Prevent writes to /usr, /boot, and /etc
24 | ProtectSystem=full
25 | 
26 | # Prevent accessing /home, /root and /run/user
27 | ProtectHome=true
28 | 
29 | # Start main service
30 | ExecStart=/usr/bin/radiobrowser -f /etc/radiobrowser/config.toml
31 | 
32 | Restart=always
33 | RestartSec=5s
34 | 


--------------------------------------------------------------------------------
/.github/workflows/rust.yml:
--------------------------------------------------------------------------------
 1 | on: [push, pull_request]
 2 | 
 3 | name: Continuous integration
 4 | 
 5 | jobs:
 6 |   check:
 7 |     name: Check
 8 |     runs-on: ubuntu-latest
 9 |     steps:
10 |       - uses: actions/checkout@v2
11 |       - uses: actions-rs/toolchain@v1
12 |         with:
13 |           profile: minimal
14 |           toolchain: stable
15 |           override: true
16 |       - uses: actions-rs/cargo@v1
17 |         with:
18 |           command: check
19 | 
20 |   test:
21 |     name: Test Suite
22 |     runs-on: ubuntu-latest
23 |     steps:
24 |       - uses: actions/checkout@v2
25 |       - uses: actions-rs/toolchain@v1
26 |         with:
27 |           profile: minimal
28 |           toolchain: stable
29 |           override: true
30 |       - uses: actions-rs/cargo@v1
31 |         with:
32 |           command: test
33 | 


--------------------------------------------------------------------------------
/src/db/db_error.rs:
--------------------------------------------------------------------------------
 1 | use std::error::Error;
 2 | use std::fmt::Display;
 3 | use std::fmt::Formatter;
 4 | use std::fmt::Result;
 5 | 
 6 | #[derive(Debug, Clone)]
 7 | pub enum DbError {
 8 |     //ConnectionError(String),
 9 |     VoteError(String),
10 |     AddStationError(String),
11 |     IllegalOrderError(String),
12 | }
13 | 
14 | impl Display for DbError {
15 |     fn fmt(&self, f: &mut Formatter) -> Result {
16 |         match *self {
17 |             //DbError::ConnectionError(ref v) => write!(f, "ConnectionError '{}'", v),
18 |             DbError::VoteError(ref v) => write!(f, "VoteError '{}'", v),
19 |             DbError::AddStationError(ref v) => write!(f, "AddStationError '{}'", v),
20 |             DbError::IllegalOrderError(ref v) => write!(f, "IllegalOrderError '{}'", v),
21 |         }
22 |     }
23 | }
24 | 
25 | impl Error for DbError {}
26 | 


--------------------------------------------------------------------------------
/src/api/data/result_message.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Serialize,Deserialize};
 2 | 
 3 | #[derive(Serialize, Deserialize)]
 4 | pub struct ResultMessage {
 5 |     ok: bool,
 6 |     message: String,
 7 | }
 8 | 
 9 | impl ResultMessage {
10 |     pub fn new(ok: bool, message: String) -> Self {
11 |         ResultMessage{
12 |             ok,
13 |             message
14 |         }
15 |     }
16 | 
17 |     pub fn serialize_xml(&self) -> std::io::Result {
18 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
19 |         xml.begin_elem("result")?;
20 |             xml.begin_elem("status")?;
21 |                 xml.attr_esc("ok", &self.ok.to_string())?;
22 |                 xml.attr_esc("message", &self.message)?;
23 |             xml.end_elem()?;
24 |         xml.end_elem()?;
25 |         xml.close()?;
26 |         xml.flush()?;
27 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
28 |     }
29 | }


--------------------------------------------------------------------------------
/Dockerfile:
--------------------------------------------------------------------------------
 1 | FROM alpine:3.14
 2 | ADD . /app
 3 | WORKDIR /app
 4 | RUN apk update
 5 | RUN apk add rustup openssl-dev gcc g++
 6 | RUN rustup-init -y
 7 | ENV PATH="/root/.cargo/bin:$PATH"
 8 | RUN cargo build --release
 9 | 
10 | FROM alpine:3.14
11 | EXPOSE 8080
12 | COPY --from=0 /app/target/release/radiobrowser-api-rust /usr/bin/
13 | COPY --from=0 /app/static/ /usr/lib/radiobrowser/static/
14 | COPY --from=0 /app/etc/config-example.toml /etc/radiobrowser/config.toml
15 | COPY --from=0 /app/etc/*.csv /etc/radiobrowser/
16 | RUN addgroup -S radiobrowser && \
17 |  adduser -S -G radiobrowser radiobrowser && \
18 |  apk add libgcc && \
19 |  mkdir -p /var/log/radiobrowser/ && \
20 |  chown -R radiobrowser:radiobrowser /var/log/radiobrowser/ && \
21 |  chmod go+r /etc/radiobrowser/config.toml
22 | ENV STATIC_FILES_DIR=/usr/lib/radiobrowser/static/
23 | USER radiobrowser:radiobrowser
24 | CMD [ "radiobrowser-api-rust", "-f", "/etc/radiobrowser/config.toml", "-vvv"]
25 | 


--------------------------------------------------------------------------------
/src/check/favicon.rs:
--------------------------------------------------------------------------------
 1 | use website_icon_extract::ImageLink;
 2 | 
 3 | fn proximity(optimal: i32, link: &ImageLink) -> i32
 4 | {
 5 |     let width: i32 = link.width as i32;
 6 |     let height: i32 = link.height as i32;
 7 |     (optimal - (width + height) / 2).abs()
 8 | }
 9 | 
10 | pub fn get_best_icon(
11 |     mut list: Vec,
12 |     optimal: usize,
13 |     minsize: usize,
14 |     maxsize: usize,
15 | ) -> Option {
16 |     if list.len() > 0 {
17 |         let mut new_list: Vec = list
18 |             .drain(..)
19 |             .filter(|image| {
20 |                 image.width >= minsize
21 |                     && image.width <= maxsize
22 |                     && image.height >= minsize
23 |                     && image.height <= maxsize
24 |             })
25 |             .collect();
26 |         new_list.sort_unstable_by(|a, b| {
27 |             proximity(optimal as i32, b).cmp(&proximity(optimal as i32, a))
28 |         });
29 |         new_list.pop()
30 |     } else {
31 |         None
32 |     }
33 | }
34 | 


--------------------------------------------------------------------------------
/docker-compose.yml:
--------------------------------------------------------------------------------
 1 | version: "3.0"
 2 | services:
 3 |   api:
 4 |     build: ./
 5 |     image: segleralex/radiobrowser-api-rust:0.7.24
 6 |     deploy:
 7 |       replicas: 1
 8 |     networks:
 9 |       - mynet
10 |     ports:
11 |       - 8080:8080
12 |     environment:
13 |        - DATABASE_URL=mysql://radiouser:password@dbserver/radio
14 |        - CACHETYPE=redis
15 |        - CACHETTL=60sec
16 |        - CACHEURL=redis://cacheserver:6379
17 |        - HOST=0.0.0.0
18 |   cacheserver:
19 |     image: redis
20 |     networks:
21 |       - mynet
22 |   dbserver:
23 |     image: mariadb:10.4
24 |     deploy:
25 |       replicas: 1
26 |     environment:
27 |        - MYSQL_ROOT_PASSWORD=12345678
28 |        - MYSQL_USER=radiouser
29 |        - MYSQL_PASSWORD=password
30 |        - MYSQL_DATABASE=radio
31 |     networks:
32 |       - mynet
33 |     volumes:
34 |       - ./dbdata:/var/lib/mysql
35 |     command: ["mysqld","--character-set-server=utf8mb4","--collation-server=utf8mb4_unicode_ci"]
36 | networks:
37 |   mynet:
38 |     driver: "overlay"
39 |     driver_opts:
40 |       encrypted: ""
41 | 


--------------------------------------------------------------------------------
/src/db/models/mod.rs:
--------------------------------------------------------------------------------
 1 | mod extra_info;
 2 | mod station_check_item;
 3 | mod station_check_item_new;
 4 | mod station_item;
 5 | mod state;
 6 | mod station_change_item_new;
 7 | mod station_history_item;
 8 | mod station_click_item;
 9 | mod station_click_item_new;
10 | mod station_check_step_item;
11 | mod station_check_step_item_new;
12 | mod streaming_server;
13 | mod streaming_server_new;
14 | mod db_country;
15 | 
16 | pub use db_country::DBCountry;
17 | pub use station_click_item::StationClickItem;
18 | pub use station_click_item_new::StationClickItemNew;
19 | pub use station_history_item::StationHistoryItem;
20 | pub use station_change_item_new::StationChangeItemNew;
21 | pub use station_check_item::StationCheckItem;
22 | pub use station_check_item_new::StationCheckItemNew;
23 | pub use station_item::DbStationItem;
24 | pub use extra_info::ExtraInfo;
25 | pub use state::State;
26 | pub use station_check_step_item::StationCheckStepItem;
27 | pub use station_check_step_item_new::StationCheckStepItemNew;
28 | pub use streaming_server::DbStreamingServer;
29 | pub use streaming_server_new::DbStreamingServerNew;


--------------------------------------------------------------------------------
/src/api/data/mod.rs:
--------------------------------------------------------------------------------
 1 | mod api_config;
 2 | mod api_language;
 3 | mod api_streaming_server;
 4 | mod api_country;
 5 | mod result_message;
 6 | mod station_add_result;
 7 | mod station_check_step;
 8 | mod station_check;
 9 | mod station_click;
10 | mod station_history;
11 | mod station;
12 | mod status;
13 | 
14 | pub use self::api_config::ApiConfig as ApiConfig;
15 | pub use self::api_language::ApiLanguage as ApiLanguage;
16 | pub use self::api_streaming_server::ApiStreamingServer as ApiStreamingServer;
17 | pub use self::api_country::ApiCountry as ApiCountry;
18 | pub use self::result_message::ResultMessage;
19 | pub use self::station_add_result::StationAddResult;
20 | pub use self::station_check_step::StationCheckStep;
21 | pub use self::station_check::StationCheck;
22 | pub use self::station_check::StationCheckV0;
23 | pub use self::station_click::StationClick;
24 | pub use self::station_click::StationClickV0;
25 | pub use self::station_history::StationHistoryCurrent;
26 | pub use self::station_history::StationHistoryV0;
27 | pub use self::station::Station;
28 | pub use self::station::StationCachedInfo;
29 | pub use self::station::StationV0;
30 | pub use self::status::Status;


--------------------------------------------------------------------------------
/src/db/models/station_check_item.rs:
--------------------------------------------------------------------------------
 1 | use chrono::DateTime;
 2 | use chrono::Utc;
 3 | 
 4 | #[derive(Clone, Debug)]
 5 | pub struct StationCheckItem {
 6 |     pub check_id: i32,
 7 |     pub check_time_iso8601: Option>,
 8 |     pub check_time: String,
 9 |     pub check_uuid: String,
10 | 
11 |     pub station_uuid: String,
12 |     pub source: String,
13 |     pub codec: String,
14 |     pub bitrate: u32,
15 |     pub hls: bool,
16 |     pub check_ok: bool,
17 |     pub url: String,
18 | 
19 |     pub metainfo_overrides_database: bool,
20 |     pub public: Option,
21 |     pub name: Option,
22 |     pub description: Option,
23 |     pub tags: Option,
24 |     pub countrycode: Option,
25 |     pub homepage: Option,
26 |     pub favicon: Option,
27 |     pub loadbalancer: Option,
28 |     pub do_not_index: Option,
29 | 
30 |     pub countrysubdivisioncode: Option,
31 |     pub server_software: Option,
32 |     pub sampling: Option,
33 |     pub timing_ms: u128,
34 |     pub languagecodes: Option,
35 |     pub ssl_error: bool,
36 |     pub geo_lat: Option,
37 |     pub geo_long: Option,
38 | }
39 | 


--------------------------------------------------------------------------------
/src/db/models/extra_info.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Serialize,Deserialize};
 2 | use std::error::Error;
 3 | 
 4 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 5 | pub struct ExtraInfo {
 6 |     pub name: String,
 7 |     pub stationcount: u32,
 8 | }
 9 | 
10 | impl ExtraInfo {
11 |     pub fn new(name: String, stationcount:u32) -> Self {
12 |         return ExtraInfo{
13 |             name,
14 |             stationcount,
15 |         };
16 |     }
17 | 
18 |     pub fn serialize_extra_list_csv(entries: Vec) -> Result> {
19 |         let mut wtr = csv::Writer::from_writer(Vec::new());
20 | 
21 |         for entry in entries {
22 |             wtr.serialize(entry)?;
23 |         }
24 |         
25 |         wtr.flush()?;
26 |         let x: Vec = wtr.into_inner()?;
27 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
28 |     }
29 | 
30 |     pub fn serialize_extra_list(entries: Vec, tag_name: &str) -> std::io::Result {
31 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
32 |         xml.begin_elem("result")?;
33 |         for entry in entries {
34 |             xml.begin_elem(tag_name)?;
35 |             xml.attr_esc("name", &entry.name)?;
36 |             xml.attr_esc("stationcount", &entry.stationcount.to_string())?;
37 |             xml.end_elem()?;
38 |         }
39 |         xml.end_elem()?;
40 |         xml.close()?;
41 |         xml.flush()?;
42 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
43 |     }
44 | }


--------------------------------------------------------------------------------
/src/api/cache/memcached.rs:
--------------------------------------------------------------------------------
 1 | //use super::generic_cache::GenericCache;
 2 | use memcache;
 3 | use std::error::Error;
 4 | 
 5 | #[derive(Debug, Clone)]
 6 | pub struct MemcachedCache {
 7 |     ttl: u16,
 8 |     cache_url: String,
 9 | }
10 | 
11 | impl MemcachedCache {
12 |     pub fn new(cache_url: String, ttl: u16) -> Self {
13 |         MemcachedCache { cache_url, ttl }
14 |     }
15 |     fn get_internal(&self, key: &str) -> Result, Box> {
16 |         let client = memcache::Client::connect(self.cache_url.clone())?;
17 |         let result = client.get(key)?;
18 |         Ok(result)
19 |     }
20 |     fn set_internal(&mut self, key: &str, value: &str, expire: u16) -> Result<(), Box> {
21 |         let client = memcache::Client::connect(self.cache_url.clone())?;
22 |         client.set(key, value, expire.into())?;
23 |         Ok(())
24 |     }
25 |     pub fn get(&self, key: &str) -> Option {
26 |         trace!("GET {}", key);
27 |         let result = self.get_internal(key);
28 |         match result {
29 |             Ok(result) => result,
30 |             Err(err) => {
31 |                 error!("Error on get of memcached value: {}", err);
32 |                 None
33 |             }
34 |         }
35 |     }
36 |     pub fn set(&mut self, key: &str, value: &str) {
37 |         trace!("SET {} {}", key.len(), key);
38 |         let result = self.set_internal(key, value, self.ttl);
39 |         if let Err(err) = result {
40 |             error!("Error on set of memcached value: {}", err);
41 |         }
42 |     }
43 | }
44 | 


--------------------------------------------------------------------------------
/src/config/data_mapping_item.rs:
--------------------------------------------------------------------------------
 1 | use reqwest::Url;
 2 | use serde::Deserialize;
 3 | use std::collections::HashMap;
 4 | use std::error::Error;
 5 | use std::fs::File;
 6 | use std::io::Read;
 7 | 
 8 | #[derive(Debug, Clone, Deserialize)]
 9 | struct DataMappingItem {
10 |     from: String,
11 |     to: String,
12 | }
13 | 
14 | pub fn read_map_csv_file(file_path: &str) -> Result, Box> {
15 |     debug!("read_map_csv_file()");
16 |     match Url::parse(file_path) {
17 |         Ok(url) => {
18 |             debug!("Remote url: {}", url);
19 |             read_map_csv_file_reader(reqwest::blocking::get(url)?)
20 |         }
21 |         Err(_) => {
22 |             debug!("Local path: {}", file_path);
23 |             read_map_csv_file_reader(File::open(file_path)?)
24 |         }
25 |     }
26 | }
27 | 
28 | fn read_map_csv_file_reader(reader: R) -> Result, Box>
29 | where
30 |     R: Read,
31 | {
32 |     let mut rdr = csv::ReaderBuilder::new()
33 |         .has_headers(true)
34 |         .delimiter(b';')
35 |         .comment(Some(b'#'))
36 |         .from_reader(reader);
37 |     let mut r: HashMap = HashMap::new();
38 |     for result in rdr.deserialize() {
39 |         let record: DataMappingItem = result?;
40 |         trace!("loaded record: {:?}", record);
41 |         if !r.contains_key(&record.from) {
42 |             r.insert(record.from, record.to);
43 |         } else {
44 |             error!("Duplicate key in file: {}", record.from);
45 |         }
46 |     }
47 |     Ok(r)
48 | }
49 | 


--------------------------------------------------------------------------------
/install_from_dist.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | set -e
 4 | 
 5 | sudo mkdir -p /usr/bin
 6 | sudo mkdir -p /usr/share/radiobrowser
 7 | sudo mkdir -p /var/log/radiobrowser
 8 | 
 9 | sudo cp target/release/radiobrowser-api-rust /usr/bin/radiobrowser
10 | sudo cp init/radiobrowser.service /etc/systemd/system
11 | sudo cp static/* /usr/share/radiobrowser/
12 | sudo cp etc/config-example.toml /etc/radiobrowser/config-example.toml
13 | sudo cp etc/language-replace.csv /etc/radiobrowser/language-replace.csv
14 | sudo cp etc/tag-replace.csv /etc/radiobrowser/tag-replace.csv
15 | sudo cp etc/language-to-code.csv /etc/radiobrowser/language-to-code.csv
16 | if [ ! -f /etc/radiobrowser/config.toml ]; then
17 |     sudo cp etc/config-example.toml /etc/radiobrowser/config.toml
18 | fi
19 | sudo cp etc/logrotate /etc/logrotate.d/radiobrowser
20 | 
21 | sudo chmod ugo+x /usr/bin/radiobrowser
22 | sudo groupadd --system radiobrowser
23 | sudo useradd --system --no-create-home --home-dir /var/lib/radiobrowser --gid radiobrowser radiobrowser
24 | 
25 | # Create log dir
26 | sudo mkdir -p /var/log/radiobrowser
27 | sudo chown radiobrowser:radiobrowser /var/log/radiobrowser
28 | 
29 | # Create home dir
30 | sudo mkdir -p /var/lib/radiobrowser
31 | sudo chown radiobrowser:radiobrowser /var/lib/radiobrowser
32 | 
33 | sudo systemctl daemon-reload
34 | 
35 | echo "Enable service with:"
36 | echo " - systemctl enable radiobrowser"
37 | echo "Start service with:"
38 | echo " - systemctl start radiobrowser"
39 | echo "Logs:"
40 | echo " - journalctl -fu radiobrowser"
41 | echo "Edit /etc/radiobrowser/config.toml to according to your needs."


--------------------------------------------------------------------------------
/src/db/models/state.rs:
--------------------------------------------------------------------------------
 1 | use std::error::Error;
 2 | use serde::{Serialize,Deserialize};
 3 | 
 4 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 5 | pub struct State {
 6 |     name: String,
 7 |     country: String,
 8 |     stationcount: u32,
 9 | }
10 | 
11 | impl State {
12 |     pub fn new(name: String, country: String, stationcount: u32) -> Self {
13 |         State {
14 |             name,
15 |             country,
16 |             stationcount,
17 |         }
18 |     }
19 | 
20 |     pub fn serialize_state_list_csv(entries: Vec) -> Result> {
21 |         let mut wtr = csv::Writer::from_writer(Vec::new());
22 | 
23 |         for entry in entries {
24 |             wtr.serialize(entry)?;
25 |         }
26 |         
27 |         wtr.flush()?;
28 |         let x: Vec = wtr.into_inner()?;
29 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
30 |     }
31 | 
32 |     pub fn serialize_state_list(entries: Vec) -> std::io::Result {
33 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
34 |         xml.begin_elem("result")?;
35 |         for entry in entries {
36 |             xml.begin_elem("state")?;
37 |             xml.attr_esc("name", &entry.name)?;
38 |             xml.attr_esc("country", &entry.country)?;
39 |             xml.attr_esc("stationcount", &entry.stationcount.to_string())?;
40 |             xml.end_elem()?;
41 |         }
42 |         xml.end_elem()?;
43 |         xml.close()?;
44 |         xml.flush()?;
45 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
46 |     }
47 | }


--------------------------------------------------------------------------------
/src/api/cache/redis.rs:
--------------------------------------------------------------------------------
 1 | //use super::generic_cache::GenericCache;
 2 | use redis;
 3 | use redis::Commands;
 4 | use std::error::Error;
 5 | 
 6 | #[derive(Debug, Clone)]
 7 | pub struct RedisCache {
 8 |     ttl: u16,
 9 |     cache_url: String,
10 | }
11 | 
12 | impl RedisCache {
13 |     pub fn new(cache_url: String, ttl: u16) -> Self {
14 |         RedisCache { cache_url, ttl }
15 |     }
16 |     fn get_internal(&self, key: &str) -> Result, Box> {
17 |         let client = redis::Client::open(self.cache_url.clone())?;
18 |         let mut con = client.get_connection()?;
19 |         let result = con.get(key);
20 |         match result {
21 |             Ok(result) => Ok(Some(result)),
22 |             Err(_) => Ok(None),
23 |         }
24 |     }
25 |     fn set_internal(&mut self, key: &str, value: &str, expire: u16) -> Result<(), Box> {
26 |         let client = redis::Client::open(self.cache_url.clone())?;
27 |         let mut con = client.get_connection()?;
28 |         let expire: usize = expire.into();
29 |         con.set_ex(key, value, expire)?;
30 |         Ok(())
31 |     }
32 |     pub fn get(&self, key: &str) -> Option {
33 |         trace!("GET {}", key);
34 |         let result = self.get_internal(key);
35 |         match result {
36 |             Ok(result) => result,
37 |             Err(err) => {
38 |                 error!("Error on get of redis value: {}", err);
39 |                 None
40 |             }
41 |         }
42 |     }
43 |     pub fn set(&mut self, key: &str, value: &str) {
44 |         trace!("SET {} {}", key.len(), key);
45 |         let result = self.set_internal(key, value, self.ttl);
46 |         if let Err(err) = result {
47 |             error!("Error on set of redis value: {}", err);
48 |         }
49 |     }
50 | }
51 | 


--------------------------------------------------------------------------------
/src/cleanup/mod.rs:
--------------------------------------------------------------------------------
 1 | use crate::DbConnection;
 2 | use std::error::Error;
 3 | 
 4 | pub fn do_cleanup(
 5 |     delete: bool,
 6 |     mut conn_new_style: C,
 7 |     click_valid_timeout: u64,
 8 |     broken_stations_never_working_timeout: u64,
 9 |     broken_stations_timeout: u64,
10 |     checks_timeout: u64,
11 |     clicks_timeout: u64,
12 | ) -> Result<(), Box> where C: DbConnection {
13 |     let checks_hour = conn_new_style.get_station_count_todo(1)?;
14 |     let checks_day = conn_new_style.get_station_count_todo(24)?;
15 |     let stations_broken = conn_new_style.get_station_count_broken()?;
16 |     let stations_working = conn_new_style.get_station_count_working()?;
17 |     let stations_todo = conn_new_style.get_station_count_todo(24)?;
18 |     let stations_deletable_never_worked =
19 |         conn_new_style.get_deletable_never_working(broken_stations_never_working_timeout)?;
20 |     let stations_deletable_were_working =
21 |         conn_new_style.get_deletable_were_working(broken_stations_timeout)?;
22 |     if delete {
23 |         conn_new_style.delete_never_working(broken_stations_never_working_timeout)?;
24 |         conn_new_style.delete_were_working(broken_stations_timeout)?;
25 |         conn_new_style.delete_old_checks(checks_timeout)?;
26 |         conn_new_style.delete_old_clicks(clicks_timeout)?;
27 |         conn_new_style.delete_removed_from_history()?;
28 |         conn_new_style.delete_unused_streaming_servers(24 * 60 * 60)?;
29 |     }
30 | 
31 |     conn_new_style.update_stations_clickcount()?;
32 |     conn_new_style.remove_unused_ip_infos_from_stationclicks(click_valid_timeout)?;
33 |     conn_new_style.calc_country_field()?;
34 | 
35 |     info!("STATS: {} Checks/Hour, {} Checks/Day, {} Working stations, {} Broken stations, {} to do, deletable {} + {}", checks_hour, checks_day, stations_working, stations_broken, stations_todo, stations_deletable_never_worked, stations_deletable_were_working);
36 |     Ok(())
37 | }
38 | 


--------------------------------------------------------------------------------
/src/api/data/station_add_result.rs:
--------------------------------------------------------------------------------
 1 | use crate::api::api_response::ApiResponse;
 2 | use std::error::Error;
 3 | use serde::{Serialize,Deserialize};
 4 | 
 5 | #[derive(Serialize, Deserialize)]
 6 | pub struct StationAddResult {
 7 |     ok: bool,
 8 |     message: String,
 9 |     uuid: String,
10 | }
11 | 
12 | impl StationAddResult {
13 |     pub fn new_ok(stationuuid: String) -> StationAddResult {
14 |         StationAddResult {
15 |             ok: true,
16 |             message: "added station successfully".to_string(),
17 |             uuid: stationuuid,
18 |         }
19 |     }
20 | 
21 |     pub fn new_err(err: &str) -> StationAddResult {
22 |         StationAddResult {
23 |             ok: false,
24 |             message: err.to_string(),
25 |             uuid: "".to_string(),
26 |         }
27 |     }
28 | 
29 |     pub fn serialize_xml(&self) -> std::io::Result {
30 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
31 |         xml.begin_elem("result")?;
32 |         xml.begin_elem("status")?;
33 |         xml.attr_esc("ok", &self.ok.to_string())?;
34 |         xml.attr_esc("message", &self.ok.to_string())?;
35 |         xml.attr_esc("uuid", &self.uuid)?;
36 |         xml.end_elem()?;
37 |         xml.end_elem()?;
38 |         xml.close()?;
39 |         xml.flush()?;
40 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
41 |     }
42 | 
43 |     pub fn from(result: Result>) -> StationAddResult {
44 |         match result {
45 |             Ok(res) => StationAddResult::new_ok(res),
46 |             Err(err) => StationAddResult::new_err(&err.to_string()),
47 |         }
48 |     }
49 | 
50 |     pub fn get_response(&self, format: &str) -> Result> {
51 |         Ok(match format {
52 |             "json" => ApiResponse::Text(serde_json::to_string(&self)?),
53 |             "xml" => ApiResponse::Text(self.serialize_xml()?),
54 |             _ => ApiResponse::UnknownContentType,
55 |         })
56 |     }
57 | }
58 | 


--------------------------------------------------------------------------------
/ansible/roles/radiobrowser/tasks/main.yml:
--------------------------------------------------------------------------------
 1 | - name: Reload package cache
 2 |   apt:
 3 |     update_cache: yes
 4 |     upgrade: "yes"
 5 | - name: Install apache2, mysql, certbot, fail2ban
 6 |   apt:
 7 |     name: apache2,mariadb-server,python3-pymysql,certbot,python3-certbot-apache
 8 | - name: Create a new database with name 'radio'
 9 |   mysql_db:
10 |     name: radio
11 |     state: present
12 |     login_unix_socket: /var/run/mysqld/mysqld.sock
13 | - name: Create database user
14 |   mysql_user:
15 |     name: radiouser
16 |     password: password
17 |     priv: 'radio.*:ALL'
18 |     state: present
19 |     login_unix_socket: /var/run/mysqld/mysqld.sock
20 | - name: Install a .deb package from the internet.
21 |   apt:
22 |     deb: "https://github.com/segler-alex/radiobrowser-api-rust/releases/download/{{version}}/radiobrowser-api-rust_{{version}}_amd64.deb"
23 | - name: Create www root
24 |   file:
25 |     state: directory
26 |     path: /var/www/radio
27 | - name: Add apache2 virtual host
28 |   template:
29 |     src: ../templates/radio-browser.conf.j2
30 |     dest: /etc/apache2/sites-available/radio-browser.conf
31 | - name: Enable apache2 module proxy
32 |   apache2_module:
33 |     state: present
34 |     name: proxy_http
35 | - name: Enable apache2 module HTTP/2
36 |   apache2_module:
37 |     state: present
38 |     name: http2
39 | - name: HTTP2 protocol config
40 |   lineinfile:
41 |     path: /etc/apache2/apache2.conf
42 |     regexp: '^Protocols h2'
43 |     line: Protocols h2 h2c http/1.1
44 | - name: Disable logging
45 |   command: a2disconf other-vhosts-access-log
46 | - name: Disable default site
47 |   command: a2dissite 000-default
48 | - name: Enable radiobrowser site
49 |   command: a2ensite radio-browser
50 | - name: Reload service apache2
51 |   service:
52 |     name: apache2
53 |     state: reloaded
54 | - name: Enable radiobrowser service
55 |   service:
56 |     name: radiobrowser
57 |     state: restarted
58 |     enabled: yes
59 | - name: Setup certbot
60 |   command: certbot --apache --agree-tos -m {{email}} -d {{inventory_hostname}} -n --redirect


--------------------------------------------------------------------------------
/.gitlab-ci.yml:
--------------------------------------------------------------------------------
 1 | stages:
 2 |   - build
 3 |   - upload
 4 |   - release
 5 | 
 6 | variables:
 7 |   PKG_DIST: "radiobrowser-dist.tar.gz"
 8 |   PKG_DEB: "radiobrowser-api-rust_${CI_COMMIT_TAG}_amd64.deb"
 9 |   PACKAGE_REGISTRY_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/releases/${CI_COMMIT_TAG}"
10 | 
11 | build:release:
12 |   image: "rust:latest"
13 |   stage: build
14 |   rules:
15 |     - if: $CI_COMMIT_TAG
16 |   script:
17 |     - rustc --version && cargo --version
18 |     - cargo install cargo-deb
19 |     - cargo build --release
20 |     - cargo deb
21 |     - bash -x builddist.sh
22 |   artifacts:
23 |     paths:
24 |       - ${PKG_DIST}
25 |       - target/debian/${PKG_DEB}
26 | 
27 | build:docker:
28 |   image: docker:latest
29 |   stage: build
30 |   rules:
31 |     - if: $CI_COMMIT_TAG
32 |   services:
33 |     - docker:dind
34 |   before_script:
35 |     - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
36 |   script:
37 |     - docker build --pull -t "$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG}" .
38 |     - docker push "$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG}"
39 |     - docker tag "$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG}" "$CI_REGISTRY_IMAGE:latest"
40 |     - docker push "$CI_REGISTRY_IMAGE:latest"
41 | 
42 | upload:
43 |   stage: upload
44 |   image: curlimages/curl:latest
45 |   rules:
46 |     - if: $CI_COMMIT_TAG
47 |   script:
48 |     - |
49 |       curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file ${PKG_DIST} "${PACKAGE_REGISTRY_URL}/${PKG_DIST}"
50 |     - |
51 |       curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file target/debian/${PKG_DEB} "${PACKAGE_REGISTRY_URL}/${PKG_DEB}"
52 | 
53 | release:
54 |   stage: release
55 |   image: registry.gitlab.com/gitlab-org/release-cli:latest
56 |   rules:
57 |     - if: $CI_COMMIT_TAG
58 |   script:
59 |     - |
60 |       release-cli create --name "Release $CI_COMMIT_TAG" --tag-name $CI_COMMIT_TAG \
61 |         --assets-link "{\"name\":\"${DARWIN_AMD64_BINARY}\",\"url\":\"${PACKAGE_REGISTRY_URL}/${PKG_DIST}\"}" \
62 |         --assets-link "{\"name\":\"${LINUX_AMD64_BINARY}\",\"url\":\"${PACKAGE_REGISTRY_URL}/${PKG_DEB}\"}"
63 | 


--------------------------------------------------------------------------------
/src/checkserver/mod.rs:
--------------------------------------------------------------------------------
 1 | use crate::db::models::DbStreamingServer;
 2 | use crate::db::DbConnection;
 3 | use icecast_stats::generate_icecast_stats_url;
 4 | use icecast_stats::IcecastStatsRoot;
 5 | use rayon::prelude::*;
 6 | use reqwest::blocking::get;
 7 | use url::Url;
 8 | 
 9 | fn single_check(server: &mut DbStreamingServer) -> Result<(), String> {
10 |     let u = Url::parse(&server.url).or(Err(String::from("URLParseError")))?;
11 |     let u = generate_icecast_stats_url(u);
12 |     let result = get(u.clone()).or(Err(String::from("FetchError")))?;
13 |     let t: IcecastStatsRoot = result.json().or(Err(String::from("ResultDecodeError")))?;
14 |     let j = serde_json::to_string(&t).or(Err(String::from("ResultToJsonError")))?;
15 |     server.status = Some(j);
16 |     server.statusurl = Some(u.to_string());
17 |     Ok(())
18 | }
19 | 
20 | pub fn do_check(
21 |     mut conn_new_style: C,
22 |     chunksize: u32,
23 |     concurrency: usize,
24 | ) -> Result<(), Box>
25 | where
26 |     C: DbConnection,
27 | {
28 |     trace!("do_check()");
29 |     let servers: Vec = conn_new_style.get_servers_to_check(24, chunksize)?;
30 | 
31 |     let pool = rayon::ThreadPoolBuilder::new()
32 |         .num_threads(concurrency)
33 |         .build()?;
34 |     let updated_streaming_servers: Vec<_> = pool.install(|| {
35 |         servers
36 |             .into_par_iter()
37 |             .map(|mut server| {
38 |                 trace!("checking {}", server.url);
39 |                 server.status = None;
40 |                 server.statusurl = None;
41 |                 match single_check(&mut server) {
42 |                     Ok(_) => {
43 |                         debug!("found icecast url at {}", server.url);
44 |                     }
45 |                     Err(err) => {
46 |                         trace!("{}: {}", err, server.url);
47 |                         server.error = Some(err);
48 |                     }
49 |                 };
50 |                 server
51 |             })
52 |             .collect()
53 |     });
54 | 
55 |     conn_new_style.update_streaming_servers(updated_streaming_servers)?;
56 |     trace!("do_check() finished");
57 | 
58 |     Ok(())
59 | }
60 | 


--------------------------------------------------------------------------------
/src/api/data/status.rs:
--------------------------------------------------------------------------------
 1 | use serde::{Serialize,Deserialize};
 2 | 
 3 | #[derive(Serialize, Deserialize)]
 4 | pub struct Status {
 5 |     pub supported_version: u32,
 6 |     pub software_version: Option,
 7 |     status: String,
 8 |     stations: u64,
 9 |     stations_broken: u64,
10 |     tags: u64,
11 |     clicks_last_hour: u64,
12 |     clicks_last_day: u64,
13 |     languages: u64,
14 |     countries: u64,
15 | }
16 | 
17 | impl Status{
18 |     pub fn new(
19 |         supported_version: u32,
20 |         software_version: Option,
21 |         status: String,
22 |         stations: u64,
23 |         stations_broken: u64,
24 |         tags: u64,
25 |         clicks_last_hour: u64,
26 |         clicks_last_day: u64,
27 |         languages: u64,
28 |         countries: u64
29 |     ) -> Self {
30 |         Status{
31 |             supported_version,
32 |             software_version,
33 |             status,
34 |             stations,
35 |             stations_broken,
36 |             tags,
37 |             clicks_last_hour,
38 |             clicks_last_day,
39 |             languages,
40 |             countries,
41 |         }
42 |     }
43 | 
44 |     pub fn serialize_xml(&self) -> std::io::Result {
45 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
46 |         xml.begin_elem("result")?;
47 |         {
48 |             xml.begin_elem("stats")?;
49 |             let s = self.status.clone();
50 |                 xml.attr_esc("supported_version", &self.supported_version.to_string())?;
51 |                 if let Some(software_version) = &self.software_version {
52 |                     xml.attr_esc("software_version", &software_version)?;
53 |                 }
54 |                 xml.attr_esc("status", &s)?;
55 |                 xml.attr_esc("stations", &self.stations.to_string())?;
56 |                 xml.attr_esc("stations_broken", &self.stations_broken.to_string())?;
57 |                 xml.attr_esc("tags", &self.tags.to_string())?;
58 |                 xml.attr_esc("clicks_last_hour", &self.clicks_last_hour.to_string())?;
59 |                 xml.attr_esc("clicks_last_day", &self.clicks_last_day.to_string())?;
60 |                 xml.attr_esc("languages", &self.languages.to_string())?;
61 |                 xml.attr_esc("countries", &self.countries.to_string())?;
62 |             xml.end_elem()?;
63 |         }
64 |         xml.end_elem()?;
65 |         xml.close()?;
66 |         xml.flush()?;
67 |         Ok(String::from_utf8(xml.into_inner()).unwrap())
68 |     }
69 | }


--------------------------------------------------------------------------------
/src/api/data/api_streaming_server.rs:
--------------------------------------------------------------------------------
 1 | use crate::api::ApiResponse;
 2 | use crate::db::models::DbStreamingServer;
 3 | use serde::{Deserialize, Serialize};
 4 | use std::error::Error;
 5 | 
 6 | #[derive(PartialEq, Serialize, Deserialize, Debug)]
 7 | pub struct ApiStreamingServer {
 8 |     pub uuid: String,
 9 |     pub url: String,
10 |     pub statusurl: Option,
11 |     pub status: Option,
12 |     pub error: Option,
13 | }
14 | 
15 | impl From for ApiStreamingServer {
16 |     fn from(item: DbStreamingServer) -> Self {
17 |         ApiStreamingServer {
18 |             uuid: item.uuid,
19 |             url: item.url,
20 |             statusurl: item.statusurl,
21 |             status: item.status,
22 |             error: item.error,
23 |         }
24 |     }
25 | }
26 | 
27 | impl ApiStreamingServer {
28 |     fn serialize_servers(servers: I) -> std::io::Result
29 |     where
30 |         I: IntoIterator,
31 |         P: Into,
32 |     {
33 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
34 |         xml.begin_elem("streamingservers")?;
35 |         for server in servers {
36 |             let server: ApiStreamingServer = server.into();
37 |             xml.begin_elem("streamingserver")?;
38 |             xml.elem_text("uuid", &server.uuid.to_string())?;
39 |             xml.elem_text("url", &server.url.to_string())?;
40 |             if let Some(statusurl) = server.statusurl {
41 |                 xml.elem_text("statusurl", &statusurl)?;
42 |             }
43 |             if let Some(error) = server.error {
44 |                 xml.elem_text("error", &error)?;
45 |             }
46 |             if let Some(status) = server.status {
47 |                 xml.begin_elem("status")?;
48 |                 xml.cdata(&status)?;
49 |                 xml.end_elem()?;
50 |             }
51 |             xml.end_elem()?;
52 |         }
53 |         xml.end_elem()?;
54 |         xml.close()?;
55 |         xml.flush()?;
56 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
57 |     }
58 | 
59 |     pub fn get_response(servers: I, format: &str) -> Result>
60 |     where
61 |         I: IntoIterator + Serialize,
62 |         P: Into,
63 |     {
64 |         let servers: Vec =
65 |             servers.into_iter().map(|server| server.into()).collect();
66 |         Ok(match format {
67 |             "json" => ApiResponse::Text(serde_json::to_string(&servers)?),
68 |             "xml" => ApiResponse::Text(ApiStreamingServer::serialize_servers(servers)?),
69 |             _ => ApiResponse::UnknownContentType,
70 |         })
71 |     }
72 | }
73 | 


--------------------------------------------------------------------------------
/src/config/config.rs:
--------------------------------------------------------------------------------
 1 | use std::time::Duration;
 2 | 
 3 | #[derive(Debug, Clone)]
 4 | pub enum CacheType {
 5 |     None,
 6 |     BuiltIn,
 7 |     Redis,
 8 |     Memcached,
 9 | }
10 | 
11 | impl From for String {
12 |     fn from(c: CacheType) -> Self {
13 |         match c {
14 |             CacheType::None => String::from("none"),
15 |             CacheType::BuiltIn => String::from("builtin"),
16 |             CacheType::Redis => String::from("redis"),
17 |             CacheType::Memcached => String::from("memcached"),
18 |         }
19 |     }
20 | }
21 | 
22 | #[derive(Debug, Clone)]
23 | pub struct Config {
24 |     pub allow_database_downgrade: bool,
25 |     pub broken_stations_never_working_timeout: Duration,
26 |     pub broken_stations_timeout: Duration,
27 |     pub check_stations: u32,
28 |     pub checks_timeout: Duration,
29 |     pub click_valid_timeout: Duration,
30 |     pub clicks_timeout: Duration,
31 |     pub concurrency: usize,
32 |     pub connection_string: String,
33 |     pub delete: bool,
34 |     pub enable_check: bool,
35 |     pub no_migrations: bool,
36 |     pub ignore_migration_errors: bool,
37 |     pub listen_host: String,
38 |     pub listen_port: i32,
39 |     pub log_dir: String,
40 |     pub log_level: usize,
41 |     pub log_json: bool,
42 |     pub max_depth: u8,
43 |     pub mirror_pull_interval: Duration,
44 |     pub pause: Duration,
45 |     pub prometheus_exporter_prefix: String,
46 |     pub prometheus_exporter: bool,
47 |     pub retries: u8,
48 |     pub server_url: String,
49 |     pub servers_pull: Vec,
50 |     pub source: String,
51 |     pub server_location: String,
52 |     pub server_country_code: String,
53 |     pub static_files_dir: String,
54 |     pub tcp_timeout: Duration,
55 |     pub threads: usize,
56 |     pub update_caches_interval: Duration,
57 |     pub useragent: String,
58 |     pub cache_type: CacheType,
59 |     pub cache_url: String,
60 |     pub cache_ttl: Duration,
61 |     pub chunk_size_changes: usize,
62 |     pub chunk_size_checks: usize,
63 |     pub max_duplicates: usize,
64 |     pub check_servers: bool,
65 |     pub check_servers_chunksize: u32,
66 |     pub language_replace_filepath: String,
67 |     pub language_to_code_filepath: String,
68 |     pub tag_replace_filepath: String,
69 |     pub enable_extract_favicon: bool,
70 |     pub recheck_existing_favicon: bool,
71 |     pub favicon_size_min: usize,
72 |     pub favicon_size_max: usize,
73 |     pub favicon_size_optimum: usize,
74 |     pub refresh_config_interval: Duration,
75 |     pub cleanup_interval: Duration,
76 |     pub sub_command: ConfigSubCommand,
77 | }
78 | 
79 | #[derive(Debug, Clone)]
80 | pub enum ConfigSubCommand {
81 |     None,
82 |     Migrate,
83 |     ResetHistory,
84 |     CleanHistory,
85 | }
86 | 


--------------------------------------------------------------------------------
/src/api/cache/builtin.rs:
--------------------------------------------------------------------------------
 1 | use std::sync::Arc;
 2 | use std::sync::Mutex;
 3 | //use super::generic_cache::GenericCache;
 4 | use std::collections::HashMap;
 5 | use std::time::{Duration, SystemTime};
 6 | 
 7 | #[derive(Debug)]
 8 | pub struct Item {
 9 |     value: String,
10 |     expire: SystemTime,
11 | }
12 | 
13 | impl Item {
14 |     pub fn new(value: String, expire: u16) -> Self {
15 |         Item {
16 |             value,
17 |             expire: SystemTime::now() + Duration::new(expire.into(), 0),
18 |         }
19 |     }
20 | }
21 | 
22 | #[derive(Debug, Clone)]
23 | pub struct BuiltinCache {
24 |     ttl: u16,
25 |     cache: Arc>>,
26 | }
27 | 
28 | impl BuiltinCache {
29 |     pub fn new(ttl: u16) -> Self {
30 |         BuiltinCache {
31 |             ttl,
32 |             cache: Arc::new(Mutex::new(HashMap::new())),
33 |         }
34 |     }
35 |     pub fn get(&self, key: &str) -> Option {
36 |         trace!("GET {}", key);
37 |         let locked = self.cache.lock();
38 |         match locked {
39 |             Ok(locked) => {
40 |                 let cached_value = locked.get(key);
41 |                 match cached_value {
42 |                     Some(item) => {
43 |                         let now = SystemTime::now();
44 |                         if item.expire > now {
45 |                             Some(item.value.clone())
46 |                         } else {
47 |                             None
48 |                         }
49 |                     }
50 |                     None => None,
51 |                 }
52 |             },
53 |             Err(err) => {
54 |                 error!("Unable to lock counter for get: {}", err);
55 |                 None
56 |             }
57 |         }
58 |     }
59 |     pub fn set(&mut self, key: &str, value: &str) {
60 |         trace!("SET {}", key);
61 |         let locked = self.cache.lock();
62 |         match locked {
63 |             Ok(mut locked) => {
64 |                 locked.remove(key);
65 |                 locked.insert(key.to_string(), Item::new(value.into(), self.ttl));
66 |             }
67 |             Err(err) => {
68 |                 error!("Unable to lock counter for set: {}", err);
69 |             }
70 |         }
71 |     }
72 |     pub fn cleanup(&mut self) {
73 |         let locked = self.cache.lock();
74 |         match locked {
75 |             Ok(mut locked) => {
76 |                 let now = SystemTime::now();
77 |                 let mut to_delete: Vec = vec![];
78 |                 for (key, value) in locked.iter() {
79 |                     if value.expire <= now {
80 |                         to_delete.push(key.clone());
81 |                     }
82 |                 }
83 | 
84 |                 for key in to_delete {
85 |                     locked.remove(&key);
86 |                 }
87 |             }
88 |             Err(err) => {
89 |                 error!("Unable to lock counter for set: {}", err);
90 |             }
91 |         }
92 |     }
93 | }
94 | 


--------------------------------------------------------------------------------
/src/api/all_params.rs:
--------------------------------------------------------------------------------
 1 | use std::error::Error;
 2 | use serde_with::skip_serializing_none;
 3 | use serde::{Serialize,Deserialize};
 4 | 
 5 | #[skip_serializing_none]
 6 | #[derive(Serialize, Deserialize)]
 7 | pub struct AllParameters {
 8 |     #[serde(rename = "u")]
 9 |     pub url: String,
10 |     #[serde(rename = "ul")]
11 |     pub param_uuids: Vec,
12 |     #[serde(rename = "ts")]
13 |     pub param_tags: Option,
14 |     #[serde(rename = "hp")]
15 |     pub param_homepage: Option,
16 |     #[serde(rename = "fv")]
17 |     pub param_favicon: Option,
18 |     #[serde(rename = "aid")]
19 |     pub param_last_changeuuid: Option,
20 |     #[serde(rename = "eid")]
21 |     pub param_last_checkuuid: Option,
22 |     #[serde(rename = "iid")]
23 |     pub param_last_clickuuid: Option,
24 |     #[serde(rename = "na")]
25 |     pub param_name: Option,
26 |     #[serde(rename = "nx")]
27 |     pub param_name_exact: bool,
28 |     #[serde(rename = "c")]
29 |     pub param_country: Option,
30 |     #[serde(rename = "cx")]
31 |     pub param_country_exact: bool,
32 |     #[serde(rename = "cc")]
33 |     pub param_countrycode: Option,
34 |     #[serde(rename = "st")]
35 |     pub param_state: Option,
36 |     #[serde(rename = "sx")]
37 |     pub param_state_exact: bool,
38 |     #[serde(rename = "lg")]
39 |     pub param_language: Option,
40 |     #[serde(rename = "lc")]
41 |     pub param_language_codes: Option,
42 |     #[serde(rename = "lx")]
43 |     pub param_language_exact: bool,
44 |     #[serde(rename = "tg")]
45 |     pub param_tag: Option,
46 |     #[serde(rename = "tx")]
47 |     pub param_tag_exact: bool,
48 |     #[serde(rename = "tl")]
49 |     pub param_tag_list: Vec,
50 |     #[serde(rename = "co")]
51 |     pub param_codec: Option,
52 |     #[serde(rename = "bi")]
53 |     pub param_bitrate_min: u32,
54 |     #[serde(rename = "ba")]
55 |     pub param_bitrate_max: u32,
56 |     #[serde(rename = "or")]
57 |     pub param_order: String,
58 |     #[serde(rename = "re")]
59 |     pub param_reverse: bool,
60 |     #[serde(rename = "hb")]
61 |     pub param_hidebroken: bool,
62 |     #[serde(rename = "hg")]
63 |     pub param_has_geo_info: Option,
64 |     #[serde(rename = "hx")]
65 |     pub param_has_extended_info: Option,
66 |     #[serde(rename = "hs")]
67 |     pub param_is_https: Option,
68 |     #[serde(rename = "of")]
69 |     pub param_offset: u32,
70 |     #[serde(rename = "li")]
71 |     pub param_limit: u32,
72 |     #[serde(rename = "se")]
73 |     pub param_seconds: u32,
74 |     #[serde(rename = "up")]
75 |     pub param_url: Option,
76 |     #[serde(rename = "ga")]
77 |     pub param_geo_lat: Option,
78 |     #[serde(rename = "go")]
79 |     pub param_geo_long: Option,
80 | }
81 | 
82 | impl AllParameters {
83 |     pub fn to_string(&self) -> Result> {
84 |         Ok(serde_json::to_string(&self)?)
85 |     }
86 | }
87 | 


--------------------------------------------------------------------------------
/src/cli.rs:
--------------------------------------------------------------------------------
 1 | use crate::db::models::StationHistoryItem;
 2 | use crate::db::DbConnection;
 3 | use crate::db::MysqlConnection;
 4 | use std::error::Error;
 5 | 
 6 | fn change_duplicated(item1: &StationHistoryItem, item2: &StationHistoryItem) -> bool {
 7 |     if item1.countrycode.eq(&item2.countrycode)
 8 |         && item1.favicon.eq(&item2.favicon)
 9 |         && item1.geo_lat.eq(&item2.geo_lat)
10 |         && item1.geo_long.eq(&item2.geo_long)
11 |         && item1.homepage.eq(&item2.homepage)
12 |         && item1.language.eq(&item2.language)
13 |         && item1.languagecodes.eq(&item2.languagecodes)
14 |         && item1.name.eq(&item2.name)
15 |         && item1.state.eq(&item2.state)
16 |         && item1.tags.eq(&item2.tags)
17 |         && item1.url.eq(&item2.url)
18 |     {
19 |         return true;
20 |     }
21 |     false
22 | }
23 | 
24 | pub fn delete_duplicate_changes(
25 |     conn: &mut MysqlConnection,
26 |     min_change_count: u32,
27 | ) -> Result<(), Box> {
28 |     debug!("delete_duplicate_changes({})", min_change_count);
29 |     let station_uuids = conn.get_stations_uuid_order_by_changes(min_change_count)?;
30 |     let mut change_uuid_to_delete = vec![];
31 |     let mut change_count = 0;
32 |     let mut station_uuid_no = 0;
33 |     let stations_uuids_count = station_uuids.len();
34 |     for station_uuid in station_uuids {
35 |         station_uuid_no += 1;
36 |         let changes = conn.get_changes(Some(station_uuid.clone()), None, 1000)?;
37 |         //let mut duplicates_current_change = 0;
38 |         change_count += changes.len();
39 |         if changes.len() > 0 {
40 |             let mut current = &changes[0];
41 |             for change_no in 1..changes.len() {
42 |                 if change_duplicated(current, &changes[change_no]) {
43 |                     //trace!("duplicate found");
44 |                     //duplicates_current_change += 1;
45 |                     change_uuid_to_delete.push(changes[change_no].changeuuid.clone());
46 |                 } else {
47 |                     current = &changes[change_no];
48 |                 }
49 |                 print!(
50 |                     "\rduplication check: {}/{} {:03}/{:03} {:06}/{:08}",
51 |                     station_uuid_no,
52 |                     stations_uuids_count,
53 |                     change_no,
54 |                     changes.len(),
55 |                     change_uuid_to_delete.len(),
56 |                     change_count
57 |                 );
58 |             }
59 |         }
60 |         /*
61 |         if duplicates_current_change > 0{
62 |             debug!("duplicates for {} found: {}/{}", station_uuid, duplicates_current_change, changes.len());
63 |         }
64 |         */
65 |     }
66 |     println!();
67 |     info!(
68 |         "duplicates found: {}/{}",
69 |         change_uuid_to_delete.len(),
70 |         change_count
71 |     );
72 |     conn.delete_change_by_uuid(&change_uuid_to_delete)?;
73 |     info!("duplicates deleted: {}", change_uuid_to_delete.len());
74 |     Ok(())
75 | }
76 | 
77 | pub fn resethistory(conn: &mut MysqlConnection) -> Result<(), Box> {
78 |     debug!("resethistory()");
79 |     conn.resethistory()?;
80 |     println!("");
81 |     Ok(())
82 | }
83 | 


--------------------------------------------------------------------------------
/src/api/cache/mod.rs:
--------------------------------------------------------------------------------
 1 | mod builtin;
 2 | mod memcached;
 3 | mod redis;
 4 | 
 5 | use std::sync::Arc;
 6 | use std::sync::Mutex;
 7 | 
 8 | pub enum GenericCacheType {
 9 |     None,
10 |     BuiltIn,
11 |     Redis,
12 |     Memcached,
13 | }
14 | 
15 | #[derive(Debug, Clone)]
16 | pub enum GenericCache {
17 |     None,
18 |     BuiltIn(Arc>),
19 |     Redis(redis::RedisCache),
20 |     Memcached(memcached::MemcachedCache),
21 | }
22 | 
23 | impl GenericCache {
24 |     pub fn new(cache_type: GenericCacheType, cache_url: String, ttl: u16) -> Self {
25 |         match cache_type {
26 |             GenericCacheType::None => GenericCache::None,
27 |             GenericCacheType::BuiltIn => {
28 |                 GenericCache::BuiltIn(Arc::new(Mutex::new(builtin::BuiltinCache::new(ttl))))
29 |             }
30 |             GenericCacheType::Redis => GenericCache::Redis(redis::RedisCache::new(cache_url, ttl)),
31 |             GenericCacheType::Memcached => {
32 |                 GenericCache::Memcached(memcached::MemcachedCache::new(cache_url, ttl))
33 |             }
34 |         }
35 |     }
36 |     pub fn set(&mut self, key: &str, value: &str) {
37 |         match self {
38 |             GenericCache::None => {}
39 |             GenericCache::BuiltIn(builtin) => {
40 |                 let builtin_locked = builtin.lock();
41 |                 match builtin_locked {
42 |                     Ok(mut builtin) => {
43 |                         builtin.set(key, value);
44 |                     }
45 |                     Err(err) => {
46 |                         error!("Unable to lock builtin cache: {}", err);
47 |                     }
48 |                 }
49 |             }
50 |             GenericCache::Redis(cache) => {
51 |                 cache.set(key, value);
52 |             }
53 |             GenericCache::Memcached(cache) => {
54 |                 cache.set(key, value);
55 |             }
56 |         };
57 |     }
58 |     pub fn get(&self, key: &str) -> Option {
59 |         match self {
60 |             GenericCache::None => None,
61 |             GenericCache::BuiltIn(builtin) => {
62 |                 let builtin_locked = builtin.lock();
63 |                 match builtin_locked {
64 |                     Ok(builtin) => builtin.get(key),
65 |                     Err(err) => {
66 |                         error!("Unable to lock builtin cache: {}", err);
67 |                         None
68 |                     }
69 |                 }
70 |             }
71 |             GenericCache::Redis(cache) => cache.get(key),
72 |             GenericCache::Memcached(cache) => cache.get(key),
73 |         }
74 |     }
75 |     pub fn cleanup(&mut self) {
76 |         if let GenericCache::BuiltIn(builtin) = self {
77 |             let builtin_locked = builtin.lock();
78 |             match builtin_locked {
79 |                 Ok(mut builtin) => builtin.cleanup(),
80 |                 Err(err) => {
81 |                     error!("Unable to lock builtin cache: {}", err);
82 |                 }
83 |             }
84 |         }
85 |     }
86 |     pub fn needs_cleanup(&self) -> bool {
87 |         if let GenericCache::BuiltIn(_) = self {
88 |             return true;
89 |         }
90 |         return false;
91 |     }
92 | }
93 | 


--------------------------------------------------------------------------------
/src/api/data/api_language.rs:
--------------------------------------------------------------------------------
 1 | use crate::api::api_response::ApiResponse;
 2 | use crate::config::convert_language_to_code;
 3 | use crate::db::models::ExtraInfo;
 4 | use serde::{Deserialize, Serialize};
 5 | use std::error::Error;
 6 | 
 7 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 8 | pub struct ApiLanguage {
 9 |     pub name: String,
10 |     pub iso_639: Option,
11 |     pub stationcount: u32,
12 | }
13 | 
14 | impl ApiLanguage {
15 |     pub fn new(name: String, iso_639: Option, stationcount: u32) -> Self {
16 |         ApiLanguage {
17 |             name,
18 |             iso_639,
19 |             stationcount,
20 |         }
21 |     }
22 | 
23 |     pub fn get_response(list: I, format: &str) -> Result>
24 |     where
25 |         I: IntoIterator,
26 |         P: Into,
27 |     {
28 |         let list = list.into_iter();
29 |         Ok(match format {
30 |             "csv" => ApiResponse::Text(ApiLanguage::serialize_to_csv(list)?),
31 |             "json" => ApiResponse::Text(ApiLanguage::serialize_to_json(list)?),
32 |             "xml" => ApiResponse::Text(ApiLanguage::serialize_to_xml(list)?),
33 |             _ => ApiResponse::UnknownContentType,
34 |         })
35 |     }
36 | 
37 |     fn serialize_to_json(entries: I) -> Result>
38 |     where
39 |         I: IntoIterator,
40 |         P: Into,
41 |     {
42 |         let list: Vec = entries.into_iter().map(|item| item.into()).collect();
43 |         Ok(serde_json::to_string(&list)?)
44 |     }
45 | 
46 |     fn serialize_to_csv(entries: I) -> Result>
47 |     where
48 |         I: IntoIterator,
49 |         P: Into,
50 |     {
51 |         let mut wtr = csv::Writer::from_writer(Vec::new());
52 | 
53 |         for entry in entries {
54 |             let p: ApiLanguage = entry.into();
55 |             wtr.serialize(p)?;
56 |         }
57 |         wtr.flush()?;
58 |         let x: Vec = wtr.into_inner()?;
59 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
60 |     }
61 | 
62 |     fn serialize_to_xml(entries: I) -> std::io::Result
63 |     where
64 |         I: IntoIterator,
65 |         P: Into,
66 |     {
67 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
68 |         xml.begin_elem("result")?;
69 |         for entry in entries {
70 |             let entry: ApiLanguage = entry.into();
71 |             xml.begin_elem("language")?;
72 |             xml.attr_esc("name", &entry.name)?;
73 |             if let Some(iso_639) = entry.iso_639 {
74 |                 xml.attr_esc("iso_639", &iso_639)?;
75 |             }
76 |             xml.attr_esc("stationcount", &entry.stationcount.to_string())?;
77 |             xml.end_elem()?;
78 |         }
79 |         xml.end_elem()?;
80 |         xml.close()?;
81 |         xml.flush()?;
82 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
83 |     }
84 | }
85 | 
86 | impl From for ApiLanguage {
87 |     fn from(item: ExtraInfo) -> Self {
88 |         let codes = convert_language_to_code(&item.name);
89 |         ApiLanguage::new(item.name, codes, item.stationcount)
90 |     }
91 | }
92 | 


--------------------------------------------------------------------------------
/docker-compose-traefik.yml:
--------------------------------------------------------------------------------
 1 | version: "3.2"
 2 | services:
 3 |   api:
 4 |     build: ./
 5 |     image: segleralex/radiobrowser-api-rust:0.7.24
 6 |     deploy:
 7 |       replicas: 1
 8 |       labels:
 9 |         - "traefik.enable=true"
10 |         - "traefik.http.middlewares.compress.compress=true"
11 |         - "traefik.http.middlewares.secureheaders.headers.framedeny=true"
12 |         - "traefik.http.middlewares.secureheaders.headers.stsSeconds=63072000"
13 |         - "traefik.http.routers.apitls.rule=Host(`${SOURCE}`)"
14 |         - "traefik.http.routers.apitls.entrypoints=websecure"
15 |         - "traefik.http.routers.apitls.middlewares=compress,secureheaders"
16 |         - "traefik.http.services.apiservice.loadbalancer.server.port=8080"
17 |     networks:
18 |       - mynet
19 |     environment:
20 |        - SOURCE
21 |        - ENABLE_CHECK
22 |        - DATABASE_URL=mysql://radiouser:password@dbserver/radio
23 |        - HOST=0.0.0.0
24 |        - CACHETYPE=redis
25 |        - CACHETTL=60sec
26 |        - CACHEURL=redis://cacheserver:6379
27 |   cacheserver:
28 |     image: redis
29 |     networks:
30 |       - mynet
31 |   dbserver:
32 |     image: mariadb:10.4
33 |     deploy:
34 |       replicas: 1
35 |     environment:
36 |        - MYSQL_ROOT_PASSWORD=12345678
37 |        - MYSQL_USER=radiouser
38 |        - MYSQL_PASSWORD=password
39 |        - MYSQL_DATABASE=radio
40 |     networks:
41 |       - mynet
42 |     volumes:
43 |       - ./dbdata:/var/lib/mysql
44 |     command: ["mysqld","--character-set-server=utf8mb4","--collation-server=utf8mb4_unicode_ci"]
45 |   reverse-proxy:
46 |     image: traefik:v2.3
47 |     # Enables the web UI and tells Traefik to listen to docker
48 |     command:
49 |       #- "--api=true"
50 |       #- "--api.insecure=true"
51 |       #- "--log.level=INFO"
52 |       - "--providers.docker=true"
53 |       - "--providers.docker.swarmMode=true"
54 |       - "--providers.docker.exposedByDefault=false"
55 |       - "--entryPoints.web.address=:80"
56 |       - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
57 |       - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
58 |       - "--entryPoints.websecure.address=:443"
59 |       - "--entrypoints.websecure.http.tls.certResolver=letsencrypt"
60 |       - "--certificatesresolvers.letsencrypt.acme.email=${EMAIL}"
61 |       - "--certificatesresolvers.letsencrypt.acme.storage=acme.json"
62 |       - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
63 |       - "--certificatesresolvers.letsencrypt.acme.httpChallenge.entryPoint=web"
64 |       - "--providers.file.filename=/traefik-dyn-config.toml"
65 |     deploy:
66 |       placement:
67 |         constraints:
68 |           - node.role == manager
69 |     ports:
70 |       - target: 80
71 |         published: 80
72 |         protocol: tcp
73 |         mode: host
74 |       - target: 443
75 |         published: 443
76 |         protocol: tcp
77 |         mode: host
78 |       #- "8080:8080"
79 |     volumes:
80 |       # So that Traefik can listen to the Docker events
81 |       - /var/run/docker.sock:/var/run/docker.sock
82 |       - ./acme.json:/acme.json
83 |       - ./traefik-dyn-config.toml:/traefik-dyn-config.toml
84 |     networks:
85 |       - mynet
86 | networks:
87 |   mynet:
88 |     driver: "overlay"
89 |     driver_opts:
90 |       encrypted: ""
91 | 


--------------------------------------------------------------------------------
/src/refresh/mod.rs:
--------------------------------------------------------------------------------
 1 | use crate::db::DbConnection;
 2 | use std;
 3 | use std::collections::HashMap;
 4 | 
 5 | pub struct RefreshCacheStatus {
 6 |     old_items: usize,
 7 |     new_items: usize,
 8 |     changed_items: usize,
 9 | }
10 | 
11 | pub fn refresh_cache_items(
12 |     pool: C,
13 |     cache_table_name: &str,
14 |     cache_column_name: &str,
15 |     station_column_name: &str,
16 | ) -> Result>
17 | where
18 |     C: DbConnection,
19 | {
20 |     let items_cached = pool.get_cached_items(cache_table_name, cache_column_name)?;
21 |     let items_current = pool.get_stations_multi_items(station_column_name)?;
22 |     let mut changed = 0;
23 |     let max_cache_item_len = 110;
24 | 
25 |     let mut to_delete = vec![];
26 |     for item_cached in items_cached.keys() {
27 |         if !items_current.contains_key(item_cached) {
28 |             to_delete.push(item_cached);
29 |         }
30 |     }
31 |     pool.remove_from_cache(to_delete, cache_table_name, cache_column_name)?;
32 | 
33 |     let mut to_insert: HashMap<&String, (u32, u32)> = HashMap::new();
34 |     for item_current in items_current.keys() {
35 |         if !items_cached.contains_key(item_current) {
36 |             if item_current.len() < max_cache_item_len {
37 |                 to_insert.insert(
38 |                     item_current,
39 |                     *items_current.get(item_current).unwrap_or(&(0, 0)),
40 |                 );
41 |             } else {
42 |                 debug!(
43 |                     "cached '{}' item too long: '{}'",
44 |                     station_column_name, item_current
45 |                 );
46 |             }
47 |         } else {
48 |             let value_new = *items_current.get(item_current).unwrap_or(&(0, 0));
49 |             let value_old = *items_cached.get(item_current).unwrap_or(&(0, 0));
50 |             if value_old != value_new {
51 |                 pool.update_cache_item(
52 |                     item_current,
53 |                     value_new.0,
54 |                     value_new.1,
55 |                     cache_table_name,
56 |                     cache_column_name,
57 |                 )?;
58 |                 changed = changed + 1;
59 |             }
60 |         }
61 |     }
62 |     pool.insert_to_cache(to_insert, cache_table_name, cache_column_name)?;
63 |     trace!(
64 |         "{}: {} -> {}, Changed: {}",
65 |         station_column_name,
66 |         items_cached.len(),
67 |         items_current.len(),
68 |         changed
69 |     );
70 |     Ok(RefreshCacheStatus {
71 |         old_items: items_cached.len(),
72 |         new_items: items_current.len(),
73 |         changed_items: changed,
74 |     })
75 | }
76 | 
77 | pub fn refresh_all_caches(pool: C) -> Result<(), Box>
78 | where
79 |     C: DbConnection + Clone,
80 | {
81 |     trace!("REFRESH START");
82 |     let tags = refresh_cache_items(pool.clone(), "TagCache", "TagName", "Tags")?;
83 |     let languages = refresh_cache_items(pool, "LanguageCache", "LanguageName", "Language")?;
84 |     debug!(
85 |         "Refresh(Tags={}->{} changed={}, Languages={}->{} changed={})",
86 |         tags.old_items,
87 |         tags.new_items,
88 |         tags.changed_items,
89 |         languages.old_items,
90 |         languages.new_items,
91 |         languages.changed_items
92 |     );
93 |     Ok(())
94 | }
95 | 


--------------------------------------------------------------------------------
/Cargo.toml:
--------------------------------------------------------------------------------
 1 | [package]
 2 | authors = ["segler_alex"]
 3 | description = "Radio-Browser Server with REST API"
 4 | license = "agpl-3.0"
 5 | name = "radiobrowser-api-rust"
 6 | readme = "README.md"
 7 | version = "0.7.24"
 8 | edition = "2021"
 9 | 
10 | [dependencies]
11 | av-stream-info-rust = "0.10.3"
12 | celes = "2.4.0"
13 | chrono = { version = "0.4.31", features = ["serde"] }
14 | clap = { version = "4.4.6", features = ["cargo", "env"] }
15 | csv = "1.2.2"
16 | dns-lookup = "2.0.3"
17 | fern = { version = "0.6.2", features = ["colored"] }
18 | handlebars = "4.4.0"
19 | hostname = "0.3.1"
20 | humantime = "2.1.0"
21 | icecast-stats = { version = "0.1.1" }
22 | log = "0.4.20"
23 | memcache = "0.17.0"
24 | mysql = "24.0.0"
25 | mysql_common = { features = ["chrono"] }
26 | native-tls = "0.2.11"
27 | once_cell = "1.18.0"
28 | percent-encoding = "2.3.0"
29 | prometheus = { version = "0.13.3" }
30 | rayon = "1.8.0"
31 | redis = { version = "0.23.3" }
32 | reqwest = { version = "0.11.20", features = ["blocking", "json"] }
33 | rouille = "3.6.2"
34 | serde = { version = "1.0.188", features = ["derive"] }
35 | serde_json = "1.0.107"
36 | serde_with = "3.3.0"
37 | signal-hook = "0.3.17"
38 | threadpool = "1.8.1"
39 | toml = "0.8.0"
40 | url = "2.4.1"
41 | uuid = { version = "1.4.1", features = ["serde", "v4"] }
42 | website-icon-extract = "0.5.2"
43 | xml_writer = "0.4.0"
44 | 
45 | [package.metadata.deb]
46 | maintainer = "sailor "
47 | copyright = "2018, sailor "
48 | depends = "$auto, systemd"
49 | extended-description = """\
50 | Radio-Browser Server with REST API
51 | 
52 | In short it is an API for an index of web streams (audio and video). Streams can be added and searched by any user of the API.
53 | 
54 | There is an official deployment of this software that is also freely usable at https://api.radio-browser.info
55 | 
56 | ## Features
57 | * Open source
58 | * Freely licensed
59 | * Well documented API
60 | * Automatic regular online checking of streams
61 | * Highliy configurable
62 | * Easy setup for multiple configurations (native, deb-packages, docker, ansible)
63 | * Implemented in Rust-lang
64 | * Multiple request types: query, json, x-www-form-urlencoded, form-data
65 | * Multiple output types: xml, json, m3u, pls, xspf, ttl, csv
66 | * Optional: multi-server setup with automatic mirroring
67 | * Optional: response caching in internal or external cache (redis, memcached)
68 | """
69 | section = "admin"
70 | priority = "optional"
71 | assets = [
72 |     ["target/release/radiobrowser-api-rust", "usr/bin/radiobrowser", "755"],
73 |     ["static/*", "usr/share/radiobrowser/", "644"],
74 |     ["init.sql", "usr/share/radiobrowser/init.sql", "644"],
75 |     ["etc/config-example.toml", "etc/radiobrowser/config-example.toml", "644"],
76 |     ["etc/config-example.toml", "etc/radiobrowser/config.toml", "644"],
77 |     ["etc/language-replace.csv", "etc/radiobrowser/language-replace.csv", "644"],
78 |     ["etc/language-to-code.csv", "etc/radiobrowser/language-to-code.csv", "644"],
79 |     ["etc/tag-replace.csv", "etc/radiobrowser/tag-replace.csv", "644"],
80 |     ["etc/logrotate", "etc/logrotate.d/radiobrowser", "644"],
81 |     ["etc/logrotate", "etc/logrotate.d/radiobrowser", "644"],
82 | ]
83 | conf-files = [
84 |     "etc/radiobrowser/config.toml",
85 | ]
86 | maintainer-scripts = "debian/"
87 | systemd-units = { unit-name = "radiobrowser", enable = false }


--------------------------------------------------------------------------------
/src/api/data/station_check_step.rs:
--------------------------------------------------------------------------------
 1 | use crate::api::api_response::ApiResponse;
 2 | use crate::db::models::StationCheckStepItem;
 3 | use chrono::DateTime;
 4 | use chrono::Utc;
 5 | use serde::{Deserialize, Serialize};
 6 | use std::error::Error;
 7 | 
 8 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 9 | pub struct StationCheckStep {
10 |     pub stepuuid: String,
11 |     pub parent_stepuuid: Option,
12 |     pub checkuuid: String,
13 |     pub stationuuid: String,
14 |     pub url: String,
15 |     pub urltype: Option,
16 |     pub error: Option,
17 |     pub creation_iso8601: DateTime,
18 | }
19 | 
20 | impl StationCheckStep {
21 |     pub fn serialize_station_checks_csv(
22 |         entries: Vec,
23 |     ) -> Result> {
24 |         let mut wtr = csv::Writer::from_writer(Vec::new());
25 | 
26 |         for entry in entries {
27 |             wtr.serialize(entry)?;
28 |         }
29 |         wtr.flush()?;
30 |         let x: Vec = wtr.into_inner()?;
31 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
32 |     }
33 | 
34 |     pub fn serialize_station_checks(entries: Vec) -> std::io::Result {
35 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
36 |         xml.begin_elem("result")?;
37 |         for entry in entries {
38 |             xml.begin_elem("checkstep")?;
39 |             xml.attr_esc("stepuuid", &entry.stepuuid)?;
40 |             if let Some(parent_stepuuid) = entry.parent_stepuuid {
41 |                 xml.attr_esc("parent_stepuuid", &parent_stepuuid)?;
42 |             }
43 |             xml.attr_esc("checkuuid", &entry.checkuuid)?;
44 |             xml.attr_esc("stationuuid", &entry.stationuuid)?;
45 |             xml.attr_esc("url", &entry.url)?;
46 |             if let Some(urltype) = entry.urltype {
47 |                 xml.attr_esc("urltype", &urltype)?;
48 |             }
49 |             if let Some(error) = entry.error {
50 |                 xml.attr_esc("error", &error)?;
51 |             }
52 |             xml.attr_esc("creation_iso8601", &entry.creation_iso8601.to_string())?;
53 |             xml.end_elem()?;
54 |         }
55 |         xml.end_elem()?;
56 |         xml.close()?;
57 |         xml.flush()?;
58 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
59 |     }
60 | 
61 |     pub fn get_response(
62 |         list: Vec,
63 |         format: &str,
64 |     ) -> Result> {
65 |         Ok(match format {
66 |             "csv" => ApiResponse::Text(StationCheckStep::serialize_station_checks_csv(list)?),
67 |             "json" => ApiResponse::Text(serde_json::to_string(&list)?),
68 |             "xml" => ApiResponse::Text(StationCheckStep::serialize_station_checks(list)?),
69 |             _ => ApiResponse::UnknownContentType,
70 |         })
71 |     }
72 | }
73 | impl From for StationCheckStep {
74 |     fn from(item: StationCheckStepItem) -> Self {
75 |         StationCheckStep {
76 |             stepuuid: item.stepuuid,
77 |             parent_stepuuid: item.parent_stepuuid,
78 |             checkuuid: item.checkuuid,
79 |             stationuuid: item.stationuuid,
80 |             url: item.url,
81 |             urltype: item.urltype,
82 |             error: item.error,
83 |             creation_iso8601: item.inserttime,
84 |         }
85 |     }
86 | }
87 | 


--------------------------------------------------------------------------------
/src/api/data/api_country.rs:
--------------------------------------------------------------------------------
 1 | use crate::db::models::DBCountry;
 2 | use crate::api::api_response::ApiResponse;
 3 | use serde::{Deserialize, Serialize};
 4 | use std::error::Error;
 5 | use celes::Country;
 6 | 
 7 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 8 | pub struct ApiCountry {
 9 |     pub name: String,
10 |     pub iso_3166_1: String,
11 |     pub stationcount: u32,
12 | }
13 | 
14 | impl ApiCountry {
15 |     /*
16 |     pub fn new(name: String, iso_3166_1: String, stationcount: u32) -> Self {
17 |         ApiCountry {
18 |             name,
19 |             iso_3166_1,
20 |             stationcount,
21 |         }
22 |     }
23 |     */
24 | 
25 |     pub fn new_with_code(iso_3166_1: String, stationcount: u32) -> Self {
26 |         let name = Country::from_alpha2(&iso_3166_1).map(|d| d.long_name).unwrap_or("");
27 |         ApiCountry {
28 |             name: name.to_string(),
29 |             iso_3166_1,
30 |             stationcount,
31 |         }
32 |     }
33 | 
34 |     pub fn get_response(list: I, format: &str) -> Result>
35 |     where
36 |         I: IntoIterator,
37 |         P: Into,
38 |     {
39 |         let list = list.into_iter();
40 |         Ok(match format {
41 |             "csv" => ApiResponse::Text(ApiCountry::serialize_to_csv(list)?),
42 |             "json" => ApiResponse::Text(ApiCountry::serialize_to_json(list)?),
43 |             "xml" => ApiResponse::Text(ApiCountry::serialize_to_xml(list)?),
44 |             _ => ApiResponse::UnknownContentType,
45 |         })
46 |     }
47 | 
48 |     fn serialize_to_json(entries: I) -> Result>
49 |     where
50 |         I: IntoIterator,
51 |         P: Into,
52 |     {
53 |         let list: Vec = entries.into_iter().map(|item| item.into()).collect();
54 |         Ok(serde_json::to_string(&list)?)
55 |     }
56 | 
57 |     fn serialize_to_csv(entries: I) -> Result>
58 |     where
59 |         I: IntoIterator,
60 |         P: Into,
61 |     {
62 |         let mut wtr = csv::Writer::from_writer(Vec::new());
63 | 
64 |         for entry in entries {
65 |             let p: ApiCountry = entry.into();
66 |             wtr.serialize(p)?;
67 |         }
68 |         wtr.flush()?;
69 |         let x: Vec = wtr.into_inner()?;
70 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
71 |     }
72 | 
73 |     fn serialize_to_xml(entries: I) -> std::io::Result
74 |     where
75 |         I: IntoIterator,
76 |         P: Into,
77 |     {
78 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
79 |         xml.begin_elem("result")?;
80 |         for entry in entries {
81 |             let entry: ApiCountry = entry.into();
82 |             xml.begin_elem("language")?;
83 |             xml.attr_esc("name", &entry.name)?;
84 |             xml.attr_esc("iso_3166_1", &entry.iso_3166_1)?;
85 |             xml.attr_esc("stationcount", &entry.stationcount.to_string())?;
86 |             xml.end_elem()?;
87 |         }
88 |         xml.end_elem()?;
89 |         xml.close()?;
90 |         xml.flush()?;
91 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
92 |     }
93 | }
94 | 
95 | impl From for ApiCountry {
96 |     fn from(item: DBCountry) -> Self {
97 |         ApiCountry::new_with_code(item.iso_3166_1, item.stationcount)
98 |     }
99 | }


--------------------------------------------------------------------------------
/src/db/models/station_check_item_new.rs:
--------------------------------------------------------------------------------
  1 | #[derive(Clone, Debug)]
  2 | pub struct StationCheckItemNew {
  3 |     pub checkuuid: Option,
  4 |     pub station_uuid: String,
  5 |     pub source: String,
  6 |     pub codec: String,
  7 |     pub bitrate: u32,
  8 |     pub sampling: Option,
  9 |     pub hls: bool,
 10 |     pub check_ok: bool,
 11 |     pub url: String,
 12 |     pub timestamp: Option,
 13 | 
 14 |     pub metainfo_overrides_database: bool,
 15 |     pub public: Option,
 16 |     pub name: Option,
 17 |     pub description: Option,
 18 |     pub tags: Option,
 19 |     pub countrycode: Option,
 20 |     pub countrysubdivisioncode: Option,
 21 |     pub languagecodes: Option,
 22 |     pub homepage: Option,
 23 |     pub favicon: Option,
 24 |     pub loadbalancer: Option,
 25 |     pub do_not_index: Option,
 26 |     pub timing_ms: u128,
 27 |     pub server_software: Option,
 28 |     pub ssl_error: bool,
 29 |     pub geo_lat: Option,
 30 |     pub geo_long: Option,
 31 | }
 32 | 
 33 | impl StationCheckItemNew {
 34 |     pub fn broken(station_uuid: String, check_uuid: String, source: String, timing_ms: u128) -> Self {
 35 |         StationCheckItemNew {
 36 |             checkuuid: Some(check_uuid),
 37 |             station_uuid,
 38 |             source,
 39 |             codec: "".to_string(),
 40 |             bitrate: 0,
 41 |             sampling: None,
 42 |             hls: false,
 43 |             check_ok: false,
 44 |             url: "".to_string(),
 45 |             timestamp: None,
 46 |             metainfo_overrides_database: false,
 47 |             public: None,
 48 |             name: None,
 49 |             description: None,
 50 |             tags: None,
 51 |             countrycode: None,
 52 |             countrysubdivisioncode: None,
 53 |             languagecodes: None,
 54 |             homepage: None,
 55 |             favicon: None,
 56 |             loadbalancer: None,
 57 |             do_not_index: None,
 58 |             timing_ms,
 59 |             server_software: None,
 60 |             ssl_error: false,
 61 |             geo_lat: None,
 62 |             geo_long: None,
 63 |         }
 64 |     }
 65 | 
 66 |     pub fn working(
 67 |         station_uuid: String,
 68 |         check_uuid: String,
 69 |         source: String,
 70 |         timing_ms: u128,
 71 |         url: String,
 72 |         info: av_stream_info_rust::StreamInfo,
 73 |     ) -> Self {
 74 |         let mut codec = info.CodecAudio.clone();
 75 |         if let Some(ref video) = info.CodecVideo {
 76 |             codec.push_str(",");
 77 |             codec.push_str(&video);
 78 |         }
 79 |         let latlong = info.GeoLatLong.clone().transpose().unwrap_or(None);
 80 |         StationCheckItemNew {
 81 |             checkuuid: Some(check_uuid),
 82 |             station_uuid,
 83 |             source,
 84 |             codec: codec,
 85 |             bitrate: info.Bitrate.unwrap_or(0),
 86 |             sampling: info.Sampling,
 87 |             hls: info.Hls,
 88 |             check_ok: true,
 89 |             url,
 90 |             timestamp: None,
 91 | 
 92 |             metainfo_overrides_database: info.OverrideIndexMetaData.unwrap_or(false),
 93 |             public: info.Public,
 94 |             name: info.Name,
 95 |             description: info.Description,
 96 |             tags: info.Genre,
 97 |             countrycode: info.CountryCode,
 98 |             countrysubdivisioncode: info.CountrySubdivisonCode,
 99 |             languagecodes: Some(info.LanguageCodes.join(",")),
100 |             homepage: info.Homepage,
101 |             favicon: info.LogoUrl,
102 |             loadbalancer: info.MainStreamUrl,
103 |             do_not_index: info.DoNotIndex,
104 |             timing_ms,
105 |             server_software: info.Server,
106 |             ssl_error: info.SslError,
107 |             geo_lat: latlong.clone().map(|y|y.lat),
108 |             geo_long: latlong.map(|y|y.long),
109 |         }
110 |     }
111 | }
112 | 


--------------------------------------------------------------------------------
/src/api/data/station_click.rs:
--------------------------------------------------------------------------------
  1 | use chrono::NaiveDateTime;
  2 | use chrono::DateTime;
  3 | use chrono::Utc;
  4 | use chrono::SecondsFormat;
  5 | use crate::api::api_response::ApiResponse;
  6 | use crate::db::models::StationClickItem;
  7 | use std::convert::TryFrom;
  8 | use std::error::Error;
  9 | use serde::{Serialize,Deserialize};
 10 | 
 11 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 12 | pub struct StationClickV0 {
 13 |     pub stationuuid: String,
 14 |     pub clickuuid: String,
 15 |     pub clicktimestamp: String,
 16 | }
 17 | 
 18 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 19 | pub struct StationClick {
 20 |     pub stationuuid: String,
 21 |     pub clickuuid: String,
 22 |     pub clicktimestamp_iso8601: Option>,
 23 |     pub clicktimestamp: String,
 24 | }
 25 | 
 26 | impl StationClick {
 27 |     pub fn new(
 28 |         stationuuid: String,
 29 |         clickuuid: String,
 30 |         clicktimestamp_iso8601: Option>,
 31 |         clicktimestamp: String,
 32 |     ) -> Self {
 33 |         StationClick {
 34 |             stationuuid,
 35 |             clickuuid,
 36 |             clicktimestamp_iso8601,
 37 |             clicktimestamp,
 38 |         }
 39 |     }
 40 | 
 41 |     pub fn serialize_station_clicks_csv(entries: Vec) -> Result> {
 42 |         let mut wtr = csv::Writer::from_writer(Vec::new());
 43 | 
 44 |         for entry in entries {
 45 |             wtr.serialize(entry)?;
 46 |         }
 47 |         
 48 |         wtr.flush()?;
 49 |         let x: Vec = wtr.into_inner()?;
 50 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
 51 |     }
 52 | 
 53 |     pub fn serialize_station_clicks(entries: Vec) -> std::io::Result {
 54 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
 55 |         xml.begin_elem("result")?;
 56 |         for entry in entries {
 57 |             xml.begin_elem("click")?;
 58 |             xml.attr_esc("stationuuid", &entry.stationuuid)?;
 59 |             xml.attr_esc("clickuuid", &entry.clickuuid)?;
 60 |             if let Some(clicktimestamp_iso8601) = entry.clicktimestamp_iso8601 {
 61 |                 xml.attr_esc("clicktimestamp_iso8601", &clicktimestamp_iso8601.to_rfc3339_opts(SecondsFormat::Secs, true))?;
 62 |             }
 63 |             xml.attr_esc("clicktimestamp", &entry.clicktimestamp)?;
 64 |             xml.end_elem()?;
 65 |         }
 66 |         xml.end_elem()?;
 67 |         xml.close()?;
 68 |         xml.flush()?;
 69 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
 70 |     }
 71 | 
 72 |     pub fn get_response(list: Vec, format: &str) -> Result> {
 73 |         Ok(match format {
 74 |             "csv" => ApiResponse::Text(StationClick::serialize_station_clicks_csv(list)?),
 75 |             "json" => ApiResponse::Text(serde_json::to_string(&list)?),
 76 |             "xml" => ApiResponse::Text(StationClick::serialize_station_clicks(list)?),
 77 |             _ => ApiResponse::UnknownContentType,
 78 |         })
 79 |     }
 80 | }
 81 | 
 82 | impl TryFrom for StationClick {
 83 |     type Error = Box;
 84 | 
 85 |     fn try_from(item: StationClickV0) -> Result {
 86 |         let clicktimestamp_iso8601 = NaiveDateTime::parse_from_str(&item.clicktimestamp, "%Y-%m-%d %H:%M:%S")
 87 |             .ok()
 88 |             .map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc));
 89 | 
 90 |         Ok(StationClick {
 91 |             stationuuid: item.stationuuid,
 92 |             clickuuid: item.clickuuid,
 93 |             clicktimestamp_iso8601,
 94 |             clicktimestamp: item.clicktimestamp,
 95 |         })
 96 |     }
 97 | }
 98 | 
 99 | impl From for StationClick {
100 |     fn from(item: StationClickItem) -> Self {
101 |         StationClick::new(
102 |             item.stationuuid,
103 |             item.clickuuid,
104 |             item.clicktimestamp_iso8601,
105 |             item.clicktimestamp,
106 |         )
107 |     }
108 | }


--------------------------------------------------------------------------------
/etc/language-to-code.csv:
--------------------------------------------------------------------------------
  1 | # mapping of language codes to language names
  2 | # imported from https://datahub.io/core/language-codes/r/language-codes.csv
  3 | from;to
  4 | "abkhazian";"ab"
  5 | "afar";"aa"
  6 | "afrikaans";"af"
  7 | "akan";"ak"
  8 | "albanian";"sq"
  9 | "amharic";"am"
 10 | "arabic";"ar"
 11 | "aragonese";"an"
 12 | "armenian";"hy"
 13 | "assamese";"as"
 14 | "avaric";"av"
 15 | "avestan";"ae"
 16 | "aymara";"ay"
 17 | "azerbaijani";"az"
 18 | "bambara";"bm"
 19 | "bashkir";"ba"
 20 | "basque";"eu"
 21 | "belarusian";"be"
 22 | "bengali";"bn"
 23 | "bihari languages";"bh"
 24 | "bislama";"bi"
 25 | "bosnian";"bs"
 26 | "breton";"br"
 27 | "bulgarian";"bg"
 28 | "burmese";"my"
 29 | "catalan";"ca"
 30 | "khmer";"km"
 31 | "chamorro";"ch"
 32 | "chechen";"ce"
 33 | "chichewa";"ny"
 34 | "chinese";"zh"
 35 | "church slavic";"cu"
 36 | "chuvash";"cv"
 37 | "cornish";"kw"
 38 | "corsican";"co"
 39 | "cree";"cr"
 40 | "croatian";"hr"
 41 | "czech";"cs"
 42 | "danish";"da"
 43 | "divehi";"dv"
 44 | "dutch";"nl"
 45 | "dzongkha";"dz"
 46 | "english";"en"
 47 | "esperanto";"eo"
 48 | "estonian";"et"
 49 | "ewe";"ee"
 50 | "faroese";"fo"
 51 | "fijian";"fj"
 52 | "finnish";"fi"
 53 | "french";"fr"
 54 | "fulah";"ff"
 55 | "gaelic";"gd"
 56 | "galician";"gl"
 57 | "ganda";"lg"
 58 | "georgian";"ka"
 59 | "german";"de"
 60 | "greek";"el"
 61 | "guarani";"gn"
 62 | "gujarati";"gu"
 63 | "haitian";"ht"
 64 | "hausa";"ha"
 65 | "hebrew";"he"
 66 | "herero";"hz"
 67 | "hindi";"hi"
 68 | "hiri motu";"ho"
 69 | "hungarian";"hu"
 70 | "icelandic";"is"
 71 | "ido";"io"
 72 | "igbo";"ig"
 73 | "indonesian";"id"
 74 | "interlingua";"ia"
 75 | "interlingue";"ie"
 76 | "inuktitut";"iu"
 77 | "inupiaq";"ik"
 78 | "irish";"ga"
 79 | "italian";"it"
 80 | "japanese";"ja"
 81 | "javanese";"jv"
 82 | "kalaallisut";"kl"
 83 | "kannada";"kn"
 84 | "kanuri";"kr"
 85 | "kashmiri";"ks"
 86 | "kazakh";"kk"
 87 | "kikuyu";"ki"
 88 | "kinyarwanda";"rw"
 89 | "kirghiz";"ky"
 90 | "komi";"kv"
 91 | "kongo";"kg"
 92 | "korean";"ko"
 93 | "kuanyama";"kj"
 94 | "kurdish";"ku"
 95 | "lao";"lo"
 96 | "latin";"la"
 97 | "latvian";"lv"
 98 | "limburgan";"li"
 99 | "lingala";"ln"
100 | "lithuanian";"lt"
101 | "luba-katanga";"lu"
102 | "luxembourgish";"lb"
103 | "macedonian";"mk"
104 | "malagasy";"mg"
105 | "malay";"ms"
106 | "malayalam";"ml"
107 | "maltese";"mt"
108 | "manx";"gv"
109 | "maori";"mi"
110 | "marathi";"mr"
111 | "marshallese";"mh"
112 | "mongolian";"mn"
113 | "nauru";"na"
114 | "navajo";"nv"
115 | "ndonga";"ng"
116 | "nepali";"ne"
117 | "north ndebele";"nd"
118 | "northern sami";"se"
119 | "norwegian bokmål";"nb"
120 | "norwegian nynorsk";"nn"
121 | "norwegian";"no"
122 | "occitan";"oc"
123 | "ojibwa";"oj"
124 | "oriya";"or"
125 | "oromo";"om"
126 | "ossetian";"os"
127 | "pali";"pi"
128 | "panjabi";"pa"
129 | "persian";"fa"
130 | "polish";"pl"
131 | "portuguese";"pt"
132 | "pushto";"ps"
133 | "quechua";"qu"
134 | "romanian";"ro"
135 | "romansh";"rm"
136 | "rundi";"rn"
137 | "russian";"ru"
138 | "samoan";"sm"
139 | "sango";"sg"
140 | "sanskrit";"sa"
141 | "sardinian";"sc"
142 | "serbian";"sr"
143 | "shona";"sn"
144 | "sichuan yi";"ii"
145 | "sindhi";"sd"
146 | "sinhala";"si"
147 | "slovak";"sk"
148 | "slovenian";"sl"
149 | "somali";"so"
150 | "sotho, southern";"st"
151 | "south ndebele";"nr"
152 | "spanish";"es"
153 | "sundanese";"su"
154 | "swahili";"sw"
155 | "swati";"ss"
156 | "swedish";"sv"
157 | "tagalog";"tl"
158 | "tahitian";"ty"
159 | "tajik";"tg"
160 | "tamil";"ta"
161 | "tatar";"tt"
162 | "telugu";"te"
163 | "thai";"th"
164 | "tibetan";"bo"
165 | "tigrinya";"ti"
166 | "tonga";"to"
167 | "tsonga";"ts"
168 | "tswana";"tn"
169 | "turkish";"tr"
170 | "turkmen";"tk"
171 | "twi";"tw"
172 | "uighur";"ug"
173 | "ukrainian";"uk"
174 | "urdu";"ur"
175 | "uzbek";"uz"
176 | "venda";"ve"
177 | "vietnamese";"vi"
178 | "volapük";"vo"
179 | "walloon";"wa"
180 | "welsh";"cy"
181 | "western frisian";"fy"
182 | "wolof";"wo"
183 | "xhosa";"xh"
184 | "yiddish";"yi"
185 | "yoruba";"yo"
186 | "zhuang";"za"
187 | "zulu";"zu"
188 | 
189 | "asturian";"ast"
190 | "bavarian";"bar"
191 | "cantonese";"yue"
192 | "hakka";"hak"
193 | "low german";"nds"
194 | "lower sorbian";"dsb"
195 | "mandarin";"cmn"
196 | "swabian";"swg"
197 | "swiss german";"gsw"
198 | "upper sorbian";"hsb"


--------------------------------------------------------------------------------
/src/logger.rs:
--------------------------------------------------------------------------------
  1 | use fern::colors::{Color, ColoredLevelConfig};
  2 | use std::io;
  3 | use serde::{Serialize,Deserialize};
  4 | 
  5 | #[derive(Serialize, Deserialize)]
  6 | struct StructuredLog {
  7 |     timestamp: String,
  8 |     level: String,
  9 |     target: String,
 10 |     message: String,
 11 | }
 12 | 
 13 | pub fn setup_logger(verbosity: usize, log_dir: &str, json: bool) -> Result<(), fern::InitError> {
 14 |     let mut base_config = fern::Dispatch::new();
 15 | 
 16 |     let log_file_path = format!("{}/main.log", log_dir);
 17 | 
 18 |     base_config = match verbosity {
 19 |         0 => base_config
 20 |             .level(log::LevelFilter::Error)
 21 |             .level_for("radiobrowser_api_rust", log::LevelFilter::Error),
 22 |         1 => base_config
 23 |             .level(log::LevelFilter::Warn)
 24 |             .level_for("radiobrowser_api_rust", log::LevelFilter::Warn),
 25 |         2 => base_config
 26 |             .level(log::LevelFilter::Info)
 27 |             .level_for("radiobrowser_api_rust", log::LevelFilter::Info),
 28 |         3 => base_config
 29 |             .level(log::LevelFilter::Info)
 30 |             .level_for("radiobrowser_api_rust", log::LevelFilter::Debug),
 31 |         _4_or_more => base_config
 32 |             .level(log::LevelFilter::Info)
 33 |             .level_for("radiobrowser_api_rust", log::LevelFilter::Trace),
 34 |     };
 35 | 
 36 |     let colors_line = ColoredLevelConfig::new()
 37 |         .error(Color::Red)
 38 |         .warn(Color::Yellow)
 39 |         .info(Color::BrightWhite)
 40 |         .debug(Color::White)
 41 |         .trace(Color::BrightBlack);
 42 | 
 43 |     let file_config = fern::Dispatch::new()
 44 |         .format(move |out, message, record| {
 45 |             if json {
 46 |                 let line = StructuredLog {
 47 |                     timestamp: chrono::Utc::now()
 48 |                         .format("%Y-%m-%dT%H:%M:%S,%f")
 49 |                         .to_string(),
 50 |                     level: record.level().to_string(),
 51 |                     target: record.target().to_string(),
 52 |                     message: message.to_string(),
 53 |                 };
 54 |                 let line_str = serde_json::to_string(&line);
 55 |                 match line_str {
 56 |                     Ok(line_str) => out.finish(format_args!("{}", line_str)),
 57 |                     Err(err) => out.finish(format_args!("Unable to encode log to JSON: {}", err)),
 58 |                 }
 59 |             } else {
 60 |                 out.finish(format_args!(
 61 |                     "{} {} {} {}",
 62 |                     chrono::Utc::now().format("%Y-%m-%dT%H:%M:%S,%f"),
 63 |                     record.level(),
 64 |                     record.target(),
 65 |                     message
 66 |                 ))
 67 |             }
 68 |         })
 69 |         .level(log::LevelFilter::Debug)
 70 |         .chain(fern::log_file(log_file_path)?);
 71 | 
 72 |     let stdout_config = fern::Dispatch::new()
 73 |         .format(move |out, message, record| {
 74 |             if json {
 75 |                 let line = StructuredLog {
 76 |                     timestamp: chrono::Utc::now()
 77 |                         .format("%Y-%m-%dT%H:%M:%S,%f")
 78 |                         .to_string(),
 79 |                     level: record.level().to_string(),
 80 |                     target: record.target().to_string(),
 81 |                     message: message.to_string(),
 82 |                 };
 83 |                 let line_str = serde_json::to_string(&line);
 84 |                 match line_str {
 85 |                     Ok(line_str) => out.finish(format_args!("{}", line_str)),
 86 |                     Err(err) => out.finish(format_args!("Unable to encode log to JSON: {}", err)),
 87 |                 }
 88 |             } else {
 89 |                 out.finish(format_args!(
 90 |                     "{} {} {} {}",
 91 |                     chrono::Utc::now().format("%Y-%m-%dT%H:%M:%S,%f"),
 92 |                     colors_line.color(record.level()),
 93 |                     record.target(),
 94 |                     message
 95 |                 ));
 96 |             }
 97 |         })
 98 |         .chain(io::stdout());
 99 | 
100 |     base_config
101 |         .chain(file_config)
102 |         .chain(stdout_config)
103 |         .apply()?;
104 |     Ok(())
105 | }
106 | 


--------------------------------------------------------------------------------
/radiobrowser-dev.toml:
--------------------------------------------------------------------------------
  1 | ## API
  2 | ## ===
  3 | ## Directory for static and template files
  4 | static-files-dir = "./static"
  5 | ## Log file path
  6 | log-dir = "./"
  7 | ## Log level 0(ERROR)-4(TRACE)
  8 | log-level = 3
  9 | ## Log in JSON format
 10 | log-json = false
 11 | ## Listening IP
 12 | listen-host = "127.0.0.1"
 13 | ## Listening port
 14 | listen-port = 8080
 15 | ## How many concurrent threads used by socket
 16 | threads = 5
 17 | ## Documentation url to be used if automatic way is not working (HTTP/1.0)
 18 | server-url = "https://de1.api.radio-browser.info"
 19 | 
 20 | ## Cache support
 21 | ## =============
 22 | ## cache type can be "none" or "builtin" or "redis" or "memcached"
 23 | cache-type = "none"
 24 | ## redis connection string
 25 | #cache-url = "redis://localhost:6379"
 26 | ## memcached connection string
 27 | #cache-url = "memcache://localhost:11211"
 28 | ## Time to live for cache items
 29 | cache-ttl = "10secs"
 30 | 
 31 | ## Database
 32 | ## ========
 33 | ## database connection string (mysql, mariadb)
 34 | database = "mysql://radiouser:password@localhost/radio"
 35 | ## Ignore errors on database migration scripts
 36 | ## ONLY use this if you know what you are doing
 37 | ignore-migration-errors = false
 38 | ## Allow database to downgrade if you start an older version
 39 | ## of the radio browser binary
 40 | allow-database-downgrade = false
 41 | 
 42 | ## Prometheus exporter
 43 | ## ===================
 44 | ## This will enable the prometheus compatible exporter on the main listening port
 45 | ## You can reach it by doing a "GET /metrics"
 46 | prometheus-exporter = true
 47 | ## Prefix for all exported keys
 48 | prometheus-exporter-prefix = "radio_browser"
 49 | 
 50 | ## Stream check
 51 | ## ============
 52 | ## Enable the checking of stations
 53 | enable-check = false
 54 | ## Concurrent checks
 55 | concurrency = 10
 56 | ## Batchsize of stations to get from the database at a time
 57 | stations = 100
 58 | ## Enable delete logic for stations
 59 | delete = true
 60 | ## Interval in seconds to wait after every batch of checks
 61 | pause = "60secs"
 62 | ## Timeout for tcp connections
 63 | tcp-timeout = "10secs"
 64 | ## Recursive depth for real stream link resolution
 65 | max-depth = 5
 66 | ## Retries for each station check until marked as broken
 67 | retries = 5
 68 | ## Hostname for the check-entries in the database, defaults to the local hostname
 69 | #source = "myhostname"
 70 | ## Freeform location string for this server
 71 | server-location = "Datacenter 2 in coolstreet"
 72 | ## 2 letter countrycode to locate this server
 73 | server-country-code = "DE"
 74 | ## User agent for the stream check
 75 | #useragent = "useragent/1.0"
 76 | 
 77 | ## Check server infos if server supports it (icecast)
 78 | server-info-check = false
 79 | ## Chunksize for checking servers
 80 | server-info-check-chunksize = 100
 81 | 
 82 | ## Check if current favicon in database still works, and remove them if not
 83 | recheck-existing-favicon = false
 84 | ## Try to extract favicon from website for empty favicons
 85 | enable-extract-favicon = false
 86 | ## Minimum (width or height) of favicons extracted
 87 | favicon-size-min = 32
 88 | ## Maximum (width or height) of favicons extracted
 89 | favicon-size-max = 256
 90 | ## Optimum size of favicons extracted
 91 | favicon-size-optimum = 128
 92 | 
 93 | ## File path to CSV for replacing languages (local path or http/https)
 94 | #replace-language-file = "https://radiobrowser.gitlab.io/radiobrowser-static-data/language-replace.csv"
 95 | replace-language-file = "./etc/language-replace.csv"
 96 | ## File path to CSV for mapping language to code (local path or http/https)
 97 | language-to-code-file = "./etc/language-to-code.csv"
 98 | ## File path to CSV for replacing tags (local path or http/https)
 99 | #replace-tag-file = "https://radiobrowser.gitlab.io/radiobrowser-static-data/tag-replace.csv"
100 | replace-tag-file = "./etc/tag-replace.csv"
101 | 
102 | ## Caches
103 | ## ======
104 | ## Update caches at an interval
105 | update-caches-interval = "5mins"
106 | 
107 | ## Cleanup
108 | ## =======
109 | ## Cleanup worker startup interval
110 | cleanup-interval = "1hour"
111 | ## The same ip cannot do clicks for the same stream in this timespan
112 | click-valid-timeout = "1day"
113 | ## Broken streams are removed after this timespan, if they have never worked.
114 | broken-stations-never-working-timeout = "3days"
115 | ## Broken streams are removed after this timespan.
116 | broken-stations-timeout = "30days"
117 | ## Checks are removed after this timespan.
118 | checks-timeout = "30days"
119 | ## Checks are removed after this timespan.
120 | clicks-timeout = "30days"
121 | ## reload / redownload some config files
122 | refresh-config-interval = "1day"
123 | 
124 | ## Mirroring
125 | ## =========
126 | ## Mirror pull interval in seconds
127 | mirror-pull-interval = "5mins"
128 | ## How many changes should be pulled in a chunk while pulling
129 | chunk-size-changes = 10000
130 | ## How many checks should be pulled in a chunk while pulling
131 | chunk-size-checks = 10000
132 | ## On values > 0 autodelete stations with same urls, order by clickcount DESC
133 | # max-duplicates = 0
134 | ## Mirror from server
135 | [pullservers]
136 | #[pullservers.alpha]
137 | #host = "http://nl1.api.radio-browser.info"
138 | #[pullservers.beta]
139 | #host = "http://de1.api.radio-browser.info"
140 | #[pullservers.gamma]
141 | #host = "http://at1.api.radio-browser.info"
142 | 


--------------------------------------------------------------------------------
/etc/config-example.toml:
--------------------------------------------------------------------------------
  1 | ## API
  2 | ## ===
  3 | ## Directory for static and template files
  4 | static-files-dir = "/usr/share/radiobrowser"
  5 | ## Log file path
  6 | log-dir = "/var/log/radiobrowser/"
  7 | ## Log level 0(ERROR)-4(TRACE)
  8 | #log-level = 1
  9 | ## Log in JSON format
 10 | # log-json = false
 11 | ## Listening IP
 12 | listen-host = "127.0.0.1"
 13 | ## Listening port
 14 | listen-port = 8080
 15 | ## How many concurrent threads used by socket
 16 | threads = 5
 17 | ## Documentation url to be used if automatic way is not working (HTTP/1.0)
 18 | server-url = "https://de1.api.radio-browser.info"
 19 | 
 20 | ## Cache support
 21 | ## =============
 22 | ## cache type can be "none" or "builtin" or "redis" or "memcached"
 23 | # cache-type = "none"
 24 | ## redis connection string
 25 | # cache-url = "redis://localhost:6379"
 26 | ## memcached connection string
 27 | # cache-url = "memcache://localhost:11211"
 28 | ## Time to live for cache items
 29 | # cache-ttl = "60secs"
 30 | 
 31 | ## Database
 32 | ## ========
 33 | ## database connection string (mysql, mariadb)
 34 | database = "mysql://radiouser:password@localhost/radio"
 35 | ## Ignore errors on database migration scripts
 36 | ## ONLY use this if you know what you are doing
 37 | ignore-migration-errors = false
 38 | ## Allow database to downgrade if you start an older version
 39 | ## of the radio browser binary
 40 | allow-database-downgrade = false
 41 | 
 42 | ## Prometheus exporter
 43 | ## ===================
 44 | ## This will enable the prometheus compatible exporter on the main listening port
 45 | ## You can reach it by doing a "GET /metrics"
 46 | prometheus-exporter = true
 47 | ## Prefix for all exported keys
 48 | prometheus-exporter-prefix = "radio_browser"
 49 | 
 50 | ## Stream check
 51 | ## ============
 52 | ## Enable the checking of stations
 53 | enable-check = false
 54 | ## Concurrent checks
 55 | concurrency = 10
 56 | ## Batchsize of stations to get from the database at a time
 57 | stations = 100
 58 | ## Enable delete logic for stations
 59 | delete = true
 60 | ## Interval in seconds to wait after every batch of checks
 61 | pause = "60secs"
 62 | ## Timeout for tcp connections
 63 | tcp-timeout = "10secs"
 64 | ## Recursive depth for real stream link resolution
 65 | max-depth = 5
 66 | ## Retries for each station check until marked as broken
 67 | retries = 5
 68 | ## Hostname for the check-entries in the database, defaults to the local hostname
 69 | #source = "myhostname"
 70 | # Freeform location string for this server
 71 | # server-location = "Datacenter 2 in coolstreet"
 72 | # 2 letter countrycode to locate this server
 73 | # server-country-code = "DE"
 74 | ## User agent for the stream check
 75 | #useragent = "useragent/1.0"
 76 | 
 77 | ## Check server infos if server supports it (icecast)
 78 | server-info-check = false
 79 | ## Chunksize for checking servers
 80 | server-info-check-chunksize = 100
 81 | 
 82 | # Check if current favicon in database still works, and remove them if not
 83 | recheck-existing-favicon = false
 84 | ## Try to extract favicon from website for empty favicons
 85 | enable-extract-favicon = false
 86 | ## Minimum (width or height) of favicons extracted
 87 | favicon-size-min = 32
 88 | ## Maximum (width or height) of favicons extracted
 89 | favicon-size-max = 256
 90 | ## Optimum size of favicons extracted
 91 | favicon-size-optimum = 128
 92 | 
 93 | ## File path to CSV for replacing languages (local path or http/https)
 94 | #replace-language-file = "https://radiobrowser.gitlab.io/radiobrowser-static-data/language-replace.csv"
 95 | replace-language-file = "/etc/radiobrowser/language-replace.csv"
 96 | ## File path to CSV for mapping language to code (local path or http/https)
 97 | language-to-code-file = "/etc/radiobrowser/language-to-code.csv"
 98 | ## File path to CSV for replacing tags (local path or http/https)
 99 | #replace-tag-file = "https://radiobrowser.gitlab.io/radiobrowser-static-data/tag-replace.csv"
100 | replace-tag-file = "/etc/radiobrowser/tag-replace.csv"
101 | 
102 | ## Caches
103 | ## ======
104 | ## Update caches at an interval
105 | update-caches-interval = "5mins"
106 | 
107 | ## Cleanup
108 | ## =======
109 | ## Cleanup worker startup interval
110 | cleanup-interval = "1hour"
111 | ## The same ip cannot do clicks for the same stream in this timespan
112 | click-valid-timeout = "1day"
113 | ## Broken streams are removed after this timespan, if they have never worked.
114 | broken-stations-never-working-timeout = "3days"
115 | ## Broken streams are removed after this timespan.
116 | broken-stations-timeout = "30days"
117 | ## Checks are removed after this timespan.
118 | checks-timeout = "30days"
119 | ## Checks are removed after this timespan.
120 | clicks-timeout = "30days"
121 | ## reload / redownload some config files
122 | refresh-config-interval = "1day"
123 | 
124 | ## Mirroring
125 | ## =========
126 | ## Mirror pull interval in seconds
127 | mirror-pull-interval = "5mins"
128 | ## How many changes should be pulled in a chunk while pulling
129 | chunk-size-changes = 10000
130 | ## How many checks should be pulled in a chunk while pulling
131 | chunk-size-checks = 10000
132 | ## On values > 0 autodelete stations with same urls, order by clickcount DESC
133 | # max-duplicates = 0
134 | ## Mirror from server
135 | [pullservers]
136 | #[pullservers.alpha]
137 | #host = "http://nl1.api.radio-browser.info"
138 | #[pullservers.beta]
139 | #host = "http://de1.api.radio-browser.info"
140 | #[pullservers.gamma]
141 | #host = "http://at1.api.radio-browser.info"
142 | 


--------------------------------------------------------------------------------
/src/api/prometheus_exporter.rs:
--------------------------------------------------------------------------------
  1 | use crate::api::api_response::ApiResponse;
  2 | use crate::db::DbConnection;
  3 | use prometheus::{
  4 |     Encoder, HistogramVec, IntCounter, IntCounterVec, IntGauge, Registry, TextEncoder,
  5 | };
  6 | use std::convert::TryInto;
  7 | use std::error::Error;
  8 | 
  9 | #[derive(Clone)]
 10 | pub struct RegistryLinks {
 11 |     pub registry: Registry,
 12 |     pub timer: HistogramVec,
 13 | 
 14 |     pub api_calls: IntCounterVec,
 15 |     pub clicks: IntCounter,
 16 |     pub cache_hits: IntCounter,
 17 |     pub cache_misses: IntCounter,
 18 | 
 19 |     pub stations_broken: IntGauge,
 20 |     pub stations_working: IntGauge,
 21 |     pub stations_todo: IntGauge,
 22 |     pub stations_deletable_never_worked: IntGauge,
 23 |     pub stations_deletable_were_working: IntGauge,
 24 |     pub country_count: IntGauge,
 25 |     pub tags_count: IntGauge,
 26 |     pub language_count: IntGauge,
 27 | }
 28 | 
 29 | pub fn create_registry(prefix: &str) -> Result> {
 30 |     // Create a Counter.
 31 |     let timer = register_histogram_vec!("timer", "Timer for the api", &["method"])?;
 32 |     let api_calls = IntCounterVec::new(
 33 |         opts!("api_calls", "Calls to the api"),
 34 |         &["method", "url", "status_code"],
 35 |     )?;
 36 |     let clicks = IntCounter::new("station_clicks", "Clicks on stations")?;
 37 |     let cache_hits = IntCounter::new("cache_hits", "Cache hits")?;
 38 |     let cache_misses = IntCounter::new("cache_misses", "Cache misses")?;
 39 | 
 40 |     let stations_broken = IntGauge::new("stations_broken", "Count of stations that are broken")?;
 41 |     let stations_working = IntGauge::new(
 42 |         "stations_working",
 43 |         "Count of stations that are working/usable",
 44 |     )?;
 45 |     let stations_todo = IntGauge::new(
 46 |         "stations_todo",
 47 |         "Count of stations that are in the queue for checking",
 48 |     )?;
 49 |     let stations_deletable_never_worked = IntGauge::new(
 50 |         "stations_deletable_never_worked",
 51 |         "Count of stations that are in the list for deletion and which never worked",
 52 |     )?;
 53 |     let stations_deletable_were_working = IntGauge::new(
 54 |         "stations_deletable_were_working",
 55 |         "Count of stations that are in the list for deletion and which worked at some point",
 56 |     )?;
 57 |     let country_count = IntGauge::new("country_count", "Count of countries")?;
 58 |     let tags_count = IntGauge::new("tags_count", "Count of tags")?;
 59 |     let language_count = IntGauge::new("language_count", "Count of languages")?;
 60 | 
 61 |     let registry = Registry::new_custom(Some(prefix.to_string()), None)?;
 62 |     registry.register(Box::new(timer.clone()))?;
 63 |     registry.register(Box::new(api_calls.clone()))?;
 64 |     registry.register(Box::new(clicks.clone()))?;
 65 |     registry.register(Box::new(cache_hits.clone()))?;
 66 |     registry.register(Box::new(cache_misses.clone()))?;
 67 |     registry.register(Box::new(stations_broken.clone()))?;
 68 |     registry.register(Box::new(stations_working.clone()))?;
 69 |     registry.register(Box::new(stations_todo.clone()))?;
 70 |     registry.register(Box::new(stations_deletable_never_worked.clone()))?;
 71 |     registry.register(Box::new(stations_deletable_were_working.clone()))?;
 72 |     registry.register(Box::new(country_count.clone()))?;
 73 |     registry.register(Box::new(tags_count.clone()))?;
 74 |     registry.register(Box::new(language_count.clone()))?;
 75 | 
 76 |     Ok(RegistryLinks {
 77 |         registry,
 78 |         timer,
 79 |         api_calls,
 80 |         clicks,
 81 |         cache_hits,
 82 |         cache_misses,
 83 |         stations_broken,
 84 |         stations_working,
 85 |         stations_todo,
 86 |         stations_deletable_never_worked,
 87 |         stations_deletable_were_working,
 88 |         country_count,
 89 |         tags_count,
 90 |         language_count,
 91 |     })
 92 | }
 93 | 
 94 | pub fn render(
 95 |     connection_new: &A,
 96 |     broken_stations_never_working_timeout: u64,
 97 |     broken_stations_timeout: u64,
 98 |     registry: RegistryLinks,
 99 | ) -> Result>
100 | where
101 |     A: DbConnection,
102 | {
103 |     let stations_broken = connection_new.get_station_count_broken()?;
104 |     let stations_working = connection_new.get_station_count_working()?;
105 |     let stations_todo = connection_new.get_station_count_todo(24)?;
106 |     let stations_deletable_never_worked =
107 |         connection_new.get_deletable_never_working(broken_stations_never_working_timeout)?;
108 |     let stations_deletable_were_working =
109 |         connection_new.get_deletable_were_working(broken_stations_timeout)?;
110 | 
111 |     let country_count = connection_new.get_country_count()?;
112 |     let tags_count = connection_new.get_tag_count()?;
113 |     let language_count = connection_new.get_language_count()?;
114 | 
115 |     registry.stations_broken.set(stations_broken.try_into()?);
116 |     registry.stations_working.set(stations_working.try_into()?);
117 |     registry.stations_todo.set(stations_todo.try_into()?);
118 |     registry
119 |         .stations_deletable_never_worked
120 |         .set(stations_deletable_never_worked.try_into()?);
121 |     registry
122 |         .stations_deletable_were_working
123 |         .set(stations_deletable_were_working.try_into()?);
124 |     registry.country_count.set(country_count.try_into()?);
125 |     registry.tags_count.set(tags_count.try_into()?);
126 |     registry.language_count.set(language_count.try_into()?);
127 | 
128 |     // Gather the metrics.
129 |     let mut buffer = vec![];
130 |     let encoder = TextEncoder::new();
131 |     let metric_families = registry.registry.gather();
132 |     encoder.encode(&metric_families, &mut buffer)?;
133 | 
134 |     // Output to the standard output.
135 |     Ok(ApiResponse::Text(String::from_utf8(buffer)?))
136 | }
137 | 


--------------------------------------------------------------------------------
/src/db/models/station_item.rs:
--------------------------------------------------------------------------------
  1 | use chrono::DateTime;
  2 | use chrono::Utc;
  3 | 
  4 | #[derive(Clone, Debug, PartialEq)]
  5 | pub struct DbStationItem {
  6 |     pub id: i32,
  7 |     pub changeuuid: String,
  8 |     pub stationuuid: String,
  9 |     pub serveruuid: Option,
 10 |     pub name: String,
 11 |     pub url: String,
 12 |     pub url_resolved: String,
 13 |     pub homepage: String,
 14 |     pub favicon: String,
 15 |     pub tags: String,
 16 |     pub country: String,
 17 |     pub countrycode: String,
 18 |     pub iso_3166_2: Option,
 19 |     pub state: String,
 20 |     pub language: String,
 21 |     pub languagecodes: String,
 22 |     pub votes: i32,
 23 |     pub lastchangetime: String,
 24 |     pub lastchangetime_iso8601: Option>,
 25 |     pub codec: String,
 26 |     pub bitrate: u32,
 27 |     pub hls: bool,
 28 |     pub lastcheckok: bool,
 29 |     pub lastchecktime: String,
 30 |     pub lastchecktime_iso8601: Option>,
 31 |     pub lastcheckoktime: String,
 32 |     pub lastcheckoktime_iso8601: Option>,
 33 |     pub lastlocalchecktime: String,
 34 |     pub lastlocalchecktime_iso8601: Option>,
 35 |     pub clicktimestamp: String,
 36 |     pub clicktimestamp_iso8601: Option>,
 37 |     pub clickcount: u32,
 38 |     pub clicktrend: i32,
 39 |     pub ssl_error: bool,
 40 |     pub geo_lat: Option,
 41 |     pub geo_long: Option,
 42 |     pub has_extended_info: Option,
 43 | }
 44 | 
 45 | impl DbStationItem {
 46 |     pub fn set_name>(&mut self, name: P) {
 47 |         if !self.name.eq(name.as_ref()) {
 48 |             debug!(
 49 |                 "station changed {}: name '{}' -> '{}'",
 50 |                 self.stationuuid,
 51 |                 self.name,
 52 |                 name.as_ref()
 53 |             );
 54 |             self.name = name.as_ref().to_string();
 55 |         }
 56 |     }
 57 | 
 58 |     pub fn set_favicon>(&mut self, favicon: P) {
 59 |         if !self.favicon.eq(favicon.as_ref()) {
 60 |             debug!(
 61 |                 "station changed {}: favicon '{}' -> '{}'",
 62 |                 self.stationuuid,
 63 |                 self.favicon,
 64 |                 favicon.as_ref()
 65 |             );
 66 |             self.favicon = favicon.as_ref().to_string();
 67 |         }
 68 |     }
 69 | 
 70 |     pub fn set_language>(&mut self, language: P) {
 71 |         if !self.language.eq(language.as_ref()) {
 72 |             debug!(
 73 |                 "station changed {}: language '{}' -> '{}'",
 74 |                 self.stationuuid,
 75 |                 self.language,
 76 |                 language.as_ref()
 77 |             );
 78 |             self.language = language.as_ref().to_string();
 79 |         }
 80 |     }
 81 | 
 82 |     pub fn set_tags>(&mut self, tags: P) {
 83 |         if !self.tags.eq(tags.as_ref()) {
 84 |             debug!(
 85 |                 "station changed {}: tags '{}' -> '{}'",
 86 |                 self.stationuuid,
 87 |                 self.tags,
 88 |                 tags.as_ref()
 89 |             );
 90 |             self.tags = tags.as_ref().to_string();
 91 |         }
 92 |     }
 93 | 
 94 |     pub fn set_countrycode>(&mut self, countrycode: P) {
 95 |         if !self.countrycode.eq(countrycode.as_ref()) {
 96 |             debug!(
 97 |                 "station changed {}: countrycode '{}' -> '{}'",
 98 |                 self.stationuuid,
 99 |                 self.countrycode,
100 |                 countrycode.as_ref()
101 |             );
102 |             self.countrycode = countrycode.as_ref().to_string();
103 |         }
104 |     }
105 | 
106 |     pub fn set_languagecodes>(&mut self, languagecodes: P) {
107 |         if !self.languagecodes.eq(languagecodes.as_ref()) {
108 |             debug!(
109 |                 "station changed {}: languagecodes '{}' -> '{}'",
110 |                 self.stationuuid,
111 |                 self.languagecodes,
112 |                 languagecodes.as_ref()
113 |             );
114 |             self.languagecodes = languagecodes.as_ref().to_string();
115 |         }
116 |     }
117 | 
118 |     pub fn set_url>(&mut self, url: P) {
119 |         if !self.url.eq(url.as_ref()) {
120 |             debug!(
121 |                 "station changed {}: url '{}' -> '{}'",
122 |                 self.stationuuid,
123 |                 self.url,
124 |                 url.as_ref()
125 |             );
126 |             self.url = url.as_ref().to_string();
127 |         }
128 |     }
129 | 
130 |     pub fn set_homepage>(&mut self, homepage: P) {
131 |         if !self.homepage.eq(homepage.as_ref()) {
132 |             debug!(
133 |                 "station changed {}: homepage '{}' -> '{}'",
134 |                 self.stationuuid,
135 |                 self.homepage,
136 |                 homepage.as_ref()
137 |             );
138 |             self.homepage = homepage.as_ref().to_string();
139 |         }
140 |     }
141 | 
142 |     pub fn set_iso_3166_2(&mut self, iso_3166_2: Option) {
143 |         if !self.iso_3166_2.eq(&iso_3166_2) {
144 |             debug!(
145 |                 "station changed {}: iso_3166_2 '{:?}' -> '{:?}'",
146 |                 self.stationuuid,
147 |                 self.iso_3166_2,
148 |                 iso_3166_2
149 |             );
150 |             self.iso_3166_2 = iso_3166_2;
151 |         }
152 |     }
153 | /*
154 |     pub fn set_geo_lat>(&mut self, geo_lat: Option) {
155 |         if !self.geo_lat.eq(&geo_lat) {
156 |             debug!(
157 |                 "station changed {}: geo_lat '{}' -> '{}'",
158 |                 self.stationuuid,
159 |                 self.geo_lat.unwrap_or_default(),
160 |                 geo_lat.unwrap_or_default()
161 |             );
162 |             self.geo_lat = geo_lat;
163 |         }
164 |     }
165 |     */
166 | }
167 | 


--------------------------------------------------------------------------------
/CODE_OF_CONDUCT.md:
--------------------------------------------------------------------------------
  1 | 
  2 | # Contributor Covenant Code of Conduct
  3 | 
  4 | ## Our Pledge
  5 | 
  6 | We as members, contributors, and leaders pledge to make participation in our
  7 | community a harassment-free experience for everyone, regardless of age, body
  8 | size, visible or invisible disability, ethnicity, sex characteristics, gender
  9 | identity and expression, level of experience, education, socio-economic status,
 10 | nationality, personal appearance, race, caste, color, religion, or sexual
 11 | identity and orientation.
 12 | 
 13 | We pledge to act and interact in ways that contribute to an open, welcoming,
 14 | diverse, inclusive, and healthy community.
 15 | 
 16 | ## Our Standards
 17 | 
 18 | Examples of behavior that contributes to a positive environment for our
 19 | community include:
 20 | 
 21 | * Demonstrating empathy and kindness toward other people
 22 | * Being respectful of differing opinions, viewpoints, and experiences
 23 | * Giving and gracefully accepting constructive feedback
 24 | * Accepting responsibility and apologizing to those affected by our mistakes,
 25 |   and learning from the experience
 26 | * Focusing on what is best not just for us as individuals, but for the overall
 27 |   community
 28 | 
 29 | Examples of unacceptable behavior include:
 30 | 
 31 | * The use of sexualized language or imagery, and sexual attention or advances of
 32 |   any kind
 33 | * Trolling, insulting or derogatory comments, and personal or political attacks
 34 | * Public or private harassment
 35 | * Publishing others' private information, such as a physical or email address,
 36 |   without their explicit permission
 37 | * Other conduct which could reasonably be considered inappropriate in a
 38 |   professional setting
 39 | 
 40 | ## Enforcement Responsibilities
 41 | 
 42 | Community leaders are responsible for clarifying and enforcing our standards of
 43 | acceptable behavior and will take appropriate and fair corrective action in
 44 | response to any behavior that they deem inappropriate, threatening, offensive,
 45 | or harmful.
 46 | 
 47 | Community leaders have the right and responsibility to remove, edit, or reject
 48 | comments, commits, code, wiki edits, issues, and other contributions that are
 49 | not aligned to this Code of Conduct, and will communicate reasons for moderation
 50 | decisions when appropriate.
 51 | 
 52 | ## Scope
 53 | 
 54 | This Code of Conduct applies within all community spaces, and also applies when
 55 | an individual is officially representing the community in public spaces.
 56 | Examples of representing our community include using an official e-mail address,
 57 | posting via an official social media account, or acting as an appointed
 58 | representative at an online or offline event.
 59 | 
 60 | ## Enforcement
 61 | 
 62 | Instances of abusive, harassing, or otherwise unacceptable behavior may be
 63 | reported to the community leaders responsible for enforcement at
 64 | segler_alex@web.de.
 65 | All complaints will be reviewed and investigated promptly and fairly.
 66 | 
 67 | All community leaders are obligated to respect the privacy and security of the
 68 | reporter of any incident.
 69 | 
 70 | ## Enforcement Guidelines
 71 | 
 72 | Community leaders will follow these Community Impact Guidelines in determining
 73 | the consequences for any action they deem in violation of this Code of Conduct:
 74 | 
 75 | ### 1. Correction
 76 | 
 77 | **Community Impact**: Use of inappropriate language or other behavior deemed
 78 | unprofessional or unwelcome in the community.
 79 | 
 80 | **Consequence**: A private, written warning from community leaders, providing
 81 | clarity around the nature of the violation and an explanation of why the
 82 | behavior was inappropriate. A public apology may be requested.
 83 | 
 84 | ### 2. Warning
 85 | 
 86 | **Community Impact**: A violation through a single incident or series of
 87 | actions.
 88 | 
 89 | **Consequence**: A warning with consequences for continued behavior. No
 90 | interaction with the people involved, including unsolicited interaction with
 91 | those enforcing the Code of Conduct, for a specified period of time. This
 92 | includes avoiding interactions in community spaces as well as external channels
 93 | like social media. Violating these terms may lead to a temporary or permanent
 94 | ban.
 95 | 
 96 | ### 3. Temporary Ban
 97 | 
 98 | **Community Impact**: A serious violation of community standards, including
 99 | sustained inappropriate behavior.
100 | 
101 | **Consequence**: A temporary ban from any sort of interaction or public
102 | communication with the community for a specified period of time. No public or
103 | private interaction with the people involved, including unsolicited interaction
104 | with those enforcing the Code of Conduct, is allowed during this period.
105 | Violating these terms may lead to a permanent ban.
106 | 
107 | ### 4. Permanent Ban
108 | 
109 | **Community Impact**: Demonstrating a pattern of violation of community
110 | standards, including sustained inappropriate behavior, harassment of an
111 | individual, or aggression toward or disparagement of classes of individuals.
112 | 
113 | **Consequence**: A permanent ban from any sort of public interaction within the
114 | community.
115 | 
116 | ## Attribution
117 | 
118 | This Code of Conduct is adapted from the [Contributor Covenant][homepage],
119 | version 2.1, available at
120 | [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
121 | 
122 | Community Impact Guidelines were inspired by
123 | [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
124 | 
125 | For answers to common questions about this code of conduct, see the FAQ at
126 | [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
127 | [https://www.contributor-covenant.org/translations][translations].
128 | 
129 | [homepage]: https://www.contributor-covenant.org
130 | [v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
131 | [Mozilla CoC]: https://github.com/mozilla/diversity
132 | [FAQ]: https://www.contributor-covenant.org/faq
133 | [translations]: https://www.contributor-covenant.org/translations
134 | 


--------------------------------------------------------------------------------
/src/api/data/api_config.rs:
--------------------------------------------------------------------------------
  1 | use crate::api::api_response::ApiResponse;
  2 | use crate::config::Config;
  3 | use std::error::Error;
  4 | use serde::{Serialize,Deserialize};
  5 | 
  6 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
  7 | pub struct ApiConfig {
  8 |     pub check_enabled: bool,
  9 |     pub prometheus_exporter_enabled: bool,
 10 |     pub pull_servers: Vec,
 11 |     pub tcp_timeout_seconds: u64,
 12 |     pub broken_stations_never_working_timeout_seconds: u64,
 13 |     pub broken_stations_timeout_seconds: u64,
 14 |     pub checks_timeout_seconds: u64,
 15 |     pub click_valid_timeout_seconds: u64,
 16 |     pub clicks_timeout_seconds: u64,
 17 |     pub mirror_pull_interval_seconds: u64,
 18 |     pub update_caches_interval_seconds: u64,
 19 |     pub server_name: String,
 20 |     pub server_location: String,
 21 |     pub server_country_code: String,
 22 |     pub check_retries: u8,
 23 |     pub check_batchsize: u32,
 24 |     pub check_pause_seconds: u64,
 25 |     pub api_threads: usize,
 26 |     pub cache_type: String,
 27 |     pub cache_ttl: u64,
 28 |     pub language_replace_filepath: String,
 29 |     pub language_to_code_filepath: String,
 30 | }
 31 | 
 32 | impl ApiConfig {
 33 |     pub fn serialize_config(config: ApiConfig) -> std::io::Result {
 34 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
 35 |         xml.begin_elem("config")?;
 36 |         xml.elem_text("check_enabled", &config.check_enabled.to_string())?;
 37 |         xml.elem_text(
 38 |             "prometheus_exporter_enabled",
 39 |             &config.prometheus_exporter_enabled.to_string(),
 40 |         )?;
 41 |         {
 42 |             xml.begin_elem("pull_servers")?;
 43 |             for server in config.pull_servers {
 44 |                 xml.elem_text("url", &server)?;
 45 |             }
 46 |             xml.end_elem()?;
 47 |         }
 48 |         xml.elem_text(
 49 |             "tcp_timeout_seconds",
 50 |             &config.tcp_timeout_seconds.to_string(),
 51 |         )?;
 52 |         xml.elem_text(
 53 |             "broken_stations_never_working_timeout_seconds",
 54 |             &config
 55 |                 .broken_stations_never_working_timeout_seconds
 56 |                 .to_string(),
 57 |         )?;
 58 |         xml.elem_text(
 59 |             "broken_stations_timeout_seconds",
 60 |             &config.broken_stations_timeout_seconds.to_string(),
 61 |         )?;
 62 |         xml.elem_text(
 63 |             "checks_timeout_seconds",
 64 |             &config.checks_timeout_seconds.to_string(),
 65 |         )?;
 66 |         xml.elem_text(
 67 |             "click_valid_timeout_seconds",
 68 |             &config.click_valid_timeout_seconds.to_string(),
 69 |         )?;
 70 |         xml.elem_text(
 71 |             "clicks_timeout_seconds",
 72 |             &config.clicks_timeout_seconds.to_string(),
 73 |         )?;
 74 |         xml.elem_text(
 75 |             "mirror_pull_interval_seconds",
 76 |             &config.mirror_pull_interval_seconds.to_string(),
 77 |         )?;
 78 |         xml.elem_text(
 79 |             "update_caches_interval_seconds",
 80 |             &config.update_caches_interval_seconds.to_string(),
 81 |         )?;
 82 |         xml.elem_text("server_name", &config.server_name)?;
 83 |         xml.elem_text("server_location", &config.server_location)?;
 84 |         xml.elem_text("server_country_code", &config.server_country_code)?;
 85 | 
 86 |         xml.elem_text("check_retries", &config.check_retries.to_string())?;
 87 |         xml.elem_text("check_batchsize", &config.check_batchsize.to_string())?;
 88 |         xml.elem_text(
 89 |             "check_pause_seconds",
 90 |             &config.check_pause_seconds.to_string(),
 91 |         )?;
 92 |         xml.elem_text("api_threads", &config.api_threads.to_string())?;
 93 |         xml.elem_text("cache_type", &config.cache_type.to_string())?;
 94 |         xml.elem_text("cache_ttl", &config.cache_ttl.to_string())?;
 95 |         xml.elem_text("language_replace_filepath", &config.language_replace_filepath)?;
 96 |         xml.elem_text("language_to_code_filepath", &config.language_to_code_filepath)?;
 97 |         xml.end_elem()?;
 98 |         xml.close()?;
 99 |         xml.flush()?;
100 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
101 |     }
102 | 
103 |     pub fn get_response(config: ApiConfig, format: &str) -> Result> {
104 |         Ok(match format {
105 |             "json" => ApiResponse::Text(serde_json::to_string(&config)?),
106 |             "xml" => ApiResponse::Text(ApiConfig::serialize_config(config)?),
107 |             _ => ApiResponse::UnknownContentType,
108 |         })
109 |     }
110 | }
111 | 
112 | impl From for ApiConfig {
113 |     fn from(item: Config) -> Self {
114 |         ApiConfig {
115 |             check_enabled: item.enable_check,
116 |             prometheus_exporter_enabled: item.prometheus_exporter,
117 |             pull_servers: item.servers_pull,
118 |             tcp_timeout_seconds: item.tcp_timeout.as_secs(),
119 |             broken_stations_never_working_timeout_seconds: item
120 |                 .broken_stations_never_working_timeout
121 |                 .as_secs(),
122 |             broken_stations_timeout_seconds: item.broken_stations_timeout.as_secs(),
123 |             checks_timeout_seconds: item.checks_timeout.as_secs(),
124 |             click_valid_timeout_seconds: item.click_valid_timeout.as_secs(),
125 |             clicks_timeout_seconds: item.clicks_timeout.as_secs(),
126 |             mirror_pull_interval_seconds: item.mirror_pull_interval.as_secs(),
127 |             update_caches_interval_seconds: item.update_caches_interval.as_secs(),
128 |             server_name: item.source,
129 |             check_retries: item.retries,
130 |             check_batchsize: item.check_stations,
131 |             check_pause_seconds: item.pause.as_secs(),
132 |             api_threads: item.threads,
133 |             cache_type: item.cache_type.into(),
134 |             cache_ttl: item.cache_ttl.as_secs(),
135 |             server_location: item.server_location,
136 |             server_country_code: item.server_country_code,
137 |             language_replace_filepath: item.language_replace_filepath,
138 |             language_to_code_filepath: item.language_to_code_filepath,
139 |         }
140 |     }
141 | }
142 | 


--------------------------------------------------------------------------------
/src/db/db_mysql/simple_migrate.rs:
--------------------------------------------------------------------------------
  1 | use mysql::TxOpts;
  2 | use mysql;
  3 | use mysql::prelude::*;
  4 | 
  5 | pub struct Migration {
  6 |     name: String,
  7 |     up: String,
  8 |     down: String,
  9 | }
 10 | 
 11 | pub struct Migrations<'a> {
 12 |     pool: &'a mysql::Pool,
 13 |     migrations_wanted: Vec,
 14 | }
 15 | 
 16 | impl<'a> Migrations<'a> {
 17 |     pub fn new(pool: &mysql::Pool) -> Migrations {
 18 |         Migrations {
 19 |             pool: pool,
 20 |             migrations_wanted: vec![],
 21 |         }
 22 |     }
 23 | 
 24 |     fn ensure_tables(&self) -> Result<(), Box> {
 25 |         self.pool.get_conn()?.query_drop(
 26 |             "CREATE TABLE IF NOT EXISTS __migrations(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, name TEXT NOT NULL, up TEXT NOT NULL, down TEXT NOT NULL);")?;
 27 |         Ok(())
 28 |     }
 29 | 
 30 |     fn get_applied_migrations(&self) -> Result, Box> {
 31 |         let list = self.pool.get_conn()?.query_map(
 32 |             "SELECT name,up,down FROM __migrations ORDER BY name;",
 33 |             |(name, up, down)| Migration { name, up, down },
 34 |         )?;
 35 |         Ok(list)
 36 |     }
 37 | 
 38 |     fn insert_db_migration(
 39 |         &self,
 40 |         conn: &mut mysql::Transaction,
 41 |         name: &str,
 42 |         up: &str,
 43 |         down: &str,
 44 |     ) -> Result<(), Box> {
 45 |         conn.exec_drop(
 46 |             "INSERT INTO __migrations(name, up , down) VALUES (:name,:up,:down);",
 47 |             params! {
 48 |                 name, up, down,
 49 |             },
 50 |         )?;
 51 |         Ok(())
 52 |     }
 53 | 
 54 |     fn delete_db_migration(
 55 |         &self,
 56 |         conn: &mut mysql::Transaction,
 57 |         name: &str,
 58 |     ) -> Result<(), Box> {
 59 |         conn.exec_drop(
 60 |             "DELETE FROM __migrations WHERE name=:name;",
 61 |             params! {
 62 |                 name,
 63 |             },
 64 |         )?;
 65 |         Ok(())
 66 |     }
 67 | 
 68 |     pub fn add_migration(&mut self, name: &str, up: &str, down: &str) {
 69 |         let m = Migration {
 70 |             name: name.to_string(),
 71 |             up: up.to_string(),
 72 |             down: down.to_string(),
 73 |         };
 74 |         self.migrations_wanted.push(m);
 75 |         self.migrations_wanted
 76 |             .sort_unstable_by(|a, b| a.name.cmp(&b.name));
 77 |     }
 78 | 
 79 |     fn apply_migration(
 80 |         &self,
 81 |         conn: &mut mysql::PooledConn,
 82 |         migration: &Migration,
 83 |         ignore_errors: bool,
 84 |     ) -> Result<(), Box> {
 85 |         info!("APPLY UP '{}'", migration.name);
 86 |         let mut conn = conn.start_transaction(TxOpts::default())?;
 87 |         let result = conn.query_drop(&migration.up);
 88 |         match result {
 89 |             Err(err) => {
 90 |                 if !ignore_errors {
 91 |                     panic!("{}", err.to_string());
 92 |                 }
 93 |             }
 94 |             _ => {}
 95 |         };
 96 |         self.insert_db_migration(&mut conn, &migration.name, &migration.up, &migration.down)?;
 97 |         conn.commit()?;
 98 |         Ok(())
 99 |     }
100 | 
101 |     fn unapply_migration(
102 |         &self,
103 |         conn: &mut mysql::PooledConn,
104 |         migration: &Migration,
105 |         ignore_errors: bool,
106 |     ) -> Result<(), Box> {
107 |         info!("APPLY DOWN '{}'", migration.name);
108 |         let mut conn = conn.start_transaction(TxOpts::default())?;
109 |         let result = conn.query_drop(&migration.down);
110 |         match result {
111 |             Err(err) => {
112 |                 if !ignore_errors {
113 |                     panic!("{}", err.to_string());
114 |                 }
115 |             }
116 |             _ => {}
117 |         };
118 |         self.delete_db_migration(&mut conn, &migration.name)?;
119 |         conn.commit()?;
120 |         Ok(())
121 |     }
122 | 
123 |     pub fn migrations_needed(
124 |         &self,
125 |     ) -> Result> {
126 |         self.ensure_tables()?;
127 | 
128 |         let migrations_applied = self.get_applied_migrations()?;
129 |         // apply all migrations, that are not applied
130 |         for wanted in self.migrations_wanted.iter() {
131 |             let mut found = false;
132 |             for applied in migrations_applied.iter() {
133 |                 if applied.name == wanted.name {
134 |                     found = true;
135 |                 }
136 |             }
137 |             if !found {
138 |                 return Ok(true);
139 |             }
140 |         }
141 | 
142 |         // unapply all migrations, that are not in wanted
143 |         for wanted in migrations_applied.iter().rev() {
144 |             let mut found = false;
145 |             for applied in self.migrations_wanted.iter() {
146 |                 if applied.name == wanted.name {
147 |                     found = true;
148 |                 }
149 |             }
150 |             if !found {
151 |                 return Ok(true);
152 |             }
153 |         }
154 | 
155 |         Ok(false)
156 |     }
157 | 
158 |     pub fn do_migrations(
159 |         &self,
160 |         ignore_errors: bool,
161 |         allow_database_downgrade: bool,
162 |     ) -> Result<(), Box> {
163 |         self.ensure_tables()?;
164 |         let mut conn = self.pool.get_conn()?;
165 | 
166 |         let migrations_applied = self.get_applied_migrations()?;
167 |         // apply all migrations, that are not applied
168 |         for wanted in self.migrations_wanted.iter() {
169 |             let mut found = false;
170 |             for applied in migrations_applied.iter() {
171 |                 if applied.name == wanted.name {
172 |                     found = true;
173 |                 }
174 |             }
175 |             if !found {
176 |                 self.apply_migration(&mut conn, &wanted, ignore_errors)?;
177 |             }
178 |         }
179 | 
180 |         // unapply all migrations, that are not in wanted
181 |         for wanted in migrations_applied.iter().rev() {
182 |             let mut found = false;
183 |             for applied in self.migrations_wanted.iter() {
184 |                 if applied.name == wanted.name {
185 |                     found = true;
186 |                 }
187 |             }
188 |             if !found {
189 |                 if allow_database_downgrade {
190 |                     self.unapply_migration(&mut conn, &wanted, ignore_errors)?;
191 |                 } else {
192 |                     panic!("Database downgrade would be neccessary! Please confirm if you really want to do that.")
193 |                 }
194 |             }
195 |         }
196 | 
197 |         Ok(())
198 |     }
199 | }
200 | 


--------------------------------------------------------------------------------
/src/api/data/station_history.rs:
--------------------------------------------------------------------------------
  1 | use chrono::NaiveDateTime;
  2 | use crate::db::models::StationHistoryItem;
  3 | use celes::Country;
  4 | use std::error::Error;
  5 | use serde::{Serialize,Deserialize};
  6 | use chrono::DateTime;
  7 | use chrono::Utc;
  8 | 
  9 | #[derive(PartialEq, Eq, Serialize, Deserialize, Debug)]
 10 | pub struct StationHistoryV0 {
 11 |     changeuuid: String,
 12 |     stationuuid: String,
 13 |     name: String,
 14 |     url: String,
 15 |     homepage: String,
 16 |     favicon: String,
 17 |     tags: String,
 18 |     country: String,
 19 |     countrycode: String,
 20 |     state: String,
 21 |     language: String,
 22 |     votes: String,
 23 |     lastchangetime: String,
 24 | }
 25 | 
 26 | #[derive(PartialEq, Serialize, Deserialize, Debug)]
 27 | pub struct StationHistoryCurrent {
 28 |     pub changeuuid: String,
 29 |     pub stationuuid: String,
 30 |     pub name: String,
 31 |     pub url: String,
 32 |     pub homepage: String,
 33 |     pub favicon: String,
 34 |     pub tags: String,
 35 |     pub country: String,
 36 |     pub countrycode: String,
 37 |     pub state: String,
 38 |     pub language: String,
 39 |     pub languagecodes: Option,
 40 |     pub votes: i32,
 41 |     pub lastchangetime: String,
 42 |     pub lastchangetime_iso8601: Option>,
 43 |     pub geo_lat: Option,
 44 |     pub geo_long: Option,
 45 | }
 46 | 
 47 | impl From for StationHistoryCurrent {
 48 |     fn from(item: StationHistoryV0) -> Self {
 49 |         let lastchangetime_iso8601 = NaiveDateTime::parse_from_str(&item.lastchangetime, "%Y-%m-%d %H:%M:%S")
 50 |             .ok()
 51 |             .map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc));
 52 | 
 53 |         StationHistoryCurrent {
 54 |             changeuuid: item.changeuuid,
 55 |             stationuuid: item.stationuuid,
 56 |             name: item.name,
 57 |             url: item.url,
 58 |             homepage: item.homepage,
 59 |             favicon: item.favicon,
 60 |             tags: item.tags,
 61 |             country: item.country,
 62 |             countrycode: item.countrycode,
 63 |             state: item.state,
 64 |             language: item.language,
 65 |             languagecodes: None,
 66 |             votes: item.votes.parse().unwrap(),
 67 |             lastchangetime: item.lastchangetime,
 68 |             lastchangetime_iso8601,
 69 |             geo_lat: None,
 70 |             geo_long: None,
 71 |         }
 72 |     }
 73 | }
 74 | 
 75 | impl From<&StationHistoryV0> for StationHistoryCurrent {
 76 |     fn from(item: &StationHistoryV0) -> Self {
 77 |         let lastchangetime_iso8601 = NaiveDateTime::parse_from_str(&item.lastchangetime, "%Y-%m-%d %H:%M:%S")
 78 |             .ok()
 79 |             .map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc));
 80 | 
 81 |         StationHistoryCurrent {
 82 |             changeuuid: item.changeuuid.clone(),
 83 |             stationuuid: item.stationuuid.clone(),
 84 |             name: item.name.clone(),
 85 |             url: item.url.clone(),
 86 |             homepage: item.homepage.clone(),
 87 |             favicon: item.favicon.clone(),
 88 |             tags: item.tags.clone(),
 89 |             country: item.country.clone(),
 90 |             countrycode: item.countrycode.clone(),
 91 |             state: item.state.clone(),
 92 |             language: item.language.clone(),
 93 |             languagecodes: None,
 94 |             votes: item.votes.parse().unwrap(),
 95 |             lastchangetime: item.lastchangetime.clone(),
 96 |             lastchangetime_iso8601,
 97 |             geo_lat: None,
 98 |             geo_long: None,
 99 |         }
100 |     }
101 | }
102 | 
103 | impl StationHistoryCurrent {
104 |     pub fn serialize_changes_list_csv(entries: Vec) -> Result> {
105 |         let mut wtr = csv::Writer::from_writer(Vec::new());
106 | 
107 |         for entry in entries {
108 |             wtr.serialize(entry)?;
109 |         }
110 |         
111 |         wtr.flush()?;
112 |         let x: Vec = wtr.into_inner()?;
113 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
114 |     }
115 | 
116 |     pub fn serialize_changes_list(entries: Vec) -> std::io::Result {
117 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
118 |         xml.begin_elem("result")?;
119 |         for entry in entries {
120 |             xml.begin_elem("station")?;
121 |             xml.attr_esc("changeuuid", &entry.changeuuid)?;
122 |             xml.attr_esc("stationuuid", &entry.stationuuid)?;
123 |             xml.attr_esc("name", &entry.name)?;
124 |             xml.attr_esc("url", &entry.url)?;
125 |             xml.attr_esc("homepage", &entry.homepage)?;
126 |             xml.attr_esc("favicon", &entry.favicon)?;
127 |             xml.attr_esc("tags", &entry.tags)?;
128 |             xml.attr_esc("country", &entry.country)?;
129 |             xml.attr_esc("countrycode", &entry.countrycode)?;
130 |             xml.attr_esc("state", &entry.state)?;
131 |             xml.attr_esc("language", &entry.language)?;
132 |             let station_votes_str = format!("{}", entry.votes);
133 |             xml.attr_esc("votes", &station_votes_str)?;
134 |             let station_lastchangetime_str = format!("{}", entry.lastchangetime);
135 |             xml.attr_esc("lastchangetime", &station_lastchangetime_str)?;
136 |             if let Some(geo_lat) = &entry.geo_lat {
137 |                 xml.attr_esc("geo_lat", &geo_lat.to_string())?;
138 |             }
139 |             if let Some(geo_long) = &entry.geo_long {
140 |                 xml.attr_esc("geo_long", &geo_long.to_string())?;
141 |             }
142 |             xml.end_elem()?;
143 |         }
144 |         xml.end_elem()?;
145 |         xml.close()?;
146 |         xml.flush()?;
147 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
148 |     }
149 | }
150 | 
151 | impl From for StationHistoryCurrent {
152 |     fn from(item: StationHistoryItem) -> Self {
153 |         StationHistoryCurrent {
154 |             changeuuid: item.changeuuid,
155 |             stationuuid: item.stationuuid,
156 |             name: item.name,
157 |             url: item.url,
158 |             homepage: item.homepage,
159 |             favicon: item.favicon,
160 |             tags: item.tags,
161 |             country: String::from(Country::from_alpha2(&item.countrycode).map(|c| c.long_name).unwrap_or("")),
162 |             countrycode: item.countrycode,
163 |             state: item.state,
164 |             language: item.language,
165 |             languagecodes: Some(item.languagecodes),
166 |             votes: item.votes,
167 |             lastchangetime: item.lastchangetime,
168 |             lastchangetime_iso8601: item.lastchangetime_iso8601,
169 |             geo_lat: item.geo_lat,
170 |             geo_long: item.geo_long,
171 |         }
172 |     }
173 | }


--------------------------------------------------------------------------------
/src/api/parameters.rs:
--------------------------------------------------------------------------------
  1 | use std::str::ParseBoolError;
  2 | use crate::api::rouille::Request;
  3 | use std::collections::HashMap;
  4 | use std::io::Read;
  5 | use url::form_urlencoded;
  6 | //use self::serde_json::value::{Map};
  7 | 
  8 | pub struct RequestParameters{
  9 |     values: HashMap
 10 | }
 11 | 
 12 | impl RequestParameters {
 13 |     pub fn new(req: &Request) -> Self {
 14 |         let mut values = HashMap::new();
 15 |         RequestParameters::decode(req, &mut values);
 16 |         RequestParameters{
 17 |             values
 18 |         }
 19 |     }
 20 | 
 21 |     fn decode(req: &Request, map: &mut HashMap) {
 22 |         let content_type_raw: &str = req.header("Content-Type").unwrap_or("nothing");
 23 |         let content_type_arr: Vec<&str> = content_type_raw.split(";").collect();
 24 |         if content_type_arr.len() == 0{
 25 |             return;
 26 |         }
 27 |         let content_type = content_type_arr[0].trim();
 28 | 
 29 |         if req.method() == "POST" {
 30 |             match content_type {
 31 |                 "multipart/form-data" =>{
 32 |                     RequestParameters::decode_form_data(req, map);
 33 |                 },
 34 |                 "application/x-www-form-urlencoded" => {
 35 |                     RequestParameters::decode_url_encoded(req, map);
 36 |                 },
 37 |                 "application/json" => {
 38 |                     RequestParameters::decode_json(req, map);
 39 |                 },
 40 |                 "nothing" => {
 41 |                     // ignore body
 42 |                 },
 43 |                 _ =>{
 44 |                     error!("unknown content type: {}", content_type);
 45 |                 }
 46 |             }
 47 |         }
 48 | 
 49 |         RequestParameters::decode_url_query(req, map);
 50 |     }
 51 | 
 52 |     fn decode_url_query(req: &Request, map: &mut HashMap) {
 53 |         let iter = form_urlencoded::parse(req.raw_query_string().as_bytes());
 54 |         for (key,val) in iter {
 55 |             trace!("application/x-www-form-urlencoded '{}' => '{}'", key, val);
 56 |             let key = String::from(key);
 57 |             if !map.contains_key(&key){
 58 |                 map.insert(key, String::from(val));
 59 |             }
 60 |         }
 61 |     }
 62 | 
 63 |     fn decode_url_encoded(req: &Request, map: &mut HashMap) {
 64 |         let mut data = req.data().unwrap();
 65 |         let mut buf = Vec::new();
 66 |         match data.read_to_end(&mut buf) {
 67 |             Ok(_) => {
 68 |                 let iter = form_urlencoded::parse(&buf);
 69 |                 for (key,val) in iter {
 70 |                     trace!("application/x-www-form-urlencoded '{}' => '{}'", key, val);
 71 |                     let key = String::from(key);
 72 |                     if !map.contains_key(&key){
 73 |                         map.insert(key, String::from(val));
 74 |                     }
 75 |                 }
 76 |             }
 77 |             Err(_)=>{
 78 |                 error!("err");
 79 |             }
 80 |         }
 81 |     }
 82 | 
 83 |     fn decode_form_data(req: &Request, map: &mut HashMap) {
 84 |         let multipart = rouille::input::multipart::get_multipart_input(&req);
 85 |         match multipart {
 86 |             Ok(mut content)=>{
 87 |                 loop{
 88 |                     let field = content.next();
 89 |                     if let Some(mut field) = field {
 90 |                         if field.is_text(){
 91 |                             let mut buf = String::new();
 92 |                             let res = field.data.read_to_string(&mut buf);
 93 |                             if let Ok(_) = res {
 94 |                                 trace!("multipart/form-data '{}' => '{}'", field.headers.name, buf);
 95 |                                 let key = String::from(&(*field.headers.name));
 96 |                                 let val = buf;
 97 |                                 if !map.contains_key(&key){
 98 |                                     map.insert(key, String::from(val));
 99 |                                 }
100 |                             }
101 |                         }
102 |                     } else {
103 |                         break;
104 |                     }
105 |                 }
106 |             }
107 |             Err(_) => {
108 |                 error!("err");
109 |             }
110 |         };
111 |     }
112 | 
113 |     fn decode_json(req: &Request, map: &mut HashMap) {
114 |         let data = req.data();
115 |         if let Some(mut data) = data {
116 |             let mut buf = Vec::new();
117 |             match data.read_to_end(&mut buf) {
118 |                 Ok(_) => {
119 |                     let v: Result, serde_json::error::Error> = serde_json::from_slice(&buf);
120 |                     match v {
121 |                         Err(_) =>{
122 |                             error!("unable to decode json");
123 |                         },
124 |                         Ok(v) => {
125 |                             for (key, value) in v {
126 |                                 trace!("application/json {} => {}", key, value);
127 |                                 if !map.contains_key(&key){
128 |                                     if let Some(value) = value.as_u64() {
129 |                                         map.insert(key, String::from(value.to_string()));
130 |                                     }
131 |                                     else if let Some(value) = value.as_i64() {
132 |                                         map.insert(key, String::from(value.to_string()));
133 |                                     }
134 |                                     else if let Some(value) = value.as_f64() {
135 |                                         map.insert(key, String::from(value.to_string()));
136 |                                     }
137 |                                     else if let Some(value) = value.as_str() {
138 |                                         map.insert(key, String::from(value));
139 |                                     }
140 |                                     else if let Some(value) = value.as_bool() {
141 |                                         map.insert(key, String::from(value.to_string()));
142 |                                     }
143 |                                     else if let Some(value) = value.as_array() {
144 |                                         let list: Vec = value.into_iter().map(|item| {
145 |                                             if item.is_string(){
146 |                                                 item.as_str().unwrap().trim().to_string()
147 |                                             }else{
148 |                                                 String::from("")
149 |                                             }
150 |                                         }).filter(|item| {
151 |                                             item != ""
152 |                                         }).collect();
153 |                                         let value = list.join(",");
154 |                                         map.insert(key, value);
155 |                                     }
156 |                                     else{
157 |                                         error!("unsupported value type in json");
158 |                                     }
159 |                                 }
160 |                             }
161 |                         }
162 |                     }
163 |                 }
164 |                 Err(_) => {
165 |                     error!("error");
166 |                 }
167 |             }
168 |         }
169 |     }
170 | 
171 |     pub fn get_string(&self, name: &str) -> Option {
172 |         let v = self.values.get(name);
173 |         if let Some(v) = v {
174 |             return Some(String::from(v));
175 |         }
176 |         None
177 |     }
178 | 
179 |     pub fn get_bool(&self, name: &str, default: bool) -> bool {
180 |         let v = self.values.get(name);
181 |         if let Some(v) = v {
182 |             let parsed = v.parse::();
183 |             if let Ok(parsed) = parsed {
184 |                 return parsed;
185 |             }
186 |         }
187 |         default
188 |     }
189 | 
190 |     pub fn get_bool_opt(&self, name: &str) -> Result, ParseBoolError> {
191 |         let v = self.values.get(name);
192 |         v.map(|v| v.parse::()).transpose()
193 |     }
194 | 
195 |     pub fn get_number(&self, name: &str, default: u32) -> u32 {
196 |         let v = self.values.get(name);
197 |         if let Some(v) = v {
198 |             let parsed = v.parse::();
199 |             if let Ok(parsed) = parsed {
200 |                 return parsed;
201 |             }else{
202 |                 error!("could not parse '{}'", v);
203 |             }
204 |         }
205 |         default
206 |     }
207 | 
208 |     pub fn get_double(&self, name: &str, default: Option) -> Option {
209 |         let v = self.values.get(name);
210 |         if let Some(v) = v {
211 |             let parsed = v.parse::();
212 |             if let Ok(parsed) = parsed {
213 |                 return Some(parsed);
214 |             }else{
215 |                 error!("could not parse '{}'", v);
216 |             }
217 |         }
218 |         default
219 |     }
220 | }


--------------------------------------------------------------------------------
/src/api/data/station_check.rs:
--------------------------------------------------------------------------------
  1 | use chrono::NaiveDateTime;
  2 | use chrono::Utc;
  3 | use chrono::DateTime;
  4 | use chrono::SecondsFormat;
  5 | use crate::api::api_response::ApiResponse;
  6 | use crate::db::models::StationCheckItem;
  7 | use std::convert::TryFrom;
  8 | use std::error::Error;
  9 | use serde::{Serialize,Deserialize};
 10 | 
 11 | #[derive(PartialEq, Eq, Serialize, Deserialize)]
 12 | pub struct StationCheckV0 {
 13 |     pub stationuuid: String,
 14 |     pub checkuuid: String,
 15 |     pub source: String,
 16 |     pub codec: String,
 17 |     pub bitrate: String,
 18 |     pub hls: String,
 19 |     pub ok: String,
 20 |     pub urlcache: String,
 21 |     pub timestamp: String,
 22 | }
 23 | 
 24 | #[derive(PartialEq, Serialize, Deserialize)]
 25 | pub struct StationCheck {
 26 |     pub stationuuid: String,
 27 |     pub checkuuid: String,
 28 |     pub source: String,
 29 |     pub codec: String,
 30 |     pub bitrate: u32,
 31 |     pub hls: u8,
 32 |     pub ok: u8,
 33 |     pub timestamp_iso8601: Option>,
 34 |     pub timestamp: String,
 35 |     pub urlcache: String,
 36 | 
 37 |     pub metainfo_overrides_database: Option,
 38 |     pub public: Option,
 39 |     pub name: Option,
 40 |     pub description: Option,
 41 |     pub tags: Option,
 42 |     pub countrycode: Option,
 43 |     pub homepage: Option,
 44 |     pub favicon: Option,
 45 |     pub loadbalancer: Option,
 46 |     pub do_not_index: Option,
 47 |     
 48 |     pub countrysubdivisioncode: Option,
 49 |     pub server_software: Option,
 50 |     pub sampling: Option,
 51 |     pub timing_ms: Option,
 52 |     pub languagecodes: Option,
 53 |     pub ssl_error: Option,
 54 |     pub geo_lat: Option,
 55 |     pub geo_long: Option,
 56 | }
 57 | 
 58 | impl StationCheck {
 59 |     pub fn new(
 60 |         stationuuid: String,
 61 |         checkuuid: String,
 62 |         source: String,
 63 |         codec: String,
 64 |         bitrate: u32,
 65 |         hls: u8,
 66 |         ok: u8,
 67 |         timestamp_iso8601: Option>,
 68 |         timestamp: String,
 69 |         urlcache: String,
 70 | 
 71 |         metainfo_overrides_database: Option,
 72 |         public: Option,
 73 |         name: Option,
 74 |         description: Option,
 75 |         tags: Option,
 76 |         countrycode: Option,
 77 |         homepage: Option,
 78 |         favicon: Option,
 79 |         loadbalancer: Option,
 80 |         do_not_index: Option,
 81 |         
 82 |         countrysubdivisioncode: Option,
 83 |         server_software: Option,
 84 |         sampling: Option,
 85 |         timing_ms: u128,
 86 |         languagecodes: Option,
 87 |         ssl_error: u8,
 88 |         geo_lat: Option,
 89 |         geo_long: Option,
 90 |     ) -> Self {
 91 |         StationCheck {
 92 |             stationuuid,
 93 |             checkuuid,
 94 |             source,
 95 |             codec,
 96 |             bitrate,
 97 |             hls,
 98 |             ok,
 99 |             timestamp_iso8601,
100 |             timestamp,
101 |             urlcache,
102 | 
103 |             metainfo_overrides_database,
104 |             public,
105 |             name,
106 |             description,
107 |             tags,
108 |             countrycode,
109 |             homepage,
110 |             favicon,
111 |             loadbalancer,
112 |             do_not_index,
113 | 
114 |             countrysubdivisioncode,
115 |             server_software,
116 |             sampling,
117 |             timing_ms: Some(timing_ms),
118 |             languagecodes,
119 |             ssl_error: Some(ssl_error),
120 |             geo_lat,
121 |             geo_long,
122 |         }
123 |     }
124 | 
125 |     pub fn serialize_station_checks_csv(entries: Vec) -> Result> {
126 |         let mut wtr = csv::Writer::from_writer(Vec::new());
127 | 
128 |         for entry in entries {
129 |             wtr.serialize(entry)?;
130 |         }
131 |         
132 |         wtr.flush()?;
133 |         let x: Vec = wtr.into_inner()?;
134 |         Ok(String::from_utf8(x).unwrap_or("encoding error".to_string()))
135 |     }
136 | 
137 |     pub fn serialize_station_checks(entries: Vec) -> std::io::Result {
138 |         let mut xml = xml_writer::XmlWriter::new(Vec::new());
139 |         xml.begin_elem("result")?;
140 |         for entry in entries {
141 |             xml.begin_elem("check")?;
142 |             xml.attr_esc("stationuuid", &entry.stationuuid)?;
143 |             xml.attr_esc("checkuuid", &entry.checkuuid)?;
144 |             xml.attr_esc("source", &entry.source)?;
145 |             xml.attr_esc("codec", &entry.codec)?;
146 |             xml.attr_esc("bitrate", &entry.bitrate.to_string())?;
147 |             xml.attr_esc("hls", &entry.hls.to_string())?;
148 |             xml.attr_esc("ok", &entry.ok.to_string())?;
149 |             xml.attr_esc("urlcache", &entry.urlcache)?;
150 |             if let Some(timestamp_iso8601) = entry.timestamp_iso8601 {
151 |                 xml.attr_esc("timestamp_iso8601", ×tamp_iso8601.to_rfc3339_opts(SecondsFormat::Secs, true))?;
152 |             }
153 |             xml.attr_esc("timestamp", &entry.timestamp)?;
154 | 
155 |             xml.attr_esc("metainfo_overrides_database", &entry.metainfo_overrides_database.unwrap_or(0).to_string())?;
156 |             xml.attr_esc("public", &entry.public.unwrap_or(0).to_string())?;
157 |             xml.attr_esc("name", &entry.name.unwrap_or_default())?;
158 |             xml.attr_esc("description", &entry.description.unwrap_or_default())?;
159 |             xml.attr_esc("tags", &entry.tags.unwrap_or_default())?;
160 |             xml.attr_esc("homepage", &entry.homepage.unwrap_or_default())?;
161 |             xml.attr_esc("loadbalancer", &entry.loadbalancer.unwrap_or_default())?;
162 |             xml.attr_esc("favicon", &entry.favicon.unwrap_or_default())?;
163 |             xml.attr_esc("countrycode", &entry.countrycode.unwrap_or_default())?;
164 | 
165 |             xml.attr_esc("countrysubdivisioncode", &entry.countrysubdivisioncode.unwrap_or_default())?;
166 |             xml.attr_esc("server_software", &entry.server_software.unwrap_or_default())?;
167 |             xml.attr_esc("sampling", &entry.sampling.unwrap_or(0).to_string())?;
168 |             xml.attr_esc("timing_ms", &entry.timing_ms.unwrap_or(0).to_string())?;
169 |             xml.attr_esc("languagecodes", &entry.languagecodes.unwrap_or_default())?;
170 |             xml.attr_esc("ssl_error", &entry.ssl_error.unwrap_or(0).to_string())?;
171 |             if let Some(geo_lat) = &entry.geo_lat {
172 |                 xml.attr_esc("geo_lat", &geo_lat.to_string())?;
173 |             }
174 |             if let Some(geo_long) = &entry.geo_long {
175 |                 xml.attr_esc("geo_long", &geo_long.to_string())?;
176 |             }
177 |             xml.end_elem()?;
178 |         }
179 |         xml.end_elem()?;
180 |         xml.close()?;
181 |         xml.flush()?;
182 |         Ok(String::from_utf8(xml.into_inner()).unwrap_or("encoding error".to_string()))
183 |     }
184 | 
185 |     pub fn get_response(list: Vec, format: &str) -> Result> {
186 |         Ok(match format {
187 |             "csv" => ApiResponse::Text(StationCheck::serialize_station_checks_csv(list)?),
188 |             "json" => ApiResponse::Text(serde_json::to_string(&list)?),
189 |             "xml" => ApiResponse::Text(StationCheck::serialize_station_checks(list)?),
190 |             _ => ApiResponse::UnknownContentType,
191 |         })
192 |     }
193 | }
194 | 
195 | impl TryFrom for StationCheck {
196 |     type Error = Box;
197 | 
198 |     fn try_from(item: StationCheckV0) -> Result {
199 |         let timestamp_iso8601 = NaiveDateTime::parse_from_str(&item.timestamp, "%Y-%m-%d %H:%M:%S")
200 |             .ok()
201 |             .map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc));
202 | 
203 |         Ok(StationCheck {
204 |             stationuuid: item.stationuuid,
205 |             checkuuid: item.checkuuid,
206 |             source: item.source,
207 |             codec: item.codec,
208 |             bitrate: item.bitrate.parse()?,
209 |             hls: item.hls.parse()?,
210 |             ok: item.ok.parse()?,
211 |             timestamp_iso8601,
212 |             timestamp: item.timestamp,
213 |             urlcache: item.urlcache,
214 |             
215 |             metainfo_overrides_database: None,
216 |             public: None,
217 |             name: None,
218 |             description: None,
219 |             tags: None,
220 |             countrycode: None,
221 |             homepage: None,
222 |             favicon: None,
223 |             loadbalancer: None,
224 |             do_not_index: None,
225 | 
226 |             countrysubdivisioncode: None,
227 |             server_software: None,
228 |             sampling: None,
229 |             timing_ms: None,
230 |             languagecodes: None,
231 |             ssl_error: None,
232 |             geo_lat: None,
233 |             geo_long: None,
234 |         })
235 |     }
236 | }
237 | 
238 | impl From for StationCheck {
239 |     fn from(item: StationCheckItem) -> Self {
240 |         StationCheck::new(
241 |             item.station_uuid,
242 |             item.check_uuid,
243 |             item.source,
244 |             item.codec,
245 |             item.bitrate,
246 |             if item.hls { 1 } else { 0 },
247 |             if item.check_ok { 1 } else { 0 },
248 |             item.check_time_iso8601,
249 |             item.check_time,
250 |             item.url,
251 | 
252 |             if item.metainfo_overrides_database {Some(1)} else {Some(0)},
253 |             item.public.map(|x| if x {1} else {0}),
254 |             item.name,
255 |             item.description,
256 |             item.tags,
257 |             item.countrycode,
258 |             item.homepage,
259 |             item.favicon,
260 |             item.loadbalancer,
261 |             item.do_not_index.map(|x| if x {1} else {0}),
262 | 
263 |             item.countrysubdivisioncode,
264 |             item.server_software,
265 |             item.sampling,
266 |             item.timing_ms,
267 |             item.languagecodes,
268 |             if item.ssl_error { 1 } else { 0 },
269 |             item.geo_lat,
270 |             item.geo_long,
271 |         )
272 |     }
273 | }


--------------------------------------------------------------------------------
/src/db/db.rs:
--------------------------------------------------------------------------------
  1 | use crate::db::models::DBCountry;
  2 | use crate::db::models::DbStreamingServerNew;
  3 | use crate::db::models::DbStreamingServer;
  4 | use crate::db::models::StationCheckStepItem;
  5 | use crate::db::models::StationCheckStepItemNew;
  6 | use crate::api::data::Station;
  7 | use crate::db::models::StationClickItemNew;
  8 | use crate::db::models::State;
  9 | use crate::db::models::ExtraInfo;
 10 | use crate::db::models::DbStationItem;
 11 | use crate::db::models::StationCheckItem;
 12 | use crate::db::models::StationCheckItemNew;
 13 | use crate::db::models::StationChangeItemNew;
 14 | use crate::db::models::StationHistoryItem;
 15 | use crate::db::models::StationClickItem;
 16 | use std::error::Error;
 17 | use std::collections::HashMap;
 18 | 
 19 | pub trait DbConnection {
 20 |     fn get_station_count_broken(&self) -> Result>;
 21 |     fn get_station_count_working(&self) -> Result>;
 22 |     fn get_station_count_todo(&self, hours: u32) -> Result>;
 23 |     fn get_deletable_never_working(&self, seconds: u64) -> Result>;
 24 |     fn get_deletable_were_working(&self, seconds: u64) -> Result>;
 25 |     fn get_tag_count(&self) -> Result>;
 26 |     fn get_country_count(&self) -> Result>;
 27 |     fn get_language_count(&self) -> Result>;
 28 |     fn get_click_count_last_hour(&self) -> Result>;
 29 |     fn get_click_count_last_day(&self) -> Result>;
 30 |     fn get_stations_to_check(&mut self, hours: u32, itemcount: u32) -> Result, Box>;
 31 |     fn get_station_by_uuid(&self, id_str: &str) -> Result,Box>;
 32 |     fn get_stations_by_uuid(&self, uuids: Vec) -> Result,Box>;
 33 |     fn get_stations_by_column_multiple(&self,column_name: &str,search: Option,exact: bool,order: &str,reverse: bool,hidebroken: bool,offset: u32,limit: u32) -> Result, Box>;
 34 |     fn get_stations_by_all(&self,order: &str,reverse: bool,hidebroken: bool,offset: u32,limit: u32) -> Result, Box>;
 35 |     fn get_stations_uuid_order_by_changes(&mut self, min_change_count: u32) -> Result, Box>;
 36 |     fn get_stations_advanced(
 37 |         &self,name: Option,name_exact: bool,country: Option,country_exact: bool,countrycode: Option,
 38 |         state: Option,state_exact: bool,language: Option,
 39 |         language_exact: bool,tag: Option,tag_exact: bool,tag_list: Vec,
 40 |         codec: Option,
 41 |         bitrate_min: u32,bitrate_max: u32,has_geo_info: Option,has_extended_info: Option, is_https: Option, order: &str,reverse: bool,hidebroken: bool,offset: u32,limit: u32) -> Result, Box>;
 42 |     fn get_changes(&self, stationuuid: Option, changeuuid: Option, limit: u32) -> Result, Box>;
 43 |     fn get_changes_for_stations(&self, station_uuids: Vec) -> Result, Box>;
 44 | 
 45 |     fn add_station_opt(&self, name: Option, url: Option, homepage: Option, favicon: Option,
 46 |         countrycode: Option, state: Option, language: Option, languagecodes: Option, tags: Option, geo_lat: Option, geo_long: Option) -> Result>;
 47 | 
 48 |     fn get_stations_broken(&self, offset: u32, limit: u32) -> Result, Box>;
 49 |     fn get_stations_topvote(&self, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 50 |     fn get_stations_topclick(&self, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 51 |     fn get_stations_lastclick(&self, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 52 |     fn get_stations_lastchange(&self, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 53 |     fn get_stations_by_column(&self,column_name: &str,search: String,exact: bool,order: &str,reverse: bool,hidebroken: bool,offset: u32,limit: u32) -> Result, Box>;
 54 |     fn get_stations_by_server_uuids(&self,uuids: Vec, order: &str,reverse: bool,hidebroken: bool,offset: u32,limit: u32) -> Result, Box>;
 55 | 
 56 |     fn get_pull_server_lastid(&self, server: &str) -> Result, Box>;
 57 |     fn set_pull_server_lastid(&self, server: &str, lastid: &str) -> Result<(),Box>;
 58 |     fn get_pull_server_lastcheckid(&self, server: &str) -> Result, Box>;
 59 |     fn set_pull_server_lastcheckid(&self, server: &str, lastcheckid: &str) -> Result<(),Box>;
 60 |     fn get_pull_server_lastclickid(&self, server: &str) -> Result, Box>;
 61 |     fn set_pull_server_lastclickid(&self, server: &str, lastclickuuid: &str) -> Result<(),Box>;
 62 | 
 63 |     fn insert_station_by_change(&self, list_station_changes: &[StationChangeItemNew], source: &str) -> Result,Box>;
 64 | 
 65 |     fn get_extra(&self, table_name: &str, column_name: &str, search: Option, order: String, reverse: bool, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 66 |     fn get_1_n(&self, column: &str, search: Option, order: String, reverse: bool, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 67 |     fn get_countries(&self, search: Option, order: String, reverse: bool, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 68 |     fn get_states(&self, country: Option, search: Option, order: String, reverse: bool, hidebroken: bool, offset: u32, limit: u32) -> Result, Box>;
 69 |     fn get_checks(&self, stationuuid: Option, checkuuid: Option, seconds: u32, include_history: bool, limit: u32) -> Result, Box>;
 70 |     fn get_clicks(&self, stationuuid: Option, clickuuid: Option, seconds: u32) -> Result, Box>;
 71 | 
 72 |     fn insert_checks(&self, list: Vec) -> Result<(Vec,Vec,Vec), Box>;
 73 |     fn update_station_with_check_data(&self, list: &Vec, local: bool) -> Result<(), Box>;
 74 |     //fn update_station(&self, station: &DbStationItem, reason: &str) -> Result<(), Box>;
 75 |     fn delete_stationhistory_more_than(&self, itemcount: u32) -> Result<(), Box>;
 76 |     fn delete_stationhistory_byid_more_than(&self, stationuuid: String, itemcount: usize) -> Result<(), Box>;
 77 | 
 78 |     fn insert_clicks(&self, list: &Vec) -> Result<(), Box>;
 79 | 
 80 |     fn delete_never_working(&mut self, seconds: u64) -> Result<(), Box>;
 81 |     fn delete_were_working(&mut self, seconds: u64) -> Result<(), Box>;
 82 |     fn get_duplicated_stations(&self, column_key: &str, max_duplicates: usize) -> Result, Box>;
 83 |     fn delete_stations(&self, stationuuids: &[String]) -> Result<(), Box>;
 84 |     fn delete_old_checks(&mut self, seconds: u64) -> Result<(), Box>;
 85 |     fn delete_old_clicks(&mut self, seconds: u64) -> Result<(), Box>;
 86 |     fn delete_removed_from_history(&mut self) -> Result<(), Box>;
 87 |     fn delete_unused_streaming_servers(&mut self, seconds: u64) -> Result<(), Box>;
 88 |     fn delete_change_by_uuid(&mut self, changeuuids: &[String]) -> Result<(), Box>;
 89 |     fn remove_unused_ip_infos_from_stationclicks(&mut self, seconds: u64) -> Result<(), Box>;
 90 |     fn calc_country_field(&mut self) -> Result<(), Box>;
 91 |     fn resethistory(&mut self) -> Result<(), Box>;
 92 |     
 93 |     fn get_stations_with_empty_icon(&mut self) -> Result, Box>;
 94 |     fn get_stations_with_non_empty_icon(&mut self) -> Result, Box>;
 95 |     fn update_station_auto(&mut self, station: &DbStationItem, reason: &str) -> Result<(), Box>;
 96 | 
 97 |     fn update_stations_clickcount(&self) -> Result<(), Box>;
 98 | 
 99 |     fn get_stations_multi_items(&self, column_name: &str) -> Result, Box>;
100 |     fn get_cached_items(&self, table_name: &str, column_name: &str) -> Result, Box>;
101 |     fn update_cache_item(&self, tag: &String, count: u32, count_working: u32, table_name: &str, column_name: &str) -> Result<(), Box>;
102 |     fn insert_to_cache(&self, tags: HashMap<&String, (u32,u32)>, table_name: &str, column_name: &str) -> Result<(), Box>;
103 |     fn remove_from_cache(&self, tags: Vec<&String>, table_name: &str, column_name: &str) -> Result<(), Box>;
104 | 
105 |     fn vote_for_station(&self, ip: &str, station: Option) -> Result>;
106 |     fn increase_clicks(&self, ip: &str, station: &DbStationItem, seconds: u64) -> Result>;
107 |     fn sync_votes(&self, list: Vec) -> Result<(), Box>;
108 | 
109 |     fn insert_station_check_steps(&mut self, station_check_steps: &[StationCheckStepItemNew]) -> Result<(),Box>;
110 |     fn select_station_check_steps(&self) -> Result,Box>;
111 |     fn select_station_check_steps_by_stations(&self, stationuuids: &[String]) -> Result,Box>;
112 |     fn delete_old_station_check_steps(&mut self, seconds: u32) -> Result<(),Box>;
113 | 
114 |     fn get_servers_to_check(&mut self, hours: u32, chunksize: u32) -> Result, Box>;
115 |     fn get_streaming_servers_by_url(&mut self, items: Vec) -> Result, Box>;
116 |     fn get_streaming_servers(&self, order: &str,reverse: bool,offset: u32,limit: u32) -> Result, Box>;
117 |     fn get_streaming_servers_by_uuids(&self, uuids: Vec, order: &str,reverse: bool,offset: u32,limit: u32) -> Result, Box>;
118 |     fn get_streaming_servers_by_station_uuids(&self, uuids: Vec, order: &str,reverse: bool,offset: u32,limit: u32) -> Result, Box>;
119 |     fn insert_streaming_servers(&mut self, items: Vec) -> Result<(), Box>;
120 |     fn update_streaming_servers(&mut self, items: Vec) -> Result<(), Box>;
121 | }
122 | 


--------------------------------------------------------------------------------
/src/db/db_mysql/conversions.rs:
--------------------------------------------------------------------------------
  1 | use crate::db::models::DbStationItem;
  2 | use crate::db::models::StationCheckItem;
  3 | use crate::db::models::StationHistoryItem;
  4 | use crate::db::models::StationClickItem;
  5 | use mysql;
  6 | use mysql::Row;
  7 | 
  8 | impl From for StationCheckItem {
  9 |     fn from(mut row: Row) -> Self {
 10 |         const ZERO: i32 = 0;
 11 |         StationCheckItem {
 12 |             check_id:                      row.take("CheckID").unwrap(),
 13 |             station_uuid:                  row.take("StationUuid").unwrap_or("".to_string()),
 14 |             check_uuid:                    row.take("CheckUuid").unwrap_or("".to_string()),
 15 |             source:                        row.take("Source").unwrap_or("".to_string()),
 16 |             codec:                         row.take_opt("Codec").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 17 |             bitrate:                       row.take_opt("Bitrate").unwrap_or(Ok(0)).unwrap_or(0),
 18 |             hls:                           row.take_opt("Hls").unwrap_or(Ok(ZERO)).unwrap_or(0) == 1,
 19 |             check_ok:                      row.take_opt("CheckOK").unwrap_or(Ok(ZERO)).unwrap_or(0) == 1,
 20 |             check_time_iso8601:            row.take_opt("CheckTime").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 21 |             check_time:                    row.take_opt("CheckTimeFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 22 |             url:                           row.take_opt("UrlCache").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 23 |             metainfo_overrides_database:   row.take_opt("MetainfoOverridesDatabase").unwrap_or(Ok(ZERO)).unwrap_or(0) == 1,
 24 |             public:                        row.take_opt("Public").transpose().unwrap_or(None).map(|x: u32| x == 1),
 25 |             name:                          row.take_opt("Name").transpose().unwrap_or(None),
 26 |             description:                   row.take_opt("Description").transpose().unwrap_or(None),
 27 |             tags:                          row.take_opt("Tags").transpose().unwrap_or(None),
 28 |             countrycode:                   row.take_opt("CountryCode").transpose().unwrap_or(None),
 29 |             homepage:                      row.take_opt("Homepage").transpose().unwrap_or(None),
 30 |             favicon:                       row.take_opt("Favicon").transpose().unwrap_or(None),
 31 |             loadbalancer:                  row.take_opt("Loadbalancer").transpose().unwrap_or(None),
 32 |             do_not_index:                  row.take_opt("DoNotIndex").transpose().unwrap_or(None),
 33 | 
 34 |             countrysubdivisioncode:        row.take_opt("CountrySubdivisionCode").transpose().unwrap_or(None),
 35 |             server_software:               row.take_opt("ServerSoftware").transpose().unwrap_or(None),
 36 |             sampling:                      row.take_opt("Sampling").transpose().unwrap_or(None),
 37 |             timing_ms:                     row.take_opt("TimingMs").unwrap_or(Ok(0)).unwrap_or(0),
 38 |             languagecodes:                 row.take_opt("LanguageCodes").transpose().unwrap_or(None),
 39 |             ssl_error:                     row.take_opt("SslError").unwrap_or(Ok(0)).unwrap_or(0) == 1,
 40 |             geo_lat:                       row.take_opt("GeoLat").transpose().unwrap_or(None),
 41 |             geo_long:                      row.take_opt("GeoLong").transpose().unwrap_or(None),
 42 |         }
 43 |     }
 44 | }
 45 | 
 46 | impl From for DbStationItem {
 47 |     fn from(mut row: Row) -> Self {
 48 |         DbStationItem {
 49 |             id:                          row.take("StationID").unwrap(),
 50 |             changeuuid:                  row.take("ChangeUuid").unwrap(),
 51 |             stationuuid:                 row.take("StationUuid").unwrap_or("".to_string()),
 52 |             serveruuid:                  row.take("ServerUuid").unwrap_or(None),
 53 |             name:                        row.take_opt("Name").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 54 |             url:                         row.take_opt("Url").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 55 |             url_resolved:                row.take_opt("UrlCache").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 56 |             codec:                       row.take_opt("Codec").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 57 |             bitrate:                     row.take_opt("Bitrate").unwrap_or(Ok(0)).unwrap_or(0),
 58 |             hls:                         row.take_opt("Hls").unwrap_or(Ok(0)).unwrap_or(0)==1,
 59 |             lastcheckok:                 row.take_opt("LastCheckOK").unwrap_or(Ok(0)).unwrap_or(0)==1,
 60 |             favicon:                     row.take_opt("Favicon").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 61 |             tags:                        row.take_opt("Tags").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 62 |             country:                     row.take_opt("Country").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 63 |             countrycode:                 row.take_opt("CountryCode").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 64 |             iso_3166_2:                  row.take_opt("CountrySubdivisionCode").transpose().unwrap_or(None),
 65 |             state:                       row.take_opt("Subcountry").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 66 |             language:                    row.take_opt("Language").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 67 |             languagecodes:               row.take_opt("LanguageCodes").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 68 |             votes:                       row.take_opt("Votes").unwrap_or(Ok(0)).unwrap_or(0),
 69 |             lastchangetime_iso8601:      row.take_opt("Creation").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 70 |             lastchangetime:              row.take_opt("CreationFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 71 |             homepage:                    row.take_opt("Homepage").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 72 |             lastchecktime_iso8601:       row.take_opt("LastCheckTime").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 73 |             lastchecktime:               row.take_opt("LastCheckTimeFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 74 |             lastcheckoktime_iso8601:     row.take_opt("LastCheckOkTime").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 75 |             lastcheckoktime:             row.take_opt("LastCheckOkTimeFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 76 |             lastlocalchecktime_iso8601:  row.take_opt("LastLocalCheckTime").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 77 |             lastlocalchecktime:          row.take_opt("LastLocalCheckTimeFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 78 |             clicktimestamp_iso8601:      row.take_opt("ClickTimestamp").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
 79 |             clicktimestamp:              row.take_opt("ClickTimestampFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 80 |             clickcount:                  row.take_opt("clickcount").unwrap_or(Ok(0)).unwrap_or(0),
 81 |             clicktrend:                  row.take_opt("ClickTrend").unwrap_or(Ok(0)).unwrap_or(0),
 82 |             ssl_error:                   row.take_opt("SslError").unwrap_or(Ok(0)).unwrap_or(0)==1,
 83 |             geo_lat:                     row.take_opt("GeoLat").transpose().unwrap_or(None),
 84 |             geo_long:                    row.take_opt("GeoLong").transpose().unwrap_or(None),
 85 |             has_extended_info:           row.take_opt("ExtendedInfo").transpose().unwrap_or(None),
 86 |         }
 87 |     }
 88 | }
 89 | 
 90 | impl From for StationHistoryItem {
 91 |     fn from(mut row: Row) -> Self {
 92 |         StationHistoryItem {
 93 |             id:                          row.take("StationChangeID").unwrap(),
 94 |             changeuuid:                  row.take("ChangeUuid").unwrap(),
 95 |             stationuuid:                 row.take("StationUuid").unwrap(),
 96 |             name:                        row.take_opt("Name").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 97 |             url:                         row.take_opt("Url").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 98 |             favicon:                     row.take_opt("Favicon").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
 99 |             tags:                        row.take_opt("Tags").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
100 |             countrycode:                 row.take_opt("CountryCode").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
101 |             state:                       row.take_opt("Subcountry").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
102 |             language:                    row.take_opt("Language").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
103 |             languagecodes:               row.take_opt("LanguageCodes").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
104 |             votes:                       row.take_opt("Votes").unwrap_or(Ok(0)).unwrap_or(0),
105 |             lastchangetime_iso8601:      row.take_opt("Creation").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
106 |             lastchangetime:              row.take_opt("CreationFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
107 |             homepage:                    row.take_opt("Homepage").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
108 |             geo_lat:                     row.take_opt("GeoLat").transpose().unwrap_or(None),
109 |             geo_long:                    row.take_opt("GeoLong").transpose().unwrap_or(None),
110 |         }
111 |     }
112 | }
113 | 
114 | impl From for StationClickItem {
115 |     fn from(mut row: Row) -> Self {
116 |         StationClickItem {
117 |             id:                        row.take("ClickID").unwrap(),
118 |             clickuuid:                 row.take("ClickUuid").unwrap(),
119 |             stationuuid:               row.take("StationUuid").unwrap(),
120 |             ip:                        row.take_opt("IP").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
121 |             clicktimestamp_iso8601:    row.take_opt("ClickTimestamp").transpose().ok().flatten().map(|x|chrono::DateTime::::from_naive_utc_and_offset(x, chrono::Utc)),
122 |             clicktimestamp:            row.take_opt("ClickTimestampFormated").unwrap_or(Ok("".to_string())).unwrap_or("".to_string()),
123 |         }
124 |     }
125 | }


--------------------------------------------------------------------------------
/src/main.rs:
--------------------------------------------------------------------------------
  1 | #[macro_use]
  2 | extern crate clap;
  3 | #[macro_use]
  4 | extern crate mysql;
  5 | #[macro_use]
  6 | extern crate log;
  7 | #[macro_use]
  8 | extern crate prometheus;
  9 | 
 10 | use crate::cli::delete_duplicate_changes;
 11 | use crate::cli::resethistory;
 12 | use crate::config::Config;
 13 | use crate::db::DbConnection;
 14 | use crate::db::MysqlConnection;
 15 | use crate::pull::UuidWithTime;
 16 | use core::fmt::Display;
 17 | use core::fmt::Formatter;
 18 | use reqwest::blocking::Client;
 19 | use signal_hook;
 20 | use signal_hook::consts::SIGHUP;
 21 | use signal_hook::iterator::Signals;
 22 | use std::error::Error;
 23 | use std::time::Duration;
 24 | use std::time::Instant;
 25 | use std::{thread, time};
 26 | 
 27 | mod api;
 28 | mod check;
 29 | mod checkserver;
 30 | mod cleanup;
 31 | mod cli;
 32 | mod config;
 33 | mod db;
 34 | mod logger;
 35 | mod pull;
 36 | mod refresh;
 37 | 
 38 | #[derive(Debug, Clone)]
 39 | enum MainError {
 40 |     LoggerInitError(String),
 41 | }
 42 | 
 43 | impl Display for MainError {
 44 |     fn fmt(&self, f: &mut Formatter) -> std::fmt::Result {
 45 |         match *self {
 46 |             MainError::LoggerInitError(ref msg) => {
 47 |                 write!(f, "Unable to initialize logger: {}", msg)
 48 |             }
 49 |         }
 50 |     }
 51 | }
 52 | 
 53 | impl Error for MainError {}
 54 | 
 55 | fn jobs(conn: C)
 56 | where
 57 |     C: DbConnection + Clone + Send,
 58 | {
 59 |     let mut once_refresh_config = false;
 60 |     let mut once_pull = true;
 61 |     let mut once_cleanup = true;
 62 |     let mut once_check = true;
 63 |     let mut once_refresh_caches = true;
 64 | 
 65 |     let mut last_time_refresh_config = Instant::now();
 66 |     let mut last_time_pull = Instant::now();
 67 |     let mut last_time_cleanup = Instant::now();
 68 |     let mut last_time_check = Instant::now();
 69 |     let mut last_time_refresh_caches = Instant::now();
 70 | 
 71 |     let mut list_deleted: Vec = vec![];
 72 |     let client = Client::new();
 73 | 
 74 |     thread::spawn(move || loop {
 75 |         let config = config::get_config()
 76 |             .expect("No config loaded")
 77 |             .lock()
 78 |             .expect("Config could not be pulled from shared memory.")
 79 |             .clone();
 80 |         if config.refresh_config_interval.as_secs() > 0
 81 |             && (once_refresh_config
 82 |                 || last_time_refresh_config.elapsed().as_secs()
 83 |                     >= config.refresh_config_interval.as_secs())
 84 |         {
 85 |             once_refresh_config = false;
 86 |             last_time_refresh_config = Instant::now();
 87 |             match config::load_all_extra_configs(&config) {
 88 |                 Ok(_) => {}
 89 |                 Err(err) => {
 90 |                     error!("Reload config: {}", err);
 91 |                 }
 92 |             }
 93 |         }
 94 | 
 95 |         if config.servers_pull.len() > 0
 96 |             && (once_pull
 97 |                 || last_time_pull.elapsed().as_secs() >= config.mirror_pull_interval.as_secs())
 98 |         {
 99 |             once_pull = false;
100 |             once_refresh_caches = true;
101 |             last_time_pull = Instant::now();
102 |             let result = pull::pull_worker(
103 |                 &client,
104 |                 conn.clone(),
105 |                 &config.servers_pull,
106 |                 config.chunk_size_changes,
107 |                 config.chunk_size_checks,
108 |                 config.max_duplicates,
109 |                 &mut list_deleted,
110 |             );
111 |             match result {
112 |                 Ok(_) => {}
113 |                 Err(err) => {
114 |                     error!("Error in pull worker: {}", err);
115 |                 }
116 |             }
117 |             // remove items from deleted list after 1 day
118 |             list_deleted.retain(|item| item.instant.elapsed().as_secs() < 3600 * 24);
119 |             debug!(
120 |                 "List of deleted station uuids (duplicates): len={}",
121 |                 list_deleted.len()
122 |             );
123 |         }
124 | 
125 |         if config.cleanup_interval.as_secs() > 0
126 |             && (once_cleanup
127 |                 || last_time_cleanup.elapsed().as_secs() >= config.cleanup_interval.as_secs())
128 |         {
129 |             once_cleanup = false;
130 |             once_refresh_caches = true;
131 |             last_time_cleanup = Instant::now();
132 |             let result = cleanup::do_cleanup(
133 |                 config.delete,
134 |                 conn.clone(),
135 |                 config.click_valid_timeout.as_secs(),
136 |                 config.broken_stations_never_working_timeout.as_secs(),
137 |                 config.broken_stations_timeout.as_secs(),
138 |                 config.checks_timeout.as_secs(),
139 |                 config.clicks_timeout.as_secs(),
140 |             );
141 |             if let Err(error) = result {
142 |                 error!("Error: {}", error);
143 |             }
144 |         }
145 | 
146 |         if config.enable_check
147 |             && (once_check || last_time_check.elapsed().as_secs() >= config.pause.as_secs())
148 |         {
149 |             trace!(
150 |                 "Check started.. (concurrency: {}, chunksize: {})",
151 |                 config.concurrency,
152 |                 config.check_stations
153 |             );
154 |             once_check = false;
155 |             once_refresh_caches = true;
156 |             last_time_check = Instant::now();
157 |             let result = check::dbcheck(
158 |                 conn.clone(),
159 |                 &config.source,
160 |                 config.concurrency,
161 |                 config.check_stations,
162 |                 config.tcp_timeout.as_secs(),
163 |                 config.max_depth,
164 |                 config.retries,
165 |                 config.check_servers,
166 |                 config.recheck_existing_favicon,
167 |                 config.enable_extract_favicon,
168 |                 config.favicon_size_min,
169 |                 config.favicon_size_max,
170 |                 config.favicon_size_optimum,
171 |             );
172 | 
173 |             match result {
174 |                 Ok(_) => {}
175 |                 Err(err) => {
176 |                     error!("Check worker error: {}", err);
177 |                 }
178 |             }
179 |             if config.check_servers {
180 |                 let result = checkserver::do_check(
181 |                     conn.clone(),
182 |                     config.check_servers_chunksize,
183 |                     config.concurrency,
184 |                 );
185 |                 match result {
186 |                     Ok(_) => {}
187 |                     Err(err) => {
188 |                         error!("Check worker error: {}", err);
189 |                     }
190 |                 }
191 |             }
192 |         }
193 | 
194 |         if config.update_caches_interval.as_secs() > 0
195 |             && (once_refresh_caches
196 |                 || last_time_refresh_caches.elapsed().as_secs()
197 |                     >= config.update_caches_interval.as_secs())
198 |         {
199 |             once_refresh_caches = false;
200 |             last_time_refresh_caches = Instant::now();
201 |             let result = refresh::refresh_all_caches(conn.clone());
202 |             match result {
203 |                 Ok(_) => {}
204 |                 Err(err) => {
205 |                     error!("Refresh worker error: {}", err);
206 |                 }
207 |             }
208 |         }
209 | 
210 |         thread::sleep(Duration::from_secs(10));
211 |     });
212 | }
213 | 
214 | fn mainloopitem(mut connection: MysqlConnection, config: Config) -> Result<(), Box> {
215 |     trace!("mainloopitem()");
216 |     if config.no_migrations {
217 |         if connection.migrations_needed()? {
218 |             debug!("Migrations are not allowed but not needed.");
219 |         } else {
220 |             panic!("Migrations are needed but not allowed by parameter!");
221 |         }
222 |     } else {
223 |         connection.do_migrations(
224 |             config.ignore_migration_errors,
225 |             config.allow_database_downgrade,
226 |         )?;
227 |     }
228 |     use config::ConfigSubCommand;
229 |     match config.sub_command {
230 |         ConfigSubCommand::Migrate => {}
231 |         ConfigSubCommand::CleanHistory => {
232 |             delete_duplicate_changes(&mut connection, 3)?;
233 |         }
234 |         ConfigSubCommand::ResetHistory => {
235 |             resethistory(&mut connection)?;
236 |         }
237 |         _ => {
238 |             jobs(connection.clone());
239 |             api::start(connection, config);
240 |         }
241 |     }
242 |     Ok(())
243 | }
244 | 
245 | fn mainloop() -> Result<(), Box> {
246 |     // load config
247 |     config::load_main_config()?;
248 |     let config = {
249 |         config::get_config()
250 |             .expect("config could not be loaded")
251 |             .lock()
252 |             .expect("could not load config from shared mem")
253 |             .clone()
254 |     };
255 |     logger::setup_logger(config.log_level, &config.log_dir, config.log_json)
256 |         .map_err(|e| MainError::LoggerInitError(e.to_string()))?;
257 |     info!("Config: {:#?}", config);
258 |     config::load_all_extra_configs(&config)?;
259 | 
260 |     let config2 = config.clone();
261 | 
262 |     if let config::ConfigSubCommand::None = config2.sub_command {
263 |         thread::spawn(|| loop {
264 |             let connection = db::MysqlConnection::new(&config2.connection_string);
265 |             match connection {
266 |                 Ok(connection) => {
267 |                     match mainloopitem(connection, config2) {
268 |                         Err(err) => {
269 |                             error!("Error: {}", err);
270 |                         }
271 |                         Ok(_) => {}
272 |                     }
273 |                     break;
274 |                 }
275 |                 Err(e) => {
276 |                     error!("DB connection error: {}", e);
277 |                     thread::sleep(time::Duration::from_millis(1000));
278 |                 }
279 |             }
280 |         });
281 |         let mut signals = Signals::new(&[SIGHUP])?;
282 |         for signal in &mut signals {
283 |             match signal {
284 |                 SIGHUP => {
285 |                     info!("received HUP, reload config");
286 |                     config::load_main_config()?;
287 |                     config::load_all_extra_configs(&config)?;
288 |                 }
289 |                 _ => unreachable!(),
290 |             }
291 |         }
292 |     } else {
293 |         let connection = db::MysqlConnection::new(&config2.connection_string);
294 |         match connection {
295 |             Ok(connection) => {
296 |                 mainloopitem(connection, config2)?;
297 |             }
298 |             Err(e) => {
299 |                 error!("DB connection error: {}", e);
300 |                 thread::sleep(time::Duration::from_millis(1000));
301 |             }
302 |         }
303 |     }
304 | 
305 |     Ok(())
306 | }
307 | 
308 | fn main() -> Result<(), Box> {
309 |     mainloop()?;
310 |     Ok(())
311 | }
312 | 


--------------------------------------------------------------------------------
/CHANGELOG.md:
--------------------------------------------------------------------------------
  1 | # Changelog
  2 | All notable changes to this project will be documented in this file.
  3 | 
  4 | The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
  5 | and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  6 | 
  7 | ## [Unreleased]
  8 | ## [0.7.24] 2022-05-01
  9 | ### Fixed
 10 | * CLI: cleanup history did not correctly find duplicates
 11 | * CHECK: update all fields for stations
 12 | * CHECK: do check first and then local corrections
 13 | ## [0.7.23] 2022-04-18
 14 | ### Added
 15 | * CLI: basic cli commands for database maintenance
 16 | * API: countrycode field to countries endpoint (#148)
 17 | ### Changed
 18 | * API: Deprecated countrycodes endpoint
 19 | * API: support uppercase stationuuids (#147)
 20 | ### Fixed
 21 | * CHECK: Station compare and update does not create new history entries if not needed anymore
 22 | 
 23 | ## [0.7.22] 2021-11-10
 24 | ### Fixed
 25 | * CHECK: check for http result code for favicons
 26 | * PULL: insert duplicates
 27 | 
 28 | ## [0.7.21] 2021-11-09
 29 | ### Added
 30 | * CHECK: check websites for favicons, if favicon is currently empty
 31 | * CLEANUP: recheck favicon every day if still existing and delete from station if not
 32 | * CONFIG: regular automatic load/download of config
 33 | * CONFIG: reload main config on HUP and regular automatic config reload (does not get used for db connection and api, but for regular jobs)
 34 | * CONFIG: added error log level, now there are 5 log levels, 0-4
 35 | * CHECK: tags replace
 36 | * API: added identifier and image tags to xspf output
 37 | 
 38 | ### Changed
 39 | * CLEANUP: add station change entries into the database for language and favicon updates
 40 | * DB: use transactions for migrations
 41 | * DB: allow update of existing stations with pulled changes
 42 | 
 43 | ### Fixed
 44 | * CONFIG: replace config in shared memory
 45 | 
 46 | ## [0.7.20] 2021-10-16
 47 | ### Added
 48 | * CLEANUP: language replace, language to code
 49 | * API: iso 639 code output to language list
 50 | * DB: use binary collation for tags to fix compare
 51 | * CONFIG: allow pulling of replacement csv info from web
 52 | * CONFIG: reload csv list files on HUP signal
 53 | 
 54 | ## [0.7.19] 2021-10-11
 55 | ### Changed
 56 | * DOCKER: use alpine as base image
 57 | 
 58 | ### Fixed
 59 | * CHECK: update stations with new languagecodes
 60 | * DB: check for empty lists in streaming servers api
 61 | * DB: collect streaming servers only if enabled
 62 | 
 63 | ## [0.7.18] 2021-10-10
 64 | ### Changed
 65 | * DB: removed fragment and query from streaming server urls
 66 | * CHECK: removed fragment and query from streaming server urls
 67 | * CLEANUP: remove unreferenced streaming server after 1 day
 68 | 
 69 | ## [0.7.17] 2021-10-10
 70 | ### Added
 71 | * API: new order type "changetimestamp"
 72 | * API: streaming servers
 73 | * DB: streaming servers
 74 | * CHECK: streaming servers
 75 | 
 76 | ## [0.7.16] 2021-09-29
 77 | ### Fixed
 78 | * DB: save geolat and geolong to checks table
 79 | 
 80 | ## [0.7.15] 2021-09-05
 81 | ### Added
 82 | * API: search by extended info marker
 83 | * API: search by is_https
 84 | * API: exposed iso_3166_2 in station lists
 85 | 
 86 | ### Fixed
 87 | * DB: countrysubdivisioncode is now able to hold full 6 chars
 88 | 
 89 | ### Updated
 90 | * dependencies
 91 | 
 92 | ## [0.7.14] 2021-05-31
 93 | ### Fixed
 94 | * CHECK: new av-stream-info version does fix handling of depth limit for playlists
 95 | 
 96 | ## [0.7.13] 2021-05-02
 97 | ### Added
 98 | * API: limit and offset parameters to countries, countrycodes, languages, tags, states, codecs endpoints
 99 | * PULL: automatically remove duplicates from database
100 | 
101 | ## [0.7.12] 2021-04-22
102 | ### Added
103 | * API: multiple station endpoints do not support limit,offset and hidebroken parameters
104 | * DOCS: added missing hidebroken to advanced search
105 | 
106 | ### Fixed
107 | * API: accept numbers in json post
108 | 
109 | ### Removed
110 | * API: /stations/improvable
111 | 
112 | ## [0.7.11] 2021-04-16
113 | ### Fixed
114 | * CHECK: insert of station checks did not always use pregenerated uuids
115 | 
116 | ## [0.7.10] 2021-04-16
117 | ### Added
118 | * API: fields server_location and server_country_code to /json/config and /xml/config
119 | * CHECK: added attribute ssl_error to /json/checks endpoint
120 | * API: /json/checksteps, /xml/checksteps, /csv/checksteps
121 | * API: languagecodes, geo_lat, geo_long
122 | * CHECK: extract languagecodes, geo_lat, geo_long
123 | * API: iso 8601 datetime fields
124 | 
125 | ### Changed
126 | * CHECK: do not ignore streams with broken ssl, just mark them
127 | * Dependency upgrade: av-stream-info-rust 0.10.0
128 | 
129 | ### Fixed
130 | * CHECK: do not throw away important check
131 | 
132 | ## [0.7.9] 2021-04-07
133 | ### Fixed
134 | * PULL: fixed importing external extra check info
135 | 
136 | ## [0.7.8] 2021-04-07
137 | ### Added
138 | * DB: save more information from checks (ServerSoftware, Sampling, LanguageCodes, TimingMs, CountrySubdivisionCode)
139 | * API: add more information for checks
140 | 
141 | ## [0.7.7] 2021-04-05
142 | ### Added
143 | * CLEAN: clean urls in db
144 | ### Changed
145 | * API: check url and homepage on insert if correct
146 | * CHECK: always ignore text/html urls
147 | 
148 | ## [0.7.6] 2021-04-03
149 | ### Added
150 | * API: limit parameter for format/station/changed
151 | * API: limit parameter for format/checks 
152 | * PULL: parameter chunk-size-changes
153 | * PULL: parameter chunk-size-checks
154 | 
155 | ### Fixed
156 | * DB: database name is now allowed to be different from "radio"
157 | * CHECK: bitrate detection of stream works now on multiple bitrate headers
158 | * PULL: try to import stations that are not existing locally for pulled checks
159 | * PULL: votes syncing (#111)
160 | 
161 | ### Changed
162 | * Dependency upgrade: prometheus 0.12.0
163 | * Dependency upgrade: redis 0.20.0
164 | * Dependency upgrade: reqwest 0.11.2
165 | * Dependency upgrade: av-stream-info-rust 0.8.2
166 | 
167 | ## [0.7.5] 2021-01-05
168 | ### Changed
169 | * Use do-not-index header of streams to delete them from the database
170 | * Override "Url" field in database if stream override is set
171 | * Support additional stream headers from https://www.stream-meta.info/version_2_headers.html
172 | 
173 | ### Fixed
174 | * DOCKER: added root certificates to container
175 | 
176 | ## [0.7.4] 2020-12-11
177 | ### Changed
178 | - PKG: removed mysql dependency from deb package, this enables use of mariadb
179 | ### Fixed
180 | * Docker build
181 | 
182 | ## [0.7.3] 2020-12-08
183 | ### Added
184 | - API: CSV output
185 | 
186 | ### Changed
187 | - API: ignore country parameter on station add. it is autogenerated from countrycode (ISO 3166-1 Alpha2).
188 | - API: allow only countrycodes with the exact length of 2 on insert.
189 | - API: ignore case on subtable selects (/format/codecs, /format/countries, /format/countrycodes)
190 | - CLEAN: calculate country column in database from countrycode every cleanup cycle
191 | - CLEAN: remove stations from history, that do not have an active entry in main station table
192 | 
193 | ## [0.7.2] 2020-09-14
194 | ### Fixed
195 | - API: clicks are counted correctly
196 | 
197 | ## [0.7.1] 2020-06-17
198 | ### Added
199 | - API: endpoint for fetching multiple streams by uuid
200 | - PULL: use correct user agent
201 | 
202 | ### Fixed
203 | - CACHE: shorter keys to make memcached work
204 | 
205 | ## [0.7.0] 2020-06-15
206 | ### Added
207 | - API: Response caching with builtin, redis and memcached
208 | 
209 | ### Changed
210 | - LOG: Ignore content-type nothing in requests
211 | 
212 | ## [0.6.16] 2020-05-27
213 | ### Fixed
214 | - METRICS: do not expose search information in call counts
215 | 
216 | ## [0.6.15] 2020-05-27
217 | ### Added
218 | - Support for JSON log format
219 | - Show api call timing information in log file
220 | - Split up api calls in prometheus endpoint with tags
221 | 
222 | ### Fixed
223 | - CLI: Verbose did not work
224 | - Fixed station voting from IPv6 address (Fixed: #69)
225 | 
226 | ### Changed
227 | - DEPENDENCY: Use fern logger instead of env_logger
228 | - API: Use limit parameter if limit is not otherwise provided (Fixed: #64)
229 | 
230 | ### Removed
231 | - API: Old style click count from metrics
232 | 
233 | ## [0.6.14] 2020-04-13
234 | ### Fixed
235 | - PLS files had title and file content reversed
236 | 
237 | ## [0.6.13] 2020-04-13
238 | ### Fixed
239 | - Always add CORS header, even on errors
240 | 
241 | ### Added
242 | - Answer OPTIONS requests for better support of CORS
243 | 
244 | ## [0.6.12] 2020-04-12
245 | ### Added
246 | - METRICS: station_clicks, api_calls
247 | - API: codec parameter to advanced search
248 | 
249 | ### Fixed
250 | - Use utc timestamp for clicks in database
251 | 
252 | ## [0.6.11] 2020-02-17
253 | ### Added
254 | - API: Added stationuuid to m3u output
255 | 
256 | ### Fixed
257 | - API: Filter order column for extra tables
258 | 
259 | ## [0.6.10] 2020-02-01
260 | ### Fixed
261 | - API: Wrong links in docs
262 | - API: Faster select from stationcheck view with added index
263 | - SYNC: Only update stations with changed votes
264 | - SYNC: Faster update of station's clicks with added index
265 | 
266 | ## [0.6.9] 2020-02-01
267 | ### Added
268 | - DEB: logrotate config
269 | - API: more exports to metrics
270 | - API: config endpoint
271 | 
272 | ### Fixed
273 | - API: return change lists (stations, checks, clicks) from start if lastuuid not found
274 | 
275 | ### Changed
276 | - CLEANUP: remove all non http/https content from favicon field of stations
277 | - ANSIBLE: disabled all apache2 logging by default
278 | - CONFIG: type errors do not get ignored with default value
279 | - CONFIG: use human readable durations instead of fixed seconds or hours
280 | 
281 | ## [0.6.8] 2020-01-22
282 | ### Fixed
283 | - Fixed wrong sql delete
284 | - Ignore station checks and clicks on pull, when there is no station
285 | 
286 | ### Added
287 | - Show "hidebroken" in docs for station query
288 | 
289 | ## [0.6.7] 2020-01-19
290 | ### Fixed
291 | - Migrations on mysql
292 | 
293 | ## [0.6.6] 2020-01-19
294 | ### Added
295 | - Simple sync of votes, may drop some votes
296 | 
297 | ### Fixed
298 | - Broken stations were not marked
299 | - Make foreign keys of StationClick and StationCheckHistory on delete cascade
300 | 
301 | ## [0.6.5] 2020-01-18
302 | ### Added
303 | - Ansible role and example playbook for debian/ubuntu
304 | 
305 | ### Fixed
306 | - Station checks
307 | - Always use UTC time in database
308 | - Faster check insert with mysql 5.7
309 | 
310 | ### Changed
311 | - Default install paths changed
312 | - IPs for clicks are only kept until not needed anymore (default 24 hours)
313 | 
314 | ## [0.6.4] 2020-01-14
315 | ### Fixed
316 | - Insert of checks does now ignore duplicates
317 | - Resuse checkuuids of pullserver, do not generate new ones on pull
318 | - Incremental results of /format/stations/changed, /format/checks and /format/clicks do work reliably
319 | 
320 | ### Changed
321 | - Replaced StationCheck table with view on StationCheckHistory
322 | - Migration deletes contents of StationCheckHistory, to remove unconnected uuids
323 | - Migration deletes contents of PullServers to force re-pulling of every server
324 | 
325 | ## [0.6.3] 2020-01-12
326 | ### Added
327 | - Script for building distribution file
328 | 
329 | ### Changed
330 | - Debian package
331 | - Improved readme
332 | 
333 | ## [0.6.2] 2020-01-11
334 | ### Added
335 | - Sync for clicks
336 | - Api for clicks /format/clicks
337 | 
338 | ### Changed
339 | - Document "/format/url" endpoint as station click endpoint
340 | - Document returned structs "station", "checks"
341 | - Faster sync
342 | 
343 | ### Fixed
344 | - Clickcount calculation for each station
345 | - Error in /format/stations/changed endpoint
346 | 
347 | ## [0.6.1] 2020-01-08
348 | ### Added
349 | - Prometheus docs
350 | 
351 | ### Fixed
352 | - SQL error in endpoint /checks?seconds=x
353 | - Database column mixup for state and language
354 | 
355 | ### Changed
356 | - Run as non root user in docker by default
357 | 
358 | ## [0.6.0] - 2020-01-06
359 | ### Added
360 | - Changelog
361 | - Check fields: "metainfo_overrides_database", "public", "name", "description", "tags", "countrycode", "homepage", "favicon", "loadbalancer"
362 | 
363 | ### Removed
364 | - "id" from all of the API, because database id should not be exposed
365 | - Endpoint /stations/broken 
366 | 
367 | ### Changed
368 | - Faster station import from pull source
369 | - Restructured more code to the new style of connecting to the database, this should enable other types of databases (e.g.: postgresql) on the long term, this means also more error checking and use of transactions.
370 | 
371 | ### Fixed
372 | - Output lastchangetime and countrycode in /json/stations/changed
373 | - Correctly collect and summarize different sources of station checks
374 | - "numberofentries" added to pls output
375 | - Less usages of "unwrap()"
376 | 
377 | ## [0.5.1] - 2019-12-11
378 | ### Added
379 | - "url_resolved" field to station lists
380 | - "random" order to station lists
381 | 
382 | ### Removed
383 | - "negativevotes" field from station lists
384 | 
385 | ### Fixed
386 | - Fixed json result of vote endpoint
387 | - Clean up tag and language fields on import
388 | - Documentation cleanup and extensions
389 | 
390 | ## [0.5.0] - 2019-12-08
391 | ### Added
392 | - First documented release :)
393 | 


--------------------------------------------------------------------------------