├── .gitattributes ├── docs ├── .gitignore ├── FDDB-Exporter-Logo.jpg ├── resources │ ├── grafana-dashboard.png │ ├── example-response-rolling-averages.json │ ├── example-response-products.json │ ├── example-response-stats.json │ ├── example-response.json │ └── example-document.bson ├── visualization │ ├── grafana-dashboard.md │ └── flutter-app.md ├── package.json ├── index.md ├── introduction │ ├── index.md │ └── getting-started.md ├── details │ ├── exports-and-data.md │ ├── docker.md │ ├── helm.md │ ├── persistence.md │ └── configuration.md └── .vitepress │ └── config.js ├── renovate.json ├── .mvn └── wrapper │ └── maven-wrapper.properties ├── docker ├── Dockerfile ├── mongo-init.js └── docker-compose.yml ├── .gitignore ├── src ├── test │ ├── resources │ │ ├── application-test.yaml │ │ ├── __files │ │ │ └── update │ │ │ │ ├── existing-incomplete-entry.json │ │ │ │ └── expected-after-update.json │ │ └── domain │ │ │ ├── fddbdata-2024-08-29.json │ │ │ └── fddbdata-2024-08-27.json │ └── java │ │ └── dev │ │ └── itobey │ │ └── adapter │ │ └── api │ │ └── fddb │ │ └── exporter │ │ ├── config │ │ └── TestConfig.java │ │ ├── service │ │ ├── TimeframeCalculatorTest.java │ │ ├── TelegramServiceTest.java │ │ ├── EnvironmentDetectorTest.java │ │ ├── TelemetryServiceTest.java │ │ ├── InfluxDBServiceTest.java │ │ ├── ExportServiceTest.java │ │ ├── StatsServiceTest.java │ │ └── FddbParserServiceTest.java │ │ └── rest │ │ ├── v2 │ │ ├── CorrelationResourceV2Test.java │ │ ├── FddbDataMigrationResourceV2Test.java │ │ ├── FddbDataExportResourceV2Test.java │ │ ├── FddbDataExportResourceV2WebMvcTest.java │ │ ├── FddbDataStatsResourceV2Test.java │ │ ├── FddbDataExportResourceV2IT.java │ │ └── FddbDataQueryResourceV2Test.java │ │ └── v1 │ │ ├── FddbDataResourceWebMvcTest.java │ │ ├── FddbDataResourceIT.java │ │ └── FddbDataResourceTest.java └── main │ ├── java │ └── dev │ │ └── itobey │ │ └── adapter │ │ └── api │ │ └── fddb │ │ └── exporter │ │ ├── domain │ │ ├── ExecutionMode.java │ │ ├── projection │ │ │ └── ProductWithDate.java │ │ ├── Product.java │ │ └── FddbData.java │ │ ├── exception │ │ ├── ParseException.java │ │ ├── AuthenticationException.java │ │ └── GlobalExceptionHandler.java │ │ ├── annotation │ │ ├── RequiresInfluxDb.java │ │ └── RequiresMongoDb.java │ │ ├── dto │ │ ├── correlation │ │ │ ├── CorrelationInputDto.java │ │ │ ├── Correlations.java │ │ │ ├── CorrelationOutputDto.java │ │ │ └── CorrelationDetail.java │ │ ├── TimeframeDTO.java │ │ ├── telemetry │ │ │ └── TelemetryDto.java │ │ ├── ProductWithDateDTO.java │ │ ├── ExportResultDTO.java │ │ ├── RollingAveragesDTO.java │ │ ├── DateRangeDTO.java │ │ ├── ProductDTO.java │ │ ├── FddbDataDTO.java │ │ └── StatsDTO.java │ │ ├── config │ │ ├── TelegramConfig.java │ │ ├── FddbFeignConfig.java │ │ ├── TelemetryFeignConfig.java │ │ ├── ConditionalMongoConfig.java │ │ ├── InfluxDBConfig.java │ │ ├── PersistenceCheckConfig.java │ │ ├── FddbExporterProperties.java │ │ ├── FddbRequestInterceptor.java │ │ └── OpenApiConfig.java │ │ ├── repository │ │ └── FddbDataRepository.java │ │ ├── adapter │ │ ├── TelemetryApi.java │ │ ├── FddbAdapter.java │ │ └── FddbApi.java │ │ ├── mapper │ │ └── FddbDataMapper.java │ │ ├── aspect │ │ ├── MongoDbEnabledAspect.java │ │ └── InfluxDbEnabledAspect.java │ │ ├── ExporterApplication.java │ │ ├── service │ │ ├── DataMigrationService.java │ │ ├── ExportService.java │ │ ├── TelegramService.java │ │ ├── TimeframeCalculator.java │ │ ├── telemetry │ │ │ ├── EnvironmentDetector.java │ │ │ └── TelemetryService.java │ │ ├── Scheduler.java │ │ ├── persistence │ │ │ ├── InfluxDBService.java │ │ │ ├── PersistenceService.java │ │ │ └── MongoDBService.java │ │ ├── FddbParserService.java │ │ └── FddbDataService.java │ │ ├── actuator │ │ └── FddbHealthIndicator.java │ │ └── rest │ │ ├── v1 │ │ └── CorrelationResourceV1.java │ │ └── v2 │ │ ├── CorrelationResourceV2.java │ │ ├── FddbDataMigrationResourceV2.java │ │ ├── FddbDataStatsResourceV2.java │ │ └── FddbDataExportResourceV2.java │ └── resources │ └── application.yml ├── .github └── workflows │ ├── ci.yml │ ├── release.yml │ └── pages.yml ├── CHANGELOG.md ├── LICENSE.txt └── README.md /.gitattributes: -------------------------------------------------------------------------------- 1 | *.html linguist-detectable=false 2 | -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- 1 | node_modules 2 | .vitepress/dist 3 | .vitepress/cache -------------------------------------------------------------------------------- /renovate.json: -------------------------------------------------------------------------------- 1 | { 2 | "extends": [ 3 | "config:recommended" 4 | ] 5 | } 6 | -------------------------------------------------------------------------------- /docs/FDDB-Exporter-Logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itobey/fddb-exporter/HEAD/docs/FDDB-Exporter-Logo.jpg -------------------------------------------------------------------------------- /docs/resources/grafana-dashboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itobey/fddb-exporter/HEAD/docs/resources/grafana-dashboard.png -------------------------------------------------------------------------------- /.mvn/wrapper/maven-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionType=only-script 2 | distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.12/apache-maven-3.9.12-bin.zip 3 | -------------------------------------------------------------------------------- /docker/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM eclipse-temurin:21 2 | RUN mkdir /opt/app 3 | COPY target/fddbexporter-*.jar /opt/app/app.jar 4 | 5 | EXPOSE 8080 6 | 7 | ENV TZ=Europe/Berlin 8 | CMD ["java", "-jar", "/opt/app/app.jar"] -------------------------------------------------------------------------------- /docker/mongo-init.js: -------------------------------------------------------------------------------- 1 | db = db.getSiblingDB('fddb'); 2 | db.createUser({ 3 | user: "mongodb_fddb_user", 4 | pwd: "mongodb_fddb_password", 5 | roles: [ 6 | { role: "readWrite", db: "fddb" } 7 | ] 8 | }); 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | Thumbs.db 2 | .DS_Store 3 | .gradle 4 | build/ 5 | target/ 6 | out/ 7 | .idea 8 | *.iml 9 | *.ipr 10 | *.iws 11 | .project 12 | .settings 13 | .classpath 14 | .factorypath 15 | src/main/resources/application-nuc.yml -------------------------------------------------------------------------------- /src/test/resources/application-test.yaml: -------------------------------------------------------------------------------- 1 | fddb-exporter: 2 | persistence: 3 | mongodb: 4 | enabled: true 5 | influxdb: 6 | enabled: true 7 | influxdb: 8 | url: http://localhost:8086 # port is set in integration test 9 | token: token 10 | org: test-org 11 | bucket: test-bucket -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/domain/ExecutionMode.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.domain; 2 | 3 | /** 4 | * ExecutionMode enum represents the different modes in which the application can run for telemetry purposes. 5 | */ 6 | public enum ExecutionMode { 7 | KUBERNETES, 8 | CONTAINER, 9 | JAR 10 | } 11 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/domain/projection/ProductWithDate.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.domain.projection; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.Product; 4 | import lombok.Data; 5 | 6 | import java.time.LocalDate; 7 | 8 | @Data 9 | public class ProductWithDate { 10 | 11 | private LocalDate date; 12 | private Product product; 13 | 14 | } 15 | -------------------------------------------------------------------------------- /docs/resources/example-response-rolling-averages.json: -------------------------------------------------------------------------------- 1 | { 2 | "fromDate": "2024-01-01", 3 | "toDate": "2024-01-07", 4 | "averages": { 5 | "avgTotalCalories": 3054.4285714285716, 6 | "avgTotalFat": 123.92857142857143, 7 | "avgTotalCarbs": 303.75714285714287, 8 | "avgTotalSugar": 85.65714285714286, 9 | "avgTotalProtein": 136.18571428571428, 10 | "avgTotalFibre": 25.82857142857143 11 | } 12 | } 13 | 14 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/exception/ParseException.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.exception; 2 | 3 | public class ParseException extends RuntimeException { 4 | 5 | // Parameterless Constructor 6 | public ParseException() { 7 | } 8 | 9 | // Constructor that accepts a message 10 | public ParseException(String message) { 11 | super(message); 12 | } 13 | 14 | } 15 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/annotation/RequiresInfluxDb.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.annotation; 2 | 3 | import java.lang.annotation.ElementType; 4 | import java.lang.annotation.Retention; 5 | import java.lang.annotation.RetentionPolicy; 6 | import java.lang.annotation.Target; 7 | 8 | @Target(ElementType.METHOD) 9 | @Retention(RetentionPolicy.RUNTIME) 10 | public @interface RequiresInfluxDb { 11 | } 12 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/annotation/RequiresMongoDb.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.annotation; 2 | 3 | import java.lang.annotation.ElementType; 4 | import java.lang.annotation.Retention; 5 | import java.lang.annotation.RetentionPolicy; 6 | import java.lang.annotation.Target; 7 | 8 | @Target(ElementType.METHOD) 9 | @Retention(RetentionPolicy.RUNTIME) 10 | public @interface RequiresMongoDb { 11 | } 12 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/correlation/CorrelationInputDto.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto.correlation; 2 | 3 | import lombok.Data; 4 | 5 | import java.util.List; 6 | 7 | @Data 8 | public class CorrelationInputDto { 9 | 10 | private List inclusionKeywords; 11 | private List exclusionKeywords; 12 | private List occurrenceDates; 13 | private String startDate; 14 | 15 | } 16 | -------------------------------------------------------------------------------- /docs/visualization/grafana-dashboard.md: -------------------------------------------------------------------------------- 1 | # Grafana Dashboard 2 | 3 | ## Screenshots 4 | 5 | After gathering all the data, it's easy to display graphs based on it in Grafana. As Grafana cannot use MongoDB as a 6 | data source, it's necessary to use InfluxDB for this. You may use 7 | my [Grafana-Dashboard](https://github.com/itobey/fddb-exporter/blob/master/docs/resources/grafana-dashboard.json) 8 | or build your own dashboard. 9 | 10 | ![image](../resources/grafana-dashboard.png) -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/correlation/Correlations.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto.correlation; 2 | 3 | import lombok.Data; 4 | 5 | @Data 6 | public class Correlations { 7 | 8 | private CorrelationDetail across3Days; 9 | private CorrelationDetail across2Days; 10 | private CorrelationDetail sameDay; 11 | private CorrelationDetail oneDayBefore; 12 | private CorrelationDetail twoDaysBefore; 13 | 14 | } 15 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/exception/AuthenticationException.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.exception; 2 | 3 | public class AuthenticationException extends RuntimeException { 4 | 5 | // Parameterless Constructor 6 | public AuthenticationException() { 7 | } 8 | 9 | // Constructor that accepts a message 10 | public AuthenticationException(String message) { 11 | super(message); 12 | } 13 | 14 | } 15 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: Java CI with Maven 2 | 3 | on: 4 | push: 5 | branches: [ master ] 6 | pull_request: 7 | branches: [ master ] 8 | workflow_dispatch: 9 | inputs: 10 | branch: 11 | description: 'Branch to run the workflow on' 12 | required: true 13 | default: 'master' 14 | 15 | jobs: 16 | run-ci-workflow: 17 | uses: itobey/workflow-templates/.github/workflows/ci.yml@master 18 | with: 19 | image_name: fddb-exporter 20 | secrets: inherit 21 | -------------------------------------------------------------------------------- /docs/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "fddb-exporter", 3 | "description": "Exports data from fddb.info to a database and supports endpoints to query the data", 4 | "type": "module", 5 | "scripts": { 6 | "docs:dev": "vitepress dev .", 7 | "docs:build": "vitepress build .", 8 | "docs:serve": "vitepress serve ." 9 | }, 10 | "repository": { 11 | "type": "git", 12 | "url": "git+ssh://git@github.com:itobey/fddb-exporter.git" 13 | }, 14 | "devDependencies": { 15 | "vitepress": "^1.4.1" 16 | } 17 | } 18 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/correlation/CorrelationOutputDto.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto.correlation; 2 | 3 | import lombok.Data; 4 | 5 | import java.time.LocalDate; 6 | import java.util.List; 7 | 8 | @Data 9 | public class CorrelationOutputDto { 10 | 11 | private Correlations correlations; 12 | private List matchedProducts; 13 | private List matchedDates; 14 | private int amountMatchedProducts; 15 | private int amountMatchedDates; 16 | 17 | } 18 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/TimeframeDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.FddbApi; 4 | import lombok.Builder; 5 | import lombok.Data; 6 | import lombok.RequiredArgsConstructor; 7 | 8 | /** 9 | * Contains the timeframe used to connect to the @{@link FddbApi}. 10 | */ 11 | @RequiredArgsConstructor 12 | @Builder 13 | @Data 14 | public class TimeframeDTO { 15 | 16 | private final long from; 17 | private final long to; 18 | 19 | } 20 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/domain/Product.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.domain; 2 | 3 | import lombok.AllArgsConstructor; 4 | import lombok.Data; 5 | import lombok.NoArgsConstructor; 6 | 7 | @Data 8 | @NoArgsConstructor 9 | @AllArgsConstructor 10 | public class Product { 11 | 12 | private String name; 13 | private String amount; 14 | private double calories; 15 | private double fat; 16 | private double carbs; 17 | private double protein; 18 | private String link; 19 | 20 | } 21 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/telemetry/TelemetryDto.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto.telemetry; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.ExecutionMode; 4 | import lombok.Data; 5 | 6 | @Data 7 | public class TelemetryDto { 8 | 9 | private String mailHash; 10 | private long documentCount; 11 | private long pointCount; 12 | private boolean mongodbEnabled; 13 | private boolean influxdbEnabled; 14 | private ExecutionMode executionMode; 15 | private String appVersion; 16 | 17 | } 18 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/TelegramConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import com.pengrad.telegrambot.TelegramBot; 4 | import lombok.RequiredArgsConstructor; 5 | import org.springframework.context.annotation.Bean; 6 | import org.springframework.context.annotation.Configuration; 7 | 8 | @Configuration 9 | @RequiredArgsConstructor 10 | public class TelegramConfig { 11 | 12 | private final FddbExporterProperties properties; 13 | 14 | @Bean 15 | public TelegramBot telegramBot() { 16 | return new TelegramBot(properties.getNotification().getTelegram().getToken()); 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /docs/resources/example-response-products.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "date": "2023-01-06", 4 | "product": { 5 | "name": "Pizza Brot", 6 | "amount": "150 g", 7 | "calories": 391.0, 8 | "fat": 5.1, 9 | "carbs": 71.1, 10 | "protein": 15.0, 11 | "link": "/db/en/food/marziale_pizza_brot/index.html" 12 | } 13 | }, 14 | { 15 | "date": "2023-02-02", 16 | "product": { 17 | "name": "Pizza Parma", 18 | "amount": "350 g", 19 | "calories": 735.0, 20 | "fat": 20.0, 21 | "carbs": 108.9, 22 | "protein": 25.6, 23 | "link": "/db/en/food/diverse_pizza_parma/index.html" 24 | } 25 | } 26 | ] -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/FddbFeignConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import lombok.RequiredArgsConstructor; 4 | import org.springframework.context.annotation.Bean; 5 | import org.springframework.context.annotation.Configuration; 6 | import org.springframework.context.annotation.Profile; 7 | 8 | @Configuration 9 | @Profile("!test") 10 | @RequiredArgsConstructor 11 | public class FddbFeignConfig { 12 | 13 | private final FddbExporterProperties properties; 14 | 15 | @Bean 16 | public FddbRequestInterceptor fddbRequestInterceptor() { 17 | return new FddbRequestInterceptor(properties); 18 | } 19 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/repository/FddbDataRepository.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.repository; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; 5 | import org.springframework.data.mongodb.repository.MongoRepository; 6 | 7 | import java.time.LocalDate; 8 | import java.util.Optional; 9 | 10 | @ConditionalOnProperty(name = "fddb-exporter.persistence.mongodb.enabled", havingValue = "true") 11 | public interface FddbDataRepository extends MongoRepository { 12 | 13 | Optional findFirstByDate(LocalDate date); 14 | 15 | } 16 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/ProductWithDateDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Data; 6 | import lombok.NoArgsConstructor; 7 | 8 | import java.time.LocalDate; 9 | 10 | @Data 11 | @NoArgsConstructor 12 | @AllArgsConstructor 13 | @Schema(description = "A product with its consumption date") 14 | public class ProductWithDateDTO { 15 | 16 | @Schema(description = "Date when the product was consumed", example = "2024-12-22") 17 | private LocalDate date; 18 | 19 | @Schema(description = "Product information") 20 | private ProductDTO product; 21 | 22 | } 23 | -------------------------------------------------------------------------------- /.github/workflows/release.yml: -------------------------------------------------------------------------------- 1 | name: Release Workflow 2 | 3 | on: 4 | workflow_dispatch: 5 | inputs: 6 | release_version: 7 | description: 'Version number for the release' 8 | required: true 9 | default: '1.0.0' 10 | snapshot_version: 11 | description: 'Version number for the next snapshot' 12 | required: true 13 | default: '1.0.1-SNAPSHOT' 14 | 15 | jobs: 16 | run-release-workflow: 17 | uses: itobey/workflow-templates/.github/workflows/release.yml@master 18 | with: 19 | image_name: fddb-exporter 20 | snapshot_version: ${{ github.event.inputs.snapshot_version }} 21 | release_version: ${{ github.event.inputs.release_version }} 22 | secrets: inherit -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/config/TestConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.service.telemetry.TelemetryService; 4 | import org.mockito.Mockito; 5 | import org.springframework.boot.test.context.TestConfiguration; 6 | import org.springframework.context.annotation.Bean; 7 | import org.springframework.context.annotation.Primary; 8 | 9 | /** 10 | * This class is used to mock the service beans for testing purposes. 11 | */ 12 | @TestConfiguration 13 | public class TestConfig { 14 | 15 | @Bean 16 | @Primary 17 | public TelemetryService telemetryService() { 18 | return Mockito.mock(TelemetryService.class); 19 | } 20 | 21 | } 22 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/adapter/TelemetryApi.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.adapter; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.TelemetryFeignConfig; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.telemetry.TelemetryDto; 5 | import org.springframework.cloud.openfeign.FeignClient; 6 | import org.springframework.web.bind.annotation.PostMapping; 7 | import org.springframework.web.bind.annotation.RequestBody; 8 | 9 | @FeignClient(name = "telemetryApi", url = "${fddb-exporter.telemetry.url}", configuration = TelemetryFeignConfig.class) 10 | public interface TelemetryApi { 11 | 12 | @PostMapping("/api/v1/fddb-exporter") 13 | void sendTelemetryData(@RequestBody TelemetryDto telemetryDto); 14 | } 15 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/TelemetryFeignConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import feign.auth.BasicAuthRequestInterceptor; 4 | import lombok.RequiredArgsConstructor; 5 | import org.springframework.context.annotation.Bean; 6 | import org.springframework.context.annotation.Configuration; 7 | 8 | @Configuration 9 | @RequiredArgsConstructor 10 | public class TelemetryFeignConfig { 11 | 12 | private final FddbExporterProperties properties; 13 | 14 | @Bean 15 | public BasicAuthRequestInterceptor basicAuthRequestInterceptor() { 16 | return new BasicAuthRequestInterceptor(properties.getTelemetry().getUsername(), 17 | properties.getTelemetry().getToken()); 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/adapter/FddbAdapter.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.adapter; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 4 | import lombok.RequiredArgsConstructor; 5 | import lombok.extern.slf4j.Slf4j; 6 | import org.springframework.stereotype.Service; 7 | 8 | /** 9 | * Adapter to the FDDB API. 10 | */ 11 | @Service 12 | @Slf4j 13 | @RequiredArgsConstructor 14 | public class FddbAdapter { 15 | 16 | private final FddbApi fddbApi; 17 | 18 | public String retrieveDataToTimeframe(TimeframeDTO timeframeDTO) { 19 | log.debug("retrieving fddb data for timeframe {}", timeframeDTO); 20 | return fddbApi.getDiary(timeframeDTO.getFrom(), timeframeDTO.getTo()); 21 | } 22 | 23 | } 24 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/ExportResultDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Data; 6 | import lombok.NoArgsConstructor; 7 | 8 | import java.util.List; 9 | 10 | @Data 11 | @NoArgsConstructor 12 | @AllArgsConstructor 13 | @Schema(description = "Result of an export operation") 14 | public class ExportResultDTO { 15 | @Schema(description = "Dates that were successfully exported", example = "[\"2024-12-20\", \"2024-12-21\"]") 16 | private List successfulDays; 17 | 18 | @Schema(description = "Dates that failed to export", example = "[\"2024-12-22\"]") 19 | private List unsuccessfulDays; 20 | } -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: home 3 | 4 | hero: 5 | name: FDDB Exporter 6 | text: Export data from fddb.info to a database and query your data 7 | actions: 8 | - theme: brand 9 | text: Documentation 10 | link: /introduction/index.md 11 | - theme: alt 12 | text: View on GitHub 13 | link: https://github.com/itobey/fddb-exporter 14 | 15 | features: 16 | - title: MongoDB persistence 17 | details: Persist all your nutritional diary information to MongoDB including every product you have ever entered. 18 | - title: InfluxDB persistence 19 | details: Persist daily totals to InfluxDB for easy querying with Grafana. 20 | - title: REST API support 21 | details: Access your data through a REST API and search for specific products. 22 | --- 23 | -------------------------------------------------------------------------------- /docs/introduction/index.md: -------------------------------------------------------------------------------- 1 | # What is FDDB Exporter? 2 | 3 | FDDB Exporter is a tool designed to extract nutritional data from [FDDB.info](https://fddb.info/) and store it in a 4 | MongoDB / InfluxDB database. 5 | 6 | You may want to do this for the following reasons: 7 | 8 | - FDDB only stores entries for up to 2 years for premium members, and even less for free users. Storing your data in a 9 | database allows you to keep your data for as long as you want. 10 | - You can query your data to see on which days you have entered specific products. This is especially useful if you 11 | suspect food allergies 12 | or sensitivities to quickly identify problematic products. 13 | - You can use the data to create summaries or graphs and charts to visualize your nutritional intake over time. 14 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/correlation/CorrelationDetail.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto.correlation; 2 | 3 | import lombok.AccessLevel; 4 | import lombok.Data; 5 | import lombok.Setter; 6 | 7 | import java.text.DecimalFormat; 8 | import java.text.DecimalFormatSymbols; 9 | import java.util.List; 10 | import java.util.Locale; 11 | 12 | @Data 13 | public class CorrelationDetail { 14 | 15 | private static final DecimalFormat df = new DecimalFormat("#.##", new DecimalFormatSymbols(Locale.US)); 16 | 17 | 18 | @Setter(AccessLevel.NONE) 19 | private double percentage; 20 | private List matchedDates; 21 | private int matchedDays; 22 | 23 | public void setPercentage(double percentage) { 24 | this.percentage = Double.parseDouble(df.format(percentage)); 25 | } 26 | 27 | } 28 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/ConditionalMongoConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; 4 | import org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration; 5 | import org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration; 6 | import org.springframework.context.annotation.Configuration; 7 | import org.springframework.context.annotation.Import; 8 | 9 | @Configuration 10 | @ConditionalOnProperty(name = "fddb-exporter.persistence.mongodb.enabled", havingValue = "true") 11 | @Import({MongoAutoConfiguration.class, MongoDataAutoConfiguration.class}) 12 | public class ConditionalMongoConfig { 13 | // This class is intentionally empty. Its purpose is to conditionally import MongoDB configurations. 14 | } 15 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/InfluxDBConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import com.influxdb.client.InfluxDBClient; 4 | import com.influxdb.client.InfluxDBClientFactory; 5 | import lombok.RequiredArgsConstructor; 6 | import org.springframework.context.annotation.Bean; 7 | import org.springframework.context.annotation.Configuration; 8 | 9 | @Configuration 10 | @RequiredArgsConstructor 11 | public class InfluxDBConfig { 12 | 13 | private final FddbExporterProperties properties; 14 | 15 | @Bean 16 | public InfluxDBClient influxDBClient() { 17 | return InfluxDBClientFactory.create(properties.getInfluxdb().getUrl(), 18 | properties.getInfluxdb().getToken().toCharArray(), 19 | properties.getInfluxdb().getOrg(), 20 | properties.getInfluxdb().getBucket()); 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/RollingAveragesDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Builder; 6 | import lombok.Data; 7 | import lombok.NoArgsConstructor; 8 | 9 | /** 10 | * Contains rolling averages for a given date range. 11 | * Used for statistics reporting. 12 | */ 13 | @Data 14 | @Builder 15 | @NoArgsConstructor 16 | @AllArgsConstructor 17 | @Schema(description = "Rolling averages for a date range") 18 | public class RollingAveragesDTO { 19 | @Schema(description = "Start date of the range", example = "2024-01-01") 20 | String fromDate; 21 | 22 | @Schema(description = "End date of the range", example = "2024-12-31") 23 | String toDate; 24 | 25 | @Schema(description = "Average nutritional values for the date range") 26 | StatsDTO.Averages averages; 27 | } 28 | -------------------------------------------------------------------------------- /docs/resources/example-response-stats.json: -------------------------------------------------------------------------------- 1 | { 2 | "amountEntries": 606, 3 | "firstEntryDate": "2023-01-01", 4 | "mostRecentMissingDay": "2024-12-20", 5 | "entryPercentage": 95.1, 6 | "uniqueProducts": 150, 7 | "averageTotals": { 8 | "avgTotalCalories": 2505.7, 9 | "avgTotalFat": 125.7, 10 | "avgTotalCarbs": 204.4, 11 | "avgTotalSugar": 63.4, 12 | "avgTotalProtein": 118.0, 13 | "avgTotalFibre": 18.4 14 | }, 15 | "highestCaloriesDay": { 16 | "date": "2024-07-31", 17 | "total": 5317.0 18 | }, 19 | "highestFatDay": { 20 | "date": "2023-09-23", 21 | "total": 260.3 22 | }, 23 | "highestCarbsDay": { 24 | "date": "2024-07-31", 25 | "total": 501.7 26 | }, 27 | "highestProteinDay": { 28 | "date": "2024-08-03", 29 | "total": 234.1 30 | }, 31 | "highestFibreDay": { 32 | "date": "2023-05-09", 33 | "total": 53.8 34 | }, 35 | "highestSugarDay": { 36 | "date": "2023-10-21", 37 | "total": 220.4 38 | } 39 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/DateRangeDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import jakarta.validation.constraints.NotNull; 4 | import jakarta.validation.constraints.Pattern; 5 | import lombok.AllArgsConstructor; 6 | import lombok.Builder; 7 | import lombok.Data; 8 | import lombok.NoArgsConstructor; 9 | 10 | /** 11 | * Contains a date range with from and to dates. 12 | * Used for batch exports and date range queries. 13 | */ 14 | @AllArgsConstructor 15 | @NoArgsConstructor 16 | @Builder 17 | @Data 18 | public class DateRangeDTO { 19 | 20 | @NotNull(message = "From date cannot be null") 21 | @Pattern(regexp = "\\d{4}-\\d{2}-\\d{2}", message = "From date must be in the format YYYY-MM-DD") 22 | private String fromDate; 23 | 24 | @NotNull(message = "To date cannot be null") 25 | @Pattern(regexp = "\\d{4}-\\d{2}-\\d{2}", message = "To date must be in the format YYYY-MM-DD") 26 | private String toDate; 27 | 28 | } 29 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/PersistenceCheckConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import jakarta.annotation.PostConstruct; 4 | import lombok.extern.slf4j.Slf4j; 5 | import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression; 6 | import org.springframework.context.annotation.Bean; 7 | import org.springframework.context.annotation.Configuration; 8 | 9 | @Configuration 10 | @Slf4j 11 | @ConditionalOnExpression("${fddb-exporter.persistence.mongodb.enabled} == false && ${fddb-exporter.persistence.influxdb.enabled} == false") 12 | public class PersistenceCheckConfig { 13 | 14 | @PostConstruct 15 | public void exitApplication() { 16 | log.error("ERROR: Both MongoDB and InfluxDB are disabled. At least one persistence layer must be enabled."); 17 | System.exit(1); 18 | } 19 | 20 | @Bean 21 | public String dummyBean() { 22 | return "Dummy bean to ensure configuration class is loaded"; 23 | } 24 | } 25 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/adapter/FddbApi.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.adapter; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbFeignConfig; 4 | import org.springframework.cloud.openfeign.FeignClient; 5 | import org.springframework.web.bind.annotation.GetMapping; 6 | import org.springframework.web.bind.annotation.RequestParam; 7 | 8 | /** 9 | * API for FDDB.info 10 | */ 11 | @FeignClient(name = "fddbApi", url = "${fddb-exporter.fddb.url}", configuration = FddbFeignConfig.class) 12 | public interface FddbApi { 13 | 14 | /** 15 | * API call to get the diary containing all necessary data. 16 | * 17 | * @param from beginning of timeframe for search of data 18 | * @param to end of timeframe for search of data 19 | * @return the HTML response containing the data 20 | */ 21 | @GetMapping("/db/i18n/myday20/?lang=en&q={to}&p={from}") 22 | String getDiary( 23 | @RequestParam("from") long from, 24 | @RequestParam("to") long to 25 | ); 26 | } 27 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/mapper/FddbDataMapper.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.mapper; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.projection.ProductWithDate; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.FddbDataDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.ProductWithDateDTO; 7 | import org.mapstruct.Mapper; 8 | import org.mapstruct.Mapping; 9 | import org.mapstruct.MappingTarget; 10 | 11 | import java.util.List; 12 | 13 | @Mapper(componentModel = "spring") 14 | public interface FddbDataMapper { 15 | 16 | @Mapping(target = "id", ignore = true) 17 | void updateFddbData(@MappingTarget FddbData target, FddbData source); 18 | 19 | FddbDataDTO toFddbDataDTO(FddbData fddbData); 20 | 21 | List toFddbDataDTO(List fddbData); 22 | 23 | ProductWithDateDTO toProductWithDateDto(ProductWithDate product); 24 | 25 | List toProductWithDateDto(List product); 26 | 27 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/aspect/MongoDbEnabledAspect.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.aspect; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 4 | import lombok.RequiredArgsConstructor; 5 | import org.aspectj.lang.ProceedingJoinPoint; 6 | import org.aspectj.lang.annotation.Around; 7 | import org.aspectj.lang.annotation.Aspect; 8 | import org.springframework.http.ResponseEntity; 9 | import org.springframework.stereotype.Component; 10 | 11 | @Aspect 12 | @Component 13 | @RequiredArgsConstructor 14 | public class MongoDbEnabledAspect { 15 | 16 | private final FddbExporterProperties properties; 17 | 18 | @Around("@annotation(dev.itobey.adapter.api.fddb.exporter.annotation.RequiresMongoDb)") 19 | public Object checkMongoDbEnabled(ProceedingJoinPoint joinPoint) throws Throwable { 20 | if (!properties.getPersistence().getMongodb().isEnabled()) { 21 | return ResponseEntity.badRequest().body("This operation requires MongoDB to be enabled"); 22 | } 23 | return joinPoint.proceed(); 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/aspect/InfluxDbEnabledAspect.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.aspect; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 4 | import lombok.RequiredArgsConstructor; 5 | import org.aspectj.lang.ProceedingJoinPoint; 6 | import org.aspectj.lang.annotation.Around; 7 | import org.aspectj.lang.annotation.Aspect; 8 | import org.springframework.http.ResponseEntity; 9 | import org.springframework.stereotype.Component; 10 | 11 | @Aspect 12 | @Component 13 | @RequiredArgsConstructor 14 | public class InfluxDbEnabledAspect { 15 | 16 | private final FddbExporterProperties properties; 17 | 18 | @Around("@annotation(dev.itobey.adapter.api.fddb.exporter.annotation.RequiresInfluxDb)") 19 | public Object checkInfluxDbEnabled(ProceedingJoinPoint joinPoint) throws Throwable { 20 | if (!properties.getPersistence().getInfluxdb().isEnabled()) { 21 | return ResponseEntity.badRequest().body("This operation requires InfluxDB to be enabled"); 22 | } 23 | return joinPoint.proceed(); 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/domain/FddbData.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.domain; 2 | 3 | import lombok.Data; 4 | import lombok.EqualsAndHashCode; 5 | import org.springframework.data.annotation.Id; 6 | import org.springframework.data.mongodb.core.mapping.Document; 7 | 8 | import java.time.LocalDate; 9 | import java.util.List; 10 | 11 | @Document(collection = "fddb") 12 | @Data 13 | public class FddbData { 14 | 15 | @Id 16 | @EqualsAndHashCode.Exclude 17 | private String id; 18 | private LocalDate date; 19 | private List products; 20 | private double totalCalories; 21 | private double totalFat; 22 | private double totalCarbs; 23 | private double totalSugar; 24 | private double totalProtein; 25 | private double totalFibre; 26 | 27 | public String toDailyTotalsString() { 28 | return "FddbData{" + 29 | "date=" + date + 30 | ", totalCalories=" + totalCalories + 31 | ", totalFat=" + totalFat + 32 | ", totalCarbs=" + totalCarbs + 33 | ", totalSugar=" + totalSugar + 34 | ", totalProtein=" + totalProtein + 35 | ", totalFibre=" + totalFibre + 36 | '}'; 37 | } 38 | 39 | } 40 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/TimeframeCalculatorTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 4 | import org.junit.jupiter.api.Test; 5 | 6 | import java.time.LocalDate; 7 | 8 | import static org.junit.jupiter.api.Assertions.assertTrue; 9 | 10 | /** 11 | * Test for @{@link TimeframeCalculator} 12 | * These are not highly useful, but well, better than nothing. 13 | */ 14 | public class TimeframeCalculatorTest { 15 | 16 | @Test 17 | public void calculateTimeframeForYesterday_shouldCreateTimeframeObject() { 18 | // given 19 | TimeframeCalculator timeframeCalculator = new TimeframeCalculator(); 20 | // when 21 | TimeframeDTO timeframeDTO = timeframeCalculator.calculateTimeframeForYesterday(); 22 | // then 23 | assertTrue(timeframeDTO.getFrom() < timeframeDTO.getTo()); 24 | } 25 | 26 | @Test 27 | public void calculateTimeframeFor_shouldCreateTimeframeObject() { 28 | // given 29 | TimeframeCalculator timeframeCalculator = new TimeframeCalculator(); 30 | // when 31 | TimeframeDTO timeframeDTO = timeframeCalculator.calculateTimeframeFor(LocalDate.now()); 32 | // then 33 | assertTrue(timeframeDTO.getFrom() < timeframeDTO.getTo()); 34 | } 35 | 36 | } 37 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/ProductDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Data; 6 | import lombok.NoArgsConstructor; 7 | 8 | @Data 9 | @NoArgsConstructor 10 | @AllArgsConstructor 11 | @Schema(description = "A consumed product with nutritional information") 12 | public class ProductDTO { 13 | 14 | @Schema(description = "Product name", example = "Banana") 15 | private String name; 16 | 17 | @Schema(description = "Amount consumed", example = "100 g") 18 | private String amount; 19 | 20 | @Schema(description = "Calories", example = "89.0") 21 | private double calories; 22 | 23 | @Schema(description = "Fat in grams", example = "0.3") 24 | private double fat; 25 | 26 | @Schema(description = "Carbs in grams", example = "23.0") 27 | private double carbs; 28 | 29 | @Schema(description = "Protein in grams", example = "1.1") 30 | private double protein; 31 | 32 | @Schema(description = "Link to product page on FDDB", example = "https://fddb.info/db/de/lebensmittel/...") 33 | private String link; 34 | 35 | // default to "0 g" if the amount is null, to have a consistent API. 36 | public String getAmount() { 37 | return amount == null ? "0 g" : amount; 38 | } 39 | 40 | } 41 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/ExporterApplication.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter; 2 | 3 | import com.ulisesbocchio.jasyptspringboot.annotation.EnableEncryptableProperties; 4 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 5 | import org.springframework.boot.SpringApplication; 6 | import org.springframework.boot.autoconfigure.SpringBootApplication; 7 | import org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration; 8 | import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration; 9 | import org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration; 10 | import org.springframework.boot.context.properties.EnableConfigurationProperties; 11 | import org.springframework.cloud.openfeign.EnableFeignClients; 12 | import org.springframework.scheduling.annotation.EnableScheduling; 13 | 14 | @SpringBootApplication(exclude = { 15 | DataSourceAutoConfiguration.class, 16 | MongoAutoConfiguration.class, 17 | MongoDataAutoConfiguration.class 18 | }) 19 | @EnableScheduling 20 | @EnableFeignClients 21 | @EnableConfigurationProperties(FddbExporterProperties.class) 22 | @EnableEncryptableProperties 23 | public class ExporterApplication { 24 | 25 | public static void main(String[] args) { 26 | SpringApplication.run(ExporterApplication.class, args); 27 | } 28 | 29 | } 30 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/DataMigrationService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.InfluxDBService; 5 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.MongoDBService; 6 | import lombok.RequiredArgsConstructor; 7 | import lombok.extern.slf4j.Slf4j; 8 | import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; 9 | import org.springframework.stereotype.Service; 10 | 11 | import java.util.List; 12 | 13 | @Service 14 | @Slf4j 15 | @RequiredArgsConstructor 16 | @ConditionalOnProperty(name = { 17 | "fddb-exporter.persistence.mongodb.enabled", 18 | "fddb-exporter.persistence.influxdb.enabled" 19 | }, havingValue = "true") 20 | public class DataMigrationService { 21 | 22 | private final MongoDBService mongoDbService; 23 | private final InfluxDBService influxDbService; 24 | 25 | public int migrateMongoDbEntriesToInfluxDb() { 26 | List allEntries = mongoDbService.findAllEntries(); 27 | int amountEntries = allEntries.size(); 28 | log.info("migrating {} entries from MongoDB to InfluxDB...", amountEntries); 29 | allEntries.forEach(influxDbService::saveToInfluxDB); 30 | log.info("migration completed"); 31 | return amountEntries; 32 | } 33 | 34 | } 35 | -------------------------------------------------------------------------------- /.github/workflows/pages.yml: -------------------------------------------------------------------------------- 1 | name: Deploy documentation to Pages 2 | 3 | on: 4 | push: 5 | branches: [ master ] 6 | 7 | workflow_dispatch: 8 | 9 | permissions: 10 | contents: read 11 | pages: write 12 | id-token: write 13 | 14 | concurrency: 15 | group: pages 16 | cancel-in-progress: false 17 | 18 | defaults: 19 | run: 20 | working-directory: ./docs 21 | 22 | jobs: 23 | build: 24 | runs-on: ubuntu-latest 25 | steps: 26 | - name: Checkout 27 | uses: actions/checkout@v6 28 | with: 29 | fetch-depth: 0 # Not needed if lastUpdated is not enabled 30 | - name: Setup Node 31 | uses: actions/setup-node@v6 32 | with: 33 | node-version: 24 34 | cache: npm 35 | cache-dependency-path: docs/package-lock.json 36 | - name: Setup Pages 37 | uses: actions/configure-pages@v5 38 | - name: Install dependencies 39 | run: npm ci 40 | - name: Build with VitePress 41 | run: npm run docs:build 42 | - name: Upload artifact 43 | uses: actions/upload-pages-artifact@v4 44 | with: 45 | path: docs/.vitepress/dist 46 | 47 | deploy: 48 | environment: 49 | name: github-pages 50 | url: ${{ steps.deployment.outputs.page_url }} 51 | needs: build 52 | runs-on: ubuntu-latest 53 | name: Deploy 54 | steps: 55 | - name: Deploy to GitHub Pages 56 | id: deployment 57 | uses: actions/deploy-pages@v4 -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/ExportService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.FddbAdapter; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 7 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 8 | import lombok.RequiredArgsConstructor; 9 | import lombok.extern.slf4j.Slf4j; 10 | import org.springframework.stereotype.Service; 11 | 12 | import java.time.LocalDateTime; 13 | import java.time.ZoneOffset; 14 | 15 | @Service 16 | @RequiredArgsConstructor 17 | @Slf4j 18 | public class ExportService { 19 | 20 | private final FddbAdapter fddbAdapter; 21 | private final FddbParserService fddbParserService; 22 | 23 | public FddbData exportData(TimeframeDTO timeframeDTO) throws AuthenticationException, ParseException { 24 | String response = fddbAdapter.retrieveDataToTimeframe(timeframeDTO); 25 | log.trace("HTML response: {}", response); 26 | FddbData fddbData = fddbParserService.parseDiary(response); 27 | LocalDateTime dateOfExport = LocalDateTime.ofEpochSecond(timeframeDTO.getFrom(), 0, ZoneOffset.UTC); 28 | fddbData.setDate(dateOfExport.toLocalDate()); 29 | log.debug("handling dataset: {}", fddbData); 30 | return fddbData; 31 | } 32 | 33 | } 34 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/FddbDataDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Data; 6 | import lombok.NoArgsConstructor; 7 | 8 | import java.time.LocalDate; 9 | import java.util.List; 10 | 11 | @Data 12 | @NoArgsConstructor 13 | @AllArgsConstructor 14 | @Schema(description = "FDDB nutrition data for a specific day") 15 | public class FddbDataDTO { 16 | 17 | @Schema(description = "Unique identifier", example = "507f1f77bcf86cd799439011") 18 | private String id; 19 | 20 | @Schema(description = "Date of the entry", example = "2024-12-22") 21 | private LocalDate date; 22 | 23 | @Schema(description = "List of products consumed on this day") 24 | private List products; 25 | 26 | @Schema(description = "Total calories for the day", example = "2000.5") 27 | private double totalCalories; 28 | 29 | @Schema(description = "Total fat in grams", example = "65.3") 30 | private double totalFat; 31 | 32 | @Schema(description = "Total carbs in grams", example = "250.2") 33 | private double totalCarbs; 34 | 35 | @Schema(description = "Total sugar in grams", example = "50.1") 36 | private double totalSugar; 37 | 38 | @Schema(description = "Total protein in grams", example = "80.4") 39 | private double totalProtein; 40 | 41 | @Schema(description = "Total fibre in grams", example = "25.6") 42 | private double totalFibre; 43 | 44 | } 45 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/CorrelationResourceV2Test.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationInputDto; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationOutputDto; 5 | import dev.itobey.adapter.api.fddb.exporter.service.CorrelationService; 6 | import org.junit.jupiter.api.Tag; 7 | import org.junit.jupiter.api.Test; 8 | import org.junit.jupiter.api.extension.ExtendWith; 9 | import org.mockito.InjectMocks; 10 | import org.mockito.Mock; 11 | import org.mockito.junit.jupiter.MockitoExtension; 12 | 13 | import static org.junit.jupiter.api.Assertions.assertEquals; 14 | import static org.mockito.Mockito.when; 15 | 16 | /** 17 | * Unit test for v2 Correlation API. 18 | */ 19 | @ExtendWith(MockitoExtension.class) 20 | @Tag("v2") 21 | class CorrelationResourceV2Test { 22 | 23 | @Mock 24 | private CorrelationService correlationService; 25 | 26 | @InjectMocks 27 | private CorrelationResourceV2 correlationResourceV2; 28 | 29 | @Test 30 | void testCreateCorrelation() { 31 | CorrelationInputDto inputDto = new CorrelationInputDto(); 32 | CorrelationOutputDto expectedOutput = new CorrelationOutputDto(); 33 | 34 | when(correlationService.createCorrelation(inputDto)).thenReturn(expectedOutput); 35 | 36 | CorrelationOutputDto response = correlationResourceV2.createCorrelation(inputDto); 37 | 38 | assertEquals(expectedOutput, response); 39 | } 40 | } 41 | 42 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/TelegramService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import com.pengrad.telegrambot.TelegramBot; 4 | import com.pengrad.telegrambot.model.request.ParseMode; 5 | import com.pengrad.telegrambot.request.SendMessage; 6 | import com.pengrad.telegrambot.response.SendResponse; 7 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 8 | import lombok.RequiredArgsConstructor; 9 | import lombok.extern.slf4j.Slf4j; 10 | import org.springframework.stereotype.Service; 11 | 12 | /** 13 | * Service to send messages to Telegram. 14 | */ 15 | @Service 16 | @RequiredArgsConstructor 17 | @Slf4j 18 | public class TelegramService { 19 | 20 | private final FddbExporterProperties properties; 21 | private final TelegramBot telegramBot; 22 | 23 | /** 24 | * Sends a message to the configured Telegram chat. 25 | * 26 | * @param message The message to send. 27 | */ 28 | public void sendMessage(String message) { 29 | FddbExporterProperties.Notification.Telegram telegram = properties.getNotification().getTelegram(); 30 | SendMessage request = new SendMessage(telegram.getChatId(), message).parseMode(ParseMode.Markdown); 31 | SendResponse response = telegramBot.execute(request); 32 | 33 | if (response.isOk()) { 34 | log.debug("Telegram message sent successfully"); 35 | } else { 36 | log.error("Failed to send Telegram message: {}", response.description()); 37 | } 38 | } 39 | 40 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/TimeframeCalculator.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 4 | import org.springframework.stereotype.Service; 5 | 6 | import java.time.LocalDate; 7 | import java.time.ZoneId; 8 | import java.time.ZonedDateTime; 9 | 10 | /** 11 | * Calculator for everything time and date related. 12 | */ 13 | @Service 14 | public class TimeframeCalculator { 15 | 16 | private static final ZoneId ZONE_BERLIN = ZoneId.of("Europe/Berlin"); 17 | private static final int OFFSET_HOURS = 2; 18 | private static final int DAY_HOURS = 24; 19 | 20 | /** 21 | * Calculates the {@link TimeframeDTO} containing the epoch seconds from and to of yesterday. 22 | * 23 | * @return a {@link TimeframeDTO} object containing the values 24 | */ 25 | public TimeframeDTO calculateTimeframeForYesterday() { 26 | return calculateTimeframeFor(LocalDate.now(ZONE_BERLIN).minusDays(1)); 27 | } 28 | 29 | /** 30 | * Calculates the {@link TimeframeDTO} containing the epoch seconds from and to of the given date. 31 | * 32 | * @param date the date for which to calculate the timeframe 33 | * @return a {@link TimeframeDTO} object containing the values 34 | */ 35 | public TimeframeDTO calculateTimeframeFor(LocalDate date) { 36 | ZonedDateTime startOfDay = date.atStartOfDay(ZONE_BERLIN).plusHours(OFFSET_HOURS); 37 | ZonedDateTime endOfDay = startOfDay.plusHours(DAY_HOURS); 38 | 39 | return new TimeframeDTO(startOfDay.toEpochSecond(), endOfDay.toEpochSecond() - 1); 40 | } 41 | } -------------------------------------------------------------------------------- /src/test/resources/__files/update/existing-incomplete-entry.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "66db51ad6d4e833b37e7dcd6", 3 | "date": "2024-09-06", 4 | "products": [ 5 | { 6 | "name": "British Heritage, Premium Cheddar Mature", 7 | "amount": "50 g", 8 | "calories": 205.0, 9 | "fat": 17.5, 10 | "carbs": 0.3, 11 | "protein": 12.0, 12 | "link": "/db/en/food/diverse_british_heritage_premium_cheddar_mature/index.html" 13 | }, 14 | { 15 | "name": "Burger Sauce", 16 | "amount": "40 ml", 17 | "calories": 79.0, 18 | "fat": 5.6, 19 | "carbs": 6.4, 20 | "protein": 0.4, 21 | "link": "/db/en/food/bulls-eye_burger_sauce/index.html" 22 | }, 23 | { 24 | "name": "Burger patty", 25 | "amount": "250 g", 26 | "calories": 617.0, 27 | "fat": 47.5, 28 | "carbs": 0.3, 29 | "protein": 47.5, 30 | "link": "/db/en/food/butchers_by_penny_burger_patty/index.html" 31 | }, 32 | { 33 | "name": "Butterschmalz", 34 | "amount": "10 g", 35 | "calories": 90.0, 36 | "fat": 10.0, 37 | "carbs": 0.0, 38 | "protein": 0.0, 39 | "link": "/db/en/food/meggle_butterschmalz/index.html" 40 | }, 41 | { 42 | "name": "Hamburger Brötchen, Glutenfrei", 43 | "amount": "150 g", 44 | "calories": 339.0, 45 | "fat": 3.9, 46 | "carbs": 66.0, 47 | "protein": 4.8, 48 | "link": "/db/en/food/schaer_hamburger_broetchen_glutenfrei/index.html" 49 | } 50 | ], 51 | "totalCalories": 1330.0, 52 | "totalFat": 84.5, 53 | "totalCarbs": 72.9, 54 | "totalSugar": 12.1, 55 | "totalProtein": 64.7, 56 | "totalFibre": 8.6 57 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataMigrationResourceV2Test.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.service.DataMigrationService; 4 | import org.junit.jupiter.api.Tag; 5 | import org.junit.jupiter.api.Test; 6 | import org.junit.jupiter.api.extension.ExtendWith; 7 | import org.mockito.InjectMocks; 8 | import org.mockito.Mock; 9 | import org.mockito.junit.jupiter.MockitoExtension; 10 | import org.springframework.http.HttpStatus; 11 | import org.springframework.http.ResponseEntity; 12 | 13 | import static org.junit.jupiter.api.Assertions.*; 14 | import static org.mockito.Mockito.when; 15 | 16 | /** 17 | * Unit test for v2 Migration API. 18 | */ 19 | @ExtendWith(MockitoExtension.class) 20 | @Tag("v2") 21 | class FddbDataMigrationResourceV2Test { 22 | 23 | @Mock 24 | private DataMigrationService dataMigrationService; 25 | 26 | @InjectMocks 27 | private FddbDataMigrationResourceV2 fddbDataMigrationResourceV2; 28 | 29 | @Test 30 | void testMigrateMongoDbEntriesToInfluxDb() { 31 | int expectedMigratedCount = 42; 32 | when(dataMigrationService.migrateMongoDbEntriesToInfluxDb()).thenReturn(expectedMigratedCount); 33 | 34 | ResponseEntity response = fddbDataMigrationResourceV2.migrateMongoDbEntriesToInfluxDb(); 35 | 36 | assertEquals(HttpStatus.OK, response.getStatusCode()); 37 | assertNotNull(response.getBody()); 38 | assertTrue(response.getBody().contains(String.valueOf(expectedMigratedCount))); 39 | assertTrue(response.getBody().contains("Migrated")); 40 | assertTrue(response.getBody().contains("entries to InfluxDB")); 41 | } 42 | } 43 | 44 | -------------------------------------------------------------------------------- /docs/details/exports-and-data.md: -------------------------------------------------------------------------------- 1 | # Exports and Data 2 | 3 | ## Web Scraping Approach 4 | 5 | The FDDB Exporter uses web scraping to extract comprehensive nutritional data from fddb.info diary entries. While 6 | fddb.info offers CSV exports, these exports lack important details like product links and complete nutritional 7 | information. By using web scraping the application captures: 8 | 9 | - Complete nutritional values (calories, fat, carbs, protein, sugar, fiber) 10 | - Product names and amounts 11 | - Direct links to product pages 12 | - Daily totals and individual entries 13 | 14 | The scraping process is handled through authenticated sessions to access the diary pages, ensuring all data is retrieved 15 | accurately and completely. This approach provides richer data for analysis and storage compared to the limited CSV 16 | export option. 17 | 18 | ## Scheduled Exports 19 | 20 | The FDDB Exporter includes a built-in scheduler that automatically exports your nutritional data daily. By default, it 21 | runs at 3 AM and exports the previous day's data. You can customize the schedule using the 22 | `FDDB-EXPORTER_SCHEDULER_CRON` environment variable with any valid cron expression. Be aware that the cron expression 23 | is a Spring Boot cron expression, so make sure to use the correct format. 24 | 25 | Configuration options: 26 | 27 | - `FDDB-EXPORTER_SCHEDULER_ENABLED`: Enable/disable the scheduler (defaults to true) 28 | - `FDDB-EXPORTER_SCHEDULER_CRON`: Set custom schedule (defaults to `0 0 3 * * *`) 29 | 30 | For cases where you need data outside the scheduled exports, the [REST API](/details/rest-api.md) provides flexible 31 | endpoints to export data for 32 | specific timeframes or retrieve data from a certain number of days back. 33 | 34 | For an overview of how the data is stored in MongoDB, refer to the [persistence](/details/persistence.md) section. -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## Unreleased 4 | 5 | ### Changed 6 | 7 | - **BREAKING**: Removed `last7DaysAverage` and `last30DaysAverage` fields from `/api/v1/fddbdata/stats` endpoint 8 | - Stats endpoint now returns only total averages and highest values per category 9 | 10 | ### Added 11 | 12 | - New `/api/v1/fddbdata/stats/averages` endpoint to calculate rolling averages for an explicit date range (use 13 | `fromDate` and `toDate`, format YYYY-MM-DD) 14 | 15 | ### Migration Guide 16 | 17 | If you were using the `last7DaysAverage` or `last30DaysAverage` fields: 18 | 19 | - Replace with calls to `/api/v1/fddbdata/stats/averages?fromDate=YYYY-MM-DD&toDate=YYYY-MM-DD` (for example, to get the 20 | last 7 full days set `toDate` to yesterday and `fromDate` to 7 days before yesterday) 21 | 22 | ## 1.6.3 23 | 24 | - dependency updates 25 | 26 | ## 1.6.2 27 | 28 | - fixed the issue originally targeted for 1.6.1 but missed due to incomplete fix 29 | 30 | ## 1.6.1 31 | 32 | - fixed issue with null amount database entries in API 33 | 34 | ## 1.6.0 35 | 36 | - added correlation API 37 | 38 | ## 1.5.0 39 | 40 | - added InfluxDB as additional persistence layer 41 | 42 | ## 1.4.0 43 | 44 | - added telemetry for anonymous usage statistics 45 | 46 | ## 1.3.0 47 | 48 | - added endpoint to retrieve stats for saved data 49 | 50 | ## 1.2.2 51 | 52 | - fixed an issue with the scheduler not running as intended 53 | 54 | ## 1.2.1 55 | 56 | - fixed an issue with updating database entries 57 | 58 | ## 1.2.0 59 | 60 | - updated product query endpoint 61 | 62 | ## 1.1.0 63 | 64 | - added Spring Actuator for healthchecks 65 | 66 | ## 1.0.0 67 | 68 | - Complete redesign of the application 69 | - Switched persistence layer to MongoDB 70 | - Updated API endpoints 71 | 72 | ## 0.3 73 | 74 | - Upgraded to Spring Boot 3 and JDK 21 75 | 76 | ## 0.2.1 77 | 78 | - Fixed login button detection due to FDDB website changes 79 | 80 | ## 0.2 81 | 82 | - Added endpoint to retrieve data for a specific number of past days 83 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/actuator/FddbHealthIndicator.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.actuator; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.FddbAdapter; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 6 | import dev.itobey.adapter.api.fddb.exporter.service.FddbParserService; 7 | import dev.itobey.adapter.api.fddb.exporter.service.TimeframeCalculator; 8 | import lombok.RequiredArgsConstructor; 9 | import lombok.extern.slf4j.Slf4j; 10 | import org.jsoup.Jsoup; 11 | import org.jsoup.nodes.Document; 12 | import org.springframework.boot.actuate.health.Health; 13 | import org.springframework.boot.actuate.health.HealthIndicator; 14 | import org.springframework.stereotype.Component; 15 | 16 | /** 17 | * This class checks if the login to FDDB is successful and reports this back to Actuator. 18 | */ 19 | @Component("fddb-login-check") 20 | @RequiredArgsConstructor 21 | @Slf4j 22 | public class FddbHealthIndicator implements HealthIndicator { 23 | 24 | private final FddbParserService fddbParserService; 25 | private final FddbAdapter fddbAdapter; 26 | private final TimeframeCalculator timeframeCalculator; 27 | 28 | @Override 29 | public Health health() { 30 | log.debug("running healthcheck to check authentication to FDDB"); 31 | TimeframeDTO timeframeDTO = timeframeCalculator.calculateTimeframeForYesterday(); 32 | String html = fddbAdapter.retrieveDataToTimeframe(timeframeDTO); 33 | Document doc = Jsoup.parse(html, "UTF-8"); 34 | try { 35 | fddbParserService.checkAuthentication(doc); 36 | return Health.up().withDetail("FDDB Status", "Authentication seems valid").build(); 37 | } catch (AuthenticationException authenticationException) { 38 | return Health.down().withDetail("FDDB Status", "Not functioning properly, Authentication seems invalid").build(); 39 | } 40 | } 41 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/exception/GlobalExceptionHandler.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.exception; 2 | 3 | import org.springframework.http.HttpStatus; 4 | import org.springframework.validation.FieldError; 5 | import org.springframework.web.bind.MethodArgumentNotValidException; 6 | import org.springframework.web.bind.annotation.ExceptionHandler; 7 | import org.springframework.web.bind.annotation.ResponseStatus; 8 | import org.springframework.web.bind.annotation.RestControllerAdvice; 9 | 10 | import java.time.DateTimeException; 11 | import java.util.HashMap; 12 | import java.util.Map; 13 | 14 | @RestControllerAdvice 15 | public class GlobalExceptionHandler { 16 | 17 | @ResponseStatus(HttpStatus.BAD_REQUEST) 18 | @ExceptionHandler({MethodArgumentNotValidException.class, DateTimeException.class}) 19 | public Map handleValidationExceptions(Exception ex) { 20 | Map errors = new HashMap<>(); 21 | if (ex instanceof MethodArgumentNotValidException methodArgumentNotValidException) { 22 | methodArgumentNotValidException.getBindingResult().getAllErrors().forEach((error) -> { 23 | String fieldName = ((FieldError) error).getField(); 24 | String errorMessage = error.getDefaultMessage(); 25 | errors.put(fieldName, errorMessage); 26 | }); 27 | } 28 | if (ex instanceof DateTimeException dateTimeException) { 29 | errors.put("dateTimeError", dateTimeException.getMessage()); 30 | } 31 | return errors; 32 | } 33 | 34 | @ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR) 35 | @ExceptionHandler({AuthenticationException.class}) 36 | public Map handleAuthenticationExceptions(Exception ex) { 37 | Map errors = new HashMap<>(); 38 | if (ex instanceof AuthenticationException authenticationException) { 39 | errors.put("authenticationError", authenticationException.getMessage()); 40 | } 41 | return errors; 42 | } 43 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/telemetry/EnvironmentDetector.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service.telemetry; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.ExecutionMode; 4 | import lombok.RequiredArgsConstructor; 5 | import org.springframework.core.env.Environment; 6 | import org.springframework.stereotype.Component; 7 | 8 | import java.io.File; 9 | import java.nio.file.Files; 10 | import java.nio.file.Paths; 11 | 12 | /** 13 | * This class is used to detect the execution mode of the application for telemetry purposes. 14 | */ 15 | @Component 16 | @RequiredArgsConstructor 17 | public class EnvironmentDetector { 18 | 19 | private final Environment environment; 20 | 21 | public ExecutionMode getExecutionMode() { 22 | if (isRunningInKubernetes()) { 23 | return ExecutionMode.KUBERNETES; 24 | } else if (isRunningInPlainContainer()) { 25 | return ExecutionMode.CONTAINER; 26 | } else { 27 | return ExecutionMode.JAR; 28 | } 29 | } 30 | 31 | private boolean isRunningInKubernetes() { 32 | return environment.containsProperty("KUBERNETES_SERVICE_HOST") || 33 | environment.containsProperty("KUBERNETES_SERVICE_PORT"); 34 | } 35 | 36 | private boolean isRunningInPlainContainer() { 37 | // Check for Docker-specific environment variable 38 | if (environment.containsProperty("DOCKER_CONTAINER")) { 39 | return true; 40 | } 41 | 42 | // Check for the presence of .dockerenv file 43 | if (new File("/.dockerenv").exists()) { 44 | return true; 45 | } 46 | 47 | // Check cgroup 48 | try { 49 | String cgroup = new String(Files.readAllBytes(Paths.get("/proc/1/cgroup"))); 50 | return cgroup.contains("docker") || cgroup.contains("containerd"); 51 | } catch (Exception e) { 52 | // File doesn't exist or can't be read, probably not in a container 53 | } 54 | 55 | return false; 56 | } 57 | } 58 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/FddbExporterProperties.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import lombok.Data; 4 | import org.springframework.boot.context.properties.ConfigurationProperties; 5 | 6 | @ConfigurationProperties(prefix = "fddb-exporter") 7 | @Data 8 | public class FddbExporterProperties { 9 | 10 | private Fddb fddb; 11 | private Scheduler scheduler; 12 | private Telemetry telemetry; 13 | private Persistence persistence; 14 | private Influxdb influxdb; 15 | private Notification notification; 16 | 17 | @Data 18 | public static class Fddb { 19 | private String url; 20 | private String username; 21 | private String password; 22 | private int minDaysBack; 23 | private int maxDaysBack; 24 | } 25 | 26 | @Data 27 | public static class Scheduler { 28 | private boolean enabled; 29 | private String cron; 30 | } 31 | 32 | @Data 33 | public static class Telemetry { 34 | private String url; 35 | private String username; 36 | private String token; 37 | private String cron; 38 | } 39 | 40 | @Data 41 | public static class Persistence { 42 | private MongoDB mongodb; 43 | private Influxdb influxdb; 44 | 45 | @Data 46 | public static class MongoDB { 47 | private boolean enabled; 48 | } 49 | 50 | @Data 51 | public static class Influxdb { 52 | private boolean enabled; 53 | } 54 | } 55 | 56 | @Data 57 | public static class Influxdb { 58 | private String url; 59 | private String token; 60 | private String org; 61 | private String bucket; 62 | } 63 | 64 | @Data 65 | public static class Notification { 66 | private Telegram telegram; 67 | private boolean enabled; 68 | 69 | @Data 70 | public static class Telegram { 71 | private String token; 72 | private String chatId; 73 | } 74 | } 75 | 76 | } 77 | -------------------------------------------------------------------------------- /src/main/resources/application.yml: -------------------------------------------------------------------------------- 1 | spring: 2 | application: 3 | name: fddb-exporter 4 | data: 5 | mongodb: 6 | host: localhost 7 | port: 27017 8 | database: fddb 9 | username: mongodb_fddb_user 10 | password: mongodb_fddb_password 11 | cloud: 12 | discovery: 13 | client: 14 | composite-indicator: 15 | enabled: false 16 | 17 | springdoc: 18 | api-docs: 19 | enabled: true 20 | path: /api-docs 21 | swagger-ui: 22 | enabled: true 23 | path: /swagger-ui.html 24 | groups-order: desc 25 | tags-sorter: alpha 26 | operations-sorter: alpha 27 | display-request-duration: true 28 | try-it-out-enabled: true 29 | show-actuator: false 30 | 31 | management: 32 | endpoints: 33 | web: 34 | exposure: 35 | include: health, scheduledtasks 36 | endpoint: 37 | health: 38 | probes: 39 | enabled: true 40 | show-details: always 41 | group: 42 | liveness: 43 | include: fddb-login-check,livenessState 44 | readiness: 45 | include: readinessState 46 | exclude: fddb-login-check 47 | health: 48 | defaults: 49 | enabled: true 50 | diskSpace: 51 | enabled: false 52 | 53 | fddb-exporter: 54 | fddb: 55 | url: https://fddb.info 56 | username: --- 57 | password: --- 58 | min-days-back: 1 59 | max-days-back: 365 60 | scheduler: 61 | enabled: true 62 | cron: "0 0 3 * * *" 63 | telemetry: 64 | url: https://telemetry.itobey.dev 65 | username: fddb-exporter 66 | token: geXXjmYGbxfCxe3EX8onXH7yyg8ywxy9 #hard coded to prevent bots from accessing public endpoint 67 | cron: "0 0 4 * * *" 68 | persistence: 69 | mongodb: 70 | enabled: true 71 | influxdb: 72 | enabled: false 73 | influxdb: 74 | url: http://localhost:8086 75 | token: --- 76 | org: primary 77 | bucket: fddb-exporter 78 | notification: 79 | enabled: true 80 | telegram: 81 | token: --- 82 | chatId: --- 83 | 84 | logging: 85 | level: 86 | root: INFO -------------------------------------------------------------------------------- /docker/docker-compose.yml: -------------------------------------------------------------------------------- 1 | #for local testing 2 | version: '3.8' 3 | 4 | services: 5 | mongodb: 6 | image: mongo:8.2.2 7 | container_name: mongodb 8 | ports: 9 | - "27017:27017" 10 | environment: 11 | MONGO_INITDB_ROOT_USERNAME: admin 12 | MONGO_INITDB_ROOT_PASSWORD: adminpassword 13 | MONGO_INITDB_DATABASE: fddb 14 | volumes: 15 | - mongo-data:/data/db 16 | - ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro 17 | networks: 18 | - container_network 19 | 20 | influxdb: 21 | image: influxdb:2.8 22 | container_name: influxdb 23 | ports: 24 | - "8086:8086" 25 | environment: 26 | DOCKER_INFLUXDB_INIT_MODE: setup 27 | DOCKER_INFLUXDB_INIT_USERNAME: admin 28 | DOCKER_INFLUXDB_INIT_PASSWORD: password 29 | DOCKER_INFLUXDB_INIT_ORG: primary 30 | DOCKER_INFLUXDB_INIT_BUCKET: fddb-exporter 31 | DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: token 32 | volumes: 33 | - influxdb-data:/var/lib/influxdb2 34 | networks: 35 | - container_network 36 | 37 | fddb-exporter: 38 | image: ghcr.io/itobey/fddb-exporter 39 | container_name: fddb-exporter 40 | ports: 41 | - "8080:8080" 42 | environment: 43 | SPRING_DATA_MONGODB_HOST: mongodb 44 | SPRING_DATA_MONGODB_PORT: 27017 45 | SPRING_DATA_MONGODB_DATABASE: fddb 46 | SPRING_DATA_MONGODB_USERNAME: mongodb_fddb_user 47 | SPRING_DATA_MONGODB_PASSWORD: mongodb_fddb_password 48 | FDDB-EXPORTER_PERSISTENCE_MONGODB_ENABLED: true 49 | FDDB-EXPORTER_PERSISTENCE_INFLUXDB_ENABLED: true 50 | FDDB-EXPORTER_INFLUXDB_TOKEN: token 51 | FDDB-EXPORTER_INFLUXDB_URL: http://influxdb:8086 52 | FDDB-EXPORTER_INFLUXDB_ORG: primary 53 | FDDB-EXPORTER_INFLUXDB_BUCKET: fddb-exporter 54 | FDDB-EXPORTER_FDDB_USERNAME: --- 55 | FDDB-EXPORTER_FDDB_PASSWORD: --- 56 | TZ: Europe/Berlin 57 | depends_on: 58 | - mongodb 59 | - influxdb 60 | 61 | networks: 62 | - container_network 63 | 64 | networks: 65 | container_network: 66 | driver: bridge 67 | 68 | volumes: 69 | mongo-data: 70 | influxdb-data: -------------------------------------------------------------------------------- /docs/details/docker.md: -------------------------------------------------------------------------------- 1 | # Docker 2 | 3 | ## Docker Image 4 | 5 | The application can be run in a Docker container. 6 | Githubs [GHCR Repository](https://github.com/itobey/fddb-exporter/pkgs/container/fddb-exporter) currently 7 | hosts an amd64 and arm64 architecture official image. There is also a [Helm Chart](/details/helm.md) available for 8 | deployment. 9 | If you want to build your own image, you can use the 10 | [Dockerfile](https://github.com/itobey/fddb-exporter/blob/master/docker/Dockerfile) in this repository. 11 | 12 | Running the image is as simple as: 13 | 14 | ``` 15 | docker run ghcr.io/itobey/fddb-exporter:latest 16 | ``` 17 | 18 | The following environment variables are used to configure the Docker image. 19 | 20 | ## Container Configuration 21 | 22 | The configuration of the Docker image can be done via environment variables. See the table on the 23 | [configuration](/details/configuration.md) page for a list of all available options. 24 | FDDB-Exporter uses the timezone of the environment for persisting data. For this reason, it is recommended to 25 | configure the timezone of the container with the `TZ` environment variable. 26 | 27 | ## Docker Compose 28 | 29 | The following `docker-compose.yml` file can be used to start the FDDB Exporter container. 30 | 31 | ``` 32 | docker-compose -f docker/docker-compose.yml up -d 33 | ``` 34 | 35 | This is an excerpt from the 36 | full [docker-compose.yaml](https://github.com/itobey/fddb-exporter/blob/master/docker/docker-compose.yml) 37 | file which starts the FDDB Exporter container along with 38 | a MongoDB and InfluxDB container: 39 | 40 | ```yaml 41 | version: '3.8' 42 | 43 | services: 44 | fddb-exporter: 45 | image: ghcr.io/itobey/fddb-exporter 46 | container_name: fddb-exporter 47 | ports: 48 | - "8080:8080" 49 | environment: 50 | SPRING_DATA_MONGODB_HOST: mongodb 51 | SPRING_DATA_MONGODB_PORT: 27017 52 | SPRING_DATA_MONGODB_DATABASE: fddb 53 | SPRING_DATA_MONGODB_USERNAME: mongodb_fddb_user 54 | SPRING_DATA_MONGODB_PASSWORD: mongodb_fddb_password 55 | FDDB-EXPORTER_FDDB_USERNAME: --- 56 | FDDB-EXPORTER_FDDB_PASSWORD: --- 57 | TZ: Europe/Berlin 58 | ... 59 | ``` -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataExportResourceV2Test.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.ExportResultDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 6 | import org.junit.jupiter.api.Tag; 7 | import org.junit.jupiter.api.Test; 8 | import org.junit.jupiter.api.extension.ExtendWith; 9 | import org.mockito.InjectMocks; 10 | import org.mockito.Mock; 11 | import org.mockito.junit.jupiter.MockitoExtension; 12 | import org.springframework.http.HttpStatus; 13 | import org.springframework.http.ResponseEntity; 14 | 15 | import static org.junit.jupiter.api.Assertions.assertEquals; 16 | import static org.mockito.Mockito.when; 17 | 18 | /** 19 | * Unit test for v2 Export API. 20 | */ 21 | @ExtendWith(MockitoExtension.class) 22 | @Tag("v2") 23 | class FddbDataExportResourceV2Test { 24 | 25 | @Mock 26 | private FddbDataService fddbDataService; 27 | 28 | @InjectMocks 29 | private FddbDataExportResourceV2 fddbDataExportResourceV2; 30 | 31 | @Test 32 | void testExportForTimerange() { 33 | DateRangeDTO mockRequest = new DateRangeDTO(); 34 | ExportResultDTO mockResult = new ExportResultDTO(); 35 | when(fddbDataService.exportForTimerange(mockRequest)).thenReturn(mockResult); 36 | 37 | ResponseEntity response = fddbDataExportResourceV2.exportForTimerange(mockRequest); 38 | 39 | assertEquals(HttpStatus.OK, response.getStatusCode()); 40 | assertEquals(mockResult, response.getBody()); 41 | } 42 | 43 | @Test 44 | void testExportForDaysBack() { 45 | int days = 7; 46 | boolean includeToday = true; 47 | ExportResultDTO mockResult = new ExportResultDTO(); 48 | when(fddbDataService.exportForDaysBack(days, includeToday)).thenReturn(mockResult); 49 | 50 | ResponseEntity response = fddbDataExportResourceV2.exportForDaysBack(days, includeToday); 51 | 52 | assertEquals(HttpStatus.OK, response.getStatusCode()); 53 | assertEquals(mockResult, response.getBody()); 54 | } 55 | } 56 | 57 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataExportResourceV2WebMvcTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import com.fasterxml.jackson.databind.ObjectMapper; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 6 | import lombok.SneakyThrows; 7 | import org.junit.jupiter.api.Tag; 8 | import org.junit.jupiter.api.Test; 9 | import org.springframework.beans.factory.annotation.Autowired; 10 | import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest; 11 | import org.springframework.boot.test.mock.mockito.MockBean; 12 | import org.springframework.http.MediaType; 13 | import org.springframework.test.web.servlet.MockMvc; 14 | 15 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; 16 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; 17 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; 18 | 19 | /** 20 | * WebMvc test for v2 Export API. 21 | */ 22 | @WebMvcTest(FddbDataExportResourceV2.class) 23 | @Tag("v2") 24 | @SuppressWarnings("deprecation") 25 | class FddbDataExportResourceV2WebMvcTest { 26 | 27 | @Autowired 28 | private MockMvc mockMvc; 29 | @Autowired 30 | private ObjectMapper objectMapper; 31 | @MockBean 32 | @SuppressWarnings("unused") 33 | private FddbDataService fddbDataService; 34 | 35 | @Test 36 | @SneakyThrows 37 | void exportForTimerange_InvalidInput() { 38 | DateRangeDTO invalidBatchExport = DateRangeDTO.builder() 39 | .fromDate("2023/01/01") // Invalid format 40 | .toDate(null) // Null value 41 | .build(); 42 | 43 | mockMvc.perform(post("/api/v2/fddbdata") 44 | .contentType(MediaType.APPLICATION_JSON) 45 | .content(objectMapper.writeValueAsString(invalidBatchExport))) 46 | .andExpect(status().isBadRequest()) 47 | .andExpect(jsonPath("$.fromDate").value("From date must be in the format YYYY-MM-DD")) 48 | .andExpect(jsonPath("$.toDate").value("To date cannot be null")); 49 | } 50 | } 51 | 52 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | This software is licensed under the MIT License, with an additional restriction via the "Commons Clause." While you are free to use, modify, and distribute the software, you are prohibited from selling it or using it in services that derive their primary value from the functionality of the software. 2 | 3 | (c) Copyright 2024 by itobey 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 6 | 7 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 10 | 11 | "Commons Clause" License Condition v1.0 12 | 13 | The Software is provided to you by the Licensor under the License, as defined below, subject to the following condition. 14 | 15 | Without limiting other conditions in the License, the grant of rights under the License will not include, and the License does not grant to you, the right to Sell the Software. 16 | 17 | For purposes of the foregoing, "Sell" means practicing any or all of the rights granted to you under the License to provide to third parties, for a fee or other consideration (including without limitation fees for hosting or consulting/ support services related to the Software), a product or service whose value derives, entirely or substantially, from the functionality of the Software. Any license notice or attribution required by the License must also include this Commons Clause License Condition notice. 18 | 19 | Software: All "FDDB Exporter" associated files. 20 | License: MIT 21 | Licensor: itobey -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/TelegramServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import com.pengrad.telegrambot.TelegramBot; 4 | import com.pengrad.telegrambot.request.SendMessage; 5 | import com.pengrad.telegrambot.response.SendResponse; 6 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 7 | import org.junit.jupiter.api.Test; 8 | import org.junit.jupiter.api.extension.ExtendWith; 9 | import org.mockito.Answers; 10 | import org.mockito.InjectMocks; 11 | import org.mockito.Mock; 12 | import org.mockito.junit.jupiter.MockitoExtension; 13 | 14 | import static org.mockito.ArgumentMatchers.any; 15 | import static org.mockito.Mockito.*; 16 | 17 | @ExtendWith(MockitoExtension.class) 18 | class TelegramServiceTest { 19 | 20 | @InjectMocks 21 | private TelegramService telegramService; 22 | 23 | @Mock 24 | private TelegramBot telegramBot; 25 | @Mock(answer = Answers.RETURNS_DEEP_STUBS) 26 | private FddbExporterProperties properties; 27 | 28 | @Test 29 | void shouldSendMessageSuccessfully() { 30 | // given 31 | SendResponse sendResponse = mock(SendResponse.class); 32 | when(properties.getNotification().getTelegram().getChatId()).thenReturn("test-chat-id"); 33 | when(telegramBot.execute(any(SendMessage.class))).thenReturn(sendResponse); 34 | when(sendResponse.isOk()).thenReturn(true); 35 | 36 | // when 37 | telegramService.sendMessage("Test message"); 38 | 39 | // then 40 | verify(telegramBot).execute(any(SendMessage.class)); 41 | verify(sendResponse).isOk(); 42 | } 43 | 44 | @Test 45 | void shouldHandleFailedMessage() { 46 | // given 47 | SendResponse sendResponse = mock(SendResponse.class); 48 | when(properties.getNotification().getTelegram().getChatId()).thenReturn("test-chat-id"); 49 | when(telegramBot.execute(any(SendMessage.class))).thenReturn(sendResponse); 50 | when(sendResponse.isOk()).thenReturn(false); 51 | when(sendResponse.description()).thenReturn("Error sending message"); 52 | 53 | // when 54 | telegramService.sendMessage("Test message"); 55 | 56 | // then 57 | verify(telegramBot).execute(any(SendMessage.class)); 58 | verify(sendResponse).isOk(); 59 | verify(sendResponse).description(); 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/Scheduler.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 4 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 5 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 6 | import dev.itobey.adapter.api.fddb.exporter.service.telemetry.TelemetryService; 7 | import lombok.RequiredArgsConstructor; 8 | import lombok.extern.slf4j.Slf4j; 9 | import org.springframework.context.annotation.Configuration; 10 | import org.springframework.scheduling.annotation.SchedulingConfigurer; 11 | import org.springframework.scheduling.config.ScheduledTaskRegistrar; 12 | 13 | /** 14 | * This class is used to schedule the export of FDDb data and telemetry data. 15 | */ 16 | @Configuration 17 | @RequiredArgsConstructor 18 | @Slf4j 19 | public class Scheduler implements SchedulingConfigurer { 20 | 21 | private final FddbDataService fddbDataService; 22 | private final TelemetryService telemetryService; 23 | private final FddbExporterProperties properties; 24 | private final TelegramService telegramService; 25 | 26 | @Override 27 | public void configureTasks(ScheduledTaskRegistrar taskRegistrar) { 28 | if (properties.getScheduler().isEnabled()) { 29 | taskRegistrar.addCronTask(this::runFddbExportForYesterday, properties.getScheduler().getCron()); 30 | } 31 | taskRegistrar.addCronTask(this::sendTelemetryData, properties.getTelemetry().getCron()); 32 | } 33 | 34 | private void sendTelemetryData() { 35 | log.debug("sending telemetry data"); 36 | telemetryService.sendTelemetryData(); 37 | } 38 | 39 | private void runFddbExportForYesterday() { 40 | log.trace("starting scheduled export"); 41 | try { 42 | fddbDataService.exportForDaysBack(1, false); 43 | } catch (AuthenticationException authenticationException) { 44 | log.error("not logged in - skipping job execution"); 45 | } catch (ParseException parseException) { 46 | String errorMessage = "data for yesterday cannot be parsed, skipping this day"; 47 | log.warn(errorMessage, parseException); 48 | if (properties.getNotification().isEnabled()) { 49 | telegramService.sendMessage(errorMessage); 50 | } 51 | } 52 | } 53 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v1/FddbDataResourceWebMvcTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v1; 2 | 3 | import com.fasterxml.jackson.databind.ObjectMapper; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.service.DataMigrationService; 6 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 7 | import lombok.SneakyThrows; 8 | import org.junit.jupiter.api.Tag; 9 | import org.junit.jupiter.api.Test; 10 | import org.springframework.beans.factory.annotation.Autowired; 11 | import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest; 12 | import org.springframework.boot.test.mock.mockito.MockBean; 13 | import org.springframework.http.MediaType; 14 | import org.springframework.test.web.servlet.MockMvc; 15 | 16 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; 17 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; 18 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; 19 | 20 | /** 21 | * WebMvc test for deprecated v1 API compatibility. 22 | * 23 | * @deprecated Tests deprecated v1 API. Create new tests for v2 controllers. 24 | */ 25 | @Deprecated 26 | @WebMvcTest(FddbDataResourceV1.class) 27 | @Tag("v1-compat") 28 | class FddbDataResourceWebMvcTest { 29 | 30 | @Autowired 31 | private MockMvc mockMvc; 32 | @Autowired 33 | private ObjectMapper objectMapper; 34 | @MockBean 35 | private FddbDataService fddbDataService; 36 | @MockBean 37 | private DataMigrationService dataMigrationService; 38 | 39 | @Test 40 | @SneakyThrows 41 | void exportForTimerange_InvalidInput() { 42 | DateRangeDTO invalidBatchExport = DateRangeDTO.builder() 43 | .fromDate("2023/01/01") // Invalid format 44 | .toDate(null) // Null value 45 | .build(); 46 | 47 | mockMvc.perform(post("/api/v1/fddbdata") 48 | .contentType(MediaType.APPLICATION_JSON) 49 | .content(objectMapper.writeValueAsString(invalidBatchExport))) 50 | .andExpect(status().isBadRequest()) 51 | .andExpect(jsonPath("$.fromDate").value("From date must be in the format YYYY-MM-DD")) 52 | .andExpect(jsonPath("$.toDate").value("To date cannot be null")); 53 | } 54 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/rest/v1/CorrelationResourceV1.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v1; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationInputDto; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationOutputDto; 5 | import dev.itobey.adapter.api.fddb.exporter.service.CorrelationService; 6 | import io.swagger.v3.oas.annotations.tags.Tag; 7 | import lombok.RequiredArgsConstructor; 8 | import lombok.extern.slf4j.Slf4j; 9 | import org.springframework.validation.annotation.Validated; 10 | import org.springframework.web.bind.annotation.PostMapping; 11 | import org.springframework.web.bind.annotation.RequestBody; 12 | import org.springframework.web.bind.annotation.RequestMapping; 13 | import org.springframework.web.bind.annotation.RestController; 14 | 15 | /** 16 | * Provides REST API endpoints for correlation analysis. 17 | *

18 | * The API endpoints are mapped to the "/api/v1/correlation" path. 19 | * 20 | * @deprecated This is the v1 API compatibility layer. Please migrate to 21 | * {@link dev.itobey.adapter.api.fddb.exporter.rest.v2.CorrelationResourceV2}. 22 | * See docs/migration/v1-to-v2.md for migration guide. 23 | * This v1 API will be removed after 2026-06-30. 24 | */ 25 | // DEPRECATED: Remove after 2026-06-30 when all clients have migrated to v2 26 | @Deprecated 27 | @RestController 28 | @RequestMapping("/api/v1/correlation") 29 | @Slf4j 30 | @Validated 31 | @RequiredArgsConstructor 32 | @Tag(name = "Correlation (DEPRECATED v1)", description = "⚠️ DEPRECATED: Legacy v1 API - Will be removed after 2026-06-30. Please use v2 API endpoints.") 33 | public class CorrelationResourceV1 { 34 | 35 | private final CorrelationService correlationService; 36 | 37 | /** 38 | * Create a correlation analysis between two data series. 39 | * 40 | * @param correlationInputDto input data containing the series to correlate 41 | * @return correlation analysis result 42 | * @deprecated Use {@link dev.itobey.adapter.api.fddb.exporter.rest.v2.CorrelationResourceV2#createCorrelation(CorrelationInputDto)} 43 | */ 44 | @Deprecated 45 | @PostMapping 46 | public CorrelationOutputDto createCorrelation(@RequestBody CorrelationInputDto correlationInputDto) { 47 | log.debug("V1 (DEPRECATED): Creating correlation analysis"); 48 | return correlationService.createCorrelation(correlationInputDto); 49 | } 50 | 51 | } 52 | 53 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/CorrelationResourceV2.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationInputDto; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.correlation.CorrelationOutputDto; 5 | import dev.itobey.adapter.api.fddb.exporter.service.CorrelationService; 6 | import io.swagger.v3.oas.annotations.Operation; 7 | import io.swagger.v3.oas.annotations.media.Content; 8 | import io.swagger.v3.oas.annotations.media.Schema; 9 | import io.swagger.v3.oas.annotations.responses.ApiResponse; 10 | import io.swagger.v3.oas.annotations.responses.ApiResponses; 11 | import io.swagger.v3.oas.annotations.tags.Tag; 12 | import lombok.RequiredArgsConstructor; 13 | import lombok.extern.slf4j.Slf4j; 14 | import org.springframework.validation.annotation.Validated; 15 | import org.springframework.web.bind.annotation.PostMapping; 16 | import org.springframework.web.bind.annotation.RequestBody; 17 | import org.springframework.web.bind.annotation.RequestMapping; 18 | import org.springframework.web.bind.annotation.RestController; 19 | 20 | /** 21 | * V2 REST API for correlation analysis. 22 | *

23 | * Provides endpoints for calculating correlations between different data series. 24 | *

25 | * The API endpoints are mapped to the "/api/v2/correlation" path. 26 | * 27 | * @since 2.0.0 28 | */ 29 | @RestController 30 | @RequestMapping("/api/v2/correlation") 31 | @Slf4j 32 | @Validated 33 | @RequiredArgsConstructor 34 | @Tag(name = "Correlation Analysis", description = "Correlation analysis between data series") 35 | public class CorrelationResourceV2 { 36 | 37 | private final CorrelationService correlationService; 38 | 39 | /** 40 | * Create a correlation analysis between two data series. 41 | * 42 | * @param correlationInputDto input data containing the series to correlate 43 | * @return correlation analysis result 44 | */ 45 | @Operation(summary = "Create correlation analysis", description = "Calculate correlation between two data series") 46 | @ApiResponses(value = { 47 | @ApiResponse(responseCode = "200", description = "Correlation calculated successfully", 48 | content = @Content(mediaType = "application/json", schema = @Schema(implementation = CorrelationOutputDto.class))) 49 | }) 50 | @PostMapping 51 | public CorrelationOutputDto createCorrelation(@RequestBody CorrelationInputDto correlationInputDto) { 52 | log.debug("V2: Creating correlation analysis"); 53 | return correlationService.createCorrelation(correlationInputDto); 54 | } 55 | } 56 | 57 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/EnvironmentDetectorTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.ExecutionMode; 4 | import dev.itobey.adapter.api.fddb.exporter.service.telemetry.EnvironmentDetector; 5 | import org.junit.jupiter.api.Test; 6 | import org.junit.jupiter.api.extension.ExtendWith; 7 | import org.mockito.InjectMocks; 8 | import org.mockito.Mock; 9 | import org.mockito.MockedStatic; 10 | import org.mockito.Mockito; 11 | import org.mockito.junit.jupiter.MockitoExtension; 12 | import org.springframework.core.env.Environment; 13 | 14 | import java.nio.file.Files; 15 | import java.nio.file.Path; 16 | import java.nio.file.Paths; 17 | 18 | import static org.junit.jupiter.api.Assertions.assertEquals; 19 | import static org.mockito.Mockito.when; 20 | 21 | @ExtendWith(MockitoExtension.class) 22 | class EnvironmentDetectorTest { 23 | 24 | @InjectMocks 25 | private EnvironmentDetector environmentDetector; 26 | 27 | @Mock 28 | private Environment environment; 29 | 30 | @Test 31 | void getExecutionMode_whenExecutionIsKubernetes_shouldReturnKubernetes() { 32 | when(environment.containsProperty("KUBERNETES_SERVICE_HOST")).thenReturn(true); 33 | assertEquals(ExecutionMode.KUBERNETES, environmentDetector.getExecutionMode()); 34 | } 35 | 36 | @Test 37 | void getExecutionMode_whenExecutionIsContainerAndEnvSet_shouldReturnContainer() { 38 | when(environment.containsProperty("KUBERNETES_SERVICE_HOST")).thenReturn(false); 39 | when(environment.containsProperty("KUBERNETES_SERVICE_PORT")).thenReturn(false); 40 | when(environment.containsProperty("DOCKER_CONTAINER")).thenReturn(true); 41 | assertEquals(ExecutionMode.CONTAINER, environmentDetector.getExecutionMode()); 42 | } 43 | 44 | @Test 45 | void getExecutionMode_whenExecutionIsContainerAndCgroupMatches_shouldReturnContainer() { 46 | Path mockPath = Mockito.mock(Path.class); 47 | try (MockedStatic mockedPaths = Mockito.mockStatic(Paths.class); 48 | MockedStatic mockedFiles = Mockito.mockStatic(Files.class)) { 49 | 50 | mockedPaths.when(() -> Paths.get("/proc/1/cgroup")).thenReturn(mockPath); 51 | mockedFiles.when(() -> Files.readAllBytes(mockPath)).thenReturn("docker".getBytes()); 52 | 53 | assertEquals(ExecutionMode.CONTAINER, environmentDetector.getExecutionMode()); 54 | } 55 | } 56 | 57 | @Test 58 | void testGetExecutionMode_Jar() { 59 | assertEquals(ExecutionMode.JAR, environmentDetector.getExecutionMode()); 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /docs/details/helm.md: -------------------------------------------------------------------------------- 1 | # Helm 2 | 3 | ## Helm Chart 4 | 5 | [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/fddb-exporter)](https://artifacthub.io/packages/helm/fddb-exporter/fddb-exporter) 6 | 7 | The official Helm Chart can be used to deploy the FDDB-Exporter application to a Kubernetes cluster. 8 | 9 | ``` 10 | helm install fddb-exporter oci://ghcr.io/itobey/charts/fddb-exporter --version 1.1.4 11 | ``` 12 | 13 | Or checkout the [Fddb-Exporter Chart](https://github.com/itobey/charts/tree/master/fddb-exporter) yourself. 14 | 15 | To see an example of how to use the Helm Chart in an umbrella chart for deployment 16 | with [ArgoCD](https://argo-cd.readthedocs.io/en/stable/), you can check 17 | out [this repository](https://github.com/itobey/k3s-nuc/blob/master/deploy/fddb-exporter/Chart.yaml) 18 | 19 | Currently the Helm Chart only deploys the application and does not include any additional services like MongoDB and 20 | InfluxDB. 21 | 22 | ## Configuration 23 | 24 | The following configuration options are an excerpt from the `values.yaml` file to configure the Helm Chart. 25 | The full list of available options can be found in 26 | the [values.yaml](https://artifacthub.io/packages/helm/fddb-exporter/fddb-exporter?modal=values) file. 27 | 28 | ### Secret Management 29 | 30 | To authenticate with FDDB, MongoDB, and InfluxDB, you have two options: 31 | 32 | 1. Using `secretRef`: 33 | You can reference an existing Kubernetes secret by specifying the secretRef option. Use secretRef.name to point to 34 | the desired secret. 35 | 36 | 2. Using `username` and `password`: 37 | Alternatively, you can provide credentials directly using the username and password options. If this method is used, 38 | a new Kubernetes secret containing the password will be created automatically. 39 | 40 | Important: 41 | If both `secretRef` and direct username/password options are provided, the `secretRef` will take precedence. 42 | 43 | Any additional configuration options for the application will be stored in a ConfigMap. Both secrets and ConfigMaps will 44 | be exposed as environment variables and mounted into the container for access. 45 | 46 | ```yaml 47 | mongodb: 48 | enabled: true 49 | host: "localhost" 50 | port: "27017" 51 | database: "database" 52 | username: "username" 53 | password: "password" 54 | secretRef: 55 | name: "" 56 | 57 | influxdb: 58 | enabled: false 59 | url: "http://localhost:8086" 60 | org: "primary" 61 | bucket: "fddb-exporter" 62 | token: "token" 63 | secretRef: 64 | name: "" 65 | 66 | fddb: 67 | auth: 68 | username: "username" 69 | password: "password" 70 | secretRef: 71 | name: "" 72 | 73 | timezone: "Europe/Berlin" 74 | ``` -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataMigrationResourceV2.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.annotation.RequiresInfluxDb; 4 | import dev.itobey.adapter.api.fddb.exporter.annotation.RequiresMongoDb; 5 | import dev.itobey.adapter.api.fddb.exporter.service.DataMigrationService; 6 | import io.swagger.v3.oas.annotations.Operation; 7 | import io.swagger.v3.oas.annotations.media.Content; 8 | import io.swagger.v3.oas.annotations.responses.ApiResponse; 9 | import io.swagger.v3.oas.annotations.responses.ApiResponses; 10 | import io.swagger.v3.oas.annotations.tags.Tag; 11 | import lombok.extern.slf4j.Slf4j; 12 | import org.springframework.beans.factory.annotation.Autowired; 13 | import org.springframework.http.ResponseEntity; 14 | import org.springframework.validation.annotation.Validated; 15 | import org.springframework.web.bind.annotation.PostMapping; 16 | import org.springframework.web.bind.annotation.RequestMapping; 17 | import org.springframework.web.bind.annotation.RestController; 18 | 19 | /** 20 | * V2 REST API for data migration operations. 21 | *

22 | * Provides endpoints for: 23 | * - Migrating data from MongoDB to InfluxDB 24 | *

25 | * The API endpoints are mapped to the "/api/v2/migration" path. 26 | * 27 | * @since 2.0.0 28 | */ 29 | @RestController 30 | @RequestMapping("/api/v2/migration") 31 | @Slf4j 32 | @Validated 33 | @Tag(name = "FDDB Data Migration", description = "Data migration operations between storage backends") 34 | public class FddbDataMigrationResourceV2 { 35 | 36 | @Autowired(required = false) 37 | private DataMigrationService dataMigrationService; 38 | 39 | /** 40 | * Migrate all MongoDB entries to InfluxDB. 41 | * 42 | * @return the number of entries migrated 43 | */ 44 | @Operation(summary = "Migrate data to InfluxDB", description = "Migrate all MongoDB entries to InfluxDB") 45 | @ApiResponses(value = { 46 | @ApiResponse(responseCode = "200", description = "Migration completed successfully", content = @Content), 47 | @ApiResponse(responseCode = "503", description = "MongoDB or InfluxDB not available", content = @Content) 48 | }) 49 | @PostMapping("/toInfluxDb") 50 | @RequiresMongoDb 51 | @RequiresInfluxDb 52 | public ResponseEntity migrateMongoDbEntriesToInfluxDb() { 53 | log.info("V2: Starting migration from MongoDB to InfluxDB"); 54 | int amountEntries = dataMigrationService.migrateMongoDbEntriesToInfluxDb(); 55 | log.info("V2: Migration completed. Migrated {} entries", amountEntries); 56 | return ResponseEntity.ok("Migrated " + amountEntries + " entries to InfluxDB"); 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /src/test/resources/__files/update/expected-after-update.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "66db51ad6d4e833b37e7dcd6", 3 | "date": "2024-09-06", 4 | "products": [ 5 | { 6 | "name": "Cheddar Käse (Subway)", 7 | "amount": "28 g", 8 | "calories": 113.0, 9 | "fat": 8.7, 10 | "carbs": 0.6, 11 | "protein": 7.0, 12 | "link": "/db/en/food/subway_cheddar_kaese_subway/index.html" 13 | }, 14 | { 15 | "name": "Subway Chicken Teriyaki", 16 | "amount": "466 g", 17 | "calories": 750.0, 18 | "fat": 6.1, 19 | "carbs": 88.5, 20 | "protein": 37.3, 21 | "link": "/db/en/food/subway_subway_chicken_teriyaki_1176162/index.html" 22 | }, 23 | { 24 | "name": "Subway Sauce, Asiago Caesar", 25 | "amount": "42 g", 26 | "calories": 200.0, 27 | "fat": 18.5, 28 | "carbs": 3.6, 29 | "protein": 1.3, 30 | "link": "/db/en/food/subway_subway_sauce_asiago_caesar/index.html" 31 | }, 32 | { 33 | "name": "Kinder Duo", 34 | "amount": "50 g", 35 | "calories": 262.0, 36 | "fat": 14.1, 37 | "carbs": 29.9, 38 | "protein": 3.6, 39 | "link": "/db/en/food/ferrero_kinder_duo_1054818/index.html" 40 | }, 41 | { 42 | "name": "British Heritage, Premium Cheddar Mature", 43 | "amount": "50 g", 44 | "calories": 205.0, 45 | "fat": 17.5, 46 | "carbs": 0.3, 47 | "protein": 12.0, 48 | "link": "/db/en/food/diverse_british_heritage_premium_cheddar_mature/index.html" 49 | }, 50 | { 51 | "name": "Burger Sauce", 52 | "amount": "40 ml", 53 | "calories": 79.0, 54 | "fat": 5.6, 55 | "carbs": 6.4, 56 | "protein": 0.4, 57 | "link": "/db/en/food/bulls-eye_burger_sauce/index.html" 58 | }, 59 | { 60 | "name": "Burger patty", 61 | "amount": "250 g", 62 | "calories": 617.0, 63 | "fat": 47.5, 64 | "carbs": 0.3, 65 | "protein": 47.5, 66 | "link": "/db/en/food/butchers_by_penny_burger_patty/index.html" 67 | }, 68 | { 69 | "name": "Butterschmalz", 70 | "amount": "10 g", 71 | "calories": 90.0, 72 | "fat": 10.0, 73 | "carbs": 0.0, 74 | "protein": 0.0, 75 | "link": "/db/en/food/meggle_butterschmalz/index.html" 76 | }, 77 | { 78 | "name": "Hamburger Brötchen, Glutenfrei", 79 | "amount": "150 g", 80 | "calories": 339.0, 81 | "fat": 3.9, 82 | "carbs": 66.0, 83 | "protein": 4.8, 84 | "link": "/db/en/food/schaer_hamburger_broetchen_glutenfrei/index.html" 85 | } 86 | ], 87 | "totalCalories": 2656.0, 88 | "totalFat": 131.7, 89 | "totalCarbs": 195.5, 90 | "totalSugar": 50.4, 91 | "totalProtein": 113.9, 92 | "totalFibre": 14.6 93 | } -------------------------------------------------------------------------------- /docs/introduction/getting-started.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # Getting started 8 | 9 | This section shortly describes how to get started with the FDDB Exporter. 10 | 11 | ## Prerequisites 12 | 13 | - Docker or Java 21+ runtime environment 14 | - A valid FDDB.info account 15 | - MongoDB instance (optional, but recommended for all data) 16 | - InfluxDB instance (optional, but recommended for storing daily totals) 17 | 18 | ## Installation and Deployment 19 | 20 | Using a containerized environment is the recommended way to run this application. But you can also run it on your 21 | local machine with a Java 21+ runtime environment. 22 | 23 | ### Pre-built Docker Image 24 | 25 | You can use the pre-built Docker image to quickly set up the application. You have two options here if you do **not** 26 | want to deploy a Helm Chart. 27 | More information about the Docker image configuration can be found on this [detail page](/details/docker.md). 28 | 29 | 1. Pull the pre-built Docker image: 30 | ``` 31 | docker run ghcr.io/itobey/fddb-exporter:latest 32 | ``` 33 | 34 | 2. Use the provided [docker-compose.yaml](https://github.com/itobey/fddb-exporter/blob/master/docker/docker-compose.yml) 35 | file to start the FDDB Exporter container along with 36 | a MongoDB and InfluxDB container. 37 | ``` 38 | docker-compose -f docker/docker-compose.yml up -d 39 | ``` 40 | 41 | ### Pre-built Helm Chart 42 | 43 | [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/fddb-exporter)](https://artifacthub.io/packages/helm/fddb-exporter/fddb-exporter) 44 | 45 | You may also use the pre-built Helm Chart to deploy the application. More information about the Helm Chart 46 | configuration can be found on this [detail page](/details/helm.md). 47 | 48 | - Use the pre-built Helm Chart: 49 | ``` 50 | helm install fddb-exporter oci://ghcr.io/itobey/charts/fddb-exporter --version 1.1.4 51 | ``` 52 | 53 | - or checkout the [Fddb-Exporter Chart](https://github.com/itobey/charts/tree/master/fddb-exporter) yourself 54 | 55 | ### Pre-built Jar File 56 | 57 | Download the pre-built jar file from the [release page](https://github.com/itobey/fddb-exporter/releases) and run it 58 | with Java 21+ runtime environment. 59 | 60 | ### Building from Source 61 | 62 | You can also build the application from source yourself. For the following steps, you need a Java 21+ runtime 63 | environment and Maven installed. 64 | 65 | 1. Clone the repository: 66 | ``` 67 | git clone https://github.com/itobey/fddb-exporter.git 68 | cd fddb-exporter 69 | ``` 70 | 71 | 2. Build the application: 72 | ``` 73 | mvn clean install 74 | ``` 75 | 76 | 3. (optionally) Build the Docker image: 77 | ``` 78 | docker build -f docker/Dockerfile -t fddb-exporter . 79 | ``` -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataStatsResourceV2Test.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.RollingAveragesDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.StatsDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 7 | import org.junit.jupiter.api.Tag; 8 | import org.junit.jupiter.api.Test; 9 | import org.junit.jupiter.api.extension.ExtendWith; 10 | import org.mockito.InjectMocks; 11 | import org.mockito.Mock; 12 | import org.mockito.junit.jupiter.MockitoExtension; 13 | import org.springframework.http.HttpStatus; 14 | import org.springframework.http.ResponseEntity; 15 | 16 | import static org.junit.jupiter.api.Assertions.assertEquals; 17 | import static org.mockito.Mockito.when; 18 | 19 | /** 20 | * Unit test for v2 Stats API. 21 | */ 22 | @ExtendWith(MockitoExtension.class) 23 | @Tag("v2") 24 | class FddbDataStatsResourceV2Test { 25 | 26 | @Mock 27 | private FddbDataService fddbDataService; 28 | 29 | @InjectMocks 30 | private FddbDataStatsResourceV2 fddbDataStatsResourceV2; 31 | 32 | @Test 33 | void testGetStats() { 34 | StatsDTO mockStats = new StatsDTO(); 35 | when(fddbDataService.getStats()).thenReturn(mockStats); 36 | 37 | ResponseEntity response = fddbDataStatsResourceV2.getStats(); 38 | 39 | assertEquals(HttpStatus.OK, response.getStatusCode()); 40 | assertEquals(mockStats, response.getBody()); 41 | } 42 | 43 | @Test 44 | void testGetRollingAverages_Success() { 45 | DateRangeDTO dateRangeDTO = DateRangeDTO.builder() 46 | .fromDate("2023-01-01") 47 | .toDate("2023-01-31") 48 | .build(); 49 | RollingAveragesDTO mockAverages = new RollingAveragesDTO(); 50 | when(fddbDataService.getRollingAverages(dateRangeDTO)).thenReturn(mockAverages); 51 | 52 | ResponseEntity response = fddbDataStatsResourceV2.getRollingAverages(dateRangeDTO); 53 | 54 | assertEquals(HttpStatus.OK, response.getStatusCode()); 55 | assertEquals(mockAverages, response.getBody()); 56 | } 57 | 58 | @Test 59 | void testGetRollingAverages_InvalidDateRange() { 60 | DateRangeDTO dateRangeDTO = DateRangeDTO.builder() 61 | .fromDate("2023-01-31") 62 | .toDate("2023-01-01") 63 | .build(); 64 | String errorMessage = "From date must be before to date"; 65 | when(fddbDataService.getRollingAverages(dateRangeDTO)) 66 | .thenThrow(new IllegalArgumentException(errorMessage)); 67 | 68 | ResponseEntity response = fddbDataStatsResourceV2.getRollingAverages(dateRangeDTO); 69 | 70 | assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode()); 71 | assertEquals(errorMessage, response.getBody()); 72 | } 73 | } 74 | 75 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/FddbRequestInterceptor.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 4 | import feign.RequestInterceptor; 5 | import feign.RequestTemplate; 6 | import lombok.RequiredArgsConstructor; 7 | import org.springframework.http.HttpEntity; 8 | import org.springframework.http.HttpHeaders; 9 | import org.springframework.http.MediaType; 10 | import org.springframework.http.ResponseEntity; 11 | import org.springframework.util.LinkedMultiValueMap; 12 | import org.springframework.util.MultiValueMap; 13 | import org.springframework.web.client.RestTemplate; 14 | 15 | import java.nio.charset.StandardCharsets; 16 | import java.util.Base64; 17 | import java.util.List; 18 | 19 | @RequiredArgsConstructor 20 | public class FddbRequestInterceptor implements RequestInterceptor { 21 | 22 | private final FddbExporterProperties properties; 23 | 24 | private String fddbCookie; 25 | 26 | @Override 27 | public void apply(RequestTemplate template) { 28 | if (template.feignTarget().url().startsWith(properties.getFddb().getUrl())) { 29 | if (fddbCookie == null) { 30 | fddbCookie = login(template.feignTarget().url()); 31 | } 32 | String password = properties.getFddb().getPassword(); 33 | String username = properties.getFddb().getUsername(); 34 | template.header("Cookie", "fddb=" + fddbCookie); 35 | String auth = Base64.getEncoder().encodeToString((username + ":" + password).getBytes(StandardCharsets.UTF_8)); 36 | template.header("Authorization", "Basic " + auth); 37 | } 38 | } 39 | 40 | private String login(String baseUrl) { 41 | RestTemplate restTemplate = new RestTemplate(); 42 | HttpHeaders headers = new HttpHeaders(); 43 | headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED); 44 | 45 | MultiValueMap map = new LinkedMultiValueMap<>(); 46 | map.add("loginemailorusername", properties.getFddb().getUsername()); 47 | map.add("loginpassword", properties.getFddb().getPassword()); 48 | 49 | HttpEntity> request = new HttpEntity<>(map, headers); 50 | 51 | ResponseEntity response = restTemplate.postForEntity( 52 | baseUrl + "/db/i18n/account/?lang=de&action=login", 53 | request, 54 | String.class 55 | ); 56 | 57 | List cookies = response.getHeaders().get(HttpHeaders.SET_COOKIE); 58 | if (cookies != null) { 59 | for (String cookie : cookies) { 60 | if (cookie.startsWith("fddb=")) { 61 | return cookie.split(";")[0].substring(5); 62 | } 63 | } 64 | } 65 | throw new AuthenticationException("Login to FDDB not successful, please check credentials"); 66 | } 67 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/TelemetryServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.TelemetryApi; 4 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 5 | import dev.itobey.adapter.api.fddb.exporter.domain.ExecutionMode; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.telemetry.TelemetryDto; 7 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.PersistenceService; 8 | import dev.itobey.adapter.api.fddb.exporter.service.telemetry.EnvironmentDetector; 9 | import dev.itobey.adapter.api.fddb.exporter.service.telemetry.TelemetryService; 10 | import org.junit.jupiter.api.BeforeEach; 11 | import org.junit.jupiter.api.Test; 12 | import org.junit.jupiter.api.extension.ExtendWith; 13 | import org.mockito.Answers; 14 | import org.mockito.ArgumentCaptor; 15 | import org.mockito.InjectMocks; 16 | import org.mockito.Mock; 17 | import org.mockito.junit.jupiter.MockitoExtension; 18 | import org.springframework.boot.info.BuildProperties; 19 | 20 | import static org.junit.jupiter.api.Assertions.assertEquals; 21 | import static org.junit.jupiter.api.Assertions.assertNotNull; 22 | import static org.mockito.Mockito.*; 23 | 24 | @ExtendWith(MockitoExtension.class) 25 | class TelemetryServiceTest { 26 | 27 | @InjectMocks 28 | private TelemetryService telemetryService; 29 | 30 | @Mock 31 | private TelemetryApi telemetryApi; 32 | @Mock 33 | private PersistenceService persistenceService; 34 | @Mock 35 | private EnvironmentDetector environmentDetector; 36 | @Mock 37 | private BuildProperties buildProperties; 38 | @Mock(answer = Answers.RETURNS_DEEP_STUBS) 39 | private FddbExporterProperties properties; 40 | 41 | @BeforeEach 42 | void setUp() { 43 | } 44 | 45 | @Test 46 | void sendTelemetryData_shouldSendTelemetryData() { 47 | // given 48 | when(persistenceService.countAllEntries()).thenReturn(10L); 49 | when(environmentDetector.getExecutionMode()).thenReturn(ExecutionMode.CONTAINER); 50 | when(buildProperties.getVersion()).thenReturn("1.0.0"); 51 | when(properties.getFddb().getUsername()).thenReturn("test@example.com"); 52 | when(properties.getPersistence().getMongodb().isEnabled()).thenReturn(true); 53 | when(properties.getPersistence().getInfluxdb().isEnabled()).thenReturn(true); 54 | 55 | // when 56 | telemetryService.sendTelemetryData(); 57 | 58 | // then 59 | ArgumentCaptor telemetryDtoCaptor = ArgumentCaptor.forClass(TelemetryDto.class); 60 | verify(telemetryApi, times(1)).sendTelemetryData(telemetryDtoCaptor.capture()); 61 | 62 | TelemetryDto capturedDto = telemetryDtoCaptor.getValue(); 63 | assertNotNull(capturedDto); 64 | assertEquals(10L, capturedDto.getDocumentCount()); 65 | assertEquals("973dfe463ec85785f5f95af5ba3906eedb2d931c24e69824a89ea65dba4e813b", capturedDto.getMailHash()); 66 | } 67 | 68 | } 69 | -------------------------------------------------------------------------------- /docs/resources/example-response.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "66d18658bc73187ea859f67c", 3 | "date": "2024-08-28", 4 | "products": [ 5 | { 6 | "name": "Waffel Al limone, gutenfrei", 7 | "amount": "64 g", 8 | "calories": 331.0, 9 | "fat": 16.6, 10 | "carbs": 42.9, 11 | "protein": 1.7, 12 | "link": "/db/en/food/schaer_waffel_al_limone_gutenfrei/index.html" 13 | }, 14 | { 15 | "name": "Senf", 16 | "amount": "30 g", 17 | "calories": 26.0, 18 | "fat": 1.2, 19 | "carbs": 1.8, 20 | "protein": 1.8, 21 | "link": "/db/en/food/durchschnittswert_senf/index.html" 22 | }, 23 | { 24 | "name": "Heinz 50% Ketchup", 25 | "amount": "30 ml", 26 | "calories": 19.0, 27 | "fat": 0.0, 28 | "carbs": 3.6, 29 | "protein": 0.4, 30 | "link": "/db/en/food/heinz_heinz_50prozent_ketchup/index.html" 31 | }, 32 | { 33 | "name": "Fleischkäse", 34 | "amount": "164 g", 35 | "calories": 501.0, 36 | "fat": 39.4, 37 | "carbs": 0.5, 38 | "protein": 30.4, 39 | "link": "/db/en/food/metzger_fleischkaese/index.html" 40 | }, 41 | { 42 | "name": "Ciabatta, mit Sauerteig", 43 | "amount": "50 g", 44 | "calories": 116.0, 45 | "fat": 1.7, 46 | "carbs": 21.0, 47 | "protein": 2.3, 48 | "link": "/db/en/food/schaer_ciabatta_mit_sauerteig/index.html" 49 | }, 50 | { 51 | "name": "Panini Rolls", 52 | "amount": "75 g", 53 | "calories": 176.0, 54 | "fat": 2.2, 55 | "carbs": 33.8, 56 | "protein": 2.8, 57 | "link": "/db/en/food/schaer_panini_rolls/index.html" 58 | }, 59 | { 60 | "name": "Vollmilch, 3,5% Fett", 61 | "amount": "40 ml", 62 | "calories": 26.0, 63 | "fat": 1.4, 64 | "carbs": 1.9, 65 | "protein": 1.2, 66 | "link": "/db/en/food/molkerei_vollmilch_3_5prozent_fett/index.html" 67 | }, 68 | { 69 | "name": "Yuzu Yakitori Chicken Bowl", 70 | "amount": "414 g", 71 | "calories": 691.0, 72 | "fat": 12.8, 73 | "carbs": 115.9, 74 | "protein": 28.6, 75 | "link": "/db/en/food/kaufland_yuzu_yakitori_chicken_bowl/index.html" 76 | }, 77 | { 78 | "name": "inside out lachs sake", 79 | "amount": "237 g", 80 | "calories": 391.0, 81 | "fat": 13.0, 82 | "carbs": 52.1, 83 | "protein": 14.7, 84 | "link": "/db/en/food/diverse_inside_out_lachs_sake/index.html" 85 | }, 86 | { 87 | "name": "High Protein Lemon Mousse", 88 | "amount": "200 g", 89 | "calories": 160.0, 90 | "fat": 5.2, 91 | "carbs": 12.0, 92 | "protein": 20.0, 93 | "link": "/db/en/food/ehrmann_high_protein_lemon_mousse/index.html" 94 | } 95 | ], 96 | "totalCalories": 2437.0, 97 | "totalFat": 93.5, 98 | "totalCarbs": 285.5, 99 | "totalSugar": 76.3, 100 | "totalProtein": 103.9, 101 | "totalFibre": 10.3 102 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/InfluxDBServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import com.influxdb.client.InfluxDBClient; 4 | import com.influxdb.client.WriteApi; 5 | import com.influxdb.client.domain.WritePrecision; 6 | import com.influxdb.client.write.Point; 7 | import com.influxdb.exceptions.InfluxException; 8 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.InfluxDBService; 9 | import org.junit.jupiter.api.BeforeEach; 10 | import org.junit.jupiter.api.Test; 11 | import org.junit.jupiter.api.extension.ExtendWith; 12 | import org.mockito.ArgumentCaptor; 13 | import org.mockito.Captor; 14 | import org.mockito.InjectMocks; 15 | import org.mockito.Mock; 16 | import org.mockito.junit.jupiter.MockitoExtension; 17 | 18 | import java.time.Instant; 19 | 20 | import static org.assertj.core.api.Assertions.assertThat; 21 | import static org.assertj.core.api.Assertions.assertThatThrownBy; 22 | import static org.mockito.Mockito.*; 23 | 24 | @ExtendWith(MockitoExtension.class) 25 | class InfluxDBServiceTest { 26 | 27 | private static final String MEASUREMENT = "dailyTotals"; 28 | private static final String FIELD = "testField"; 29 | private static final double VALUE = 42.0; 30 | 31 | @Mock 32 | private InfluxDBClient influxDBClient; 33 | 34 | @Mock 35 | private WriteApi writeApi; 36 | 37 | @Captor 38 | private ArgumentCaptor pointCaptor; 39 | 40 | @InjectMocks 41 | private InfluxDBService influxDBService; 42 | 43 | @BeforeEach 44 | void setUp() { 45 | when(influxDBClient.makeWriteApi()).thenReturn(writeApi); 46 | } 47 | 48 | @Test 49 | void writeData_shouldWritePointToInfluxDB() { 50 | // given 51 | Instant time = Instant.now(); 52 | 53 | // when 54 | influxDBService.writeData(FIELD, VALUE, time); 55 | 56 | // then 57 | verify(influxDBClient, times(1)).makeWriteApi(); 58 | verify(writeApi, times(1)).writePoint(pointCaptor.capture()); 59 | verify(writeApi, times(1)).close(); 60 | 61 | Point capturedPoint = pointCaptor.getValue(); 62 | String lineProtocol = capturedPoint.toLineProtocol(); 63 | String[] split = lineProtocol.split(" "); 64 | assert capturedPoint.getTime() != null; 65 | Instant capturedTime = Instant.ofEpochSecond(0, capturedPoint.getTime().longValue()); 66 | assertThat(split[0]).isEqualTo(MEASUREMENT); 67 | assertThat(split[1]).isEqualTo(FIELD + "=" + VALUE); 68 | assertThat(capturedTime).isEqualTo(time); 69 | assertThat(capturedPoint.getPrecision()).isEqualTo(WritePrecision.NS); 70 | } 71 | 72 | @Test 73 | void writeData_shouldThrowExceptionWhenWriteFails() { 74 | // given 75 | doThrow(new InfluxException("Write failed")).when(writeApi).writePoint(any(Point.class)); 76 | 77 | // when/then 78 | assertThatThrownBy(() -> influxDBService.writeData(FIELD, VALUE, Instant.now())) 79 | .isInstanceOf(InfluxException.class) 80 | .hasMessage("Write failed"); 81 | } 82 | } 83 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/config/OpenApiConfig.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.config; 2 | 3 | import io.swagger.v3.oas.models.OpenAPI; 4 | import io.swagger.v3.oas.models.info.Contact; 5 | import io.swagger.v3.oas.models.info.Info; 6 | import io.swagger.v3.oas.models.info.License; 7 | import io.swagger.v3.oas.models.servers.Server; 8 | import org.springdoc.core.models.GroupedOpenApi; 9 | import org.springframework.context.annotation.Bean; 10 | import org.springframework.context.annotation.Configuration; 11 | 12 | import java.util.List; 13 | 14 | @Configuration 15 | public class OpenApiConfig { 16 | 17 | @Bean 18 | public GroupedOpenApi v1Api() { 19 | return GroupedOpenApi.builder() 20 | .group("v1-deprecated") 21 | .pathsToMatch("/api/v1/**") 22 | .displayName("API v1 (DEPRECATED)") 23 | .build(); 24 | } 25 | 26 | @Bean 27 | public GroupedOpenApi v2Api() { 28 | return GroupedOpenApi.builder() 29 | .group("v2") 30 | .pathsToMatch("/api/v2/**") 31 | .displayName("API v2") 32 | .build(); 33 | } 34 | 35 | @Bean 36 | public OpenAPI customOpenAPI() { 37 | return new OpenAPI() 38 | .info(new Info() 39 | .title("FDDB Exporter API") 40 | .description(""" 41 | REST API for the FDDB Exporter application. 42 | 43 | This API provides endpoints for: 44 | - Querying FDDB nutrition data entries 45 | - Exporting data from FDDB for specified date ranges 46 | - Retrieving statistics and rolling averages 47 | - Managing data migrations between storage backends 48 | - Calculating correlations between data series 49 | 50 | **Note:** v1 API is deprecated and will be removed after 2026-06-30. 51 | Please migrate to v2 API. See migration guide: /docs/migration/v1-to-v2.md 52 | """) 53 | .version("2.0.0") 54 | .contact(new Contact() 55 | .name("FDDB Exporter") 56 | .url("https://github.com/itobey/fddb-exporter")) 57 | .license(new License() 58 | .name("MIT") 59 | .url("https://opensource.org/licenses/MIT"))) 60 | .servers(List.of( 61 | new Server() 62 | .url("http://localhost:8080") 63 | .description("Local development server"), 64 | new Server() 65 | .url("https://api.example.com") 66 | .description("Production server") 67 | )); 68 | } 69 | } 70 | 71 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataExportResourceV2IT.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import com.fasterxml.jackson.databind.ObjectMapper; 4 | import dev.itobey.adapter.api.fddb.exporter.config.TestConfig; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.PersistenceService; 7 | import lombok.SneakyThrows; 8 | import org.junit.jupiter.api.Tag; 9 | import org.junit.jupiter.api.Test; 10 | import org.junit.jupiter.params.ParameterizedTest; 11 | import org.junit.jupiter.params.provider.CsvSource; 12 | import org.springframework.beans.factory.annotation.Autowired; 13 | import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc; 14 | import org.springframework.boot.test.context.SpringBootTest; 15 | import org.springframework.context.annotation.Import; 16 | import org.springframework.http.MediaType; 17 | import org.springframework.test.context.ActiveProfiles; 18 | import org.springframework.test.web.servlet.MockMvc; 19 | 20 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; 21 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; 22 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; 23 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; 24 | 25 | /** 26 | * Integration test for v2 Export API. 27 | */ 28 | @SpringBootTest 29 | @AutoConfigureMockMvc 30 | @ActiveProfiles("test") 31 | @Import(TestConfig.class) 32 | @Tag("v2") 33 | class FddbDataExportResourceV2IT { 34 | 35 | @Autowired 36 | private MockMvc mockMvc; 37 | @Autowired 38 | private ObjectMapper objectMapper; 39 | @Autowired 40 | @SuppressWarnings("unused") 41 | private PersistenceService persistenceService; 42 | 43 | @Test 44 | @SneakyThrows 45 | void exportForTimerange_fromDateAfterToDate_shouldThrowException() { 46 | DateRangeDTO invalidBatchExport = DateRangeDTO.builder() 47 | .fromDate("2023-01-01") 48 | .toDate("2022-01-01") 49 | .build(); 50 | 51 | mockMvc.perform(post("/api/v2/fddbdata") 52 | .contentType(MediaType.APPLICATION_JSON) 53 | .content(objectMapper.writeValueAsString(invalidBatchExport))) 54 | .andExpect(status().isBadRequest()) 55 | .andExpect(jsonPath("$.dateTimeError") 56 | .value("The 'from' date cannot be after the 'to' date")); 57 | } 58 | 59 | @ParameterizedTest 60 | @CsvSource({ 61 | "0, true", 62 | "0, false", 63 | "366, true", 64 | "366, false" 65 | }) 66 | @SneakyThrows 67 | void exportForDaysBack_whenDayOutsidePermittedRange_shouldThrowException(int days, boolean includeToday) { 68 | mockMvc.perform(get("/api/v2/fddbdata/export?days=" + days + "&includeToday=" + includeToday)) 69 | .andExpect(status().isBadRequest()) 70 | .andExpect(jsonPath("$.dateTimeError") 71 | .value("Days back must be between 1 and 365")); 72 | } 73 | } 74 | 75 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/ExportServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.FddbAdapter; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.TimeframeDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 7 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 8 | import lombok.SneakyThrows; 9 | import org.junit.jupiter.api.Test; 10 | import org.junit.jupiter.api.extension.ExtendWith; 11 | import org.mockito.InjectMocks; 12 | import org.mockito.Mock; 13 | import org.mockito.junit.jupiter.MockitoExtension; 14 | 15 | import java.time.LocalDate; 16 | 17 | import static org.junit.jupiter.api.Assertions.*; 18 | import static org.mockito.Mockito.verifyNoInteractions; 19 | import static org.mockito.Mockito.when; 20 | 21 | @ExtendWith(MockitoExtension.class) 22 | class ExportServiceTest { 23 | 24 | @InjectMocks 25 | private ExportService exportService; 26 | 27 | @Mock 28 | private FddbAdapter fddbAdapter; 29 | 30 | @Mock 31 | private FddbParserService fddbParserService; 32 | 33 | @Test 34 | @SneakyThrows 35 | void exportData_whenExportSuccessful_shouldReturnResult() { 36 | // Given 37 | long fromTimestamp = 1625097600L; // 2021-07-01 00:00:00 UTC 38 | TimeframeDTO timeframeDTO = new TimeframeDTO(fromTimestamp, fromTimestamp + 86400); // One day later // 2021-07-01 00:00:00 UTC to 2021-07-02 00:00:00 UTC 39 | String mockResponse = "mock response data"; 40 | FddbData mockParsedData = new FddbData(); 41 | 42 | when(fddbAdapter.retrieveDataToTimeframe(timeframeDTO)).thenReturn(mockResponse); 43 | when(fddbParserService.parseDiary(mockResponse)).thenReturn(mockParsedData); 44 | 45 | // When 46 | FddbData result = exportService.exportData(timeframeDTO); 47 | 48 | // Then 49 | assertNotNull(result); 50 | assertEquals(LocalDate.of(2021, 7, 1), result.getDate()); 51 | } 52 | 53 | @Test 54 | @SneakyThrows 55 | void exportData_whenAuthenticationExceptionThrown_shouldThrowException() { 56 | // Given 57 | TimeframeDTO timeframeDTO = new TimeframeDTO(1625097600L, 1625184000L); 58 | when(fddbAdapter.retrieveDataToTimeframe(timeframeDTO)).thenThrow(new AuthenticationException("Authentication failed")); 59 | 60 | // When & Then 61 | assertThrows(AuthenticationException.class, () -> exportService.exportData(timeframeDTO)); 62 | verifyNoInteractions(fddbParserService); 63 | } 64 | 65 | @Test 66 | @SneakyThrows 67 | void exportData_whenParseExceptionThrown_shouldThrowException() { 68 | // Given 69 | TimeframeDTO timeframeDTO = new TimeframeDTO(1625097600L, 1625184000L); 70 | String mockResponse = "invalid response data"; 71 | when(fddbAdapter.retrieveDataToTimeframe(timeframeDTO)).thenReturn(mockResponse); 72 | when(fddbParserService.parseDiary(mockResponse)).thenThrow(new ParseException("Parsing failed")); 73 | 74 | // When & Then 75 | assertThrows(ParseException.class, () -> exportService.exportData(timeframeDTO)); 76 | } 77 | 78 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/dto/StatsDTO.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.dto; 2 | 3 | import io.swagger.v3.oas.annotations.media.Schema; 4 | import lombok.AllArgsConstructor; 5 | import lombok.Builder; 6 | import lombok.Data; 7 | import lombok.NoArgsConstructor; 8 | 9 | import java.time.LocalDate; 10 | 11 | @Data 12 | @Builder 13 | @NoArgsConstructor 14 | @AllArgsConstructor 15 | @Schema(description = "Statistics about FDDB data entries") 16 | public class StatsDTO { 17 | @Schema(description = "Total number of entries in the database", example = "365") 18 | long amountEntries; 19 | 20 | @Schema(description = "Date of the first entry", example = "2024-01-01") 21 | LocalDate firstEntryDate; 22 | 23 | @Schema(description = "Percentage of days with entries", example = "95.5") 24 | double entryPercentage; 25 | 26 | @Schema(description = "Number of unique products consumed", example = "150") 27 | long uniqueProducts; 28 | 29 | @Schema(description = "Average nutritional values across all entries") 30 | Averages averageTotals; 31 | 32 | @Schema(description = "Day with highest calories") 33 | DayStats highestCaloriesDay; 34 | 35 | @Schema(description = "Day with highest fat intake") 36 | DayStats highestFatDay; 37 | 38 | @Schema(description = "Day with highest carbs intake") 39 | DayStats highestCarbsDay; 40 | 41 | @Schema(description = "Day with highest protein intake") 42 | DayStats highestProteinDay; 43 | 44 | @Schema(description = "Day with highest fibre intake") 45 | DayStats highestFibreDay; 46 | 47 | @Schema(description = "Day with highest sugar intake") 48 | DayStats highestSugarDay; 49 | 50 | @Schema(description = "Most recent day with no entry (excluding today). Returns date or 'only available with MongoDB' when MongoDB is not configured", example = "2024-12-20") 51 | Object mostRecentMissingDay; 52 | 53 | @Data 54 | @Builder 55 | @NoArgsConstructor 56 | @AllArgsConstructor 57 | @Schema(description = "Average nutritional values") 58 | public static class Averages { 59 | @Schema(description = "Average calories per day", example = "2000.5") 60 | double avgTotalCalories; 61 | 62 | @Schema(description = "Average fat per day in grams", example = "65.3") 63 | double avgTotalFat; 64 | 65 | @Schema(description = "Average carbs per day in grams", example = "250.2") 66 | double avgTotalCarbs; 67 | 68 | @Schema(description = "Average sugar per day in grams", example = "50.1") 69 | double avgTotalSugar; 70 | 71 | @Schema(description = "Average protein per day in grams", example = "80.4") 72 | double avgTotalProtein; 73 | 74 | @Schema(description = "Average fibre per day in grams", example = "25.6") 75 | double avgTotalFibre; 76 | } 77 | 78 | @Data 79 | @Builder 80 | @NoArgsConstructor 81 | @AllArgsConstructor 82 | @Schema(description = "Statistics for a specific day") 83 | public static class DayStats { 84 | @Schema(description = "Date of the entry", example = "2024-12-22") 85 | LocalDate date; 86 | 87 | @Schema(description = "Total value for that day", example = "2500.5") 88 | double total; 89 | } 90 | } 91 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v1/FddbDataResourceIT.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v1; 2 | 3 | import com.fasterxml.jackson.databind.ObjectMapper; 4 | import dev.itobey.adapter.api.fddb.exporter.config.TestConfig; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.PersistenceService; 7 | import lombok.SneakyThrows; 8 | import org.junit.jupiter.api.Tag; 9 | import org.junit.jupiter.api.Test; 10 | import org.junit.jupiter.params.ParameterizedTest; 11 | import org.junit.jupiter.params.provider.CsvSource; 12 | import org.springframework.beans.factory.annotation.Autowired; 13 | import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc; 14 | import org.springframework.boot.test.context.SpringBootTest; 15 | import org.springframework.context.annotation.Import; 16 | import org.springframework.http.MediaType; 17 | import org.springframework.test.context.ActiveProfiles; 18 | import org.springframework.test.web.servlet.MockMvc; 19 | 20 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; 21 | import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; 22 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; 23 | import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; 24 | 25 | /** 26 | * Integration test for deprecated v1 API compatibility. 27 | * 28 | * @deprecated Tests deprecated v1 API endpoints. Create new tests for v2 controllers. 29 | */ 30 | @Deprecated 31 | @SpringBootTest 32 | @AutoConfigureMockMvc 33 | @ActiveProfiles("test") 34 | @Import(TestConfig.class) 35 | @Tag("v1-compat") 36 | class FddbDataResourceIT { 37 | 38 | @Autowired 39 | private MockMvc mockMvc; 40 | @Autowired 41 | private ObjectMapper objectMapper; 42 | @Autowired 43 | private PersistenceService persistenceService; 44 | 45 | @Test 46 | @SneakyThrows 47 | void exportForTimerange_fromDateAfterToDate_shouldThrowException() { 48 | DateRangeDTO invalidBatchExport = DateRangeDTO.builder() 49 | .fromDate("2023-01-01") 50 | .toDate("2022-01-01") 51 | .build(); 52 | 53 | mockMvc.perform(post("/api/v1/fddbdata") 54 | .contentType(MediaType.APPLICATION_JSON) 55 | .content(objectMapper.writeValueAsString(invalidBatchExport))) 56 | .andExpect(status().isBadRequest()) 57 | .andExpect(jsonPath("$.dateTimeError") 58 | .value("The 'from' date cannot be after the 'to' date")); 59 | } 60 | 61 | @ParameterizedTest 62 | @CsvSource({ 63 | "0, true", 64 | "0, false", 65 | "366, true", 66 | "366, false" 67 | }) 68 | @SneakyThrows 69 | void exportForDaysBack_whenDayOutsidePermittedRange_shouldThrowException(int days, boolean includeToday) { 70 | mockMvc.perform(get("/api/v1/fddbdata/export?days=" + days + "&includeToday=" + includeToday)) 71 | .andExpect(status().isBadRequest()) 72 | .andExpect(jsonPath("$.dateTimeError") 73 | .value("Days back must be between 1 and 365")); 74 | } 75 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataQueryResourceV2Test.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.FddbDataDTO; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.ProductWithDateDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 6 | import org.junit.jupiter.api.Tag; 7 | import org.junit.jupiter.api.Test; 8 | import org.junit.jupiter.api.extension.ExtendWith; 9 | import org.mockito.InjectMocks; 10 | import org.mockito.Mock; 11 | import org.mockito.junit.jupiter.MockitoExtension; 12 | import org.springframework.http.HttpStatus; 13 | import org.springframework.http.ResponseEntity; 14 | 15 | import java.util.Arrays; 16 | import java.util.List; 17 | import java.util.Optional; 18 | 19 | import static org.junit.jupiter.api.Assertions.assertEquals; 20 | import static org.mockito.Mockito.when; 21 | 22 | /** 23 | * Unit test for v2 Query API. 24 | */ 25 | @ExtendWith(MockitoExtension.class) 26 | @Tag("v2") 27 | class FddbDataQueryResourceV2Test { 28 | 29 | @Mock 30 | private FddbDataService fddbDataService; 31 | 32 | @InjectMocks 33 | private FddbDataQueryResourceV2 fddbDataQueryResourceV2; 34 | 35 | @Test 36 | void testFindAllEntries() { 37 | List mockData = Arrays.asList(new FddbDataDTO(), new FddbDataDTO()); 38 | when(fddbDataService.findAllEntries()).thenReturn(mockData); 39 | 40 | ResponseEntity> response = fddbDataQueryResourceV2.findAllEntries(); 41 | 42 | assertEquals(HttpStatus.OK, response.getStatusCode()); 43 | assertEquals(mockData, response.getBody()); 44 | } 45 | 46 | @Test 47 | void testFindByDate_ValidDate() { 48 | String validDate = "2023-01-01"; 49 | FddbDataDTO mockData = new FddbDataDTO(); 50 | when(fddbDataService.findByDate(validDate)).thenReturn(Optional.of(mockData)); 51 | 52 | ResponseEntity response = fddbDataQueryResourceV2.findByDate(validDate); 53 | 54 | assertEquals(HttpStatus.OK, response.getStatusCode()); 55 | assertEquals(Optional.of(mockData), response.getBody()); 56 | } 57 | 58 | @Test 59 | void testFindByDate_InvalidDate() { 60 | String invalidDate = "2023-1-1"; 61 | 62 | ResponseEntity response = fddbDataQueryResourceV2.findByDate(invalidDate); 63 | 64 | assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode()); 65 | assertEquals("Date must be in the format YYYY-MM-DD", response.getBody()); 66 | } 67 | 68 | @Test 69 | void testFindByDate_NotFound() { 70 | String validDate = "2023-01-01"; 71 | when(fddbDataService.findByDate(validDate)).thenReturn(Optional.empty()); 72 | 73 | ResponseEntity response = fddbDataQueryResourceV2.findByDate(validDate); 74 | 75 | assertEquals(HttpStatus.NOT_FOUND, response.getStatusCode()); 76 | } 77 | 78 | @Test 79 | void testFindByProduct() { 80 | String productName = "TestProduct"; 81 | List mockData = Arrays.asList(new ProductWithDateDTO(), new ProductWithDateDTO()); 82 | when(fddbDataService.findByProduct(productName)).thenReturn(mockData); 83 | 84 | ResponseEntity> response = fddbDataQueryResourceV2.findByProduct(productName); 85 | 86 | assertEquals(HttpStatus.OK, response.getStatusCode()); 87 | assertEquals(mockData, response.getBody()); 88 | } 89 | } 90 | 91 | -------------------------------------------------------------------------------- /docs/visualization/flutter-app.md: -------------------------------------------------------------------------------- 1 | # Flutter App 2 | 3 | The FDDB Exporter App is a Flutter-based mobile application that serves as a user-friendly frontend for 4 | the [FDDB Exporter](https://github.com/itobey/fddb-exporter) backend system. This companion app provides convenient 5 | access to the backend API endpoints through an intuitive mobile interface. 6 | 7 | ## Overview 8 | 9 | The Flutter app is available on [GitHub](https://github.com/itobey/fddb-exporter-app) and provides a complete mobile 10 | interface for interacting with the FDDB Exporter backend. Without the self-hosted backend, this app will not function 11 | properly as it relies on those API endpoints. 12 | 13 | ## Features 14 | 15 | The app supports the following key features: 16 | 17 | - **Export Data**: Fetch nutrition data by days-back or by a specific date range 18 | - **Daily Search**: Retrieve and view daily nutrition information by date 19 | - **Product Search**: Find specific products by name 20 | - **Statistics**: View aggregated nutrition statistics 21 | - **Correlation Analysis**: Explore correlations between nutritional data using filters for keywords, date ranges, and 22 | occurrences 23 | - **Settings**: Configure the API endpoint connection to your backend 24 | 25 | ## Screenshots 26 | 27 | | Daily Search | Product Search | Correlation Analysis | 28 | |:--------------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------:| 29 | | ![Daily Search](https://raw.githubusercontent.com/itobey/fddb-exporter-app/refs/heads/master/docs/images/daily-search.jpg) | ![Product Search](https://raw.githubusercontent.com/itobey/fddb-exporter-app/refs/heads/master/docs/images/product-search.jpg) | ![Correlation Output](https://raw.githubusercontent.com/itobey/fddb-exporter-app/refs/heads/master/docs/images/correlation-output.jpg) | 30 | 31 | ## Installation 32 | 33 | You can either: 34 | 35 | - Build the application yourself following the instructions in the GitHub repository 36 | - Download a prebuilt APK from the [GitHub releases page](https://github.com/itobey/fddb-exporter-app/releases) 37 | 38 | ## Configuration 39 | 40 | The app requires minimal configuration: 41 | 42 | - The API endpoint is configurable via the in-app Settings screen 43 | - Default endpoint is set to `http://localhost:8080` 44 | - The configuration is stored using SharedPreferences 45 | 46 | ## Requirements 47 | 48 | - Flutter SDK: ^3.5.1 49 | - Dart SDK: ^3.5.1 50 | - A running instance of the [FDDB Exporter](https://github.com/itobey/fddb-exporter) backend 51 | 52 | ## Technical Details 53 | 54 | The application is built with Flutter, making it compatible with both Android and iOS platforms. It communicates with 55 | the backend REST API to fetch and display data, offering a complete mobile experience for managing your nutrition data 56 | from the FDDB Exporter system. 57 | 58 | For more information about the backend system, refer to 59 | the [FDDB Exporter GitHub repository](https://github.com/itobey/fddb-exporter). -------------------------------------------------------------------------------- /docs/.vitepress/config.js: -------------------------------------------------------------------------------- 1 | 2 | const ref = "menu"; 3 | const base = "/fddb-exporter/" 4 | 5 | export default { 6 | // site-level options 7 | title: 'FDDB Exporter', 8 | description: 'Export data from fddb.info to a database and query your data', 9 | 10 | base: base, 11 | themeConfig: { 12 | lastUpdated: true, 13 | appRef: ref, 14 | outline: [2, 4], 15 | search: { 16 | provider: "local" 17 | }, 18 | footer: { 19 | message: 20 | 'Released under the MIT + Commons Clause License.', 21 | }, 22 | nav: [ 23 | { 24 | text: ref, 25 | items: [ 26 | { text: "Changelog", link: "https://github.com/itobey/fddb-exporter/blob/master/CHANGELOG.md" }, 27 | { text: "Issues", link: "https://github.com/itobey/fddb-exporter/issues" } 28 | ] 29 | } 30 | ], 31 | socialLinks: [ 32 | { 33 | icon: { 34 | svg: '' 35 | }, 36 | link: "https://github.com/itobey/fddb-exporter" 37 | } 38 | ], 39 | sidebar: [ 40 | { 41 | text: "Introduction", 42 | collapsible: true, 43 | items: [ 44 | { text: "What is FDDB Exporter?", link: "/introduction/index.html" }, 45 | { text: "Getting started", link: "/introduction/getting-started.html" } 46 | ] 47 | }, 48 | { 49 | text: "Details", 50 | collapsible: true, 51 | items: [ 52 | { text: "Configuration", link: "/details/configuration.html" }, 53 | { text: "Docker", link: "/details/docker.html" }, 54 | { text: "Helm", link: "/details/helm.html" }, 55 | { text: "Exports and data", link: "/details/exports-and-data.html" }, 56 | { text: "Persistence", link: "/details/persistence.html" }, 57 | { text: "REST API", link: "/details/rest-api.html" }, 58 | { text: "Correlation API", link: "/details/correlation-api.html" } 59 | ] 60 | }, 61 | { 62 | text: "Visualization", 63 | collapsible: true, 64 | items: [ 65 | { text: "Grafana Dashboard", link: "/visualization/grafana-dashboard.html" }, 66 | { text: "Flutter App", link: "/visualization/flutter-app.html" } 67 | ] 68 | } 69 | ] 70 | }, 71 | ignoreDeadLinks: [ 72 | // ignore all localhost links 73 | /^https?:\/\/localhost/, 74 | ], 75 | cleanUrls: true 76 | } 77 | -------------------------------------------------------------------------------- /docs/resources/example-document.bson: -------------------------------------------------------------------------------- 1 | { 2 | "_id": ObjectId("66d18658bc73187ea859f67d"), 3 | "date": ISODate("2024-08-28T22:00:00.000+0000"), 4 | "products": [ 5 | { 6 | "name": "Ciabatta, mit Sauerteig", 7 | "amount": "100 g", 8 | "calories": 231.0, 9 | "fat": 3.3, 10 | "carbs": 42.0, 11 | "protein": 4.6, 12 | "link": "/db/en/food/schaer_ciabatta_mit_sauerteig/index.html" 13 | }, 14 | { 15 | "name": "Cevapcici", 16 | "amount": "400 g", 17 | "calories": 936.0, 18 | "fat": 71.2, 19 | "carbs": 2.0, 20 | "protein": 70.8, 21 | "link": "/db/en/food/kaufland_cevapcici/index.html" 22 | }, 23 | { 24 | "name": "Erdnussmus, creamy", 25 | "amount": "25 ml", 26 | "calories": 155.0, 27 | "fat": 12.5, 28 | "carbs": 2.2, 29 | "protein": 7.0, 30 | "link": "/db/en/food/kaufland_erdnussmus_creamy/index.html" 31 | }, 32 | { 33 | "name": "Helles Brot, glutenfrei", 34 | "amount": "100 g", 35 | "calories": 211.0, 36 | "fat": 2.7, 37 | "carbs": 41.0, 38 | "protein": 2.4, 39 | "link": "/db/en/food/schaer_helles_brot_glutenfrei/index.html" 40 | }, 41 | { 42 | "name": "Marmelade, Durchschnitt", 43 | "amount": "15 g", 44 | "calories": 43.0, 45 | "fat": 0.0, 46 | "carbs": 10.4, 47 | "protein": 0.0, 48 | "link": "/db/en/food/durchschnittswert_marmelade_durchschnitt/index.html" 49 | }, 50 | { 51 | "name": "Vollmilch, 3,5% Fett", 52 | "amount": "40 ml", 53 | "calories": 26.0, 54 | "fat": 1.4, 55 | "carbs": 1.9, 56 | "protein": 1.2, 57 | "link": "/db/en/food/molkerei_vollmilch_3_5prozent_fett/index.html" 58 | }, 59 | { 60 | "name": "Paprikalyoner", 61 | "amount": "39 g", 62 | "calories": 110.0, 63 | "fat": 10.1, 64 | "carbs": 0.4, 65 | "protein": 4.3, 66 | "link": "/db/en/food/real_paprikalyoner_727510/index.html" 67 | }, 68 | { 69 | "name": "Laugenbrötchen", 70 | "amount": "72 g", 71 | "calories": 205.0, 72 | "fat": 1.2, 73 | "carbs": 40.8, 74 | "protein": 6.9, 75 | "link": "/db/en/food/baecker_laugenbroetchen/index.html" 76 | }, 77 | { 78 | "name": "Gemischter Salat, Salat Dressing", 79 | "amount": "100 g", 80 | "calories": 164.0, 81 | "fat": 14.0, 82 | "carbs": 6.9, 83 | "protein": 1.1, 84 | "link": "/db/en/food/lidl_gemischter_salat/index.html" 85 | }, 86 | { 87 | "name": "pilzragout", 88 | "amount": "150 g", 89 | "calories": 142.0, 90 | "fat": 4.7, 91 | "carbs": 20.4, 92 | "protein": 4.4, 93 | "link": "/db/en/food/klueh_pilzragout/index.html" 94 | }, 95 | { 96 | "name": "Gnocchi", 97 | "amount": "250 g", 98 | "calories": 375.0, 99 | "fat": 5.5, 100 | "carbs": 76.3, 101 | "protein": 9.5, 102 | "link": "/db/en/food/durchschnittswert_gnocchi/index.html" 103 | }, 104 | { 105 | "name": "Butter", 106 | "amount": "5 g", 107 | "calories": 37.0, 108 | "fat": 4.2, 109 | "carbs": 0.0, 110 | "protein": 0.0, 111 | "link": "/db/en/food/durchschnittswert_butter/index.html" 112 | } 113 | ], 114 | "totalCalories": 2633.0, 115 | "totalFat": 130.8, 116 | "totalCarbs": 244.2, 117 | "totalSugar": 33.8, 118 | "totalProtein": 112.2, 119 | "totalFibre": 25.6, 120 | "_class": "dev.itobey.adapter.api.fddb.exporter.domain.FddbData" 121 | } -------------------------------------------------------------------------------- /src/test/resources/domain/fddbdata-2024-08-29.json: -------------------------------------------------------------------------------- 1 | { 2 | "date": "2024-08-29T00:00:00.000", 3 | "products": [ 4 | { 5 | "name": "Ciabatta, mit Sauerteig", 6 | "amount": "100 g", 7 | "calories": 231.0, 8 | "fat": 3.3, 9 | "carbs": 42.0, 10 | "protein": 4.6, 11 | "link": "https://fddb.info/db/en/food/schaer_ciabatta_mit_sauerteig/index.html" 12 | }, 13 | { 14 | "name": "Cevapcici", 15 | "amount": "400 g", 16 | "calories": 936.0, 17 | "fat": 71.2, 18 | "carbs": 2.0, 19 | "protein": 70.8, 20 | "link": "https://fddb.info/db/en/food/kaufland_cevapcici/index.html" 21 | }, 22 | { 23 | "name": "Erdnussmus, creamy", 24 | "amount": "25 ml", 25 | "calories": 155.0, 26 | "fat": 12.5, 27 | "carbs": 2.2, 28 | "protein": 7.0, 29 | "link": "https://fddb.info/db/en/food/kaufland_erdnussmus_creamy/index.html" 30 | }, 31 | { 32 | "name": "Helles Brot, glutenfrei", 33 | "amount": "100 g", 34 | "calories": 211.0, 35 | "fat": 2.7, 36 | "carbs": 41.0, 37 | "protein": 2.4, 38 | "link": "https://fddb.info/db/en/food/schaer_helles_brot_glutenfrei/index.html" 39 | }, 40 | { 41 | "name": "Marmelade, Durchschnitt", 42 | "amount": "15 g", 43 | "calories": 43.0, 44 | "fat": 0.0, 45 | "carbs": 10.4, 46 | "protein": 0.0, 47 | "link": "https://fddb.info/db/en/food/durchschnittswert_marmelade_durchschnitt/index.html" 48 | }, 49 | { 50 | "name": "Vollmilch, 3,5% Fett", 51 | "amount": "40 ml", 52 | "calories": 26.0, 53 | "fat": 1.4, 54 | "carbs": 1.9, 55 | "protein": 1.2, 56 | "link": "https://fddb.info/db/en/food/molkerei_vollmilch_3_5prozent_fett/index.html" 57 | }, 58 | { 59 | "name": "Paprikalyoner", 60 | "amount": "39 g", 61 | "calories": 110.0, 62 | "fat": 10.1, 63 | "carbs": 0.4, 64 | "protein": 4.3, 65 | "link": "https://fddb.info/db/en/food/real_paprikalyoner_727510/index.html" 66 | }, 67 | { 68 | "name": "Laugenbrötchen", 69 | "amount": "72 g", 70 | "calories": 205.0, 71 | "fat": 1.2, 72 | "carbs": 40.8, 73 | "protein": 6.9, 74 | "link": "https://fddb.info/db/en/food/baecker_laugenbroetchen/index.html" 75 | }, 76 | { 77 | "name": "Gemischter Salat, Salat Dressing", 78 | "amount": "100 g", 79 | "calories": 164.0, 80 | "fat": 14.0, 81 | "carbs": 6.9, 82 | "protein": 1.1, 83 | "link": "https://fddb.info/db/en/food/lidl_gemischter_salat/index.html" 84 | }, 85 | { 86 | "name": "pilzragout", 87 | "amount": "150 g", 88 | "calories": 142.0, 89 | "fat": 4.7, 90 | "carbs": 20.4, 91 | "protein": 4.4, 92 | "link": "https://fddb.info/db/en/food/klueh_pilzragout/index.html" 93 | }, 94 | { 95 | "name": "Gnocchi", 96 | "amount": "250 g", 97 | "calories": 375.0, 98 | "fat": 5.5, 99 | "carbs": 76.3, 100 | "protein": 9.5, 101 | "link": "https://fddb.info/db/en/food/durchschnittswert_gnocchi/index.html" 102 | }, 103 | { 104 | "name": "Butter", 105 | "amount": "5 g", 106 | "calories": 37.0, 107 | "fat": 4.2, 108 | "carbs": 0.0, 109 | "protein": 0.0, 110 | "link": "https://fddb.info/db/en/food/durchschnittswert_butter/index.html" 111 | } 112 | ], 113 | "totalCalories": 2633.0, 114 | "totalFat": 130.8, 115 | "totalCarbs": 244.2, 116 | "totalSugar": 33.8, 117 | "totalProtein": 112.2, 118 | "totalFibre": 25.6 119 | } 120 | 121 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/persistence/InfluxDBService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service.persistence; 2 | 3 | import com.influxdb.client.InfluxDBClient; 4 | import com.influxdb.client.QueryApi; 5 | import com.influxdb.client.WriteApi; 6 | import com.influxdb.client.domain.WritePrecision; 7 | import com.influxdb.client.write.Point; 8 | import com.influxdb.query.FluxTable; 9 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 10 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 11 | import lombok.RequiredArgsConstructor; 12 | import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; 13 | import org.springframework.stereotype.Service; 14 | 15 | import java.time.Instant; 16 | import java.time.ZoneId; 17 | import java.util.List; 18 | import java.util.Map; 19 | 20 | /** 21 | * Service class for persisting data to InfluxDB. 22 | */ 23 | @Service 24 | @RequiredArgsConstructor 25 | @ConditionalOnProperty(name = "fddb-exporter.persistence.influxdb.enabled", havingValue = "true") 26 | public class InfluxDBService { 27 | 28 | public static final String DAILY_TOTALS = "dailyTotals"; 29 | 30 | private final InfluxDBClient influxDBClient; 31 | 32 | private final FddbExporterProperties properties; 33 | 34 | /** 35 | * Saves the given FddbData object to InfluxDB - but only uses the total values of the FddbData object. 36 | * The diary entries are not saved, because InfluxDB is not a document database. 37 | * 38 | * @param fddbData The FddbData object containing the data to be saved. 39 | */ 40 | public void saveToInfluxDB(FddbData fddbData) { 41 | Instant time = fddbData.getDate().atStartOfDay(ZoneId.systemDefault()).toInstant(); 42 | Map metrics = Map.of( 43 | "calories", fddbData.getTotalCalories(), 44 | "fat", fddbData.getTotalFat(), 45 | "carbs", fddbData.getTotalCarbs(), 46 | "sugar", fddbData.getTotalSugar(), 47 | "fibre", fddbData.getTotalFibre(), 48 | "protein", fddbData.getTotalProtein() 49 | ); 50 | 51 | metrics.forEach((metric, value) -> 52 | writeData(metric, value, time) 53 | ); 54 | } 55 | 56 | /** 57 | * Writes data as a Point to InfluxDB. 58 | * 59 | * @param field The field to be written. 60 | * @param value The value to be written. 61 | * @param time The time of the data point. 62 | */ 63 | public void writeData(String field, double value, Instant time) { 64 | try (WriteApi writeApi = influxDBClient.makeWriteApi()) { 65 | Point point = Point.measurement(DAILY_TOTALS) 66 | .addField(field, value) 67 | .time(time, WritePrecision.NS); 68 | 69 | writeApi.writePoint(point); 70 | } 71 | } 72 | 73 | /** 74 | * Returns the amount of data points in the database, similar to the "count" function in SQL. 75 | * 76 | * @return the amount of data points in the database. 77 | */ 78 | public long getDataPointCount() { 79 | QueryApi queryApi = influxDBClient.getQueryApi(); 80 | String flux = "from(bucket:\"" + properties.getInfluxdb().getBucket() + "\")" + 81 | " |> range(start: 0)" + 82 | " |> filter(fn: (r) => r._measurement == \"" + DAILY_TOTALS + "\")" + 83 | " |> count()"; 84 | List result = queryApi.query(flux); 85 | 86 | return result.stream() 87 | .flatMap(table -> table.getRecords().stream()) 88 | .findFirst() 89 | .map(record -> record.getValue() instanceof Long ? (Long) record.getValue() : 0L) 90 | .orElse(0L); 91 | } 92 | 93 | } 94 | 95 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/telemetry/TelemetryService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service.telemetry; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.adapter.TelemetryApi; 4 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 5 | import dev.itobey.adapter.api.fddb.exporter.domain.ExecutionMode; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.telemetry.TelemetryDto; 7 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.PersistenceService; 8 | import jakarta.annotation.PostConstruct; 9 | import lombok.RequiredArgsConstructor; 10 | import lombok.extern.slf4j.Slf4j; 11 | import org.springframework.boot.info.BuildProperties; 12 | import org.springframework.stereotype.Service; 13 | 14 | import java.nio.charset.StandardCharsets; 15 | import java.security.MessageDigest; 16 | import java.security.NoSuchAlgorithmException; 17 | 18 | /** 19 | * This service is used to send telemetry data. No personal data is sent. 20 | * Only the mail hash is sent along with the document count and the execution mode to determine how the exporter is used. 21 | * See README.md for more information. 22 | */ 23 | @Service 24 | @RequiredArgsConstructor 25 | @Slf4j 26 | public class TelemetryService { 27 | 28 | private final TelemetryApi telemetryApi; 29 | private final PersistenceService persistenceService; 30 | private final EnvironmentDetector environmentDetector; 31 | private final BuildProperties buildProperties; 32 | private final FddbExporterProperties properties; 33 | 34 | public void sendTelemetryData() { 35 | ExecutionMode executionMode = environmentDetector.getExecutionMode(); 36 | String mailHash = hashMail(properties.getFddb().getUsername()); 37 | TelemetryDto telemetryDto = new TelemetryDto(); 38 | boolean mongoDbEnabled = properties.getPersistence().getMongodb().isEnabled(); 39 | if (mongoDbEnabled) { 40 | long documentCount = persistenceService.countAllEntries(); 41 | telemetryDto.setDocumentCount(documentCount); 42 | } 43 | boolean influxDbEnabled = properties.getPersistence().getInfluxdb().isEnabled(); 44 | if (influxDbEnabled) { 45 | long pointCount = persistenceService.countAllInfluxDbPoints(); 46 | telemetryDto.setPointCount(pointCount); 47 | } 48 | telemetryDto.setMailHash(mailHash); 49 | telemetryDto.setMongodbEnabled(mongoDbEnabled); 50 | telemetryDto.setInfluxdbEnabled(influxDbEnabled); 51 | telemetryDto.setExecutionMode(executionMode); 52 | telemetryDto.setAppVersion(buildProperties.getVersion()); 53 | log.debug("sending telemetry data: {}", telemetryDto); 54 | telemetryApi.sendTelemetryData(telemetryDto); 55 | } 56 | 57 | @PostConstruct 58 | private void init() { 59 | log.debug("sending telemetry data on startup"); 60 | sendTelemetryData(); 61 | } 62 | 63 | private String hashMail(String mail) { 64 | try { 65 | MessageDigest digest = MessageDigest.getInstance("SHA-256"); 66 | byte[] encodedHash = digest.digest(mail.getBytes(StandardCharsets.UTF_8)); 67 | return bytesToHex(encodedHash); 68 | } catch (NoSuchAlgorithmException e) { 69 | log.error("SHA-256 algorithm not found", e); 70 | return ""; 71 | } 72 | } 73 | 74 | private static String bytesToHex(byte[] hash) { 75 | StringBuilder hexString = new StringBuilder(2 * hash.length); 76 | for (byte b : hash) { 77 | String hex = Integer.toHexString(0xff & b); 78 | if (hex.length() == 1) { 79 | hexString.append('0'); 80 | } 81 | hexString.append(hex); 82 | } 83 | return hexString.toString(); 84 | } 85 | 86 | } 87 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/persistence/PersistenceService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service.persistence; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 5 | import dev.itobey.adapter.api.fddb.exporter.domain.projection.ProductWithDate; 6 | import dev.itobey.adapter.api.fddb.exporter.mapper.FddbDataMapper; 7 | import dev.itobey.adapter.api.fddb.exporter.repository.FddbDataRepository; 8 | import lombok.extern.slf4j.Slf4j; 9 | import org.springframework.beans.factory.annotation.Autowired; 10 | import org.springframework.stereotype.Service; 11 | 12 | import java.time.LocalDate; 13 | import java.util.List; 14 | import java.util.Optional; 15 | 16 | /** 17 | * Provides persistence-related services for managing {@link FddbData} objects. 18 | *

19 | * This service class is responsible for saving and retrieving {@link FddbData} objects to/from the database. 20 | * It provides methods to find the first {@link FddbData} object by a given date, save a new {@link FddbData} object, 21 | * and search for {@link FddbData} objects by product name. 22 | */ 23 | @Service 24 | @Slf4j 25 | public class PersistenceService { 26 | 27 | @Autowired 28 | private FddbDataMapper fddbDataMapper; 29 | @Autowired(required = false) 30 | private FddbDataRepository fddbDataRepository; 31 | @Autowired(required = false) 32 | private MongoDBService mongoDBService; 33 | @Autowired(required = false) 34 | private InfluxDBService influxDBService; 35 | @Autowired 36 | private FddbExporterProperties properties; 37 | 38 | public long countAllEntries() { 39 | return mongoDBService.countAllEntries(); 40 | } 41 | 42 | public long countAllInfluxDbPoints() { 43 | return influxDBService.getDataPointCount(); 44 | } 45 | 46 | public List findAllEntries() { 47 | return mongoDBService.findAllEntries(); 48 | } 49 | 50 | public List findByProduct(String name) { 51 | return mongoDBService.findByProduct(name); 52 | } 53 | 54 | public Optional findByDate(LocalDate date) { 55 | return mongoDBService.findByDate(date); 56 | } 57 | 58 | public void saveOrUpdate(FddbData dataToPersist) { 59 | saveToMongoDbIfEnabled(dataToPersist); 60 | saveToInfluxDbIfEnabled(dataToPersist); 61 | } 62 | 63 | private void saveToInfluxDbIfEnabled(FddbData dataToPersist) { 64 | if (properties.getPersistence().getInfluxdb().isEnabled()) { 65 | log.info("writing point to influxdb: {}", dataToPersist.toDailyTotalsString()); 66 | influxDBService.saveToInfluxDB(dataToPersist); 67 | } 68 | } 69 | 70 | private void saveToMongoDbIfEnabled(FddbData dataToPersist) { 71 | if (properties.getPersistence().getMongodb().isEnabled()) { 72 | Optional optionalOfDbEntry = mongoDBService.findByDate(dataToPersist.getDate()); 73 | if (optionalOfDbEntry.isPresent()) { 74 | FddbData existingFddbData = optionalOfDbEntry.get(); 75 | log.debug("updating existing database entry for {}", dataToPersist.getDate()); 76 | updateDataIfNotIdentical(dataToPersist, existingFddbData); 77 | } else { 78 | FddbData savedEntry = fddbDataRepository.save(dataToPersist); 79 | log.info("created entry in database: {}", savedEntry); 80 | } 81 | } 82 | } 83 | 84 | private void updateDataIfNotIdentical(FddbData dataToPersist, FddbData existingFddbData) { 85 | if (!dataToPersist.equals(existingFddbData)) { 86 | fddbDataMapper.updateFddbData(existingFddbData, dataToPersist); 87 | FddbData updatedEntry = fddbDataRepository.save(existingFddbData); 88 | log.info("updated entry: {}", updatedEntry); 89 | } else { 90 | log.info("entry already exported, skipping: {}", dataToPersist); 91 | } 92 | } 93 | 94 | } 95 | -------------------------------------------------------------------------------- /docs/details/persistence.md: -------------------------------------------------------------------------------- 1 | # Persistence 2 | 3 | ## Persistence Layers 4 | 5 | The FDDB Exporter supports two persistence layers: MongoDB and InfluxDB. 6 | Either one can be enabled or disabled, while MongoDB is the default. You can also choose to use both at the same time. 7 | Only InfluxDB 2.x is supported, as the application uses the new InfluxDB 2.x API. 8 | 9 | Because of missing AVX instruction support of an older NUC model, MongoDB has been tested with the ancient version 10 | `4.4.13`. 11 | In principle, the application should work with any newer version of MongoDB. Integration-tests have been performed with 12 | MongoDB `8.0.11`. 13 | 14 | You can enable each persistence layer with an environment variable set to `true`. 15 | 16 | - `FDDB-EXPORTER_PERSISTENCE_INFLUXDB_ENABLED` 17 | - `FDDB-EXPORTER_PERSISTENCE_MONGODB_ENABLED` 18 | 19 | For further configuration, see the [configuration details](/details/configuration.md). 20 | 21 | If you enabled InfluxDB after already having exported data, you can use the [REST API](/details/rest-api.md) to migrate 22 | your data from MongoDB to InfluxDB, so you don't have to re-export your data. Depending on the size of your 23 | data, this may take a while. 24 | 25 | ## Data Structure 26 | 27 | Depending on the persistence layer chosen, the exported data is structured differently. MongoDB contains all 28 | data in a collection, while InfluxDB stores only daily totals for graphical representation in Grafana. 29 | 30 | ### MongoDB Collection 31 | 32 | The exported data is structured in a MongoDB collection, with each document representing a single diary entry. 33 | Sugar values of single products are not stored in the collection, because they are not part of the FDDB diary overview. 34 | This data could be retrieved from the product page, which would require a lot of additional requests. For this reason, 35 | this data is currently missing and only daily totals of sugar are stored. 36 | 37 | The collection contains the following fields - this is an abbreviated example. For a full example, see the 38 | [example data](https://github.com/itobey/fddb-exporter/blob/master/docs/resources/example-document.bson). 39 | 40 | ```json 41 | { 42 | "_id": ObjectId( 43 | "66d18658bc73187ea859f67d" 44 | ), 45 | "date": ISODate( 46 | "2024-08-28T22:00:00.000+0000" 47 | ), 48 | "products": [ 49 | { 50 | "name": "Cevapcici", 51 | "amount": "400 g", 52 | "calories": 936.0, 53 | "fat": 71.2, 54 | "carbs": 2.0, 55 | "protein": 70.8, 56 | "link": "/db/en/food/kaufland_cevapcici/index.html" 57 | }, 58 | ... 59 | ], 60 | "totalCalories": 2633.0, 61 | "totalFat": 130.8, 62 | "totalCarbs": 244.2, 63 | "totalSugar": 33.8, 64 | "totalProtein": 112.2, 65 | "totalFibre": 25.6 66 | } 67 | ``` 68 | 69 | ### InfluxDB Points 70 | 71 | The FDDB Exporter stores daily totals as measurement points in InfluxDB, which is ideal for time-series data 72 | visualization. Each day's nutritional values are stored as separate points with the following metrics: 73 | 74 | - calories 75 | - fat 76 | - carbs 77 | - sugar 78 | - fibre 79 | - protein 80 | 81 | This structure makes it particularly effective for creating time-based visualizations in tools like Grafana, where you 82 | can track trends and patterns in your nutritional data over time. 83 | 84 | ### Time and Date 85 | 86 | Both MongoDB and InfluxDB store timestamps at the beginning of each day (00:00:00) in UTC, derived from the configured 87 | timezone of the environment. For example, if your timezone is set to Europe/Berlin, a diary entry for January 15th will 88 | be stored with the timestamp "2024-01-14T22:00:00Z" (UTC). This consistent UTC timestamp handling ensures accurate data 89 | representation across both persistence layers while respecting local time zones for display purposes. For information on 90 | how to configure the timezone, see the [configuration details](/details/configuration.md). 91 | 92 | ### Querying Data 93 | 94 | The FDDB Exporter provides a REST API to query data. This is an easy way to retrieve data from the database in JSON 95 | format. For more information, see the [REST API](/details/rest-api.md). -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/StatsServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.StatsDTO; 5 | import org.bson.Document; 6 | import org.junit.jupiter.api.Test; 7 | import org.junit.jupiter.api.extension.ExtendWith; 8 | import org.mockito.InjectMocks; 9 | import org.mockito.Mock; 10 | import org.mockito.junit.jupiter.MockitoExtension; 11 | import org.springframework.data.mongodb.core.MongoTemplate; 12 | import org.springframework.data.mongodb.core.aggregation.Aggregation; 13 | import org.springframework.data.mongodb.core.aggregation.AggregationResults; 14 | import org.springframework.data.mongodb.core.query.Query; 15 | 16 | import java.time.LocalDate; 17 | import java.util.Collections; 18 | 19 | import static org.assertj.core.api.Assertions.assertThat; 20 | import static org.mockito.ArgumentMatchers.any; 21 | import static org.mockito.ArgumentMatchers.eq; 22 | import static org.mockito.Mockito.when; 23 | 24 | @ExtendWith(MockitoExtension.class) 25 | class StatsServiceTest { 26 | 27 | @Mock 28 | private MongoTemplate mongoTemplate; 29 | 30 | @InjectMocks 31 | private StatsService statsService; 32 | 33 | @Test 34 | void getStats_shouldGatherAllData() { 35 | // given 36 | when(mongoTemplate.count(any(Query.class), eq(StatsService.COLLECTION_NAME))).thenReturn(100L); 37 | 38 | FddbData mockData = new FddbData(); 39 | mockData.setDate(LocalDate.of(2023, 1, 1)); 40 | when(mongoTemplate.findOne(any(Query.class), eq(FddbData.class), eq(StatsService.COLLECTION_NAME))).thenReturn(mockData); 41 | 42 | StatsDTO.Averages mockAverages = StatsDTO.Averages.builder() 43 | .avgTotalCalories(2000) 44 | .avgTotalFat(70) 45 | .avgTotalCarbs(250) 46 | .avgTotalSugar(50) 47 | .avgTotalProtein(100) 48 | .avgTotalFibre(30) 49 | .build(); 50 | 51 | Document rawResults = new Document(); 52 | AggregationResults mockResults = new AggregationResults<>(Collections.singletonList(mockAverages), rawResults); 53 | when(mongoTemplate.aggregate(any(Aggregation.class), eq(StatsService.COLLECTION_NAME), eq(StatsDTO.Averages.class))) 54 | .thenReturn(mockResults); 55 | 56 | StatsDTO.DayStats mockDayStats = StatsDTO.DayStats.builder() 57 | .date(LocalDate.of(2023, 5, 1)) 58 | .total(3000) 59 | .build(); 60 | 61 | AggregationResults mockDayStatsResults = new AggregationResults<>(Collections.singletonList(mockDayStats), rawResults); 62 | when(mongoTemplate.aggregate(any(Aggregation.class), eq(StatsService.COLLECTION_NAME), eq(StatsDTO.DayStats.class))) 63 | .thenReturn(mockDayStatsResults); 64 | 65 | AggregationResults mockUniqueProductsResults = new AggregationResults<>( 66 | Collections.singletonList(new Document("uniqueCount", 42L)), rawResults); 67 | when(mongoTemplate.aggregate(any(Aggregation.class), eq(StatsService.COLLECTION_NAME), eq(Document.class))) 68 | .thenReturn(mockUniqueProductsResults); 69 | 70 | // when 71 | StatsDTO result = statsService.getStats(); 72 | 73 | // then 74 | assertThat(result).isNotNull(); 75 | assertThat(result.getAmountEntries()).isEqualTo(100L); 76 | assertThat(result.getFirstEntryDate()).isEqualTo(LocalDate.of(2023, 1, 1)); 77 | assertThat(result.getEntryPercentage()).isGreaterThan(0); 78 | assertThat(result.getUniqueProducts()).isEqualTo(42L); 79 | assertThat(result.getAverageTotals()).isEqualTo(mockAverages); 80 | assertThat(result.getHighestCaloriesDay()).isEqualTo(mockDayStats); 81 | assertThat(result.getHighestFatDay()).isEqualTo(mockDayStats); 82 | assertThat(result.getHighestCarbsDay()).isEqualTo(mockDayStats); 83 | assertThat(result.getHighestProteinDay()).isEqualTo(mockDayStats); 84 | assertThat(result.getHighestFibreDay()).isEqualTo(mockDayStats); 85 | assertThat(result.getHighestSugarDay()).isEqualTo(mockDayStats); 86 | assertThat(result.getMostRecentMissingDay()).isNotNull(); 87 | } 88 | 89 | } 90 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataStatsResourceV2.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.annotation.RequiresMongoDb; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.RollingAveragesDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.StatsDTO; 7 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 8 | import io.swagger.v3.oas.annotations.Operation; 9 | import io.swagger.v3.oas.annotations.media.Content; 10 | import io.swagger.v3.oas.annotations.media.Schema; 11 | import io.swagger.v3.oas.annotations.responses.ApiResponse; 12 | import io.swagger.v3.oas.annotations.responses.ApiResponses; 13 | import io.swagger.v3.oas.annotations.tags.Tag; 14 | import jakarta.validation.Valid; 15 | import lombok.RequiredArgsConstructor; 16 | import lombok.extern.slf4j.Slf4j; 17 | import org.springframework.http.ResponseEntity; 18 | import org.springframework.validation.annotation.Validated; 19 | import org.springframework.web.bind.annotation.GetMapping; 20 | import org.springframework.web.bind.annotation.RequestMapping; 21 | import org.springframework.web.bind.annotation.RestController; 22 | 23 | /** 24 | * V2 REST API for FDDB data statistics. 25 | *

26 | * Provides endpoints for: 27 | * - Retrieving overall statistics 28 | * - Calculating rolling averages for a date range 29 | *

30 | * The API endpoints are mapped to the "/api/v2/stats" path. 31 | * 32 | * @since 2.0.0 33 | */ 34 | @RestController 35 | @RequestMapping("/api/v2/stats") 36 | @Slf4j 37 | @Validated 38 | @RequiredArgsConstructor 39 | @Tag(name = "FDDB Data Statistics", description = "Statistics and analytics for FDDB data") 40 | public class FddbDataStatsResourceV2 { 41 | 42 | private final FddbDataService fddbDataService; 43 | 44 | /** 45 | * Get overall statistics for FDDB data. 46 | * 47 | * @return statistics including counts and other metrics 48 | */ 49 | @Operation(summary = "Get overall statistics", description = "Retrieves overall statistics for FDDB data") 50 | @ApiResponses(value = { 51 | @ApiResponse(responseCode = "200", description = "Statistics retrieved successfully", 52 | content = @Content(mediaType = "application/json", schema = @Schema(implementation = StatsDTO.class))), 53 | @ApiResponse(responseCode = "503", description = "MongoDB not available", content = @Content) 54 | }) 55 | @GetMapping 56 | @RequiresMongoDb 57 | public ResponseEntity getStats() { 58 | log.debug("V2: Retrieving FDDB data statistics"); 59 | return ResponseEntity.ok(fddbDataService.getStats()); 60 | } 61 | 62 | /** 63 | * Get rolling averages for a specified date range. 64 | *

65 | * Example: /api/v2/stats/averages?fromDate=2024-01-01&toDate=2024-01-31 66 | * 67 | * @param dateRangeDTO the date range for which to calculate averages (including both from and to dates) 68 | * @return rolling averages for the specified date range 69 | */ 70 | @Operation(summary = "Get rolling averages", description = "Calculate rolling averages for a specified date range") 71 | @ApiResponses(value = { 72 | @ApiResponse(responseCode = "200", description = "Rolling averages calculated successfully", 73 | content = @Content(mediaType = "application/json", schema = @Schema(implementation = RollingAveragesDTO.class))), 74 | @ApiResponse(responseCode = "400", description = "Invalid date range", content = @Content), 75 | @ApiResponse(responseCode = "503", description = "MongoDB not available", content = @Content) 76 | }) 77 | @GetMapping("/averages") 78 | @RequiresMongoDb 79 | public ResponseEntity getRollingAverages(@Valid DateRangeDTO dateRangeDTO) { 80 | log.debug("V2: Calculating rolling averages for date range: {} to {}", 81 | dateRangeDTO.getFromDate(), dateRangeDTO.getToDate()); 82 | try { 83 | RollingAveragesDTO result = fddbDataService.getRollingAverages(dateRangeDTO); 84 | return ResponseEntity.ok(result); 85 | } catch (IllegalArgumentException illegalArgumentException) { 86 | return ResponseEntity.badRequest().body(illegalArgumentException.getMessage()); 87 | } 88 | } 89 | } 90 | 91 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/rest/v2/FddbDataExportResourceV2.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v2; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.ExportResultDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 6 | import io.swagger.v3.oas.annotations.Operation; 7 | import io.swagger.v3.oas.annotations.Parameter; 8 | import io.swagger.v3.oas.annotations.media.Content; 9 | import io.swagger.v3.oas.annotations.media.Schema; 10 | import io.swagger.v3.oas.annotations.responses.ApiResponse; 11 | import io.swagger.v3.oas.annotations.responses.ApiResponses; 12 | import io.swagger.v3.oas.annotations.tags.Tag; 13 | import jakarta.validation.Valid; 14 | import lombok.RequiredArgsConstructor; 15 | import lombok.extern.slf4j.Slf4j; 16 | import org.springframework.http.ResponseEntity; 17 | import org.springframework.validation.annotation.Validated; 18 | import org.springframework.web.bind.annotation.*; 19 | 20 | /** 21 | * V2 REST API for exporting FDDB data. 22 | *

23 | * Provides endpoints for: 24 | * - Exporting data for a specified date range 25 | * - Exporting data for a specified number of days back 26 | *

27 | * The API endpoints are mapped to the "/api/v2/fddbdata/export" path. 28 | * 29 | * @since 2.0.0 30 | */ 31 | @RestController 32 | @RequestMapping("/api/v2/fddbdata") 33 | @Slf4j 34 | @Validated 35 | @RequiredArgsConstructor 36 | @Tag(name = "FDDB Data Export", description = "Export FDDB data for specified date ranges") 37 | public class FddbDataExportResourceV2 { 38 | 39 | private final FddbDataService fddbDataService; 40 | 41 | /** 42 | * Export data for all days contained in the given timeframe as a batch. 43 | * 44 | * @param dateRangeDTO the date range which should be exported 45 | * @return HTTP 200 and export result with saved and updated entries 46 | */ 47 | @Operation(summary = "Export data for a date range", description = "Export FDDB data for all days in the specified timeframe") 48 | @ApiResponses(value = { 49 | @ApiResponse(responseCode = "200", description = "Export completed successfully", 50 | content = @Content(mediaType = "application/json", schema = @Schema(implementation = ExportResultDTO.class))), 51 | @ApiResponse(responseCode = "400", description = "Invalid date range", content = @Content) 52 | }) 53 | @PostMapping 54 | public ResponseEntity exportForTimerange(@Valid @RequestBody DateRangeDTO dateRangeDTO) { 55 | log.info("V2: Exporting data for timerange: {} to {}", 56 | dateRangeDTO.getFromDate(), dateRangeDTO.getToDate()); 57 | ExportResultDTO result = fddbDataService.exportForTimerange(dateRangeDTO); 58 | return ResponseEntity.ok(result); 59 | } 60 | 61 | /** 62 | * Export data for the given amount of days back from today. 63 | *

64 | * Example: /api/v2/fddbdata/export?days=2&includeToday=true 65 | *

66 | * If includeToday is true, the current day will be exported as well. 67 | * 68 | * @param days the amount of days that should be exported 69 | * @param includeToday true, if the current day should be included as well 70 | * @return a list of saved and updated data points 71 | */ 72 | @Operation(summary = "Export data for recent days", description = "Export FDDB data for a specified number of days back from today") 73 | @ApiResponses(value = { 74 | @ApiResponse(responseCode = "200", description = "Export completed successfully", 75 | content = @Content(mediaType = "application/json", schema = @Schema(implementation = ExportResultDTO.class))) 76 | }) 77 | @GetMapping("/export") 78 | public ResponseEntity exportForDaysBack( 79 | @Parameter(description = "Number of days to export", example = "7", required = true) 80 | @RequestParam int days, 81 | @Parameter(description = "Whether to include today in the export", example = "false") 82 | @RequestParam(defaultValue = "false") boolean includeToday) { 83 | log.info("V2: Exporting data for {} days back (includeToday={})", days, includeToday); 84 | ExportResultDTO result = fddbDataService.exportForDaysBack(days, includeToday); 85 | return ResponseEntity.ok(result); 86 | } 87 | } 88 | 89 | -------------------------------------------------------------------------------- /src/test/resources/domain/fddbdata-2024-08-27.json: -------------------------------------------------------------------------------- 1 | { 2 | "date": "2024-08-27T00:00:00.000", 3 | "products": [ 4 | { 5 | "name": "Bio Ziegenjogurt mild", 6 | "amount": "125 g", 7 | "calories": 112.0, 8 | "fat": 7.5, 9 | "carbs": 4.5, 10 | "protein": 6.6, 11 | "link": "https://fddb.info/db/en/food/andechser_molkerei_bio_ziegenjogurt_mild_99503/index.html" 12 | }, 13 | { 14 | "name": "Skyr (Bioland)", 15 | "amount": "200 g", 16 | "calories": 120.0, 17 | "fat": 0.4, 18 | "carbs": 7.0, 19 | "protein": 22.0, 20 | "link": "https://fddb.info/db/en/food/kaufland_skyr_bioland/index.html" 21 | }, 22 | { 23 | "name": "Kinder Duo", 24 | "amount": "25 g", 25 | "calories": 131.0, 26 | "fat": 7.0, 27 | "carbs": 15.0, 28 | "protein": 1.8, 29 | "link": "https://fddb.info/db/en/food/ferrero_kinder_duo_1054818/index.html" 30 | }, 31 | { 32 | "name": "Impact Whey Protein, Cookies & Cream", 33 | "amount": "1 Scoop", 34 | "calories": 102.0, 35 | "fat": 1.8, 36 | "carbs": 1.8, 37 | "protein": 19.8, 38 | "link": "https://fddb.info/db/en/food/myprotein_impact_whey_protein_cookies_und_cream_flavour_516098/index.html" 39 | }, 40 | { 41 | "name": "Haferflocken", 42 | "amount": "40 g", 43 | "calories": 149.0, 44 | "fat": 2.8, 45 | "carbs": 23.5, 46 | "protein": 5.4, 47 | "link": "https://fddb.info/db/en/food/durchschnittswert_haferflocken/index.html" 48 | }, 49 | { 50 | "name": "Granatapfel Kerne", 51 | "amount": "50 g", 52 | "calories": 36.0, 53 | "fat": 1.0, 54 | "carbs": 6.0, 55 | "protein": 0.7, 56 | "link": "https://fddb.info/db/en/food/rewe_beste_wahl_granatapfel_kerne_1155693/index.html" 57 | }, 58 | { 59 | "name": "Cornflakes, glutenfrei", 60 | "amount": "30 g", 61 | "calories": 111.0, 62 | "fat": 0.3, 63 | "carbs": 24.0, 64 | "protein": 2.4, 65 | "link": "https://fddb.info/db/en/food/schaer_cornflakes_glutenfrei/index.html" 66 | }, 67 | { 68 | "name": "Chiasamen", 69 | "amount": "30 g", 70 | "calories": 123.0, 71 | "fat": 9.9, 72 | "carbs": 1.1, 73 | "protein": 6.6, 74 | "link": "https://fddb.info/db/en/food/k-bio_chiasamen_869980/index.html" 75 | }, 76 | { 77 | "name": "Leinsamen, geschrotet", 78 | "amount": "10 g", 79 | "calories": 54.0, 80 | "fat": 4.2, 81 | "carbs": 0.3, 82 | "protein": 2.2, 83 | "link": "https://fddb.info/db/en/food/alnatura_leinsamen_geschrotet_11146/index.html" 84 | }, 85 | { 86 | "name": "Rice Cracker", 87 | "amount": "25 g", 88 | "calories": 105.0, 89 | "fat": 2.2, 90 | "carbs": 19.3, 91 | "protein": 1.9, 92 | "link": "https://fddb.info/db/en/food/mitsuba_mitsuba/index.html" 93 | }, 94 | { 95 | "name": "Nudeln, gekocht", 96 | "amount": "600 g", 97 | "calories": 823.0, 98 | "fat": 12.6, 99 | "carbs": 150.0, 100 | "protein": 27.0, 101 | "link": "https://fddb.info/db/en/food/durchschnittswert_nudeln_gekocht/index.html" 102 | }, 103 | { 104 | "name": "Tomatensauce", 105 | "amount": "300 ml", 106 | "calories": 169.0, 107 | "fat": 17.7, 108 | "carbs": 17.7, 109 | "protein": 6.6, 110 | "link": "https://fddb.info/db/en/food/diverse_tomatensauce_923194/index.html" 111 | }, 112 | { 113 | "name": "Thunfisch-Filets, im eigenen Saft und Au", 114 | "amount": "60 g", 115 | "calories": 66.0, 116 | "fat": 0.4, 117 | "carbs": 0.1, 118 | "protein": 15.6, 119 | "link": "https://fddb.info/db/en/food/nixe_thunfisch-filets_im_eigenen_saft_und_aufguss/index.html" 120 | }, 121 | { 122 | "name": "Vollmilch, 3,5% Fett", 123 | "amount": "40 ml", 124 | "calories": 26.0, 125 | "fat": 1.4, 126 | "carbs": 1.9, 127 | "protein": 1.2, 128 | "link": "https://fddb.info/db/en/food/molkerei_vollmilch_3_5prozent_fett/index.html" 129 | } 130 | ], 131 | "totalCalories": 2128.0, 132 | "totalFat": 69.3, 133 | "totalCarbs": 272.0, 134 | "totalSugar": 44.1, 135 | "totalProtein": 119.7, 136 | "totalFibre": 24.0 137 | } 138 | 139 | -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/rest/v1/FddbDataResourceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.rest.v1; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.dto.DateRangeDTO; 4 | import dev.itobey.adapter.api.fddb.exporter.dto.ExportResultDTO; 5 | import dev.itobey.adapter.api.fddb.exporter.dto.FddbDataDTO; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.ProductWithDateDTO; 7 | import dev.itobey.adapter.api.fddb.exporter.service.FddbDataService; 8 | import org.junit.jupiter.api.Tag; 9 | import org.junit.jupiter.api.Test; 10 | import org.junit.jupiter.api.extension.ExtendWith; 11 | import org.mockito.InjectMocks; 12 | import org.mockito.Mock; 13 | import org.mockito.junit.jupiter.MockitoExtension; 14 | import org.springframework.http.HttpStatus; 15 | import org.springframework.http.ResponseEntity; 16 | 17 | import java.util.Arrays; 18 | import java.util.List; 19 | import java.util.Optional; 20 | 21 | import static org.junit.jupiter.api.Assertions.assertEquals; 22 | import static org.mockito.Mockito.when; 23 | 24 | /** 25 | * Test for deprecated v1 API compatibility. 26 | * 27 | * @deprecated Tests deprecated v1 API. See FddbDataQueryResourceV2Test, FddbDataExportResourceV2Test, etc. 28 | */ 29 | @Deprecated 30 | @ExtendWith(MockitoExtension.class) 31 | @Tag("v1-compat") 32 | class FddbDataResourceTest { 33 | 34 | @Mock 35 | private FddbDataService fddbDataService; 36 | 37 | @InjectMocks 38 | private FddbDataResourceV1 fddbDataResource; 39 | 40 | @Test 41 | void testFindAllEntries() { 42 | List mockData = Arrays.asList(new FddbDataDTO(), new FddbDataDTO()); 43 | when(fddbDataService.findAllEntries()).thenReturn(mockData); 44 | 45 | ResponseEntity> response = fddbDataResource.findAllEntries(); 46 | 47 | assertEquals(HttpStatus.OK, response.getStatusCode()); 48 | assertEquals(mockData, response.getBody()); 49 | } 50 | 51 | @Test 52 | void testFindByDate_ValidDate() { 53 | String validDate = "2023-01-01"; 54 | FddbDataDTO mockData = new FddbDataDTO(); 55 | when(fddbDataService.findByDate(validDate)).thenReturn(Optional.of(mockData)); 56 | 57 | ResponseEntity response = fddbDataResource.findByDate(validDate); 58 | 59 | assertEquals(HttpStatus.OK, response.getStatusCode()); 60 | assertEquals(Optional.of(mockData), response.getBody()); 61 | } 62 | 63 | @Test 64 | void testFindByDate_InvalidDate() { 65 | String invalidDate = "2023-1-1"; 66 | 67 | ResponseEntity response = fddbDataResource.findByDate(invalidDate); 68 | 69 | assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode()); 70 | assertEquals("Date must be in the format YYYY-MM-DD", response.getBody()); 71 | } 72 | 73 | @Test 74 | void testFindByDate_NotFound() { 75 | String validDate = "2023-01-01"; 76 | when(fddbDataService.findByDate(validDate)).thenReturn(Optional.empty()); 77 | 78 | ResponseEntity response = fddbDataResource.findByDate(validDate); 79 | 80 | assertEquals(HttpStatus.NOT_FOUND, response.getStatusCode()); 81 | } 82 | 83 | @Test 84 | void testFindByProduct() { 85 | String productName = "TestProduct"; 86 | List mockData = Arrays.asList(new ProductWithDateDTO(), new ProductWithDateDTO()); 87 | when(fddbDataService.findByProduct(productName)).thenReturn(mockData); 88 | 89 | ResponseEntity> response = fddbDataResource.findByProduct(productName); 90 | 91 | assertEquals(HttpStatus.OK, response.getStatusCode()); 92 | assertEquals(mockData, response.getBody()); 93 | } 94 | 95 | @Test 96 | void testExportForTimerange() { 97 | DateRangeDTO mockRequest = new DateRangeDTO(); 98 | ExportResultDTO mockResult = new ExportResultDTO(); 99 | when(fddbDataService.exportForTimerange(mockRequest)).thenReturn(mockResult); 100 | 101 | ResponseEntity response = fddbDataResource.exportForTimerange(mockRequest); 102 | 103 | assertEquals(HttpStatus.OK, response.getStatusCode()); 104 | assertEquals(mockResult, response.getBody()); 105 | } 106 | 107 | @Test 108 | void testExportForDaysBack() { 109 | int days = 7; 110 | boolean includeToday = true; 111 | ExportResultDTO mockResult = new ExportResultDTO(); 112 | when(fddbDataService.exportForDaysBack(days, includeToday)).thenReturn(mockResult); 113 | 114 | ResponseEntity response = fddbDataResource.exportForDaysBack(days, includeToday); 115 | 116 | assertEquals(HttpStatus.OK, response.getStatusCode()); 117 | assertEquals(mockResult, response.getBody()); 118 | } 119 | } -------------------------------------------------------------------------------- /src/test/java/dev/itobey/adapter/api/fddb/exporter/service/FddbParserServiceTest.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.Product; 5 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 6 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 7 | import lombok.SneakyThrows; 8 | import org.assertj.core.api.Assertions; 9 | import org.junit.jupiter.api.BeforeEach; 10 | import org.junit.jupiter.api.Test; 11 | import org.springframework.core.io.ClassPathResource; 12 | import org.springframework.core.io.Resource; 13 | 14 | import java.nio.charset.StandardCharsets; 15 | import java.nio.file.Files; 16 | import java.nio.file.Path; 17 | import java.util.List; 18 | 19 | import static org.junit.jupiter.api.Assertions.assertEquals; 20 | import static org.junit.jupiter.api.Assertions.assertNotNull; 21 | 22 | /** 23 | * Based on https://fddb.info/db/i18n/myday20/?%20\%20lang=de&p=1724105323&q=1724191723 24 | */ 25 | class FddbParserServiceTest { 26 | 27 | private FddbParserService fddbParserService; 28 | 29 | @BeforeEach 30 | void setUp() { 31 | fddbParserService = new FddbParserService(); 32 | } 33 | 34 | @Test 35 | @SneakyThrows 36 | void parseDiary_whenLoggedInAndDataAvailable_shouldParseAccordingly() { 37 | // Given 38 | Resource resource = new ClassPathResource("valid-response.html"); 39 | Path path = resource.getFile().toPath(); 40 | String content = Files.readString(path, StandardCharsets.UTF_8); 41 | 42 | // When 43 | FddbData fddbData = fddbParserService.parseDiary(content); 44 | 45 | // Then 46 | assertNotNull(fddbData); 47 | assertEquals(2565, fddbData.getTotalCalories()); 48 | assertEquals(110.4, fddbData.getTotalFat()); 49 | assertEquals(246.2, fddbData.getTotalCarbs()); 50 | assertEquals(126.4, fddbData.getTotalProtein()); 51 | assertEquals(51, fddbData.getTotalSugar()); 52 | assertEquals(18.3, fddbData.getTotalFibre()); 53 | 54 | List products = fddbData.getProducts(); 55 | assertEquals(19, products.size()); 56 | 57 | // Check a few specific products 58 | Product pizza = products.getFirst(); 59 | assertEquals("Pizza", pizza.getName()); 60 | assertEquals("150 g", pizza.getAmount()); 61 | assertEquals(300, pizza.getCalories()); 62 | assertEquals(10.5, pizza.getFat()); 63 | assertEquals(30, pizza.getCarbs()); 64 | assertEquals(13.5, pizza.getProtein()); 65 | assertEquals("https://fddb.info/db/en/food/selbstgemacht_pizza/index.html", pizza.getLink()); 66 | 67 | Product amaranth = products.get(1); 68 | assertEquals("Bio Amaranth gepufft", amaranth.getName()); 69 | assertEquals("10 g", amaranth.getAmount()); 70 | assertEquals(37, amaranth.getCalories()); 71 | assertEquals(0.5, amaranth.getFat()); 72 | assertEquals(6.5, amaranth.getCarbs()); 73 | assertEquals(1.2, amaranth.getProtein()); 74 | assertEquals("https://fddb.info/db/en/food/antersdorfer_muehle_bio_amaranth_gepufft/index.html", amaranth.getLink()); 75 | 76 | Product senf = products.get(18); 77 | assertEquals("Senf", senf.getName()); 78 | assertEquals("30 g", senf.getAmount()); 79 | assertEquals(26, senf.getCalories()); 80 | assertEquals(1.2, senf.getFat()); 81 | assertEquals(1.8, senf.getCarbs()); 82 | assertEquals(1.8, senf.getProtein()); 83 | assertEquals("https://fddb.info/db/en/food/durchschnittswert_senf/index.html", senf.getLink()); 84 | } 85 | 86 | @Test 87 | @SneakyThrows 88 | void parseDiary_whenNotLoggedIn_shouldThrowException() { 89 | // Given 90 | Resource resource = new ClassPathResource("__files/unauthenticated.html"); 91 | Path path = resource.getFile().toPath(); 92 | String content = Files.readString(path, StandardCharsets.UTF_8); 93 | 94 | // When; Then 95 | Assertions.assertThatExceptionOfType(AuthenticationException.class) 96 | .isThrownBy(() -> fddbParserService.parseDiary(content)); 97 | } 98 | 99 | @Test 100 | @SneakyThrows 101 | void parseDiary_whenLoggedInAndNoDataAvailable_shouldThrowException() { 102 | // Given 103 | Resource resource = new ClassPathResource("no-data-available.html"); 104 | Path path = resource.getFile().toPath(); 105 | String content = Files.readString(path, StandardCharsets.UTF_8); 106 | 107 | // When; Then 108 | Assertions.assertThatExceptionOfType(ParseException.class) 109 | .isThrownBy(() -> fddbParserService.parseDiary(content)); 110 | } 111 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

2 | 3 |

Export data from FDDB.info with ease and flexibility

4 | 5 | ## [Documentation](https://itobey.github.io/fddb-exporter/) 6 | 7 | [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/fddb-exporter)](https://artifacthub.io/packages/search?repo=fddb-exporter) 8 | [![Build Status](https://img.shields.io/github/actions/workflow/status/itobey/fddb-exporter/ci.yml?style=flat-square)](https://github.com/itobey/fddb-exporter/actions/workflows/ci.yml) 9 | [![Release Version](https://img.shields.io/github/release/itobey/fddb-exporter.svg?style=flat-square&color=9CF)](https://github.com/itobey/fddb-exporter/releases) 10 | [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square)](https://www.gnu.org/licenses/mit.txt) 11 | [![Commit Activity](https://img.shields.io/github/commit-activity/m/itobey/fddb-exporter.svg?style=flat-square)](https://github.com/itobey/fddb-exporter/commits/master) 12 | [![Last Commit](https://img.shields.io/github/last-commit/itobey/fddb-exporter.svg?style=flat-square&color=FF9900)](https://github.com/itobey/fddb-exporter/commits/master) 13 | 14 |
15 | 16 | # Overview 17 | 18 | FDDB Exporter is a tool designed to extract nutritional data from [FDDB.info](https://fddb.info/) and store it in a 19 | MongoDB/InfluxDB database. 20 | This application is especially useful for individuals who want to keep their FDDB diaries for themselves. 21 | FDDB only stores entries for up to 2 years for premium members, and even less for free users. 22 | Additionally, it is very handy if you want to query your data to see on which days you have entered specific products. 23 | See the [documentation](https://itobey.github.io/fddb-exporter/) for a deep dive. 24 | There is also a [Flutter app](https://github.com/itobey/fddb-exporter-app) available as a frontend. 25 | 26 | # Key Features 27 | 28 | - Exports daily nutritional totals: calories, fat, carbohydrates, sugar, protein, and fiber 29 | - Stores detailed information on consumed products, including name, amount, nutritional values, and link to the product 30 | page 31 | - Supports scheduled daily exports and manual exports for specific date ranges 32 | - Provides a RESTful API for data retrieval and export operations 33 | - Interactive Swagger UI for easy API exploration and testing 34 | - A special API endpoint to find correlations to matching dates for checking food allergies 35 | 36 | # Prerequisites 37 | 38 | - Docker or Java 21+ runtime environment 39 | - A valid FDDB.info account 40 | - A running database (MongoDB and/or InfluxDB) 41 | - MongoDB instance (used for storing all data) 42 | - InfluxDB instance (used for storing daily totals) 43 | 44 | # Quick Start 45 | 46 | Once the application is running, you can access the interactive API documentation at: 47 | 48 | **Swagger UI:** `http://localhost:8080/swagger-ui.html` 49 | 50 | See [Swagger UI Setup Guide](docs/SWAGGER-UI-SETUP.md) for detailed documentation. 51 | 52 | # Technology Stack 53 | 54 | - [Spring Boot](https://spring.io/projects/spring-boot) 55 | - Java 21 56 | - MongoDB 57 | - Docker (optional) 58 | 59 | # Privacy 60 | 61 | This application does not collect any personal data. All data is stored locally on your device. Your FDDB credentials 62 | are only used to log in to the FDDB website and fetch the data. To determine how this tool is used (and how important it 63 | is to maintain it), the application sends some anonymous data to my server. The mail address is hashed and cannot be 64 | used to identify you. Along with the hash of the mail address, the following data is sent: amount of documents in the 65 | database, what persistence layer is used, the version of the application and the environment (container, Kubernetes or 66 | plain java). Feel free to audit the code 67 | yourself [here](./src/main/java/dev/itobey/adapter/api/fddb/exporter/service/telemetry/TelemetryService.java). 68 | If you still have any concerns, feel free to contact me or open an issue. 69 | 70 | # Roadmap 71 | 72 | I plan on implementing the following features in the future: 73 | 74 | - [x] Helm Chart for deployment 75 | - [x] product search API: to get only relevant data instead of the entire day 76 | - [ ] product search API: limit search by date or weekday instead of searching and returning every day 77 | - [x] correlation API: find products correlating with given dates 78 | - [x] new stats endpoint: display some stats of your data 79 | - [x] ARM container release 80 | - [ ] Alerting feature to notify when the Scheduler run failed 81 | - [x] accompanying Flutter app as a frontend 82 | - [x] InfluxDB as additional persistence layer 83 | 84 | If you have another feature in mind please open up an issue or contact me. 85 | 86 | # Resource Usage 87 | 88 | The service typically uses around 300 MB of RAM with minimal CPU usage when idle. 89 | 90 | # Contributing 91 | 92 | Contributions are welcome! Please feel free to submit a Pull Request or open an issue. 93 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/persistence/MongoDBService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service.persistence; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.projection.ProductWithDate; 5 | import dev.itobey.adapter.api.fddb.exporter.repository.FddbDataRepository; 6 | import org.springframework.beans.factory.annotation.Autowired; 7 | import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; 8 | import org.springframework.data.mongodb.core.MongoTemplate; 9 | import org.springframework.data.mongodb.core.aggregation.Aggregation; 10 | import org.springframework.data.mongodb.core.aggregation.AggregationOperation; 11 | import org.springframework.data.mongodb.core.aggregation.AggregationResults; 12 | import org.springframework.data.mongodb.core.query.Criteria; 13 | import org.springframework.stereotype.Service; 14 | 15 | import java.time.LocalDate; 16 | import java.util.ArrayList; 17 | import java.util.List; 18 | import java.util.Optional; 19 | 20 | import static org.springframework.data.mongodb.core.aggregation.Aggregation.*; 21 | 22 | /** 23 | * Provides MongoDB-related services for managing {@link FddbData} objects. 24 | *

25 | * This does not use Lomboks constructor, because the required=false is not supported by Lombok. 26 | */ 27 | @Service 28 | @ConditionalOnProperty(name = "fddb-exporter.persistence.mongodb.enabled", havingValue = "true") 29 | public class MongoDBService { 30 | 31 | @Autowired(required = false) 32 | private FddbDataRepository fddbDataRepository; 33 | @Autowired(required = false) 34 | private MongoTemplate mongoTemplate; 35 | 36 | public long countAllEntries() { 37 | return fddbDataRepository.count(); 38 | } 39 | 40 | /** 41 | * Retrieves all {@link FddbData} objects stored in the database. 42 | * 43 | * @return a list of all {@link FddbData} objects 44 | */ 45 | public List findAllEntries() { 46 | return fddbDataRepository.findAll(); 47 | } 48 | 49 | /** 50 | * Retrieves an entry to a given date. 51 | * 52 | * @param date the date to find an entry to 53 | * @return an Optional of {@link FddbData} 54 | */ 55 | public Optional findByDate(LocalDate date) { 56 | return fddbDataRepository.findFirstByDate(date); 57 | } 58 | 59 | /** 60 | * Searches for a product name and returns the date with the product details. 61 | * Unfortunately an aggregation annotation query did not work, maybe because I'm stuck with Mongo 4.4. 62 | * 63 | * @param name the name of the product 64 | * @return a list of matches 65 | */ 66 | public List findByProduct(String name) { 67 | AggregationOperation match = match(Criteria.where("products.name").regex(name, "i")); 68 | AggregationOperation unwind = unwind("products"); 69 | AggregationOperation secondMatch = match(Criteria.where("products.name").regex(name, "i")); 70 | AggregationOperation project = project() 71 | .andExpression("date").as("date") 72 | .and("products").as("product"); 73 | 74 | Aggregation aggregation = newAggregation(match, unwind, secondMatch, project); 75 | 76 | AggregationResults results = mongoTemplate.aggregate( 77 | aggregation, "fddb", ProductWithDate.class); 78 | 79 | return results.getMappedResults(); 80 | } 81 | 82 | public List findByProductsWithExclusions(List includeNames, List excludeNames, LocalDate startDate) { 83 | List operations = new ArrayList<>(); 84 | 85 | // Add date filter first 86 | if (startDate != null) { 87 | operations.add(match(Criteria.where("date").gte(startDate))); 88 | } 89 | 90 | // Unwind products array 91 | operations.add(unwind("products")); 92 | 93 | // Match stage after unwind to filter individual products 94 | if (!includeNames.isEmpty()) { 95 | List includeList = includeNames.stream() 96 | .map(name -> Criteria.where("products.name").regex(name, "i")) 97 | .toList(); 98 | operations.add(match(new Criteria().orOperator(includeList.toArray(new Criteria[0])))); 99 | } 100 | 101 | if (!excludeNames.isEmpty()) { 102 | List excludeList = excludeNames.stream() 103 | .map(name -> Criteria.where("products.name").regex(name, "i")) 104 | .toList(); 105 | operations.add(match(new Criteria().norOperator(excludeList.toArray(new Criteria[0])))); 106 | } 107 | 108 | // Project required fields 109 | operations.add(project() 110 | .andExpression("date").as("date") 111 | .and("products").as("product")); 112 | 113 | Aggregation aggregation = newAggregation(operations); 114 | 115 | AggregationResults results = mongoTemplate.aggregate( 116 | aggregation, "fddb", ProductWithDate.class); 117 | 118 | return results.getMappedResults(); 119 | } 120 | 121 | } 122 | -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/FddbParserService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.Product; 5 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 6 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 7 | import lombok.extern.slf4j.Slf4j; 8 | import org.jsoup.Jsoup; 9 | import org.jsoup.nodes.Document; 10 | import org.jsoup.nodes.Element; 11 | import org.jsoup.select.Elements; 12 | import org.springframework.stereotype.Service; 13 | 14 | import java.util.ArrayList; 15 | import java.util.List; 16 | 17 | @Service 18 | @Slf4j 19 | public class FddbParserService { 20 | 21 | private static final String XPATH_AUTH_STATUS = "//div[@class='quicklinks']/a[contains(@class, 'v2hdlnk') and (text()='Anmelden' or text()='Login')]"; 22 | private static final String XPATH_PRODUCT_TABLE = "//table[@class='myday-table-std']/tbody/tr"; 23 | private static final String XPATH_SUGAR = "//*[@id=\"content\"]/div[3]/div[2]/div[2]/div/table[2]/tbody/tr[4]/td[2]/span"; 24 | private static final String XPATH_FIBER = "//*[@id=\"content\"]/div[3]/div[2]/div[2]/div/table[2]/tbody/tr[8]/td[2]/b"; 25 | 26 | public FddbData parseDiary(String input) throws AuthenticationException, ParseException { 27 | Document doc = Jsoup.parse(input, "UTF-8"); 28 | checkAuthentication(doc); 29 | checkDataAvailable(doc); 30 | 31 | FddbData fddbData = new FddbData(); 32 | List products = parseProducts(doc); 33 | fddbData.setProducts(products); 34 | 35 | setDayTotals(fddbData, doc); 36 | 37 | return fddbData; 38 | } 39 | 40 | public void checkAuthentication(Document doc) throws AuthenticationException { 41 | Elements authStatus = doc.selectXpath(XPATH_AUTH_STATUS); 42 | if (!authStatus.isEmpty()) { 43 | String errorMsg = "Login to FDDB not successful, please check credentials"; 44 | log.error(errorMsg); 45 | throw new AuthenticationException(errorMsg); 46 | } 47 | } 48 | 49 | private List parseProducts(Document doc) { 50 | List products = new ArrayList<>(); 51 | Elements rows = doc.selectXpath(XPATH_PRODUCT_TABLE); 52 | 53 | for (int i = 0; i < rows.size() - 1; i++) { 54 | Element row = rows.get(i); 55 | Elements columns = row.select("td"); 56 | 57 | if (columns.size() <= 1 || isCategoryRow(columns)) { 58 | continue; 59 | } 60 | 61 | products.add(createProduct(columns)); 62 | } 63 | 64 | return products; 65 | } 66 | 67 | private boolean isCategoryRow(Elements columns) { 68 | return columns.stream() 69 | .anyMatch(column -> { 70 | Element span = column.selectFirst("span[style]"); 71 | return span != null && span.attr("style").replaceAll("\\s+", "").contains("color:#AAAAAA"); 72 | }); 73 | } 74 | 75 | private Product createProduct(Elements columns) { 76 | Product product = new Product(); 77 | Element productLink = columns.get(0).selectFirst("a"); 78 | 79 | if (productLink != null) { 80 | setProductNameAndAmount(product, productLink); 81 | product.setLink(productLink.attr("href")); 82 | } 83 | 84 | product.setCalories(extractNumber(columns.get(2).text())); 85 | product.setFat(extractNumber(columns.get(3).text())); 86 | product.setCarbs(extractNumber(columns.get(4).text())); 87 | product.setProtein(extractNumber(columns.get(5).text())); 88 | 89 | return product; 90 | } 91 | 92 | private void setProductNameAndAmount(Product product, Element productLink) { 93 | String fullName = productLink.text(); 94 | String[] parts = fullName.split(" ", 3); 95 | if (parts.length == 3) { 96 | product.setAmount(parts[0] + " " + parts[1]); 97 | product.setName(parts[2]); 98 | } else { 99 | product.setName(fullName); 100 | } 101 | } 102 | 103 | private void setDayTotals(FddbData fddbData, Document doc) { 104 | Elements lastRow = doc.selectXpath(XPATH_PRODUCT_TABLE + "[last()]/td"); 105 | fddbData.setTotalCalories(extractNumber(lastRow.get(2).text())); 106 | fddbData.setTotalFat(extractNumber(lastRow.get(3).text())); 107 | fddbData.setTotalCarbs(extractNumber(lastRow.get(4).text())); 108 | fddbData.setTotalProtein(extractNumber(lastRow.get(5).text())); 109 | fddbData.setTotalSugar(extractNumber(doc.selectXpath(XPATH_SUGAR).text())); 110 | fddbData.setTotalSugar(extractNumber(doc.selectXpath(XPATH_SUGAR).text())); 111 | fddbData.setTotalFibre(extractNumber(doc.selectXpath(XPATH_FIBER).text())); 112 | } 113 | 114 | private double extractNumber(String text) { 115 | return Double.parseDouble(text.replaceAll("[^0-9.]", "")); 116 | } 117 | 118 | private void checkDataAvailable(Document doc) throws ParseException { 119 | Elements lastRow = doc.selectXpath(XPATH_PRODUCT_TABLE + "[last()]/td"); 120 | if (lastRow.isEmpty()) { 121 | throw new ParseException("cannot parse input. it's likely there is no data available for the given day"); 122 | } 123 | } 124 | } -------------------------------------------------------------------------------- /src/main/java/dev/itobey/adapter/api/fddb/exporter/service/FddbDataService.java: -------------------------------------------------------------------------------- 1 | package dev.itobey.adapter.api.fddb.exporter.service; 2 | 3 | import dev.itobey.adapter.api.fddb.exporter.config.FddbExporterProperties; 4 | import dev.itobey.adapter.api.fddb.exporter.domain.FddbData; 5 | import dev.itobey.adapter.api.fddb.exporter.domain.projection.ProductWithDate; 6 | import dev.itobey.adapter.api.fddb.exporter.dto.*; 7 | import dev.itobey.adapter.api.fddb.exporter.exception.AuthenticationException; 8 | import dev.itobey.adapter.api.fddb.exporter.exception.ParseException; 9 | import dev.itobey.adapter.api.fddb.exporter.mapper.FddbDataMapper; 10 | import dev.itobey.adapter.api.fddb.exporter.service.persistence.PersistenceService; 11 | import lombok.RequiredArgsConstructor; 12 | import lombok.extern.slf4j.Slf4j; 13 | import org.springframework.stereotype.Service; 14 | 15 | import java.time.DateTimeException; 16 | import java.time.LocalDate; 17 | import java.util.ArrayList; 18 | import java.util.List; 19 | import java.util.Optional; 20 | import java.util.stream.IntStream; 21 | 22 | import static java.time.temporal.ChronoUnit.DAYS; 23 | 24 | @Service 25 | @RequiredArgsConstructor 26 | @Slf4j 27 | public class FddbDataService { 28 | 29 | private final TimeframeCalculator timeframeCalculator; 30 | private final ExportService exportService; 31 | private final PersistenceService persistenceService; 32 | private final FddbDataMapper fddbDataMapper; 33 | private final StatsService statsService; 34 | private final FddbExporterProperties properties; 35 | 36 | public List findAllEntries() { 37 | List allEntries = persistenceService.findAllEntries(); 38 | return fddbDataMapper.toFddbDataDTO(allEntries); 39 | } 40 | 41 | public List findByProduct(String name) { 42 | List productsWithDate = persistenceService.findByProduct(name); 43 | return fddbDataMapper.toProductWithDateDto(productsWithDate); 44 | } 45 | 46 | public Optional findByDate(String dateString) { 47 | LocalDate date = LocalDate.parse(dateString); 48 | Optional fddbDataOptional = persistenceService.findByDate(date); 49 | return fddbDataOptional.map(fddbDataMapper::toFddbDataDTO); 50 | } 51 | 52 | public ExportResultDTO exportForTimerange(DateRangeDTO dateRangeDTO) { 53 | LocalDate from = LocalDate.parse(dateRangeDTO.getFromDate()); 54 | LocalDate to = LocalDate.parse(dateRangeDTO.getToDate()); 55 | 56 | if (from.isAfter(to)) { 57 | throw new DateTimeException("The 'from' date cannot be after the 'to' date"); 58 | } 59 | 60 | long amountDaysToExport = DAYS.between(from, to) + 1; 61 | 62 | List successfulDays = new ArrayList<>(); 63 | List unsuccessfulDays = new ArrayList<>(); 64 | 65 | IntStream.range(0, (int) amountDaysToExport) 66 | .mapToObj(from::plusDays) 67 | .forEach(date -> { 68 | try { 69 | exportForDate(date); 70 | successfulDays.add(date.toString()); 71 | } catch (ParseException parseException) { 72 | unsuccessfulDays.add(date.toString()); 73 | } 74 | // AuthenticationException is not caught and will halt the process 75 | }); 76 | 77 | ExportResultDTO result = new ExportResultDTO(); 78 | result.setSuccessfulDays(successfulDays); 79 | result.setUnsuccessfulDays(unsuccessfulDays); 80 | return result; 81 | } 82 | 83 | public ExportResultDTO exportForDaysBack(int days, boolean includeToday) { 84 | // safety net to prevent accidents 85 | int maxDaysBack = properties.getFddb().getMaxDaysBack(); 86 | int minDaysBack = properties.getFddb().getMinDaysBack(); 87 | if (days < minDaysBack || days > maxDaysBack) { 88 | throw new DateTimeException("Days back must be between " + minDaysBack + " and " + maxDaysBack); 89 | } 90 | 91 | LocalDate to = includeToday ? LocalDate.now() : LocalDate.now().minusDays(1); 92 | LocalDate from = to.minusDays(days - 1); 93 | 94 | DateRangeDTO timeframe = DateRangeDTO.builder() 95 | .fromDate(from.toString()) 96 | .toDate(to.toString()) 97 | .build(); 98 | 99 | return exportForTimerange(timeframe); 100 | } 101 | 102 | public StatsDTO getStats() { 103 | return statsService.getStats(); 104 | } 105 | 106 | public RollingAveragesDTO getRollingAverages(DateRangeDTO dateRangeDTO) { 107 | LocalDate fromDate = LocalDate.parse(dateRangeDTO.getFromDate()); 108 | LocalDate toDate = LocalDate.parse(dateRangeDTO.getToDate()); 109 | 110 | StatsDTO.Averages averages = statsService.getAveragesForDateRange(fromDate, toDate); 111 | return RollingAveragesDTO.builder() 112 | .fromDate(dateRangeDTO.getFromDate()) 113 | .toDate(dateRangeDTO.getToDate()) 114 | .averages(averages) 115 | .build(); 116 | } 117 | 118 | private void exportForDate(LocalDate date) throws ParseException, AuthenticationException { 119 | log.debug("exporting data for {}", date); 120 | TimeframeDTO timeframeDTO = timeframeCalculator.calculateTimeframeFor(date); 121 | FddbData fddbData = exportService.exportData(timeframeDTO); 122 | persistenceService.saveOrUpdate(fddbData); 123 | } 124 | } -------------------------------------------------------------------------------- /docs/details/configuration.md: -------------------------------------------------------------------------------- 1 | # Configuration 2 | 3 | ## Configuration Options 4 | 5 | The FDDB Exporter application is a [Spring Boot](https://spring.io/projects/spring-boot) 3 application. 6 | It is pre-configured with a basic configuration embedded in the application. However, some properties need to be 7 | configured to make the application work for your use case and environment. The easiest way to do this is via 8 | environment variables. 9 | 10 | For further methods of configuring Spring Boot applications, please refer to 11 | the [official documentation](https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config). 12 | 13 | ## Environment Variables 14 | 15 | This page lists all available environment variables and their default values. Usually, you only need to change 16 | the username and password for the FDDB connection and the settings for your preferred database connection. 17 | 18 | - For more information about how to configure the Docker image, please refer to 19 | the [Docker details](/details/docker.md). 20 | - For more information about how to configure the Helm Chart, please refer to the [Helm details](/details/helm.md). 21 | 22 | ### FDBB Configuration 23 | 24 | The application requires a valid FDDB.info account to work. The following environment variables are used to configure 25 | the FDDB connection. 26 | 27 | | Variable | Default | Description | 28 | |-------------------------------|-------------------|----------------------------------| 29 | | `FDDB-EXPORTER_FDDB_USERNAME` | - | Your FDDB.info username or email | 30 | | `FDDB-EXPORTER_FDDB_PASSWORD` | - | Your FDDB.info password | 31 | | `FDDB-EXPORTER_FDDB_URL` | https://fddb.info | FDDB website URL | 32 | 33 | ### Export Configuration 34 | 35 | For more information about the scheduler and how the export works, see [Export details](/details/exports-and-data.md). 36 | 37 | | Variable | Default | Description | 38 | |------------------------------------|-------------|---------------------------------------------------------------| 39 | | `FDDB-EXPORTER_FDDB_MIN-DAYS-BACK` | 1 | Min limit of days back export for REST API | 40 | | `FDDB-EXPORTER_FDDB_MAX-DAYS-BACK` | 365 | Max limit of days back export for REST API | 41 | | `FDDB-EXPORTER_SCHEDULER_ENABLED` | true | Enable/disable the daily export scheduler | 42 | | `FDDB-EXPORTER_SCHEDULER_CRON` | 0 0 3 * * * | Scheduler cron expression (default: 3 AM daily) (Spring cron) | 43 | 44 | ### MongoDB Configuration 45 | 46 | MongoDB is used by default as persistence for the application. The following environment variables are used to configure 47 | the MongoDB connection. You can disable MongoDB persistence by setting `FDDB-EXPORTER_PERSISTENCE_MONGODB_ENABLED` to 48 | `false`. However, in this case, InfluxDB is necessary as persistence. For more information about persistence, see 49 | [Persistence details](/details/persistence.md). 50 | 51 | | Variable | Default | Description | 52 | |---------------------------------------------|-----------------------|----------------------------| 53 | | `FDDB-EXPORTER_PERSISTENCE_MONGODB_ENABLED` | true | Use MongoDB as persistence | 54 | | `SPRING_DATA_MONGODB_HOST` | localhost | MongoDB host | 55 | | `SPRING_DATA_MONGODB_PORT` | 27017 | MongoDB port | 56 | | `SPRING_DATA_MONGODB_DATABASE` | fddb | MongoDB database name | 57 | | `SPRING_DATA_MONGODB_USERNAME` | mongodb_fddb_user | MongoDB username | 58 | | `SPRING_DATA_MONGODB_PASSWORD` | mongodb_fddb_password | MongoDB password | 59 | 60 | ### InfluxDB Configuration 61 | 62 | InfluxDB is disabled by default. The following environment variables are used to configure the InfluxDB connection. You 63 | can enable InfluxDB persistence by setting `FDDB-EXPORTER_PERSISTENCE_INFLUXDB_ENABLED` to `true`. The token needs to 64 | have permissions to write to the specified bucket. The application will only work with InfluxDB 2.x. 65 | For more information about persistence, see [Persistence details](/details/persistence.md). 66 | 67 | | Variable | Default | Description | 68 | |----------------------------------------------|-----------------------|--------------------------------------| 69 | | `FDDB-EXPORTER_PERSISTENCE_INFLUXDB_ENABLED` | false | Use InfluxDB as persistence | 70 | | `FDDB-EXPORTER_INFLUXDB_URL` | http://localhost:8086 | URL to InfluxDB | 71 | | `FDDB-EXPORTER_INFLUXDB_ORG` | primary | InfluxDB Org | 72 | | `FDDB-EXPORTER_INFLUXDB_TOKEN` | token | Token for authentication in InfluxDB | 73 | | `FDDB-EXPORTER_INFLUXDB_BUCKET` | fddb-exporter | InfluxDB bucket | 74 | 75 | ### Logging Configuration 76 | 77 | | Variable | Default | Description | 78 | |----------------------|---------|-----------------------| 79 | | `LOGGING_LEVEL_ROOT` | info | Application log level | 80 | 81 | --- 82 | 83 | Because environment variables are plain text, it is recommended to use a secure method of storing and passing the 84 | credentials. If you are using the Helm chart, this is already handled for you. --------------------------------------------------------------------------------