- Added the `deadlineWithinWindowMatch` function to match raw deadline fields stored as Unix seconds or milliseconds, ensuring accurate filtering in MongoDB queries.
- Updated the `ClosingSoon` method in the repository to utilize the new deadline matching logic, improving query accuracy for upcoming deadlines.
- Introduced a new unit test, `TestDeadlineWithinWindowMatchCoversSecondsAndMilliseconds`, to validate the functionality of the deadline matching logic, ensuring both seconds and milliseconds are correctly handled.
- Enhanced comments in the `widget_cache` and `search_list_cache` files for better clarity on cache behavior and staleness.
This update improves the accuracy of deadline handling in the dashboard service and strengthens the test coverage for related functionalities.
- Introduced a new `widgetCache` structure to manage caching for various dashboard widgets, improving performance and reducing redundant data retrieval.
- Updated the `ClosingSoon` and `Trend` methods in the service to utilize the new caching mechanism, ensuring efficient data access and reducing load on the database.
- Enhanced the `NoticeTypes` and `Countries` methods to implement caching, further optimizing the dashboard's responsiveness.
- Added a new `searchListCache` implementation in the tender service to cache search results, improving the efficiency of tender searches and reducing database load.
- Implemented robust error handling and logging for cache operations, ensuring better visibility into potential issues.
This update significantly enhances the performance and scalability of the dashboard and tender services by integrating effective caching strategies, leading to improved user experience and resource management.
- Introduced a new `translatedNoticesScanner` interface for scanning MinIO for daily translated notice counts.
- Enhanced the `repository` struct to include fields for managing translated notice scope and caching.
- Implemented the `ScanTranslatedNotices` method in the `StorageClient` to retrieve daily counts and total from MinIO.
- Updated the `Statistics` method in the repository to utilize the new translated notices scope, improving data accuracy.
- Added unit tests for the translated notices functionality, ensuring robust validation of the new features.
This update significantly enhances the dashboard's ability to handle translated notice statistics, improving overall data management and reporting capabilities.
- Added new field `Days` and `ScrapedTED` to `SummaryResponse` for tracking daily TED scrape counts.
- Updated `SummaryQuery` to include `Days` parameter for querying scraped TED data.
- Modified `Summary` handler to accept and process the new `Days` parameter.
- Refactored `Summary` method in the repository to support the new `Days` parameter and improved aggregation logic.
- Enhanced caching logic in the service layer to utilize a composite cache key based on `closingWindowSec` and `days`, improving cache management and retrieval efficiency.
This update improves the dashboard's functionality by providing more detailed insights into TED scraping activities and optimizing the caching strategy for better performance.
- Introduced the `scrapedDocumentsScanner` interface to facilitate scanning of scraped documents from MinIO, returning both procedure summaries and daily document counts.
- Updated the `ListProceduresWithDocuments` method to utilize the new scanning functionality, improving data retrieval efficiency.
- Enhanced the `scrapedDocumentsPerDay` method to filter daily counts based on a specified start date, ensuring accurate reporting of document statistics.
- Added unit tests for the new scanning logic and daily counts filtering, ensuring robust functionality and error handling.
This update enhances the dashboard's document management capabilities, providing better insights into scraped documents and their daily counts.
- Introduced caching mechanisms for summary and statistics in the dashboard service, improving performance by reducing redundant data retrieval.
- Refactored the Summary method to utilize MongoDB aggregation for more efficient data processing and retrieval.
- Added synchronization features using singleflight to prevent duplicate processing of requests for cached data.
- Updated the repository to include a cachedScrapedTendersScope method, enhancing the efficiency of scraped document statistics retrieval.
This update significantly optimizes the dashboard's performance and data handling capabilities, ensuring faster response times and reduced load on the database.
- Introduced the ProcedureDocumentsLister interface to list contract folders with scraped documents, enhancing the accuracy of document-scrape statistics.
- Updated the dashboard repository to accept ProcedureDocumentsLister as a dependency, allowing for improved data retrieval.
- Implemented tests for the new functionality, ensuring proper handling of scraped document folder IDs and error propagation.
This update enhances the dashboard's capability to manage and report on scraped documents, improving overall system efficiency and data integrity.
- Updated `InitAISummarizerClient` to accept `mongoManager` for tracking translation success.
- Introduced new `Statistics` endpoint in the dashboard to fetch scraping and translation statistics.
- Enhanced `TranslationWorker` to utilize the new success counter for tracking successful translations.
- Added necessary data structures and query forms for statistics reporting.
This refactor improves the tracking of AI translation success and provides new insights through the dashboard statistics.
Mazyar