Refactor statistics repository to use created_at for historical data retrieval

- Updated the `Statistics` method to utilize `created_at` instead of `processing_metadata.scraped_at` for fetching daily counts, ensuring accurate historical data representation.
- Removed redundant conditions in the `scrapedDocumentsPerDay` method, streamlining the query logic for better performance and clarity.
- Added a new index on `source` and `created_at` to optimize database queries related to scraped documents.

This update enhances the accuracy of data retrieval in the dashboard statistics, improving the overall efficiency of the tender management system.
This commit is contained in:
Mazyar
2026-06-15 23:39:32 +03:30
parent 31d8efef0b
commit 7f84746400
2 changed files with 12 additions and 10 deletions
+4
View File
@@ -43,6 +43,10 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Re
{Key: "source", Value: 1},
{Key: "processing_metadata.scraped_at", Value: 1},
}),
*orm.NewIndex("source_created_at_idx", bson.D{
{Key: "source", Value: 1},
{Key: "created_at", Value: 1},
}),
*orm.NewIndex("main_classification_idx", bson.D{{Key: "main_classification", Value: 1}}),
*orm.NewIndex("contract_notice_id_idx", bson.D{{Key: "contract_notice_id", Value: 1}}),
// One row per TED ContractNoticeID when present (parallel scrape races).