Add index for documents scraped with keyset pagination in tender repository

- Introduced a new index `documents_scraped_created_at_id_idx` in the tender repository to support keyset pagination for documents marked as scraped, sorted by creation date.
- This enhancement improves the efficiency of querying scraped documents, facilitating better performance in data retrieval and management.

This update significantly optimizes the tender repository's handling of scraped documents, enhancing overall data access capabilities.
This commit is contained in:
Mazyar
2026-07-11 17:00:48 +03:30
parent 1df44ec8ca
commit 932b0cf24e
+7
View File
@@ -172,6 +172,13 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Te
{Key: "processing_metadata.documents_scraped", Value: 1},
{Key: "processing_metadata.documents_scraped_at", Value: -1},
}),
// Panel list: documents_scraped=true sorted by created_at desc (keyset pagination).
*orm.NewIndex("documents_scraped_created_at_id_idx", bson.D{
{Key: "created_at", Value: -1},
{Key: "_id", Value: -1},
}).WithPartialFilterExpression(bson.M{
"processing_metadata.documents_scraped": true,
}),
*orm.NewIndex("scraped_documents_scraped_at_idx", bson.D{
{Key: "scraped_documents.scraped_at", Value: 1},
}),