Compare commits

...

28 Commits

Author SHA1 Message Date
m.nazemi 6ee81c3581 Merge pull request 'Add documents scraped filter functionality and related tests' (#51) from documents-scraped into develop
continuous-integration/drone/push Build is passing
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/51
2026-07-12 22:03:13 +03:30
Mazyar d55b2685ca Merge branch 'documents-scraped' of https://repo.ravanertebat.com/TM/tm_back into documents-scraped 2026-07-12 22:02:43 +03:30
Mazyar 04c4102170 Enhance documents scraped filter with cancellation handling and caching improvements
- Updated `documents_scraped_filter.go` to extend the caching mechanism for contract folder IDs, increasing the cache TTL to 15 minutes to prevent premature invalidation during slow scans.
- Modified the `cachedContractFolderIDsWithDocuments` method to detach from the caller's cancellation, ensuring that shared scans are not aborted by client disconnects or timeouts.
- Introduced a new test, `TestCachedContractFolderIDsSurvivesCallerCancellation`, in `documents_scraped_filter_test.go` to validate the behavior of the caching mechanism under cancellation scenarios.
- Enhanced the `scanStartedProcedureLister` to manage scan initiation and cancellation more effectively, improving the robustness of the document retrieval process.

This update improves the reliability and performance of the tender management system's document filtering capabilities, particularly in scenarios with long-running scans.
2026-07-12 21:54:21 +03:30
Mazyar 0fde5772e2 Add documents scraped filter functionality and related tests
- Introduced `documents_scraped_filter.go` to handle filtering tenders based on the presence of documents in MinIO, utilizing a caching mechanism to optimize performance.
- Implemented `enrichDocumentsScrapedFilter` and `cachedContractFolderIDsWithDocuments` methods in the `tenderService` for efficient retrieval of contract folder IDs with documents.
- Added unit tests in `documents_scraped_filter_test.go` to validate the new filtering logic and caching behavior, ensuring accurate results and error handling.
- Updated `SearchForm` documentation to clarify the caching aspect of `ContractFolderIDsWithDocuments`.
- Enhanced comments in the handler and repository to reflect the new filtering logic and its implications.

This update improves the tender search functionality by efficiently managing document presence checks, leading to better performance and user experience in the tender management system.
2026-07-12 21:54:21 +03:30
m.nazemi c35fa331f9 Merge branch 'develop' into documents-scraped 2026-07-12 21:46:03 +03:30
m.nazemi c8616940ff Merge pull request 'Add tender submission functionality and related routes' (#52) from tender-submission into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/52
2026-07-12 21:45:23 +03:30
Mazyar fa3d466579 Enhance tender approval and submission handling
- Added a new error for submission sync failures, improving error handling in the tender approval process.
- Updated the `ToggleTenderApproval` method to handle the new error, providing clearer responses for sync issues.
- Introduced a method to reopen terminal submissions when tender approvals are resubmitted, enhancing submission state management.
- Implemented optimistic locking in the repository to prevent concurrent updates, improving data integrity.
- Added unit tests for new submission handling logic, ensuring robust functionality and error management.

This update strengthens the tender approval workflow and submission handling, ensuring better error reporting and state management.
2026-07-12 21:41:53 +03:30
Mazyar 0cce9ef1b5 Add tender submission functionality and related routes
- Introduced the `tender_submission` domain, including entity, repository, service, and handler implementations for managing tender submissions.
- Added new routes for both admin and public access to tender submissions, allowing for listing, ensuring, and retrieving submissions by ID and tender.
- Enhanced the tender approval service to synchronize submission workflows with approval changes, ensuring proper state management.
- Implemented validation and response structures for tender submission operations, improving API consistency and usability.
- Added unit tests for tender submission status transitions and workflow logic, ensuring robust functionality.

This update enhances the tender management system by providing comprehensive support for tender submissions, improving overall workflow and user experience.
2026-07-12 14:54:42 +03:30
Mazyar 5c93e0f01b Add documents scraped filter functionality and related tests
- Introduced `documents_scraped_filter.go` to handle filtering tenders based on the presence of documents in MinIO, utilizing a caching mechanism to optimize performance.
- Implemented `enrichDocumentsScrapedFilter` and `cachedContractFolderIDsWithDocuments` methods in the `tenderService` for efficient retrieval of contract folder IDs with documents.
- Added unit tests in `documents_scraped_filter_test.go` to validate the new filtering logic and caching behavior, ensuring accurate results and error handling.
- Updated `SearchForm` documentation to clarify the caching aspect of `ContractFolderIDsWithDocuments`.
- Enhanced comments in the handler and repository to reflect the new filtering logic and its implications.

This update improves the tender search functionality by efficiently managing document presence checks, leading to better performance and user experience in the tender management system.
2026-07-12 00:20:39 +03:30
m.nazemi 10385e997b Merge pull request 'Remove ContentXML field from Tender entity and related repository method' (#50) from content-xml into develop
continuous-integration/drone/push Build is passing
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/50
Reviewed-by: Hadi Barzegar <barzagarhadi@gmail.com>
2026-07-11 21:47:55 +03:30
m.nazemi adfd291245 Merge branch 'develop' into content-xml 2026-07-11 21:47:44 +03:30
m.nazemi 26a593306d Merge pull request 'Enhance dashboard service with widget caching and improved query handling' (#49) from panel-widget into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/49
2026-07-11 21:47:14 +03:30
Mazyar c5e62a4061 Implement deadline matching logic for dashboard repository and enhance unit tests
- Added the `deadlineWithinWindowMatch` function to match raw deadline fields stored as Unix seconds or milliseconds, ensuring accurate filtering in MongoDB queries.
- Updated the `ClosingSoon` method in the repository to utilize the new deadline matching logic, improving query accuracy for upcoming deadlines.
- Introduced a new unit test, `TestDeadlineWithinWindowMatchCoversSecondsAndMilliseconds`, to validate the functionality of the deadline matching logic, ensuring both seconds and milliseconds are correctly handled.
- Enhanced comments in the `widget_cache` and `search_list_cache` files for better clarity on cache behavior and staleness.

This update improves the accuracy of deadline handling in the dashboard service and strengthens the test coverage for related functionalities.
2026-07-11 21:44:22 +03:30
Mazyar 81b0d94ba3 Enhance dashboard service with widget caching and improved query handling
- Introduced a new `widgetCache` structure to manage caching for various dashboard widgets, improving performance and reducing redundant data retrieval.
- Updated the `ClosingSoon` and `Trend` methods in the service to utilize the new caching mechanism, ensuring efficient data access and reducing load on the database.
- Enhanced the `NoticeTypes` and `Countries` methods to implement caching, further optimizing the dashboard's responsiveness.
- Added a new `searchListCache` implementation in the tender service to cache search results, improving the efficiency of tender searches and reducing database load.
- Implemented robust error handling and logging for cache operations, ensuring better visibility into potential issues.

This update significantly enhances the performance and scalability of the dashboard and tender services by integrating effective caching strategies, leading to improved user experience and resource management.
2026-07-11 21:44:22 +03:30
m.nazemi 3e2700bc36 Merge pull request 'Refactor notification service to support multiple delivery channels' (#48) from notif-channel into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/48
2026-07-11 21:12:11 +03:30
Mazyar aafae2a26f Enhance notification entity and service for improved delivery channel handling
- Introduced new event types for notifications, including email and push, to support a broader range of delivery methods.
- Added a `persistedEventType` function to map delivery channels to their corresponding event types, improving clarity and maintainability.
- Updated the `sendToRecipient` method to ensure in-app notifications are always persisted, enhancing the reliability of notification records.
- Improved error logging for in-app notification persistence failures, providing better context for debugging.

This update enhances the notification system's flexibility and reliability by supporting multiple delivery channels and improving error handling.
2026-07-11 21:07:37 +03:30
Mazyar 9d8ce12816 Refactor notification service to support multiple delivery channels
- Updated the `sendToRecipient` method to iterate over the channels specified in the notification request, allowing notifications to be persisted for each channel.
- Modified the `persistNotification` method to accept a channel parameter, enhancing the logic for handling different delivery methods (email, push, in-app).
- Improved error logging to include the channel information when persisting notifications fails, providing better context for debugging.

This update enhances the flexibility of the notification service by enabling it to handle multiple delivery channels more effectively, improving overall notification management.
2026-07-11 21:07:37 +03:30
Mazyar b388af3518 Remove ContentXML field from Tender entity and related repository method
- Eliminated the `ContentXML` field from the `Tender` struct in the entity definition, streamlining the data model.
- Removed the `FindTendersWithContentXML` method from the `TenderRepository` interface, as it is no longer necessary.
- Updated the `ToTender` method in the `NoticeWorker` to reflect these changes, ensuring consistency across the codebase.

This update simplifies the tender data structure and repository interface, improving maintainability and clarity in the tender management system.
2026-07-11 20:54:29 +03:30
Mazyar 932b0cf24e Add index for documents scraped with keyset pagination in tender repository
- Introduced a new index `documents_scraped_created_at_id_idx` in the tender repository to support keyset pagination for documents marked as scraped, sorted by creation date.
- This enhancement improves the efficiency of querying scraped documents, facilitating better performance in data retrieval and management.

This update significantly optimizes the tender repository's handling of scraped documents, enhancing overall data access capabilities.
2026-07-11 17:00:48 +03:30
Mazyar 1df44ec8ca Add translated notices scanning functionality to dashboard repository
continuous-integration/drone/push Build is passing
- Introduced a new `translatedNoticesScanner` interface for scanning MinIO for daily translated notice counts.
- Enhanced the `repository` struct to include fields for managing translated notice scope and caching.
- Implemented the `ScanTranslatedNotices` method in the `StorageClient` to retrieve daily counts and total from MinIO.
- Updated the `Statistics` method in the repository to utilize the new translated notices scope, improving data accuracy.
- Added unit tests for the translated notices functionality, ensuring robust validation of the new features.

This update significantly enhances the dashboard's ability to handle translated notice statistics, improving overall data management and reporting capabilities.
2026-07-10 17:05:38 +03:30
Mazyar 843e1508df Enhance tender search functionality with pagination and filtering improvements
continuous-integration/drone/push Build is passing
- Updated the tender search handler to include a new `include_total` query parameter, allowing clients to request total count and pagination metadata for search results.
- Refactored the `tenderSearchListProjection` to exclude heavy fields from the list response, optimizing data retrieval.
- Modified the `buildSearchFilter` method to utilize the `processing_metadata.documents_scraped` flag for filtering, improving search accuracy.
- Added unit tests for the new pagination features and search filter logic, ensuring robust validation of the search functionality.

This update significantly enhances the tender search experience by providing more flexible pagination options and improving the efficiency of data handling.
2026-07-10 16:23:46 +03:30
Mazyar 2fbf329182 Refactor tender approval logic to streamline submission mode handling
continuous-integration/drone/push Build is passing
- Removed the resetting of `SubmissionMode` in the `Reject` method of the `TenderApproval` entity, ensuring that the submission mode is preserved upon rejection.
- Updated the `GetCompanyTenderApprovalStats` method in the repository to count submissions by mode without filtering by status, enhancing the accuracy of statistics.
- Adjusted the `ToggleTenderApproval` method in the service to set the `SubmissionMode` when rejecting a tender, ensuring consistent handling of submission modes across approval states.

This update improves the clarity and functionality of tender approval processes, ensuring that submission modes are correctly managed during approval and rejection scenarios.
2026-07-08 03:02:57 +03:30
Mazyar 81a94b4879 Enhance tender and company context handling with new features and error management
continuous-integration/drone/push Build is passing
- Introduced `PickAssignedCompanyID` function to determine the appropriate company ID for company-scoped operations, improving company assignment logic.
- Added `ResolveMongoID` method in the tender service to map MongoDB IDs and AI procedure references to canonical tender IDs, enhancing ID resolution.
- Updated `ToggleTenderApproval` to handle company ID validation and improve error handling, ensuring robust processing of tender approvals.
- Enhanced logging for error scenarios in the tender approval service, providing better insights into failures during operations.
- Refactored the `GetByID` and `GetByTenderAndCompany` methods in the tender approval repository to utilize custom error types for improved clarity.

This update significantly improves the handling of company and tender operations, enhancing error management and overall service reliability.
2026-07-08 02:55:55 +03:30
Mazyar 11e0b44ebe Add cache staleness check for recommendation page
continuous-integration/drone/push Build is passing
- Introduced a new method `isRecommendedPageCacheStale` in the `tenderService` to determine if the cached recommendations for a company are outdated based on the count of cached recommendations and the latest AI recommendations.
- Updated the `recommendFromPageCache` method to utilize the new cache staleness check, ensuring that outdated caches are not used for recommendations.
- This enhancement improves the accuracy of recommendations by ensuring that users receive the most up-to-date information, thereby enhancing the overall user experience in the tender recommendation service.
2026-07-08 02:41:00 +03:30
Mazyar f68b6d7787 Refactor recommendation response handling and enhance enrichment logic
continuous-integration/drone/push Build is passing
- Removed the `buildRecommendedTenderResponsesParallel` method and replaced it with `buildRecommendedTenderListResponses`, simplifying the response building process.
- Introduced `enrichRecommendedTenderResponsesParallel` to handle enrichment of tender responses in parallel, improving performance.
- Updated the `Recommend` method to utilize the new enrichment logic, ensuring timely and efficient processing of recommendations.
- Cleaned up unused functions and imports in `recommendation_page_cache.go` and `recommendation_response.go`, enhancing code clarity and maintainability.

This update streamlines the recommendation response handling and improves the overall efficiency of the tender recommendation service.
2026-07-08 02:27:25 +03:30
Mazyar 784c3d6563 Refactor recommendation page cache handling and configuration
continuous-integration/drone/push Build is passing
- Removed the `RecommendationPageCacheLanguages` configuration from `AISummarizerConfig` to streamline cache management.
- Updated the `companyService` and `tenderService` to utilize the new `InvalidateRecommendedTendersPageCache` method for cache invalidation, enhancing clarity and efficiency.
- Refactored the `invalidateRecommendedTendersPageCache` method to eliminate unnecessary context parameters, simplifying the function signature.
- Improved the handling of page cache refresh logic by consolidating language handling within the `tenderService`, ensuring consistent behavior across services.
- Cleaned up related tests and removed deprecated functions to maintain code quality and readability.

This update enhances the maintainability of the recommendation caching system by simplifying configuration and improving cache invalidation logic.
2026-07-08 02:20:16 +03:30
Mazyar 0b74e9ad23 Enhance tender recommendation caching and page refresh logic
continuous-integration/drone/push Build is passing
- Introduced a new `RecommendedTendersPageCacheRefresher` interface to manage the asynchronous refresh of recommendation pages in Redis, improving cache management.
- Updated the `companyService` to support setting page cache languages and refreshing the recommended tenders page cache based on company IDs.
- Enhanced the `tenderService` to build and invalidate recommended tenders page caches, ensuring timely updates and efficient retrieval of cached recommendations.
- Added configuration options for recommendation page cache languages in the `AISummarizerConfig`, allowing for flexible language support.
- Implemented unit tests for the new caching logic and page refresh functionality, ensuring robust validation of the recommendation caching process.

This update significantly improves the efficiency and responsiveness of the tender recommendation service by integrating enhanced caching mechanisms and page refresh capabilities.
2026-07-08 02:05:36 +03:30
Mazyar b28bc23975 Enhance tender reference handling with improved deduplication and test coverage
- Updated the `markResolvedRecommendedTenderSeen` function to accept an additional `tenderRef` parameter, allowing for better tracking of tender references and preventing duplicates.
- Refactored the `GetByProcedureReferences` method in the repository to combine indexed notice lookups with contract-folder candidates, ensuring older notice references resolve correctly after tender merges.
- Introduced a new `findTendersByContractFolderIDs` method to streamline the retrieval of tenders by contract folder IDs, enhancing performance and maintainability.
- Added unit tests for new functionality in `ai_reference_test.go` and `recommendation_filter_test.go`, ensuring robust validation of tender reference mapping and deduplication logic.

This update significantly improves the handling of tender references and enhances the overall test coverage, ensuring more reliable and efficient processing of tender data.
2026-07-08 01:55:33 +03:30
53 changed files with 4467 additions and 251 deletions
+29 -7
View File
@@ -119,6 +119,7 @@ import (
"tm/internal/notification"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/internal/tender_submission"
"tm/internal/user"
"tm/pkg/filestore"
"tm/pkg/security"
@@ -201,6 +202,7 @@ func main() {
tenderRepository := tender.NewRepository(mongoManager, logger)
feedbackRepo := feedback.NewRepository(mongoManager, logger)
tenderApprovalRepository := tender_approval.NewRepository(mongoManager, logger)
tenderSubmissionRepository := tender_submission.NewRepository(mongoManager, logger)
inquiryRepository := inquiry.NewRepository(mongoManager, logger)
contactRepository := contact.NewContactRepository(mongoManager, logger)
cmsRepository := cms.NewRepository(mongoManager, logger)
@@ -210,7 +212,7 @@ func main() {
cardRepository := kanban.NewCardRepository(mongoManager, logger)
notificationRepository := notification.NewRepository(mongoManager, logger)
logger.Info("Repositories initialized successfully", map[string]interface{}{
"repositories": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "contact", "cms", "notice", "kanban_board", "kanban_column", "kanban_card", "notification"},
"repositories": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "contact", "cms", "notice", "kanban_board", "kanban_column", "kanban_card", "notification"},
})
// Initialize validation services
@@ -242,9 +244,28 @@ func main() {
categoryService := company_category.NewService(categoryRepository, logger)
companyService := company.NewService(companyRepository, categoryService, fileStoreService, aiRecommendationClient, aiPipelineClient, redisClient, conf.AISummarizer.RecommendationCacheTTL, logger)
customerService := customer.NewService(customerRepository, logger, customerAuthService, companyService, redisClient, notificationSDK, customerValidator, auditLogger)
tenderService := tender.NewService(tenderRepository, companyService, customerService, tenderApprovalRepository, logger, aiSummarizerClient, aiSummarizerStorage, conf.AISummarizer.DefaultLanguage)
tenderService := tender.NewService(
tenderRepository,
companyService,
customerService,
tenderApprovalRepository,
logger,
aiSummarizerClient,
aiSummarizerStorage,
redisClient,
conf.AISummarizer.RecommendationCacheTTL,
conf.AISummarizer.DefaultLanguage,
)
if configurer, ok := companyService.(interface {
SetRecommendedTendersPageCacheRefresher(company.RecommendedTendersPageCacheRefresher)
}); ok {
if refresher, ok := tenderService.(company.RecommendedTendersPageCacheRefresher); ok {
configurer.SetRecommendedTendersPageCacheRefresher(refresher)
}
}
feedbackService := feedback.NewService(feedbackRepo, tenderService, companyService, logger)
tenderApprovalService := tender_approval.NewService(tenderApprovalRepository, tenderService, logger)
tenderSubmissionService := tender_submission.NewService(tenderSubmissionRepository, tenderApprovalRepository, tenderService, logger)
tenderApprovalService := tender_approval.NewService(tenderApprovalRepository, tenderService, logger, tenderSubmissionService)
inquiryService := inquiry.NewService(inquiryRepository, notificationSDK, logger, xssPolicy)
flagService := assets.NewService(logger, conf.Assets.FlagsPath)
notificationService := notification.NewService(notificationSDK, notificationRepository, userService, customerService, logger)
@@ -258,7 +279,7 @@ func main() {
auditLogRepository := auditlog.NewRepository(auditStore)
auditLogService := auditlog.NewService(auditLogRepository, logger)
logger.Info("Services initialized successfully", map[string]interface{}{
"services": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
"services": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
})
// Initialize handlers with services
@@ -269,6 +290,7 @@ func main() {
tenderHandler := tender.NewHandler(tenderService, logger)
feedbackHandler := feedback.NewHandler(feedbackService, userHandler, customerHandler, logger)
tenderApprovalHandler := tender_approval.NewHandler(tenderApprovalService, logger)
tenderSubmissionHandler := tender_submission.NewHandler(tenderSubmissionService, logger)
inquiryHandler := inquiry.NewHandler(inquiryService, logger, hcaptchaVerifier, xssPolicy)
flagHandler := assets.NewHandler(flagService, logger)
notificationHandler := notification.NewHandler(notificationService)
@@ -281,7 +303,7 @@ func main() {
aiPipelineHandler := ai_pipeline.NewHandler(aiPipelineService)
auditLogHandler := auditlog.NewHandler(auditLogService)
logger.Info("Handlers initialized successfully", map[string]interface{}{
"handlers": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "filestore", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
"handlers": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "filestore", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
})
// Initialize HTTP server
@@ -290,8 +312,8 @@ func main() {
router.SetupCSPSecurity(e)
// Register routes
router.RegisterAdminRoutes(e, userHandler, companyHandler, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, dashboardHandler, aiPipelineHandler, auditLogHandler, xssPolicy)
router.RegisterPublicRoutes(e, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, companyHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, xssPolicy)
router.RegisterAdminRoutes(e, userHandler, companyHandler, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, tenderSubmissionHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, dashboardHandler, aiPipelineHandler, auditLogHandler, xssPolicy)
router.RegisterPublicRoutes(e, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, tenderSubmissionHandler, companyHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, xssPolicy)
router.RegisterDocumentScraperRoutes(e, documentScraperHandler, conf.DocumentScraper.APIKey)
// Start server
+24 -2
View File
@@ -17,6 +17,7 @@ import (
"tm/internal/notification"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/internal/tender_submission"
"tm/internal/user"
"tm/pkg/filestore"
"tm/pkg/security"
@@ -36,7 +37,7 @@ func SetupCSPSecurity(e *echo.Echo) {
e.POST("/api/v1/csp-report", security.CSPReportHandler)
}
func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler *company.Handler, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, dashboardHandler *dashboard.Handler, aiPipelineHandler *ai_pipeline.Handler, auditLogHandler *auditlog.Handler, xssPolicy *security.XSSPolicy) {
func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler *company.Handler, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, tenderSubmissionHandler *tender_submission.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, dashboardHandler *dashboard.Handler, aiPipelineHandler *ai_pipeline.Handler, auditLogHandler *auditlog.Handler, xssPolicy *security.XSSPolicy) {
adminV1 := e.Group("/admin/v1")
// Admin Users Routes
@@ -156,6 +157,15 @@ func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler
tenderApprovalGP.GET("/status/:status", tenderApprovalHandler.GetTenderApprovalsByStatus)
}
// Admin Tender-Submissions Routes
tenderSubmissionGP := adminV1.Group("/tender-submissions")
{
tenderSubmissionGP.Use(userHandler.AuthMiddleware())
tenderSubmissionGP.GET("", tenderSubmissionHandler.AdminListSubmissions)
tenderSubmissionGP.GET("/stats", tenderSubmissionHandler.AdminGetSubmissionStats)
tenderSubmissionGP.GET("/:id", tenderSubmissionHandler.AdminGetSubmissionByID)
}
// Admin Inquiry Routes
inquiryGP := adminV1.Group("/inquiries")
{
@@ -260,7 +270,7 @@ func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler
}
}
func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, companyHandler *company.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, xssPolicy *security.XSSPolicy) {
func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, tenderSubmissionHandler *tender_submission.Handler, companyHandler *company.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, xssPolicy *security.XSSPolicy) {
v1 := e.Group("/api/v1")
customerGP := v1.Group("/profile")
@@ -317,6 +327,18 @@ func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tende
tenderApprovalGP.GET("/stats", tenderApprovalHandler.GetCompanyTenderApprovalStats)
}
// Public Tender Submissions Routes
tenderSubmissionGP := v1.Group("/tender-submissions")
{
tenderSubmissionGP.Use(customerHandler.AuthMiddleware(), customerHandler.CompanyContextMiddleware())
tenderSubmissionGP.GET("", tenderSubmissionHandler.ListCompanySubmissions)
tenderSubmissionGP.GET("/stats", tenderSubmissionHandler.GetCompanySubmissionStats)
tenderSubmissionGP.POST("/tender/:tender_id/ensure", tenderSubmissionHandler.EnsureSubmission)
tenderSubmissionGP.GET("/tender/:tender_id", tenderSubmissionHandler.GetSubmissionByTender)
tenderSubmissionGP.PATCH("/:id/status", tenderSubmissionHandler.UpdateSubmissionStatus)
tenderSubmissionGP.GET("/:id", tenderSubmissionHandler.GetSubmissionByID)
}
// Public Company Routes
companiesGP := v1.Group("/companies")
{
+13 -13
View File
@@ -47,17 +47,17 @@ type WorkerConfig struct {
// AISummarizerConfig holds configuration for the external AI summarizer service.
type AISummarizerConfig struct {
APIBaseURL string `env:"AI_SUMMARIZER_API_BASE_URL" envDefault:""`
APITimeout time.Duration `env:"AI_SUMMARIZER_API_TIMEOUT" envDefault:"120s"`
APIRetryCount int `env:"AI_SUMMARIZER_API_RETRY_COUNT" envDefault:"2"`
APIRetryDelay time.Duration `env:"AI_SUMMARIZER_API_RETRY_DELAY" envDefault:"3s"`
DefaultLanguage string `env:"AI_SUMMARIZER_DEFAULT_LANGUAGE" envDefault:"en"`
TranslationLanguages string `env:"AI_SUMMARIZER_TRANSLATION_LANGUAGES" envDefault:"en"`
MinioEndpoint string `env:"AI_SUMMARIZER_MINIO_ENDPOINT" envDefault:""`
MinioAccessKey string `env:"AI_SUMMARIZER_MINIO_ACCESS_KEY" envDefault:""`
MinioSecretKey string `env:"AI_SUMMARIZER_MINIO_SECRET_KEY" envDefault:""`
MinioUseSSL bool `env:"AI_SUMMARIZER_MINIO_USE_SSL" envDefault:"false"`
MinioRegion string `env:"AI_SUMMARIZER_MINIO_REGION" envDefault:"us-east-1"`
MinioBucket string `env:"AI_SUMMARIZER_MINIO_BUCKET" envDefault:"opplens"`
RecommendationCacheTTL time.Duration `env:"AI_RECOMMENDATION_CACHE_TTL" envDefault:"0"`
APIBaseURL string `env:"AI_SUMMARIZER_API_BASE_URL" envDefault:""`
APITimeout time.Duration `env:"AI_SUMMARIZER_API_TIMEOUT" envDefault:"120s"`
APIRetryCount int `env:"AI_SUMMARIZER_API_RETRY_COUNT" envDefault:"2"`
APIRetryDelay time.Duration `env:"AI_SUMMARIZER_API_RETRY_DELAY" envDefault:"3s"`
DefaultLanguage string `env:"AI_SUMMARIZER_DEFAULT_LANGUAGE" envDefault:"en"`
TranslationLanguages string `env:"AI_SUMMARIZER_TRANSLATION_LANGUAGES" envDefault:"en"`
MinioEndpoint string `env:"AI_SUMMARIZER_MINIO_ENDPOINT" envDefault:""`
MinioAccessKey string `env:"AI_SUMMARIZER_MINIO_ACCESS_KEY" envDefault:""`
MinioSecretKey string `env:"AI_SUMMARIZER_MINIO_SECRET_KEY" envDefault:""`
MinioUseSSL bool `env:"AI_SUMMARIZER_MINIO_USE_SSL" envDefault:"false"`
MinioRegion string `env:"AI_SUMMARIZER_MINIO_REGION" envDefault:"us-east-1"`
MinioBucket string `env:"AI_SUMMARIZER_MINIO_BUCKET" envDefault:"opplens"`
RecommendationCacheTTL time.Duration `env:"AI_RECOMMENDATION_CACHE_TTL" envDefault:"0"`
}
-1
View File
@@ -565,7 +565,6 @@ func (w *NoticeWorker) ToTender(n *notice.Notice) (*tender.Tender, error) {
t.ProcurementProjectID = n.ProcurementProjectID
t.SourceFileURL = n.SourceFileURL
t.SourceFileName = n.SourceFileName
t.ContentXML = n.ContentXML
// Preserve document-scraping/summarization flags on the tender (managed by
// downstream workers), but refresh the notice-side processing metadata.
t.ProcessingMetadata.ScrapedAt = n.ProcessingMetadata.ScrapedAt
+8
View File
@@ -0,0 +1,8 @@
package company
// RecommendedTendersPageCacheRefresher pre-builds resolved recommendation pages in Redis.
// Implemented by the tender service on the web API; optional on the worker.
type RecommendedTendersPageCacheRefresher interface {
ScheduleRefreshRecommendedTendersPageCache(companyID string)
InvalidateRecommendedTendersPageCache(companyID string)
}
+5
View File
@@ -464,7 +464,11 @@ func (s *companyService) cacheAIRecommendations(ctx context.Context, companyID s
"company_id": companyID,
"error": err.Error(),
})
return
}
s.invalidateRecommendedTendersPageCache(companyID)
s.scheduleRecommendedTendersPageCacheRefresh(companyID)
}
func (s *companyService) recommendationCacheExpiration() time.Duration {
@@ -485,6 +489,7 @@ func (s *companyService) invalidateAIRecommendationCache(ctx context.Context, co
"error": err.Error(),
})
}
s.invalidateRecommendedTendersPageCache(companyID)
}
func (s *companyService) buildOnboardingDocuments(fileIDs []string) (string, error) {
@@ -0,0 +1,20 @@
package company
func (s *companyService) invalidateRecommendedTendersPageCache(companyID string) {
if s.pageCacheRefresher == nil {
return
}
s.pageCacheRefresher.InvalidateRecommendedTendersPageCache(companyID)
}
func (s *companyService) scheduleRecommendedTendersPageCacheRefresh(companyID string) {
if s.pageCacheRefresher == nil {
return
}
s.pageCacheRefresher.ScheduleRefreshRecommendedTendersPageCache(companyID)
}
// SetRecommendedTendersPageCacheRefresher wires async page-cache rebuilds from the tender service.
func (s *companyService) SetRecommendedTendersPageCacheRefresher(refresher RecommendedTendersPageCacheRefresher) {
s.pageCacheRefresher = refresher
}
+1
View File
@@ -60,6 +60,7 @@ type companyService struct {
fileStore filestore.FileStoreService
aiRecommendationClient AIRecommendationClient
aiPipelineStatusClient AIPipelineStatusClient
pageCacheRefresher RecommendedTendersPageCacheRefresher
redisClient redis.Client
recommendationCacheTTL time.Duration
logger logger.Logger
+25 -1
View File
@@ -6,7 +6,10 @@ import (
"strings"
)
var errCompanyNotAssigned = errors.New("company is not assigned to customer")
// ErrCompanyNotAssigned is returned when the requested company is not assigned to the customer.
var ErrCompanyNotAssigned = errors.New("company is not assigned to customer")
var errCompanyNotAssigned = ErrCompanyNotAssigned
func normalizeAssignedCompanyIDs(assigned []string) []string {
normalized := make([]string, 0, len(assigned))
@@ -56,6 +59,27 @@ func pickActiveCompanyID(assigned []string, tokenCompanyID, requestedCompanyID s
return assigned[0], nil
}
// PickAssignedCompanyID chooses the company for a company-scoped write (e.g. tender approval).
// When requestedCompanyID is set it must be in assigned; otherwise activeCompanyID is used.
func PickAssignedCompanyID(activeCompanyID string, assigned []string, requestedCompanyID string) (string, error) {
activeCompanyID = strings.TrimSpace(activeCompanyID)
requestedCompanyID = strings.TrimSpace(requestedCompanyID)
if requestedCompanyID != "" {
for _, id := range normalizeAssignedCompanyIDs(assigned) {
if id == requestedCompanyID {
return requestedCompanyID, nil
}
}
return "", ErrCompanyNotAssigned
}
if activeCompanyID == "" {
return "", errors.New("company ID is required")
}
return activeCompanyID, nil
}
// ResolveCompanyContext resolves the active company and assigned companies for an authenticated customer.
func (s *customerService) ResolveCompanyContext(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, []string, error) {
customer, err := s.repository.GetByID(ctx, customerID)
+46
View File
@@ -27,6 +27,11 @@ type scrapedDocumentsScanner interface {
ScanScrapedDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, map[string]int64, error)
}
// translatedNoticesScanner optionally scans MinIO for per-day translated notice counts.
type translatedNoticesScanner interface {
ScanTranslatedNotices(ctx context.Context) (map[string]int64, int64, error)
}
const defaultValueCurrency = "EUR"
// Repository defines dashboard data access.
@@ -51,6 +56,11 @@ type repository struct {
scrapedScopeExpiry time.Time
scrapedScopeCached bool
scrapedScopeGroup singleflight.Group
translatedScopeMu sync.Mutex
translatedScope translatedNoticesScope
translatedScopeExpiry time.Time
translatedScopeCached bool
translatedScopeGroup singleflight.Group
}
// NewRepository creates a dashboard repository backed by the tenders collection.
@@ -431,6 +441,22 @@ func (r *repository) ClosingSoon(ctx context.Context, limit int, windowSec int64
windowEnd := now + windowSec
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{
"$or": bson.A{
bson.M{
"$and": bson.A{
bson.M{"submission_deadline": bson.M{"$gt": 0}},
deadlineWithinWindowMatch("submission_deadline", now, windowEnd),
},
},
bson.M{
"$and": bson.A{
noSubmissionDeadlineClause(),
deadlineWithinWindowMatch("tender_deadline", now, windowEnd),
},
},
},
}}},
{{Key: "$addFields", Value: bson.M{"effective_deadline": effectiveDeadlineExpr()}}},
{{Key: "$match", Value: bson.M{
"effective_deadline": bson.M{"$gt": now, "$lte": windowEnd},
@@ -570,6 +596,26 @@ func trendTimestampField(metric string) string {
}
}
// deadlineWithinWindowMatch matches raw deadline fields stored as Unix seconds or
// milliseconds. normalizeTimestampExpr treats values > 1e12 as ms; indexed pre-filters
// must accept both encodings to stay equivalent to filtering on normalized deadlines.
func deadlineWithinWindowMatch(field string, now, windowEnd int64) bson.M {
return bson.M{
"$or": bson.A{
bson.M{field: bson.M{"$gt": now, "$lte": windowEnd}},
bson.M{field: bson.M{"$gt": now * 1000, "$lte": windowEnd * 1000}},
},
}
}
func noSubmissionDeadlineClause() bson.M {
return bson.M{"$or": bson.A{
bson.M{"submission_deadline": bson.M{"$exists": false}},
bson.M{"submission_deadline": nil},
bson.M{"submission_deadline": bson.M{"$lte": 0}},
}}
}
// effectiveDeadlineExpr is used for closing-soon (submission first, per dashboard-api.md).
func effectiveDeadlineExpr() bson.M {
return normalizeDeadlineFieldExpr(
+27
View File
@@ -42,3 +42,30 @@ func TestFacetCurrencyValueDecodesBSOND(t *testing.T) {
t.Fatalf("expected 12345.67, got %f", total)
}
}
func TestDeadlineWithinWindowMatchCoversSecondsAndMilliseconds(t *testing.T) {
now := int64(1_700_000_000)
windowEnd := now + 86_400
match := deadlineWithinWindowMatch("submission_deadline", now, windowEnd)
orClause, ok := match["$or"].(bson.A)
if !ok || len(orClause) != 2 {
t.Fatalf("expected two range branches, got %#v", match)
}
secRange, ok := orClause[0].(bson.M)["submission_deadline"].(bson.M)
if !ok {
t.Fatalf("expected seconds range on submission_deadline, got %#v", orClause[0])
}
if secRange["$gt"] != now || secRange["$lte"] != windowEnd {
t.Fatalf("unexpected seconds range: %#v", secRange)
}
msRange, ok := orClause[1].(bson.M)["submission_deadline"].(bson.M)
if !ok {
t.Fatalf("expected milliseconds range on submission_deadline, got %#v", orClause[1])
}
if msRange["$gt"] != now*1000 || msRange["$lte"] != windowEnd*1000 {
t.Fatalf("unexpected milliseconds range: %#v", msRange)
}
}
+185 -68
View File
@@ -60,19 +60,30 @@ type service struct {
statisticsMu sync.Mutex
statistics map[int]cacheEntry[*StatisticsReportResponse]
statisticsGroup singleflight.Group
trendCache *widgetCache[*TrendResponse]
countriesCache *widgetCache[*CountriesResponse]
noticeTypesCache *widgetCache[*NoticeTypesResponse]
closingSoonCache *widgetCache[*ClosingSoonResponse]
recentCache *widgetCache[*RecentResponse]
}
// NewService creates a dashboard service.
func NewService(repo Repository, log logger.Logger, redisClient redis.Client) Service {
s := &service{
repo: repo,
logger: log,
redis: redisClient,
summary: make(map[string]cacheEntry[*SummaryResponse]),
statistics: make(map[int]cacheEntry[*StatisticsReportResponse]),
repo: repo,
logger: log,
redis: redisClient,
summary: make(map[string]cacheEntry[*SummaryResponse]),
statistics: make(map[int]cacheEntry[*StatisticsReportResponse]),
trendCache: newWidgetCache[*TrendResponse](redisClient, log, "trend"),
countriesCache: newWidgetCache[*CountriesResponse](redisClient, log, "countries"),
noticeTypesCache: newWidgetCache[*NoticeTypesResponse](redisClient, log, "notice-types"),
closingSoonCache: newWidgetCache[*ClosingSoonResponse](redisClient, log, "closing-soon"),
recentCache: newWidgetCache[*RecentResponse](redisClient, log, "recent"),
}
go s.warmStatisticsCache()
go s.warmSummaryCache()
go s.warmWidgetCaches()
return s
}
@@ -129,102 +140,134 @@ func (s *service) Summary(ctx context.Context, query SummaryQuery) (*SummaryResp
func (s *service) Trend(ctx context.Context, query TrendQuery) (*TrendResponse, error) {
days := trendDays(query.Days)
metric := normalizeTrendMetric(query.Metric)
startUnix := trendStartUnix(days)
cacheKey := fmt.Sprintf("%d:%s", days, metric)
s.logger.Info("Fetching dashboard trend", map[string]interface{}{
"days": days,
"metric": metric,
})
return s.trendCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*TrendResponse, error) {
startUnix := trendStartUnix(days)
counts, err := s.repo.Trend(ctx, days, metric, startUnix)
if err != nil {
s.logger.Error("Failed to fetch dashboard trend", map[string]interface{}{
"error": err.Error(),
s.logger.Info("Fetching dashboard trend", map[string]interface{}{
"days": days,
"metric": metric,
})
return nil, fmt.Errorf("dashboard trend: %w", err)
}
series := fillTrendSeries(days, counts)
counts, err := s.repo.Trend(loadCtx, days, metric, startUnix)
if err != nil {
s.logger.Error("Failed to fetch dashboard trend", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard trend: %w", err)
}
return &TrendResponse{
Metric: metric,
Days: days,
Series: series,
}, nil
return &TrendResponse{
Metric: metric,
Days: days,
Series: fillTrendSeries(days, counts),
}, nil
})
}
func (s *service) Countries(ctx context.Context, query CountriesQuery) (*CountriesResponse, error) {
limit := countriesLimit(query.Limit)
cacheKey := strconv.Itoa(limit)
s.logger.Info("Fetching dashboard countries", map[string]interface{}{
"limit": limit,
})
out, err := s.repo.Countries(ctx, limit)
if err != nil {
s.logger.Error("Failed to fetch dashboard countries", map[string]interface{}{
"error": err.Error(),
return s.countriesCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*CountriesResponse, error) {
s.logger.Info("Fetching dashboard countries", map[string]interface{}{
"limit": limit,
})
return nil, fmt.Errorf("dashboard countries: %w", err)
}
return out, nil
out, err := s.repo.Countries(loadCtx, limit)
if err != nil {
s.logger.Error("Failed to fetch dashboard countries", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard countries: %w", err)
}
return out, nil
})
}
func (s *service) NoticeTypes(ctx context.Context) (*NoticeTypesResponse, error) {
s.logger.Info("Fetching dashboard notice types", nil)
return s.noticeTypesCache.Get(ctx, "all", func(loadCtx context.Context) (*NoticeTypesResponse, error) {
s.logger.Info("Fetching dashboard notice types", nil)
out, err := s.repo.NoticeTypes(ctx)
if err != nil {
s.logger.Error("Failed to fetch dashboard notice types", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard notice types: %w", err)
}
out, err := s.repo.NoticeTypes(loadCtx)
if err != nil {
s.logger.Error("Failed to fetch dashboard notice types", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard notice types: %w", err)
}
return out, nil
return out, nil
})
}
func (s *service) ClosingSoon(ctx context.Context, query ClosingSoonQuery) (*ClosingSoonResponse, error) {
limit := listLimit(query.Limit)
windowHours := closingWindowHours(query.Window)
windowSec := int64(windowHours) * 3600
cacheKey := fmt.Sprintf("%d:%d", limit, windowSec)
s.logger.Info("Fetching dashboard closing soon", map[string]interface{}{
"limit": limit,
"window": windowHours,
})
items, err := s.repo.ClosingSoon(ctx, limit, windowSec)
if err != nil {
s.logger.Error("Failed to fetch dashboard closing soon", map[string]interface{}{
"error": err.Error(),
return s.closingSoonCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*ClosingSoonResponse, error) {
s.logger.Info("Fetching dashboard closing soon", map[string]interface{}{
"limit": limit,
"window": windowHours,
})
return nil, fmt.Errorf("dashboard closing soon: %w", err)
}
return &ClosingSoonResponse{Items: items}, nil
items, err := s.repo.ClosingSoon(loadCtx, limit, windowSec)
if err != nil {
s.logger.Error("Failed to fetch dashboard closing soon", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard closing soon: %w", err)
}
return &ClosingSoonResponse{Items: items}, nil
})
}
func (s *service) Recent(ctx context.Context, query RecentQuery) (*RecentResponse, error) {
limit := listLimit(query.Limit)
s.logger.Info("Fetching dashboard recent tenders", map[string]interface{}{
"limit": limit,
})
items, nextCursor, err := s.repo.Recent(ctx, limit, strings.TrimSpace(query.Cursor))
if err != nil {
s.logger.Error("Failed to fetch dashboard recent tenders", map[string]interface{}{
"error": err.Error(),
cursor := strings.TrimSpace(query.Cursor)
if cursor != "" {
s.logger.Info("Fetching dashboard recent tenders", map[string]interface{}{
"limit": limit,
})
return nil, fmt.Errorf("dashboard recent: %w", err)
items, nextCursor, err := s.repo.Recent(ctx, limit, cursor)
if err != nil {
s.logger.Error("Failed to fetch dashboard recent tenders", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard recent: %w", err)
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
cacheKey := strconv.Itoa(limit)
return s.recentCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*RecentResponse, error) {
s.logger.Info("Fetching dashboard recent tenders", map[string]interface{}{
"limit": limit,
})
items, nextCursor, err := s.repo.Recent(loadCtx, limit, "")
if err != nil {
s.logger.Error("Failed to fetch dashboard recent tenders", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard recent: %w", err)
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
})
}
func (s *service) Statistics(ctx context.Context, query StatisticsQuery) (*StatisticsReportResponse, error) {
@@ -339,6 +382,80 @@ func (s *service) loadStatistics(ctx context.Context, days int) (*StatisticsRepo
return out.(*StatisticsReportResponse), nil
}
func (s *service) warmWidgetCaches() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
defaultWindowSec := int64(defaultClosingWindowHours) * 3600
warmed := 0
if s.trendCache.WarmFromRedis(ctx, fmt.Sprintf("%d:%s", defaultTrendDays, "created")) {
warmed++
}
if s.countriesCache.WarmFromRedis(ctx, strconv.Itoa(defaultCountriesLimit)) {
warmed++
}
if s.noticeTypesCache.WarmFromRedis(ctx, "all") {
warmed++
}
if s.closingSoonCache.WarmFromRedis(ctx, fmt.Sprintf("%d:%d", defaultListLimit, defaultWindowSec)) {
warmed++
}
if s.recentCache.WarmFromRedis(ctx, strconv.Itoa(defaultListLimit)) {
warmed++
}
if warmed > 0 {
s.logger.Info("Dashboard widget caches warmed from Redis", map[string]interface{}{
"warmed": warmed,
})
}
go s.trendCache.Reload(fmt.Sprintf("%d:%s", defaultTrendDays, "created"), func(loadCtx context.Context) (*TrendResponse, error) {
return s.loadTrend(loadCtx, TrendQuery{Days: defaultTrendDays, Metric: "created"})
})
go s.countriesCache.Reload(strconv.Itoa(defaultCountriesLimit), func(loadCtx context.Context) (*CountriesResponse, error) {
return s.repo.Countries(loadCtx, defaultCountriesLimit)
})
go s.noticeTypesCache.Reload("all", func(loadCtx context.Context) (*NoticeTypesResponse, error) {
return s.repo.NoticeTypes(loadCtx)
})
go s.closingSoonCache.Reload(fmt.Sprintf("%d:%d", defaultListLimit, defaultWindowSec), func(loadCtx context.Context) (*ClosingSoonResponse, error) {
items, err := s.repo.ClosingSoon(loadCtx, defaultListLimit, defaultWindowSec)
if err != nil {
return nil, err
}
return &ClosingSoonResponse{Items: items}, nil
})
go s.recentCache.Reload(strconv.Itoa(defaultListLimit), func(loadCtx context.Context) (*RecentResponse, error) {
items, nextCursor, err := s.repo.Recent(loadCtx, defaultListLimit, "")
if err != nil {
return nil, err
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
})
}
func (s *service) loadTrend(ctx context.Context, query TrendQuery) (*TrendResponse, error) {
days := trendDays(query.Days)
metric := normalizeTrendMetric(query.Metric)
startUnix := trendStartUnix(days)
counts, err := s.repo.Trend(ctx, days, metric, startUnix)
if err != nil {
return nil, err
}
return &TrendResponse{
Metric: metric,
Days: days,
Series: fillTrendSeries(days, counts),
}, nil
}
func (s *service) warmSummaryCache() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
+96 -11
View File
@@ -33,6 +33,13 @@ type scrapedTendersScope struct {
totalTenderCount int64 // >0 when MinIO procedure count should be used instead of Mongo Count
}
// translatedNoticesScope holds MinIO-backed translation statistics when available.
type translatedNoticesScope struct {
fromMinIO bool
dailyCounts map[string]int64
totalCount int64
}
func (r *repository) Statistics(ctx context.Context, days int, startUnix int64) (*StatisticsReportResponse, error) {
_ = ctx
@@ -47,6 +54,7 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
totalTranslated int64
totalScrapedTED int64
scrapedScope scrapedTendersScope
translatedScope translatedNoticesScope
)
var wg sync.WaitGroup
@@ -78,23 +86,26 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
wg.Add(1)
go func() {
defer wg.Done()
scope, err := r.cachedTranslatedNoticesScope(context.Background())
if err != nil {
recordFailure("translated_scope", err)
return
}
translatedScope = scope
if scope.fromMinIO {
return
}
qctx, cancel := context.WithTimeout(context.Background(), statisticsMongoQueryTimeout)
defer cancel()
counts, err := r.translatedNoticesPerDay(qctx, startDay, endDay)
counts, err := r.translatedNoticesPerDayFromMetrics(qctx, startDay, endDay)
if err != nil {
recordFailure("translated_notices", err)
translatedNotices = map[string]int64{}
return
} else {
translatedNotices = counts
}
translatedNotices = counts
}()
wg.Add(1)
go func() {
defer wg.Done()
qctx, cancel := context.WithTimeout(context.Background(), statisticsMongoQueryTimeout)
defer cancel()
total, err := r.metricsCounter.Get(qctx, orm.AITranslationSuccessCounterKey)
if err != nil {
@@ -118,6 +129,11 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
wg.Wait()
if translatedScope.fromMinIO {
translatedNotices = filterDailyCountsSince(translatedScope.dailyCounts, startUnix)
totalTranslated = translatedScope.totalCount
}
wg.Add(1)
go func() {
defer wg.Done()
@@ -224,10 +240,79 @@ func (r *repository) scrapedDocumentsPerDayFromMongo(ctx context.Context, startU
return decodeDailyCounts(ctx, cursor)
}
func (r *repository) translatedNoticesPerDay(ctx context.Context, startDay, endDay time.Time) (map[string]int64, error) {
func (r *repository) translatedNoticesPerDayFromMetrics(ctx context.Context, startDay, endDay time.Time) (map[string]int64, error) {
return r.metricsCounter.GetDailyCounts(ctx, startDay, endDay)
}
func (r *repository) cachedTranslatedNoticesScope(_ context.Context) (translatedNoticesScope, error) {
now := time.Now()
r.translatedScopeMu.Lock()
if r.translatedScopeCached && now.Before(r.translatedScopeExpiry) {
scope := r.translatedScope
r.translatedScopeMu.Unlock()
return scope, nil
}
r.translatedScopeMu.Unlock()
v, err, _ := r.translatedScopeGroup.Do("translated-scope", func() (interface{}, error) {
r.translatedScopeMu.Lock()
if r.translatedScopeCached && time.Now().Before(r.translatedScopeExpiry) {
scope := r.translatedScope
r.translatedScopeMu.Unlock()
return scope, nil
}
r.translatedScopeMu.Unlock()
resolveCtx, cancel := context.WithTimeout(context.Background(), scrapedScopeResolveTimeout)
defer cancel()
scope, resolveErr := r.resolveTranslatedNoticesScope(resolveCtx)
if resolveErr != nil {
if r.logger != nil {
r.logger.Warn("MinIO translation scope unavailable, using metrics counter fallback for dashboard statistics", map[string]interface{}{
"error": resolveErr.Error(),
})
}
return translatedNoticesScope{}, nil
}
r.translatedScopeMu.Lock()
r.translatedScope = scope
r.translatedScopeExpiry = time.Now().Add(scrapedScopeCacheTTL)
r.translatedScopeCached = true
r.translatedScopeMu.Unlock()
return scope, nil
})
if err != nil {
return translatedNoticesScope{}, err
}
return v.(translatedNoticesScope), nil
}
func (r *repository) resolveTranslatedNoticesScope(ctx context.Context) (translatedNoticesScope, error) {
if r.procedureLister == nil {
return translatedNoticesScope{}, nil
}
scanner, ok := r.procedureLister.(translatedNoticesScanner)
if !ok {
return translatedNoticesScope{}, nil
}
dailyCounts, total, err := scanner.ScanTranslatedNotices(ctx)
if err != nil {
return translatedNoticesScope{}, err
}
return translatedNoticesScope{
fromMinIO: true,
dailyCounts: dailyCounts,
totalCount: total,
}, nil
}
func (r *repository) totalScrapedTenders(ctx context.Context, scope scrapedTendersScope) (int64, error) {
if scope.zero {
return 0, nil
@@ -29,6 +29,10 @@ type mockProcedureDocumentsLister struct {
procedures []ai_summarizer.ProcedureDocumentsSummary
dailyCounts map[string]int64
err error
translatedDailyCounts map[string]int64
translatedTotal int64
translatedErr error
}
func (m *mockProcedureDocumentsLister) ListProceduresWithDocuments(context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
@@ -45,6 +49,13 @@ func (m *mockProcedureDocumentsLister) ScanScrapedDocuments(context.Context) ([]
return m.procedures, m.dailyCounts, nil
}
func (m *mockProcedureDocumentsLister) ScanTranslatedNotices(context.Context) (map[string]int64, int64, error) {
if m.translatedErr != nil {
return nil, 0, m.translatedErr
}
return m.translatedDailyCounts, m.translatedTotal, nil
}
func TestScrapedDocumentsFolderIDsUsesMinIOProcedures(t *testing.T) {
repo := &repository{
procedureLister: &mockProcedureDocumentsLister{
@@ -259,3 +270,41 @@ func TestMongoScrapedTendersScopeMatchesStatisticsFilter(t *testing.T) {
t.Fatalf("expected documents_scraped filter, got %#v", scope.match)
}
}
func TestResolveTranslatedNoticesScopeUsesMinIOCounts(t *testing.T) {
repo := &repository{
procedureLister: &mockProcedureDocumentsLister{
translatedDailyCounts: map[string]int64{
"2026-07-01": 4,
"2026-07-09": 2,
},
translatedTotal: 6,
},
}
scope, err := repo.resolveTranslatedNoticesScope(context.Background())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !scope.fromMinIO {
t.Fatal("expected MinIO-backed translation scope")
}
if scope.totalCount != 6 {
t.Fatalf("expected total 6, got %d", scope.totalCount)
}
if scope.dailyCounts["2026-07-09"] != 2 {
t.Fatalf("expected 2 on 2026-07-09, got %d", scope.dailyCounts["2026-07-09"])
}
}
func TestResolveTranslatedNoticesScopeWithoutLister(t *testing.T) {
repo := &repository{}
scope, err := repo.resolveTranslatedNoticesScope(context.Background())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if scope.fromMinIO {
t.Fatal("expected metrics fallback without lister")
}
}
+230
View File
@@ -0,0 +1,230 @@
package dashboard
import (
"context"
"encoding/json"
"fmt"
"sync"
"time"
"tm/pkg/logger"
"tm/pkg/redis"
goredis "github.com/redis/go-redis/v9"
"golang.org/x/sync/singleflight"
)
const (
// widgetCacheTTL is the fresh window before a background reload is triggered.
widgetCacheTTL = 60 * time.Second
// widgetStaleGraceTTL is how long stale entries may be served while reloading.
// Dashboard widgets have no write-side invalidation; expect up to this lag after
// data changes (e.g. a new tender may not appear in cached widgets immediately).
widgetStaleGraceTTL = 5 * time.Minute
widgetReloadTimeout = 2 * time.Minute
)
type widgetLoader[T any] func(ctx context.Context) (T, error)
type widgetCache[T any] struct {
mu sync.Mutex
entries map[string]cacheEntry[T]
group singleflight.Group
redis redis.Client
logger logger.Logger
keyPrefix string
ttl time.Duration
staleGrace time.Duration
}
func newWidgetCache[T any](redisClient redis.Client, log logger.Logger, keyPrefix string) *widgetCache[T] {
return &widgetCache[T]{
entries: make(map[string]cacheEntry[T]),
redis: redisClient,
logger: log,
keyPrefix: keyPrefix,
ttl: widgetCacheTTL,
staleGrace: widgetStaleGraceTTL,
}
}
func (c *widgetCache[T]) Get(ctx context.Context, key string, load widgetLoader[T]) (T, error) {
var zero T
// Returned values are shared across concurrent callers via singleflight and stale
// hits. Callers must treat them as read-only through JSON serialization.
if value, ok := c.fresh(key); ok {
return value, nil
}
if value, ok := c.stale(key); ok {
go c.Reload(key, load)
return value, nil
}
if value, ok := c.fromRedis(ctx, key); ok {
c.store(key, value)
go c.Reload(key, load)
return value, nil
}
out, err, _ := c.group.Do(key, func() (interface{}, error) {
if value, ok := c.fresh(key); ok {
return value, nil
}
result, err := load(context.WithoutCancel(ctx))
if err != nil {
return zero, err
}
c.store(key, result)
c.storeRedis(context.Background(), key, result)
return result, nil
})
if err != nil {
return zero, err
}
return out.(T), nil
}
func (c *widgetCache[T]) Reload(key string, load widgetLoader[T]) {
ctx, cancel := context.WithTimeout(context.Background(), widgetReloadTimeout)
defer cancel()
_, _, _ = c.group.Do("reload:"+key, func() (interface{}, error) {
result, err := load(ctx)
if err != nil {
if c.logger != nil {
c.logger.Warn("Failed to refresh dashboard widget cache", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return nil, err
}
c.store(key, result)
c.storeRedis(context.Background(), key, result)
return result, nil
})
}
func (c *widgetCache[T]) WarmFromRedis(ctx context.Context, key string) bool {
value, ok := c.fromRedis(ctx, key)
if !ok {
return false
}
c.store(key, value)
return true
}
func (c *widgetCache[T]) fresh(key string) (T, bool) {
var zero T
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
entry, ok := c.entries[key]
if !ok || now.After(entry.expiresAt) {
return zero, false
}
return entry.value, true
}
func (c *widgetCache[T]) stale(key string) (T, bool) {
var zero T
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
entry, ok := c.entries[key]
if !ok || now.After(entry.staleUntil) {
if ok {
delete(c.entries, key)
}
return zero, false
}
return entry.value, true
}
func (c *widgetCache[T]) store(key string, value T) {
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
c.entries[key] = cacheEntry[T]{
expiresAt: now.Add(c.ttl),
staleUntil: now.Add(c.ttl + c.staleGrace),
value: value,
}
}
func (c *widgetCache[T]) redisKey(key string) string {
return fmt.Sprintf("dashboard:%s:%s", c.keyPrefix, key)
}
func (c *widgetCache[T]) fromRedis(ctx context.Context, key string) (T, bool) {
var zero T
if c.redis == nil {
return zero, false
}
raw, err := c.redis.Get(ctx, c.redisKey(key))
if err != nil {
if err != goredis.Nil && c.logger != nil {
c.logger.Warn("Failed to read dashboard widget cache from Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return zero, false
}
var value T
if err := json.Unmarshal([]byte(raw), &value); err != nil {
if c.logger != nil {
c.logger.Warn("Failed to decode dashboard widget cache from Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
_ = c.redis.Del(ctx, c.redisKey(key))
return zero, false
}
return value, true
}
func (c *widgetCache[T]) storeRedis(ctx context.Context, key string, value T) {
if c.redis == nil {
return
}
encoded, err := json.Marshal(value)
if err != nil {
if c.logger != nil {
c.logger.Warn("Failed to encode dashboard widget cache for Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return
}
ttl := c.ttl + c.staleGrace
if err := c.redis.Set(ctx, c.redisKey(key), string(encoded), ttl); err != nil && c.logger != nil {
c.logger.Warn("Failed to store dashboard widget cache in Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
}
+20 -2
View File
@@ -28,8 +28,26 @@ const (
DeliveryChannelInApp DeliveryChannel = "in_app"
)
// EventTypeInApp identifies notifications stored for the in-app notification center.
const EventTypeInApp = "in_app"
// Persisted event types for stored notifications (admin list / notification center).
const (
EventTypeInApp = "in_app"
EventTypeEmail = "email"
EventTypePush = "push"
)
// persistedEventType maps a delivery channel to the event_type stored in MongoDB.
func persistedEventType(channel DeliveryChannel) (string, bool) {
switch channel {
case DeliveryChannelEmail:
return EventTypeEmail, true
case DeliveryChannelPush:
return EventTypePush, true
case DeliveryChannelInApp:
return EventTypeInApp, true
default:
return "", false
}
}
// DeliveryStatus represents the delivery status of notification
type NotificationStatus string
+33 -8
View File
@@ -99,13 +99,27 @@ func (s *notificationService) Send(ctx context.Context, req *NotificationRequest
}
func (s *notificationService) sendToRecipient(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string) {
if err := s.persistNotification(ctx, recipient, req, link, image); err != nil {
// Always persist an in-app record so every send appears in the notification center.
if err := s.persistNotification(ctx, recipient, req, link, image, DeliveryChannelInApp); err != nil {
s.logger.Error("Failed to persist in-app notification", map[string]interface{}{
"error": err.Error(),
"user_id": recipient.UserID,
})
}
for _, channel := range req.Channels {
if channel == DeliveryChannelInApp {
continue
}
if err := s.persistNotification(ctx, recipient, req, link, image, channel); err != nil {
s.logger.Error("Failed to persist notification", map[string]interface{}{
"error": err.Error(),
"user_id": recipient.UserID,
"channel": channel,
})
}
}
metadata := map[string]any{
"tender": strings.TrimSpace(req.Tender),
}
@@ -149,15 +163,26 @@ func (s *notificationService) sendToRecipient(ctx context.Context, recipient not
}
}
func (s *notificationService) persistNotification(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string) error {
methods := map[string]string{
"in_app": "true",
func (s *notificationService) persistNotification(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string, channel DeliveryChannel) error {
eventType, ok := persistedEventType(channel)
if !ok {
return nil
}
if slices.Contains(req.Channels, DeliveryChannelEmail) && recipient.Email != "" {
methods := map[string]string{}
switch channel {
case DeliveryChannelEmail:
if recipient.Email == "" {
return nil
}
methods["email"] = recipient.Email
}
if slices.Contains(req.Channels, DeliveryChannelPush) && len(recipient.DeviceTokens) > 0 {
case DeliveryChannelPush:
if len(recipient.DeviceTokens) == 0 {
return nil
}
methods["push"] = recipient.DeviceTokens[0]
case DeliveryChannelInApp:
methods["in_app"] = "true"
}
status := string(DeliveryStatusSent)
@@ -173,7 +198,7 @@ func (s *notificationService) persistNotification(ctx context.Context, recipient
Image: image,
Priority: string(req.Priority),
Type: string(req.Type),
EventType: EventTypeInApp,
EventType: eventType,
Status: status,
Methods: methods,
Metadata: map[string]any{
+39 -1
View File
@@ -1,6 +1,10 @@
package tender
import "testing"
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestParseAIProcedureRef(t *testing.T) {
tests := []struct {
@@ -84,3 +88,37 @@ func TestMapProcedureReferencesToTenders(t *testing.T) {
t.Fatalf("expected folder-2 ref to resolve, got %+v", got["PROC_folder-2/notice-2"])
}
}
func TestMapProcedureReferencesToTendersUsesFolderCandidatesWhenNoticeLookupMisses(t *testing.T) {
// Notice-old is only present on the merged tender loaded via contract_folder_id.
candidates := []Tender{
{
ContractFolderID: "folder-1",
NoticePublicationID: "notice-new",
RelatedNoticePublicationIDs: []string{"notice-old"},
},
}
refs := []ProcedureReference{
{Ref: "PROC_folder-1/notice-old", ContractFolderID: "folder-1", NoticePublicationID: "notice-old"},
}
got := mapProcedureReferencesToTenders(candidates, refs)
if len(got) != 1 {
t.Fatalf("mapProcedureReferencesToTenders() len = %d, want 1", len(got))
}
if got["PROC_folder-1/notice-old"].NoticePublicationID != "notice-new" {
t.Fatalf("expected folder candidate to resolve notice-old ref, got %+v", got["PROC_folder-1/notice-old"])
}
}
func TestMergeTenderCandidatesDedupesByMongoID(t *testing.T) {
first := Tender{ContractFolderID: "folder-1", NoticePublicationID: "notice-a"}
first.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
second := Tender{ContractFolderID: "folder-1", NoticePublicationID: "notice-b"}
second.ID = first.ID
got := mergeTenderCandidates([]Tender{first}, []Tender{second})
if len(got) != 1 {
t.Fatalf("mergeTenderCandidates() len = %d, want 1", len(got))
}
}
+120
View File
@@ -0,0 +1,120 @@
package tender
import (
"context"
"errors"
"fmt"
"strings"
"time"
"tm/pkg/ai_summarizer"
)
const (
// contractFoldersWithDocsCacheTTL avoids scanning MinIO on every documents_scraped list request.
// Kept >= resolve timeout so a slow scan is not immediately invalidated for waiters.
contractFoldersWithDocsCacheTTL = 15 * time.Minute
// contractFoldersWithDocsResolveTimeout bounds a single MinIO procedure scan.
contractFoldersWithDocsResolveTimeout = 3 * time.Minute
)
type contractFoldersWithDocsCacheEntry struct {
expiresAt time.Time
folderIDs []string
valid bool
}
// enrichDocumentsScrapedFilter resolves contract_folder_id values that currently have
// documents in MinIO and stores them on the search form for the repository filter.
func (s *tenderService) enrichDocumentsScrapedFilter(ctx context.Context, form *SearchForm) error {
if form == nil || !form.DocumentsScraped {
return nil
}
folderIDs, err := s.cachedContractFolderIDsWithDocuments(ctx)
if err != nil {
s.logger.Error("Failed to resolve tenders with scraped documents from MinIO", map[string]interface{}{
"error": err.Error(),
})
if errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
return fmt.Errorf("failed to resolve tenders with scraped documents: MinIO connection unavailable: %w", err)
}
return fmt.Errorf("failed to resolve tenders with scraped documents: %w", err)
}
form.ContractFolderIDsWithDocuments = folderIDs
return nil
}
func (s *tenderService) cachedContractFolderIDsWithDocuments(ctx context.Context) ([]string, error) {
now := time.Now()
s.contractFoldersCacheMu.Lock()
if s.contractFoldersCache.valid && now.Before(s.contractFoldersCache.expiresAt) {
ids := append([]string(nil), s.contractFoldersCache.folderIDs...)
s.contractFoldersCacheMu.Unlock()
return ids, nil
}
s.contractFoldersCacheMu.Unlock()
v, err, _ := s.contractFoldersCacheGroup.Do("contract-folders-with-docs", func() (interface{}, error) {
s.contractFoldersCacheMu.Lock()
if s.contractFoldersCache.valid && time.Now().Before(s.contractFoldersCache.expiresAt) {
ids := append([]string(nil), s.contractFoldersCache.folderIDs...)
s.contractFoldersCacheMu.Unlock()
return ids, nil
}
s.contractFoldersCacheMu.Unlock()
// Detach from the caller's cancellation so a client disconnect or proxy timeout
// does not abort the shared singleflight scan for every waiter.
resolveCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), contractFoldersWithDocsResolveTimeout)
defer cancel()
procedures, listErr := s.listProceduresWithDocuments(resolveCtx)
if listErr != nil {
return nil, listErr
}
folderIDs := contractFolderIDsFromProcedures(procedures)
s.contractFoldersCacheMu.Lock()
s.contractFoldersCache = contractFoldersWithDocsCacheEntry{
expiresAt: time.Now().Add(contractFoldersWithDocsCacheTTL),
folderIDs: folderIDs,
valid: true,
}
s.contractFoldersCacheMu.Unlock()
return append([]string(nil), folderIDs...), nil
})
if err != nil {
return nil, err
}
ids, ok := v.([]string)
if !ok {
return nil, fmt.Errorf("unexpected contract folder cache value type %T", v)
}
return append([]string(nil), ids...), nil
}
func contractFolderIDsFromProcedures(procedures []ai_summarizer.ProcedureDocumentsSummary) []string {
folderIDs := make([]string, 0, len(procedures))
seen := make(map[string]struct{}, len(procedures))
for _, proc := range procedures {
if proc.DocumentCount <= 0 {
continue
}
id := strings.TrimSpace(proc.ContractFolderID)
if id == "" {
continue
}
if _, ok := seen[id]; ok {
continue
}
seen[id] = struct{}{}
folderIDs = append(folderIDs, id)
}
return folderIDs
}
@@ -0,0 +1,241 @@
package tender
import (
"context"
"errors"
"fmt"
"io"
"sync/atomic"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"tm/pkg/ai_summarizer"
)
type mockProcedureDocumentLister struct {
procedures []ai_summarizer.ProcedureDocumentsSummary
err error
calls atomic.Int32
}
func (m *mockProcedureDocumentLister) ListProceduresWithDocuments(context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
m.calls.Add(1)
if m.err != nil {
return nil, m.err
}
return m.procedures, nil
}
type stubProcedureDocumentStorage struct {
lister interface {
ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error)
}
}
func (s stubProcedureDocumentStorage) GetSummaryFromStorage(context.Context, string, string) (string, error) {
return "", ai_summarizer.ErrObjectNotFound
}
func (s stubProcedureDocumentStorage) GetDocumentSummariesFromStorage(context.Context, string, string) ([]ai_summarizer.DocumentSummary, error) {
return nil, nil
}
func (s stubProcedureDocumentStorage) GetTranslationFromStorage(context.Context, string, string, string) (ai_summarizer.StoredTranslation, error) {
return ai_summarizer.StoredTranslation{}, ai_summarizer.ErrTranslationNotReady
}
func (s stubProcedureDocumentStorage) ListDocuments(context.Context, string, string) ([]ai_summarizer.StoredDocument, error) {
return nil, nil
}
func (s stubProcedureDocumentStorage) DownloadDocument(context.Context, string) (io.ReadCloser, error) {
return nil, ai_summarizer.ErrObjectNotFound
}
func (s stubProcedureDocumentStorage) ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
return s.lister.ListProceduresWithDocuments(ctx)
}
func TestBuildSearchFilterDocumentsScrapedUsesMinIOFolderIDs(t *testing.T) {
repo := &tenderRepository{}
filter := repo.buildSearchFilter(&SearchForm{
DocumentsScraped: true,
ContractFolderIDsWithDocuments: []string{"folder-a", "folder-b"},
})
in, ok := filter["contract_folder_id"].(bson.M)["$in"].([]string)
if !ok {
t.Fatalf("expected contract_folder_id $in filter, got %#v", filter)
}
if len(in) != 2 || in[0] != "folder-a" || in[1] != "folder-b" {
t.Fatalf("unexpected folder ids: %#v", in)
}
if filter["processing_metadata.documents_scraped"] != nil {
t.Fatalf("expected no stale mongo documents_scraped flag, got %#v", filter)
}
}
func TestContractFolderIDsSearchClauseChunksLargeLists(t *testing.T) {
folderIDs := make([]string, maxContractFolderInClause+2)
for i := range folderIDs {
folderIDs[i] = fmt.Sprintf("folder-%d", i)
}
clause := contractFolderIDsSearchClause(folderIDs)
or, ok := clause["$or"].([]bson.M)
if !ok || len(or) != 2 {
t.Fatalf("expected chunked $or filter, got %#v", clause)
}
}
func TestCachedContractFolderIDsWithDocumentsUsesCache(t *testing.T) {
lister := &mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 2},
{ContractFolderID: "folder-2", DocumentCount: 0},
{ContractFolderID: "folder-3", DocumentCount: 1},
},
}
svc := &tenderService{
aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister},
}
first, err := svc.cachedContractFolderIDsWithDocuments(context.Background())
if err != nil {
t.Fatalf("first lookup failed: %v", err)
}
if len(first) != 2 || first[0] != "folder-1" || first[1] != "folder-3" {
t.Fatalf("unexpected folder ids: %#v", first)
}
if lister.calls.Load() != 1 {
t.Fatalf("expected one MinIO scan, got %d", lister.calls.Load())
}
second, err := svc.cachedContractFolderIDsWithDocuments(context.Background())
if err != nil {
t.Fatalf("second lookup failed: %v", err)
}
if len(second) != 2 {
t.Fatalf("unexpected cached folder ids: %#v", second)
}
if lister.calls.Load() != 1 {
t.Fatalf("expected cache hit without extra MinIO scan, got %d calls", lister.calls.Load())
}
}
func TestEnrichDocumentsScrapedFilterPropagatesMinIOError(t *testing.T) {
svc := &tenderService{
aiSummarizerStorage: stubProcedureDocumentStorage{
lister: &mockProcedureDocumentLister{
err: ai_summarizer.ErrMinIOUnavailable,
},
},
logger: noopTenderTestLogger{},
}
form := &SearchForm{DocumentsScraped: true}
err := svc.enrichDocumentsScrapedFilter(context.Background(), form)
if err == nil {
t.Fatal("expected MinIO error")
}
if !errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
t.Fatalf("expected wrapped MinIO unavailable error, got %v", err)
}
}
func TestContractFolderIDsFromProceduresDedupesAndSkipsEmpty(t *testing.T) {
ids := contractFolderIDsFromProcedures([]ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "a", DocumentCount: 1},
{ContractFolderID: "a", DocumentCount: 2},
{ContractFolderID: " ", DocumentCount: 1},
{ContractFolderID: "b", DocumentCount: 0},
})
if len(ids) != 1 || ids[0] != "a" {
t.Fatalf("unexpected ids: %#v", ids)
}
}
type scanStartedProcedureLister struct {
mockProcedureDocumentLister
started chan struct{}
}
func (m *scanStartedProcedureLister) ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
if m.started != nil {
select {
case <-m.started:
default:
close(m.started)
}
}
time.Sleep(50 * time.Millisecond)
if err := ctx.Err(); err != nil {
return nil, err
}
return m.mockProcedureDocumentLister.ListProceduresWithDocuments(ctx)
}
func TestCachedContractFolderIDsSurvivesCallerCancellation(t *testing.T) {
started := make(chan struct{})
lister := &scanStartedProcedureLister{
mockProcedureDocumentLister: mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 1},
},
},
started: started,
}
svc := &tenderService{aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
errCh := make(chan error, 1)
var ids []string
go func() {
var err error
ids, err = svc.cachedContractFolderIDsWithDocuments(ctx)
errCh <- err
}()
select {
case <-started:
case <-time.After(2 * time.Second):
t.Fatal("MinIO scan did not start")
}
cancel()
if err := <-errCh; err != nil {
t.Fatalf("expected shared scan to finish despite caller cancel: %v", err)
}
if len(ids) != 1 || ids[0] != "folder-1" {
t.Fatalf("unexpected folder ids: %#v", ids)
}
}
func TestContractFoldersWithDocsCacheExpires(t *testing.T) {
lister := &mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 1},
},
}
svc := &tenderService{aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister}}
if _, err := svc.cachedContractFolderIDsWithDocuments(context.Background()); err != nil {
t.Fatalf("lookup failed: %v", err)
}
svc.contractFoldersCacheMu.Lock()
svc.contractFoldersCache.expiresAt = time.Now().Add(-time.Second)
svc.contractFoldersCacheMu.Unlock()
if _, err := svc.cachedContractFolderIDsWithDocuments(context.Background()); err != nil {
t.Fatalf("lookup after expiry failed: %v", err)
}
if lister.calls.Load() != 2 {
t.Fatalf("expected cache refresh after expiry, got %d calls", lister.calls.Load())
}
}
-1
View File
@@ -88,7 +88,6 @@ type Tender struct {
Source TenderSource `bson:"source" json:"source"`
SourceFileURL string `bson:"source_file_url" json:"source_file_url"`
SourceFileName string `bson:"source_file_name" json:"source_file_name"`
ContentXML string `bson:"content_xml,omitempty" json:"content_xml,omitempty"`
ProcessingMetadata ProcessingMetadata `bson:"processing_metadata" json:"processing_metadata"`
// AIOverallSummary optional denormalized overall summary for search/list (e.g. synced from AI pipeline storage).
AIOverallSummary string `bson:"ai_overall_summary,omitempty" json:"ai_overall_summary,omitempty"`
+74 -1
View File
@@ -64,12 +64,85 @@ type SearchForm struct {
OnlyActiveDeadlines bool `query:"only_active_deadlines" valid:"optional"`
Language *string `query:"lang" valid:"optional"`
DocumentsScraped bool `query:"documents_scraped" valid:"optional"`
// ContractFolderIDsWithDocuments is populated by the service from MinIO when DocumentsScraped is true.
// ContractFolderIDsWithDocuments is populated by the service from a cached MinIO scan when DocumentsScraped is true.
ContractFolderIDsWithDocuments []string `query:"-" valid:"optional"`
// ExcludeRejectedTenders hides company-rejected tenders from recommendation results.
ExcludeRejectedTenders bool `query:"-" valid:"optional"`
}
// HasListFilters reports whether the form applies any MongoDB list filters.
func (f *SearchForm) HasListFilters() bool {
if f == nil {
return false
}
if f.DocumentsScraped || f.OnlyActiveDeadlines || f.ExcludeRejectedTenders {
return true
}
if f.Search != nil && strings.TrimSpace(*f.Search) != "" {
return true
}
if f.Title != nil && strings.TrimSpace(*f.Title) != "" {
return true
}
if f.Description != nil && strings.TrimSpace(*f.Description) != "" {
return true
}
if f.NoticeType != nil && strings.TrimSpace(*f.NoticeType) != "" {
return true
}
if len(f.NoticeTypes) > 0 || len(f.FormTypes) > 0 {
return true
}
if f.ProcurementType != nil && strings.TrimSpace(*f.ProcurementType) != "" {
return true
}
if f.Country != nil && strings.TrimSpace(*f.Country) != "" {
return true
}
if f.CountryCode != nil && strings.TrimSpace(*f.CountryCode) != "" {
return true
}
if len(f.CountryCodes) > 0 || len(f.RegionCodes) > 0 {
return true
}
if f.MainClassification != nil && strings.TrimSpace(*f.MainClassification) != "" {
return true
}
if len(f.Classifications) > 0 || len(f.CpvCodes) > 0 {
return true
}
if f.MinEstimatedValue != nil || f.MaxEstimatedValue != nil || f.Currency != "" {
return true
}
if f.CreatedAt != nil || f.CreatedAtFrom != nil || f.CreatedAtTo != nil {
return true
}
if f.TenderDeadline != nil || f.TenderDeadlineFrom != nil || f.TenderDeadlineTo != nil ||
f.DeadlineFrom != nil || f.DeadlineTo != nil {
return true
}
if f.PublicationDate != nil || f.PublicationDateFrom != nil || f.PublicationDateTo != nil {
return true
}
if f.SubmissionDeadline != nil || f.SubmissionDateAliasFrom != nil || f.SubmissionDateAliasTo != nil ||
f.SubmissionDateFrom != nil || f.SubmissionDateTo != nil {
return true
}
if len(f.Status) > 0 || len(f.Source) > 0 || len(f.Languages) > 0 {
return true
}
if f.BuyerOrganizationID != nil && strings.TrimSpace(*f.BuyerOrganizationID) != "" {
return true
}
if f.NoticePublicationID != nil && strings.TrimSpace(*f.NoticePublicationID) != "" {
return true
}
if len(f.ContractFolderIDsWithDocuments) > 0 {
return true
}
return false
}
// SearchResponse represents the response for listing tenders
type SearchResponse struct {
Tenders []TenderResponse `json:"tenders"`
+4 -2
View File
@@ -160,10 +160,11 @@ func (h *TenderHandler) Delete(c echo.Context) error {
// @Param languages query []string false "Filter by languages (comma-separated)"
// @Param buyer_organization_id query string false "Filter by buyer organization ID"
// @Param notice_publication_id query string false "Filter by TED notice publication ID"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO (cached scan)"
// @Param limit query int false "Number of items per page (default: 20, max: 100)"
// @Param offset query int false "Number of items to skip (default: 0). Prefer cursor for deep pages."
// @Param cursor query string false "Opaque cursor from previous page meta.next_cursor"
// @Param include_total query bool false "When true, include meta.total and meta.pages (slower on large collections)"
// @Param sort_by query string false "Sort by field"
// @Param sort_order query string false "Sort order (asc or desc)"
// @Success 200 {object} response.APIResponse{data=map[string]interface{}}
@@ -270,10 +271,11 @@ func (h *TenderHandler) AdminRecommendTenders(c echo.Context) error {
// @Param languages query []string false "Filter by languages (comma-separated)"
// @Param buyer_organization_id query string false "Filter by buyer organization ID"
// @Param notice_publication_id query string false "Filter by TED notice publication ID"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO (cached scan)"
// @Param limit query int false "Number of items per page (default: 20, max: 100)"
// @Param offset query int false "Number of items to skip (default: 0). Prefer cursor for deep pages."
// @Param cursor query string false "Opaque cursor from previous page meta.next_cursor"
// @Param include_total query bool false "When true, include meta.total and meta.pages (slower on large collections)"
// @Param sort_by query string false "Sort by field"
// @Param sort_order query string false "Sort order (asc or desc)"
// @Success 200 {object} response.APIResponse{data=SearchResponse}
+10 -7
View File
@@ -97,19 +97,22 @@ func addTenderIDsToSet(set map[string]struct{}, ids []string) {
// markResolvedRecommendedTenderSeen records a resolved tender and reports whether it was already seen.
// Recommendations may reference the same tender via different AI refs (procedure ref vs Mongo ID,
// or multiple notice publication IDs for one merged tender).
func markResolvedRecommendedTenderSeen(seen map[string]struct{}, tender Tender) bool {
func markResolvedRecommendedTenderSeen(seen map[string]struct{}, tender Tender, tenderRef string) bool {
if seen == nil {
return false
}
tenderID := strings.TrimSpace(tender.GetID())
if tenderID == "" {
key := strings.TrimSpace(tender.GetID())
if key == "" {
key = strings.TrimSpace(tenderRef)
}
if key == "" {
return false
}
if _, exists := seen[key]; exists {
return true
}
if _, exists := seen[tenderID]; exists {
return true
}
seen[tenderID] = struct{}{}
seen[key] = struct{}{}
return false
}
@@ -132,8 +132,8 @@ func TestMarkResolvedRecommendedTenderSeen(t *testing.T) {
sample.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
seen := make(map[string]struct{})
first := markResolvedRecommendedTenderSeen(seen, sample)
second := markResolvedRecommendedTenderSeen(seen, sample)
first := markResolvedRecommendedTenderSeen(seen, sample, "PROC_folder/notice")
second := markResolvedRecommendedTenderSeen(seen, sample, "PROC_folder/notice")
if first {
t.Fatalf("first sighting should not be skipped, id=%q seen=%v", sample.GetID(), seen)
}
@@ -143,7 +143,7 @@ func TestMarkResolvedRecommendedTenderSeen(t *testing.T) {
otherRefSameTender := Tender{}
otherRefSameTender.ID = sample.ID
if !markResolvedRecommendedTenderSeen(seen, otherRefSameTender) {
if !markResolvedRecommendedTenderSeen(seen, otherRefSameTender, "PROC_folder/other-notice") {
t.Fatalf("duplicate tender via different ref should be skipped")
}
}
+1 -1
View File
@@ -69,7 +69,7 @@ func (s *tenderService) buildRecommendationPage(
if len(form.Status) > 0 && !statusAllowed(tender.Status, form.Status) {
continue
}
if markResolvedRecommendedTenderSeen(seenTenderIDs, tender) {
if markResolvedRecommendedTenderSeen(seenTenderIDs, tender, tenderRef) {
continue
}
@@ -0,0 +1,355 @@
package tender
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/redis/go-redis/v9"
"tm/internal/company"
"tm/pkg/response"
)
const (
recommendedTendersPageCacheKeyPrefix = "ai:recommendations:page:"
recommendationPageCacheReadTimeout = 3 * time.Second
recommendationPageCacheBuildTimeout = 5 * time.Minute
recommendationPageCacheConcurrency = 8
)
type cachedRecommendedTendersPage struct {
BuiltAt int64 `json:"built_at"`
CompanyID string `json:"company_id"`
Language string `json:"language"`
Items []TenderResponse `json:"items"`
}
func recommendedTendersPageCacheKey(companyID, language string) string {
return recommendedTendersPageCacheKeyPrefix +
strings.TrimSpace(companyID) + ":" +
strings.ToLower(strings.TrimSpace(language))
}
// ScheduleRefreshRecommendedTendersPageCache rebuilds the page cache for the service default language.
func (s *tenderService) ScheduleRefreshRecommendedTendersPageCache(companyID string) {
s.schedulePageCacheRefreshForLanguage(companyID, s.defaultLanguage)
}
// InvalidateRecommendedTendersPageCache drops the cached page for the service default language.
func (s *tenderService) InvalidateRecommendedTendersPageCache(companyID string) {
companyID = strings.TrimSpace(companyID)
if companyID == "" || s.redisClient == nil {
return
}
ctx, cancel := context.WithTimeout(context.Background(), recommendationPageCacheReadTimeout)
defer cancel()
s.deleteRecommendedTendersPageCache(ctx, companyID, s.defaultLanguage)
}
func (s *tenderService) schedulePageCacheRefreshForLanguage(companyID, language string) {
companyID = strings.TrimSpace(companyID)
language = strings.ToLower(strings.TrimSpace(language))
if companyID == "" || language == "" || s.redisClient == nil {
return
}
go func() {
ctx, cancel := context.WithTimeout(context.Background(), recommendationPageCacheBuildTimeout)
defer cancel()
if err := s.buildRecommendedTendersPageCache(ctx, companyID, language); err != nil {
s.logger.Warn("Failed to refresh recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
}
}()
}
func (s *tenderService) buildRecommendedTendersPageCache(ctx context.Context, companyID, language string) error {
recommendations, err := s.companyService.GetAIRecommendations(ctx, companyID)
if err != nil {
return err
}
if len(recommendations) == 0 {
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil
}
items, err := s.buildAllRankedRecommendations(ctx, recommendations)
if err != nil {
return err
}
if len(items) == 0 {
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil
}
responses := s.buildRecommendedTenderListResponses(items, language)
page := cachedRecommendedTendersPage{
BuiltAt: time.Now().Unix(),
CompanyID: companyID,
Language: language,
Items: responses,
}
encoded, err := json.Marshal(page)
if err != nil {
return fmt.Errorf("encode recommended tenders page cache: %w", err)
}
writeCtx, cancel := context.WithTimeout(ctx, recommendationPageCacheReadTimeout)
defer cancel()
if err := s.redisClient.Set(writeCtx, recommendedTendersPageCacheKey(companyID, language), string(encoded), s.pageCacheTTL); err != nil {
return fmt.Errorf("store recommended tenders page cache: %w", err)
}
s.logger.Info("Recommended tenders page cache built", map[string]interface{}{
"company_id": companyID,
"language": language,
"count": len(responses),
})
return nil
}
func (s *tenderService) getRecommendedTendersPageCache(ctx context.Context, companyID, language string) ([]TenderResponse, bool) {
if s.redisClient == nil {
return nil, false
}
readCtx, cancel := context.WithTimeout(ctx, recommendationPageCacheReadTimeout)
defer cancel()
raw, err := s.redisClient.Get(readCtx, recommendedTendersPageCacheKey(companyID, language))
if err != nil {
if err != redis.Nil {
s.logger.Warn("Failed to read recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
}
return nil, false
}
var page cachedRecommendedTendersPage
if err := json.Unmarshal([]byte(raw), &page); err != nil {
s.logger.Warn("Failed to decode recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil, false
}
if len(page.Items) == 0 {
return nil, false
}
return page.Items, true
}
func (s *tenderService) deleteRecommendedTendersPageCache(ctx context.Context, companyID, language string) {
if s.redisClient == nil {
return
}
_ = s.redisClient.Del(ctx, recommendedTendersPageCacheKey(companyID, language))
}
func (s *tenderService) buildAllRankedRecommendations(
ctx context.Context,
recommendations []company.RecommendedTenderResponse,
) ([]rankedRecommendedTender, error) {
seenTenderIDs := make(map[string]struct{}, len(recommendations))
out := make([]rankedRecommendedTender, 0, len(recommendations))
for start := 0; start < len(recommendations); start += recommendationResolveBatchSize {
end := start + recommendationResolveBatchSize
if end > len(recommendations) {
end = len(recommendations)
}
tenderByRef, err := s.resolveRecommendedTenders(ctx, recommendations[start:end])
if err != nil {
return nil, err
}
for _, rec := range recommendations[start:end] {
tenderRef := strings.TrimSpace(rec.TenderID)
tender, ok := tenderByRef[tenderRef]
if !ok {
continue
}
if markResolvedRecommendedTenderSeen(seenTenderIDs, tender, tenderRef) {
continue
}
out = append(out, rankedRecommendedTender{
tender: tender,
rank: rec.Rank,
analysis: rec.Analysis,
tenderRef: tenderRef,
})
}
}
return out, nil
}
func (s *tenderService) isRecommendedPageCacheStale(ctx context.Context, companyID string, cachedCount int) bool {
companyID = strings.TrimSpace(companyID)
if companyID == "" || cachedCount == 0 {
return false
}
recommendations, err := s.companyService.GetAIRecommendations(ctx, companyID)
if err != nil || len(recommendations) == 0 {
return false
}
return cachedCount < len(recommendations)
}
func (s *tenderService) recommendFromPageCache(
ctx context.Context,
companyID string,
companyIDs []string,
form *SearchForm,
pagination *response.Pagination,
lang string,
) (*SearchResponse, bool) {
var cached []TenderResponse
if len(companyIDs) > 0 {
cached = s.mergeCachedRecommendationPages(ctx, companyIDs, lang)
} else {
items, ok := s.getRecommendedTendersPageCache(ctx, companyID, lang)
if !ok {
return nil, false
}
cached = items
}
if len(cached) == 0 {
return nil, false
}
if len(companyIDs) == 0 && s.isRecommendedPageCacheStale(ctx, companyID, len(cached)) {
return nil, false
}
excluded, excludedByCompany, err := s.loadRecommendationExclusions(ctx, form, companyID, companyIDs)
if err != nil {
return nil, false
}
now := time.Now().Unix()
filtered := filterCachedRecommendedTenders(cached, form, companyIDs, excluded, excludedByCompany, now)
total := int64(len(filtered))
start := pagination.Offset
if start > len(filtered) {
start = len(filtered)
}
end := start + pagination.Limit
if end > len(filtered) {
end = len(filtered)
}
paged := append([]TenderResponse(nil), filtered[start:end]...)
paged = s.enrichRecommendedTenderResponsesParallel(ctx, paged, lang)
return &SearchResponse{
Tenders: paged,
Metadata: pagination.ListMeta(total, "", end < len(filtered), start),
}, true
}
func (s *tenderService) mergeCachedRecommendationPages(ctx context.Context, companyIDs []string, lang string) []TenderResponse {
normalized := normalizeRecommendationCompanyScope(companyIDs)
if len(normalized) == 0 {
return nil
}
lists := make([][]TenderResponse, 0, len(normalized))
for _, companyID := range normalized {
items, ok := s.getRecommendedTendersPageCache(ctx, companyID, lang)
if !ok || len(items) == 0 {
return nil
}
lists = append(lists, items)
}
merged := make([]TenderResponse, 0)
for _, list := range lists {
merged = append(merged, list...)
}
return dedupeCachedRecommendedTendersByID(merged)
}
func dedupeCachedRecommendedTendersByID(items []TenderResponse) []TenderResponse {
seen := make(map[string]struct{}, len(items))
out := make([]TenderResponse, 0, len(items))
for _, item := range items {
key := strings.TrimSpace(item.ID)
if key == "" {
key = strings.TrimSpace(item.ProcedureRef)
}
if key == "" {
continue
}
if _, exists := seen[key]; exists {
continue
}
seen[key] = struct{}{}
out = append(out, item)
}
return out
}
func filterCachedRecommendedTenders(
items []TenderResponse,
form *SearchForm,
companyIDs []string,
excluded map[string]struct{},
excludedByCompany map[string]map[string]struct{},
now int64,
) []TenderResponse {
if len(items) == 0 {
return nil
}
out := make([]TenderResponse, 0, len(items))
for _, item := range items {
tender := tenderFromRecommendationResponse(item)
tenderRef := strings.TrimSpace(item.ProcedureRef)
if len(companyIDs) > 0 {
if isRecommendedTenderExcludedForAllCompanies(tender, tenderRef, excludedByCompany, companyIDs) {
continue
}
} else if isRecommendedTenderExcluded(tender, tenderRef, excluded) {
continue
}
if form.OnlyActiveDeadlines && !tender.IsRecommendable(now) {
continue
}
if len(form.Status) > 0 && !statusAllowed(tender.Status, form.Status) {
continue
}
out = append(out, item)
}
return out
}
// RecommendedTendersPageCacheKeyPrefix is shared with company invalidation logic.
const RecommendedTendersPageCacheKeyPrefix = recommendedTendersPageCacheKeyPrefix
// RecommendedTendersPageCacheKey builds the Redis key for a cached recommendation page.
func RecommendedTendersPageCacheKey(companyID, language string) string {
return recommendedTendersPageCacheKey(companyID, language)
}
@@ -0,0 +1,44 @@
package tender
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestFilterCachedRecommendedTendersAppliesActiveAndRejectedFilters(t *testing.T) {
activeID, _ := bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
expiredID, _ := bson.ObjectIDFromHex("507f1f77bcf86cd799439012")
items := []TenderResponse{
{
ID: activeID.Hex(),
Status: TenderStatusActive,
PublicationDate: 1,
SubmissionDeadline: 9999999999,
ProcedureRef: "PROC_folder-a/notice-a",
Rank: 1,
},
{
ID: expiredID.Hex(),
Status: TenderStatusExpired,
ProcedureRef: "PROC_folder-b/notice-b",
Rank: 2,
},
}
form := &SearchForm{
OnlyActiveDeadlines: true,
ExcludeRejectedTenders: true,
Status: []string{string(TenderStatusActive)},
}
excluded := map[string]struct{}{expiredID.Hex(): {}}
got := filterCachedRecommendedTenders(items, form, nil, excluded, nil, 100)
if len(got) != 1 {
t.Fatalf("len = %d, want 1", len(got))
}
if got[0].ID != activeID.Hex() {
t.Fatalf("got ID %q, want active tender", got[0].ID)
}
}
@@ -6,6 +6,9 @@ import (
"strings"
"sync"
"go.mongodb.org/mongo-driver/v2/bson"
"golang.org/x/sync/errgroup"
"tm/pkg/ai_summarizer"
)
@@ -68,6 +71,72 @@ func (s *tenderService) buildRecommendedTenderListResponses(items []rankedRecomm
return out
}
func (s *tenderService) enrichRecommendedTenderResponsesParallel(ctx context.Context, responses []TenderResponse, language string) []TenderResponse {
if len(responses) == 0 || s.aiSummarizerStorage == nil {
return responses
}
out := make([]TenderResponse, len(responses))
if len(responses) == 1 {
out[0] = s.enrichRecommendedTenderResponse(ctx, responses[0], language, newRecommendationTranslationCache())
return out
}
translationCache := newRecommendationTranslationCache()
group, groupCtx := errgroup.WithContext(ctx)
group.SetLimit(recommendationPageCacheConcurrency)
for i := range responses {
i := i
group.Go(func() error {
out[i] = s.enrichRecommendedTenderResponse(groupCtx, responses[i], language, translationCache)
return nil
})
}
if err := group.Wait(); err != nil {
for i, resp := range responses {
out[i] = s.enrichRecommendedTenderResponse(ctx, resp, language, translationCache)
}
}
return out
}
func (s *tenderService) enrichRecommendedTenderResponse(
ctx context.Context,
resp TenderResponse,
language string,
cache *recommendationTranslationCache,
) TenderResponse {
enriched := resp
tender := tenderFromRecommendationResponse(resp)
s.enrichWithTranslationTimeout(
ctx,
&tender,
&enriched,
language,
cache,
recommendTranslationEnrichmentTimeout,
)
return enriched
}
func tenderFromRecommendationResponse(item TenderResponse) Tender {
var tender Tender
if id, err := bson.ObjectIDFromHex(strings.TrimSpace(item.ID)); err == nil {
tender.ID = id
}
tender.ContractFolderID = item.ContractFolderID
tender.NoticePublicationID = item.NoticePublicationID
tender.RelatedNoticePublicationIDs = append([]string(nil), item.RelatedNoticePublicationIDs...)
tender.Status = item.Status
tender.PublicationDate = item.PublicationDate
tender.SubmissionDeadline = item.SubmissionDeadline
tender.TenderDeadline = item.TenderDeadline
return tender
}
func (s *tenderService) loadRecommendationExclusions(
ctx context.Context,
form *SearchForm,
+152 -39
View File
@@ -13,6 +13,9 @@ import (
"go.mongodb.org/mongo-driver/v2/mongo"
)
// maxContractFolderInClause caps each MongoDB $in batch for documents_scraped MinIO cross-check filters.
const maxContractFolderInClause = 5000
// TenderRepository interface defines tender data access methods
type TenderRepository interface {
// Tender CRUD operations
@@ -27,7 +30,6 @@ type TenderRepository interface {
// GetLatestByContractFolderIDs returns the most recently updated tender per distinct contract_folder_id.
GetLatestByContractFolderIDs(ctx context.Context, contractFolderIDs []string) (map[string]*Tender, error)
GetByProcurementProjectID(ctx context.Context, procurementProjectID string) (*Tender, error)
FindTendersWithContentXML(ctx context.Context, limit, skip int) ([]Tender, error)
GetTenderCountByCountry(ctx context.Context) (map[string]int64, error)
GetTenderCountByType(ctx context.Context) (map[string]int64, error)
GetTenderCountByClassification(ctx context.Context) (map[string]int64, error)
@@ -64,10 +66,17 @@ func tenderSearchListProjection() bson.M {
"modifications": 0,
"source_file_url": 0,
"source_file_name": 0,
// Not used by list response mapping; can be large per tender.
// Detail-only or bulky fields; list rows use title and summary metadata only.
"description": 0,
"ai_overall_summary": 0,
"procurement_lots": 0,
"awarded": 0,
"cancellation_reason": 0,
"suspension_reason": 0,
"translations": 0,
"organizations": 0,
"review_organization": 0,
"processing_metadata.enrichment_data": 0,
"processing_metadata.translated_data": 0,
"processing_metadata.parsing_errors": 0,
"processing_metadata.validation_errors": 0,
@@ -123,6 +132,15 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Te
*orm.NewIndex("main_classification_idx", bson.D{{Key: "main_classification", Value: 1}}),
*orm.NewIndex("publication_date_idx", bson.D{{Key: "publication_date", Value: -1}}),
*orm.NewIndex("tender_deadline_idx", bson.D{{Key: "tender_deadline", Value: 1}}),
*orm.NewIndex("submission_deadline_idx", bson.D{{Key: "submission_deadline", Value: 1}}),
*orm.NewIndex("closing_soon_submission_idx", bson.D{
{Key: "submission_deadline", Value: 1},
{Key: "status", Value: 1},
}),
*orm.NewIndex("closing_soon_tender_deadline_idx", bson.D{
{Key: "tender_deadline", Value: 1},
{Key: "status", Value: 1},
}),
*orm.NewIndex("estimated_value_idx", bson.D{{Key: "estimated_value", Value: -1}}),
*orm.NewIndex("created_at_idx", bson.D{{Key: "created_at", Value: -1}}),
// Stable keyset pagination for default admin list (created_at desc).
@@ -165,6 +183,13 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Te
{Key: "processing_metadata.documents_scraped", Value: 1},
{Key: "processing_metadata.documents_scraped_at", Value: -1},
}),
// Panel list: documents_scraped=true sorted by created_at desc (keyset pagination).
*orm.NewIndex("documents_scraped_created_at_id_idx", bson.D{
{Key: "created_at", Value: -1},
{Key: "_id", Value: -1},
}).WithPartialFilterExpression(bson.M{
"processing_metadata.documents_scraped": true,
}),
*orm.NewIndex("scraped_documents_scraped_at_idx", bson.D{
{Key: "scraped_documents.scraped_at", Value: 1},
}),
@@ -249,12 +274,19 @@ func (r *tenderRepository) GetByIDs(ctx context.Context, ids []string) ([]Tender
objectIDs := make([]bson.ObjectID, 0, len(batch))
for _, id := range batch {
objectID, err := bson.ObjectIDFromHex(id)
objectID, err := bson.ObjectIDFromHex(strings.TrimSpace(id))
if err != nil {
return nil, err
r.logger.Warn("Skipping invalid tender ID in batch lookup", map[string]interface{}{
"tender_id": id,
"error": err.Error(),
})
continue
}
objectIDs = append(objectIDs, objectID)
}
if len(objectIDs) == 0 {
continue
}
pagination := orm.Pagination{
Limit: len(objectIDs),
@@ -362,13 +394,18 @@ func (r *tenderRepository) GetByProcedureReference(ctx context.Context, contract
const procedureReferenceLookupBatchSize = 250
// GetByProcedureReferences loads tenders for multiple AI procedure refs using indexed notice lookups.
const contractFolderLookupBatchSize = 250
// GetByProcedureReferences loads tenders for multiple AI procedure refs.
// It combines indexed notice lookups with contract-folder candidates so older notice
// references still resolve after tender merges (same behavior as before the perf refactor).
func (r *tenderRepository) GetByProcedureReferences(ctx context.Context, refs []ProcedureReference) (map[string]Tender, error) {
out := make(map[string]Tender)
if len(refs) == 0 {
return out, nil
}
folderIDs := make([]string, 0, len(refs))
noticeIDs := make([]string, 0, len(refs))
for _, ref := range refs {
folder := strings.TrimSpace(ref.ContractFolderID)
@@ -376,15 +413,12 @@ func (r *tenderRepository) GetByProcedureReferences(ctx context.Context, refs []
if folder == "" || notice == "" {
continue
}
folderIDs = append(folderIDs, folder)
noticeIDs = append(noticeIDs, notice)
}
uniqueNotices := uniqueNonEmptyTrimmedStrings(noticeIDs)
if len(uniqueNotices) == 0 {
return out, nil
}
candidates := make([]Tender, 0, len(uniqueNotices))
noticeCandidates := make([]Tender, 0, len(uniqueNotices))
for start := 0; start < len(uniqueNotices); start += procedureReferenceLookupBatchSize {
end := start + procedureReferenceLookupBatchSize
if end > len(uniqueNotices) {
@@ -395,12 +429,79 @@ func (r *tenderRepository) GetByProcedureReferences(ctx context.Context, refs []
if err != nil {
return nil, err
}
candidates = append(candidates, batch...)
noticeCandidates = append(noticeCandidates, batch...)
}
folderCandidates, err := r.findTendersByContractFolderIDs(ctx, folderIDs)
if err != nil {
return nil, err
}
candidates := mergeTenderCandidates(noticeCandidates, folderCandidates)
return mapProcedureReferencesToTenders(candidates, refs), nil
}
func (r *tenderRepository) findTendersByContractFolderIDs(ctx context.Context, folderIDs []string) ([]Tender, error) {
uniqueFolders := uniqueNonEmptyTrimmedStrings(folderIDs)
if len(uniqueFolders) == 0 {
return nil, nil
}
out := make([]Tender, 0, len(uniqueFolders))
for start := 0; start < len(uniqueFolders); start += contractFolderLookupBatchSize {
end := start + contractFolderLookupBatchSize
if end > len(uniqueFolders) {
end = len(uniqueFolders)
}
batch := uniqueFolders[start:end]
findLimit := len(batch) * 5
if minLimit := len(batch) + 1; findLimit < minLimit {
findLimit = minLimit
}
filter := bson.M{"contract_folder_id": bson.M{"$in": batch}}
pagination := orm.Pagination{
Limit: findLimit,
SortField: "updated_at",
SortOrder: -1,
SkipCount: true,
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination)
if err != nil {
r.logger.Error("Failed to get tenders by contract folder IDs", map[string]interface{}{
"count": len(batch),
"error": err.Error(),
})
return nil, err
}
out = append(out, result.Items...)
}
return out, nil
}
func mergeTenderCandidates(groups ...[]Tender) []Tender {
seen := make(map[string]struct{})
out := make([]Tender, 0)
for _, group := range groups {
for i := range group {
tenderID := strings.TrimSpace(group[i].GetID())
if tenderID == "" {
out = append(out, group[i])
continue
}
if _, exists := seen[tenderID]; exists {
continue
}
seen[tenderID] = struct{}{}
out = append(out, group[i])
}
}
return out
}
func (r *tenderRepository) findTendersByNoticePublicationIDs(ctx context.Context, noticeIDs []string) ([]Tender, error) {
if len(noticeIDs) == 0 {
return nil, nil
@@ -413,7 +514,7 @@ func (r *tenderRepository) findTendersByNoticePublicationIDs(ctx context.Context
},
}
pagination := orm.Pagination{
Limit: len(noticeIDs) * 2,
Limit: len(noticeIDs) * 3,
SortField: "updated_at",
SortOrder: -1,
SkipCount: true,
@@ -577,27 +678,6 @@ func (r *tenderRepository) GetByProcurementProjectID(ctx context.Context, procur
return &result.Items[0], nil
}
// FindTendersWithContentXML returns tenders that still have TED XML (for backfill after notices were deleted).
func (r *tenderRepository) FindTendersWithContentXML(ctx context.Context, limit, skip int) ([]Tender, error) {
filter := bson.M{"content_xml": bson.M{"$exists": true, "$ne": ""}}
pagination := orm.Pagination{
Limit: limit,
Skip: skip,
SortField: "_id",
SortOrder: 1,
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination)
if err != nil {
r.logger.Error("Failed to list tenders with content_xml", map[string]interface{}{
"error": err.Error(),
})
return nil, err
}
return result.Items, nil
}
// Update updates an existing tender
func (r *tenderRepository) Update(ctx context.Context, tender *Tender) error {
tender.UpdatedAt = time.Now().Unix()
@@ -650,6 +730,15 @@ func (r *tenderRepository) Search(ctx context.Context, form *SearchForm, paginat
sortOrder = pagination.SortOrder
}
listOpts := orm.ListPaginationOptions{Projection: tenderSearchListProjection()}
if pagination.IncludeTotal {
if pagination.Cursor != "" {
listOpts.IncludeCount = true
}
} else {
listOpts.SkipCount = true
}
mongoPagination, err := orm.BuildListPagination(
pagination.Limit,
pagination.Offset,
@@ -657,7 +746,7 @@ func (r *tenderRepository) Search(ctx context.Context, form *SearchForm, paginat
sort,
sortOrder,
filter,
orm.ListPaginationOptions{Projection: tenderSearchListProjection()},
listOpts,
)
if err != nil {
return nil, err
@@ -1294,11 +1383,7 @@ func (r *tenderRepository) buildSearchFilter(form *SearchForm) bson.M {
}
if form.DocumentsScraped {
folderIDs := dedupeNonEmptyStrings(form.ContractFolderIDsWithDocuments)
scrapedClause := bson.M{"contract_folder_id": bson.M{"$in": folderIDs}}
if len(folderIDs) == 0 {
scrapedClause = bson.M{"_id": bson.M{"$exists": false}}
}
scrapedClause := documentsScrapedSearchClause(form)
if len(filter) == 0 {
return scrapedClause
}
@@ -1317,3 +1402,31 @@ func (r *tenderRepository) buildSearchFilter(form *SearchForm) bson.M {
return filter
}
// documentsScrapedSearchClause restricts list queries to tenders whose contract_folder_id
// currently has documents in MinIO (folder IDs are populated by the service layer).
func documentsScrapedSearchClause(form *SearchForm) bson.M {
if form == nil {
return bson.M{"contract_folder_id": bson.M{"$in": []string{}}}
}
return contractFolderIDsSearchClause(form.ContractFolderIDsWithDocuments)
}
func contractFolderIDsSearchClause(folderIDs []string) bson.M {
if len(folderIDs) == 0 {
return bson.M{"contract_folder_id": bson.M{"$in": []string{}}}
}
if len(folderIDs) <= maxContractFolderInClause {
return bson.M{"contract_folder_id": bson.M{"$in": folderIDs}}
}
or := make([]bson.M, 0, (len(folderIDs)+maxContractFolderInClause-1)/maxContractFolderInClause)
for i := 0; i < len(folderIDs); i += maxContractFolderInClause {
end := i + maxContractFolderInClause
if end > len(folderIDs) {
end = len(folderIDs)
}
or = append(or, bson.M{"contract_folder_id": bson.M{"$in": folderIDs[i:end]}})
}
return bson.M{"$or": or}
}
+32
View File
@@ -0,0 +1,32 @@
package tender
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestBuildSearchFilterDocumentsScrapedWithoutFolderIDsMatchesNothing(t *testing.T) {
repo := &tenderRepository{}
filter := repo.buildSearchFilter(&SearchForm{DocumentsScraped: true})
in, ok := filter["contract_folder_id"].(bson.M)["$in"].([]string)
if !ok || len(in) != 0 {
t.Fatalf("expected empty contract_folder_id $in filter, got %#v", filter)
}
}
func TestTenderSearchListProjectionExcludesHeavyFields(t *testing.T) {
projection := tenderSearchListProjection()
for _, field := range []string{
"content_xml",
"description",
"ai_overall_summary",
"procurement_lots",
"document_summaries",
} {
if projection[field] != 0 {
t.Fatalf("expected %s to be excluded from list projection", field)
}
}
}
+240
View File
@@ -0,0 +1,240 @@
package tender
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
goredis "github.com/redis/go-redis/v9"
"tm/pkg/response"
)
const (
searchListCacheTTL = 60 * time.Second
// searchListCacheStaleTTL bounds how long the default unfiltered list may lag after
// writes. Only isCacheableSearchList queries use this cache (~5 min max staleness).
searchListCacheStaleTTL = 5 * time.Minute
searchListCacheKeyPrefix = "tender:search:list:"
)
type searchListCacheEntry struct {
expiresAt time.Time
staleUntil time.Time
value *SearchResponse
}
type cachedSearchListPayload struct {
Tenders []TenderResponse `json:"tenders"`
Metadata *response.Meta `json:"metadata,omitempty"`
}
func searchResponseToCachePayload(resp *SearchResponse) *cachedSearchListPayload {
if resp == nil {
return nil
}
return &cachedSearchListPayload{
Tenders: resp.Tenders,
Metadata: resp.Metadata,
}
}
func cachePayloadToSearchResponse(payload *cachedSearchListPayload) *SearchResponse {
if payload == nil {
return nil
}
return &SearchResponse{
Tenders: payload.Tenders,
Metadata: payload.Metadata,
}
}
func isCacheableSearchList(form *SearchForm, pagination *response.Pagination) bool {
if form == nil || pagination == nil {
return false
}
if form.HasListFilters() {
return false
}
if form.CompanyID != nil && strings.TrimSpace(*form.CompanyID) != "" {
return false
}
if form.CustomerID != nil && strings.TrimSpace(*form.CustomerID) != "" {
return false
}
if len(form.CompanyIDs) > 0 {
return false
}
if pagination.Cursor != "" || pagination.IncludeTotal || pagination.Offset != 0 {
return false
}
if pagination.Limit <= 0 || pagination.Limit > 20 {
return false
}
sortBy := pagination.SortBy
if sortBy == "" {
sortBy = "created_at"
}
sortOrder := pagination.SortOrder
if sortOrder == "" {
sortOrder = "desc"
}
return sortBy == "created_at" && sortOrder == "desc"
}
func searchListCacheKey(pagination *response.Pagination, language string) string {
return fmt.Sprintf("%d:%s", pagination.Limit, strings.ToLower(strings.TrimSpace(language)))
}
func (s *tenderService) cachedSearchList(ctx context.Context, form *SearchForm, pagination *response.Pagination, language string, load func(context.Context) (*SearchResponse, error)) (*SearchResponse, error) {
if !isCacheableSearchList(form, pagination) {
return load(ctx)
}
// Cached *SearchResponse values (including the Tenders slice) are shared across
// concurrent callers via singleflight and stale hits; treat as read-only.
cacheKey := searchListCacheKey(pagination, language)
redisKey := searchListCacheKeyPrefix + cacheKey
if cached, ok := s.getFreshSearchListCache(cacheKey); ok {
return cached, nil
}
if stale, ok := s.getStaleSearchListCache(cacheKey); ok {
go s.refreshSearchListCache(cacheKey, redisKey, load)
return stale, nil
}
if cached, ok := s.getRedisSearchListCache(ctx, redisKey); ok {
s.storeSearchListCache(cacheKey, cached)
go s.refreshSearchListCache(cacheKey, redisKey, load)
return cached, nil
}
out, err, _ := s.searchListCacheGroup.Do(cacheKey, func() (interface{}, error) {
if cached, ok := s.getFreshSearchListCache(cacheKey); ok {
return cached, nil
}
result, err := load(context.WithoutCancel(ctx))
if err != nil {
return nil, err
}
s.storeSearchListCache(cacheKey, result)
s.storeRedisSearchListCache(context.Background(), redisKey, result)
return result, nil
})
if err != nil {
return nil, err
}
return out.(*SearchResponse), nil
}
func (s *tenderService) refreshSearchListCache(cacheKey, redisKey string, load func(context.Context) (*SearchResponse, error)) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
result, err := load(ctx)
if err != nil {
s.logger.Warn("Failed to refresh tender search list cache", map[string]interface{}{
"cache_key": cacheKey,
"error": err.Error(),
})
return
}
s.storeSearchListCache(cacheKey, result)
s.storeRedisSearchListCache(context.Background(), redisKey, result)
}
func (s *tenderService) getFreshSearchListCache(cacheKey string) (*SearchResponse, bool) {
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
entry, ok := s.searchListCache[cacheKey]
if !ok || time.Now().After(entry.expiresAt) {
return nil, false
}
return entry.value, true
}
func (s *tenderService) getStaleSearchListCache(cacheKey string) (*SearchResponse, bool) {
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
entry, ok := s.searchListCache[cacheKey]
if !ok || time.Now().After(entry.staleUntil) {
if ok {
delete(s.searchListCache, cacheKey)
}
return nil, false
}
return entry.value, true
}
func (s *tenderService) storeSearchListCache(cacheKey string, value *SearchResponse) {
now := time.Now()
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
s.searchListCache[cacheKey] = searchListCacheEntry{
expiresAt: now.Add(searchListCacheTTL),
staleUntil: now.Add(searchListCacheTTL + searchListCacheStaleTTL),
value: value,
}
}
func (s *tenderService) getRedisSearchListCache(ctx context.Context, redisKey string) (*SearchResponse, bool) {
if s.redisClient == nil {
return nil, false
}
raw, err := s.redisClient.Get(ctx, redisKey)
if err != nil {
if err != goredis.Nil {
s.logger.Warn("Failed to read tender search list cache from Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
}
return nil, false
}
var cached cachedSearchListPayload
if err := json.Unmarshal([]byte(raw), &cached); err != nil {
s.logger.Warn("Failed to decode tender search list cache from Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
_ = s.redisClient.Del(ctx, redisKey)
return nil, false
}
return cachePayloadToSearchResponse(&cached), true
}
func (s *tenderService) storeRedisSearchListCache(ctx context.Context, redisKey string, value *SearchResponse) {
if s.redisClient == nil || value == nil {
return
}
encoded, err := json.Marshal(searchResponseToCachePayload(value))
if err != nil {
s.logger.Warn("Failed to encode tender search list cache for Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
return
}
if err := s.redisClient.Set(ctx, redisKey, string(encoded), searchListCacheTTL+searchListCacheStaleTTL); err != nil {
s.logger.Warn("Failed to store tender search list cache in Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
}
}
+82 -33
View File
@@ -10,15 +10,18 @@ import (
"path"
"path/filepath"
"strings"
"sync"
"time"
"tm/internal/company"
"tm/internal/customer"
"tm/pkg/ai_summarizer"
"tm/pkg/logger"
orm "tm/pkg/mongo"
"tm/pkg/redis"
"tm/pkg/response"
"golang.org/x/sync/errgroup"
"golang.org/x/sync/singleflight"
)
// AISummarizerClient defines the interface for on-demand AI operations.
@@ -54,6 +57,8 @@ type Service interface {
// Tender operations
GetByID(ctx context.Context, id string, language string) (*TenderResponse, error)
GetByIDs(ctx context.Context, ids []string) (map[string]*TenderResponse, error)
// ResolveMongoID maps a MongoDB id or AI procedure ref to the canonical tender MongoDB id.
ResolveMongoID(ctx context.Context, tenderRef string) (string, error)
Update(ctx context.Context, req UpdateTenderRequest) (*TenderResponse, error)
Delete(ctx context.Context, id string) error
Search(ctx context.Context, req *SearchForm, pagination *response.Pagination) (*SearchResponse, error)
@@ -89,12 +94,22 @@ type tenderService struct {
logger logger.Logger
aiSummarizerClient AISummarizerClient
aiSummarizerStorage AISummarizerStorage
redisClient redis.Client
pageCacheTTL time.Duration
defaultLanguage string
searchListCacheMu sync.Mutex
searchListCache map[string]searchListCacheEntry
searchListCacheGroup singleflight.Group
contractFoldersCacheMu sync.Mutex
contractFoldersCache contractFoldersWithDocsCacheEntry
contractFoldersCacheGroup singleflight.Group
}
const (
aiSummaryEnrichmentTimeout = 2 * time.Second
aiTranslationEnrichmentTimeout = 2 * time.Second
aiSummaryEnrichmentTimeout = 2 * time.Second
aiTranslationEnrichmentTimeout = 2 * time.Second
recommendTranslationEnrichmentTimeout = 5 * time.Second
)
var (
@@ -111,6 +126,8 @@ func NewService(
logger logger.Logger,
aiClient AISummarizerClient,
aiStorage AISummarizerStorage,
redisClient redis.Client,
recommendationPageCacheTTL time.Duration,
defaultLanguage string,
) Service {
if strings.TrimSpace(defaultLanguage) == "" {
@@ -125,7 +142,10 @@ func NewService(
logger: logger,
aiSummarizerClient: aiClient,
aiSummarizerStorage: aiStorage,
redisClient: redisClient,
pageCacheTTL: recommendationPageCacheTTL,
defaultLanguage: strings.ToLower(defaultLanguage),
searchListCache: make(map[string]searchListCacheEntry),
}
}
@@ -173,6 +193,35 @@ func (s *tenderService) GetByIDs(ctx context.Context, ids []string) (map[string]
return out, nil
}
// ResolveMongoID maps a MongoDB id or AI procedure ref (PROC_{folder}/{notice}) to the canonical tender id.
func (s *tenderService) ResolveMongoID(ctx context.Context, tenderRef string) (string, error) {
tenderRef = strings.TrimSpace(tenderRef)
if tenderRef == "" {
return "", fmt.Errorf("tender reference is required")
}
if IsMongoObjectIDRef(tenderRef) {
if _, err := s.repository.GetByID(ctx, tenderRef); err != nil {
return "", fmt.Errorf("tender not found")
}
return tenderRef, nil
}
if folder, notice, ok := ParseAIProcedureRef(tenderRef); ok {
t, err := s.repository.GetByProcedureReference(ctx, folder, notice)
if err != nil || t == nil {
return "", fmt.Errorf("tender not found")
}
return t.GetID(), nil
}
if _, err := s.repository.GetByID(ctx, tenderRef); err == nil {
return tenderRef, nil
}
return "", fmt.Errorf("tender not found")
}
// buildDetailResponse maps a tender to API form and enriches translation (MinIO) and summary.
func (s *tenderService) buildDetailResponse(ctx context.Context, tender *Tender, language string) *TenderResponse {
resp := tender.ToResponseWithLanguage("")
@@ -207,6 +256,17 @@ func (s *tenderService) enrichWithAISummary(ctx context.Context, t *Tender, resp
// enrichWithTranslation overlays title/description from MinIO when the pipeline has finished
// for the requested language. Does not call the AI translate API on GET.
func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, resp *TenderResponse, language string, cache *recommendationTranslationCache) {
s.enrichWithTranslationTimeout(ctx, t, resp, language, cache, aiTranslationEnrichmentTimeout)
}
func (s *tenderService) enrichWithTranslationTimeout(
ctx context.Context,
t *Tender,
resp *TenderResponse,
language string,
cache *recommendationTranslationCache,
enrichmentTimeout time.Duration,
) {
if t == nil || resp == nil {
return
}
@@ -226,7 +286,7 @@ func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, re
}
}
enrichCtx, cancel := context.WithTimeout(ctx, aiTranslationEnrichmentTimeout)
enrichCtx, cancel := context.WithTimeout(ctx, enrichmentTimeout)
defer cancel()
stored, usedNotice, err := s.lookupTranslationForTender(enrichCtx, t, targetLanguage)
@@ -872,8 +932,13 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
"to": form.DeadlineTo,
})
// Profile keywords are not used for tender search.
language := s.pickResponseLanguage(form.Language)
return s.cachedSearchList(ctx, form, pagination, language, func(loadCtx context.Context) (*SearchResponse, error) {
return s.searchUncached(loadCtx, form, pagination, language)
})
}
func (s *tenderService) searchUncached(ctx context.Context, form *SearchForm, pagination *response.Pagination, language string) (*SearchResponse, error) {
if form.DocumentsScraped {
if err := s.enrichDocumentsScrapedFilter(ctx, form); err != nil {
return nil, err
@@ -895,7 +960,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
if form.CompanyID == nil || *form.CompanyID == "" {
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -914,7 +979,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
// Continue without match percentage if company not found
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -937,7 +1002,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -967,6 +1032,11 @@ func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, paginat
"company_ids": companyIDs,
})
lang := s.pickResponseLanguage(form.Language)
if result, ok := s.recommendFromPageCache(ctx, companyID, companyIDs, form, pagination, lang); ok {
return result, nil
}
var (
recommendations []company.RecommendedTenderResponse
err error
@@ -983,6 +1053,10 @@ func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, paginat
return s.emptySearchResponse(pagination), nil
}
if len(companyIDs) == 0 && companyID != "" {
s.schedulePageCacheRefreshForLanguage(companyID, lang)
}
excluded, excludedByCompany, err := s.loadRecommendationExclusions(ctx, form, companyID, companyIDs)
if err != nil {
if form.ExcludeRejectedTenders {
@@ -995,7 +1069,6 @@ func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, paginat
return nil, err
}
lang := s.pickResponseLanguage(form.Language)
now := time.Now().Unix()
pageResult, err := s.buildRecommendationPage(
ctx,
@@ -1013,6 +1086,7 @@ func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, paginat
}
paged := s.buildRecommendedTenderListResponses(pageResult.items, lang)
paged = s.enrichRecommendedTenderResponsesParallel(ctx, paged, lang)
return &SearchResponse{
Tenders: paged,
@@ -1698,31 +1772,6 @@ func (s *tenderService) listProceduresWithDocuments(ctx context.Context) ([]ai_s
return procedures, nil
}
func (s *tenderService) enrichDocumentsScrapedFilter(ctx context.Context, form *SearchForm) error {
procedures, err := s.listProceduresWithDocuments(ctx)
if err != nil {
s.logger.Error("Failed to resolve tenders with scraped documents from MinIO", map[string]interface{}{
"error": err.Error(),
})
if errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
return fmt.Errorf("failed to resolve tenders with scraped documents: MinIO connection unavailable: %w", err)
}
return fmt.Errorf("failed to resolve tenders with scraped documents: %w", err)
}
folderIDs := make([]string, 0, len(procedures))
for _, proc := range procedures {
if proc.DocumentCount <= 0 {
continue
}
if id := strings.TrimSpace(proc.ContractFolderID); id != "" {
folderIDs = append(folderIDs, id)
}
}
form.ContractFolderIDsWithDocuments = folderIDs
return nil
}
func (s *tenderService) resolveRecommendedTenders(ctx context.Context, recommendations []company.RecommendedTenderResponse) (map[string]Tender, error) {
out := make(map[string]Tender)
procRefs := make([]ProcedureReference, 0, len(recommendations))
-1
View File
@@ -148,7 +148,6 @@ func (ta *TenderApproval) Submit() {
// Reject rejects the tender with a reason
func (ta *TenderApproval) Reject() {
ta.Status = ApprovalStatusRejected
ta.SubmissionMode = ""
ta.CreatedAt = time.Now().Unix()
}
+1
View File
@@ -14,4 +14,5 @@ var (
ErrInvalidCompanyID = errors.New("invalid company ID")
ErrUnauthorized = errors.New("unauthorized to perform this action")
ErrInvalidData = errors.New("invalid tender approval data")
ErrSubmissionSyncFailed = errors.New("failed to sync tender submission workflow")
)
+1
View File
@@ -81,6 +81,7 @@ type CompanyTenderApprovalStatsResponse struct {
// ToggleTenderApprovalForm represents the form for toggling a tender approval
type ToggleTenderApprovalForm struct {
TenderID string `json:"tender_id" valid:"required"`
CompanyID *string `json:"company_id,omitempty" valid:"optional"`
Status ApprovalStatus `json:"status" valid:"required,in(submitted|rejected)"`
SubmissionMode SubmissionMode `json:"submission_mode" valid:"required,in(self-apply|partnership)"`
}
+24 -5
View File
@@ -1,7 +1,10 @@
package tender_approval
import (
"errors"
"strconv"
"strings"
"tm/internal/customer"
"tm/pkg/logger"
"tm/pkg/response"
@@ -42,14 +45,30 @@ func (h *Handler) ToggleTenderApproval(c echo.Context) error {
return response.ValidationError(c, "Invalid request data", err.Error())
}
// Get company ID from context
companyID := c.Get("company_id").(string)
if companyID == "" {
return response.Unauthorized(c, "Unauthorized")
requestedCompanyID := strings.TrimSpace(c.QueryParam("company_id"))
if form.CompanyID != nil && strings.TrimSpace(*form.CompanyID) != "" {
requestedCompanyID = strings.TrimSpace(*form.CompanyID)
}
tenderApproval, err := h.service.ToggleTenderApproval(c.Request().Context(), form, companyID)
activeCompanyID, _ := c.Get("company_id").(string)
assignedCompanyIDs, _ := c.Get("company_ids").([]string)
companyID, err := customer.PickAssignedCompanyID(activeCompanyID, assignedCompanyIDs, requestedCompanyID)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
tenderApproval, err := h.service.ToggleTenderApproval(c.Request().Context(), form, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) {
return response.NotFound(c, "Tender not found")
}
if errors.Is(err, ErrSubmissionSyncFailed) {
return response.InternalServerError(c, "Failed to sync tender submission workflow")
}
if err.Error() == "tender approval already exists for this tender and company" {
return response.Conflict(c, err.Error())
}
+5 -5
View File
@@ -104,7 +104,7 @@ func (r *tenderApprovalRepository) GetByID(ctx context.Context, id string) (*Ten
tenderApproval, err := r.ormRepo.FindByID(ctx, id)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, errors.New("tender approval not found")
return nil, ErrTenderApprovalNotFound
}
r.logger.Error("Failed to get tender approval by ID", map[string]interface{}{
"error": err.Error(),
@@ -126,7 +126,7 @@ func (r *tenderApprovalRepository) GetByTenderAndCompany(ctx context.Context, te
tenderApproval, err := r.ormRepo.FindOne(ctx, filter)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, errors.New("tender approval not found")
return nil, ErrTenderApprovalNotFound
}
r.logger.Error("Failed to get tender approval by tender and company", map[string]interface{}{
"error": err.Error(),
@@ -516,11 +516,11 @@ func (r *tenderApprovalRepository) GetCompanyTenderApprovalStats(ctx context.Con
rejectedFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusRejected)}
rejectedTenders, _ := r.ormRepo.Count(ctx, rejectedFilter)
// Count by submission mode for this company
selfApplyFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusSubmitted), "submission_mode": string(SubmissionModeSelfApply)}
// Count by submission mode for this company (includes submitted and rejected decisions).
selfApplyFilter := bson.M{"company_id": companyID, "submission_mode": string(SubmissionModeSelfApply)}
selfApplyCount, _ := r.ormRepo.Count(ctx, selfApplyFilter)
partnershipFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusSubmitted), "submission_mode": string(SubmissionModePartnership)}
partnershipFilter := bson.M{"company_id": companyID, "submission_mode": string(SubmissionModePartnership)}
partnershipCount, _ := r.ormRepo.Count(ctx, partnershipFilter)
stats := &CompanyTenderApprovalStatsResponse{
+110 -28
View File
@@ -4,12 +4,20 @@ import (
"context"
"errors"
"math"
"strings"
"time"
"tm/internal/tender"
"tm/pkg/logger"
"tm/pkg/response"
)
// SubmissionLifecycleHook synchronizes tender submission workflow with approval changes.
type SubmissionLifecycleHook interface {
OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error
OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error
OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error
}
// Service defines business logic for tender approval operations
type Service interface {
// Core tender approval operations
@@ -33,7 +41,7 @@ type Service interface {
SearchTenderApprovalsByCriteria(ctx context.Context, criteria *TenderApprovalSearchCriteria) ([]*TenderApproval, error)
// Business operations
ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID string) (*TenderApprovalWithTenderResponse, error)
ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID, customerID string) (*TenderApprovalWithTenderResponse, error)
// Statistics and analytics
GetTenderApprovalStats(ctx context.Context) (*TenderApprovalStatsResponse, error)
@@ -50,17 +58,19 @@ type Service interface {
// tenderApprovalService implements the Service interface
type tenderApprovalService struct {
repository Repository
tenderService tender.Service
logger logger.Logger
repository Repository
tenderService tender.Service
logger logger.Logger
submissionHook SubmissionLifecycleHook
}
// NewService creates a new tender approval service
func NewService(repository Repository, tenderService tender.Service, logger logger.Logger) Service {
func NewService(repository Repository, tenderService tender.Service, logger logger.Logger, submissionHook SubmissionLifecycleHook) Service {
return &tenderApprovalService{
repository: repository,
tenderService: tenderService,
logger: logger,
repository: repository,
tenderService: tenderService,
logger: logger,
submissionHook: submissionHook,
}
}
@@ -852,11 +862,35 @@ func (s *tenderApprovalService) ListTenderApprovalsWithTender(ctx context.Contex
return meta, nil
}
func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID string) (*TenderApprovalWithTenderResponse, error) {
existing, err := s.repository.GetByTenderAndCompany(ctx, req.TenderID, companyID)
func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID, customerID string) (*TenderApprovalWithTenderResponse, error) {
companyID = strings.TrimSpace(companyID)
if companyID == "" {
return nil, ErrInvalidCompanyID
}
tenderID, err := s.tenderService.ResolveMongoID(ctx, req.TenderID)
if err != nil {
s.logger.Warn("Failed to resolve tender for approval toggle", map[string]interface{}{
"tender_ref": req.TenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, ErrTenderNotFound
}
existing, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if !errors.Is(err, ErrTenderApprovalNotFound) {
s.logger.Error("Failed to load tender approval for toggle", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, err
}
approval := &TenderApproval{
TenderID: req.TenderID,
TenderID: tenderID,
CompanyID: companyID,
SubmissionMode: req.SubmissionMode,
Status: req.Status,
@@ -865,7 +899,7 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
err = s.repository.Create(ctx, approval)
if err != nil {
s.logger.Error("Failed to create new tender approval", map[string]interface{}{
"tender_id": req.TenderID,
"tender_id": tenderID,
"company_id": companyID,
"status": req.Status,
"submission_mode": req.SubmissionMode,
@@ -874,13 +908,22 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
return nil, err
}
return approval.ToResponseWithTender(nil), nil
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, true); err != nil {
if deleteErr := s.repository.Delete(ctx, approval.GetID()); deleteErr != nil {
s.logger.Error("Failed to rollback tender approval after submission sync failure", map[string]interface{}{
"tender_approval_id": approval.GetID(),
"error": deleteErr.Error(),
})
}
return nil, ErrSubmissionSyncFailed
}
return s.toggleResponseWithTender(ctx, approval)
}
if existing.Status == req.Status {
err = s.repository.Delete(ctx, existing.GetID())
if err != nil {
s.logger.Error("Failed to update tender approval", map[string]interface{}{
s.logger.Error("Failed to delete tender approval during toggle", map[string]interface{}{
"error": err.Error(),
})
return nil, err
@@ -892,24 +935,21 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
existing.Status = "unrejected"
}
return existing.ToResponseWithTender(nil), nil
} else if existing.Status == req.Status && existing.SubmissionMode != req.SubmissionMode {
existing.SubmissionMode = req.SubmissionMode
err = s.repository.Update(ctx, existing)
if err != nil {
s.logger.Error("Failed to update tender approval", map[string]interface{}{
"error": err.Error(),
})
return nil, err
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, false); err != nil {
return nil, ErrSubmissionSyncFailed
}
return existing.ToResponseWithTender(nil), nil
}
if existing.Status == ApprovalStatusSubmitted {
existing.Reject()
} else {
if existing.Status == ApprovalStatusRejected && req.Status == ApprovalStatusSubmitted {
existing.Submit()
existing.SubmissionMode = req.SubmissionMode
} else if existing.Status == ApprovalStatusSubmitted && req.Status == ApprovalStatusRejected {
existing.Reject()
existing.SubmissionMode = req.SubmissionMode
} else {
existing.Status = req.Status
existing.SubmissionMode = req.SubmissionMode
}
err = s.repository.Update(ctx, existing)
@@ -920,7 +960,49 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
return nil, err
}
return existing.ToResponseWithTender(nil), nil
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, true); err != nil {
return nil, ErrSubmissionSyncFailed
}
return s.toggleResponseWithTender(ctx, existing)
}
func (s *tenderApprovalService) syncSubmissionAfterToggle(ctx context.Context, tenderID, companyID, customerID string, status ApprovalStatus, active bool) error {
if s.submissionHook == nil {
return nil
}
var err error
if !active {
err = s.submissionHook.OnApprovalRemoved(ctx, tenderID, companyID)
} else if status == ApprovalStatusSubmitted {
err = s.submissionHook.OnApprovalSubmitted(ctx, tenderID, companyID, customerID)
} else if status == ApprovalStatusRejected {
err = s.submissionHook.OnApprovalRejected(ctx, tenderID, companyID, customerID)
}
if err != nil {
s.logger.Error("Failed to sync tender submission workflow after approval toggle", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"status": status,
"active": active,
"error": err.Error(),
})
}
return err
}
func (s *tenderApprovalService) toggleResponseWithTender(ctx context.Context, approval *TenderApproval) (*TenderApprovalWithTenderResponse, error) {
if approval == nil {
return nil, ErrInvalidData
}
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
return approval.ToResponseWithTender(nil), nil
}
return approval.ToResponseWithTender(listTenderDetailsFromResponse(tenderResponse)), nil
}
// GetCompanyTenderApprovalStats returns company-specific tender approval statistics
+168
View File
@@ -0,0 +1,168 @@
package tender_submission
import (
"time"
"tm/pkg/mongo"
)
// SubmissionStatus represents the current step in the tender submission workflow.
type SubmissionStatus string
const (
StatusOpportunity SubmissionStatus = "opportunity"
StatusValidating SubmissionStatus = "validating"
StatusPartnerPending SubmissionStatus = "partner_pending"
StatusApplying SubmissionStatus = "applying"
StatusSentWaiting SubmissionStatus = "sent_waiting"
StatusRejected SubmissionStatus = "rejected"
StatusContract SubmissionStatus = "contract"
StatusNotApplied SubmissionStatus = "not_applied"
)
// SubmissionStage groups statuses into high-level workflow stages for UI display.
type SubmissionStage string
const (
StageOpportunity SubmissionStage = "opportunity"
StageInProgress SubmissionStage = "in_progress"
StageSubmitted SubmissionStage = "submitted"
StageNotApplied SubmissionStage = "not_applied"
)
// StatusHistory records a status change in the submission workflow.
type StatusHistory struct {
Status SubmissionStatus `bson:"status" json:"status"`
Reason string `bson:"reason,omitempty" json:"reason,omitempty"`
ChangedBy string `bson:"changed_by" json:"changed_by"`
ChangedAt int64 `bson:"changed_at" json:"changed_at"`
Description string `bson:"description,omitempty" json:"description,omitempty"`
}
// TenderSubmission tracks a company's tender submission workflow.
type TenderSubmission struct {
mongo.Model `bson:",inline"`
TenderID string `bson:"tender_id" json:"tender_id"`
CompanyID string `bson:"company_id" json:"company_id"`
CustomerID string `bson:"customer_id,omitempty" json:"customer_id,omitempty"`
Status SubmissionStatus `bson:"status" json:"status"`
Stage SubmissionStage `bson:"stage" json:"stage"`
History []StatusHistory `bson:"status_history" json:"status_history"`
}
func StageForStatus(status SubmissionStatus) SubmissionStage {
switch status {
case StatusOpportunity:
return StageOpportunity
case StatusValidating, StatusPartnerPending, StatusApplying:
return StageInProgress
case StatusSentWaiting, StatusRejected, StatusContract:
return StageSubmitted
case StatusNotApplied:
return StageNotApplied
default:
return StageOpportunity
}
}
func IsTerminalStatus(status SubmissionStatus) bool {
switch status {
case StatusRejected, StatusContract, StatusNotApplied:
return true
default:
return false
}
}
func AllowedNextStatuses(current SubmissionStatus) []SubmissionStatus {
transitions := map[SubmissionStatus][]SubmissionStatus{
StatusOpportunity: {StatusValidating, StatusNotApplied},
StatusValidating: {StatusPartnerPending, StatusApplying, StatusNotApplied},
StatusPartnerPending: {StatusApplying, StatusNotApplied},
StatusApplying: {StatusSentWaiting, StatusNotApplied},
StatusSentWaiting: {StatusRejected, StatusContract},
StatusRejected: {},
StatusContract: {},
StatusNotApplied: {},
}
return transitions[current]
}
func (ts *TenderSubmission) GetID() string {
return ts.ID.Hex()
}
func (ts *TenderSubmission) SetCreatedAt(timestamp int64) {
ts.CreatedAt = timestamp
}
func (ts *TenderSubmission) SetUpdatedAt(timestamp int64) {
ts.UpdatedAt = timestamp
}
func (ts *TenderSubmission) AddStatusChange(status SubmissionStatus, reason, changedBy, description string) {
ts.History = append(ts.History, StatusHistory{
Status: status,
Reason: reason,
ChangedBy: changedBy,
ChangedAt: time.Now().Unix(),
Description: description,
})
ts.Status = status
ts.Stage = StageForStatus(status)
ts.SetUpdatedAt(time.Now().Unix())
}
// ReopenForApproval resets a terminal submission when the tender is approved again.
// This is only used by the approval sync path, not by user-initiated status updates.
func (ts *TenderSubmission) ReopenForApproval(changedBy, description string) {
ts.AddStatusChange(StatusOpportunity, "approval_resubmitted", changedBy, description)
}
func NewTenderSubmission(tenderID, companyID, customerID string) *TenderSubmission {
now := time.Now().Unix()
ts := &TenderSubmission{
TenderID: tenderID,
CompanyID: companyID,
CustomerID: customerID,
Status: StatusOpportunity,
Stage: StageOpportunity,
}
ts.SetCreatedAt(now)
ts.SetUpdatedAt(now)
ts.AddStatusChange(StatusOpportunity, "", customerID, "Submission workflow started")
return ts
}
func (ts *TenderSubmission) ToResponse() *TenderSubmissionResponse {
history := make([]StatusHistoryResponse, len(ts.History))
for i, h := range ts.History {
history[i] = StatusHistoryResponse{
Status: string(h.Status),
Reason: h.Reason,
ChangedBy: h.ChangedBy,
ChangedAt: h.ChangedAt,
Description: h.Description,
}
}
return &TenderSubmissionResponse{
ID: ts.GetID(),
TenderID: ts.TenderID,
CompanyID: ts.CompanyID,
CustomerID: ts.CustomerID,
Status: string(ts.Status),
Stage: string(ts.Stage),
History: history,
CreatedAt: ts.CreatedAt,
UpdatedAt: ts.UpdatedAt,
}
}
func (ts *TenderSubmission) ToResponseWithTender(tender *TenderDetails) *TenderSubmissionWithTenderResponse {
resp := ts.ToResponse()
return &TenderSubmissionWithTenderResponse{
TenderSubmissionResponse: *resp,
Tender: tender,
}
}
+98
View File
@@ -0,0 +1,98 @@
package tender_submission
import "testing"
func TestStageForStatus(t *testing.T) {
tests := []struct {
status SubmissionStatus
stage SubmissionStage
}{
{StatusOpportunity, StageOpportunity},
{StatusValidating, StageInProgress},
{StatusPartnerPending, StageInProgress},
{StatusApplying, StageInProgress},
{StatusSentWaiting, StageSubmitted},
{StatusRejected, StageSubmitted},
{StatusContract, StageSubmitted},
{StatusNotApplied, StageNotApplied},
}
for _, tt := range tests {
if got := StageForStatus(tt.status); got != tt.stage {
t.Fatalf("StageForStatus(%q) = %q, want %q", tt.status, got, tt.stage)
}
}
}
func TestAllowedNextStatuses(t *testing.T) {
tests := []struct {
current SubmissionStatus
next SubmissionStatus
allowed bool
}{
{StatusOpportunity, StatusValidating, true},
{StatusOpportunity, StatusNotApplied, true},
{StatusOpportunity, StatusApplying, false},
{StatusValidating, StatusPartnerPending, true},
{StatusValidating, StatusApplying, true},
{StatusValidating, StatusNotApplied, true},
{StatusPartnerPending, StatusApplying, true},
{StatusApplying, StatusSentWaiting, true},
{StatusSentWaiting, StatusContract, true},
{StatusSentWaiting, StatusRejected, true},
{StatusContract, StatusRejected, false},
{StatusNotApplied, StatusValidating, false},
}
for _, tt := range tests {
allowed := false
for _, candidate := range AllowedNextStatuses(tt.current) {
if candidate == tt.next {
allowed = true
break
}
}
if allowed != tt.allowed {
t.Fatalf("transition %q -> %q allowed=%v, want %v", tt.current, tt.next, allowed, tt.allowed)
}
}
}
func TestAddStatusChangeUpdatesStage(t *testing.T) {
submission := NewTenderSubmission("tender-1", "company-1", "customer-1")
submission.AddStatusChange(StatusValidating, "", "customer-1", "Accepted tender")
if submission.Status != StatusValidating {
t.Fatalf("status = %q, want %q", submission.Status, StatusValidating)
}
if submission.Stage != StageInProgress {
t.Fatalf("stage = %q, want %q", submission.Stage, StageInProgress)
}
if len(submission.History) != 2 {
t.Fatalf("history length = %d, want 2", len(submission.History))
}
}
func TestValidateStatusTransition(t *testing.T) {
service := &tenderSubmissionService{}
if err := service.validateStatusTransition(StatusOpportunity, StatusValidating); err != nil {
t.Fatalf("expected valid transition, got %v", err)
}
if err := service.validateStatusTransition(StatusContract, StatusRejected); err == nil {
t.Fatal("expected terminal status error")
}
err := service.validateStatusTransition(StatusOpportunity, StatusApplying)
if err == nil {
t.Fatal("expected invalid transition error")
}
transitionErr, ok := err.(*InvalidStatusTransitionError)
if !ok {
t.Fatalf("expected InvalidStatusTransitionError, got %T", err)
}
if transitionErr.CurrentStatus != string(StatusOpportunity) {
t.Fatalf("current status = %q", transitionErr.CurrentStatus)
}
}
+46
View File
@@ -0,0 +1,46 @@
package tender_submission
import (
"errors"
"fmt"
"strings"
)
var (
ErrTenderSubmissionNotFound = errors.New("tender submission not found")
ErrTenderSubmissionExists = errors.New("tender submission already exists for this tender and company")
ErrInvalidStatus = errors.New("invalid tender submission status")
ErrInvalidStage = errors.New("invalid tender submission stage")
ErrTenderNotFound = errors.New("tender not found")
ErrApprovalRequired = errors.New("tender must be approved before starting submission workflow")
ErrInvalidStatusTransition = errors.New("invalid status transition")
ErrTerminalStatus = errors.New("tender submission is in a terminal state and cannot be updated")
ErrConcurrentUpdate = errors.New("tender submission was modified concurrently")
)
// InvalidStatusTransitionError is returned when a status change is not allowed.
type InvalidStatusTransitionError struct {
CurrentStatus string
NewStatus string
Allowed []string
}
func (e *InvalidStatusTransitionError) Error() string {
if len(e.Allowed) == 0 {
return fmt.Sprintf("Tender submission status is already %s and cannot be changed", e.CurrentStatus)
}
return fmt.Sprintf(
"Cannot change tender submission status from %s to %s. Allowed next statuses: %s",
e.CurrentStatus,
e.NewStatus,
strings.Join(e.Allowed, ", "),
)
}
func NewInvalidStatusTransitionError(currentStatus, newStatus string, allowed []string) *InvalidStatusTransitionError {
return &InvalidStatusTransitionError{
CurrentStatus: currentStatus,
NewStatus: newStatus,
Allowed: allowed,
}
}
+100
View File
@@ -0,0 +1,100 @@
package tender_submission
import "tm/pkg/response"
// UpdateSubmissionStatusForm updates the workflow status of a tender submission.
type UpdateSubmissionStatusForm struct {
Status SubmissionStatus `json:"status" valid:"required,in(opportunity|validating|partner_pending|applying|sent_waiting|rejected|contract|not_applied)"`
Reason string `json:"reason,omitempty" valid:"optional,length(0|500)"`
Description string `json:"description,omitempty" valid:"optional,length(0|1000)"`
}
// ListTenderSubmissionsForm filters company tender submissions.
type ListTenderSubmissionsForm struct {
TenderID *string `query:"tender_id" valid:"optional"`
CompanyID *string `query:"company_id" valid:"optional"`
Status []string `query:"status" valid:"optional"`
Stage []string `query:"stage" valid:"optional"`
From *int64 `query:"created_from" valid:"optional"`
To *int64 `query:"created_to" valid:"optional"`
Limit *int `query:"limit" valid:"optional,range(1|100)"`
Offset *int `query:"offset" valid:"optional,range(0|1000000)"`
SortBy *string `query:"sort_by" valid:"optional,in(created_at|updated_at|status|stage)"`
SortOrder *string `query:"sort_order" valid:"optional,in(asc|desc)"`
}
// TenderSubmissionResponse is the API response for a tender submission.
type TenderSubmissionResponse struct {
ID string `json:"id"`
TenderID string `json:"tender_id"`
CompanyID string `json:"company_id"`
CustomerID string `json:"customer_id,omitempty"`
Status string `json:"status"`
Stage string `json:"stage"`
History []StatusHistoryResponse `json:"status_history"`
CreatedAt int64 `json:"created_at"`
UpdatedAt int64 `json:"updated_at"`
}
// StatusHistoryResponse is the API representation of a status change.
type StatusHistoryResponse struct {
Status string `json:"status"`
Reason string `json:"reason,omitempty"`
ChangedBy string `json:"changed_by"`
ChangedAt int64 `json:"changed_at"`
Description string `json:"description,omitempty"`
}
// TenderSubmissionWithTenderResponse includes tender details.
type TenderSubmissionWithTenderResponse struct {
TenderSubmissionResponse
Tender *TenderDetails `json:"tender,omitempty"`
}
// TenderSubmissionListResponse is a paginated list of submissions with tender details.
type TenderSubmissionListResponse struct {
Submissions []*TenderSubmissionWithTenderResponse `json:"submissions"`
Metadata *response.Meta `json:"metadata"`
}
// TenderDetails represents essential tender information for submission responses.
type TenderDetails struct {
ID string `json:"id"`
NoticePublicationID string `json:"notice_publication_id"`
PublicationDate int64 `json:"publication_date"`
Title string `json:"title"`
Description string `json:"description"`
ProcurementTypeCode string `json:"procurement_type_code"`
ProcedureCode string `json:"procedure_code"`
EstimatedValue float64 `json:"estimated_value"`
Currency string `json:"currency"`
Duration string `json:"duration"`
DurationUnit string `json:"duration_unit"`
TenderDeadline int64 `json:"tender_deadline"`
SubmissionDeadline int64 `json:"submission_deadline"`
CountryCode string `json:"country_code"`
BuyerOrganization *OrganizationResponse `json:"buyer_organization"`
Status string `json:"status"`
}
// OrganizationResponse represents buyer organization info.
type OrganizationResponse struct {
Name string `json:"name"`
}
// CompanySubmissionStatsResponse contains submission counts grouped by stage and status.
type CompanySubmissionStatsResponse struct {
CompanyID string `json:"company_id"`
Total int64 `json:"total"`
ByStage map[string]int64 `json:"by_stage"`
ByStatus map[string]int64 `json:"by_status"`
LastUpdated int64 `json:"last_updated"`
}
// AdminSubmissionStatsResponse contains global submission statistics.
type AdminSubmissionStatsResponse struct {
Total int64 `json:"total"`
ByStage map[string]int64 `json:"by_stage"`
ByStatus map[string]int64 `json:"by_status"`
LastUpdated int64 `json:"last_updated"`
}
+334
View File
@@ -0,0 +1,334 @@
package tender_submission
import (
"errors"
"strings"
"tm/internal/customer"
"tm/pkg/logger"
"tm/pkg/response"
"github.com/labstack/echo/v4"
)
// Handler handles HTTP requests for tender submission operations.
type Handler struct {
service Service
logger logger.Logger
}
func NewHandler(service Service, logger logger.Logger) *Handler {
return &Handler{service: service, logger: logger}
}
// ListCompanySubmissions lists tender submissions for the authenticated company.
// @Summary List company tender submissions
// @Description Retrieve paginated tender submission workflows for the authenticated company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param status query []string false "Filter by status"
// @Param stage query []string false "Filter by stage"
// @Param created_from query int false "Filter by created date from (Unix timestamp)"
// @Param created_to query int false "Filter by created date to (Unix timestamp)"
// @Param limit query int false "Page size (default 20, max 100)"
// @Param offset query int false "Pagination offset"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionListResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions [get]
func (h *Handler) ListCompanySubmissions(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
form, err := response.Parse[ListTenderSubmissionsForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
result, err := h.service.ListByCompanyWithTender(c.Request().Context(), companyID, form)
if err != nil {
return response.InternalServerError(c, "Failed to list tender submissions")
}
return response.SuccessWithMeta(c, result.Submissions, result.Metadata, "Tender submissions retrieved successfully")
}
// EnsureSubmission ensures a submission workflow exists for an approved tender.
// @Summary Ensure tender submission workflow
// @Description Create or return the submission workflow for an approved tender
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param tender_id path string true "Tender ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 400 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/tender/{tender_id}/ensure [post]
func (h *Handler) EnsureSubmission(c echo.Context) error {
companyID, err := h.resolveCompanyID(c)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
tenderID := strings.TrimSpace(c.Param("tender_id"))
if tenderID == "" {
return response.BadRequest(c, "Tender ID is required", "")
}
result, err := h.service.EnsureForApprovedTender(c.Request().Context(), tenderID, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) {
return response.NotFound(c, "Tender not found")
}
if errors.Is(err, ErrApprovalRequired) {
return response.BadRequest(c, "Tender must be approved before starting submission workflow", "")
}
return response.InternalServerError(c, "Failed to ensure tender submission")
}
return response.Success(c, result, "Tender submission ensured successfully")
}
// GetSubmissionByID retrieves a tender submission by ID for the authenticated company.
// @Summary Get tender submission by ID
// @Description Retrieve a tender submission workflow with tender details for the authenticated company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/{id} [get]
func (h *Handler) GetSubmissionByID(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
result, err := h.service.GetByIDForCompanyWithTender(c.Request().Context(), id, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// GetSubmissionByTender retrieves a submission by tender and company.
// @Summary Get tender submission by tender
// @Description Retrieve the submission workflow for a specific tender and company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param tender_id path string true "Tender ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/tender/{tender_id} [get]
func (h *Handler) GetSubmissionByTender(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
tenderID := strings.TrimSpace(c.Param("tender_id"))
if tenderID == "" {
return response.BadRequest(c, "Tender ID is required", "")
}
result, err := h.service.GetByTenderAndCompanyWithTender(c.Request().Context(), tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) || errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// UpdateSubmissionStatus updates the workflow status of a submission.
// @Summary Update tender submission status
// @Description Move a tender submission to the next workflow step
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Param body body UpdateSubmissionStatusForm true "Status update"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 400 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 422 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/{id}/status [patch]
func (h *Handler) UpdateSubmissionStatus(c echo.Context) error {
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
form, err := response.Parse[UpdateSubmissionStatusForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
companyID, err := h.resolveCompanyID(c)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
result, err := h.service.UpdateStatus(c.Request().Context(), id, form, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
var transitionErr *InvalidStatusTransitionError
if errors.As(err, &transitionErr) {
return response.ValidationError(c, "Invalid status transition", err.Error())
}
if errors.Is(err, ErrTerminalStatus) {
return response.ValidationError(c, "Submission is in a terminal state", err.Error())
}
if errors.Is(err, ErrConcurrentUpdate) {
return response.Conflict(c, "Tender submission was modified concurrently")
}
return response.InternalServerError(c, "Failed to update tender submission status")
}
return response.Success(c, result, "Tender submission status updated successfully")
}
// GetCompanySubmissionStats returns submission statistics for the company.
// @Summary Get company tender submission statistics
// @Description Retrieve submission counts grouped by stage and status
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Success 200 {object} response.APIResponse{data=CompanySubmissionStatsResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/stats [get]
func (h *Handler) GetCompanySubmissionStats(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
stats, err := h.service.GetCompanyStats(c.Request().Context(), companyID)
if err != nil {
return response.InternalServerError(c, "Failed to get submission statistics")
}
return response.Success(c, stats, "Submission statistics retrieved successfully")
}
// AdminListSubmissions lists tender submissions for admin users.
// @Summary Admin list tender submissions
// @Description Retrieve paginated tender submissions across companies
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Param company_id query string false "Filter by company ID"
// @Param tender_id query string false "Filter by tender ID"
// @Param status query []string false "Filter by status"
// @Param stage query []string false "Filter by stage"
// @Param limit query int false "Page size"
// @Param offset query int false "Pagination offset"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionListResponse}
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions [get]
func (h *Handler) AdminListSubmissions(c echo.Context) error {
form, err := response.Parse[ListTenderSubmissionsForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
result, err := h.service.AdminListWithTender(c.Request().Context(), form)
if err != nil {
return response.InternalServerError(c, "Failed to list tender submissions")
}
return response.SuccessWithMeta(c, result.Submissions, result.Metadata, "Tender submissions retrieved successfully")
}
// AdminGetSubmissionByID retrieves a submission by ID for admin users.
// @Summary Admin get tender submission by ID
// @Description Retrieve a tender submission workflow with tender details
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions/{id} [get]
func (h *Handler) AdminGetSubmissionByID(c echo.Context) error {
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
result, err := h.service.GetByIDWithTender(c.Request().Context(), id)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// AdminGetSubmissionStats returns global submission statistics.
// @Summary Admin get tender submission statistics
// @Description Retrieve global submission counts grouped by stage and status
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Success 200 {object} response.APIResponse{data=AdminSubmissionStatsResponse}
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions/stats [get]
func (h *Handler) AdminGetSubmissionStats(c echo.Context) error {
stats, err := h.service.GetGlobalStats(c.Request().Context())
if err != nil {
return response.InternalServerError(c, "Failed to get submission statistics")
}
return response.Success(c, stats, "Submission statistics retrieved successfully")
}
func (h *Handler) resolveCompanyID(c echo.Context) (string, error) {
requestedCompanyID := strings.TrimSpace(c.QueryParam("company_id"))
activeCompanyID, _ := c.Get("company_id").(string)
assignedCompanyIDs, _ := c.Get("company_ids").([]string)
return customer.PickAssignedCompanyID(activeCompanyID, assignedCompanyIDs, requestedCompanyID)
}
+346
View File
@@ -0,0 +1,346 @@
package tender_submission
import (
"context"
"errors"
"time"
"tm/pkg/logger"
"tm/pkg/mongo"
"go.mongodb.org/mongo-driver/v2/bson"
mongodriver "go.mongodb.org/mongo-driver/v2/mongo"
)
const maxListLimit = 100
// Repository defines data access for tender submissions.
type Repository interface {
Create(ctx context.Context, submission *TenderSubmission) error
GetByID(ctx context.Context, id string) (*TenderSubmission, error)
GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*TenderSubmission, error)
Update(ctx context.Context, submission *TenderSubmission) error
UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error
Delete(ctx context.Context, id string) error
ListByCompany(ctx context.Context, companyID string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error)
Search(ctx context.Context, tenderID, companyID *string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error)
GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error)
GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error)
}
type tenderSubmissionRepository struct {
ormRepo mongo.Repository[TenderSubmission]
collection *mongodriver.Collection
logger logger.Logger
}
func NewRepository(mongoManager *mongo.ConnectionManager, logger logger.Logger) Repository {
indexes := []mongo.Index{
*mongo.CreateUniqueIndex("tender_company_idx", bson.D{{Key: "tender_id", Value: 1}, {Key: "company_id", Value: 1}}),
*mongo.NewIndex("tender_id_idx", bson.D{{Key: "tender_id", Value: 1}}),
*mongo.NewIndex("company_id_idx", bson.D{{Key: "company_id", Value: 1}}),
*mongo.NewIndex("status_idx", bson.D{{Key: "status", Value: 1}}),
*mongo.NewIndex("stage_idx", bson.D{{Key: "stage", Value: 1}}),
*mongo.NewIndex("created_at_idx", bson.D{{Key: "created_at", Value: -1}}),
*mongo.NewIndex("updated_at_idx", bson.D{{Key: "updated_at", Value: -1}}),
}
if err := mongoManager.CreateIndexes("tender_submissions", indexes); err != nil {
logger.Warn("Failed to create tender submission indexes", map[string]interface{}{
"error": err.Error(),
})
}
collection := mongoManager.GetCollection("tender_submissions")
ormRepo := mongo.NewRepository[TenderSubmission](collection, logger)
return &tenderSubmissionRepository{ormRepo: ormRepo, collection: collection, logger: logger}
}
func (r *tenderSubmissionRepository) Create(ctx context.Context, submission *TenderSubmission) error {
now := time.Now().Unix()
submission.SetCreatedAt(now)
submission.SetUpdatedAt(now)
if err := r.ormRepo.Create(ctx, submission); err != nil {
if mongo.IsDuplicateKeyError(err) {
return ErrTenderSubmissionExists
}
r.logger.Error("Failed to create tender submission", map[string]interface{}{
"error": err.Error(),
"tender_id": submission.TenderID,
"company_id": submission.CompanyID,
})
return err
}
r.logger.Info("Tender submission created", map[string]interface{}{
"submission_id": submission.GetID(),
"tender_id": submission.TenderID,
"company_id": submission.CompanyID,
"status": submission.Status,
})
return nil
}
func (r *tenderSubmissionRepository) GetByID(ctx context.Context, id string) (*TenderSubmission, error) {
submission, err := r.ormRepo.FindByID(ctx, id)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to get tender submission by ID", map[string]interface{}{
"error": err.Error(),
"submission_id": id,
})
return nil, err
}
return submission, nil
}
func (r *tenderSubmissionRepository) GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*TenderSubmission, error) {
filter := bson.M{"tender_id": tenderID, "company_id": companyID}
submission, err := r.ormRepo.FindOne(ctx, filter)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to get tender submission by tender and company", map[string]interface{}{
"error": err.Error(),
"tender_id": tenderID,
"company_id": companyID,
})
return nil, err
}
return submission, nil
}
func (r *tenderSubmissionRepository) Update(ctx context.Context, submission *TenderSubmission) error {
submission.SetUpdatedAt(time.Now().Unix())
if err := r.ormRepo.Update(ctx, submission); err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to update tender submission", map[string]interface{}{
"error": err.Error(),
"submission_id": submission.GetID(),
})
return err
}
return nil
}
func (r *tenderSubmissionRepository) UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error {
objectID, err := bson.ObjectIDFromHex(submission.GetID())
if err != nil {
return err
}
newUpdatedAt := time.Now().Unix()
submission.SetUpdatedAt(newUpdatedAt)
filter := bson.M{"_id": objectID, "updated_at": expectedUpdatedAt}
result, err := r.collection.UpdateOne(ctx, filter, bson.M{"$set": submission})
if err != nil {
r.logger.Error("Failed to update tender submission with optimistic lock", map[string]interface{}{
"error": err.Error(),
"submission_id": submission.GetID(),
})
return err
}
if result.MatchedCount == 0 {
return ErrConcurrentUpdate
}
return nil
}
func (r *tenderSubmissionRepository) Delete(ctx context.Context, id string) error {
if err := r.ormRepo.Delete(ctx, id); err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to delete tender submission", map[string]interface{}{
"error": err.Error(),
"submission_id": id,
})
return err
}
return nil
}
func (r *tenderSubmissionRepository) ListByCompany(ctx context.Context, companyID string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error) {
companyIDPtr := companyID
return r.Search(ctx, nil, &companyIDPtr, statuses, stages, from, to, limit, offset, sortBy, sortOrder)
}
func (r *tenderSubmissionRepository) Search(ctx context.Context, tenderID, companyID *string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error) {
limit = capListLimit(limit)
if offset < 0 {
offset = 0
}
filter := bson.M{}
if tenderID != nil && *tenderID != "" {
filter["tender_id"] = *tenderID
}
if companyID != nil && *companyID != "" {
filter["company_id"] = *companyID
}
if len(statuses) > 0 {
filter["status"] = bson.M{"$in": statuses}
}
if len(stages) > 0 {
filter["stage"] = bson.M{"$in": stages}
}
if from != nil || to != nil {
dateFilter := bson.M{}
if from != nil {
dateFilter["$gte"] = *from
}
if to != nil {
dateFilter["$lte"] = *to
}
filter["created_at"] = dateFilter
}
if sortBy == "" {
sortBy = "updated_at"
}
if sortOrder == "" {
sortOrder = "desc"
}
pagination := mongo.NewPaginationBuilder().Limit(limit).Skip(offset)
if sortOrder == "asc" {
pagination = pagination.SortAsc(sortBy)
} else {
pagination = pagination.SortDesc(sortBy)
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination.Build())
if err != nil {
r.logger.Error("Failed to search tender submissions", map[string]interface{}{
"error": err.Error(),
})
return nil, 0, err
}
items := make([]*TenderSubmission, len(result.Items))
for i := range result.Items {
items[i] = &result.Items[i]
}
return items, result.TotalCount, nil
}
func (r *tenderSubmissionRepository) GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error) {
pipeline := mongodriver.Pipeline{
{{Key: "$match", Value: bson.M{"company_id": companyID}}},
{{Key: "$facet", Value: bson.M{
"total": bson.A{bson.M{"$count": "count"}},
"by_stage": bson.A{bson.M{"$group": bson.M{"_id": "$stage", "count": bson.M{"$sum": 1}}}},
"by_status": bson.A{bson.M{"$group": bson.M{"_id": "$status", "count": bson.M{"$sum": 1}}}},
}}},
}
results, err := r.ormRepo.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
stats := &CompanySubmissionStatsResponse{
CompanyID: companyID,
ByStage: map[string]int64{},
ByStatus: map[string]int64{},
LastUpdated: time.Now().Unix(),
}
if len(results) == 0 {
return stats, nil
}
facet := results[0]
if totalArr, ok := facet["total"].(bson.A); ok && len(totalArr) > 0 {
if doc, ok := totalArr[0].(bson.M); ok {
stats.Total = int64Value(doc["count"])
}
}
stats.ByStage = countMapFromFacet(facet["by_stage"])
stats.ByStatus = countMapFromFacet(facet["by_status"])
return stats, nil
}
func (r *tenderSubmissionRepository) GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error) {
pipeline := mongodriver.Pipeline{
{{Key: "$facet", Value: bson.M{
"total": bson.A{bson.M{"$count": "count"}},
"by_stage": bson.A{bson.M{"$group": bson.M{"_id": "$stage", "count": bson.M{"$sum": 1}}}},
"by_status": bson.A{bson.M{"$group": bson.M{"_id": "$status", "count": bson.M{"$sum": 1}}}},
}}},
}
results, err := r.ormRepo.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
stats := &AdminSubmissionStatsResponse{
ByStage: map[string]int64{},
ByStatus: map[string]int64{},
LastUpdated: time.Now().Unix(),
}
if len(results) == 0 {
return stats, nil
}
facet := results[0]
if totalArr, ok := facet["total"].(bson.A); ok && len(totalArr) > 0 {
if doc, ok := totalArr[0].(bson.M); ok {
stats.Total = int64Value(doc["count"])
}
}
stats.ByStage = countMapFromFacet(facet["by_stage"])
stats.ByStatus = countMapFromFacet(facet["by_status"])
return stats, nil
}
func countMapFromFacet(raw interface{}) map[string]int64 {
out := map[string]int64{}
arr, ok := raw.(bson.A)
if !ok {
return out
}
for _, item := range arr {
doc, ok := item.(bson.M)
if !ok {
continue
}
key, _ := doc["_id"].(string)
if key == "" {
continue
}
out[key] = int64Value(doc["count"])
}
return out
}
func int64Value(v interface{}) int64 {
switch n := v.(type) {
case int32:
return int64(n)
case int64:
return n
case float64:
return int64(n)
default:
return 0
}
}
func capListLimit(limit int) int {
if limit <= 0 {
return 20
}
if limit > maxListLimit {
return maxListLimit
}
return limit
}
+451
View File
@@ -0,0 +1,451 @@
package tender_submission
import (
"context"
"errors"
"strings"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/pkg/logger"
"tm/pkg/response"
)
// TenderApprovalReader verifies that a tender is approved before workflow actions.
type TenderApprovalReader interface {
GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*tender_approval.TenderApproval, error)
}
// Service defines tender submission business logic.
type Service interface {
EnsureForApprovedTender(ctx context.Context, tenderID, companyID, customerID string) (*TenderSubmissionWithTenderResponse, error)
GetByIDWithTender(ctx context.Context, id string) (*TenderSubmissionWithTenderResponse, error)
GetByIDForCompanyWithTender(ctx context.Context, id, companyID string) (*TenderSubmissionWithTenderResponse, error)
GetByTenderAndCompanyWithTender(ctx context.Context, tenderID, companyID string) (*TenderSubmissionWithTenderResponse, error)
ListByCompanyWithTender(ctx context.Context, companyID string, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error)
UpdateStatus(ctx context.Context, id string, form *UpdateSubmissionStatusForm, companyID, changedBy string) (*TenderSubmissionWithTenderResponse, error)
GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error)
GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error)
AdminListWithTender(ctx context.Context, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error)
OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error
OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error
OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error
}
type tenderSubmissionService struct {
repository Repository
approvalReader TenderApprovalReader
tenderService tender.Service
logger logger.Logger
}
func NewService(repository Repository, approvalReader TenderApprovalReader, tenderService tender.Service, logger logger.Logger) Service {
return &tenderSubmissionService{
repository: repository,
approvalReader: approvalReader,
tenderService: tenderService,
logger: logger,
}
}
func (s *tenderSubmissionService) EnsureForApprovedTender(ctx context.Context, tenderID, companyID, customerID string) (*TenderSubmissionWithTenderResponse, error) {
resolvedTenderID, err := s.tenderService.ResolveMongoID(ctx, tenderID)
if err != nil {
s.logger.Warn("Failed to resolve tender for submission ensure", map[string]interface{}{
"tender_ref": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, ErrTenderNotFound
}
if err := s.requireSubmittedApproval(ctx, resolvedTenderID, companyID); err != nil {
return nil, err
}
submission, err := s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
if !errors.Is(err, ErrTenderSubmissionNotFound) {
return nil, err
}
submission = NewTenderSubmission(resolvedTenderID, companyID, customerID)
if err := s.repository.Create(ctx, submission); err != nil {
if !errors.Is(err, ErrTenderSubmissionExists) {
return nil, err
}
submission, err = s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
return nil, err
}
}
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByIDWithTender(ctx context.Context, id string) (*TenderSubmissionWithTenderResponse, error) {
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByIDForCompanyWithTender(ctx context.Context, id, companyID string) (*TenderSubmissionWithTenderResponse, error) {
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
if submission.CompanyID != companyID {
return nil, ErrTenderSubmissionNotFound
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByTenderAndCompanyWithTender(ctx context.Context, tenderID, companyID string) (*TenderSubmissionWithTenderResponse, error) {
resolvedTenderID, err := s.tenderService.ResolveMongoID(ctx, tenderID)
if err != nil {
return nil, ErrTenderNotFound
}
submission, err := s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
return nil, err
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) ListByCompanyWithTender(ctx context.Context, companyID string, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error) {
limit, offset, sortBy, sortOrder := listParams(form)
statuses := parseStatuses(form.Status)
stages := parseStages(form.Stage)
submissions, total, err := s.repository.ListByCompany(ctx, companyID, statuses, stages, form.From, form.To, limit, offset, sortBy, sortOrder)
if err != nil {
s.logger.Error("Failed to list tender submissions", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
return nil, errors.New("failed to list tender submissions")
}
return &TenderSubmissionListResponse{
Submissions: s.enrichSubmissions(ctx, submissions, false),
Metadata: listMetadata(total, limit, offset),
}, nil
}
func (s *tenderSubmissionService) AdminListWithTender(ctx context.Context, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error) {
limit, offset, sortBy, sortOrder := listParams(form)
statuses := parseStatuses(form.Status)
stages := parseStages(form.Stage)
submissions, total, err := s.repository.Search(ctx, form.TenderID, form.CompanyID, statuses, stages, form.From, form.To, limit, offset, sortBy, sortOrder)
if err != nil {
s.logger.Error("Failed to admin list tender submissions", map[string]interface{}{
"error": err.Error(),
})
return nil, errors.New("failed to list tender submissions")
}
return &TenderSubmissionListResponse{
Submissions: s.enrichSubmissions(ctx, submissions, false),
Metadata: listMetadata(total, limit, offset),
}, nil
}
func (s *tenderSubmissionService) UpdateStatus(ctx context.Context, id string, form *UpdateSubmissionStatusForm, companyID, changedBy string) (*TenderSubmissionWithTenderResponse, error) {
s.logger.Info("Updating tender submission status", map[string]interface{}{
"submission_id": id,
"company_id": companyID,
"new_status": form.Status,
"changed_by": changedBy,
})
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
if companyID != "" && submission.CompanyID != companyID {
return nil, ErrTenderSubmissionNotFound
}
expectedUpdatedAt := submission.UpdatedAt
if err := s.validateStatusTransition(submission.Status, form.Status); err != nil {
s.logger.Error("Invalid tender submission status transition", map[string]interface{}{
"submission_id": id,
"current_status": submission.Status,
"new_status": form.Status,
"error": err.Error(),
})
return nil, err
}
submission.AddStatusChange(form.Status, form.Reason, changedBy, form.Description)
if err := s.repository.UpdateIfUpdatedAt(ctx, submission, expectedUpdatedAt); err != nil {
if errors.Is(err, ErrConcurrentUpdate) {
return nil, ErrConcurrentUpdate
}
return nil, errors.New("failed to update tender submission status")
}
s.logger.Info("Tender submission status updated", map[string]interface{}{
"submission_id": id,
"status": submission.Status,
"stage": submission.Stage,
})
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error) {
stats, err := s.repository.GetCompanyStats(ctx, companyID)
if err != nil {
s.logger.Error("Failed to get company submission stats", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
return nil, errors.New("failed to get submission statistics")
}
return stats, nil
}
func (s *tenderSubmissionService) GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error) {
stats, err := s.repository.GetGlobalStats(ctx)
if err != nil {
s.logger.Error("Failed to get global submission stats", map[string]interface{}{
"error": err.Error(),
})
return nil, errors.New("failed to get submission statistics")
}
return stats, nil
}
func (s *tenderSubmissionService) OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error {
existing, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil && !errors.Is(err, ErrTenderSubmissionNotFound) {
return err
}
if existing != nil {
if IsTerminalStatus(existing.Status) {
existing.ReopenForApproval(customerID, "Tender re-approved")
return s.repository.Update(ctx, existing)
}
return nil
}
submission := NewTenderSubmission(tenderID, companyID, customerID)
if err := s.repository.Create(ctx, submission); err != nil {
if errors.Is(err, ErrTenderSubmissionExists) {
existing, getErr := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if getErr != nil {
return getErr
}
if IsTerminalStatus(existing.Status) {
existing.ReopenForApproval(customerID, "Tender re-approved")
return s.repository.Update(ctx, existing)
}
return nil
}
s.logger.Error("Failed to create submission on approval", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return err
}
s.logger.Info("Submission workflow created from approval", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
})
return nil
}
func (s *tenderSubmissionService) OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error {
submission, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return nil
}
return err
}
if submission.Status != StatusOpportunity {
return nil
}
if err := s.repository.Delete(ctx, submission.GetID()); err != nil {
s.logger.Error("Failed to remove opportunity submission after approval toggle", map[string]interface{}{
"submission_id": submission.GetID(),
"error": err.Error(),
})
return err
}
return nil
}
func (s *tenderSubmissionService) OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error {
submission, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return nil
}
return err
}
if IsTerminalStatus(submission.Status) {
return nil
}
if err := s.transitionSubmission(submission, StatusNotApplied, "approval_rejected", changedBy, "Tender approval was rejected"); err != nil {
var transitionErr *InvalidStatusTransitionError
if errors.Is(err, ErrTerminalStatus) || errors.As(err, &transitionErr) {
s.logger.Warn("Skipping approval rejection sync due to invalid transition", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"current_status": submission.Status,
"target_status": StatusNotApplied,
})
return nil
}
return err
}
return s.repository.Update(ctx, submission)
}
func (s *tenderSubmissionService) transitionSubmission(submission *TenderSubmission, next SubmissionStatus, reason, changedBy, description string) error {
if err := s.validateStatusTransition(submission.Status, next); err != nil {
return err
}
submission.AddStatusChange(next, reason, changedBy, description)
return nil
}
func (s *tenderSubmissionService) requireSubmittedApproval(ctx context.Context, tenderID, companyID string) error {
approval, err := s.approvalReader.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, tender_approval.ErrTenderApprovalNotFound) {
return ErrApprovalRequired
}
return err
}
if approval.Status != tender_approval.ApprovalStatusSubmitted {
return ErrApprovalRequired
}
return nil
}
func (s *tenderSubmissionService) validateStatusTransition(current, next SubmissionStatus) error {
if IsTerminalStatus(current) {
return ErrTerminalStatus
}
for _, allowed := range AllowedNextStatuses(current) {
if allowed == next {
return nil
}
}
allowed := AllowedNextStatuses(current)
allowedStrings := make([]string, len(allowed))
for i, status := range allowed {
allowedStrings[i] = string(status)
}
return NewInvalidStatusTransitionError(string(current), string(next), allowedStrings)
}
func (s *tenderSubmissionService) responseWithTender(ctx context.Context, submission *TenderSubmission, fullDetails bool) (*TenderSubmissionWithTenderResponse, error) {
tenderResp, err := s.tenderService.GetByID(ctx, submission.TenderID, "")
if err != nil {
s.logger.Warn("Failed to load tender for submission response", map[string]interface{}{
"tender_id": submission.TenderID,
"error": err.Error(),
})
return submission.ToResponseWithTender(nil), nil
}
var details *TenderDetails
if fullDetails {
details = fullTenderDetailsFromResponse(tenderResp)
} else {
details = listTenderDetailsFromResponse(tenderResp)
}
return submission.ToResponseWithTender(details), nil
}
func listParams(form *ListTenderSubmissionsForm) (limit, offset int, sortBy, sortOrder string) {
limit = 20
offset = 0
sortBy = "updated_at"
sortOrder = "desc"
if form == nil {
return limit, offset, sortBy, sortOrder
}
if form.Limit != nil && *form.Limit > 0 {
limit = *form.Limit
}
if limit > maxListLimit {
limit = maxListLimit
}
if form.Offset != nil && *form.Offset >= 0 {
offset = *form.Offset
}
if form.SortBy != nil && strings.TrimSpace(*form.SortBy) != "" {
sortBy = strings.TrimSpace(*form.SortBy)
}
if form.SortOrder != nil && strings.TrimSpace(*form.SortOrder) != "" {
sortOrder = strings.TrimSpace(*form.SortOrder)
}
return limit, offset, sortBy, sortOrder
}
func parseStatuses(values []string) []SubmissionStatus {
if len(values) == 0 {
return nil
}
out := make([]SubmissionStatus, 0, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value == "" {
continue
}
out = append(out, SubmissionStatus(value))
}
return out
}
func parseStages(values []string) []SubmissionStage {
if len(values) == 0 {
return nil
}
out := make([]SubmissionStage, 0, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value == "" {
continue
}
out = append(out, SubmissionStage(value))
}
return out
}
func listMetadata(total int64, limit, offset int) *response.Meta {
pages := int(total) / limit
if limit > 0 && int(total)%limit > 0 {
pages++
}
page := 1
if limit > 0 {
page = (offset / limit) + 1
}
return &response.Meta{
Total: int(total),
Limit: limit,
Offset: offset,
Page: page,
Pages: pages,
}
}
+261
View File
@@ -0,0 +1,261 @@
package tender_submission
import (
"context"
"errors"
"testing"
"tm/pkg/logger"
)
type mockRepository struct {
byTenderCompany map[string]*TenderSubmission
createErr error
updateErr error
deleteErr error
}
func submissionKey(tenderID, companyID string) string {
return tenderID + ":" + companyID
}
func newMockRepository() *mockRepository {
return &mockRepository{byTenderCompany: map[string]*TenderSubmission{}}
}
func (m *mockRepository) Create(_ context.Context, submission *TenderSubmission) error {
if m.createErr != nil {
return m.createErr
}
key := submissionKey(submission.TenderID, submission.CompanyID)
if _, exists := m.byTenderCompany[key]; exists {
return ErrTenderSubmissionExists
}
copy := *submission
m.byTenderCompany[key] = &copy
return nil
}
func (m *mockRepository) GetByID(_ context.Context, id string) (*TenderSubmission, error) {
for _, submission := range m.byTenderCompany {
if submission.GetID() == id {
copy := *submission
return &copy, nil
}
}
return nil, ErrTenderSubmissionNotFound
}
func (m *mockRepository) GetByTenderAndCompany(_ context.Context, tenderID, companyID string) (*TenderSubmission, error) {
submission, ok := m.byTenderCompany[submissionKey(tenderID, companyID)]
if !ok {
return nil, ErrTenderSubmissionNotFound
}
copy := *submission
return &copy, nil
}
func (m *mockRepository) Update(_ context.Context, submission *TenderSubmission) error {
if m.updateErr != nil {
return m.updateErr
}
key := submissionKey(submission.TenderID, submission.CompanyID)
if _, ok := m.byTenderCompany[key]; !ok {
return ErrTenderSubmissionNotFound
}
copy := *submission
m.byTenderCompany[key] = &copy
return nil
}
func (m *mockRepository) UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error {
current, err := m.GetByTenderAndCompany(ctx, submission.TenderID, submission.CompanyID)
if err != nil {
return err
}
if current.UpdatedAt != expectedUpdatedAt {
return ErrConcurrentUpdate
}
return m.Update(ctx, submission)
}
func (m *mockRepository) Delete(_ context.Context, id string) error {
if m.deleteErr != nil {
return m.deleteErr
}
for key, submission := range m.byTenderCompany {
if submission.GetID() == id {
delete(m.byTenderCompany, key)
return nil
}
}
return ErrTenderSubmissionNotFound
}
func (m *mockRepository) ListByCompany(context.Context, string, []SubmissionStatus, []SubmissionStage, *int64, *int64, int, int, string, string) ([]*TenderSubmission, int64, error) {
return nil, 0, nil
}
func (m *mockRepository) Search(context.Context, *string, *string, []SubmissionStatus, []SubmissionStage, *int64, *int64, int, int, string, string) ([]*TenderSubmission, int64, error) {
return nil, 0, nil
}
func (m *mockRepository) GetCompanyStats(context.Context, string) (*CompanySubmissionStatsResponse, error) {
return nil, nil
}
func (m *mockRepository) GetGlobalStats(context.Context) (*AdminSubmissionStatsResponse, error) {
return nil, nil
}
func (m *mockRepository) seed(submission *TenderSubmission) {
copy := *submission
m.byTenderCompany[submissionKey(submission.TenderID, submission.CompanyID)] = &copy
}
func newTestService(repo Repository) *tenderSubmissionService {
return &tenderSubmissionService{
repository: repo,
approvalReader: nil,
tenderService: nil,
logger: testLogger{},
}
}
type testLogger struct{}
func (testLogger) Debug(string, map[string]interface{}) {}
func (testLogger) Info(string, map[string]interface{}) {}
func (testLogger) Warn(string, map[string]interface{}) {}
func (testLogger) Error(string, map[string]interface{}) {}
func (testLogger) Fatal(string, map[string]interface{}) {}
func (testLogger) WithFields(map[string]interface{}) logger.Logger {
return testLogger{}
}
func (testLogger) Sync() error { return nil }
func TestOnApprovalSubmittedReopensTerminalSubmission(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusNotApplied, "approval_rejected", "customer-1", "Rejected")
repo.seed(existing)
if err := service.OnApprovalSubmitted(context.Background(), "tender-1", "company-1", "customer-2"); err != nil {
t.Fatalf("OnApprovalSubmitted() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusOpportunity {
t.Fatalf("status = %q, want %q", updated.Status, StatusOpportunity)
}
}
func TestOnApprovalSubmittedCreatesWhenMissing(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
if err := service.OnApprovalSubmitted(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalSubmitted() error = %v", err)
}
created, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if created.Status != StatusOpportunity {
t.Fatalf("status = %q, want %q", created.Status, StatusOpportunity)
}
}
func TestOnApprovalRejectedUsesValidatedTransition(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
repo.seed(existing)
if err := service.OnApprovalRejected(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalRejected() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusNotApplied {
t.Fatalf("status = %q, want %q", updated.Status, StatusNotApplied)
}
}
func TestOnApprovalRejectedSkipsInvalidTransitionFromSentWaiting(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
existing.AddStatusChange(StatusApplying, "", "customer-1", "Ready")
existing.AddStatusChange(StatusSentWaiting, "", "customer-1", "Submitted")
repo.seed(existing)
if err := service.OnApprovalRejected(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalRejected() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusSentWaiting {
t.Fatalf("status = %q, want %q", updated.Status, StatusSentWaiting)
}
}
func TestOnApprovalRemovedDeletesOpportunityOnly(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
opportunity := NewTenderSubmission("tender-1", "company-1", "customer-1")
repo.seed(opportunity)
if err := service.OnApprovalRemoved(context.Background(), "tender-1", "company-1"); err != nil {
t.Fatalf("OnApprovalRemoved() error = %v", err)
}
if _, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1"); !errors.Is(err, ErrTenderSubmissionNotFound) {
t.Fatalf("expected submission to be deleted, got err = %v", err)
}
inProgress := NewTenderSubmission("tender-2", "company-1", "customer-1")
inProgress.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
repo.seed(inProgress)
if err := service.OnApprovalRemoved(context.Background(), "tender-2", "company-1"); err != nil {
t.Fatalf("OnApprovalRemoved() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-2", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusValidating {
t.Fatalf("status = %q, want %q", updated.Status, StatusValidating)
}
}
func TestGetByIDForCompanyWithTenderEnforcesOwnership(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
submission := NewTenderSubmission("tender-1", "company-1", "customer-1")
repo.seed(submission)
_, err := service.GetByIDForCompanyWithTender(context.Background(), submission.GetID(), "company-2")
if !errors.Is(err, ErrTenderSubmissionNotFound) {
t.Fatalf("expected not found for other company, got %v", err)
}
}
@@ -0,0 +1,110 @@
package tender_submission
import (
"context"
"tm/internal/tender"
)
func listTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
Title: resp.Title,
Description: resp.Description,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func fullTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
NoticePublicationID: resp.NoticePublicationID,
PublicationDate: resp.PublicationDate,
Title: resp.Title,
Description: resp.Description,
ProcurementTypeCode: resp.ProcurementTypeCode,
ProcedureCode: resp.ProcedureCode,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
Duration: resp.Duration,
DurationUnit: resp.DurationUnit,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func uniqueTenderIDs(submissions []*TenderSubmission) []string {
if len(submissions) == 0 {
return nil
}
seen := make(map[string]struct{}, len(submissions))
ids := make([]string, 0, len(submissions))
for _, submission := range submissions {
if submission == nil || submission.TenderID == "" {
continue
}
if _, ok := seen[submission.TenderID]; ok {
continue
}
seen[submission.TenderID] = struct{}{}
ids = append(ids, submission.TenderID)
}
return ids
}
func (s *tenderSubmissionService) loadTendersByIDs(ctx context.Context, tenderIDs []string) map[string]*tender.TenderResponse {
if len(tenderIDs) == 0 {
return map[string]*tender.TenderResponse{}
}
tenders, err := s.tenderService.GetByIDs(ctx, tenderIDs)
if err != nil {
s.logger.Error("Failed to batch load tenders for submissions", map[string]interface{}{
"count": len(tenderIDs),
"error": err.Error(),
})
return map[string]*tender.TenderResponse{}
}
return tenders
}
func (s *tenderSubmissionService) enrichSubmissions(ctx context.Context, submissions []*TenderSubmission, fullDetails bool) []*TenderSubmissionWithTenderResponse {
tenderMap := s.loadTendersByIDs(ctx, uniqueTenderIDs(submissions))
responses := make([]*TenderSubmissionWithTenderResponse, len(submissions))
for i, submission := range submissions {
var details *TenderDetails
if tenderResp, ok := tenderMap[submission.TenderID]; ok {
if fullDetails {
details = fullTenderDetailsFromResponse(tenderResp)
} else {
details = listTenderDetailsFromResponse(tenderResp)
}
}
responses[i] = submission.ToResponseWithTender(details)
}
return responses
}
+22
View File
@@ -389,3 +389,25 @@ func (t *TenderJSON) translationForLanguage(language string) (StoredTranslation,
}
return t.storedTranslationText(language)
}
// hasAnyTranslation reports whether tender.json contains at least one non-empty
// title translation (via translation_status.done or the translations map).
func (t *TenderJSON) hasAnyTranslation() bool {
if t == nil {
return false
}
for _, lang := range t.translationsDone() {
if _, ok := t.storedTranslationText(lang); ok {
return true
}
}
if t.Translations == nil {
return false
}
for _, entry := range t.Translations {
if strings.TrimSpace(entry.Title) != "" {
return true
}
}
return false
}
+83
View File
@@ -654,6 +654,89 @@ func (s *StorageClient) ScanScrapedDocuments(ctx context.Context) ([]ProcedureDo
return results, dailyCounts, nil
}
// ScanTranslatedNotices scans MinIO for tender.json objects with stored translations
// and returns daily notice counts keyed by UTC date (YYYY-MM-DD) plus the total.
func (s *StorageClient) ScanTranslatedNotices(ctx context.Context) (map[string]int64, int64, error) {
s.logger.Debug("Scanning translated notices from MinIO", map[string]interface{}{
"bucket": s.config.MinioBucket,
"prefix": procedurePrefix,
})
dailyCounts := make(map[string]int64)
var total int64
seen := make(map[string]struct{})
var fatalErr error
appendFromPrefix := func(prefix string) bool {
objectCh := s.client.ListObjects(ctx, s.config.MinioBucket, minio.ListObjectsOptions{
Prefix: prefix,
Recursive: true,
})
for object := range objectCh {
if object.Err != nil {
fatalErr = s.logMinIOFailure("list_translated_notices", object.Err, map[string]interface{}{
"prefix": prefix,
})
return false
}
key := object.Key
if !strings.HasSuffix(key, "/tender.json") {
continue
}
if _, dup := seen[key]; dup {
continue
}
seen[key] = struct{}{}
tenderJSON, err := s.readTenderJSONAtKey(ctx, key)
if err != nil {
if errors.Is(err, ErrObjectNotFound) {
continue
}
if errors.Is(err, ErrMinIOUnavailable) {
fatalErr = err
return false
}
s.logger.Debug("Skipping tender.json during translation scan", map[string]interface{}{
"object_key": key,
"error": err.Error(),
})
continue
}
if !tenderJSON.hasAnyTranslation() {
continue
}
total++
modified := normalizeUnixSeconds(object.LastModified.Unix())
if modified > 0 {
date := time.Unix(modified, 0).UTC().Format("2006-01-02")
dailyCounts[date]++
}
}
return true
}
if !appendFromPrefix(procedurePrefix) {
return nil, 0, fatalErr
}
if len(seen) == 0 {
if !appendFromPrefix("") {
return nil, 0, fatalErr
}
}
if fatalErr != nil {
return nil, 0, fatalErr
}
s.logger.Info("Scanned translated notices from storage", map[string]interface{}{
"bucket": s.config.MinioBucket,
"total_count": total,
})
return dailyCounts, total, nil
}
func normalizeUnixSeconds(ts int64) int64 {
if ts > 1_000_000_000_000 {
return ts / 1000
+8 -5
View File
@@ -64,12 +64,15 @@ func NewPagination(c echo.Context) (*Pagination, error) {
sortOrder = "desc"
}
includeTotal := c.QueryParam("include_total") == "true"
return &Pagination{
Limit: limit,
Offset: offset,
Cursor: cursor,
SortBy: sortBy,
SortOrder: sortOrder,
Limit: limit,
Offset: offset,
Cursor: cursor,
SortBy: sortBy,
SortOrder: sortOrder,
IncludeTotal: includeTotal,
}, nil
}
+16 -1
View File
@@ -18,11 +18,26 @@ func TestNewPaginationDefaults(t *testing.T) {
if err != nil {
t.Fatalf("NewPagination: %v", err)
}
if p.Limit != 10 || p.Offset != 0 || p.Cursor != "" {
if p.Limit != 10 || p.Offset != 0 || p.Cursor != "" || p.IncludeTotal {
t.Fatalf("unexpected pagination: %+v", p)
}
}
func TestNewPaginationIncludeTotal(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/?include_total=true", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
p, err := NewPagination(c)
if err != nil {
t.Fatalf("NewPagination: %v", err)
}
if !p.IncludeTotal {
t.Fatal("expected include_total=true")
}
}
func TestNewPaginationCursorConflict(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/?limit=10&offset=5&cursor=abc", nil)
+6 -5
View File
@@ -38,11 +38,12 @@ type Meta struct {
}
type Pagination struct {
Limit int `query:"limit" valid:"optional,range(1|100)" default:"10"`
Offset int `query:"offset" valid:"optional,range(0|1000000)" default:"0"`
Cursor string `query:"cursor" valid:"optional"`
SortBy string `query:"sort_by" valid:"optional"`
SortOrder string `query:"sort_order" valid:"optional,in(asc|desc)"`
Limit int `query:"limit" valid:"optional,range(1|100)" default:"10"`
Offset int `query:"offset" valid:"optional,range(0|1000000)" default:"0"`
Cursor string `query:"cursor" valid:"optional"`
SortBy string `query:"sort_by" valid:"optional"`
SortOrder string `query:"sort_order" valid:"optional,in(asc|desc)"`
IncludeTotal bool `query:"include_total" valid:"optional"`
}
// Success returns a successful response