Compare commits

...

47 Commits

Author SHA1 Message Date
m.nazemi fa16d4080a Merge pull request 'Add unit tests for backfilling notice location from buyer' (#53) from TM-485 into develop
continuous-integration/drone/push Build is passing
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/53
Reviewed-by: Hadi Barzegar <barzagarhadi@gmail.com>
2026-07-13 14:20:08 +03:30
Mazyar e4f7c4a04c Add unit tests for backfilling notice location from buyer
- Introduced `mapper_test.go` to validate the functionality of backfilling notice location details from the buyer organization.
- Implemented two test cases: one to ensure the location fields are correctly populated from the buyer when they are empty, and another to verify that existing values are not overwritten.
- Enhanced the `mapper.go` file by adding the `backfillNoticeLocationFromBuyer` function to facilitate this logic.

This update improves the reliability of notice data handling within the tender management system by ensuring accurate location information is maintained.
2026-07-13 13:44:31 +03:30
Mazyar 51a1a6aa82 Refactor AI pipeline terminology for consistency
continuous-integration/drone/push Build is passing
- Updated comments and logging messages in the worker and related files to replace "daily-run" with "auto run" for clarity and consistency.
- Adjusted the `WorkerConfig` struct to reflect the new terminology in configuration settings.
- Renamed functions and test cases to align with the updated terminology, enhancing code readability and maintainability.

This change improves the clarity of the AI pipeline's functionality within the tender management system.
2026-07-13 11:54:58 +03:30
m.nazemi 6ee81c3581 Merge pull request 'Add documents scraped filter functionality and related tests' (#51) from documents-scraped into develop
continuous-integration/drone/push Build is passing
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/51
2026-07-12 22:03:13 +03:30
Mazyar d55b2685ca Merge branch 'documents-scraped' of https://repo.ravanertebat.com/TM/tm_back into documents-scraped 2026-07-12 22:02:43 +03:30
Mazyar 04c4102170 Enhance documents scraped filter with cancellation handling and caching improvements
- Updated `documents_scraped_filter.go` to extend the caching mechanism for contract folder IDs, increasing the cache TTL to 15 minutes to prevent premature invalidation during slow scans.
- Modified the `cachedContractFolderIDsWithDocuments` method to detach from the caller's cancellation, ensuring that shared scans are not aborted by client disconnects or timeouts.
- Introduced a new test, `TestCachedContractFolderIDsSurvivesCallerCancellation`, in `documents_scraped_filter_test.go` to validate the behavior of the caching mechanism under cancellation scenarios.
- Enhanced the `scanStartedProcedureLister` to manage scan initiation and cancellation more effectively, improving the robustness of the document retrieval process.

This update improves the reliability and performance of the tender management system's document filtering capabilities, particularly in scenarios with long-running scans.
2026-07-12 21:54:21 +03:30
Mazyar 0fde5772e2 Add documents scraped filter functionality and related tests
- Introduced `documents_scraped_filter.go` to handle filtering tenders based on the presence of documents in MinIO, utilizing a caching mechanism to optimize performance.
- Implemented `enrichDocumentsScrapedFilter` and `cachedContractFolderIDsWithDocuments` methods in the `tenderService` for efficient retrieval of contract folder IDs with documents.
- Added unit tests in `documents_scraped_filter_test.go` to validate the new filtering logic and caching behavior, ensuring accurate results and error handling.
- Updated `SearchForm` documentation to clarify the caching aspect of `ContractFolderIDsWithDocuments`.
- Enhanced comments in the handler and repository to reflect the new filtering logic and its implications.

This update improves the tender search functionality by efficiently managing document presence checks, leading to better performance and user experience in the tender management system.
2026-07-12 21:54:21 +03:30
m.nazemi c35fa331f9 Merge branch 'develop' into documents-scraped 2026-07-12 21:46:03 +03:30
m.nazemi c8616940ff Merge pull request 'Add tender submission functionality and related routes' (#52) from tender-submission into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/52
2026-07-12 21:45:23 +03:30
Mazyar fa3d466579 Enhance tender approval and submission handling
- Added a new error for submission sync failures, improving error handling in the tender approval process.
- Updated the `ToggleTenderApproval` method to handle the new error, providing clearer responses for sync issues.
- Introduced a method to reopen terminal submissions when tender approvals are resubmitted, enhancing submission state management.
- Implemented optimistic locking in the repository to prevent concurrent updates, improving data integrity.
- Added unit tests for new submission handling logic, ensuring robust functionality and error management.

This update strengthens the tender approval workflow and submission handling, ensuring better error reporting and state management.
2026-07-12 21:41:53 +03:30
Mazyar 0cce9ef1b5 Add tender submission functionality and related routes
- Introduced the `tender_submission` domain, including entity, repository, service, and handler implementations for managing tender submissions.
- Added new routes for both admin and public access to tender submissions, allowing for listing, ensuring, and retrieving submissions by ID and tender.
- Enhanced the tender approval service to synchronize submission workflows with approval changes, ensuring proper state management.
- Implemented validation and response structures for tender submission operations, improving API consistency and usability.
- Added unit tests for tender submission status transitions and workflow logic, ensuring robust functionality.

This update enhances the tender management system by providing comprehensive support for tender submissions, improving overall workflow and user experience.
2026-07-12 14:54:42 +03:30
Mazyar 5c93e0f01b Add documents scraped filter functionality and related tests
- Introduced `documents_scraped_filter.go` to handle filtering tenders based on the presence of documents in MinIO, utilizing a caching mechanism to optimize performance.
- Implemented `enrichDocumentsScrapedFilter` and `cachedContractFolderIDsWithDocuments` methods in the `tenderService` for efficient retrieval of contract folder IDs with documents.
- Added unit tests in `documents_scraped_filter_test.go` to validate the new filtering logic and caching behavior, ensuring accurate results and error handling.
- Updated `SearchForm` documentation to clarify the caching aspect of `ContractFolderIDsWithDocuments`.
- Enhanced comments in the handler and repository to reflect the new filtering logic and its implications.

This update improves the tender search functionality by efficiently managing document presence checks, leading to better performance and user experience in the tender management system.
2026-07-12 00:20:39 +03:30
m.nazemi 10385e997b Merge pull request 'Remove ContentXML field from Tender entity and related repository method' (#50) from content-xml into develop
continuous-integration/drone/push Build is passing
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/50
Reviewed-by: Hadi Barzegar <barzagarhadi@gmail.com>
2026-07-11 21:47:55 +03:30
m.nazemi adfd291245 Merge branch 'develop' into content-xml 2026-07-11 21:47:44 +03:30
m.nazemi 26a593306d Merge pull request 'Enhance dashboard service with widget caching and improved query handling' (#49) from panel-widget into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/49
2026-07-11 21:47:14 +03:30
Mazyar c5e62a4061 Implement deadline matching logic for dashboard repository and enhance unit tests
- Added the `deadlineWithinWindowMatch` function to match raw deadline fields stored as Unix seconds or milliseconds, ensuring accurate filtering in MongoDB queries.
- Updated the `ClosingSoon` method in the repository to utilize the new deadline matching logic, improving query accuracy for upcoming deadlines.
- Introduced a new unit test, `TestDeadlineWithinWindowMatchCoversSecondsAndMilliseconds`, to validate the functionality of the deadline matching logic, ensuring both seconds and milliseconds are correctly handled.
- Enhanced comments in the `widget_cache` and `search_list_cache` files for better clarity on cache behavior and staleness.

This update improves the accuracy of deadline handling in the dashboard service and strengthens the test coverage for related functionalities.
2026-07-11 21:44:22 +03:30
Mazyar 81b0d94ba3 Enhance dashboard service with widget caching and improved query handling
- Introduced a new `widgetCache` structure to manage caching for various dashboard widgets, improving performance and reducing redundant data retrieval.
- Updated the `ClosingSoon` and `Trend` methods in the service to utilize the new caching mechanism, ensuring efficient data access and reducing load on the database.
- Enhanced the `NoticeTypes` and `Countries` methods to implement caching, further optimizing the dashboard's responsiveness.
- Added a new `searchListCache` implementation in the tender service to cache search results, improving the efficiency of tender searches and reducing database load.
- Implemented robust error handling and logging for cache operations, ensuring better visibility into potential issues.

This update significantly enhances the performance and scalability of the dashboard and tender services by integrating effective caching strategies, leading to improved user experience and resource management.
2026-07-11 21:44:22 +03:30
m.nazemi 3e2700bc36 Merge pull request 'Refactor notification service to support multiple delivery channels' (#48) from notif-channel into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/48
2026-07-11 21:12:11 +03:30
Mazyar aafae2a26f Enhance notification entity and service for improved delivery channel handling
- Introduced new event types for notifications, including email and push, to support a broader range of delivery methods.
- Added a `persistedEventType` function to map delivery channels to their corresponding event types, improving clarity and maintainability.
- Updated the `sendToRecipient` method to ensure in-app notifications are always persisted, enhancing the reliability of notification records.
- Improved error logging for in-app notification persistence failures, providing better context for debugging.

This update enhances the notification system's flexibility and reliability by supporting multiple delivery channels and improving error handling.
2026-07-11 21:07:37 +03:30
Mazyar 9d8ce12816 Refactor notification service to support multiple delivery channels
- Updated the `sendToRecipient` method to iterate over the channels specified in the notification request, allowing notifications to be persisted for each channel.
- Modified the `persistNotification` method to accept a channel parameter, enhancing the logic for handling different delivery methods (email, push, in-app).
- Improved error logging to include the channel information when persisting notifications fails, providing better context for debugging.

This update enhances the flexibility of the notification service by enabling it to handle multiple delivery channels more effectively, improving overall notification management.
2026-07-11 21:07:37 +03:30
Mazyar b388af3518 Remove ContentXML field from Tender entity and related repository method
- Eliminated the `ContentXML` field from the `Tender` struct in the entity definition, streamlining the data model.
- Removed the `FindTendersWithContentXML` method from the `TenderRepository` interface, as it is no longer necessary.
- Updated the `ToTender` method in the `NoticeWorker` to reflect these changes, ensuring consistency across the codebase.

This update simplifies the tender data structure and repository interface, improving maintainability and clarity in the tender management system.
2026-07-11 20:54:29 +03:30
Mazyar 932b0cf24e Add index for documents scraped with keyset pagination in tender repository
- Introduced a new index `documents_scraped_created_at_id_idx` in the tender repository to support keyset pagination for documents marked as scraped, sorted by creation date.
- This enhancement improves the efficiency of querying scraped documents, facilitating better performance in data retrieval and management.

This update significantly optimizes the tender repository's handling of scraped documents, enhancing overall data access capabilities.
2026-07-11 17:00:48 +03:30
Mazyar 1df44ec8ca Add translated notices scanning functionality to dashboard repository
continuous-integration/drone/push Build is passing
- Introduced a new `translatedNoticesScanner` interface for scanning MinIO for daily translated notice counts.
- Enhanced the `repository` struct to include fields for managing translated notice scope and caching.
- Implemented the `ScanTranslatedNotices` method in the `StorageClient` to retrieve daily counts and total from MinIO.
- Updated the `Statistics` method in the repository to utilize the new translated notices scope, improving data accuracy.
- Added unit tests for the translated notices functionality, ensuring robust validation of the new features.

This update significantly enhances the dashboard's ability to handle translated notice statistics, improving overall data management and reporting capabilities.
2026-07-10 17:05:38 +03:30
Mazyar 843e1508df Enhance tender search functionality with pagination and filtering improvements
continuous-integration/drone/push Build is passing
- Updated the tender search handler to include a new `include_total` query parameter, allowing clients to request total count and pagination metadata for search results.
- Refactored the `tenderSearchListProjection` to exclude heavy fields from the list response, optimizing data retrieval.
- Modified the `buildSearchFilter` method to utilize the `processing_metadata.documents_scraped` flag for filtering, improving search accuracy.
- Added unit tests for the new pagination features and search filter logic, ensuring robust validation of the search functionality.

This update significantly enhances the tender search experience by providing more flexible pagination options and improving the efficiency of data handling.
2026-07-10 16:23:46 +03:30
Mazyar 2fbf329182 Refactor tender approval logic to streamline submission mode handling
continuous-integration/drone/push Build is passing
- Removed the resetting of `SubmissionMode` in the `Reject` method of the `TenderApproval` entity, ensuring that the submission mode is preserved upon rejection.
- Updated the `GetCompanyTenderApprovalStats` method in the repository to count submissions by mode without filtering by status, enhancing the accuracy of statistics.
- Adjusted the `ToggleTenderApproval` method in the service to set the `SubmissionMode` when rejecting a tender, ensuring consistent handling of submission modes across approval states.

This update improves the clarity and functionality of tender approval processes, ensuring that submission modes are correctly managed during approval and rejection scenarios.
2026-07-08 03:02:57 +03:30
Mazyar 81a94b4879 Enhance tender and company context handling with new features and error management
continuous-integration/drone/push Build is passing
- Introduced `PickAssignedCompanyID` function to determine the appropriate company ID for company-scoped operations, improving company assignment logic.
- Added `ResolveMongoID` method in the tender service to map MongoDB IDs and AI procedure references to canonical tender IDs, enhancing ID resolution.
- Updated `ToggleTenderApproval` to handle company ID validation and improve error handling, ensuring robust processing of tender approvals.
- Enhanced logging for error scenarios in the tender approval service, providing better insights into failures during operations.
- Refactored the `GetByID` and `GetByTenderAndCompany` methods in the tender approval repository to utilize custom error types for improved clarity.

This update significantly improves the handling of company and tender operations, enhancing error management and overall service reliability.
2026-07-08 02:55:55 +03:30
Mazyar 11e0b44ebe Add cache staleness check for recommendation page
continuous-integration/drone/push Build is passing
- Introduced a new method `isRecommendedPageCacheStale` in the `tenderService` to determine if the cached recommendations for a company are outdated based on the count of cached recommendations and the latest AI recommendations.
- Updated the `recommendFromPageCache` method to utilize the new cache staleness check, ensuring that outdated caches are not used for recommendations.
- This enhancement improves the accuracy of recommendations by ensuring that users receive the most up-to-date information, thereby enhancing the overall user experience in the tender recommendation service.
2026-07-08 02:41:00 +03:30
Mazyar f68b6d7787 Refactor recommendation response handling and enhance enrichment logic
continuous-integration/drone/push Build is passing
- Removed the `buildRecommendedTenderResponsesParallel` method and replaced it with `buildRecommendedTenderListResponses`, simplifying the response building process.
- Introduced `enrichRecommendedTenderResponsesParallel` to handle enrichment of tender responses in parallel, improving performance.
- Updated the `Recommend` method to utilize the new enrichment logic, ensuring timely and efficient processing of recommendations.
- Cleaned up unused functions and imports in `recommendation_page_cache.go` and `recommendation_response.go`, enhancing code clarity and maintainability.

This update streamlines the recommendation response handling and improves the overall efficiency of the tender recommendation service.
2026-07-08 02:27:25 +03:30
Mazyar 784c3d6563 Refactor recommendation page cache handling and configuration
continuous-integration/drone/push Build is passing
- Removed the `RecommendationPageCacheLanguages` configuration from `AISummarizerConfig` to streamline cache management.
- Updated the `companyService` and `tenderService` to utilize the new `InvalidateRecommendedTendersPageCache` method for cache invalidation, enhancing clarity and efficiency.
- Refactored the `invalidateRecommendedTendersPageCache` method to eliminate unnecessary context parameters, simplifying the function signature.
- Improved the handling of page cache refresh logic by consolidating language handling within the `tenderService`, ensuring consistent behavior across services.
- Cleaned up related tests and removed deprecated functions to maintain code quality and readability.

This update enhances the maintainability of the recommendation caching system by simplifying configuration and improving cache invalidation logic.
2026-07-08 02:20:16 +03:30
Mazyar 0b74e9ad23 Enhance tender recommendation caching and page refresh logic
continuous-integration/drone/push Build is passing
- Introduced a new `RecommendedTendersPageCacheRefresher` interface to manage the asynchronous refresh of recommendation pages in Redis, improving cache management.
- Updated the `companyService` to support setting page cache languages and refreshing the recommended tenders page cache based on company IDs.
- Enhanced the `tenderService` to build and invalidate recommended tenders page caches, ensuring timely updates and efficient retrieval of cached recommendations.
- Added configuration options for recommendation page cache languages in the `AISummarizerConfig`, allowing for flexible language support.
- Implemented unit tests for the new caching logic and page refresh functionality, ensuring robust validation of the recommendation caching process.

This update significantly improves the efficiency and responsiveness of the tender recommendation service by integrating enhanced caching mechanisms and page refresh capabilities.
2026-07-08 02:05:36 +03:30
Mazyar b28bc23975 Enhance tender reference handling with improved deduplication and test coverage
- Updated the `markResolvedRecommendedTenderSeen` function to accept an additional `tenderRef` parameter, allowing for better tracking of tender references and preventing duplicates.
- Refactored the `GetByProcedureReferences` method in the repository to combine indexed notice lookups with contract-folder candidates, ensuring older notice references resolve correctly after tender merges.
- Introduced a new `findTendersByContractFolderIDs` method to streamline the retrieval of tenders by contract folder IDs, enhancing performance and maintainability.
- Added unit tests for new functionality in `ai_reference_test.go` and `recommendation_filter_test.go`, ensuring robust validation of tender reference mapping and deduplication logic.

This update significantly improves the handling of tender references and enhances the overall test coverage, ensuring more reliable and efficient processing of tender data.
2026-07-08 01:55:33 +03:30
Mazyar f108733c2a Enhance tender recommendation service with improved caching and pagination
continuous-integration/drone/push Build is passing
- Introduced a context with a timeout for Redis cache retrieval in the `getCachedAIRecommendations` method, ensuring more robust handling of potential delays.
- Added a new file `recommendation_page.go` to encapsulate the logic for building recommendation pages, improving code organization and readability.
- Implemented a `buildRecommendationPage` method to handle pagination and batch processing of recommendations, enhancing performance and user experience.
- Created a new test file `recommendation_page_test.go` to validate the functionality of the recommendation page building process, ensuring correct pagination and batch resolution.

This update significantly improves the efficiency and maintainability of the tender recommendation service by optimizing caching and implementing structured pagination handling.
2026-07-08 01:32:27 +03:30
Mazyar 2980a3beb4 Refactor tender recommendation response handling and improve test coverage
continuous-integration/drone/push Build is passing
- Renamed the test function to `TestBuildRecommendedTenderListResponsesPreservesOrderWithoutStorage` for clarity and updated its logic to reflect changes in the recommendation response building process.
- Removed unnecessary sleep in the test to enhance performance.
- Refactored the `buildRecommendedTenderResponsesParallel` method to `buildRecommendedTenderListResponses`, simplifying the response building without relying on MinIO translation lookups.
- Updated the repository's `mapProcedureReferencesToTenders` function to improve efficiency by grouping candidates by their contract folder IDs, enhancing the mapping process.

This update streamlines the recommendation response handling and enhances the test suite for better maintainability and performance.
2026-07-08 01:23:21 +03:30
Mazyar adaae4c4bc Refactor tender approval service to optimize tender detail retrieval
continuous-integration/drone/push Build is passing
- Replaced the direct fetching of tender details within the `GetTenderApprovalsByCompanyIDWithTender`, `GetTenderApprovalsByStatusWithTender`, and `GetTenderApprovalsBySubmissionModeWithTender` methods with a new approach that loads tender details in bulk using `loadTendersByIDs`.
- Introduced helper functions `fullTenderDetailsFromResponse` and `listTenderDetailsFromResponse` to streamline the mapping of tender responses to `TenderDetails`.
- Added a new file `tender_details.go` to encapsulate tender detail mapping logic, improving code organization and readability.

This update enhances the performance of the tender approval service by reducing redundant calls to fetch tender details, leading to more efficient data handling.
2026-07-08 01:17:17 +03:30
Mazyar 107469c328 Enhance tender recommendation service with caching and parallel processing
continuous-integration/drone/push Build is passing
- Introduced a caching mechanism for translation responses in the tender recommendation service, improving efficiency by reducing redundant translation requests.
- Refactored the `enrichWithTranslation` method to support an optional translation cache, allowing for faster retrieval of stored translations.
- Implemented parallel processing for building recommended tender responses, utilizing goroutines to enhance performance and responsiveness.
- Added unit tests to validate the new caching behavior and ensure the correct application of translations in various scenarios.

This update significantly improves the performance and responsiveness of the tender recommendation service by integrating caching and parallel processing, enhancing the overall user experience.
2026-07-08 00:57:52 +03:30
Mazyar 0e4fadaf29 Enhance company recommendation caching and pipeline integration
continuous-integration/drone/push Build is passing
- Updated the company service to include a new method for scheduling the refresh of cached AI recommendations after the AI pipeline execution.
- Introduced a new interface for managing cached recommendation refreshes, improving the separation of concerns within the service layer.
- Enhanced the worker initialization to include Redis client support, allowing for better management of recommendation caching.
- Added functionality to list company IDs with existing recommendation caches, ensuring efficient updates post-pipeline runs.
- Implemented unit tests to validate the new recommendation refresh logic and ensure proper handling of various scenarios.

This update significantly improves the handling of AI recommendations by integrating caching mechanisms with the AI pipeline, enhancing overall system performance and responsiveness.
2026-07-08 00:45:53 +03:30
Mazyar 4e5296d5dd Add deduplication and normalization for tender recommendations
continuous-integration/drone/push Build is passing
- Introduced a new function `dedupeRecommendationResponsesByTenderID` to remove duplicate tender recommendations based on their IDs, ensuring unique entries in the recommendation list.
- Updated the `mergeRecommendedTenders` function to utilize the new deduplication logic, streamlining the merging process of recommendation lists.
- Enhanced the `fetchAIRecommendations` method to apply deduplication on the fetched recommendations, improving data integrity.
- Added unit tests for the new deduplication function to validate its behavior and ensure correct handling of various input scenarios.

This update enhances the recommendation handling by ensuring that duplicate tender entries are effectively managed, improving the overall quality of the recommendations provided to users.
2026-07-07 23:00:16 +03:30
Mazyar 1ad0206e61 Refactor AI pipeline handling to support daily runs
continuous-integration/drone/push Build is passing
- Renamed and refactored the AI pipeline auto run functionality to a daily run, enhancing clarity and purpose.
- Introduced a new `AIPipelineDailyWorker` to manage the daily execution of the AI pipeline, replacing the previous auto run implementation.
- Updated configuration fields and logging messages to reflect the change from auto to daily run, ensuring consistent terminology throughout the codebase.
- Removed the obsolete `ai_pipeline_auto.go` file to streamline the worker structure.

This update improves the maintainability and readability of the AI pipeline management by clearly distinguishing between auto and daily run functionalities.
2026-07-07 12:17:03 +03:30
Mazyar 89faa08b1c Add functionality to manage external links for companies
continuous-integration/drone/push Build is passing
- Introduced the ability to append external links to company profiles through a new API endpoint.
- Enhanced the `Company` entity to include a `Links` field for storing external resource links.
- Created `AddLinksForm` for validating incoming link data and implemented corresponding logic in the service layer.
- Added error handling for link validation, ensuring only valid URLs are accepted and limiting the number of links to 20.
- Implemented unit tests for link management functions, including sanitization and merging of links.
- Updated relevant API documentation to reflect the new functionality.

This update significantly enhances the company management capabilities by allowing the addition of external links, improving the overall user experience and data richness in company profiles.
2026-07-06 00:05:48 +03:30
Mazyar 3aeb8630e3 Add company selection handling in customer forms with unit tests
continuous-integration/drone/push Build is passing
- Introduced new functionality in `form_companies.go` to handle company selection from various input formats, including legacy `company_ids` and company summary objects.
- Implemented `UnmarshalJSON` methods for `CreateCustomerForm`, `UpdateCustomerForm`, and `AssignCompaniesForm` to support flexible company data parsing.
- Added unit tests in `form_companies_test.go` to validate the correct unmarshalling of company IDs and summaries, ensuring robust handling of different input scenarios.
- Enhanced the `GetByCompanies` method in the customer repository to directly use string company IDs, improving data retrieval efficiency.

This update significantly improves the handling of company data in customer forms, ensuring accurate processing and validation of company selections, while expanding test coverage for these functionalities.
2026-07-05 21:20:28 +03:30
Mazyar a762430bca Add unit tests for tender recommendation translation handling
continuous-integration/drone/push Build is passing
- Introduced a new test file `recommendation_translation_test.go` to validate the behavior of the tender recommendation service regarding translation handling.
- Implemented tests to ensure that the service correctly applies stored translations and retains original text when no translation is available.
- Enhanced the `Recommend` method in the tender service to utilize a structured approach for managing recommended tenders, improving clarity and maintainability.

This update improves the testing coverage for translation handling in tender recommendations, ensuring accurate responses based on available translations and original content.
2026-07-05 21:01:42 +03:30
Mazyar 3221a31ac8 Refactor tender service methods for rejected tenders and enhance unit tests
continuous-integration/drone/push Build is passing
- Renamed `buildRejectedTenderIDSetForCompanies` to `buildRejectedTenderIDSetsForCompanies` to better reflect its functionality of returning a map of excluded tender IDs by company.
- Updated the implementation to return a nested map structure for better organization of excluded tender IDs per company.
- Added a new test function `TestIsRecommendedTenderExcludedForAllCompanies` to validate the visibility of tenders based on company-specific exclusions.
- Enhanced existing tests to ensure comprehensive coverage of the new logic for handling rejected tenders.

This update improves the clarity and functionality of the tender recommendation process by refining the handling of rejected tenders and expanding test coverage.
2026-07-04 13:56:07 +03:30
Mazyar 4c48e0bb3b Add unit test for mapping procedure references to tenders
continuous-integration/drone/push Build is passing
- Introduced a new test function `TestMapProcedureReferencesToTenders` to validate the mapping of procedure references to corresponding tenders.
- Enhanced the `GetByProcedureReferences` method in the tender repository to utilize a more efficient filtering mechanism based on unique contract folder IDs.
- Updated pagination logic to dynamically adjust limits based on the number of unique folders and references, ensuring optimal data retrieval.

This update improves the testing coverage for tender mapping functionality and optimizes the repository's data fetching strategy, enhancing overall performance and reliability.
2026-07-04 13:39:14 +03:30
Mazyar 467090e5d2 Enhance AI recommendations handling with validation and caching improvements
continuous-integration/drone/push Build is passing
- Introduced a new method to validate company IDs for AI recommendations, ensuring all requested companies exist before processing.
- Added caching logic to retrieve AI recommendations efficiently, reducing unnecessary calls and improving performance.
- Utilized errgroup for concurrent processing of AI recommendations across multiple companies, enhancing responsiveness.
- Updated the tender service to handle rejected tenders more effectively by incorporating concurrent fetching of rejected tender IDs.

This update significantly improves the AI recommendations process by ensuring accurate company validation and optimizing data retrieval through caching and concurrency, enhancing overall system performance and user experience.
2026-07-04 13:28:41 +03:30
Mazyar 1edf42187d Implement AI recommendations for multiple companies and enhance company context handling
continuous-integration/drone/push Build is passing
- Added functionality to retrieve merged AI recommendations for multiple companies, improving the relevance of tender suggestions based on company-specific data.
- Introduced normalization functions to clean and deduplicate company IDs, ensuring accurate processing of recommendations.
- Enhanced the company context resolution in customer middleware to support multiple assigned companies, improving the handling of company-specific requests.
- Updated the tender recommendation logic to utilize the new merged recommendations and handle exclusions for rejected tenders accordingly.
- Added unit tests to verify the new recommendation merging logic and company ID normalization, ensuring robust functionality.

This update significantly enhances the tender recommendation process by allowing for more comprehensive and relevant suggestions based on multiple company contexts, improving user experience and satisfaction.
2026-07-04 13:00:33 +03:30
m.nazemi f8c98da132 Merge pull request 'Enhance tender service to exclude company-rejected tenders from recommendations' (#47) from recommendation-approval into develop
Reviewed-on: https://repo.ravanertebat.com/TM/tm_back/pulls/47
2026-07-04 10:46:28 +03:30
Mazyar fe3a71ba2b Enhance tender service to exclude company-rejected tenders from recommendations
- Updated the tender service to include a new dependency for listing rejected tenders by company, allowing for more refined tender recommendations.
- Introduced a new field in the SearchForm to specify whether to exclude rejected tenders from the recommendation results.
- Enhanced the Recommend method to filter out rejected tenders based on the new exclusion logic, improving the relevance of AI-ranked tender recommendations.
- Added unit tests to verify the exclusion logic for rejected tenders, ensuring robust functionality.

This update improves the tender recommendation process by ensuring that company-rejected tenders are not included in the results, enhancing user experience and satisfaction.
2026-07-03 19:55:01 +03:30
78 changed files with 7096 additions and 589 deletions
+31 -9
View File
@@ -119,6 +119,7 @@ import (
"tm/internal/notification"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/internal/tender_submission"
"tm/internal/user"
"tm/pkg/filestore"
"tm/pkg/security"
@@ -201,6 +202,7 @@ func main() {
tenderRepository := tender.NewRepository(mongoManager, logger)
feedbackRepo := feedback.NewRepository(mongoManager, logger)
tenderApprovalRepository := tender_approval.NewRepository(mongoManager, logger)
tenderSubmissionRepository := tender_submission.NewRepository(mongoManager, logger)
inquiryRepository := inquiry.NewRepository(mongoManager, logger)
contactRepository := contact.NewContactRepository(mongoManager, logger)
cmsRepository := cms.NewRepository(mongoManager, logger)
@@ -210,7 +212,7 @@ func main() {
cardRepository := kanban.NewCardRepository(mongoManager, logger)
notificationRepository := notification.NewRepository(mongoManager, logger)
logger.Info("Repositories initialized successfully", map[string]interface{}{
"repositories": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "contact", "cms", "notice", "kanban_board", "kanban_column", "kanban_card", "notification"},
"repositories": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "contact", "cms", "notice", "kanban_board", "kanban_column", "kanban_card", "notification"},
})
// Initialize validation services
@@ -240,11 +242,30 @@ func main() {
userService := user.NewService(userRepository, logger, userAuthService, userValidator, notificationSDK, redisClient, auditLogger)
categoryService := company_category.NewService(categoryRepository, logger)
companyService := company.NewService(companyRepository, categoryService, fileStoreService, aiRecommendationClient, redisClient, conf.AISummarizer.RecommendationCacheTTL, logger)
companyService := company.NewService(companyRepository, categoryService, fileStoreService, aiRecommendationClient, aiPipelineClient, redisClient, conf.AISummarizer.RecommendationCacheTTL, logger)
customerService := customer.NewService(customerRepository, logger, customerAuthService, companyService, redisClient, notificationSDK, customerValidator, auditLogger)
tenderService := tender.NewService(tenderRepository, companyService, customerService, logger, aiSummarizerClient, aiSummarizerStorage, conf.AISummarizer.DefaultLanguage)
tenderService := tender.NewService(
tenderRepository,
companyService,
customerService,
tenderApprovalRepository,
logger,
aiSummarizerClient,
aiSummarizerStorage,
redisClient,
conf.AISummarizer.RecommendationCacheTTL,
conf.AISummarizer.DefaultLanguage,
)
if configurer, ok := companyService.(interface {
SetRecommendedTendersPageCacheRefresher(company.RecommendedTendersPageCacheRefresher)
}); ok {
if refresher, ok := tenderService.(company.RecommendedTendersPageCacheRefresher); ok {
configurer.SetRecommendedTendersPageCacheRefresher(refresher)
}
}
feedbackService := feedback.NewService(feedbackRepo, tenderService, companyService, logger)
tenderApprovalService := tender_approval.NewService(tenderApprovalRepository, tenderService, logger)
tenderSubmissionService := tender_submission.NewService(tenderSubmissionRepository, tenderApprovalRepository, tenderService, logger)
tenderApprovalService := tender_approval.NewService(tenderApprovalRepository, tenderService, logger, tenderSubmissionService)
inquiryService := inquiry.NewService(inquiryRepository, notificationSDK, logger, xssPolicy)
flagService := assets.NewService(logger, conf.Assets.FlagsPath)
notificationService := notification.NewService(notificationSDK, notificationRepository, userService, customerService, logger)
@@ -254,11 +275,11 @@ func main() {
documentScraperService := document_scraper.NewService(tenderRepository, aiPipelineClient, logger)
dashboardRepository := dashboard.NewRepository(mongoManager, logger, procedureDocumentsLister)
dashboardService := dashboard.NewService(dashboardRepository, logger, redisClient)
aiPipelineService := ai_pipeline.NewService(aiPipelineClient, logger, tenderService)
aiPipelineService := ai_pipeline.NewService(aiPipelineClient, logger, tenderService, companyService)
auditLogRepository := auditlog.NewRepository(auditStore)
auditLogService := auditlog.NewService(auditLogRepository, logger)
logger.Info("Services initialized successfully", map[string]interface{}{
"services": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
"services": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
})
// Initialize handlers with services
@@ -269,6 +290,7 @@ func main() {
tenderHandler := tender.NewHandler(tenderService, logger)
feedbackHandler := feedback.NewHandler(feedbackService, userHandler, customerHandler, logger)
tenderApprovalHandler := tender_approval.NewHandler(tenderApprovalService, logger)
tenderSubmissionHandler := tender_submission.NewHandler(tenderSubmissionService, logger)
inquiryHandler := inquiry.NewHandler(inquiryService, logger, hcaptchaVerifier, xssPolicy)
flagHandler := assets.NewHandler(flagService, logger)
notificationHandler := notification.NewHandler(notificationService)
@@ -281,7 +303,7 @@ func main() {
aiPipelineHandler := ai_pipeline.NewHandler(aiPipelineService)
auditLogHandler := auditlog.NewHandler(auditLogService)
logger.Info("Handlers initialized successfully", map[string]interface{}{
"handlers": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "filestore", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
"handlers": []string{"customer", "user", "company", "category", "tender", "feedback", "tender_approval", "tender_submission", "inquiry", "flag", "notification", "contact", "cms", "scraper", "kanban", "filestore", "document_scraper", "dashboard", "ai_pipeline", "auditlog"},
})
// Initialize HTTP server
@@ -290,8 +312,8 @@ func main() {
router.SetupCSPSecurity(e)
// Register routes
router.RegisterAdminRoutes(e, userHandler, companyHandler, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, dashboardHandler, aiPipelineHandler, auditLogHandler, xssPolicy)
router.RegisterPublicRoutes(e, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, companyHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, xssPolicy)
router.RegisterAdminRoutes(e, userHandler, companyHandler, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, tenderSubmissionHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, dashboardHandler, aiPipelineHandler, auditLogHandler, xssPolicy)
router.RegisterPublicRoutes(e, customerHandler, tenderHandler, feedbackHandler, tenderApprovalHandler, tenderSubmissionHandler, companyHandler, inquiryHandler, categoryHandler, flagHandler, notificationHandler, contactHandler, cmsHandler, kanbanHandler, fileStoreHandler, xssPolicy)
router.RegisterDocumentScraperRoutes(e, documentScraperHandler, conf.DocumentScraper.APIKey)
// Start server
+25 -2
View File
@@ -17,6 +17,7 @@ import (
"tm/internal/notification"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/internal/tender_submission"
"tm/internal/user"
"tm/pkg/filestore"
"tm/pkg/security"
@@ -36,7 +37,7 @@ func SetupCSPSecurity(e *echo.Echo) {
e.POST("/api/v1/csp-report", security.CSPReportHandler)
}
func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler *company.Handler, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, dashboardHandler *dashboard.Handler, aiPipelineHandler *ai_pipeline.Handler, auditLogHandler *auditlog.Handler, xssPolicy *security.XSSPolicy) {
func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler *company.Handler, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, tenderSubmissionHandler *tender_submission.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, dashboardHandler *dashboard.Handler, aiPipelineHandler *ai_pipeline.Handler, auditLogHandler *auditlog.Handler, xssPolicy *security.XSSPolicy) {
adminV1 := e.Group("/admin/v1")
// Admin Users Routes
@@ -88,6 +89,7 @@ func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler
companiesGP.PATCH("/:id/status", companyHandler.UpdateStatus)
companiesGP.PATCH("/:id/verification", companyHandler.UpdateVerification)
companiesGP.POST("/:id/documents", companyHandler.UploadDocuments)
companiesGP.POST("/:id/links", companyHandler.AddLinks)
}
// Admin Categories Routes
@@ -155,6 +157,15 @@ func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler
tenderApprovalGP.GET("/status/:status", tenderApprovalHandler.GetTenderApprovalsByStatus)
}
// Admin Tender-Submissions Routes
tenderSubmissionGP := adminV1.Group("/tender-submissions")
{
tenderSubmissionGP.Use(userHandler.AuthMiddleware())
tenderSubmissionGP.GET("", tenderSubmissionHandler.AdminListSubmissions)
tenderSubmissionGP.GET("/stats", tenderSubmissionHandler.AdminGetSubmissionStats)
tenderSubmissionGP.GET("/:id", tenderSubmissionHandler.AdminGetSubmissionByID)
}
// Admin Inquiry Routes
inquiryGP := adminV1.Group("/inquiries")
{
@@ -259,7 +270,7 @@ func RegisterAdminRoutes(e *echo.Echo, userHandler *user.Handler, companyHandler
}
}
func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, companyHandler *company.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, xssPolicy *security.XSSPolicy) {
func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tenderHandler *tender.TenderHandler, feedbackHandler *feedback.Handler, tenderApprovalHandler *tender_approval.Handler, tenderSubmissionHandler *tender_submission.Handler, companyHandler *company.Handler, inquiryHandler *inquiry.Handler, categoryHandler *company_category.Handler, flagHandler *assets.Handler, notificationHandler *notification.NotificationHandler, contactHandler *contact.Handler, cmsHandler *cms.Handler, kanbanHandler *kanban.Handler, fileStoreHandler *filestore.Handler, xssPolicy *security.XSSPolicy) {
v1 := e.Group("/api/v1")
customerGP := v1.Group("/profile")
@@ -316,6 +327,18 @@ func RegisterPublicRoutes(e *echo.Echo, customerHandler *customer.Handler, tende
tenderApprovalGP.GET("/stats", tenderApprovalHandler.GetCompanyTenderApprovalStats)
}
// Public Tender Submissions Routes
tenderSubmissionGP := v1.Group("/tender-submissions")
{
tenderSubmissionGP.Use(customerHandler.AuthMiddleware(), customerHandler.CompanyContextMiddleware())
tenderSubmissionGP.GET("", tenderSubmissionHandler.ListCompanySubmissions)
tenderSubmissionGP.GET("/stats", tenderSubmissionHandler.GetCompanySubmissionStats)
tenderSubmissionGP.POST("/tender/:tender_id/ensure", tenderSubmissionHandler.EnsureSubmission)
tenderSubmissionGP.GET("/tender/:tender_id", tenderSubmissionHandler.GetSubmissionByTender)
tenderSubmissionGP.PATCH("/:id/status", tenderSubmissionHandler.UpdateSubmissionStatus)
tenderSubmissionGP.GET("/:id", tenderSubmissionHandler.GetSubmissionByID)
}
// Public Company Routes
companiesGP := v1.Group("/companies")
{
+79 -15
View File
@@ -2,11 +2,14 @@ package bootstrap
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"time"
"tm/cmd/worker/workers"
"tm/internal/company"
"tm/internal/company_category"
"tm/internal/notice"
notificationDomain "tm/internal/notification"
"tm/internal/tender"
@@ -16,11 +19,12 @@ import (
"tm/pkg/mongo"
"tm/pkg/notification"
"tm/pkg/ollama"
"tm/pkg/redis"
"tm/pkg/schedule"
)
// aiPipelineAutoRunMu ensures only one AI pipeline auto run executes at a time (startup catch-up and cron).
var aiPipelineAutoRunMu sync.Mutex
// aiPipelineDailyRunMu ensures only one AI pipeline auto run executes at a time (startup catch-up and cron).
var aiPipelineDailyRunMu sync.Mutex
// Init Application Configuration
func InitConfig() (*Config, error) {
@@ -82,7 +86,7 @@ func InitMongoDB(conf config.MongoConfig, log logger.Logger) *mongo.ConnectionMa
}
// InitWorker
func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger logger.Logger, notify notification.SDK, aiClient *ai_summarizer.Client, aiStorage *ai_summarizer.StorageClient) *schedule.CronScheduler {
func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger logger.Logger, notify notification.SDK, aiClient *ai_summarizer.Client, aiStorage *ai_summarizer.StorageClient, redisClient redis.Client) *schedule.CronScheduler {
// Debug: Log worker config
appLogger.Info("Worker configuration", map[string]interface{}{
"worker_interval": config.Worker.Interval,
@@ -102,6 +106,26 @@ func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger
tenderRepo := tender.NewRepository(mongoManager, appLogger)
notificationRepo := notificationDomain.NewRepository(mongoManager, appLogger)
var recommendationRefresher company.Service
if config.Worker.RecommendationRefreshAfterPipelineEnabled && aiClient != nil && redisClient != nil {
companyRepo := company.NewRepository(mongoManager, appLogger)
categoryRepo := company_category.NewRepository(mongoManager, appLogger)
categoryService := company_category.NewService(categoryRepo, appLogger)
recommendationRefresher = company.NewService(
companyRepo,
categoryService,
nil,
aiClient,
aiClient,
redisClient,
config.AISummarizer.RecommendationCacheTTL,
appLogger,
)
appLogger.Info("Company recommendation refresh after pipeline enabled", map[string]interface{}{})
} else if config.Worker.RecommendationRefreshAfterPipelineEnabled {
appLogger.Warn("Company recommendation refresh after pipeline disabled: AI client or Redis unavailable", map[string]interface{}{})
}
// Create a single shared cron scheduler for all recurring jobs
scheduler := schedule.NewCronScheduler(appLogger, mongoManager, noticeRepo)
@@ -213,14 +237,14 @@ func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger
} else if aiClient != nil {
dailyJobTracker := schedule.NewDailyJobTracker(mongoManager, appLogger)
runAIPipelineAuto := func(trigger string) {
aiPipelineAutoRunMu.Lock()
defer aiPipelineAutoRunMu.Unlock()
runAIPipelineDaily := func(trigger string) {
aiPipelineDailyRunMu.Lock()
defer aiPipelineDailyRunMu.Unlock()
ctx := context.Background()
today := time.Now().Local()
completed, checkErr := dailyJobTracker.IsCompleted(ctx, schedule.AIPipelineAutoJobName, today)
completed, checkErr := dailyJobTracker.IsCompleted(ctx, schedule.AIPipelineDailyJobName, today)
if checkErr != nil {
appLogger.Error("Failed to check AI pipeline auto completion status", map[string]interface{}{
"trigger": trigger,
@@ -236,26 +260,33 @@ func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger
}
if trigger == "startup" {
appLogger.Info("Running startup catch-up for today's AI pipeline auto", map[string]interface{}{
appLogger.Info("Running startup catch-up for today's AI pipeline auto run", map[string]interface{}{
"date": today.Format("02/01/2006"),
})
} else {
appLogger.Info("Running scheduled AI pipeline auto", map[string]interface{}{
appLogger.Info("Running scheduled AI pipeline auto run", map[string]interface{}{
"date": today.Format("02/01/2006"),
})
}
worker := workers.NewAIPipelineAutoWorker(appLogger, aiClient)
worker := workers.NewAIPipelineDailyWorker(appLogger, aiClient)
if err := worker.Run(); err != nil {
appLogger.Error("AI pipeline auto run failed", map[string]interface{}{
if errors.Is(err, ai_summarizer.ErrPipelineJobRunning) {
appLogger.Info("AI pipeline auto deferred: job already running", map[string]interface{}{
"trigger": trigger,
"date": today.Format("02/01/2006"),
})
} else {
appLogger.Error("AI pipeline auto failed", map[string]interface{}{
"trigger": trigger,
"date": today.Format("02/01/2006"),
"error": err.Error(),
})
}
return
}
if markErr := dailyJobTracker.MarkCompleted(ctx, schedule.AIPipelineAutoJobName, today); markErr != nil {
if markErr := dailyJobTracker.MarkCompleted(ctx, schedule.AIPipelineDailyJobName, today); markErr != nil {
appLogger.Error("Failed to mark AI pipeline auto as completed", map[string]interface{}{
"trigger": trigger,
"date": today.Format("02/01/2006"),
@@ -263,15 +294,19 @@ func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger
})
}
appLogger.Info("AI pipeline auto run completed successfully", map[string]interface{}{
appLogger.Info("AI pipeline auto completed successfully", map[string]interface{}{
"trigger": trigger,
"date": today.Format("02/01/2006"),
})
if recommendationRefresher != nil {
recommendationRefresher.ScheduleRefreshCachedAIRecommendationsAfterPipeline()
}
}
scheduler.AddJob(schedule.Job{
Name: "AI Pipeline Auto Job",
Func: func() { runAIPipelineAuto("scheduled") },
Func: func() { runAIPipelineDaily("scheduled") },
Expr: config.Worker.AIPipelineAutoInterval,
})
appLogger.Info("Scheduled AI pipeline auto worker", map[string]interface{}{
@@ -279,7 +314,7 @@ func InitWorker(config Config, mongoManager *mongo.ConnectionManager, appLogger
})
go func() {
runAIPipelineAuto("startup")
runAIPipelineDaily("startup")
}()
} else {
appLogger.Warn("AI summarizer client not available, AI pipeline auto worker is disabled", map[string]interface{}{})
@@ -379,6 +414,35 @@ func InitAISummarizerStorage(conf AISummarizerConfig, log logger.Logger) *ai_sum
return storage
}
// InitRedis initializes the Redis client used by worker jobs.
func InitRedis(conf config.RedisConfig, log logger.Logger) redis.Client {
connectionConfig := &redis.Config{
Host: conf.Host,
Port: conf.Port,
Password: conf.Password,
DB: conf.DB,
PoolSize: conf.PoolSize,
}
redisClient, err := redis.NewClient(connectionConfig, log)
if err != nil {
log.Error("Failed to initialize Redis connection for worker", map[string]interface{}{
"error": err.Error(),
"host": conf.Host,
"port": conf.Port,
})
panic(fmt.Sprintf("Failed to initialize Redis connection: %v", err))
}
log.Info("Redis connection initialized for worker", map[string]interface{}{
"host": conf.Host,
"port": conf.Port,
"pool_size": conf.PoolSize,
})
return redisClient
}
func parseTranslationLanguages(conf AISummarizerConfig) []string {
raw := strings.TrimSpace(conf.TranslationLanguages)
if raw == "" {
+5 -1
View File
@@ -8,6 +8,7 @@ import (
// Config defines the worker application configuration
type Config struct {
Database config.DatabaseConfig
Redis config.RedisConfig
Logging config.LoggingConfig
Notification config.NotificationConfig
Ollama config.OllamaConfig
@@ -38,8 +39,10 @@ type WorkerConfig struct {
// AIPipelineAutoEnabled schedules the daily AI pipeline auto run on the worker.
// On-demand pipeline routes on web are unaffected.
AIPipelineAutoEnabled bool `env:"WORKER_AI_PIPELINE_AUTO_ENABLED" envDefault:"true"`
// AIPipelineAutoInterval runs once per day by default at 11:00 UTC (after TED scrape at 10:00).
// AIPipelineAutoInterval runs pipeline auto once per day by default at 11:00 UTC (after TED scrape at 10:00).
AIPipelineAutoInterval string `env:"WORKER_AI_PIPELINE_AUTO_INTERVAL" envDefault:"0 0 11 * * *"`
// RecommendationRefreshAfterPipelineEnabled refreshes cached company recommendations after the daily AI pipeline auto run.
RecommendationRefreshAfterPipelineEnabled bool `env:"WORKER_RECOMMENDATION_REFRESH" envDefault:"true"`
}
// AISummarizerConfig holds configuration for the external AI summarizer service.
@@ -56,4 +59,5 @@ type AISummarizerConfig struct {
MinioUseSSL bool `env:"AI_SUMMARIZER_MINIO_USE_SSL" envDefault:"false"`
MinioRegion string `env:"AI_SUMMARIZER_MINIO_REGION" envDefault:"us-east-1"`
MinioBucket string `env:"AI_SUMMARIZER_MINIO_BUCKET" envDefault:"opplens"`
RecommendationCacheTTL time.Duration `env:"AI_RECOMMENDATION_CACHE_TTL" envDefault:"0"`
}
+11 -1
View File
@@ -40,9 +40,19 @@ func main() {
// Initialize AI summarizer client (translation worker)
aiSummarizerClient := bootstrap.InitAISummarizerClient(config.AISummarizer, mongoManager, appLogger)
aiSummarizerStorage := bootstrap.InitAISummarizerStorage(config.AISummarizer, appLogger)
redisClient := bootstrap.InitRedis(config.Redis, appLogger)
defer func() {
if redisClient != nil {
if err := redisClient.Close(); err != nil {
appLogger.Error("Failed to close Redis connection", map[string]interface{}{
"error": err.Error(),
})
}
}
}()
// Initialize Worker
scheduler := bootstrap.InitWorker(*config, mongoManager, appLogger, notificationService, aiSummarizerClient, aiSummarizerStorage)
scheduler := bootstrap.InitWorker(*config, mongoManager, appLogger, notificationService, aiSummarizerClient, aiSummarizerStorage, redisClient)
// Set up signal handling for graceful shutdown
signalChan := make(chan os.Signal, 1)
@@ -2,29 +2,29 @@ package workers
import (
"context"
"errors"
"time"
"tm/pkg/ai_summarizer"
"tm/pkg/logger"
)
// AIPipelineAutoWorker triggers the Opplens AI pipeline auto run (sync + complete missing work).
type AIPipelineAutoWorker struct {
// AIPipelineDailyWorker triggers the scheduled Opplens AI pipeline auto run.
type AIPipelineDailyWorker struct {
Logger logger.Logger
AIClient *ai_summarizer.Client
}
// NewAIPipelineAutoWorker creates a worker that calls POST /pipeline/auto on the AI service.
func NewAIPipelineAutoWorker(log logger.Logger, aiClient *ai_summarizer.Client) *AIPipelineAutoWorker {
return &AIPipelineAutoWorker{
// NewAIPipelineDailyWorker creates a worker that calls POST /pipeline/auto on the AI service.
func NewAIPipelineDailyWorker(log logger.Logger, aiClient *ai_summarizer.Client) *AIPipelineDailyWorker {
return &AIPipelineDailyWorker{
Logger: log,
AIClient: aiClient,
}
}
// Run starts the pipeline auto job. A 409 response means a run is already in progress and is not an error.
func (w *AIPipelineAutoWorker) Run() error {
// Run starts the pipeline auto job. A 409 response means a run is already in progress and is returned as an error
// so the daily job tracker does not mark today as completed before the run actually starts.
func (w *AIPipelineDailyWorker) Run() error {
if w.AIClient == nil {
w.Logger.Warn("AI pipeline auto worker skipped: AI client is not configured", map[string]interface{}{})
return nil
@@ -37,10 +37,6 @@ func (w *AIPipelineAutoWorker) Run() error {
resp, err := w.AIClient.PipelineAuto(ctx)
if err != nil {
if errors.Is(err, ai_summarizer.ErrPipelineJobRunning) {
w.Logger.Info("AI pipeline auto skipped: job already running", map[string]interface{}{})
return nil
}
w.Logger.Error("AI pipeline auto worker failed", map[string]interface{}{
"error": err.Error(),
})
-1
View File
@@ -565,7 +565,6 @@ func (w *NoticeWorker) ToTender(n *notice.Notice) (*tender.Tender, error) {
t.ProcurementProjectID = n.ProcurementProjectID
t.SourceFileURL = n.SourceFileURL
t.SourceFileName = n.SourceFileName
t.ContentXML = n.ContentXML
// Preserve document-scraping/summarization flags on the tender (managed by
// downstream workers), but refresh the notice-side processing metadata.
t.ProcessingMetadata.ScrapedAt = n.ProcessingMetadata.ScrapedAt
+26 -1
View File
@@ -48,6 +48,11 @@ type Service interface {
GetMinioStats(ctx context.Context) (ai_summarizer.PipelineMinioStatsResponse, error)
}
// CachedRecommendationRefresher schedules a background refresh of company recommendation caches.
type CachedRecommendationRefresher interface {
ScheduleRefreshCachedAIRecommendationsAfterPipeline()
}
// ScrapedDocumentMetadataSyncer persists MinIO scraped-document metadata onto tender records.
type ScrapedDocumentMetadataSyncer interface {
SyncScrapedDocumentsFromStorage(ctx context.Context, contractFolderID, noticePublicationID string) error
@@ -57,14 +62,21 @@ type service struct {
client Client
logger logger.Logger
metadataSyncer ScrapedDocumentMetadataSyncer
recommendationRefresher CachedRecommendationRefresher
}
// NewService creates an AI pipeline admin service.
func NewService(client Client, log logger.Logger, metadataSyncer ScrapedDocumentMetadataSyncer) Service {
func NewService(
client Client,
log logger.Logger,
metadataSyncer ScrapedDocumentMetadataSyncer,
recommendationRefresher CachedRecommendationRefresher,
) Service {
return &service{
client: client,
logger: log,
metadataSyncer: metadataSyncer,
recommendationRefresher: recommendationRefresher,
}
}
@@ -219,6 +231,8 @@ func (s *service) Sync(ctx context.Context) (*ai_summarizer.PipelineSyncResponse
return nil, fmt.Errorf("pipeline sync failed: %w", err)
}
s.scheduleRecommendationRefreshAfterPipeline()
return resp, nil
}
@@ -322,6 +336,8 @@ func (s *service) DailyRun(ctx context.Context) (*ai_summarizer.PipelineStartedR
return nil, fmt.Errorf("pipeline daily-run failed: %w", err)
}
s.scheduleRecommendationRefreshAfterPipeline()
return resp, nil
}
@@ -340,6 +356,8 @@ func (s *service) Auto(ctx context.Context) (*ai_summarizer.PipelineStartedRespo
return nil, fmt.Errorf("pipeline auto failed: %w", err)
}
s.scheduleRecommendationRefreshAfterPipeline()
return resp, nil
}
@@ -425,6 +443,13 @@ func (s *service) GetMinioStats(ctx context.Context) (ai_summarizer.PipelineMini
return resp, nil
}
func (s *service) scheduleRecommendationRefreshAfterPipeline() {
if s.recommendationRefresher == nil {
return
}
s.recommendationRefresher.ScheduleRefreshCachedAIRecommendationsAfterPipeline()
}
func validateTenderBatch(forms []TenderRefForm) ([]ai_summarizer.TenderRef, error) {
if len(forms) == 0 {
return nil, ErrEmptyTenderBatch
+9
View File
@@ -53,6 +53,9 @@ type Company struct {
// File references stored in GridFS
DocumentFileIDs []string `bson:"document_file_ids" json:"document_file_ids,omitempty"`
// External resource links (e.g. company profile pages, portfolios)
Links []CompanyLink `bson:"links,omitempty" json:"links,omitempty"`
// Tags for categorization and search
Tags *CompanyTags `bson:"tags,omitempty" json:"tags,omitempty"`
@@ -97,6 +100,12 @@ func (c *Company) GetUpdatedAt() int64 {
return c.UpdatedAt
}
// CompanyLink represents an external URL associated with a company.
type CompanyLink struct {
URL string `bson:"url" json:"url"`
Title *string `bson:"title,omitempty" json:"title,omitempty"`
}
// Address represents a company's address
type Address struct {
Street string `bson:"street" json:"street"`
+16
View File
@@ -25,6 +25,7 @@ type (
Phone *string `json:"phone,omitempty" valid:"optional,length(10|20)" example:"+1234567890"`
Email *string `json:"email,omitempty" valid:"optional,email" example:"info@opplens.com"`
DocumentFileIDs []string `json:"document_file_ids,omitempty" valid:"optional"`
Links []CompanyLinkForm `json:"links,omitempty" valid:"optional"`
// Tags for categorization and search
Tags *CompanyTagsForm `json:"tags,omitempty"`
@@ -35,6 +36,12 @@ type (
Timezone *string `json:"timezone,omitempty" valid:"optional,length(3|50)" example:"UTC"`
}
// CompanyLinkForm represents the form for a company link.
CompanyLinkForm struct {
URL string `json:"url" valid:"required,url" example:"https://example.com/profile"`
Title *string `json:"title,omitempty" valid:"optional,length(1|200)" example:"Company profile"`
}
// AddressForm represents the form for company address
AddressForm struct {
Street string `json:"street" valid:"required,length(5|200)" example:"123 Main St"`
@@ -90,6 +97,11 @@ type StatusForm struct {
Reason *string `json:"reason" valid:"optional,length(10|500)"`
}
// AddLinksForm is the request body for appending links to a company.
type AddLinksForm struct {
Links []CompanyLinkForm `json:"links" valid:"required"`
}
// VerificationForm represents the form for updating company verification status
type VerificationForm struct {
IsVerified bool `json:"is_verified"`
@@ -128,6 +140,7 @@ type (
Phone *string `json:"phone,omitempty"`
Email *string `json:"email,omitempty"`
DocumentFileIDs []string `json:"document_file_ids,omitempty"`
Links []CompanyLink `json:"links,omitempty"`
Tags *CompanyTagsResponse `json:"tags,omitempty"`
IsVerified bool `json:"is_verified"`
IsCompliant bool `json:"is_compliant"`
@@ -167,6 +180,7 @@ func (c *Company) ToResponse(categories []*CategoryResponse) *CompanyResponse {
Phone: c.Phone,
Email: c.Email,
DocumentFileIDs: c.DocumentFileIDs,
Links: c.Links,
Tags: c.Tags.ToResponse(categories),
IsVerified: c.IsVerified,
IsCompliant: c.IsCompliant,
@@ -209,6 +223,7 @@ type CompanyProfileResponse struct {
Industry string `json:"industry"`
FoundedYear *int `json:"founded_year,omitempty"`
DocumentFileIDs []string `json:"document_file_ids,omitempty"`
Links []CompanyLink `json:"links,omitempty"`
CreatedAt int64 `json:"created_at"`
}
@@ -221,6 +236,7 @@ func (c *Company) ToProfileResponse() *CompanyProfileResponse {
Industry: c.Industry,
FoundedYear: c.FoundedYear,
DocumentFileIDs: c.DocumentFileIDs,
Links: c.Links,
CreatedAt: c.CreatedAt,
}
}
+69
View File
@@ -101,6 +101,21 @@ func (h *Handler) StartOnboarding(c echo.Context) error {
// @Security BearerAuth
// @Router /api/v1/recommend [post]
func (h *Handler) RecommendTenders(c echo.Context) error {
if companyIDs, ok := c.Get("company_ids").([]string); ok && len(companyIDs) > 0 {
result, err := h.service.GetMergedAIRecommendations(c.Request().Context(), companyIDs)
if err != nil {
if errors.Is(err, ErrAIRecommendationNotConfigured) {
return response.ServiceUnavailable(c, "AI recommendation service is not configured")
}
if err.Error() == "company not found" {
return response.NotFound(c, "Company not found")
}
return response.InternalServerError(c, "Failed to retrieve AI recommendations")
}
return response.Success(c, result, "AI recommendations retrieved successfully")
}
companyID, err := user.GetCompanyIDFromContext(c)
if err != nil {
return response.Unauthorized(c, "User not authenticated")
@@ -153,6 +168,9 @@ func (h *Handler) Create(c echo.Context) error {
err.Error() == "failed to validate categories" {
return response.BadRequest(c, err.Error(), "Invalid category data")
}
if err.Error() == "invalid link URL" || err.Error() == "too many links provided" {
return response.BadRequest(c, err.Error(), "")
}
return response.InternalServerError(c, "Failed to create company")
}
@@ -223,6 +241,9 @@ func (h *Handler) Update(c echo.Context) error {
err.Error() == "failed to validate categories" {
return response.BadRequest(c, err.Error(), "Invalid category data")
}
if err.Error() == "invalid link URL" || err.Error() == "too many links provided" {
return response.BadRequest(c, err.Error(), "")
}
return response.InternalServerError(c, "Failed to update company")
}
@@ -440,6 +461,54 @@ func (h *Handler) UploadDocuments(c echo.Context) error {
return response.Success(c, result, message)
}
// AddLinks appends external links to a company (Web Panel)
// @Summary Add company links
// @Description Append one or more external URLs to the company (max 20 links per request)
// @Tags Admin-Companies
// @Accept json
// @Produce json
// @Param id path string true "Company ID"
// @Param links body AddLinksForm true "Links to append"
// @Success 200 {object} response.APIResponse{data=AddLinksResponse} "Links added successfully"
// @Failure 400 {object} response.APIResponse "Bad request - No links or invalid URLs"
// @Failure 404 {object} response.APIResponse "Not found - Company not found"
// @Failure 422 {object} response.APIResponse "Validation error - Invalid request data"
// @Failure 500 {object} response.APIResponse "Internal server error"
// @Security BearerAuth
// @Router /admin/v1/companies/{id}/links [post]
func (h *Handler) AddLinks(c echo.Context) error {
id := c.Param("id")
form, err := response.Parse[AddLinksForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
if len(form.Links) > maxCompanyLinks {
return response.ValidationError(c, "Too many links", "Maximum 20 links per request")
}
result, err := h.service.AddLinks(c.Request().Context(), id, form)
if err != nil {
switch err.Error() {
case "company not found":
return response.NotFound(c, "Company not found")
case "no links provided", "no valid links provided", "invalid link URL":
return response.BadRequest(c, err.Error(), "")
case "too many links in request", "company link limit exceeded":
return response.ValidationError(c, err.Error(), "")
default:
return response.InternalServerError(c, "Failed to add company links")
}
}
message := "Company links added successfully"
if len(result.Added) == 0 {
message = "No new company links were added"
}
return response.Success(c, result, message)
}
func collectCompanyUploadFiles(form *multipart.Form) []*multipart.FileHeader {
if form == nil {
return nil
+164
View File
@@ -0,0 +1,164 @@
package company
import (
"context"
"errors"
"strings"
"github.com/asaskevich/govalidator"
)
const maxCompanyLinks = 20
// AddLinksResponse is returned after appending links to a company.
type AddLinksResponse struct {
Links []CompanyLink `json:"links"`
Added []CompanyLink `json:"added"`
}
// AddLinks appends validated links to a company.
func (s *companyService) AddLinks(ctx context.Context, companyID string, form *AddLinksForm) (*AddLinksResponse, error) {
if form == nil || len(form.Links) == 0 {
return nil, errors.New("no links provided")
}
if len(form.Links) > maxCompanyLinks {
return nil, errors.New("too many links in request")
}
company, err := s.repository.GetByID(ctx, companyID)
if err != nil {
return nil, errors.New("company not found")
}
newLinks, err := sanitizeCompanyLinks(convertCompanyLinkForms(form.Links))
if err != nil {
return nil, err
}
if len(newLinks) == 0 {
return nil, errors.New("no valid links provided")
}
existing := sanitizeStoredCompanyLinks(company.Links)
merged, added := mergeCompanyLinks(existing, newLinks)
if len(merged) > maxCompanyLinks {
return nil, errors.New("company link limit exceeded")
}
company.Links = merged
if err := s.repository.Update(ctx, company); err != nil {
s.logger.Error("Failed to save company links", map[string]interface{}{
"error": err.Error(),
"company_id": companyID,
})
return nil, errors.New("failed to update company links")
}
s.logger.Info("Company links added", map[string]interface{}{
"company_id": companyID,
"added_count": len(added),
"links_total": len(company.Links),
})
s.triggerAIOnboardingAsync(companyID)
return &AddLinksResponse{
Links: company.Links,
Added: added,
}, nil
}
func convertCompanyLinkForms(forms []CompanyLinkForm) []CompanyLink {
if len(forms) == 0 {
return nil
}
links := make([]CompanyLink, 0, len(forms))
for _, form := range forms {
links = append(links, CompanyLink{
URL: form.URL,
Title: form.Title,
})
}
return links
}
func sanitizeCompanyLinks(links []CompanyLink) ([]CompanyLink, error) {
if len(links) == 0 {
return nil, nil
}
sanitized := make([]CompanyLink, 0, len(links))
for _, link := range links {
url := strings.TrimSpace(link.URL)
if url == "" {
continue
}
if !govalidator.IsURL(url) {
return nil, errors.New("invalid link URL")
}
var title *string
if link.Title != nil {
trimmed := strings.TrimSpace(*link.Title)
if trimmed != "" {
title = &trimmed
}
}
sanitized = append(sanitized, CompanyLink{
URL: url,
Title: title,
})
}
return sanitized, nil
}
func sanitizeStoredCompanyLinks(links []CompanyLink) []CompanyLink {
sanitized, err := sanitizeCompanyLinks(links)
if err != nil || len(sanitized) == 0 {
return nil
}
return sanitized
}
func mergeCompanyLinks(existing, additional []CompanyLink) ([]CompanyLink, []CompanyLink) {
seen := make(map[string]struct{}, len(existing)+len(additional))
merged := make([]CompanyLink, 0, len(existing)+len(additional))
added := make([]CompanyLink, 0, len(additional))
appendUnique := func(items []CompanyLink, trackAdded bool) {
for _, item := range items {
key := strings.ToLower(strings.TrimSpace(item.URL))
if key == "" {
continue
}
if _, ok := seen[key]; ok {
continue
}
seen[key] = struct{}{}
merged = append(merged, item)
if trackAdded {
added = append(added, item)
}
}
}
appendUnique(existing, false)
appendUnique(additional, true)
return merged, added
}
func companyLinkURLs(links []CompanyLink) []string {
if len(links) == 0 {
return nil
}
urls := make([]string, 0, len(links))
for _, link := range links {
url := strings.TrimSpace(link.URL)
if url != "" {
urls = append(urls, url)
}
}
return urls
}
+70
View File
@@ -0,0 +1,70 @@
package company
import "testing"
func TestMergeCompanyLinks(t *testing.T) {
existing := []CompanyLink{
{URL: "https://example.com/a"},
{URL: "https://example.com/b"},
}
additional := []CompanyLink{
{URL: " https://example.com/b "},
{URL: "https://Example.com/C"},
{URL: " "},
}
merged, added := mergeCompanyLinks(existing, additional)
if len(merged) != 3 {
t.Fatalf("mergeCompanyLinks() merged len = %d, want 3", len(merged))
}
if len(added) != 1 {
t.Fatalf("mergeCompanyLinks() added len = %d, want 1", len(added))
}
if added[0].URL != "https://Example.com/C" {
t.Fatalf("added link = %+v, want https://Example.com/C", added[0])
}
}
func TestSanitizeCompanyLinks(t *testing.T) {
title := " Profile "
got, err := sanitizeCompanyLinks([]CompanyLink{
{URL: " https://example.com ", Title: &title},
{URL: "not-a-url"},
})
if err == nil {
t.Fatal("sanitizeCompanyLinks() expected invalid URL error")
}
got, err = sanitizeCompanyLinks([]CompanyLink{
{URL: "https://example.com", Title: &title},
})
if err != nil {
t.Fatalf("sanitizeCompanyLinks() unexpected error: %v", err)
}
if len(got) != 1 {
t.Fatalf("sanitizeCompanyLinks() len = %d, want 1", len(got))
}
if got[0].URL != "https://example.com" {
t.Fatalf("sanitized URL = %q, want https://example.com", got[0].URL)
}
if got[0].Title == nil || *got[0].Title != "Profile" {
t.Fatalf("sanitized title = %+v, want Profile", got[0].Title)
}
}
func TestCompanyLinkURLs(t *testing.T) {
got := companyLinkURLs([]CompanyLink{
{URL: " https://a.test "},
{URL: " "},
{URL: "https://b.test"},
})
want := []string{"https://a.test", "https://b.test"}
if len(got) != len(want) {
t.Fatalf("companyLinkURLs() len = %d, want %d", len(got), len(want))
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("companyLinkURLs()[%d] = %q, want %q", i, got[i], want[i])
}
}
}
+8
View File
@@ -0,0 +1,8 @@
package company
// RecommendedTendersPageCacheRefresher pre-builds resolved recommendation pages in Redis.
// Implemented by the tender service on the web API; optional on the worker.
type RecommendedTendersPageCacheRefresher interface {
ScheduleRefreshRecommendedTendersPageCache(companyID string)
InvalidateRecommendedTendersPageCache(companyID string)
}
+126 -7
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/redis/go-redis/v9"
"golang.org/x/sync/errgroup"
"tm/pkg/ai_summarizer"
)
@@ -24,6 +25,31 @@ const (
recommendationFetchInitialDelay = 10 * time.Second
)
func normalizeRecommendationCompanyIDs(companyIDs []string) []string {
normalized := make([]string, 0, len(companyIDs))
seen := make(map[string]struct{}, len(companyIDs))
for _, companyID := range companyIDs {
clean := strings.TrimSpace(companyID)
if clean == "" {
continue
}
if _, ok := seen[clean]; ok {
continue
}
seen[clean] = struct{}{}
normalized = append(normalized, clean)
}
return normalized
}
func mergeRecommendedTenders(recommendationLists ...[]RecommendedTenderResponse) []RecommendedTenderResponse {
merged := make([]RecommendedTenderResponse, 0)
for _, recommendations := range recommendationLists {
merged = append(merged, recommendations...)
}
return dedupeRecommendationResponsesByTenderID(merged)
}
func aiRecommendationCacheKey(companyID string) string {
return aiRecommendationCacheKeyPrefix + companyID
}
@@ -132,6 +158,7 @@ func (s *companyService) startAIOnboardingInternal(ctx context.Context, companyI
s.logger.Info("Starting AI company onboarding", map[string]interface{}{
"company_id": companyID,
"documents_count": len(company.DocumentFileIDs),
"links_count": len(company.Links),
"has_website_url": websiteURL != "",
})
@@ -139,6 +166,7 @@ func (s *companyService) startAIOnboardingInternal(ctx context.Context, companyI
CompanyID: companyID,
Documents: documents,
WebsiteURL: websiteURL,
Links: companyLinkURLs(company.Links),
})
if err != nil {
s.logger.Error("AI onboarding request failed", map[string]interface{}{
@@ -163,6 +191,10 @@ func (s *companyService) GetAIRecommendations(ctx context.Context, companyID str
return nil, errors.New("company ID is required")
}
if cached, ok := s.getCachedAIRecommendations(ctx, companyID); ok {
return cached, nil
}
if _, err := s.repository.GetByID(ctx, companyID); err != nil {
s.logger.Error("Failed to load company for AI recommendations", map[string]interface{}{
"company_id": companyID,
@@ -171,17 +203,74 @@ func (s *companyService) GetAIRecommendations(ctx context.Context, companyID str
return nil, errors.New("company not found")
}
if cached, ok := s.getCachedAIRecommendations(ctx, companyID); ok {
return cached, nil
}
s.logger.Info("AI recommendations cache miss; returning empty list", map[string]interface{}{
s.logger.Debug("AI recommendations cache miss; returning empty list", map[string]interface{}{
"company_id": companyID,
})
return []RecommendedTenderResponse{}, nil
}
func (s *companyService) validateRecommendationCompanyIDs(ctx context.Context, companyIDs []string) error {
companies, err := s.repository.GetByIDs(ctx, companyIDs)
if err != nil {
s.logger.Error("Failed to load companies for merged AI recommendations", map[string]interface{}{
"company_ids": companyIDs,
"error": err.Error(),
})
return errors.New("company not found")
}
if len(companies) != len(companyIDs) {
s.logger.Error("Some companies were not found for merged AI recommendations", map[string]interface{}{
"requested_count": len(companyIDs),
"loaded_count": len(companies),
})
return errors.New("company not found")
}
return nil
}
func (s *companyService) getCachedAIRecommendationsOrEmpty(ctx context.Context, companyID string) []RecommendedTenderResponse {
if cached, ok := s.getCachedAIRecommendations(ctx, companyID); ok {
return cached
}
s.logger.Debug("AI recommendations cache miss; returning empty list", map[string]interface{}{
"company_id": companyID,
})
return []RecommendedTenderResponse{}
}
// GetMergedAIRecommendations unions cached AI recommendations across multiple companies.
func (s *companyService) GetMergedAIRecommendations(ctx context.Context, companyIDs []string) ([]RecommendedTenderResponse, error) {
if s.aiRecommendationClient == nil {
return nil, ErrAIRecommendationNotConfigured
}
normalized := normalizeRecommendationCompanyIDs(companyIDs)
if len(normalized) == 0 {
return []RecommendedTenderResponse{}, nil
}
recommendationLists := make([][]RecommendedTenderResponse, len(normalized))
group, groupCtx := errgroup.WithContext(ctx)
group.Go(func() error {
return s.validateRecommendationCompanyIDs(groupCtx, normalized)
})
for i, companyID := range normalized {
i := i
companyID := companyID
group.Go(func() error {
recommendationLists[i] = s.getCachedAIRecommendationsOrEmpty(groupCtx, companyID)
return nil
})
}
if err := group.Wait(); err != nil {
return nil, err
}
return mergeRecommendedTenders(recommendationLists...), nil
}
func (s *companyService) scheduleRecommendationRefreshAfterOnboarding(companyID string) {
if s.aiRecommendationClient == nil {
return
@@ -279,6 +368,10 @@ func (s *companyService) fetchAIRecommendations(ctx context.Context, companyID s
return nil, fmt.Errorf("failed to fetch AI recommendations: %w", err)
}
return dedupeRecommendationResponsesByTenderID(itemsToRecommendationResponses(items)), nil
}
func itemsToRecommendationResponses(items []ai_summarizer.RecommendedTender) []RecommendedTenderResponse {
responses := make([]RecommendedTenderResponse, 0, len(items))
for _, item := range items {
responses = append(responses, RecommendedTenderResponse{
@@ -287,8 +380,26 @@ func (s *companyService) fetchAIRecommendations(ctx context.Context, companyID s
Analysis: item.Analysis,
})
}
return responses
}
return responses, nil
func dedupeRecommendationResponsesByTenderID(responses []RecommendedTenderResponse) []RecommendedTenderResponse {
seen := make(map[string]struct{}, len(responses))
out := make([]RecommendedTenderResponse, 0, len(responses))
for _, recommendation := range responses {
tenderID := strings.TrimSpace(recommendation.TenderID)
if tenderID == "" {
continue
}
if _, ok := seen[tenderID]; ok {
continue
}
seen[tenderID] = struct{}{}
recommendation.TenderID = tenderID
recommendation.Analysis = strings.TrimSpace(recommendation.Analysis)
out = append(out, recommendation)
}
return out
}
func (s *companyService) getCachedAIRecommendations(ctx context.Context, companyID string) ([]RecommendedTenderResponse, bool) {
@@ -296,7 +407,10 @@ func (s *companyService) getCachedAIRecommendations(ctx context.Context, company
return nil, false
}
raw, err := s.redisClient.Get(ctx, aiRecommendationCacheKey(companyID))
readCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
raw, err := s.redisClient.Get(readCtx, aiRecommendationCacheKey(companyID))
if err != nil {
if err != redis.Nil {
s.logger.Warn("Failed to read AI recommendation cache", map[string]interface{}{
@@ -350,7 +464,11 @@ func (s *companyService) cacheAIRecommendations(ctx context.Context, companyID s
"company_id": companyID,
"error": err.Error(),
})
return
}
s.invalidateRecommendedTendersPageCache(companyID)
s.scheduleRecommendedTendersPageCacheRefresh(companyID)
}
func (s *companyService) recommendationCacheExpiration() time.Duration {
@@ -371,6 +489,7 @@ func (s *companyService) invalidateAIRecommendationCache(ctx context.Context, co
"error": err.Error(),
})
}
s.invalidateRecommendedTendersPageCache(companyID)
}
func (s *companyService) buildOnboardingDocuments(fileIDs []string) (string, error) {
@@ -0,0 +1,20 @@
package company
func (s *companyService) invalidateRecommendedTendersPageCache(companyID string) {
if s.pageCacheRefresher == nil {
return
}
s.pageCacheRefresher.InvalidateRecommendedTendersPageCache(companyID)
}
func (s *companyService) scheduleRecommendedTendersPageCacheRefresh(companyID string) {
if s.pageCacheRefresher == nil {
return
}
s.pageCacheRefresher.ScheduleRefreshRecommendedTendersPageCache(companyID)
}
// SetRecommendedTendersPageCacheRefresher wires async page-cache rebuilds from the tender service.
func (s *companyService) SetRecommendedTendersPageCacheRefresher(refresher RecommendedTendersPageCacheRefresher) {
s.pageCacheRefresher = refresher
}
@@ -0,0 +1,221 @@
package company
import (
"context"
"strings"
"sync/atomic"
"time"
"golang.org/x/sync/errgroup"
"tm/pkg/ai_summarizer"
)
// AIPipelineStatusClient reports pipeline job status from the AI service.
type AIPipelineStatusClient interface {
GetPipelineLastAutoRun(ctx context.Context) (*ai_summarizer.PipelineReportResponse, error)
}
const (
recommendationPipelineWaitInitialDelay = 5 * time.Minute
recommendationPipelineWaitMaxAttempts = 24
recommendationRefreshConcurrency = 3
)
// ScheduleRefreshCachedAIRecommendationsAfterPipeline waits for the AI pipeline auto run to finish,
// then re-fetches ranked tenders for every company that already has a recommendation cache.
func (s *companyService) ScheduleRefreshCachedAIRecommendationsAfterPipeline() {
if s.aiRecommendationClient == nil {
return
}
go func() {
ctx := context.Background()
s.logger.Info("Scheduling AI recommendation cache refresh after pipeline sync", map[string]interface{}{})
if err := s.refreshCachedAIRecommendationsAfterPipeline(ctx); err != nil {
s.logger.Error("AI recommendation cache refresh after pipeline failed", map[string]interface{}{
"error": err.Error(),
})
return
}
s.logger.Info("AI recommendation cache refresh after pipeline completed", map[string]interface{}{})
}()
}
func (s *companyService) refreshCachedAIRecommendationsAfterPipeline(ctx context.Context) error {
if err := s.waitForPipelineAutoCompletion(ctx); err != nil {
return err
}
companyIDs, err := s.listCompanyIDsWithRecommendationCache(ctx)
if err != nil {
return err
}
if len(companyIDs) == 0 {
s.logger.Info("No companies with recommendation cache to refresh after pipeline", map[string]interface{}{})
return nil
}
s.logger.Info("Refreshing AI recommendation caches after pipeline", map[string]interface{}{
"company_count": len(companyIDs),
})
group, groupCtx := errgroup.WithContext(ctx)
group.SetLimit(recommendationRefreshConcurrency)
var refreshed atomic.Int32
var failed atomic.Int32
for _, companyID := range companyIDs {
companyID := companyID
group.Go(func() error {
if err := s.refreshAIRecommendationsCache(groupCtx, companyID); err != nil {
s.logger.Error("Failed to refresh AI recommendation cache for company", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
failed.Add(1)
return nil
}
refreshed.Add(1)
return nil
})
}
if err := group.Wait(); err != nil {
return err
}
s.logger.Info("AI recommendation cache refresh summary", map[string]interface{}{
"company_count": len(companyIDs),
"refreshed": refreshed.Load(),
"failed": failed.Load(),
})
return nil
}
func (s *companyService) waitForPipelineAutoCompletion(ctx context.Context) error {
if s.aiPipelineStatusClient == nil {
s.logger.Info("AI pipeline status client not configured; waiting before recommendation refresh", map[string]interface{}{
"delay_sec": int(recommendationPipelineWaitInitialDelay.Seconds()),
})
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(recommendationPipelineWaitInitialDelay):
}
return nil
}
delay := recommendationPipelineWaitInitialDelay
for attempt := 1; attempt <= recommendationPipelineWaitMaxAttempts; attempt++ {
if attempt == 1 {
s.logger.Info("Waiting before first AI pipeline status check", map[string]interface{}{
"delay_sec": int(delay.Seconds()),
})
} else {
s.logger.Info("Retrying AI pipeline status check", map[string]interface{}{
"attempt": attempt,
"delay_sec": int(delay.Seconds()),
})
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(delay):
}
if attempt > 1 {
delay *= 2
if delay > 30*time.Minute {
delay = 30 * time.Minute
}
}
report, err := s.aiPipelineStatusClient.GetPipelineLastAutoRun(ctx)
if err != nil {
s.logger.Warn("Failed to read AI pipeline last-auto-run status", map[string]interface{}{
"attempt": attempt,
"error": err.Error(),
})
if attempt == recommendationPipelineWaitMaxAttempts {
s.logger.Warn("Proceeding with recommendation refresh without pipeline status", map[string]interface{}{})
return nil
}
continue
}
if !isPipelineRunInProgress(report) {
s.logger.Info("AI pipeline auto run finished; refreshing recommendation caches", map[string]interface{}{
"attempt": attempt,
"status": strings.TrimSpace(report.Status),
})
return nil
}
if attempt == recommendationPipelineWaitMaxAttempts {
s.logger.Warn("AI pipeline still in progress after max wait; refreshing recommendation caches anyway", map[string]interface{}{
"attempts": attempt,
"status": strings.TrimSpace(report.Status),
})
return nil
}
}
return nil
}
func isPipelineRunInProgress(report *ai_summarizer.PipelineReportResponse) bool {
if report == nil {
return true
}
status := strings.ToLower(strings.TrimSpace(report.Status))
switch status {
case "", "running", "in_progress", "started", "processing", "pending", "queued":
return true
default:
return false
}
}
func (s *companyService) listCompanyIDsWithRecommendationCache(ctx context.Context) ([]string, error) {
allIDs, err := s.repository.ListIDs(ctx)
if err != nil {
s.logger.Error("Failed to list companies for recommendation refresh", map[string]interface{}{
"error": err.Error(),
})
return nil, err
}
cached := make([]string, 0, len(allIDs))
for _, companyID := range allIDs {
if _, ok := s.getCachedAIRecommendations(ctx, companyID); ok {
cached = append(cached, companyID)
}
}
return cached, nil
}
// refreshAIRecommendationsCache updates Redis when the AI service returns a non-empty ranked list.
// Empty or failed fetches leave the existing cache untouched.
func (s *companyService) refreshAIRecommendationsCache(ctx context.Context, companyID string) error {
responses, err := s.fetchAIRecommendations(ctx, companyID)
if err != nil {
return err
}
if len(responses) == 0 {
s.logger.Info("Skipping recommendation cache update after pipeline: AI returned empty list", map[string]interface{}{
"company_id": companyID,
})
return nil
}
s.cacheAIRecommendations(ctx, companyID, responses)
s.logger.Info("AI recommendation cache refreshed after pipeline", map[string]interface{}{
"company_id": companyID,
"count": len(responses),
})
return nil
}
@@ -0,0 +1,30 @@
package company
import (
"testing"
"tm/pkg/ai_summarizer"
)
func TestIsPipelineRunInProgress(t *testing.T) {
tests := []struct {
name string
report *ai_summarizer.PipelineReportResponse
want bool
}{
{name: "nil report", report: nil, want: true},
{name: "empty status", report: &ai_summarizer.PipelineReportResponse{}, want: true},
{name: "running", report: &ai_summarizer.PipelineReportResponse{Status: "running"}, want: true},
{name: "in progress", report: &ai_summarizer.PipelineReportResponse{Status: "in_progress"}, want: true},
{name: "completed", report: &ai_summarizer.PipelineReportResponse{Status: "completed"}, want: false},
{name: "failed", report: &ai_summarizer.PipelineReportResponse{Status: "failed"}, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isPipelineRunInProgress(tt.report); got != tt.want {
t.Fatalf("isPipelineRunInProgress() = %v, want %v", got, tt.want)
}
})
}
}
+64
View File
@@ -0,0 +1,64 @@
package company
import "testing"
func TestDedupeRecommendationResponsesByTenderID(t *testing.T) {
input := []RecommendedTenderResponse{
{Rank: 1, TenderID: " PROC-1 ", Analysis: " first "},
{Rank: 2, TenderID: "PROC-2", Analysis: "second"},
{Rank: 1, TenderID: "PROC-2", Analysis: "duplicate"},
{Rank: 4, TenderID: " ", Analysis: "ignored"},
}
got := dedupeRecommendationResponsesByTenderID(input)
if len(got) != 2 {
t.Fatalf("dedupeRecommendationResponsesByTenderID() len = %d, want 2", len(got))
}
if got[0].TenderID != "PROC-1" || got[0].Analysis != "first" {
t.Fatalf("first recommendation = %+v, want trimmed PROC-1/first", got[0])
}
if got[1].TenderID != "PROC-2" || got[1].Rank != 2 {
t.Fatalf("second recommendation = %+v, want original PROC-2 rank 2", got[1])
}
}
func TestMergeRecommendedTenders(t *testing.T) {
first := []RecommendedTenderResponse{
{Rank: 1, TenderID: " PROC-1 ", Analysis: " first "},
{Rank: 2, TenderID: "PROC-2", Analysis: "second"},
}
second := []RecommendedTenderResponse{
{Rank: 1, TenderID: "PROC-2", Analysis: "duplicate"},
{Rank: 3, TenderID: "PROC-3", Analysis: "third"},
{Rank: 4, TenderID: " ", Analysis: "ignored"},
}
got := mergeRecommendedTenders(first, second)
if len(got) != 3 {
t.Fatalf("mergeRecommendedTenders() len = %d, want 3", len(got))
}
if got[0].TenderID != "PROC-1" || got[0].Analysis != "first" {
t.Fatalf("first recommendation = %+v, want trimmed PROC-1/first", got[0])
}
if got[1].TenderID != "PROC-2" || got[1].Rank != 2 {
t.Fatalf("second recommendation = %+v, want original PROC-2 rank 2", got[1])
}
if got[2].TenderID != "PROC-3" {
t.Fatalf("third recommendation = %+v, want PROC-3", got[2])
}
}
func TestNormalizeRecommendationCompanyIDs(t *testing.T) {
got := normalizeRecommendationCompanyIDs([]string{" company-a ", "", "company-b", "company-a"})
want := []string{"company-a", "company-b"}
if len(got) != len(want) {
t.Fatalf("normalizeRecommendationCompanyIDs() len = %d, want %d", len(got), len(want))
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("normalizeRecommendationCompanyIDs()[%d] = %q, want %q", i, got[i], want[i])
}
}
}
+59 -7
View File
@@ -24,6 +24,7 @@ type Repository interface {
GetByRegistrationNumber(ctx context.Context, registrationNumber string) (*Company, error)
GetByTaxID(ctx context.Context, taxID string) (*Company, error)
Search(ctx context.Context, form *SearchForm, pagination *response.Pagination) (*orm.PaginatedResult[Company], error)
ListIDs(ctx context.Context) ([]string, error)
Delete(ctx context.Context, id string) error
UpdateStatus(ctx context.Context, id string, status CompanyStatus) error
UpdateVerification(ctx context.Context, id string, isVerified, isCompliant bool, complianceNotes *string) error
@@ -170,6 +171,7 @@ func (r *companyRepository) Update(ctx context.Context, company *Company) error
company.SetUpdatedAt(time.Now().Unix())
clearDocuments := company.DocumentFileIDs != nil && len(company.DocumentFileIDs) == 0
clearLinks := company.Links != nil && len(company.Links) == 0
// Use ORM to update company
err := r.ormRepo.Update(ctx, company)
@@ -181,16 +183,23 @@ func (r *companyRepository) Update(ctx context.Context, company *Company) error
return err
}
// BSON omitempty can skip empty slices in $set, so persist an explicit empty array.
if clearDocuments {
_, err = r.collection.UpdateOne(ctx, bson.M{"_id": company.ID}, bson.M{
"$set": bson.M{
"document_file_ids": []string{},
// BSON omitempty can skip empty slices in $set, so persist explicit empty arrays.
if clearDocuments || clearLinks {
setFields := bson.M{
"updated_at": company.UpdatedAt,
},
}
if clearDocuments {
setFields["document_file_ids"] = []string{}
}
if clearLinks {
setFields["links"] = []CompanyLink{}
}
_, err = r.collection.UpdateOne(ctx, bson.M{"_id": company.ID}, bson.M{
"$set": setFields,
})
if err != nil {
r.logger.Error("Failed to clear company document file IDs", map[string]interface{}{
r.logger.Error("Failed to clear company collection fields", map[string]interface{}{
"error": err.Error(),
"company_id": company.ID,
})
@@ -375,6 +384,49 @@ func (r *companyRepository) Search(ctx context.Context, form *SearchForm, pagina
return result, nil
}
// ListIDs returns every company document ID.
func (r *companyRepository) ListIDs(ctx context.Context) ([]string, error) {
const pageSize = 500
ids := make([]string, 0)
skip := 0
for {
result, err := r.ormRepo.FindAll(ctx, bson.M{}, orm.Pagination{
Limit: pageSize,
Skip: skip,
SortField: "_id",
SortOrder: 1,
SkipCount: true,
Projection: bson.M{"_id": 1},
})
if err != nil {
r.logger.Error("Failed to list company IDs", map[string]interface{}{
"error": err.Error(),
"skip": skip,
})
return nil, err
}
if len(result.Items) == 0 {
break
}
for i := range result.Items {
id := strings.TrimSpace(result.Items[i].GetID())
if id != "" {
ids = append(ids, id)
}
}
if len(result.Items) < pageSize {
break
}
skip += pageSize
}
return ids, nil
}
// UpdateStatus updates company status
func (r *companyRepository) UpdateStatus(ctx context.Context, id string, status CompanyStatus) error {
// Get company first
+32 -1
View File
@@ -36,6 +36,9 @@ type Service interface {
// UploadDocuments uploads files and attaches them to a company
UploadDocuments(ctx context.Context, companyID, uploadedBy string, files []*multipart.FileHeader) (*UploadDocumentsResponse, error)
// AddLinks appends external URLs to a company
AddLinks(ctx context.Context, companyID string, form *AddLinksForm) (*AddLinksResponse, error)
// DetachDocumentFileID removes a file ID from companies when the file is deleted
DetachDocumentFileID(ctx context.Context, fileID string) error
@@ -44,6 +47,10 @@ type Service interface {
// GetAIRecommendations returns ranked tender recommendations from the AI team
GetAIRecommendations(ctx context.Context, companyID string) ([]RecommendedTenderResponse, error)
GetMergedAIRecommendations(ctx context.Context, companyIDs []string) ([]RecommendedTenderResponse, error)
// ScheduleRefreshCachedAIRecommendationsAfterPipeline re-fetches cached company recommendations after tender sync.
ScheduleRefreshCachedAIRecommendationsAfterPipeline()
}
// companyService implements the Service interface
@@ -52,6 +59,8 @@ type companyService struct {
categoryService company_category.Service
fileStore filestore.FileStoreService
aiRecommendationClient AIRecommendationClient
aiPipelineStatusClient AIPipelineStatusClient
pageCacheRefresher RecommendedTendersPageCacheRefresher
redisClient redis.Client
recommendationCacheTTL time.Duration
logger logger.Logger
@@ -63,6 +72,7 @@ func NewService(
categoryService company_category.Service,
fileStore filestore.FileStoreService,
aiRecommendationClient AIRecommendationClient,
aiPipelineStatusClient AIPipelineStatusClient,
redisClient redis.Client,
recommendationCacheTTL time.Duration,
logger logger.Logger,
@@ -72,6 +82,7 @@ func NewService(
categoryService: categoryService,
fileStore: fileStore,
aiRecommendationClient: aiRecommendationClient,
aiPipelineStatusClient: aiPipelineStatusClient,
redisClient: redisClient,
recommendationCacheTTL: recommendationCacheTTL,
logger: logger,
@@ -117,6 +128,14 @@ func (s *companyService) Create(ctx context.Context, form *CompanyForm) (*Compan
}
}
links, err := sanitizeCompanyLinks(convertCompanyLinkForms(form.Links))
if err != nil {
return nil, err
}
if len(links) > maxCompanyLinks {
return nil, errors.New("too many links provided")
}
// Create company
company := &Company{
Name: form.Name,
@@ -134,6 +153,7 @@ func (s *companyService) Create(ctx context.Context, form *CompanyForm) (*Compan
Phone: form.Phone,
Email: form.Email,
DocumentFileIDs: s.sanitizeDocumentFileIDs(form.DocumentFileIDs),
Links: links,
Tags: s.convertCompanyTagsForm(form.Tags),
IsVerified: false,
IsCompliant: false,
@@ -143,7 +163,7 @@ func (s *companyService) Create(ctx context.Context, form *CompanyForm) (*Compan
}
// Save to database
err := s.repository.Create(ctx, company)
err = s.repository.Create(ctx, company)
if err != nil {
s.logger.Error("Failed to create company", map[string]interface{}{
"error": err.Error(),
@@ -303,6 +323,17 @@ func (s *companyService) Update(ctx context.Context, id string, form *CompanyFor
company.DocumentFileIDs = s.sanitizeDocumentFileIDs(form.DocumentFileIDs)
}
if form.Links != nil {
links, linkErr := sanitizeCompanyLinks(convertCompanyLinkForms(form.Links))
if linkErr != nil {
return nil, linkErr
}
if len(links) > maxCompanyLinks {
return nil, errors.New("too many links provided")
}
company.Links = links
}
if form.Tags != nil {
company.Tags = s.convertCompanyTagsForm(form.Tags)
}
+61 -7
View File
@@ -6,12 +6,33 @@ import (
"strings"
)
var errCompanyNotAssigned = errors.New("company is not assigned to customer")
// ErrCompanyNotAssigned is returned when the requested company is not assigned to the customer.
var ErrCompanyNotAssigned = errors.New("company is not assigned to customer")
var errCompanyNotAssigned = ErrCompanyNotAssigned
func normalizeAssignedCompanyIDs(assigned []string) []string {
normalized := make([]string, 0, len(assigned))
seen := make(map[string]struct{}, len(assigned))
for _, id := range assigned {
clean := strings.TrimSpace(id)
if clean == "" {
continue
}
if _, ok := seen[clean]; ok {
continue
}
seen[clean] = struct{}{}
normalized = append(normalized, clean)
}
return normalized
}
// pickActiveCompanyID chooses the company context for a customer request.
// When requestedCompanyID is set it must be in assigned; otherwise a still-valid
// token company is kept; if the token company was removed, the first assignment is used.
func pickActiveCompanyID(assigned []string, tokenCompanyID, requestedCompanyID string) (string, error) {
assigned = normalizeAssignedCompanyIDs(assigned)
if len(assigned) == 0 {
return "", nil
}
@@ -38,12 +59,45 @@ func pickActiveCompanyID(assigned []string, tokenCompanyID, requestedCompanyID s
return assigned[0], nil
}
// ResolveActiveCompanyID resolves the active company for an authenticated customer.
func (s *customerService) ResolveActiveCompanyID(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, error) {
customer, err := s.repository.GetByID(ctx, customerID)
if err != nil {
return "", errors.New("customer not found")
// PickAssignedCompanyID chooses the company for a company-scoped write (e.g. tender approval).
// When requestedCompanyID is set it must be in assigned; otherwise activeCompanyID is used.
func PickAssignedCompanyID(activeCompanyID string, assigned []string, requestedCompanyID string) (string, error) {
activeCompanyID = strings.TrimSpace(activeCompanyID)
requestedCompanyID = strings.TrimSpace(requestedCompanyID)
if requestedCompanyID != "" {
for _, id := range normalizeAssignedCompanyIDs(assigned) {
if id == requestedCompanyID {
return requestedCompanyID, nil
}
}
return "", ErrCompanyNotAssigned
}
return pickActiveCompanyID(customer.Companies, tokenCompanyID, requestedCompanyID)
if activeCompanyID == "" {
return "", errors.New("company ID is required")
}
return activeCompanyID, nil
}
// ResolveCompanyContext resolves the active company and assigned companies for an authenticated customer.
func (s *customerService) ResolveCompanyContext(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, []string, error) {
customer, err := s.repository.GetByID(ctx, customerID)
if err != nil {
return "", nil, errors.New("customer not found")
}
assigned := normalizeAssignedCompanyIDs(customer.Companies)
companyID, err := pickActiveCompanyID(assigned, tokenCompanyID, requestedCompanyID)
if err != nil {
return "", nil, err
}
return companyID, assigned, nil
}
// ResolveActiveCompanyID resolves the active company for an authenticated customer.
func (s *customerService) ResolveActiveCompanyID(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, error) {
companyID, _, err := s.ResolveCompanyContext(ctx, customerID, tokenCompanyID, requestedCompanyID)
return companyID, err
}
+152
View File
@@ -0,0 +1,152 @@
package customer
import (
"encoding/json"
"fmt"
"strings"
)
type companySelectionObject struct {
ID string `json:"id"`
}
// decodeCompanySelection parses company IDs sent as strings, legacy company_ids,
// or company summary objects returned by GET customer.
func decodeCompanySelection(raw json.RawMessage) ([]string, error) {
if len(raw) == 0 {
return nil, fmt.Errorf("empty company selection")
}
if string(raw) == "null" {
return []string{}, nil
}
var ids []string
if err := json.Unmarshal(raw, &ids); err == nil {
return ids, nil
}
var objects []companySelectionObject
if err := json.Unmarshal(raw, &objects); err == nil {
result := make([]string, 0, len(objects))
for _, object := range objects {
if id := strings.TrimSpace(object.ID); id != "" {
result = append(result, id)
}
}
return result, nil
}
return nil, fmt.Errorf("companies must be an array of IDs or objects with id")
}
func mergeCompanySelectionFields(raw map[string]json.RawMessage) (json.RawMessage, bool, error) {
if value, ok := raw["companies"]; ok {
return value, true, nil
}
if value, ok := raw["company_ids"]; ok {
return value, true, nil
}
return nil, false, nil
}
func unmarshalFormWithoutCompanyFields(data []byte, raw map[string]json.RawMessage, target any) error {
delete(raw, "companies")
delete(raw, "company_ids")
trimmed, err := json.Marshal(raw)
if err != nil {
return err
}
return json.Unmarshal(trimmed, target)
}
// UnmarshalJSON accepts companies, legacy company_ids, and company summary objects.
func (f *CreateCustomerForm) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
selectionRaw, present, err := mergeCompanySelectionFields(raw)
if err != nil {
return err
}
type alias CreateCustomerForm
if err := unmarshalFormWithoutCompanyFields(data, raw, (*alias)(f)); err != nil {
return err
}
if !present {
return nil
}
companies, err := decodeCompanySelection(selectionRaw)
if err != nil {
return err
}
f.Companies = companies
return nil
}
// UnmarshalJSON accepts companies, legacy company_ids, and company summary objects.
func (f *UpdateCustomerForm) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
selectionRaw, present, err := mergeCompanySelectionFields(raw)
if err != nil {
return err
}
type alias UpdateCustomerForm
if err := unmarshalFormWithoutCompanyFields(data, raw, (*alias)(f)); err != nil {
return err
}
if !present {
return nil
}
companies, err := decodeCompanySelection(selectionRaw)
if err != nil {
return err
}
f.Companies = &companies
return nil
}
// UnmarshalJSON accepts companies, legacy company_ids, and company summary objects.
func (f *AssignCompaniesForm) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
selectionRaw, present, err := mergeCompanySelectionFields(raw)
if err != nil {
return err
}
type alias AssignCompaniesForm
if err := unmarshalFormWithoutCompanyFields(data, raw, (*alias)(f)); err != nil {
return err
}
if !present {
return nil
}
companies, err := decodeCompanySelection(selectionRaw)
if err != nil {
return err
}
f.Companies = companies
return nil
}
+92
View File
@@ -0,0 +1,92 @@
package customer
import (
"encoding/json"
"testing"
)
func TestCreateCustomerForm_UnmarshalJSON_CompanyIDs(t *testing.T) {
payload := `{
"type": "individual",
"role": "analyst",
"username": "testuser",
"email": "test@example.com",
"password": "Test!1234",
"company_ids": ["507f1f77bcf86cd799439011", "507f1f77bcf86cd799439012"]
}`
var form CreateCustomerForm
if err := json.Unmarshal([]byte(payload), &form); err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := []string{"507f1f77bcf86cd799439011", "507f1f77bcf86cd799439012"}
if len(form.Companies) != len(want) {
t.Fatalf("Companies len = %d, want %d", len(form.Companies), len(want))
}
for i, id := range want {
if form.Companies[i] != id {
t.Fatalf("Companies[%d] = %q, want %q", i, form.Companies[i], id)
}
}
}
func TestUpdateCustomerForm_UnmarshalJSON_CompanySummaries(t *testing.T) {
payload := `{
"type": "individual",
"role": "analyst",
"username": "testuser",
"email": "test@example.com",
"companies": [
{"id": "507f1f77bcf86cd799439011", "name": "Acme"},
{"id": "507f1f77bcf86cd799439012", "name": "Globex"}
]
}`
var form UpdateCustomerForm
if err := json.Unmarshal([]byte(payload), &form); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if form.Companies == nil {
t.Fatal("Companies should be set")
}
want := []string{"507f1f77bcf86cd799439011", "507f1f77bcf86cd799439012"}
if len(*form.Companies) != len(want) {
t.Fatalf("Companies len = %d, want %d", len(*form.Companies), len(want))
}
for i, id := range want {
if (*form.Companies)[i] != id {
t.Fatalf("Companies[%d] = %q, want %q", i, (*form.Companies)[i], id)
}
}
}
func TestUpdateCustomerForm_UnmarshalJSON_OmittedCompanies(t *testing.T) {
payload := `{
"type": "individual",
"role": "analyst",
"username": "testuser",
"email": "test@example.com"
}`
var form UpdateCustomerForm
if err := json.Unmarshal([]byte(payload), &form); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if form.Companies != nil {
t.Fatalf("Companies = %v, want nil when omitted", form.Companies)
}
}
func TestAssignCompaniesForm_UnmarshalJSON_LegacyField(t *testing.T) {
payload := `{"company_ids": ["507f1f77bcf86cd799439011"]}`
var form AssignCompaniesForm
if err := json.Unmarshal([]byte(payload), &form); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(form.Companies) != 1 || form.Companies[0] != "507f1f77bcf86cd799439011" {
t.Fatalf("Companies = %v", form.Companies)
}
}
+2 -1
View File
@@ -76,7 +76,7 @@ func (h *Handler) CompanyContextMiddleware() echo.MiddlewareFunc {
tokenCompanyID, _ := c.Get("company_id").(string)
requestedCompanyID := strings.TrimSpace(c.QueryParam("company_id"))
companyID, err := h.service.ResolveActiveCompanyID(
companyID, companyIDs, err := h.service.ResolveCompanyContext(
c.Request().Context(),
customerID,
tokenCompanyID,
@@ -90,6 +90,7 @@ func (h *Handler) CompanyContextMiddleware() echo.MiddlewareFunc {
}
c.Set("company_id", companyID)
c.Set("company_ids", companyIDs)
return next(c)
}
}
+5 -10
View File
@@ -249,16 +249,11 @@ func (r *customerRepository) GetByRole(ctx context.Context, role CustomerRole) (
// GetByCompanies retrieves customers by companies
func (r *customerRepository) GetByCompanies(ctx context.Context, companies []string) ([]Customer, error) {
companyIDs := make([]bson.ObjectID, 0)
for _, company := range companies {
id, err := bson.ObjectIDFromHex(company)
if err != nil {
return nil, err
}
companyIDs = append(companyIDs, id)
if len(companies) == 0 {
return []Customer{}, nil
}
customers, err := r.ormRepo.FindAll(ctx, bson.M{"companies": bson.M{"$in": companyIDs}}, orm.NewPaginationBuilder().Build())
customers, err := r.ormRepo.FindAll(ctx, bson.M{"companies": bson.M{"$in": companies}}, orm.NewPaginationBuilder().Build())
if err != nil {
return nil, err
}
@@ -274,9 +269,9 @@ func (r *customerRepository) GetByCompanyID(ctx context.Context, companyID strin
SortBy(pagination.SortBy, pagination.SortOrder).
Build()
// Filter by company ID and active status
// Filter by assigned company ID and active status
filter := bson.M{
"company_id": companyID,
"companies": companyID,
"status": bson.M{"$ne": CustomerStatusInactive},
}
+4
View File
@@ -54,6 +54,10 @@ type Service interface {
AssignRole(ctx context.Context, customerID string, form *AssignRoleForm) (*AssignRoleResponse, error)
AssignCompanies(ctx context.Context, customerID string, Companies []string) error
// ResolveCompanyContext returns the active company and all assigned companies
// for an authenticated customer request.
ResolveCompanyContext(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, []string, error)
// ResolveActiveCompanyID returns the company context for an authenticated customer.
ResolveActiveCompanyID(ctx context.Context, customerID, tokenCompanyID, requestedCompanyID string) (string, error)
}
+46
View File
@@ -27,6 +27,11 @@ type scrapedDocumentsScanner interface {
ScanScrapedDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, map[string]int64, error)
}
// translatedNoticesScanner optionally scans MinIO for per-day translated notice counts.
type translatedNoticesScanner interface {
ScanTranslatedNotices(ctx context.Context) (map[string]int64, int64, error)
}
const defaultValueCurrency = "EUR"
// Repository defines dashboard data access.
@@ -51,6 +56,11 @@ type repository struct {
scrapedScopeExpiry time.Time
scrapedScopeCached bool
scrapedScopeGroup singleflight.Group
translatedScopeMu sync.Mutex
translatedScope translatedNoticesScope
translatedScopeExpiry time.Time
translatedScopeCached bool
translatedScopeGroup singleflight.Group
}
// NewRepository creates a dashboard repository backed by the tenders collection.
@@ -431,6 +441,22 @@ func (r *repository) ClosingSoon(ctx context.Context, limit int, windowSec int64
windowEnd := now + windowSec
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{
"$or": bson.A{
bson.M{
"$and": bson.A{
bson.M{"submission_deadline": bson.M{"$gt": 0}},
deadlineWithinWindowMatch("submission_deadline", now, windowEnd),
},
},
bson.M{
"$and": bson.A{
noSubmissionDeadlineClause(),
deadlineWithinWindowMatch("tender_deadline", now, windowEnd),
},
},
},
}}},
{{Key: "$addFields", Value: bson.M{"effective_deadline": effectiveDeadlineExpr()}}},
{{Key: "$match", Value: bson.M{
"effective_deadline": bson.M{"$gt": now, "$lte": windowEnd},
@@ -570,6 +596,26 @@ func trendTimestampField(metric string) string {
}
}
// deadlineWithinWindowMatch matches raw deadline fields stored as Unix seconds or
// milliseconds. normalizeTimestampExpr treats values > 1e12 as ms; indexed pre-filters
// must accept both encodings to stay equivalent to filtering on normalized deadlines.
func deadlineWithinWindowMatch(field string, now, windowEnd int64) bson.M {
return bson.M{
"$or": bson.A{
bson.M{field: bson.M{"$gt": now, "$lte": windowEnd}},
bson.M{field: bson.M{"$gt": now * 1000, "$lte": windowEnd * 1000}},
},
}
}
func noSubmissionDeadlineClause() bson.M {
return bson.M{"$or": bson.A{
bson.M{"submission_deadline": bson.M{"$exists": false}},
bson.M{"submission_deadline": nil},
bson.M{"submission_deadline": bson.M{"$lte": 0}},
}}
}
// effectiveDeadlineExpr is used for closing-soon (submission first, per dashboard-api.md).
func effectiveDeadlineExpr() bson.M {
return normalizeDeadlineFieldExpr(
+27
View File
@@ -42,3 +42,30 @@ func TestFacetCurrencyValueDecodesBSOND(t *testing.T) {
t.Fatalf("expected 12345.67, got %f", total)
}
}
func TestDeadlineWithinWindowMatchCoversSecondsAndMilliseconds(t *testing.T) {
now := int64(1_700_000_000)
windowEnd := now + 86_400
match := deadlineWithinWindowMatch("submission_deadline", now, windowEnd)
orClause, ok := match["$or"].(bson.A)
if !ok || len(orClause) != 2 {
t.Fatalf("expected two range branches, got %#v", match)
}
secRange, ok := orClause[0].(bson.M)["submission_deadline"].(bson.M)
if !ok {
t.Fatalf("expected seconds range on submission_deadline, got %#v", orClause[0])
}
if secRange["$gt"] != now || secRange["$lte"] != windowEnd {
t.Fatalf("unexpected seconds range: %#v", secRange)
}
msRange, ok := orClause[1].(bson.M)["submission_deadline"].(bson.M)
if !ok {
t.Fatalf("expected milliseconds range on submission_deadline, got %#v", orClause[1])
}
if msRange["$gt"] != now*1000 || msRange["$lte"] != windowEnd*1000 {
t.Fatalf("unexpected milliseconds range: %#v", msRange)
}
}
+126 -9
View File
@@ -60,6 +60,11 @@ type service struct {
statisticsMu sync.Mutex
statistics map[int]cacheEntry[*StatisticsReportResponse]
statisticsGroup singleflight.Group
trendCache *widgetCache[*TrendResponse]
countriesCache *widgetCache[*CountriesResponse]
noticeTypesCache *widgetCache[*NoticeTypesResponse]
closingSoonCache *widgetCache[*ClosingSoonResponse]
recentCache *widgetCache[*RecentResponse]
}
// NewService creates a dashboard service.
@@ -70,9 +75,15 @@ func NewService(repo Repository, log logger.Logger, redisClient redis.Client) Se
redis: redisClient,
summary: make(map[string]cacheEntry[*SummaryResponse]),
statistics: make(map[int]cacheEntry[*StatisticsReportResponse]),
trendCache: newWidgetCache[*TrendResponse](redisClient, log, "trend"),
countriesCache: newWidgetCache[*CountriesResponse](redisClient, log, "countries"),
noticeTypesCache: newWidgetCache[*NoticeTypesResponse](redisClient, log, "notice-types"),
closingSoonCache: newWidgetCache[*ClosingSoonResponse](redisClient, log, "closing-soon"),
recentCache: newWidgetCache[*RecentResponse](redisClient, log, "recent"),
}
go s.warmStatisticsCache()
go s.warmSummaryCache()
go s.warmWidgetCaches()
return s
}
@@ -129,6 +140,9 @@ func (s *service) Summary(ctx context.Context, query SummaryQuery) (*SummaryResp
func (s *service) Trend(ctx context.Context, query TrendQuery) (*TrendResponse, error) {
days := trendDays(query.Days)
metric := normalizeTrendMetric(query.Metric)
cacheKey := fmt.Sprintf("%d:%s", days, metric)
return s.trendCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*TrendResponse, error) {
startUnix := trendStartUnix(days)
s.logger.Info("Fetching dashboard trend", map[string]interface{}{
@@ -136,7 +150,7 @@ func (s *service) Trend(ctx context.Context, query TrendQuery) (*TrendResponse,
"metric": metric,
})
counts, err := s.repo.Trend(ctx, days, metric, startUnix)
counts, err := s.repo.Trend(loadCtx, days, metric, startUnix)
if err != nil {
s.logger.Error("Failed to fetch dashboard trend", map[string]interface{}{
"error": err.Error(),
@@ -144,23 +158,24 @@ func (s *service) Trend(ctx context.Context, query TrendQuery) (*TrendResponse,
return nil, fmt.Errorf("dashboard trend: %w", err)
}
series := fillTrendSeries(days, counts)
return &TrendResponse{
Metric: metric,
Days: days,
Series: series,
Series: fillTrendSeries(days, counts),
}, nil
})
}
func (s *service) Countries(ctx context.Context, query CountriesQuery) (*CountriesResponse, error) {
limit := countriesLimit(query.Limit)
cacheKey := strconv.Itoa(limit)
return s.countriesCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*CountriesResponse, error) {
s.logger.Info("Fetching dashboard countries", map[string]interface{}{
"limit": limit,
})
out, err := s.repo.Countries(ctx, limit)
out, err := s.repo.Countries(loadCtx, limit)
if err != nil {
s.logger.Error("Failed to fetch dashboard countries", map[string]interface{}{
"error": err.Error(),
@@ -169,12 +184,14 @@ func (s *service) Countries(ctx context.Context, query CountriesQuery) (*Countri
}
return out, nil
})
}
func (s *service) NoticeTypes(ctx context.Context) (*NoticeTypesResponse, error) {
return s.noticeTypesCache.Get(ctx, "all", func(loadCtx context.Context) (*NoticeTypesResponse, error) {
s.logger.Info("Fetching dashboard notice types", nil)
out, err := s.repo.NoticeTypes(ctx)
out, err := s.repo.NoticeTypes(loadCtx)
if err != nil {
s.logger.Error("Failed to fetch dashboard notice types", map[string]interface{}{
"error": err.Error(),
@@ -183,19 +200,22 @@ func (s *service) NoticeTypes(ctx context.Context) (*NoticeTypesResponse, error)
}
return out, nil
})
}
func (s *service) ClosingSoon(ctx context.Context, query ClosingSoonQuery) (*ClosingSoonResponse, error) {
limit := listLimit(query.Limit)
windowHours := closingWindowHours(query.Window)
windowSec := int64(windowHours) * 3600
cacheKey := fmt.Sprintf("%d:%d", limit, windowSec)
return s.closingSoonCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*ClosingSoonResponse, error) {
s.logger.Info("Fetching dashboard closing soon", map[string]interface{}{
"limit": limit,
"window": windowHours,
})
items, err := s.repo.ClosingSoon(ctx, limit, windowSec)
items, err := s.repo.ClosingSoon(loadCtx, limit, windowSec)
if err != nil {
s.logger.Error("Failed to fetch dashboard closing soon", map[string]interface{}{
"error": err.Error(),
@@ -204,16 +224,18 @@ func (s *service) ClosingSoon(ctx context.Context, query ClosingSoonQuery) (*Clo
}
return &ClosingSoonResponse{Items: items}, nil
})
}
func (s *service) Recent(ctx context.Context, query RecentQuery) (*RecentResponse, error) {
limit := listLimit(query.Limit)
cursor := strings.TrimSpace(query.Cursor)
if cursor != "" {
s.logger.Info("Fetching dashboard recent tenders", map[string]interface{}{
"limit": limit,
})
items, nextCursor, err := s.repo.Recent(ctx, limit, strings.TrimSpace(query.Cursor))
items, nextCursor, err := s.repo.Recent(ctx, limit, cursor)
if err != nil {
s.logger.Error("Failed to fetch dashboard recent tenders", map[string]interface{}{
"error": err.Error(),
@@ -227,6 +249,27 @@ func (s *service) Recent(ctx context.Context, query RecentQuery) (*RecentRespons
}, nil
}
cacheKey := strconv.Itoa(limit)
return s.recentCache.Get(ctx, cacheKey, func(loadCtx context.Context) (*RecentResponse, error) {
s.logger.Info("Fetching dashboard recent tenders", map[string]interface{}{
"limit": limit,
})
items, nextCursor, err := s.repo.Recent(loadCtx, limit, "")
if err != nil {
s.logger.Error("Failed to fetch dashboard recent tenders", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("dashboard recent: %w", err)
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
})
}
func (s *service) Statistics(ctx context.Context, query StatisticsQuery) (*StatisticsReportResponse, error) {
days := trendDays(query.Days)
if cached, ok := s.cachedStatistics(days); ok {
@@ -339,6 +382,80 @@ func (s *service) loadStatistics(ctx context.Context, days int) (*StatisticsRepo
return out.(*StatisticsReportResponse), nil
}
func (s *service) warmWidgetCaches() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
defaultWindowSec := int64(defaultClosingWindowHours) * 3600
warmed := 0
if s.trendCache.WarmFromRedis(ctx, fmt.Sprintf("%d:%s", defaultTrendDays, "created")) {
warmed++
}
if s.countriesCache.WarmFromRedis(ctx, strconv.Itoa(defaultCountriesLimit)) {
warmed++
}
if s.noticeTypesCache.WarmFromRedis(ctx, "all") {
warmed++
}
if s.closingSoonCache.WarmFromRedis(ctx, fmt.Sprintf("%d:%d", defaultListLimit, defaultWindowSec)) {
warmed++
}
if s.recentCache.WarmFromRedis(ctx, strconv.Itoa(defaultListLimit)) {
warmed++
}
if warmed > 0 {
s.logger.Info("Dashboard widget caches warmed from Redis", map[string]interface{}{
"warmed": warmed,
})
}
go s.trendCache.Reload(fmt.Sprintf("%d:%s", defaultTrendDays, "created"), func(loadCtx context.Context) (*TrendResponse, error) {
return s.loadTrend(loadCtx, TrendQuery{Days: defaultTrendDays, Metric: "created"})
})
go s.countriesCache.Reload(strconv.Itoa(defaultCountriesLimit), func(loadCtx context.Context) (*CountriesResponse, error) {
return s.repo.Countries(loadCtx, defaultCountriesLimit)
})
go s.noticeTypesCache.Reload("all", func(loadCtx context.Context) (*NoticeTypesResponse, error) {
return s.repo.NoticeTypes(loadCtx)
})
go s.closingSoonCache.Reload(fmt.Sprintf("%d:%d", defaultListLimit, defaultWindowSec), func(loadCtx context.Context) (*ClosingSoonResponse, error) {
items, err := s.repo.ClosingSoon(loadCtx, defaultListLimit, defaultWindowSec)
if err != nil {
return nil, err
}
return &ClosingSoonResponse{Items: items}, nil
})
go s.recentCache.Reload(strconv.Itoa(defaultListLimit), func(loadCtx context.Context) (*RecentResponse, error) {
items, nextCursor, err := s.repo.Recent(loadCtx, defaultListLimit, "")
if err != nil {
return nil, err
}
return &RecentResponse{
Items: items,
NextCursor: nextCursor,
}, nil
})
}
func (s *service) loadTrend(ctx context.Context, query TrendQuery) (*TrendResponse, error) {
days := trendDays(query.Days)
metric := normalizeTrendMetric(query.Metric)
startUnix := trendStartUnix(days)
counts, err := s.repo.Trend(ctx, days, metric, startUnix)
if err != nil {
return nil, err
}
return &TrendResponse{
Metric: metric,
Days: days,
Series: fillTrendSeries(days, counts),
}, nil
}
func (s *service) warmSummaryCache() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
+96 -11
View File
@@ -33,6 +33,13 @@ type scrapedTendersScope struct {
totalTenderCount int64 // >0 when MinIO procedure count should be used instead of Mongo Count
}
// translatedNoticesScope holds MinIO-backed translation statistics when available.
type translatedNoticesScope struct {
fromMinIO bool
dailyCounts map[string]int64
totalCount int64
}
func (r *repository) Statistics(ctx context.Context, days int, startUnix int64) (*StatisticsReportResponse, error) {
_ = ctx
@@ -47,6 +54,7 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
totalTranslated int64
totalScrapedTED int64
scrapedScope scrapedTendersScope
translatedScope translatedNoticesScope
)
var wg sync.WaitGroup
@@ -78,23 +86,26 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
wg.Add(1)
go func() {
defer wg.Done()
scope, err := r.cachedTranslatedNoticesScope(context.Background())
if err != nil {
recordFailure("translated_scope", err)
return
}
translatedScope = scope
if scope.fromMinIO {
return
}
qctx, cancel := context.WithTimeout(context.Background(), statisticsMongoQueryTimeout)
defer cancel()
counts, err := r.translatedNoticesPerDay(qctx, startDay, endDay)
counts, err := r.translatedNoticesPerDayFromMetrics(qctx, startDay, endDay)
if err != nil {
recordFailure("translated_notices", err)
translatedNotices = map[string]int64{}
return
}
} else {
translatedNotices = counts
}()
wg.Add(1)
go func() {
defer wg.Done()
qctx, cancel := context.WithTimeout(context.Background(), statisticsMongoQueryTimeout)
defer cancel()
}
total, err := r.metricsCounter.Get(qctx, orm.AITranslationSuccessCounterKey)
if err != nil {
@@ -118,6 +129,11 @@ func (r *repository) Statistics(ctx context.Context, days int, startUnix int64)
wg.Wait()
if translatedScope.fromMinIO {
translatedNotices = filterDailyCountsSince(translatedScope.dailyCounts, startUnix)
totalTranslated = translatedScope.totalCount
}
wg.Add(1)
go func() {
defer wg.Done()
@@ -224,10 +240,79 @@ func (r *repository) scrapedDocumentsPerDayFromMongo(ctx context.Context, startU
return decodeDailyCounts(ctx, cursor)
}
func (r *repository) translatedNoticesPerDay(ctx context.Context, startDay, endDay time.Time) (map[string]int64, error) {
func (r *repository) translatedNoticesPerDayFromMetrics(ctx context.Context, startDay, endDay time.Time) (map[string]int64, error) {
return r.metricsCounter.GetDailyCounts(ctx, startDay, endDay)
}
func (r *repository) cachedTranslatedNoticesScope(_ context.Context) (translatedNoticesScope, error) {
now := time.Now()
r.translatedScopeMu.Lock()
if r.translatedScopeCached && now.Before(r.translatedScopeExpiry) {
scope := r.translatedScope
r.translatedScopeMu.Unlock()
return scope, nil
}
r.translatedScopeMu.Unlock()
v, err, _ := r.translatedScopeGroup.Do("translated-scope", func() (interface{}, error) {
r.translatedScopeMu.Lock()
if r.translatedScopeCached && time.Now().Before(r.translatedScopeExpiry) {
scope := r.translatedScope
r.translatedScopeMu.Unlock()
return scope, nil
}
r.translatedScopeMu.Unlock()
resolveCtx, cancel := context.WithTimeout(context.Background(), scrapedScopeResolveTimeout)
defer cancel()
scope, resolveErr := r.resolveTranslatedNoticesScope(resolveCtx)
if resolveErr != nil {
if r.logger != nil {
r.logger.Warn("MinIO translation scope unavailable, using metrics counter fallback for dashboard statistics", map[string]interface{}{
"error": resolveErr.Error(),
})
}
return translatedNoticesScope{}, nil
}
r.translatedScopeMu.Lock()
r.translatedScope = scope
r.translatedScopeExpiry = time.Now().Add(scrapedScopeCacheTTL)
r.translatedScopeCached = true
r.translatedScopeMu.Unlock()
return scope, nil
})
if err != nil {
return translatedNoticesScope{}, err
}
return v.(translatedNoticesScope), nil
}
func (r *repository) resolveTranslatedNoticesScope(ctx context.Context) (translatedNoticesScope, error) {
if r.procedureLister == nil {
return translatedNoticesScope{}, nil
}
scanner, ok := r.procedureLister.(translatedNoticesScanner)
if !ok {
return translatedNoticesScope{}, nil
}
dailyCounts, total, err := scanner.ScanTranslatedNotices(ctx)
if err != nil {
return translatedNoticesScope{}, err
}
return translatedNoticesScope{
fromMinIO: true,
dailyCounts: dailyCounts,
totalCount: total,
}, nil
}
func (r *repository) totalScrapedTenders(ctx context.Context, scope scrapedTendersScope) (int64, error) {
if scope.zero {
return 0, nil
@@ -29,6 +29,10 @@ type mockProcedureDocumentsLister struct {
procedures []ai_summarizer.ProcedureDocumentsSummary
dailyCounts map[string]int64
err error
translatedDailyCounts map[string]int64
translatedTotal int64
translatedErr error
}
func (m *mockProcedureDocumentsLister) ListProceduresWithDocuments(context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
@@ -45,6 +49,13 @@ func (m *mockProcedureDocumentsLister) ScanScrapedDocuments(context.Context) ([]
return m.procedures, m.dailyCounts, nil
}
func (m *mockProcedureDocumentsLister) ScanTranslatedNotices(context.Context) (map[string]int64, int64, error) {
if m.translatedErr != nil {
return nil, 0, m.translatedErr
}
return m.translatedDailyCounts, m.translatedTotal, nil
}
func TestScrapedDocumentsFolderIDsUsesMinIOProcedures(t *testing.T) {
repo := &repository{
procedureLister: &mockProcedureDocumentsLister{
@@ -259,3 +270,41 @@ func TestMongoScrapedTendersScopeMatchesStatisticsFilter(t *testing.T) {
t.Fatalf("expected documents_scraped filter, got %#v", scope.match)
}
}
func TestResolveTranslatedNoticesScopeUsesMinIOCounts(t *testing.T) {
repo := &repository{
procedureLister: &mockProcedureDocumentsLister{
translatedDailyCounts: map[string]int64{
"2026-07-01": 4,
"2026-07-09": 2,
},
translatedTotal: 6,
},
}
scope, err := repo.resolveTranslatedNoticesScope(context.Background())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !scope.fromMinIO {
t.Fatal("expected MinIO-backed translation scope")
}
if scope.totalCount != 6 {
t.Fatalf("expected total 6, got %d", scope.totalCount)
}
if scope.dailyCounts["2026-07-09"] != 2 {
t.Fatalf("expected 2 on 2026-07-09, got %d", scope.dailyCounts["2026-07-09"])
}
}
func TestResolveTranslatedNoticesScopeWithoutLister(t *testing.T) {
repo := &repository{}
scope, err := repo.resolveTranslatedNoticesScope(context.Background())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if scope.fromMinIO {
t.Fatal("expected metrics fallback without lister")
}
}
+230
View File
@@ -0,0 +1,230 @@
package dashboard
import (
"context"
"encoding/json"
"fmt"
"sync"
"time"
"tm/pkg/logger"
"tm/pkg/redis"
goredis "github.com/redis/go-redis/v9"
"golang.org/x/sync/singleflight"
)
const (
// widgetCacheTTL is the fresh window before a background reload is triggered.
widgetCacheTTL = 60 * time.Second
// widgetStaleGraceTTL is how long stale entries may be served while reloading.
// Dashboard widgets have no write-side invalidation; expect up to this lag after
// data changes (e.g. a new tender may not appear in cached widgets immediately).
widgetStaleGraceTTL = 5 * time.Minute
widgetReloadTimeout = 2 * time.Minute
)
type widgetLoader[T any] func(ctx context.Context) (T, error)
type widgetCache[T any] struct {
mu sync.Mutex
entries map[string]cacheEntry[T]
group singleflight.Group
redis redis.Client
logger logger.Logger
keyPrefix string
ttl time.Duration
staleGrace time.Duration
}
func newWidgetCache[T any](redisClient redis.Client, log logger.Logger, keyPrefix string) *widgetCache[T] {
return &widgetCache[T]{
entries: make(map[string]cacheEntry[T]),
redis: redisClient,
logger: log,
keyPrefix: keyPrefix,
ttl: widgetCacheTTL,
staleGrace: widgetStaleGraceTTL,
}
}
func (c *widgetCache[T]) Get(ctx context.Context, key string, load widgetLoader[T]) (T, error) {
var zero T
// Returned values are shared across concurrent callers via singleflight and stale
// hits. Callers must treat them as read-only through JSON serialization.
if value, ok := c.fresh(key); ok {
return value, nil
}
if value, ok := c.stale(key); ok {
go c.Reload(key, load)
return value, nil
}
if value, ok := c.fromRedis(ctx, key); ok {
c.store(key, value)
go c.Reload(key, load)
return value, nil
}
out, err, _ := c.group.Do(key, func() (interface{}, error) {
if value, ok := c.fresh(key); ok {
return value, nil
}
result, err := load(context.WithoutCancel(ctx))
if err != nil {
return zero, err
}
c.store(key, result)
c.storeRedis(context.Background(), key, result)
return result, nil
})
if err != nil {
return zero, err
}
return out.(T), nil
}
func (c *widgetCache[T]) Reload(key string, load widgetLoader[T]) {
ctx, cancel := context.WithTimeout(context.Background(), widgetReloadTimeout)
defer cancel()
_, _, _ = c.group.Do("reload:"+key, func() (interface{}, error) {
result, err := load(ctx)
if err != nil {
if c.logger != nil {
c.logger.Warn("Failed to refresh dashboard widget cache", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return nil, err
}
c.store(key, result)
c.storeRedis(context.Background(), key, result)
return result, nil
})
}
func (c *widgetCache[T]) WarmFromRedis(ctx context.Context, key string) bool {
value, ok := c.fromRedis(ctx, key)
if !ok {
return false
}
c.store(key, value)
return true
}
func (c *widgetCache[T]) fresh(key string) (T, bool) {
var zero T
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
entry, ok := c.entries[key]
if !ok || now.After(entry.expiresAt) {
return zero, false
}
return entry.value, true
}
func (c *widgetCache[T]) stale(key string) (T, bool) {
var zero T
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
entry, ok := c.entries[key]
if !ok || now.After(entry.staleUntil) {
if ok {
delete(c.entries, key)
}
return zero, false
}
return entry.value, true
}
func (c *widgetCache[T]) store(key string, value T) {
now := time.Now()
c.mu.Lock()
defer c.mu.Unlock()
c.entries[key] = cacheEntry[T]{
expiresAt: now.Add(c.ttl),
staleUntil: now.Add(c.ttl + c.staleGrace),
value: value,
}
}
func (c *widgetCache[T]) redisKey(key string) string {
return fmt.Sprintf("dashboard:%s:%s", c.keyPrefix, key)
}
func (c *widgetCache[T]) fromRedis(ctx context.Context, key string) (T, bool) {
var zero T
if c.redis == nil {
return zero, false
}
raw, err := c.redis.Get(ctx, c.redisKey(key))
if err != nil {
if err != goredis.Nil && c.logger != nil {
c.logger.Warn("Failed to read dashboard widget cache from Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return zero, false
}
var value T
if err := json.Unmarshal([]byte(raw), &value); err != nil {
if c.logger != nil {
c.logger.Warn("Failed to decode dashboard widget cache from Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
_ = c.redis.Del(ctx, c.redisKey(key))
return zero, false
}
return value, true
}
func (c *widgetCache[T]) storeRedis(ctx context.Context, key string, value T) {
if c.redis == nil {
return
}
encoded, err := json.Marshal(value)
if err != nil {
if c.logger != nil {
c.logger.Warn("Failed to encode dashboard widget cache for Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
return
}
ttl := c.ttl + c.staleGrace
if err := c.redis.Set(ctx, c.redisKey(key), string(encoded), ttl); err != nil && c.logger != nil {
c.logger.Warn("Failed to store dashboard widget cache in Redis", map[string]interface{}{
"widget": c.keyPrefix,
"key": key,
"error": err.Error(),
})
}
}
+20 -2
View File
@@ -28,8 +28,26 @@ const (
DeliveryChannelInApp DeliveryChannel = "in_app"
)
// EventTypeInApp identifies notifications stored for the in-app notification center.
const EventTypeInApp = "in_app"
// Persisted event types for stored notifications (admin list / notification center).
const (
EventTypeInApp = "in_app"
EventTypeEmail = "email"
EventTypePush = "push"
)
// persistedEventType maps a delivery channel to the event_type stored in MongoDB.
func persistedEventType(channel DeliveryChannel) (string, bool) {
switch channel {
case DeliveryChannelEmail:
return EventTypeEmail, true
case DeliveryChannelPush:
return EventTypePush, true
case DeliveryChannelInApp:
return EventTypeInApp, true
default:
return "", false
}
}
// DeliveryStatus represents the delivery status of notification
type NotificationStatus string
+32 -7
View File
@@ -99,13 +99,27 @@ func (s *notificationService) Send(ctx context.Context, req *NotificationRequest
}
func (s *notificationService) sendToRecipient(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string) {
if err := s.persistNotification(ctx, recipient, req, link, image); err != nil {
// Always persist an in-app record so every send appears in the notification center.
if err := s.persistNotification(ctx, recipient, req, link, image, DeliveryChannelInApp); err != nil {
s.logger.Error("Failed to persist in-app notification", map[string]interface{}{
"error": err.Error(),
"user_id": recipient.UserID,
})
}
for _, channel := range req.Channels {
if channel == DeliveryChannelInApp {
continue
}
if err := s.persistNotification(ctx, recipient, req, link, image, channel); err != nil {
s.logger.Error("Failed to persist notification", map[string]interface{}{
"error": err.Error(),
"user_id": recipient.UserID,
"channel": channel,
})
}
}
metadata := map[string]any{
"tender": strings.TrimSpace(req.Tender),
}
@@ -149,15 +163,26 @@ func (s *notificationService) sendToRecipient(ctx context.Context, recipient not
}
}
func (s *notificationService) persistNotification(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string) error {
methods := map[string]string{
"in_app": "true",
func (s *notificationService) persistNotification(ctx context.Context, recipient notificationRecipient, req *NotificationRequest, link, image string, channel DeliveryChannel) error {
eventType, ok := persistedEventType(channel)
if !ok {
return nil
}
methods := map[string]string{}
switch channel {
case DeliveryChannelEmail:
if recipient.Email == "" {
return nil
}
if slices.Contains(req.Channels, DeliveryChannelEmail) && recipient.Email != "" {
methods["email"] = recipient.Email
case DeliveryChannelPush:
if len(recipient.DeviceTokens) == 0 {
return nil
}
if slices.Contains(req.Channels, DeliveryChannelPush) && len(recipient.DeviceTokens) > 0 {
methods["push"] = recipient.DeviceTokens[0]
case DeliveryChannelInApp:
methods["in_app"] = "true"
}
status := string(DeliveryStatusSent)
@@ -173,7 +198,7 @@ func (s *notificationService) persistNotification(ctx context.Context, recipient
Image: image,
Priority: string(req.Priority),
Type: string(req.Type),
EventType: EventTypeInApp,
EventType: eventType,
Status: status,
Methods: methods,
Metadata: map[string]any{
+76 -1
View File
@@ -1,6 +1,10 @@
package tender
import "testing"
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestParseAIProcedureRef(t *testing.T) {
tests := []struct {
@@ -47,3 +51,74 @@ func TestFormatAIProcedureRef(t *testing.T) {
t.Fatalf("FormatAIProcedureRef() = %q want %q", got, want)
}
}
func TestMapProcedureReferencesToTenders(t *testing.T) {
candidates := []Tender{
{
ContractFolderID: "folder-1",
NoticePublicationID: "notice-new",
RelatedNoticePublicationIDs: []string{"notice-old"},
},
{
ContractFolderID: "folder-1",
NoticePublicationID: "notice-legacy",
},
{
ContractFolderID: "folder-2",
NoticePublicationID: "notice-2",
},
}
refs := []ProcedureReference{
{Ref: "PROC_folder-1/notice-old", ContractFolderID: "folder-1", NoticePublicationID: "notice-old"},
{Ref: "PROC_folder-1/notice-legacy", ContractFolderID: "folder-1", NoticePublicationID: "notice-legacy"},
{Ref: "PROC_folder-2/notice-2", ContractFolderID: "folder-2", NoticePublicationID: "notice-2"},
}
got := mapProcedureReferencesToTenders(candidates, refs)
if len(got) != 3 {
t.Fatalf("mapProcedureReferencesToTenders() len = %d, want 3", len(got))
}
if got["PROC_folder-1/notice-old"].NoticePublicationID != "notice-new" {
t.Fatalf("expected notice-old ref to resolve to latest merged tender, got %+v", got["PROC_folder-1/notice-old"])
}
if got["PROC_folder-1/notice-legacy"].NoticePublicationID != "notice-legacy" {
t.Fatalf("expected legacy ref to resolve to legacy tender, got %+v", got["PROC_folder-1/notice-legacy"])
}
if got["PROC_folder-2/notice-2"].NoticePublicationID != "notice-2" {
t.Fatalf("expected folder-2 ref to resolve, got %+v", got["PROC_folder-2/notice-2"])
}
}
func TestMapProcedureReferencesToTendersUsesFolderCandidatesWhenNoticeLookupMisses(t *testing.T) {
// Notice-old is only present on the merged tender loaded via contract_folder_id.
candidates := []Tender{
{
ContractFolderID: "folder-1",
NoticePublicationID: "notice-new",
RelatedNoticePublicationIDs: []string{"notice-old"},
},
}
refs := []ProcedureReference{
{Ref: "PROC_folder-1/notice-old", ContractFolderID: "folder-1", NoticePublicationID: "notice-old"},
}
got := mapProcedureReferencesToTenders(candidates, refs)
if len(got) != 1 {
t.Fatalf("mapProcedureReferencesToTenders() len = %d, want 1", len(got))
}
if got["PROC_folder-1/notice-old"].NoticePublicationID != "notice-new" {
t.Fatalf("expected folder candidate to resolve notice-old ref, got %+v", got["PROC_folder-1/notice-old"])
}
}
func TestMergeTenderCandidatesDedupesByMongoID(t *testing.T) {
first := Tender{ContractFolderID: "folder-1", NoticePublicationID: "notice-a"}
first.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
second := Tender{ContractFolderID: "folder-1", NoticePublicationID: "notice-b"}
second.ID = first.ID
got := mergeTenderCandidates([]Tender{first}, []Tender{second})
if len(got) != 1 {
t.Fatalf("mergeTenderCandidates() len = %d, want 1", len(got))
}
}
+120
View File
@@ -0,0 +1,120 @@
package tender
import (
"context"
"errors"
"fmt"
"strings"
"time"
"tm/pkg/ai_summarizer"
)
const (
// contractFoldersWithDocsCacheTTL avoids scanning MinIO on every documents_scraped list request.
// Kept >= resolve timeout so a slow scan is not immediately invalidated for waiters.
contractFoldersWithDocsCacheTTL = 15 * time.Minute
// contractFoldersWithDocsResolveTimeout bounds a single MinIO procedure scan.
contractFoldersWithDocsResolveTimeout = 3 * time.Minute
)
type contractFoldersWithDocsCacheEntry struct {
expiresAt time.Time
folderIDs []string
valid bool
}
// enrichDocumentsScrapedFilter resolves contract_folder_id values that currently have
// documents in MinIO and stores them on the search form for the repository filter.
func (s *tenderService) enrichDocumentsScrapedFilter(ctx context.Context, form *SearchForm) error {
if form == nil || !form.DocumentsScraped {
return nil
}
folderIDs, err := s.cachedContractFolderIDsWithDocuments(ctx)
if err != nil {
s.logger.Error("Failed to resolve tenders with scraped documents from MinIO", map[string]interface{}{
"error": err.Error(),
})
if errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
return fmt.Errorf("failed to resolve tenders with scraped documents: MinIO connection unavailable: %w", err)
}
return fmt.Errorf("failed to resolve tenders with scraped documents: %w", err)
}
form.ContractFolderIDsWithDocuments = folderIDs
return nil
}
func (s *tenderService) cachedContractFolderIDsWithDocuments(ctx context.Context) ([]string, error) {
now := time.Now()
s.contractFoldersCacheMu.Lock()
if s.contractFoldersCache.valid && now.Before(s.contractFoldersCache.expiresAt) {
ids := append([]string(nil), s.contractFoldersCache.folderIDs...)
s.contractFoldersCacheMu.Unlock()
return ids, nil
}
s.contractFoldersCacheMu.Unlock()
v, err, _ := s.contractFoldersCacheGroup.Do("contract-folders-with-docs", func() (interface{}, error) {
s.contractFoldersCacheMu.Lock()
if s.contractFoldersCache.valid && time.Now().Before(s.contractFoldersCache.expiresAt) {
ids := append([]string(nil), s.contractFoldersCache.folderIDs...)
s.contractFoldersCacheMu.Unlock()
return ids, nil
}
s.contractFoldersCacheMu.Unlock()
// Detach from the caller's cancellation so a client disconnect or proxy timeout
// does not abort the shared singleflight scan for every waiter.
resolveCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), contractFoldersWithDocsResolveTimeout)
defer cancel()
procedures, listErr := s.listProceduresWithDocuments(resolveCtx)
if listErr != nil {
return nil, listErr
}
folderIDs := contractFolderIDsFromProcedures(procedures)
s.contractFoldersCacheMu.Lock()
s.contractFoldersCache = contractFoldersWithDocsCacheEntry{
expiresAt: time.Now().Add(contractFoldersWithDocsCacheTTL),
folderIDs: folderIDs,
valid: true,
}
s.contractFoldersCacheMu.Unlock()
return append([]string(nil), folderIDs...), nil
})
if err != nil {
return nil, err
}
ids, ok := v.([]string)
if !ok {
return nil, fmt.Errorf("unexpected contract folder cache value type %T", v)
}
return append([]string(nil), ids...), nil
}
func contractFolderIDsFromProcedures(procedures []ai_summarizer.ProcedureDocumentsSummary) []string {
folderIDs := make([]string, 0, len(procedures))
seen := make(map[string]struct{}, len(procedures))
for _, proc := range procedures {
if proc.DocumentCount <= 0 {
continue
}
id := strings.TrimSpace(proc.ContractFolderID)
if id == "" {
continue
}
if _, ok := seen[id]; ok {
continue
}
seen[id] = struct{}{}
folderIDs = append(folderIDs, id)
}
return folderIDs
}
@@ -0,0 +1,241 @@
package tender
import (
"context"
"errors"
"fmt"
"io"
"sync/atomic"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"tm/pkg/ai_summarizer"
)
type mockProcedureDocumentLister struct {
procedures []ai_summarizer.ProcedureDocumentsSummary
err error
calls atomic.Int32
}
func (m *mockProcedureDocumentLister) ListProceduresWithDocuments(context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
m.calls.Add(1)
if m.err != nil {
return nil, m.err
}
return m.procedures, nil
}
type stubProcedureDocumentStorage struct {
lister interface {
ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error)
}
}
func (s stubProcedureDocumentStorage) GetSummaryFromStorage(context.Context, string, string) (string, error) {
return "", ai_summarizer.ErrObjectNotFound
}
func (s stubProcedureDocumentStorage) GetDocumentSummariesFromStorage(context.Context, string, string) ([]ai_summarizer.DocumentSummary, error) {
return nil, nil
}
func (s stubProcedureDocumentStorage) GetTranslationFromStorage(context.Context, string, string, string) (ai_summarizer.StoredTranslation, error) {
return ai_summarizer.StoredTranslation{}, ai_summarizer.ErrTranslationNotReady
}
func (s stubProcedureDocumentStorage) ListDocuments(context.Context, string, string) ([]ai_summarizer.StoredDocument, error) {
return nil, nil
}
func (s stubProcedureDocumentStorage) DownloadDocument(context.Context, string) (io.ReadCloser, error) {
return nil, ai_summarizer.ErrObjectNotFound
}
func (s stubProcedureDocumentStorage) ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
return s.lister.ListProceduresWithDocuments(ctx)
}
func TestBuildSearchFilterDocumentsScrapedUsesMinIOFolderIDs(t *testing.T) {
repo := &tenderRepository{}
filter := repo.buildSearchFilter(&SearchForm{
DocumentsScraped: true,
ContractFolderIDsWithDocuments: []string{"folder-a", "folder-b"},
})
in, ok := filter["contract_folder_id"].(bson.M)["$in"].([]string)
if !ok {
t.Fatalf("expected contract_folder_id $in filter, got %#v", filter)
}
if len(in) != 2 || in[0] != "folder-a" || in[1] != "folder-b" {
t.Fatalf("unexpected folder ids: %#v", in)
}
if filter["processing_metadata.documents_scraped"] != nil {
t.Fatalf("expected no stale mongo documents_scraped flag, got %#v", filter)
}
}
func TestContractFolderIDsSearchClauseChunksLargeLists(t *testing.T) {
folderIDs := make([]string, maxContractFolderInClause+2)
for i := range folderIDs {
folderIDs[i] = fmt.Sprintf("folder-%d", i)
}
clause := contractFolderIDsSearchClause(folderIDs)
or, ok := clause["$or"].([]bson.M)
if !ok || len(or) != 2 {
t.Fatalf("expected chunked $or filter, got %#v", clause)
}
}
func TestCachedContractFolderIDsWithDocumentsUsesCache(t *testing.T) {
lister := &mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 2},
{ContractFolderID: "folder-2", DocumentCount: 0},
{ContractFolderID: "folder-3", DocumentCount: 1},
},
}
svc := &tenderService{
aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister},
}
first, err := svc.cachedContractFolderIDsWithDocuments(context.Background())
if err != nil {
t.Fatalf("first lookup failed: %v", err)
}
if len(first) != 2 || first[0] != "folder-1" || first[1] != "folder-3" {
t.Fatalf("unexpected folder ids: %#v", first)
}
if lister.calls.Load() != 1 {
t.Fatalf("expected one MinIO scan, got %d", lister.calls.Load())
}
second, err := svc.cachedContractFolderIDsWithDocuments(context.Background())
if err != nil {
t.Fatalf("second lookup failed: %v", err)
}
if len(second) != 2 {
t.Fatalf("unexpected cached folder ids: %#v", second)
}
if lister.calls.Load() != 1 {
t.Fatalf("expected cache hit without extra MinIO scan, got %d calls", lister.calls.Load())
}
}
func TestEnrichDocumentsScrapedFilterPropagatesMinIOError(t *testing.T) {
svc := &tenderService{
aiSummarizerStorage: stubProcedureDocumentStorage{
lister: &mockProcedureDocumentLister{
err: ai_summarizer.ErrMinIOUnavailable,
},
},
logger: noopTenderTestLogger{},
}
form := &SearchForm{DocumentsScraped: true}
err := svc.enrichDocumentsScrapedFilter(context.Background(), form)
if err == nil {
t.Fatal("expected MinIO error")
}
if !errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
t.Fatalf("expected wrapped MinIO unavailable error, got %v", err)
}
}
func TestContractFolderIDsFromProceduresDedupesAndSkipsEmpty(t *testing.T) {
ids := contractFolderIDsFromProcedures([]ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "a", DocumentCount: 1},
{ContractFolderID: "a", DocumentCount: 2},
{ContractFolderID: " ", DocumentCount: 1},
{ContractFolderID: "b", DocumentCount: 0},
})
if len(ids) != 1 || ids[0] != "a" {
t.Fatalf("unexpected ids: %#v", ids)
}
}
type scanStartedProcedureLister struct {
mockProcedureDocumentLister
started chan struct{}
}
func (m *scanStartedProcedureLister) ListProceduresWithDocuments(ctx context.Context) ([]ai_summarizer.ProcedureDocumentsSummary, error) {
if m.started != nil {
select {
case <-m.started:
default:
close(m.started)
}
}
time.Sleep(50 * time.Millisecond)
if err := ctx.Err(); err != nil {
return nil, err
}
return m.mockProcedureDocumentLister.ListProceduresWithDocuments(ctx)
}
func TestCachedContractFolderIDsSurvivesCallerCancellation(t *testing.T) {
started := make(chan struct{})
lister := &scanStartedProcedureLister{
mockProcedureDocumentLister: mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 1},
},
},
started: started,
}
svc := &tenderService{aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
errCh := make(chan error, 1)
var ids []string
go func() {
var err error
ids, err = svc.cachedContractFolderIDsWithDocuments(ctx)
errCh <- err
}()
select {
case <-started:
case <-time.After(2 * time.Second):
t.Fatal("MinIO scan did not start")
}
cancel()
if err := <-errCh; err != nil {
t.Fatalf("expected shared scan to finish despite caller cancel: %v", err)
}
if len(ids) != 1 || ids[0] != "folder-1" {
t.Fatalf("unexpected folder ids: %#v", ids)
}
}
func TestContractFoldersWithDocsCacheExpires(t *testing.T) {
lister := &mockProcedureDocumentLister{
procedures: []ai_summarizer.ProcedureDocumentsSummary{
{ContractFolderID: "folder-1", DocumentCount: 1},
},
}
svc := &tenderService{aiSummarizerStorage: stubProcedureDocumentStorage{lister: lister}}
if _, err := svc.cachedContractFolderIDsWithDocuments(context.Background()); err != nil {
t.Fatalf("lookup failed: %v", err)
}
svc.contractFoldersCacheMu.Lock()
svc.contractFoldersCache.expiresAt = time.Now().Add(-time.Second)
svc.contractFoldersCacheMu.Unlock()
if _, err := svc.cachedContractFolderIDsWithDocuments(context.Background()); err != nil {
t.Fatalf("lookup after expiry failed: %v", err)
}
if lister.calls.Load() != 2 {
t.Fatalf("expected cache refresh after expiry, got %d calls", lister.calls.Load())
}
}
-1
View File
@@ -88,7 +88,6 @@ type Tender struct {
Source TenderSource `bson:"source" json:"source"`
SourceFileURL string `bson:"source_file_url" json:"source_file_url"`
SourceFileName string `bson:"source_file_name" json:"source_file_name"`
ContentXML string `bson:"content_xml,omitempty" json:"content_xml,omitempty"`
ProcessingMetadata ProcessingMetadata `bson:"processing_metadata" json:"processing_metadata"`
// AIOverallSummary optional denormalized overall summary for search/list (e.g. synced from AI pipeline storage).
AIOverallSummary string `bson:"ai_overall_summary,omitempty" json:"ai_overall_summary,omitempty"`
+77 -1
View File
@@ -32,6 +32,7 @@ type SearchForm struct {
CountryCodes []string `query:"country_codes" valid:"optional"`
RegionCodes []string `query:"region_codes" valid:"optional"`
CompanyID *string `query:"company_id" valid:"optional"`
CompanyIDs []string `query:"-" valid:"optional"`
CustomerID *string `query:"customer_id" valid:"optional"`
Status []string `query:"status" valid:"optional"`
MainClassification *string `query:"main_classification" valid:"optional"`
@@ -63,8 +64,83 @@ type SearchForm struct {
OnlyActiveDeadlines bool `query:"only_active_deadlines" valid:"optional"`
Language *string `query:"lang" valid:"optional"`
DocumentsScraped bool `query:"documents_scraped" valid:"optional"`
// ContractFolderIDsWithDocuments is populated by the service from MinIO when DocumentsScraped is true.
// ContractFolderIDsWithDocuments is populated by the service from a cached MinIO scan when DocumentsScraped is true.
ContractFolderIDsWithDocuments []string `query:"-" valid:"optional"`
// ExcludeRejectedTenders hides company-rejected tenders from recommendation results.
ExcludeRejectedTenders bool `query:"-" valid:"optional"`
}
// HasListFilters reports whether the form applies any MongoDB list filters.
func (f *SearchForm) HasListFilters() bool {
if f == nil {
return false
}
if f.DocumentsScraped || f.OnlyActiveDeadlines || f.ExcludeRejectedTenders {
return true
}
if f.Search != nil && strings.TrimSpace(*f.Search) != "" {
return true
}
if f.Title != nil && strings.TrimSpace(*f.Title) != "" {
return true
}
if f.Description != nil && strings.TrimSpace(*f.Description) != "" {
return true
}
if f.NoticeType != nil && strings.TrimSpace(*f.NoticeType) != "" {
return true
}
if len(f.NoticeTypes) > 0 || len(f.FormTypes) > 0 {
return true
}
if f.ProcurementType != nil && strings.TrimSpace(*f.ProcurementType) != "" {
return true
}
if f.Country != nil && strings.TrimSpace(*f.Country) != "" {
return true
}
if f.CountryCode != nil && strings.TrimSpace(*f.CountryCode) != "" {
return true
}
if len(f.CountryCodes) > 0 || len(f.RegionCodes) > 0 {
return true
}
if f.MainClassification != nil && strings.TrimSpace(*f.MainClassification) != "" {
return true
}
if len(f.Classifications) > 0 || len(f.CpvCodes) > 0 {
return true
}
if f.MinEstimatedValue != nil || f.MaxEstimatedValue != nil || f.Currency != "" {
return true
}
if f.CreatedAt != nil || f.CreatedAtFrom != nil || f.CreatedAtTo != nil {
return true
}
if f.TenderDeadline != nil || f.TenderDeadlineFrom != nil || f.TenderDeadlineTo != nil ||
f.DeadlineFrom != nil || f.DeadlineTo != nil {
return true
}
if f.PublicationDate != nil || f.PublicationDateFrom != nil || f.PublicationDateTo != nil {
return true
}
if f.SubmissionDeadline != nil || f.SubmissionDateAliasFrom != nil || f.SubmissionDateAliasTo != nil ||
f.SubmissionDateFrom != nil || f.SubmissionDateTo != nil {
return true
}
if len(f.Status) > 0 || len(f.Source) > 0 || len(f.Languages) > 0 {
return true
}
if f.BuyerOrganizationID != nil && strings.TrimSpace(*f.BuyerOrganizationID) != "" {
return true
}
if f.NoticePublicationID != nil && strings.TrimSpace(*f.NoticePublicationID) != "" {
return true
}
if len(f.ContractFolderIDsWithDocuments) > 0 {
return true
}
return false
}
// SearchResponse represents the response for listing tenders
+21 -4
View File
@@ -160,10 +160,11 @@ func (h *TenderHandler) Delete(c echo.Context) error {
// @Param languages query []string false "Filter by languages (comma-separated)"
// @Param buyer_organization_id query string false "Filter by buyer organization ID"
// @Param notice_publication_id query string false "Filter by TED notice publication ID"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO (cached scan)"
// @Param limit query int false "Number of items per page (default: 20, max: 100)"
// @Param offset query int false "Number of items to skip (default: 0). Prefer cursor for deep pages."
// @Param cursor query string false "Opaque cursor from previous page meta.next_cursor"
// @Param include_total query bool false "When true, include meta.total and meta.pages (slower on large collections)"
// @Param sort_by query string false "Sort by field"
// @Param sort_order query string false "Sort order (asc or desc)"
// @Success 200 {object} response.APIResponse{data=map[string]interface{}}
@@ -270,10 +271,11 @@ func (h *TenderHandler) AdminRecommendTenders(c echo.Context) error {
// @Param languages query []string false "Filter by languages (comma-separated)"
// @Param buyer_organization_id query string false "Filter by buyer organization ID"
// @Param notice_publication_id query string false "Filter by TED notice publication ID"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO"
// @Param documents_scraped query bool false "When true, only tenders whose contract_folder_id has documents in MinIO (cached scan)"
// @Param limit query int false "Number of items per page (default: 20, max: 100)"
// @Param offset query int false "Number of items to skip (default: 0). Prefer cursor for deep pages."
// @Param cursor query string false "Opaque cursor from previous page meta.next_cursor"
// @Param include_total query bool false "When true, include meta.total and meta.pages (slower on large collections)"
// @Param sort_by query string false "Sort by field"
// @Param sort_order query string false "Sort order (asc or desc)"
// @Success 200 {object} response.APIResponse{data=SearchResponse}
@@ -326,7 +328,7 @@ func (h *TenderHandler) GetPublicTenders(c echo.Context) error {
// RecommendTenders returns AI-ranked tender recommendations for the authenticated customer's company
// @Summary Get recommended tenders
// @Description Retrieve AI-ranked tenders with full tender details. Only active tenders whose publication-based submission window has not passed are returned (submission_deadline, or 48 working hours after publication_date). AI tender_id values use PROC_{contract_folder_id}/{notice_publication_id}; each item includes rank, analysis, procedure_ref, and the standard tender fields.
// @Description Retrieve AI-ranked tenders with full tender details. Only active tenders whose publication-based submission window has not passed are returned (submission_deadline, or 48 working hours after publication_date). Company-rejected tenders are excluded. AI tender_id values use PROC_{contract_folder_id}/{notice_publication_id}; each item includes rank, analysis, procedure_ref, and the standard tender fields.
// @Tags Tenders
// @Produce json
// @Param limit query int false "Number of items per page (default: 10, max: 100)"
@@ -352,10 +354,26 @@ func (h *TenderHandler) RecommendTenders(c echo.Context) error {
form.Status = []string{string(TenderStatusActive)}
form.OnlyActiveDeadlines = true
form.ExcludeRejectedTenders = true
if customerIDStr, ok := c.Get("customer_id").(string); ok && customerIDStr != "" {
form.CustomerID = &customerIDStr
}
requestedCompanyID := ""
if form.CompanyID != nil {
requestedCompanyID = strings.TrimSpace(*form.CompanyID)
}
if requestedCompanyID != "" {
if customerCompanyID, ok := c.Get("company_id").(string); ok && customerCompanyID != "" {
form.CompanyID = &customerCompanyID
}
} else if customerCompanyIDs, ok := c.Get("company_ids").([]string); ok && len(customerCompanyIDs) > 0 {
form.CompanyIDs = append([]string(nil), customerCompanyIDs...)
} else if customerCompanyID, ok := c.Get("company_id").(string); ok && customerCompanyID != "" {
form.CompanyID = &customerCompanyID
}
return h.serveRecommendTenders(c, form)
}
@@ -819,4 +837,3 @@ func (h *TenderHandler) GetAllAISummaries(c echo.Context) error {
return response.Success(c, summaries, "AI summaries retrieved successfully")
}
+158
View File
@@ -0,0 +1,158 @@
package tender
import (
"context"
"fmt"
"strings"
"golang.org/x/sync/errgroup"
)
// CompanyRejectedTenderLister returns tender IDs the company has rejected.
type CompanyRejectedTenderLister interface {
ListRejectedTenderIDsByCompanyID(ctx context.Context, companyID string) ([]string, error)
}
func normalizeRecommendationCompanyScope(companyIDs []string) []string {
normalized := make([]string, 0, len(companyIDs))
seen := make(map[string]struct{}, len(companyIDs))
for _, companyID := range companyIDs {
clean := strings.TrimSpace(companyID)
if clean == "" {
continue
}
if _, ok := seen[clean]; ok {
continue
}
seen[clean] = struct{}{}
normalized = append(normalized, clean)
}
return normalized
}
func (s *tenderService) buildRejectedTenderIDSet(ctx context.Context, companyID string) (map[string]struct{}, error) {
excluded := make(map[string]struct{})
if s.rejectedTenderLister == nil {
return excluded, nil
}
ids, err := s.rejectedTenderLister.ListRejectedTenderIDsByCompanyID(ctx, companyID)
if err != nil {
return nil, fmt.Errorf("failed to list rejected tender exclusions: %w", err)
}
addTenderIDsToSet(excluded, ids)
return excluded, nil
}
func (s *tenderService) buildRejectedTenderIDSetsForCompanies(ctx context.Context, companyIDs []string) (map[string]map[string]struct{}, error) {
excludedByCompany := make(map[string]map[string]struct{})
if s.rejectedTenderLister == nil {
return excludedByCompany, nil
}
normalized := normalizeRecommendationCompanyScope(companyIDs)
if len(normalized) == 0 {
return excludedByCompany, nil
}
results := make([][]string, len(normalized))
group, groupCtx := errgroup.WithContext(ctx)
for i, companyID := range normalized {
i := i
companyID := companyID
group.Go(func() error {
ids, err := s.rejectedTenderLister.ListRejectedTenderIDsByCompanyID(groupCtx, companyID)
if err != nil {
return fmt.Errorf("failed to list rejected tender exclusions: %w", err)
}
results[i] = ids
return nil
})
}
if err := group.Wait(); err != nil {
return nil, err
}
for i, companyID := range normalized {
excluded := make(map[string]struct{})
addTenderIDsToSet(excluded, results[i])
excludedByCompany[companyID] = excluded
}
return excludedByCompany, nil
}
func addTenderIDsToSet(set map[string]struct{}, ids []string) {
for _, id := range ids {
id = strings.TrimSpace(id)
if id == "" {
continue
}
set[id] = struct{}{}
}
}
// markResolvedRecommendedTenderSeen records a resolved tender and reports whether it was already seen.
// Recommendations may reference the same tender via different AI refs (procedure ref vs Mongo ID,
// or multiple notice publication IDs for one merged tender).
func markResolvedRecommendedTenderSeen(seen map[string]struct{}, tender Tender, tenderRef string) bool {
if seen == nil {
return false
}
key := strings.TrimSpace(tender.GetID())
if key == "" {
key = strings.TrimSpace(tenderRef)
}
if key == "" {
return false
}
if _, exists := seen[key]; exists {
return true
}
seen[key] = struct{}{}
return false
}
func isRecommendedTenderExcluded(tender Tender, tenderRef string, excluded map[string]struct{}) bool {
if len(excluded) == 0 {
return false
}
if _, ok := excluded[tender.GetID()]; ok {
return true
}
tenderRef = strings.TrimSpace(tenderRef)
if tenderRef != "" {
if _, ok := excluded[tenderRef]; ok {
return true
}
}
procRef := FormatAIProcedureRef(tender.ContractFolderID, tender.NoticePublicationID)
if procRef != "" {
if _, ok := excluded[procRef]; ok {
return true
}
}
return false
}
func isRecommendedTenderExcludedForAllCompanies(tender Tender, tenderRef string, excludedByCompany map[string]map[string]struct{}, companyIDs []string) bool {
normalized := normalizeRecommendationCompanyScope(companyIDs)
if len(normalized) == 0 {
return false
}
for _, companyID := range normalized {
if !isRecommendedTenderExcluded(tender, tenderRef, excludedByCompany[companyID]) {
return false
}
}
return true
}
@@ -0,0 +1,149 @@
package tender
import (
"context"
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
"tm/pkg/mongo"
)
type stubRejectedTenderLister struct {
byCompany map[string][]string
}
func (s stubRejectedTenderLister) ListRejectedTenderIDsByCompanyID(_ context.Context, companyID string) ([]string, error) {
return s.byCompany[companyID], nil
}
func TestIsRecommendedTenderExcluded(t *testing.T) {
tender := Tender{
Model: mongo.Model{},
ContractFolderID: "055329e0-3d8f-4eeb-946a-85a451f2e36b",
NoticePublicationID: "00423458-2026",
}
tender.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
procRef := FormatAIProcedureRef(tender.ContractFolderID, tender.NoticePublicationID)
tests := []struct {
name string
excluded map[string]struct{}
tenderRef string
want bool
}{
{
name: "empty exclusions",
excluded: map[string]struct{}{},
want: false,
},
{
name: "excluded by mongo id",
excluded: map[string]struct{}{tender.GetID(): {}},
want: true,
},
{
name: "excluded by procedure ref",
excluded: map[string]struct{}{procRef: {}},
want: true,
},
{
name: "excluded by recommendation ref",
excluded: map[string]struct{}{procRef: {}},
tenderRef: procRef,
want: true,
},
{
name: "not excluded",
excluded: map[string]struct{}{"other-tender-id": {}},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isRecommendedTenderExcluded(tender, tt.tenderRef, tt.excluded)
if got != tt.want {
t.Fatalf("isRecommendedTenderExcluded() = %v want %v", got, tt.want)
}
})
}
}
func TestBuildRejectedTenderIDSetsForCompanies(t *testing.T) {
service := &tenderService{
rejectedTenderLister: stubRejectedTenderLister{
byCompany: map[string][]string{
"company-a": {" tender-1 ", "tender-2"},
"company-b": {"tender-2", "tender-3"},
},
},
}
got, err := service.buildRejectedTenderIDSetsForCompanies(context.Background(), []string{"company-a", "company-b", "company-a"})
if err != nil {
t.Fatalf("buildRejectedTenderIDSetsForCompanies() error = %v", err)
}
if _, ok := got["company-a"]["tender-1"]; !ok {
t.Fatalf("expected company-a exclusions to contain tender-1")
}
if _, ok := got["company-a"]["tender-2"]; !ok {
t.Fatalf("expected company-a exclusions to contain tender-2")
}
if _, ok := got["company-b"]["tender-2"]; !ok {
t.Fatalf("expected company-b exclusions to contain tender-2")
}
if _, ok := got["company-b"]["tender-3"]; !ok {
t.Fatalf("expected company-b exclusions to contain tender-3")
}
if len(got) != 2 {
t.Fatalf("buildRejectedTenderIDSetsForCompanies() len = %d, want 2", len(got))
}
}
func TestIsRecommendedTenderExcludedForAllCompanies(t *testing.T) {
tender := Tender{
Model: mongo.Model{},
ContractFolderID: "055329e0-3d8f-4eeb-946a-85a451f2e36b",
NoticePublicationID: "00423458-2026",
}
tender.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
procRef := FormatAIProcedureRef(tender.ContractFolderID, tender.NoticePublicationID)
excludedByCompany := map[string]map[string]struct{}{
"company-a": {procRef: {}},
"company-b": {},
}
if isRecommendedTenderExcludedForAllCompanies(tender, procRef, excludedByCompany, []string{"company-a", "company-b"}) {
t.Fatalf("expected tender to remain visible when not all companies rejected it")
}
excludedByCompany["company-b"][procRef] = struct{}{}
if !isRecommendedTenderExcludedForAllCompanies(tender, procRef, excludedByCompany, []string{"company-a", "company-b"}) {
t.Fatalf("expected tender to be hidden when all companies rejected it")
}
}
func TestMarkResolvedRecommendedTenderSeen(t *testing.T) {
sample := Tender{}
sample.ID, _ = bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
seen := make(map[string]struct{})
first := markResolvedRecommendedTenderSeen(seen, sample, "PROC_folder/notice")
second := markResolvedRecommendedTenderSeen(seen, sample, "PROC_folder/notice")
if first {
t.Fatalf("first sighting should not be skipped, id=%q seen=%v", sample.GetID(), seen)
}
if !second {
t.Fatalf("duplicate sighting should be skipped, id=%q seen=%v", sample.GetID(), seen)
}
otherRefSameTender := Tender{}
otherRefSameTender.ID = sample.ID
if !markResolvedRecommendedTenderSeen(seen, otherRefSameTender, "PROC_folder/other-notice") {
t.Fatalf("duplicate tender via different ref should be skipped")
}
}
+94
View File
@@ -0,0 +1,94 @@
package tender
import (
"context"
"strings"
"tm/internal/company"
)
const recommendationResolveBatchSize = 100
type recommendationPageResult struct {
items []rankedRecommendedTender
total int64
}
func (s *tenderService) buildRecommendationPage(
ctx context.Context,
recommendations []company.RecommendedTenderResponse,
form *SearchForm,
companyIDs []string,
excluded map[string]struct{},
excludedByCompany map[string]map[string]struct{},
now int64,
offset int,
limit int,
) (recommendationPageResult, error) {
seenTenderIDs := make(map[string]struct{}, len(recommendations))
pageStart := offset
pageEnd := offset + limit
if pageStart < 0 {
pageStart = 0
}
if pageEnd < pageStart {
pageEnd = pageStart
}
result := recommendationPageResult{
items: make([]rankedRecommendedTender, 0, limit),
}
for start := 0; start < len(recommendations); start += recommendationResolveBatchSize {
end := start + recommendationResolveBatchSize
if end > len(recommendations) {
end = len(recommendations)
}
tenderByRef, err := s.resolveRecommendedTenders(ctx, recommendations[start:end])
if err != nil {
return recommendationPageResult{}, err
}
for _, rec := range recommendations[start:end] {
tenderRef := strings.TrimSpace(rec.TenderID)
tender, ok := tenderByRef[tenderRef]
if !ok {
continue
}
if len(companyIDs) > 0 {
if isRecommendedTenderExcludedForAllCompanies(tender, tenderRef, excludedByCompany, companyIDs) {
continue
}
} else if isRecommendedTenderExcluded(tender, tenderRef, excluded) {
continue
}
if form.OnlyActiveDeadlines && !tender.IsRecommendable(now) {
continue
}
if len(form.Status) > 0 && !statusAllowed(tender.Status, form.Status) {
continue
}
if markResolvedRecommendedTenderSeen(seenTenderIDs, tender, tenderRef) {
continue
}
result.total++
if int(result.total) <= pageStart || len(result.items) >= limit {
continue
}
if int(result.total) > pageEnd {
continue
}
result.items = append(result.items, rankedRecommendedTender{
tender: tender,
rank: rec.Rank,
analysis: rec.Analysis,
tenderRef: tenderRef,
})
}
}
return result, nil
}
@@ -0,0 +1,355 @@
package tender
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/redis/go-redis/v9"
"tm/internal/company"
"tm/pkg/response"
)
const (
recommendedTendersPageCacheKeyPrefix = "ai:recommendations:page:"
recommendationPageCacheReadTimeout = 3 * time.Second
recommendationPageCacheBuildTimeout = 5 * time.Minute
recommendationPageCacheConcurrency = 8
)
type cachedRecommendedTendersPage struct {
BuiltAt int64 `json:"built_at"`
CompanyID string `json:"company_id"`
Language string `json:"language"`
Items []TenderResponse `json:"items"`
}
func recommendedTendersPageCacheKey(companyID, language string) string {
return recommendedTendersPageCacheKeyPrefix +
strings.TrimSpace(companyID) + ":" +
strings.ToLower(strings.TrimSpace(language))
}
// ScheduleRefreshRecommendedTendersPageCache rebuilds the page cache for the service default language.
func (s *tenderService) ScheduleRefreshRecommendedTendersPageCache(companyID string) {
s.schedulePageCacheRefreshForLanguage(companyID, s.defaultLanguage)
}
// InvalidateRecommendedTendersPageCache drops the cached page for the service default language.
func (s *tenderService) InvalidateRecommendedTendersPageCache(companyID string) {
companyID = strings.TrimSpace(companyID)
if companyID == "" || s.redisClient == nil {
return
}
ctx, cancel := context.WithTimeout(context.Background(), recommendationPageCacheReadTimeout)
defer cancel()
s.deleteRecommendedTendersPageCache(ctx, companyID, s.defaultLanguage)
}
func (s *tenderService) schedulePageCacheRefreshForLanguage(companyID, language string) {
companyID = strings.TrimSpace(companyID)
language = strings.ToLower(strings.TrimSpace(language))
if companyID == "" || language == "" || s.redisClient == nil {
return
}
go func() {
ctx, cancel := context.WithTimeout(context.Background(), recommendationPageCacheBuildTimeout)
defer cancel()
if err := s.buildRecommendedTendersPageCache(ctx, companyID, language); err != nil {
s.logger.Warn("Failed to refresh recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
}
}()
}
func (s *tenderService) buildRecommendedTendersPageCache(ctx context.Context, companyID, language string) error {
recommendations, err := s.companyService.GetAIRecommendations(ctx, companyID)
if err != nil {
return err
}
if len(recommendations) == 0 {
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil
}
items, err := s.buildAllRankedRecommendations(ctx, recommendations)
if err != nil {
return err
}
if len(items) == 0 {
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil
}
responses := s.buildRecommendedTenderListResponses(items, language)
page := cachedRecommendedTendersPage{
BuiltAt: time.Now().Unix(),
CompanyID: companyID,
Language: language,
Items: responses,
}
encoded, err := json.Marshal(page)
if err != nil {
return fmt.Errorf("encode recommended tenders page cache: %w", err)
}
writeCtx, cancel := context.WithTimeout(ctx, recommendationPageCacheReadTimeout)
defer cancel()
if err := s.redisClient.Set(writeCtx, recommendedTendersPageCacheKey(companyID, language), string(encoded), s.pageCacheTTL); err != nil {
return fmt.Errorf("store recommended tenders page cache: %w", err)
}
s.logger.Info("Recommended tenders page cache built", map[string]interface{}{
"company_id": companyID,
"language": language,
"count": len(responses),
})
return nil
}
func (s *tenderService) getRecommendedTendersPageCache(ctx context.Context, companyID, language string) ([]TenderResponse, bool) {
if s.redisClient == nil {
return nil, false
}
readCtx, cancel := context.WithTimeout(ctx, recommendationPageCacheReadTimeout)
defer cancel()
raw, err := s.redisClient.Get(readCtx, recommendedTendersPageCacheKey(companyID, language))
if err != nil {
if err != redis.Nil {
s.logger.Warn("Failed to read recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
}
return nil, false
}
var page cachedRecommendedTendersPage
if err := json.Unmarshal([]byte(raw), &page); err != nil {
s.logger.Warn("Failed to decode recommended tenders page cache", map[string]interface{}{
"company_id": companyID,
"language": language,
"error": err.Error(),
})
s.deleteRecommendedTendersPageCache(ctx, companyID, language)
return nil, false
}
if len(page.Items) == 0 {
return nil, false
}
return page.Items, true
}
func (s *tenderService) deleteRecommendedTendersPageCache(ctx context.Context, companyID, language string) {
if s.redisClient == nil {
return
}
_ = s.redisClient.Del(ctx, recommendedTendersPageCacheKey(companyID, language))
}
func (s *tenderService) buildAllRankedRecommendations(
ctx context.Context,
recommendations []company.RecommendedTenderResponse,
) ([]rankedRecommendedTender, error) {
seenTenderIDs := make(map[string]struct{}, len(recommendations))
out := make([]rankedRecommendedTender, 0, len(recommendations))
for start := 0; start < len(recommendations); start += recommendationResolveBatchSize {
end := start + recommendationResolveBatchSize
if end > len(recommendations) {
end = len(recommendations)
}
tenderByRef, err := s.resolveRecommendedTenders(ctx, recommendations[start:end])
if err != nil {
return nil, err
}
for _, rec := range recommendations[start:end] {
tenderRef := strings.TrimSpace(rec.TenderID)
tender, ok := tenderByRef[tenderRef]
if !ok {
continue
}
if markResolvedRecommendedTenderSeen(seenTenderIDs, tender, tenderRef) {
continue
}
out = append(out, rankedRecommendedTender{
tender: tender,
rank: rec.Rank,
analysis: rec.Analysis,
tenderRef: tenderRef,
})
}
}
return out, nil
}
func (s *tenderService) isRecommendedPageCacheStale(ctx context.Context, companyID string, cachedCount int) bool {
companyID = strings.TrimSpace(companyID)
if companyID == "" || cachedCount == 0 {
return false
}
recommendations, err := s.companyService.GetAIRecommendations(ctx, companyID)
if err != nil || len(recommendations) == 0 {
return false
}
return cachedCount < len(recommendations)
}
func (s *tenderService) recommendFromPageCache(
ctx context.Context,
companyID string,
companyIDs []string,
form *SearchForm,
pagination *response.Pagination,
lang string,
) (*SearchResponse, bool) {
var cached []TenderResponse
if len(companyIDs) > 0 {
cached = s.mergeCachedRecommendationPages(ctx, companyIDs, lang)
} else {
items, ok := s.getRecommendedTendersPageCache(ctx, companyID, lang)
if !ok {
return nil, false
}
cached = items
}
if len(cached) == 0 {
return nil, false
}
if len(companyIDs) == 0 && s.isRecommendedPageCacheStale(ctx, companyID, len(cached)) {
return nil, false
}
excluded, excludedByCompany, err := s.loadRecommendationExclusions(ctx, form, companyID, companyIDs)
if err != nil {
return nil, false
}
now := time.Now().Unix()
filtered := filterCachedRecommendedTenders(cached, form, companyIDs, excluded, excludedByCompany, now)
total := int64(len(filtered))
start := pagination.Offset
if start > len(filtered) {
start = len(filtered)
}
end := start + pagination.Limit
if end > len(filtered) {
end = len(filtered)
}
paged := append([]TenderResponse(nil), filtered[start:end]...)
paged = s.enrichRecommendedTenderResponsesParallel(ctx, paged, lang)
return &SearchResponse{
Tenders: paged,
Metadata: pagination.ListMeta(total, "", end < len(filtered), start),
}, true
}
func (s *tenderService) mergeCachedRecommendationPages(ctx context.Context, companyIDs []string, lang string) []TenderResponse {
normalized := normalizeRecommendationCompanyScope(companyIDs)
if len(normalized) == 0 {
return nil
}
lists := make([][]TenderResponse, 0, len(normalized))
for _, companyID := range normalized {
items, ok := s.getRecommendedTendersPageCache(ctx, companyID, lang)
if !ok || len(items) == 0 {
return nil
}
lists = append(lists, items)
}
merged := make([]TenderResponse, 0)
for _, list := range lists {
merged = append(merged, list...)
}
return dedupeCachedRecommendedTendersByID(merged)
}
func dedupeCachedRecommendedTendersByID(items []TenderResponse) []TenderResponse {
seen := make(map[string]struct{}, len(items))
out := make([]TenderResponse, 0, len(items))
for _, item := range items {
key := strings.TrimSpace(item.ID)
if key == "" {
key = strings.TrimSpace(item.ProcedureRef)
}
if key == "" {
continue
}
if _, exists := seen[key]; exists {
continue
}
seen[key] = struct{}{}
out = append(out, item)
}
return out
}
func filterCachedRecommendedTenders(
items []TenderResponse,
form *SearchForm,
companyIDs []string,
excluded map[string]struct{},
excludedByCompany map[string]map[string]struct{},
now int64,
) []TenderResponse {
if len(items) == 0 {
return nil
}
out := make([]TenderResponse, 0, len(items))
for _, item := range items {
tender := tenderFromRecommendationResponse(item)
tenderRef := strings.TrimSpace(item.ProcedureRef)
if len(companyIDs) > 0 {
if isRecommendedTenderExcludedForAllCompanies(tender, tenderRef, excludedByCompany, companyIDs) {
continue
}
} else if isRecommendedTenderExcluded(tender, tenderRef, excluded) {
continue
}
if form.OnlyActiveDeadlines && !tender.IsRecommendable(now) {
continue
}
if len(form.Status) > 0 && !statusAllowed(tender.Status, form.Status) {
continue
}
out = append(out, item)
}
return out
}
// RecommendedTendersPageCacheKeyPrefix is shared with company invalidation logic.
const RecommendedTendersPageCacheKeyPrefix = recommendedTendersPageCacheKeyPrefix
// RecommendedTendersPageCacheKey builds the Redis key for a cached recommendation page.
func RecommendedTendersPageCacheKey(companyID, language string) string {
return recommendedTendersPageCacheKey(companyID, language)
}
@@ -0,0 +1,44 @@
package tender
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestFilterCachedRecommendedTendersAppliesActiveAndRejectedFilters(t *testing.T) {
activeID, _ := bson.ObjectIDFromHex("507f1f77bcf86cd799439011")
expiredID, _ := bson.ObjectIDFromHex("507f1f77bcf86cd799439012")
items := []TenderResponse{
{
ID: activeID.Hex(),
Status: TenderStatusActive,
PublicationDate: 1,
SubmissionDeadline: 9999999999,
ProcedureRef: "PROC_folder-a/notice-a",
Rank: 1,
},
{
ID: expiredID.Hex(),
Status: TenderStatusExpired,
ProcedureRef: "PROC_folder-b/notice-b",
Rank: 2,
},
}
form := &SearchForm{
OnlyActiveDeadlines: true,
ExcludeRejectedTenders: true,
Status: []string{string(TenderStatusActive)},
}
excluded := map[string]struct{}{expiredID.Hex(): {}}
got := filterCachedRecommendedTenders(items, form, nil, excluded, nil, 100)
if len(got) != 1 {
t.Fatalf("len = %d, want 1", len(got))
}
if got[0].ID != activeID.Hex() {
t.Fatalf("got ID %q, want active tender", got[0].ID)
}
}
@@ -0,0 +1,86 @@
package tender
import (
"context"
"fmt"
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
"tm/internal/company"
)
type stubRecommendTenderRepository struct {
TenderRepository
batchCalls int
}
func (s *stubRecommendTenderRepository) GetByProcedureReferences(_ context.Context, refs []ProcedureReference) (map[string]Tender, error) {
s.batchCalls++
out := make(map[string]Tender, len(refs))
for i, ref := range refs {
tender := Tender{
ContractFolderID: ref.ContractFolderID,
NoticePublicationID: ref.NoticePublicationID,
Title: ref.Ref,
Status: TenderStatusActive,
}
tender.ID = bsonObjectIDFromCounter(i + s.batchCalls*1000)
out[ref.Ref] = tender
}
return out, nil
}
func bsonObjectIDFromCounter(n int) bson.ObjectID {
var id [12]byte
id[11] = byte(n)
id[10] = byte(n >> 8)
return bson.ObjectID(id)
}
func TestBuildRecommendationPageResolvesInBatchesAndPaginates(t *testing.T) {
repo := &stubRecommendTenderRepository{}
service := &tenderService{
repository: repo,
logger: noopTenderTestLogger{},
}
recommendations := make([]company.RecommendedTenderResponse, 0, 120)
for i := 0; i < 120; i++ {
recommendations = append(recommendations, company.RecommendedTenderResponse{
Rank: i + 1,
TenderID: fmt.Sprintf("PROC_folder/notice-%d", i),
Analysis: "fit",
})
}
form := &SearchForm{
OnlyActiveDeadlines: false,
ExcludeRejectedTenders: false,
Status: []string{string(TenderStatusActive)},
}
got, err := service.buildRecommendationPage(
context.Background(),
recommendations,
form,
nil,
nil,
nil,
0,
10,
10,
)
if err != nil {
t.Fatalf("buildRecommendationPage() error = %v", err)
}
if repo.batchCalls != 2 {
t.Fatalf("batchCalls = %d, want 2", repo.batchCalls)
}
if got.total != 120 {
t.Fatalf("total = %d, want 120", got.total)
}
if len(got.items) != 10 {
t.Fatalf("len(items) = %d, want 10", len(got.items))
}
}
+163
View File
@@ -0,0 +1,163 @@
package tender
import (
"context"
"fmt"
"strings"
"sync"
"go.mongodb.org/mongo-driver/v2/bson"
"golang.org/x/sync/errgroup"
"tm/pkg/ai_summarizer"
)
type rankedRecommendedTender struct {
tender Tender
rank int
analysis string
tenderRef string
}
type recommendationTranslationCache struct {
mu sync.Mutex
items map[string]ai_summarizer.StoredTranslation
}
func newRecommendationTranslationCache() *recommendationTranslationCache {
return &recommendationTranslationCache{
items: make(map[string]ai_summarizer.StoredTranslation),
}
}
func recommendationTranslationCacheKey(tenderID, language string) string {
return strings.TrimSpace(tenderID) + "|" + strings.TrimSpace(language)
}
func (c *recommendationTranslationCache) get(tenderID, language string) (ai_summarizer.StoredTranslation, bool) {
if c == nil {
return ai_summarizer.StoredTranslation{}, false
}
c.mu.Lock()
defer c.mu.Unlock()
stored, ok := c.items[recommendationTranslationCacheKey(tenderID, language)]
return stored, ok
}
func (c *recommendationTranslationCache) set(tenderID, language string, stored ai_summarizer.StoredTranslation) {
if c == nil {
return
}
c.mu.Lock()
defer c.mu.Unlock()
c.items[recommendationTranslationCacheKey(tenderID, language)] = stored
}
// buildRecommendedTenderListResponses builds list responses without MinIO translation lookups.
// Recommendation pages use MongoDB text; tender detail endpoints still enrich from storage.
func (s *tenderService) buildRecommendedTenderListResponses(items []rankedRecommendedTender, language string) []TenderResponse {
if len(items) == 0 {
return nil
}
out := make([]TenderResponse, len(items))
for i, item := range items {
resp := item.tender.ToResponseWithLanguage(language)
resp.Rank = item.rank
resp.Analysis = item.analysis
resp.ProcedureRef = item.tenderRef
out[i] = *resp
}
return out
}
func (s *tenderService) enrichRecommendedTenderResponsesParallel(ctx context.Context, responses []TenderResponse, language string) []TenderResponse {
if len(responses) == 0 || s.aiSummarizerStorage == nil {
return responses
}
out := make([]TenderResponse, len(responses))
if len(responses) == 1 {
out[0] = s.enrichRecommendedTenderResponse(ctx, responses[0], language, newRecommendationTranslationCache())
return out
}
translationCache := newRecommendationTranslationCache()
group, groupCtx := errgroup.WithContext(ctx)
group.SetLimit(recommendationPageCacheConcurrency)
for i := range responses {
i := i
group.Go(func() error {
out[i] = s.enrichRecommendedTenderResponse(groupCtx, responses[i], language, translationCache)
return nil
})
}
if err := group.Wait(); err != nil {
for i, resp := range responses {
out[i] = s.enrichRecommendedTenderResponse(ctx, resp, language, translationCache)
}
}
return out
}
func (s *tenderService) enrichRecommendedTenderResponse(
ctx context.Context,
resp TenderResponse,
language string,
cache *recommendationTranslationCache,
) TenderResponse {
enriched := resp
tender := tenderFromRecommendationResponse(resp)
s.enrichWithTranslationTimeout(
ctx,
&tender,
&enriched,
language,
cache,
recommendTranslationEnrichmentTimeout,
)
return enriched
}
func tenderFromRecommendationResponse(item TenderResponse) Tender {
var tender Tender
if id, err := bson.ObjectIDFromHex(strings.TrimSpace(item.ID)); err == nil {
tender.ID = id
}
tender.ContractFolderID = item.ContractFolderID
tender.NoticePublicationID = item.NoticePublicationID
tender.RelatedNoticePublicationIDs = append([]string(nil), item.RelatedNoticePublicationIDs...)
tender.Status = item.Status
tender.PublicationDate = item.PublicationDate
tender.SubmissionDeadline = item.SubmissionDeadline
tender.TenderDeadline = item.TenderDeadline
return tender
}
func (s *tenderService) loadRecommendationExclusions(
ctx context.Context,
form *SearchForm,
companyID string,
companyIDs []string,
) (map[string]struct{}, map[string]map[string]struct{}, error) {
if !form.ExcludeRejectedTenders {
return nil, nil, nil
}
if len(companyIDs) > 0 {
excludedByCompany, err := s.buildRejectedTenderIDSetsForCompanies(ctx, companyIDs)
if err != nil {
return nil, nil, fmt.Errorf("failed to build recommendation exclusions: %w", err)
}
return nil, excludedByCompany, nil
}
excluded, err := s.buildRejectedTenderIDSet(ctx, companyID)
if err != nil {
return nil, nil, fmt.Errorf("failed to build recommendation exclusions: %w", err)
}
return excluded, nil, nil
}
@@ -0,0 +1,88 @@
package tender
import (
"context"
"io"
"sync/atomic"
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
"tm/pkg/ai_summarizer"
)
type countingTranslationStorage struct {
calls atomic.Int32
}
func (s *countingTranslationStorage) GetSummaryFromStorage(context.Context, string, string) (string, error) {
return "", ai_summarizer.ErrSummaryNotReady
}
func (s *countingTranslationStorage) GetDocumentSummariesFromStorage(context.Context, string, string) ([]ai_summarizer.DocumentSummary, error) {
return nil, nil
}
func (s *countingTranslationStorage) GetTranslationFromStorage(context.Context, string, string, string) (ai_summarizer.StoredTranslation, error) {
s.calls.Add(1)
return ai_summarizer.StoredTranslation{
Title: "Translated title",
Description: "Translated description",
}, nil
}
func (s *countingTranslationStorage) ListDocuments(context.Context, string, string) ([]ai_summarizer.StoredDocument, error) {
return nil, nil
}
func (s *countingTranslationStorage) DownloadDocument(context.Context, string) (io.ReadCloser, error) {
return nil, ai_summarizer.ErrObjectNotFound
}
func TestBuildRecommendedTenderListResponsesPreservesOrderWithoutStorage(t *testing.T) {
storage := &countingTranslationStorage{}
service := &tenderService{
aiSummarizerStorage: storage,
defaultLanguage: "en",
logger: noopTenderTestLogger{},
}
items := make([]rankedRecommendedTender, 0, 4)
ids := []string{
"507f1f77bcf86cd799439011",
"507f1f77bcf86cd799439012",
"507f1f77bcf86cd799439013",
"507f1f77bcf86cd799439014",
}
for i, idHex := range ids {
tender := Tender{
ContractFolderID: "folder",
NoticePublicationID: "notice",
Title: "Original",
}
tender.ID, _ = bson.ObjectIDFromHex(idHex)
items = append(items, rankedRecommendedTender{
tender: tender,
rank: i + 1,
analysis: "fit",
tenderRef: "PROC_folder/notice",
})
}
got := service.buildRecommendedTenderListResponses(items, "fr")
if len(got) != len(items) {
t.Fatalf("len = %d, want %d", len(got), len(items))
}
for i, resp := range got {
wantRank := i + 1
if resp.Rank != wantRank {
t.Fatalf("got[%d].Rank = %d, want %d", i, resp.Rank, wantRank)
}
if resp.Title != "Original" {
t.Fatalf("got[%d].Title = %q, want MongoDB title without MinIO lookup", i, resp.Title)
}
}
if storage.calls.Load() != 0 {
t.Fatalf("translation lookups = %d, want 0", storage.calls.Load())
}
}
@@ -0,0 +1,136 @@
package tender
import (
"context"
"io"
"testing"
"tm/pkg/ai_summarizer"
"tm/pkg/logger"
)
type noopTenderTestLogger struct{}
func (noopTenderTestLogger) Debug(string, map[string]interface{}) {}
func (noopTenderTestLogger) Info(string, map[string]interface{}) {}
func (noopTenderTestLogger) Warn(string, map[string]interface{}) {}
func (noopTenderTestLogger) Error(string, map[string]interface{}) {}
func (noopTenderTestLogger) Fatal(string, map[string]interface{}) {}
func (l noopTenderTestLogger) WithFields(map[string]interface{}) logger.Logger {
return l
}
func (noopTenderTestLogger) Sync() error { return nil }
type stubRecommendationTranslationStorage struct {
translations map[string]ai_summarizer.StoredTranslation
}
func (s stubRecommendationTranslationStorage) GetSummaryFromStorage(context.Context, string, string) (string, error) {
return "", ai_summarizer.ErrSummaryNotReady
}
func (s stubRecommendationTranslationStorage) GetDocumentSummariesFromStorage(context.Context, string, string) ([]ai_summarizer.DocumentSummary, error) {
return nil, nil
}
func (s stubRecommendationTranslationStorage) GetTranslationFromStorage(_ context.Context, contractFolderID, noticePublicationID, language string) (ai_summarizer.StoredTranslation, error) {
key := contractFolderID + "|" + noticePublicationID + "|" + language
if translation, ok := s.translations[key]; ok {
return translation, nil
}
return ai_summarizer.StoredTranslation{}, ai_summarizer.ErrTranslationNotReady
}
func (s stubRecommendationTranslationStorage) ListDocuments(context.Context, string, string) ([]ai_summarizer.StoredDocument, error) {
return nil, nil
}
func (s stubRecommendationTranslationStorage) DownloadDocument(context.Context, string) (io.ReadCloser, error) {
return nil, ai_summarizer.ErrObjectNotFound
}
func TestBuildRecommendedTenderResponseAppliesStoredTranslation(t *testing.T) {
service := &tenderService{
aiSummarizerStorage: stubRecommendationTranslationStorage{
translations: map[string]ai_summarizer.StoredTranslation{
"folder-1|00123456-2026|fr": {
Title: "Titre traduit",
Description: "Description traduite",
},
},
},
defaultLanguage: "en",
logger: noopTenderTestLogger{},
}
tender := Tender{
ContractFolderID: "folder-1",
NoticePublicationID: "00123456-2026",
Title: "Original title",
Description: "Original description",
}
resp := service.buildRecommendedTenderResponse(
context.Background(),
tender,
3,
"Strong fit",
"PROC_folder-1/00123456-2026",
"fr",
nil,
)
if resp.Title != "Titre traduit" {
t.Fatalf("expected translated title, got %q", resp.Title)
}
if resp.Description != "Description traduite" {
t.Fatalf("expected translated description, got %q", resp.Description)
}
if resp.Language != "fr" {
t.Fatalf("expected response language fr, got %q", resp.Language)
}
if resp.Rank != 3 {
t.Fatalf("expected rank 3, got %d", resp.Rank)
}
if resp.Analysis != "Strong fit" {
t.Fatalf("expected analysis to be preserved, got %q", resp.Analysis)
}
if resp.ProcedureRef != "PROC_folder-1/00123456-2026" {
t.Fatalf("expected procedure ref to be preserved, got %q", resp.ProcedureRef)
}
}
func TestBuildRecommendedTenderResponseKeepsOriginalTextWithoutStoredTranslation(t *testing.T) {
service := &tenderService{
aiSummarizerStorage: stubRecommendationTranslationStorage{},
defaultLanguage: "en",
logger: noopTenderTestLogger{},
}
tender := Tender{
ContractFolderID: "folder-1",
NoticePublicationID: "00123456-2026",
Title: "Original title",
Description: "Original description",
}
resp := service.buildRecommendedTenderResponse(
context.Background(),
tender,
1,
"Fallback",
"PROC_folder-1/00123456-2026",
"fr",
nil,
)
if resp.Title != "Original title" {
t.Fatalf("expected original title, got %q", resp.Title)
}
if resp.Description != "Original description" {
t.Fatalf("expected original description, got %q", resp.Description)
}
if resp.Language != "" {
t.Fatalf("expected empty language when no translation exists, got %q", resp.Language)
}
}
+227 -57
View File
@@ -13,6 +13,9 @@ import (
"go.mongodb.org/mongo-driver/v2/mongo"
)
// maxContractFolderInClause caps each MongoDB $in batch for documents_scraped MinIO cross-check filters.
const maxContractFolderInClause = 5000
// TenderRepository interface defines tender data access methods
type TenderRepository interface {
// Tender CRUD operations
@@ -27,7 +30,6 @@ type TenderRepository interface {
// GetLatestByContractFolderIDs returns the most recently updated tender per distinct contract_folder_id.
GetLatestByContractFolderIDs(ctx context.Context, contractFolderIDs []string) (map[string]*Tender, error)
GetByProcurementProjectID(ctx context.Context, procurementProjectID string) (*Tender, error)
FindTendersWithContentXML(ctx context.Context, limit, skip int) ([]Tender, error)
GetTenderCountByCountry(ctx context.Context) (map[string]int64, error)
GetTenderCountByType(ctx context.Context) (map[string]int64, error)
GetTenderCountByClassification(ctx context.Context) (map[string]int64, error)
@@ -64,10 +66,17 @@ func tenderSearchListProjection() bson.M {
"modifications": 0,
"source_file_url": 0,
"source_file_name": 0,
// Not used by list response mapping; can be large per tender.
// Detail-only or bulky fields; list rows use title and summary metadata only.
"description": 0,
"ai_overall_summary": 0,
"procurement_lots": 0,
"awarded": 0,
"cancellation_reason": 0,
"suspension_reason": 0,
"translations": 0,
"organizations": 0,
"review_organization": 0,
"processing_metadata.enrichment_data": 0,
"processing_metadata.translated_data": 0,
"processing_metadata.parsing_errors": 0,
"processing_metadata.validation_errors": 0,
@@ -123,6 +132,15 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Te
*orm.NewIndex("main_classification_idx", bson.D{{Key: "main_classification", Value: 1}}),
*orm.NewIndex("publication_date_idx", bson.D{{Key: "publication_date", Value: -1}}),
*orm.NewIndex("tender_deadline_idx", bson.D{{Key: "tender_deadline", Value: 1}}),
*orm.NewIndex("submission_deadline_idx", bson.D{{Key: "submission_deadline", Value: 1}}),
*orm.NewIndex("closing_soon_submission_idx", bson.D{
{Key: "submission_deadline", Value: 1},
{Key: "status", Value: 1},
}),
*orm.NewIndex("closing_soon_tender_deadline_idx", bson.D{
{Key: "tender_deadline", Value: 1},
{Key: "status", Value: 1},
}),
*orm.NewIndex("estimated_value_idx", bson.D{{Key: "estimated_value", Value: -1}}),
*orm.NewIndex("created_at_idx", bson.D{{Key: "created_at", Value: -1}}),
// Stable keyset pagination for default admin list (created_at desc).
@@ -165,6 +183,13 @@ func NewRepository(mongoManager *orm.ConnectionManager, logger logger.Logger) Te
{Key: "processing_metadata.documents_scraped", Value: 1},
{Key: "processing_metadata.documents_scraped_at", Value: -1},
}),
// Panel list: documents_scraped=true sorted by created_at desc (keyset pagination).
*orm.NewIndex("documents_scraped_created_at_id_idx", bson.D{
{Key: "created_at", Value: -1},
{Key: "_id", Value: -1},
}).WithPartialFilterExpression(bson.M{
"processing_metadata.documents_scraped": true,
}),
*orm.NewIndex("scraped_documents_scraped_at_idx", bson.D{
{Key: "scraped_documents.scraped_at", Value: 1},
}),
@@ -238,25 +263,49 @@ func (r *tenderRepository) GetByIDs(ctx context.Context, ids []string) ([]Tender
return nil, nil
}
objectIDs := make([]bson.ObjectID, 0, len(ids))
for _, id := range ids {
objectID, err := bson.ObjectIDFromHex(id)
const batchSize = 500
out := make([]Tender, 0, len(ids))
for start := 0; start < len(ids); start += batchSize {
end := start + batchSize
if end > len(ids) {
end = len(ids)
}
batch := ids[start:end]
objectIDs := make([]bson.ObjectID, 0, len(batch))
for _, id := range batch {
objectID, err := bson.ObjectIDFromHex(strings.TrimSpace(id))
if err != nil {
return nil, err
r.logger.Warn("Skipping invalid tender ID in batch lookup", map[string]interface{}{
"tender_id": id,
"error": err.Error(),
})
continue
}
objectIDs = append(objectIDs, objectID)
}
if len(objectIDs) == 0 {
continue
}
result, err := r.ormRepo.FindAll(ctx, bson.M{"_id": bson.M{"$in": objectIDs}}, orm.NewPaginationBuilder().Build())
pagination := orm.Pagination{
Limit: len(objectIDs),
SortField: "_id",
SortOrder: -1,
SkipCount: true,
}
result, err := r.ormRepo.FindAll(ctx, bson.M{"_id": bson.M{"$in": objectIDs}}, pagination)
if err != nil {
r.logger.Error("Failed to get tenders by IDs", map[string]interface{}{
"count": len(ids),
"count": len(batch),
"error": err.Error(),
})
return nil, err
}
out = append(out, result.Items...)
}
return result.Items, nil
return out, nil
}
// GetByContractNoticeID retrieves a tender by contract notice ID
@@ -343,58 +392,167 @@ func (r *tenderRepository) GetByProcedureReference(ctx context.Context, contract
return &result.Items[0], nil
}
// GetByProcedureReferences loads tenders for multiple AI procedure refs in one query.
const procedureReferenceLookupBatchSize = 250
const contractFolderLookupBatchSize = 250
// GetByProcedureReferences loads tenders for multiple AI procedure refs.
// It combines indexed notice lookups with contract-folder candidates so older notice
// references still resolve after tender merges (same behavior as before the perf refactor).
func (r *tenderRepository) GetByProcedureReferences(ctx context.Context, refs []ProcedureReference) (map[string]Tender, error) {
out := make(map[string]Tender)
if len(refs) == 0 {
return out, nil
}
orClauses := make([]bson.M, 0, len(refs))
folderIDs := make([]string, 0, len(refs))
noticeIDs := make([]string, 0, len(refs))
for _, ref := range refs {
folder := strings.TrimSpace(ref.ContractFolderID)
notice := strings.TrimSpace(ref.NoticePublicationID)
if folder == "" || notice == "" {
continue
}
orClauses = append(orClauses, bson.M{
"contract_folder_id": folder,
"$or": []bson.M{
{"notice_publication_id": notice},
{"related_notice_publication_ids": notice},
},
})
}
if len(orClauses) == 0 {
return out, nil
folderIDs = append(folderIDs, folder)
noticeIDs = append(noticeIDs, notice)
}
filter := bson.M{"$or": orClauses}
uniqueNotices := uniqueNonEmptyTrimmedStrings(noticeIDs)
noticeCandidates := make([]Tender, 0, len(uniqueNotices))
for start := 0; start < len(uniqueNotices); start += procedureReferenceLookupBatchSize {
end := start + procedureReferenceLookupBatchSize
if end > len(uniqueNotices) {
end = len(uniqueNotices)
}
batch, err := r.findTendersByNoticePublicationIDs(ctx, uniqueNotices[start:end])
if err != nil {
return nil, err
}
noticeCandidates = append(noticeCandidates, batch...)
}
folderCandidates, err := r.findTendersByContractFolderIDs(ctx, folderIDs)
if err != nil {
return nil, err
}
candidates := mergeTenderCandidates(noticeCandidates, folderCandidates)
return mapProcedureReferencesToTenders(candidates, refs), nil
}
func (r *tenderRepository) findTendersByContractFolderIDs(ctx context.Context, folderIDs []string) ([]Tender, error) {
uniqueFolders := uniqueNonEmptyTrimmedStrings(folderIDs)
if len(uniqueFolders) == 0 {
return nil, nil
}
out := make([]Tender, 0, len(uniqueFolders))
for start := 0; start < len(uniqueFolders); start += contractFolderLookupBatchSize {
end := start + contractFolderLookupBatchSize
if end > len(uniqueFolders) {
end = len(uniqueFolders)
}
batch := uniqueFolders[start:end]
findLimit := len(batch) * 5
if minLimit := len(batch) + 1; findLimit < minLimit {
findLimit = minLimit
}
filter := bson.M{"contract_folder_id": bson.M{"$in": batch}}
pagination := orm.Pagination{
Limit: len(orClauses) * 2,
Limit: findLimit,
SortField: "updated_at",
SortOrder: -1,
SkipCount: true,
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination)
if err != nil {
r.logger.Error("Failed to get tenders by procedure references", map[string]interface{}{
"count": len(orClauses),
r.logger.Error("Failed to get tenders by contract folder IDs", map[string]interface{}{
"count": len(batch),
"error": err.Error(),
})
return nil, err
}
out = append(out, result.Items...)
}
return out, nil
}
func mergeTenderCandidates(groups ...[]Tender) []Tender {
seen := make(map[string]struct{})
out := make([]Tender, 0)
for _, group := range groups {
for i := range group {
tenderID := strings.TrimSpace(group[i].GetID())
if tenderID == "" {
out = append(out, group[i])
continue
}
if _, exists := seen[tenderID]; exists {
continue
}
seen[tenderID] = struct{}{}
out = append(out, group[i])
}
}
return out
}
func (r *tenderRepository) findTendersByNoticePublicationIDs(ctx context.Context, noticeIDs []string) ([]Tender, error) {
if len(noticeIDs) == 0 {
return nil, nil
}
filter := bson.M{
"$or": []bson.M{
{"notice_publication_id": bson.M{"$in": noticeIDs}},
{"related_notice_publication_ids": bson.M{"$in": noticeIDs}},
},
}
pagination := orm.Pagination{
Limit: len(noticeIDs) * 3,
SortField: "updated_at",
SortOrder: -1,
SkipCount: true,
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination)
if err != nil {
r.logger.Error("Failed to get tenders by notice publication IDs", map[string]interface{}{
"count": len(noticeIDs),
"error": err.Error(),
})
return nil, err
}
return result.Items, nil
}
func mapProcedureReferencesToTenders(candidates []Tender, refs []ProcedureReference) map[string]Tender {
byFolder := make(map[string][]Tender)
for i := range candidates {
folder := strings.TrimSpace(candidates[i].ContractFolderID)
if folder == "" {
continue
}
byFolder[folder] = append(byFolder[folder], candidates[i])
}
out := make(map[string]Tender, len(refs))
for _, ref := range refs {
for i := range result.Items {
if tenderMatchesProcedureRef(result.Items[i], ref.ContractFolderID, ref.NoticePublicationID) {
out[ref.Ref] = result.Items[i]
folder := strings.TrimSpace(ref.ContractFolderID)
for _, candidate := range byFolder[folder] {
if tenderMatchesProcedureRef(candidate, ref.ContractFolderID, ref.NoticePublicationID) {
out[ref.Ref] = candidate
break
}
}
}
return out, nil
return out
}
// GetByContractFolderID retrieves a tender by its procedure-level identifier (ContractFolderID).
@@ -520,27 +678,6 @@ func (r *tenderRepository) GetByProcurementProjectID(ctx context.Context, procur
return &result.Items[0], nil
}
// FindTendersWithContentXML returns tenders that still have TED XML (for backfill after notices were deleted).
func (r *tenderRepository) FindTendersWithContentXML(ctx context.Context, limit, skip int) ([]Tender, error) {
filter := bson.M{"content_xml": bson.M{"$exists": true, "$ne": ""}}
pagination := orm.Pagination{
Limit: limit,
Skip: skip,
SortField: "_id",
SortOrder: 1,
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination)
if err != nil {
r.logger.Error("Failed to list tenders with content_xml", map[string]interface{}{
"error": err.Error(),
})
return nil, err
}
return result.Items, nil
}
// Update updates an existing tender
func (r *tenderRepository) Update(ctx context.Context, tender *Tender) error {
tender.UpdatedAt = time.Now().Unix()
@@ -593,6 +730,15 @@ func (r *tenderRepository) Search(ctx context.Context, form *SearchForm, paginat
sortOrder = pagination.SortOrder
}
listOpts := orm.ListPaginationOptions{Projection: tenderSearchListProjection()}
if pagination.IncludeTotal {
if pagination.Cursor != "" {
listOpts.IncludeCount = true
}
} else {
listOpts.SkipCount = true
}
mongoPagination, err := orm.BuildListPagination(
pagination.Limit,
pagination.Offset,
@@ -600,7 +746,7 @@ func (r *tenderRepository) Search(ctx context.Context, form *SearchForm, paginat
sort,
sortOrder,
filter,
orm.ListPaginationOptions{Projection: tenderSearchListProjection()},
listOpts,
)
if err != nil {
return nil, err
@@ -1237,11 +1383,7 @@ func (r *tenderRepository) buildSearchFilter(form *SearchForm) bson.M {
}
if form.DocumentsScraped {
folderIDs := dedupeNonEmptyStrings(form.ContractFolderIDsWithDocuments)
scrapedClause := bson.M{"contract_folder_id": bson.M{"$in": folderIDs}}
if len(folderIDs) == 0 {
scrapedClause = bson.M{"_id": bson.M{"$exists": false}}
}
scrapedClause := documentsScrapedSearchClause(form)
if len(filter) == 0 {
return scrapedClause
}
@@ -1260,3 +1402,31 @@ func (r *tenderRepository) buildSearchFilter(form *SearchForm) bson.M {
return filter
}
// documentsScrapedSearchClause restricts list queries to tenders whose contract_folder_id
// currently has documents in MinIO (folder IDs are populated by the service layer).
func documentsScrapedSearchClause(form *SearchForm) bson.M {
if form == nil {
return bson.M{"contract_folder_id": bson.M{"$in": []string{}}}
}
return contractFolderIDsSearchClause(form.ContractFolderIDsWithDocuments)
}
func contractFolderIDsSearchClause(folderIDs []string) bson.M {
if len(folderIDs) == 0 {
return bson.M{"contract_folder_id": bson.M{"$in": []string{}}}
}
if len(folderIDs) <= maxContractFolderInClause {
return bson.M{"contract_folder_id": bson.M{"$in": folderIDs}}
}
or := make([]bson.M, 0, (len(folderIDs)+maxContractFolderInClause-1)/maxContractFolderInClause)
for i := 0; i < len(folderIDs); i += maxContractFolderInClause {
end := i + maxContractFolderInClause
if end > len(folderIDs) {
end = len(folderIDs)
}
or = append(or, bson.M{"contract_folder_id": bson.M{"$in": folderIDs[i:end]}})
}
return bson.M{"$or": or}
}
+32
View File
@@ -0,0 +1,32 @@
package tender
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestBuildSearchFilterDocumentsScrapedWithoutFolderIDsMatchesNothing(t *testing.T) {
repo := &tenderRepository{}
filter := repo.buildSearchFilter(&SearchForm{DocumentsScraped: true})
in, ok := filter["contract_folder_id"].(bson.M)["$in"].([]string)
if !ok || len(in) != 0 {
t.Fatalf("expected empty contract_folder_id $in filter, got %#v", filter)
}
}
func TestTenderSearchListProjectionExcludesHeavyFields(t *testing.T) {
projection := tenderSearchListProjection()
for _, field := range []string{
"content_xml",
"description",
"ai_overall_summary",
"procurement_lots",
"document_summaries",
} {
if projection[field] != 0 {
t.Fatalf("expected %s to be excluded from list projection", field)
}
}
}
+240
View File
@@ -0,0 +1,240 @@
package tender
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
goredis "github.com/redis/go-redis/v9"
"tm/pkg/response"
)
const (
searchListCacheTTL = 60 * time.Second
// searchListCacheStaleTTL bounds how long the default unfiltered list may lag after
// writes. Only isCacheableSearchList queries use this cache (~5 min max staleness).
searchListCacheStaleTTL = 5 * time.Minute
searchListCacheKeyPrefix = "tender:search:list:"
)
type searchListCacheEntry struct {
expiresAt time.Time
staleUntil time.Time
value *SearchResponse
}
type cachedSearchListPayload struct {
Tenders []TenderResponse `json:"tenders"`
Metadata *response.Meta `json:"metadata,omitempty"`
}
func searchResponseToCachePayload(resp *SearchResponse) *cachedSearchListPayload {
if resp == nil {
return nil
}
return &cachedSearchListPayload{
Tenders: resp.Tenders,
Metadata: resp.Metadata,
}
}
func cachePayloadToSearchResponse(payload *cachedSearchListPayload) *SearchResponse {
if payload == nil {
return nil
}
return &SearchResponse{
Tenders: payload.Tenders,
Metadata: payload.Metadata,
}
}
func isCacheableSearchList(form *SearchForm, pagination *response.Pagination) bool {
if form == nil || pagination == nil {
return false
}
if form.HasListFilters() {
return false
}
if form.CompanyID != nil && strings.TrimSpace(*form.CompanyID) != "" {
return false
}
if form.CustomerID != nil && strings.TrimSpace(*form.CustomerID) != "" {
return false
}
if len(form.CompanyIDs) > 0 {
return false
}
if pagination.Cursor != "" || pagination.IncludeTotal || pagination.Offset != 0 {
return false
}
if pagination.Limit <= 0 || pagination.Limit > 20 {
return false
}
sortBy := pagination.SortBy
if sortBy == "" {
sortBy = "created_at"
}
sortOrder := pagination.SortOrder
if sortOrder == "" {
sortOrder = "desc"
}
return sortBy == "created_at" && sortOrder == "desc"
}
func searchListCacheKey(pagination *response.Pagination, language string) string {
return fmt.Sprintf("%d:%s", pagination.Limit, strings.ToLower(strings.TrimSpace(language)))
}
func (s *tenderService) cachedSearchList(ctx context.Context, form *SearchForm, pagination *response.Pagination, language string, load func(context.Context) (*SearchResponse, error)) (*SearchResponse, error) {
if !isCacheableSearchList(form, pagination) {
return load(ctx)
}
// Cached *SearchResponse values (including the Tenders slice) are shared across
// concurrent callers via singleflight and stale hits; treat as read-only.
cacheKey := searchListCacheKey(pagination, language)
redisKey := searchListCacheKeyPrefix + cacheKey
if cached, ok := s.getFreshSearchListCache(cacheKey); ok {
return cached, nil
}
if stale, ok := s.getStaleSearchListCache(cacheKey); ok {
go s.refreshSearchListCache(cacheKey, redisKey, load)
return stale, nil
}
if cached, ok := s.getRedisSearchListCache(ctx, redisKey); ok {
s.storeSearchListCache(cacheKey, cached)
go s.refreshSearchListCache(cacheKey, redisKey, load)
return cached, nil
}
out, err, _ := s.searchListCacheGroup.Do(cacheKey, func() (interface{}, error) {
if cached, ok := s.getFreshSearchListCache(cacheKey); ok {
return cached, nil
}
result, err := load(context.WithoutCancel(ctx))
if err != nil {
return nil, err
}
s.storeSearchListCache(cacheKey, result)
s.storeRedisSearchListCache(context.Background(), redisKey, result)
return result, nil
})
if err != nil {
return nil, err
}
return out.(*SearchResponse), nil
}
func (s *tenderService) refreshSearchListCache(cacheKey, redisKey string, load func(context.Context) (*SearchResponse, error)) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
result, err := load(ctx)
if err != nil {
s.logger.Warn("Failed to refresh tender search list cache", map[string]interface{}{
"cache_key": cacheKey,
"error": err.Error(),
})
return
}
s.storeSearchListCache(cacheKey, result)
s.storeRedisSearchListCache(context.Background(), redisKey, result)
}
func (s *tenderService) getFreshSearchListCache(cacheKey string) (*SearchResponse, bool) {
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
entry, ok := s.searchListCache[cacheKey]
if !ok || time.Now().After(entry.expiresAt) {
return nil, false
}
return entry.value, true
}
func (s *tenderService) getStaleSearchListCache(cacheKey string) (*SearchResponse, bool) {
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
entry, ok := s.searchListCache[cacheKey]
if !ok || time.Now().After(entry.staleUntil) {
if ok {
delete(s.searchListCache, cacheKey)
}
return nil, false
}
return entry.value, true
}
func (s *tenderService) storeSearchListCache(cacheKey string, value *SearchResponse) {
now := time.Now()
s.searchListCacheMu.Lock()
defer s.searchListCacheMu.Unlock()
s.searchListCache[cacheKey] = searchListCacheEntry{
expiresAt: now.Add(searchListCacheTTL),
staleUntil: now.Add(searchListCacheTTL + searchListCacheStaleTTL),
value: value,
}
}
func (s *tenderService) getRedisSearchListCache(ctx context.Context, redisKey string) (*SearchResponse, bool) {
if s.redisClient == nil {
return nil, false
}
raw, err := s.redisClient.Get(ctx, redisKey)
if err != nil {
if err != goredis.Nil {
s.logger.Warn("Failed to read tender search list cache from Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
}
return nil, false
}
var cached cachedSearchListPayload
if err := json.Unmarshal([]byte(raw), &cached); err != nil {
s.logger.Warn("Failed to decode tender search list cache from Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
_ = s.redisClient.Del(ctx, redisKey)
return nil, false
}
return cachePayloadToSearchResponse(&cached), true
}
func (s *tenderService) storeRedisSearchListCache(ctx context.Context, redisKey string, value *SearchResponse) {
if s.redisClient == nil || value == nil {
return
}
encoded, err := json.Marshal(searchResponseToCachePayload(value))
if err != nil {
s.logger.Warn("Failed to encode tender search list cache for Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
return
}
if err := s.redisClient.Set(ctx, redisKey, string(encoded), searchListCacheTTL+searchListCacheStaleTTL); err != nil {
s.logger.Warn("Failed to store tender search list cache in Redis", map[string]interface{}{
"cache_key": redisKey,
"error": err.Error(),
})
}
}
+197 -83
View File
@@ -10,13 +10,18 @@ import (
"path"
"path/filepath"
"strings"
"sync"
"time"
"tm/internal/company"
"tm/internal/customer"
"tm/pkg/ai_summarizer"
"tm/pkg/logger"
orm "tm/pkg/mongo"
"tm/pkg/redis"
"tm/pkg/response"
"golang.org/x/sync/errgroup"
"golang.org/x/sync/singleflight"
)
// AISummarizerClient defines the interface for on-demand AI operations.
@@ -52,6 +57,8 @@ type Service interface {
// Tender operations
GetByID(ctx context.Context, id string, language string) (*TenderResponse, error)
GetByIDs(ctx context.Context, ids []string) (map[string]*TenderResponse, error)
// ResolveMongoID maps a MongoDB id or AI procedure ref to the canonical tender MongoDB id.
ResolveMongoID(ctx context.Context, tenderRef string) (string, error)
Update(ctx context.Context, req UpdateTenderRequest) (*TenderResponse, error)
Delete(ctx context.Context, id string) error
Search(ctx context.Context, req *SearchForm, pagination *response.Pagination) (*SearchResponse, error)
@@ -83,15 +90,26 @@ type tenderService struct {
repository TenderRepository
companyService company.Service
customerService customer.Service
rejectedTenderLister CompanyRejectedTenderLister
logger logger.Logger
aiSummarizerClient AISummarizerClient
aiSummarizerStorage AISummarizerStorage
redisClient redis.Client
pageCacheTTL time.Duration
defaultLanguage string
searchListCacheMu sync.Mutex
searchListCache map[string]searchListCacheEntry
searchListCacheGroup singleflight.Group
contractFoldersCacheMu sync.Mutex
contractFoldersCache contractFoldersWithDocsCacheEntry
contractFoldersCacheGroup singleflight.Group
}
const (
aiSummaryEnrichmentTimeout = 2 * time.Second
aiTranslationEnrichmentTimeout = 2 * time.Second
recommendTranslationEnrichmentTimeout = 5 * time.Second
)
var (
@@ -100,7 +118,18 @@ var (
)
// NewService creates a new tender service
func NewService(repository TenderRepository, companyService company.Service, customerService customer.Service, logger logger.Logger, aiClient AISummarizerClient, aiStorage AISummarizerStorage, defaultLanguage string) Service {
func NewService(
repository TenderRepository,
companyService company.Service,
customerService customer.Service,
rejectedTenderLister CompanyRejectedTenderLister,
logger logger.Logger,
aiClient AISummarizerClient,
aiStorage AISummarizerStorage,
redisClient redis.Client,
recommendationPageCacheTTL time.Duration,
defaultLanguage string,
) Service {
if strings.TrimSpace(defaultLanguage) == "" {
defaultLanguage = "en"
}
@@ -109,10 +138,14 @@ func NewService(repository TenderRepository, companyService company.Service, cus
repository: repository,
companyService: companyService,
customerService: customerService,
rejectedTenderLister: rejectedTenderLister,
logger: logger,
aiSummarizerClient: aiClient,
aiSummarizerStorage: aiStorage,
redisClient: redisClient,
pageCacheTTL: recommendationPageCacheTTL,
defaultLanguage: strings.ToLower(defaultLanguage),
searchListCache: make(map[string]searchListCacheEntry),
}
}
@@ -160,10 +193,39 @@ func (s *tenderService) GetByIDs(ctx context.Context, ids []string) (map[string]
return out, nil
}
// ResolveMongoID maps a MongoDB id or AI procedure ref (PROC_{folder}/{notice}) to the canonical tender id.
func (s *tenderService) ResolveMongoID(ctx context.Context, tenderRef string) (string, error) {
tenderRef = strings.TrimSpace(tenderRef)
if tenderRef == "" {
return "", fmt.Errorf("tender reference is required")
}
if IsMongoObjectIDRef(tenderRef) {
if _, err := s.repository.GetByID(ctx, tenderRef); err != nil {
return "", fmt.Errorf("tender not found")
}
return tenderRef, nil
}
if folder, notice, ok := ParseAIProcedureRef(tenderRef); ok {
t, err := s.repository.GetByProcedureReference(ctx, folder, notice)
if err != nil || t == nil {
return "", fmt.Errorf("tender not found")
}
return t.GetID(), nil
}
if _, err := s.repository.GetByID(ctx, tenderRef); err == nil {
return tenderRef, nil
}
return "", fmt.Errorf("tender not found")
}
// buildDetailResponse maps a tender to API form and enriches translation (MinIO) and summary.
func (s *tenderService) buildDetailResponse(ctx context.Context, tender *Tender, language string) *TenderResponse {
resp := tender.ToResponseWithLanguage("")
s.enrichWithTranslation(ctx, tender, resp, language)
s.enrichWithTranslation(ctx, tender, resp, language, nil)
s.enrichWithAISummary(ctx, tender, resp)
return resp
}
@@ -193,7 +255,18 @@ func (s *tenderService) enrichWithAISummary(ctx context.Context, t *Tender, resp
// enrichWithTranslation overlays title/description from MinIO when the pipeline has finished
// for the requested language. Does not call the AI translate API on GET.
func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, resp *TenderResponse, language string) {
func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, resp *TenderResponse, language string, cache *recommendationTranslationCache) {
s.enrichWithTranslationTimeout(ctx, t, resp, language, cache, aiTranslationEnrichmentTimeout)
}
func (s *tenderService) enrichWithTranslationTimeout(
ctx context.Context,
t *Tender,
resp *TenderResponse,
language string,
cache *recommendationTranslationCache,
enrichmentTimeout time.Duration,
) {
if t == nil || resp == nil {
return
}
@@ -202,7 +275,18 @@ func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, re
}
targetLanguage := s.resolveDetailLanguage(language)
enrichCtx, cancel := context.WithTimeout(ctx, aiTranslationEnrichmentTimeout)
if cache != nil {
if stored, ok := cache.get(t.GetID(), targetLanguage); ok {
if strings.TrimSpace(stored.Title) != "" {
resp.Title = stored.Title
resp.Description = stored.Description
resp.Language = targetLanguage
}
return
}
}
enrichCtx, cancel := context.WithTimeout(ctx, enrichmentTimeout)
defer cancel()
stored, usedNotice, err := s.lookupTranslationForTender(enrichCtx, t, targetLanguage)
@@ -220,6 +304,10 @@ func (s *tenderService) enrichWithTranslation(ctx context.Context, t *Tender, re
return
}
if cache != nil {
cache.set(t.GetID(), targetLanguage, stored)
}
resp.Title = stored.Title
resp.Description = stored.Description
resp.Language = targetLanguage
@@ -844,8 +932,13 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
"to": form.DeadlineTo,
})
// Profile keywords are not used for tender search.
language := s.pickResponseLanguage(form.Language)
return s.cachedSearchList(ctx, form, pagination, language, func(loadCtx context.Context) (*SearchResponse, error) {
return s.searchUncached(loadCtx, form, pagination, language)
})
}
func (s *tenderService) searchUncached(ctx context.Context, form *SearchForm, pagination *response.Pagination, language string) (*SearchResponse, error) {
if form.DocumentsScraped {
if err := s.enrichDocumentsScrapedFilter(ctx, form); err != nil {
return nil, err
@@ -867,7 +960,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
if form.CompanyID == nil || *form.CompanyID == "" {
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -886,7 +979,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
// Continue without match percentage if company not found
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -909,7 +1002,7 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
tenderResponses := make([]TenderResponse, 0, len(page.Items))
for _, tender := range page.Items {
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(s.pickResponseLanguage(form.Language)))
tenderResponses = append(tenderResponses, *tender.ToResponseWithLanguage(language))
}
return &SearchResponse{
@@ -921,19 +1014,38 @@ func (s *tenderService) Search(ctx context.Context, form *SearchForm, pagination
// Recommend retrieves AI-ranked tenders for the authenticated customer's company.
func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, pagination *response.Pagination) (*SearchResponse, error) {
if form.CompanyID == nil || strings.TrimSpace(*form.CompanyID) == "" {
companyIDs := normalizeRecommendationCompanyScope(form.CompanyIDs)
companyID := ""
if form.CompanyID != nil {
companyID = strings.TrimSpace(*form.CompanyID)
}
if companyID == "" && len(companyIDs) == 0 {
return nil, errors.New("company ID is required")
}
if pagination.Cursor != "" {
return nil, fmt.Errorf("%w: cursor pagination is not supported for recommendations", response.ErrInvalidPagination)
}
companyID := strings.TrimSpace(*form.CompanyID)
s.logger.Info("Recommend tenders via AI service", map[string]interface{}{
s.logger.Debug("Recommend tenders request", map[string]interface{}{
"company_id": companyID,
"company_ids": companyIDs,
})
recommendations, err := s.companyService.GetAIRecommendations(ctx, companyID)
lang := s.pickResponseLanguage(form.Language)
if result, ok := s.recommendFromPageCache(ctx, companyID, companyIDs, form, pagination, lang); ok {
return result, nil
}
var (
recommendations []company.RecommendedTenderResponse
err error
)
if len(companyIDs) > 0 {
recommendations, err = s.companyService.GetMergedAIRecommendations(ctx, companyIDs)
} else {
recommendations, err = s.companyService.GetAIRecommendations(ctx, companyID)
}
if err != nil {
return nil, err
}
@@ -941,50 +1053,56 @@ func (s *tenderService) Recommend(ctx context.Context, form *SearchForm, paginat
return s.emptySearchResponse(pagination), nil
}
tenderByRef, err := s.resolveRecommendedTenders(ctx, recommendations)
if len(companyIDs) == 0 && companyID != "" {
s.schedulePageCacheRefreshForLanguage(companyID, lang)
}
excluded, excludedByCompany, err := s.loadRecommendationExclusions(ctx, form, companyID, companyIDs)
if err != nil {
if form.ExcludeRejectedTenders {
s.logger.Error("Failed to build recommendation exclusions", map[string]interface{}{
"company_id": companyID,
"company_ids": companyIDs,
"error": err.Error(),
})
}
return nil, err
}
now := time.Now().Unix()
pageResult, err := s.buildRecommendationPage(
ctx,
recommendations,
form,
companyIDs,
excluded,
excludedByCompany,
now,
pagination.Offset,
pagination.Limit,
)
if err != nil {
return nil, err
}
lang := s.pickResponseLanguage(form.Language)
now := time.Now().Unix()
ordered := make([]TenderResponse, 0, len(recommendations))
for _, rec := range recommendations {
tenderRef := strings.TrimSpace(rec.TenderID)
tender, ok := tenderByRef[tenderRef]
if !ok {
continue
}
if form.OnlyActiveDeadlines && !tender.IsRecommendable(now) {
continue
}
if len(form.Status) > 0 && !statusAllowed(tender.Status, form.Status) {
continue
}
resp := tender.ToResponseWithLanguage(lang)
resp.Rank = rec.Rank
resp.Analysis = rec.Analysis
resp.ProcedureRef = tenderRef
ordered = append(ordered, *resp)
}
total := int64(len(ordered))
start := pagination.Offset
if start > len(ordered) {
start = len(ordered)
}
end := start + pagination.Limit
if end > len(ordered) {
end = len(ordered)
}
paged := s.buildRecommendedTenderListResponses(pageResult.items, lang)
paged = s.enrichRecommendedTenderResponsesParallel(ctx, paged, lang)
return &SearchResponse{
Tenders: ordered[start:end],
Metadata: pagination.ListMeta(total, "", end < len(ordered), start),
Tenders: paged,
Metadata: pagination.ListMeta(pageResult.total, "", int(pagination.Offset)+len(paged) < int(pageResult.total), pagination.Offset),
}, nil
}
func (s *tenderService) buildRecommendedTenderResponse(ctx context.Context, tender Tender, rank int, analysis, tenderRef, language string, cache *recommendationTranslationCache) TenderResponse {
resp := tender.ToResponseWithLanguage(language)
s.enrichWithTranslation(ctx, &tender, resp, language, cache)
resp.Rank = rank
resp.Analysis = analysis
resp.ProcedureRef = tenderRef
return *resp
}
func statusAllowed(status TenderStatus, allowed []string) bool {
for _, s := range allowed {
if string(status) == strings.TrimSpace(s) {
@@ -1654,31 +1772,6 @@ func (s *tenderService) listProceduresWithDocuments(ctx context.Context) ([]ai_s
return procedures, nil
}
func (s *tenderService) enrichDocumentsScrapedFilter(ctx context.Context, form *SearchForm) error {
procedures, err := s.listProceduresWithDocuments(ctx)
if err != nil {
s.logger.Error("Failed to resolve tenders with scraped documents from MinIO", map[string]interface{}{
"error": err.Error(),
})
if errors.Is(err, ai_summarizer.ErrMinIOUnavailable) {
return fmt.Errorf("failed to resolve tenders with scraped documents: MinIO connection unavailable: %w", err)
}
return fmt.Errorf("failed to resolve tenders with scraped documents: %w", err)
}
folderIDs := make([]string, 0, len(procedures))
for _, proc := range procedures {
if proc.DocumentCount <= 0 {
continue
}
if id := strings.TrimSpace(proc.ContractFolderID); id != "" {
folderIDs = append(folderIDs, id)
}
}
form.ContractFolderIDsWithDocuments = folderIDs
return nil
}
func (s *tenderService) resolveRecommendedTenders(ctx context.Context, recommendations []company.RecommendedTenderResponse) (map[string]Tender, error) {
out := make(map[string]Tender)
procRefs := make([]ProcedureReference, 0, len(recommendations))
@@ -1709,29 +1802,50 @@ func (s *tenderService) resolveRecommendedTenders(ctx context.Context, recommend
})
}
var (
byProcedure = make(map[string]Tender)
byMongoID = make(map[string]Tender)
)
group, groupCtx := errgroup.WithContext(ctx)
if len(procRefs) > 0 {
byProcedure, err := s.repository.GetByProcedureReferences(ctx, procRefs)
group.Go(func() error {
byProcedureResult, err := s.repository.GetByProcedureReferences(groupCtx, procRefs)
if err != nil {
return nil, fmt.Errorf("failed to load recommended tenders by procedure reference: %w", err)
return fmt.Errorf("failed to load recommended tenders by procedure reference: %w", err)
}
for ref, tender := range byProcedure {
out[ref] = tender
for ref, tender := range byProcedureResult {
byProcedure[ref] = tender
}
return nil
})
}
if len(mongoIDs) == 0 {
return out, nil
}
tenders, err := s.repository.GetByIDs(ctx, mongoIDs)
if len(mongoIDs) > 0 {
group.Go(func() error {
tenders, err := s.repository.GetByIDs(groupCtx, mongoIDs)
if err != nil {
s.logger.Error("Failed to load recommended tenders by Mongo ID", map[string]interface{}{
"error": err.Error(),
})
return nil, fmt.Errorf("failed to load recommended tenders: %w", err)
return fmt.Errorf("failed to load recommended tenders: %w", err)
}
for i := range tenders {
out[tenders[i].GetID()] = tenders[i]
byMongoID[tenders[i].GetID()] = tenders[i]
}
return nil
})
}
if err := group.Wait(); err != nil {
return nil, err
}
for ref, tender := range byProcedure {
out[ref] = tender
}
for ref, tender := range byMongoID {
out[ref] = tender
}
return out, nil
-1
View File
@@ -148,7 +148,6 @@ func (ta *TenderApproval) Submit() {
// Reject rejects the tender with a reason
func (ta *TenderApproval) Reject() {
ta.Status = ApprovalStatusRejected
ta.SubmissionMode = ""
ta.CreatedAt = time.Now().Unix()
}
+1
View File
@@ -14,4 +14,5 @@ var (
ErrInvalidCompanyID = errors.New("invalid company ID")
ErrUnauthorized = errors.New("unauthorized to perform this action")
ErrInvalidData = errors.New("invalid tender approval data")
ErrSubmissionSyncFailed = errors.New("failed to sync tender submission workflow")
)
+1
View File
@@ -81,6 +81,7 @@ type CompanyTenderApprovalStatsResponse struct {
// ToggleTenderApprovalForm represents the form for toggling a tender approval
type ToggleTenderApprovalForm struct {
TenderID string `json:"tender_id" valid:"required"`
CompanyID *string `json:"company_id,omitempty" valid:"optional"`
Status ApprovalStatus `json:"status" valid:"required,in(submitted|rejected)"`
SubmissionMode SubmissionMode `json:"submission_mode" valid:"required,in(self-apply|partnership)"`
}
+25 -9
View File
@@ -1,7 +1,10 @@
package tender_approval
import (
"errors"
"strconv"
"strings"
"tm/internal/customer"
"tm/pkg/logger"
"tm/pkg/response"
@@ -42,14 +45,30 @@ func (h *Handler) ToggleTenderApproval(c echo.Context) error {
return response.ValidationError(c, "Invalid request data", err.Error())
}
// Get company ID from context
companyID := c.Get("company_id").(string)
if companyID == "" {
return response.Unauthorized(c, "Unauthorized")
requestedCompanyID := strings.TrimSpace(c.QueryParam("company_id"))
if form.CompanyID != nil && strings.TrimSpace(*form.CompanyID) != "" {
requestedCompanyID = strings.TrimSpace(*form.CompanyID)
}
tenderApproval, err := h.service.ToggleTenderApproval(c.Request().Context(), form, companyID)
activeCompanyID, _ := c.Get("company_id").(string)
assignedCompanyIDs, _ := c.Get("company_ids").([]string)
companyID, err := customer.PickAssignedCompanyID(activeCompanyID, assignedCompanyIDs, requestedCompanyID)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
tenderApproval, err := h.service.ToggleTenderApproval(c.Request().Context(), form, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) {
return response.NotFound(c, "Tender not found")
}
if errors.Is(err, ErrSubmissionSyncFailed) {
return response.InternalServerError(c, "Failed to sync tender submission workflow")
}
if err.Error() == "tender approval already exists for this tender and company" {
return response.Conflict(c, err.Error())
}
@@ -176,7 +195,7 @@ func (h *Handler) PublicGetTenderApprovals(c echo.Context) error {
submissionMode = &mode
}
// Parse status filter; default to submitted so "Your Tenders" excludes rejected approvals
// Parse status filter
var status *ApprovalStatus
if statusStr != "" {
statusVal := ApprovalStatus(statusStr)
@@ -184,9 +203,6 @@ func (h *Handler) PublicGetTenderApprovals(c echo.Context) error {
return response.BadRequest(c, "Invalid status value", "Must be 'submitted' or 'rejected'")
}
status = &statusVal
} else {
defaultStatus := ApprovalStatusSubmitted
status = &defaultStatus
}
var from *int64
+43 -5
View File
@@ -3,11 +3,13 @@ package tender_approval
import (
"context"
"errors"
"strings"
"time"
"tm/pkg/logger"
"tm/pkg/mongo"
"go.mongodb.org/mongo-driver/v2/bson"
mongodriver "go.mongodb.org/mongo-driver/v2/mongo"
)
// Repository defines the interface for tender approval data operations
@@ -30,6 +32,7 @@ type Repository interface {
GetTenderApprovalStats(ctx context.Context) (*TenderApprovalStatsResponse, error)
GetCompanyTenderApprovalStats(ctx context.Context, companyID string) (*CompanyTenderApprovalStatsResponse, error)
SearchByCriteria(ctx context.Context, criteria *TenderApprovalSearchCriteria) ([]*TenderApproval, error)
ListRejectedTenderIDsByCompanyID(ctx context.Context, companyID string) ([]string, error)
}
// tenderApprovalRepository implements the Repository interface using the MongoDB ORM
@@ -101,7 +104,7 @@ func (r *tenderApprovalRepository) GetByID(ctx context.Context, id string) (*Ten
tenderApproval, err := r.ormRepo.FindByID(ctx, id)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, errors.New("tender approval not found")
return nil, ErrTenderApprovalNotFound
}
r.logger.Error("Failed to get tender approval by ID", map[string]interface{}{
"error": err.Error(),
@@ -123,7 +126,7 @@ func (r *tenderApprovalRepository) GetByTenderAndCompany(ctx context.Context, te
tenderApproval, err := r.ormRepo.FindOne(ctx, filter)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, errors.New("tender approval not found")
return nil, ErrTenderApprovalNotFound
}
r.logger.Error("Failed to get tender approval by tender and company", map[string]interface{}{
"error": err.Error(),
@@ -513,11 +516,11 @@ func (r *tenderApprovalRepository) GetCompanyTenderApprovalStats(ctx context.Con
rejectedFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusRejected)}
rejectedTenders, _ := r.ormRepo.Count(ctx, rejectedFilter)
// Count by submission mode for this company
selfApplyFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusSubmitted), "submission_mode": string(SubmissionModeSelfApply)}
// Count by submission mode for this company (includes submitted and rejected decisions).
selfApplyFilter := bson.M{"company_id": companyID, "submission_mode": string(SubmissionModeSelfApply)}
selfApplyCount, _ := r.ormRepo.Count(ctx, selfApplyFilter)
partnershipFilter := bson.M{"company_id": companyID, "status": string(ApprovalStatusSubmitted), "submission_mode": string(SubmissionModePartnership)}
partnershipFilter := bson.M{"company_id": companyID, "submission_mode": string(SubmissionModePartnership)}
partnershipCount, _ := r.ormRepo.Count(ctx, partnershipFilter)
stats := &CompanyTenderApprovalStatsResponse{
@@ -533,6 +536,28 @@ func (r *tenderApprovalRepository) GetCompanyTenderApprovalStats(ctx context.Con
return stats, nil
}
// ListRejectedTenderIDsByCompanyID returns distinct tender IDs rejected by a company.
func (r *tenderApprovalRepository) ListRejectedTenderIDsByCompanyID(ctx context.Context, companyID string) ([]string, error) {
pipeline := mongodriver.Pipeline{
{{Key: "$match", Value: bson.M{
"company_id": companyID,
"status": string(ApprovalStatusRejected),
}}},
{{Key: "$group", Value: bson.M{"_id": "$tender_id"}}},
}
results, err := r.ormRepo.Aggregate(ctx, pipeline)
if err != nil {
r.logger.Error("Failed to list rejected tender IDs by company", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
return nil, err
}
return aggregateStringIDs(results), nil
}
// SearchByCriteria searches tender approvals using search criteria
func (r *tenderApprovalRepository) SearchByCriteria(ctx context.Context, criteria *TenderApprovalSearchCriteria) ([]*TenderApproval, error) {
// Convert criteria to search parameters
@@ -549,3 +574,16 @@ func (r *tenderApprovalRepository) SearchByCriteria(ctx context.Context, criteri
// Use the search method
return r.Search(ctx, criteria.TenderID, criteria.CompanyID, status, submissionMode, criteria.CreatedAtFrom, criteria.CreatedAtTo, criteria.Limit, criteria.Offset, "created_at", "desc")
}
func aggregateStringIDs(results []bson.M) []string {
ids := make([]string, 0, len(results))
for _, result := range results {
switch id := result["_id"].(type) {
case string:
if strings.TrimSpace(id) != "" {
ids = append(ids, id)
}
}
}
return ids
}
+113 -164
View File
@@ -4,12 +4,20 @@ import (
"context"
"errors"
"math"
"strings"
"time"
"tm/internal/tender"
"tm/pkg/logger"
"tm/pkg/response"
)
// SubmissionLifecycleHook synchronizes tender submission workflow with approval changes.
type SubmissionLifecycleHook interface {
OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error
OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error
OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error
}
// Service defines business logic for tender approval operations
type Service interface {
// Core tender approval operations
@@ -33,7 +41,7 @@ type Service interface {
SearchTenderApprovalsByCriteria(ctx context.Context, criteria *TenderApprovalSearchCriteria) ([]*TenderApproval, error)
// Business operations
ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID string) (*TenderApprovalWithTenderResponse, error)
ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID, customerID string) (*TenderApprovalWithTenderResponse, error)
// Statistics and analytics
GetTenderApprovalStats(ctx context.Context) (*TenderApprovalStatsResponse, error)
@@ -53,14 +61,16 @@ type tenderApprovalService struct {
repository Repository
tenderService tender.Service
logger logger.Logger
submissionHook SubmissionLifecycleHook
}
// NewService creates a new tender approval service
func NewService(repository Repository, tenderService tender.Service, logger logger.Logger) Service {
func NewService(repository Repository, tenderService tender.Service, logger logger.Logger, submissionHook SubmissionLifecycleHook) Service {
return &tenderApprovalService{
repository: repository,
tenderService: tenderService,
logger: logger,
submissionHook: submissionHook,
}
}
@@ -292,40 +302,12 @@ func (s *tenderApprovalService) GetTenderApprovalsByCompanyIDWithTender(ctx cont
return nil, err
}
// Convert to response with tender details
tenders := s.loadTendersByIDs(ctx, uniqueTenderIDs(tenderApprovals))
responses := &TenderApprovalWithTenderListResponse{
Approvals: []*TenderApprovalWithTenderResponse{},
Approvals: mapApprovalsWithTender(tenderApprovals, tenders, listTenderDetailsFromResponse),
Metadata: &response.Meta{},
}
for _, approval := range tenderApprovals {
// Get tender details
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
s.logger.Warn("Failed to get tender details", map[string]interface{}{
"tender_id": approval.TenderID,
"error": err.Error(),
})
// Continue without tender details rather than failing completely
responses.Approvals = append(responses.Approvals, approval.ToResponseWithTender(nil))
continue
}
// Convert tender response to tender details
tenderDetails := &TenderDetails{
ID: tenderResponse.ID,
Title: tenderResponse.Title,
Description: tenderResponse.Description,
EstimatedValue: tenderResponse.EstimatedValue,
Currency: tenderResponse.Currency,
TenderDeadline: tenderResponse.TenderDeadline,
SubmissionDeadline: tenderResponse.SubmissionDeadline,
CountryCode: tenderResponse.CountryCode,
BuyerOrganization: &OrganizationResponse{Name: tenderResponse.BuyerOrganization.Name},
Status: string(tenderResponse.Status),
}
responses.Approvals = append(responses.Approvals, approval.ToResponseWithTender(tenderDetails))
}
responses.Metadata = &response.Meta{
Total: int(total),
@@ -780,43 +762,8 @@ func (s *tenderApprovalService) GetTenderApprovalsByStatusWithTender(ctx context
return nil, 0, err
}
// Convert to response with tender details
var responses []*TenderApprovalWithTenderResponse
for _, approval := range tenderApprovals {
// Get tender details
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
s.logger.Warn("Failed to get tender details", map[string]interface{}{
"tender_id": approval.TenderID,
"error": err.Error(),
})
// Continue without tender details rather than failing completely
responses = append(responses, approval.ToResponseWithTender(nil))
continue
}
// Convert tender response to tender details
tenderDetails := &TenderDetails{
ID: tenderResponse.ID,
NoticePublicationID: tenderResponse.NoticePublicationID,
PublicationDate: tenderResponse.PublicationDate,
Title: tenderResponse.Title,
Description: tenderResponse.Description,
ProcurementTypeCode: tenderResponse.ProcurementTypeCode,
ProcedureCode: tenderResponse.ProcurementTypeCode,
EstimatedValue: tenderResponse.EstimatedValue,
Currency: tenderResponse.Currency,
Duration: tenderResponse.Duration,
DurationUnit: tenderResponse.DurationUnit,
TenderDeadline: tenderResponse.TenderDeadline,
SubmissionDeadline: tenderResponse.SubmissionDeadline,
CountryCode: tenderResponse.CountryCode,
BuyerOrganization: &OrganizationResponse{Name: tenderResponse.BuyerOrganization.Name},
Status: string(tenderResponse.Status),
}
responses = append(responses, approval.ToResponseWithTender(tenderDetails))
}
tenders := s.loadTendersByIDs(ctx, uniqueTenderIDs(tenderApprovals))
responses := mapApprovalsWithTender(tenderApprovals, tenders, fullTenderDetailsFromResponse)
return responses, total, nil
}
@@ -828,43 +775,8 @@ func (s *tenderApprovalService) GetTenderApprovalsBySubmissionModeWithTender(ctx
return nil, 0, err
}
// Convert to response with tender details
var responses []*TenderApprovalWithTenderResponse
for _, approval := range tenderApprovals {
// Get tender details
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
s.logger.Warn("Failed to get tender details", map[string]interface{}{
"tender_id": approval.TenderID,
"error": err.Error(),
})
// Continue without tender details rather than failing completely
responses = append(responses, approval.ToResponseWithTender(nil))
continue
}
// Convert tender response to tender details
tenderDetails := &TenderDetails{
ID: tenderResponse.ID,
NoticePublicationID: tenderResponse.NoticePublicationID,
PublicationDate: tenderResponse.PublicationDate,
Title: tenderResponse.Title,
Description: tenderResponse.Description,
ProcurementTypeCode: tenderResponse.ProcurementTypeCode,
ProcedureCode: tenderResponse.ProcedureCode,
EstimatedValue: tenderResponse.EstimatedValue,
Currency: tenderResponse.Currency,
Duration: tenderResponse.Duration,
DurationUnit: tenderResponse.DurationUnit,
TenderDeadline: tenderResponse.TenderDeadline,
SubmissionDeadline: tenderResponse.SubmissionDeadline,
CountryCode: tenderResponse.CountryCode,
BuyerOrganization: &OrganizationResponse{Name: tenderResponse.BuyerOrganization.Name},
Status: string(tenderResponse.Status),
}
responses = append(responses, approval.ToResponseWithTender(tenderDetails))
}
tenders := s.loadTendersByIDs(ctx, uniqueTenderIDs(tenderApprovals))
responses := mapApprovalsWithTender(tenderApprovals, tenders, fullTenderDetailsFromResponse)
return responses, total, nil
}
@@ -926,43 +838,8 @@ func (s *tenderApprovalService) ListTenderApprovalsWithTender(ctx context.Contex
total = int64(len(tenderApprovals)) // Use actual count if total count fails
}
// Convert to response with tender details
var responses []*TenderApprovalWithTenderResponse
for _, approval := range tenderApprovals {
// Get tender details
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
s.logger.Warn("Failed to get tender details", map[string]interface{}{
"tender_id": approval.TenderID,
"error": err.Error(),
})
// Continue without tender details rather than failing completely
responses = append(responses, approval.ToResponseWithTender(nil))
continue
}
// Convert tender response to tender details
tenderDetails := &TenderDetails{
ID: tenderResponse.ID,
NoticePublicationID: tenderResponse.NoticePublicationID,
PublicationDate: tenderResponse.PublicationDate,
Title: tenderResponse.Title,
Description: tenderResponse.Description,
ProcurementTypeCode: tenderResponse.ProcurementTypeCode,
ProcedureCode: tenderResponse.ProcedureCode,
EstimatedValue: tenderResponse.EstimatedValue,
Currency: tenderResponse.Currency,
Duration: tenderResponse.Duration,
DurationUnit: tenderResponse.DurationUnit,
TenderDeadline: tenderResponse.TenderDeadline,
SubmissionDeadline: tenderResponse.SubmissionDeadline,
CountryCode: tenderResponse.CountryCode,
BuyerOrganization: &OrganizationResponse{Name: tenderResponse.BuyerOrganization.Name},
Status: string(tenderResponse.Status),
}
responses = append(responses, approval.ToResponseWithTender(tenderDetails))
}
tenders := s.loadTendersByIDs(ctx, uniqueTenderIDs(tenderApprovals))
responses := mapApprovalsWithTender(tenderApprovals, tenders, fullTenderDetailsFromResponse)
// Calculate pagination metadata
pages := int(total) / limit
@@ -985,11 +862,35 @@ func (s *tenderApprovalService) ListTenderApprovalsWithTender(ctx context.Contex
return meta, nil
}
func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID string) (*TenderApprovalWithTenderResponse, error) {
existing, err := s.repository.GetByTenderAndCompany(ctx, req.TenderID, companyID)
func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *ToggleTenderApprovalForm, companyID, customerID string) (*TenderApprovalWithTenderResponse, error) {
companyID = strings.TrimSpace(companyID)
if companyID == "" {
return nil, ErrInvalidCompanyID
}
tenderID, err := s.tenderService.ResolveMongoID(ctx, req.TenderID)
if err != nil {
s.logger.Warn("Failed to resolve tender for approval toggle", map[string]interface{}{
"tender_ref": req.TenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, ErrTenderNotFound
}
existing, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if !errors.Is(err, ErrTenderApprovalNotFound) {
s.logger.Error("Failed to load tender approval for toggle", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, err
}
approval := &TenderApproval{
TenderID: req.TenderID,
TenderID: tenderID,
CompanyID: companyID,
SubmissionMode: req.SubmissionMode,
Status: req.Status,
@@ -998,7 +899,7 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
err = s.repository.Create(ctx, approval)
if err != nil {
s.logger.Error("Failed to create new tender approval", map[string]interface{}{
"tender_id": req.TenderID,
"tender_id": tenderID,
"company_id": companyID,
"status": req.Status,
"submission_mode": req.SubmissionMode,
@@ -1007,13 +908,22 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
return nil, err
}
return approval.ToResponseWithTender(nil), nil
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, true); err != nil {
if deleteErr := s.repository.Delete(ctx, approval.GetID()); deleteErr != nil {
s.logger.Error("Failed to rollback tender approval after submission sync failure", map[string]interface{}{
"tender_approval_id": approval.GetID(),
"error": deleteErr.Error(),
})
}
return nil, ErrSubmissionSyncFailed
}
return s.toggleResponseWithTender(ctx, approval)
}
if existing.Status == req.Status {
err = s.repository.Delete(ctx, existing.GetID())
if err != nil {
s.logger.Error("Failed to update tender approval", map[string]interface{}{
s.logger.Error("Failed to delete tender approval during toggle", map[string]interface{}{
"error": err.Error(),
})
return nil, err
@@ -1025,24 +935,21 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
existing.Status = "unrejected"
}
return existing.ToResponseWithTender(nil), nil
} else if existing.Status == req.Status && existing.SubmissionMode != req.SubmissionMode {
existing.SubmissionMode = req.SubmissionMode
err = s.repository.Update(ctx, existing)
if err != nil {
s.logger.Error("Failed to update tender approval", map[string]interface{}{
"error": err.Error(),
})
return nil, err
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, false); err != nil {
return nil, ErrSubmissionSyncFailed
}
return existing.ToResponseWithTender(nil), nil
}
if existing.Status == ApprovalStatusSubmitted {
existing.Reject()
} else {
if existing.Status == ApprovalStatusRejected && req.Status == ApprovalStatusSubmitted {
existing.Submit()
existing.SubmissionMode = req.SubmissionMode
} else if existing.Status == ApprovalStatusSubmitted && req.Status == ApprovalStatusRejected {
existing.Reject()
existing.SubmissionMode = req.SubmissionMode
} else {
existing.Status = req.Status
existing.SubmissionMode = req.SubmissionMode
}
err = s.repository.Update(ctx, existing)
@@ -1053,7 +960,49 @@ func (s *tenderApprovalService) ToggleTenderApproval(ctx context.Context, req *T
return nil, err
}
return existing.ToResponseWithTender(nil), nil
if err := s.syncSubmissionAfterToggle(ctx, tenderID, companyID, customerID, req.Status, true); err != nil {
return nil, ErrSubmissionSyncFailed
}
return s.toggleResponseWithTender(ctx, existing)
}
func (s *tenderApprovalService) syncSubmissionAfterToggle(ctx context.Context, tenderID, companyID, customerID string, status ApprovalStatus, active bool) error {
if s.submissionHook == nil {
return nil
}
var err error
if !active {
err = s.submissionHook.OnApprovalRemoved(ctx, tenderID, companyID)
} else if status == ApprovalStatusSubmitted {
err = s.submissionHook.OnApprovalSubmitted(ctx, tenderID, companyID, customerID)
} else if status == ApprovalStatusRejected {
err = s.submissionHook.OnApprovalRejected(ctx, tenderID, companyID, customerID)
}
if err != nil {
s.logger.Error("Failed to sync tender submission workflow after approval toggle", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"status": status,
"active": active,
"error": err.Error(),
})
}
return err
}
func (s *tenderApprovalService) toggleResponseWithTender(ctx context.Context, approval *TenderApproval) (*TenderApprovalWithTenderResponse, error) {
if approval == nil {
return nil, ErrInvalidData
}
tenderResponse, err := s.tenderService.GetByID(ctx, approval.TenderID, "")
if err != nil {
return approval.ToResponseWithTender(nil), nil
}
return approval.ToResponseWithTender(listTenderDetailsFromResponse(tenderResponse)), nil
}
// GetCompanyTenderApprovalStats returns company-specific tender approval statistics
+110
View File
@@ -0,0 +1,110 @@
package tender_approval
import (
"context"
"tm/internal/tender"
)
func fullTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
NoticePublicationID: resp.NoticePublicationID,
PublicationDate: resp.PublicationDate,
Title: resp.Title,
Description: resp.Description,
ProcurementTypeCode: resp.ProcurementTypeCode,
ProcedureCode: resp.ProcedureCode,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
Duration: resp.Duration,
DurationUnit: resp.DurationUnit,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func listTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
Title: resp.Title,
Description: resp.Description,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func uniqueTenderIDs(approvals []*TenderApproval) []string {
if len(approvals) == 0 {
return nil
}
seen := make(map[string]struct{}, len(approvals))
ids := make([]string, 0, len(approvals))
for _, approval := range approvals {
if approval == nil || approval.TenderID == "" {
continue
}
if _, ok := seen[approval.TenderID]; ok {
continue
}
seen[approval.TenderID] = struct{}{}
ids = append(ids, approval.TenderID)
}
return ids
}
func (s *tenderApprovalService) loadTendersByIDs(ctx context.Context, tenderIDs []string) map[string]*tender.TenderResponse {
if len(tenderIDs) == 0 {
return map[string]*tender.TenderResponse{}
}
tenders, err := s.tenderService.GetByIDs(ctx, tenderIDs)
if err != nil {
s.logger.Warn("Failed to load tenders by IDs", map[string]interface{}{
"count": len(tenderIDs),
"error": err.Error(),
})
return map[string]*tender.TenderResponse{}
}
return tenders
}
func mapApprovalsWithTender(
approvals []*TenderApproval,
tenders map[string]*tender.TenderResponse,
toDetails func(*tender.TenderResponse) *TenderDetails,
) []*TenderApprovalWithTenderResponse {
responses := make([]*TenderApprovalWithTenderResponse, 0, len(approvals))
for _, approval := range approvals {
var details *TenderDetails
if tenderResp, ok := tenders[approval.TenderID]; ok {
details = toDetails(tenderResp)
}
responses = append(responses, approval.ToResponseWithTender(details))
}
return responses
}
+168
View File
@@ -0,0 +1,168 @@
package tender_submission
import (
"time"
"tm/pkg/mongo"
)
// SubmissionStatus represents the current step in the tender submission workflow.
type SubmissionStatus string
const (
StatusOpportunity SubmissionStatus = "opportunity"
StatusValidating SubmissionStatus = "validating"
StatusPartnerPending SubmissionStatus = "partner_pending"
StatusApplying SubmissionStatus = "applying"
StatusSentWaiting SubmissionStatus = "sent_waiting"
StatusRejected SubmissionStatus = "rejected"
StatusContract SubmissionStatus = "contract"
StatusNotApplied SubmissionStatus = "not_applied"
)
// SubmissionStage groups statuses into high-level workflow stages for UI display.
type SubmissionStage string
const (
StageOpportunity SubmissionStage = "opportunity"
StageInProgress SubmissionStage = "in_progress"
StageSubmitted SubmissionStage = "submitted"
StageNotApplied SubmissionStage = "not_applied"
)
// StatusHistory records a status change in the submission workflow.
type StatusHistory struct {
Status SubmissionStatus `bson:"status" json:"status"`
Reason string `bson:"reason,omitempty" json:"reason,omitempty"`
ChangedBy string `bson:"changed_by" json:"changed_by"`
ChangedAt int64 `bson:"changed_at" json:"changed_at"`
Description string `bson:"description,omitempty" json:"description,omitempty"`
}
// TenderSubmission tracks a company's tender submission workflow.
type TenderSubmission struct {
mongo.Model `bson:",inline"`
TenderID string `bson:"tender_id" json:"tender_id"`
CompanyID string `bson:"company_id" json:"company_id"`
CustomerID string `bson:"customer_id,omitempty" json:"customer_id,omitempty"`
Status SubmissionStatus `bson:"status" json:"status"`
Stage SubmissionStage `bson:"stage" json:"stage"`
History []StatusHistory `bson:"status_history" json:"status_history"`
}
func StageForStatus(status SubmissionStatus) SubmissionStage {
switch status {
case StatusOpportunity:
return StageOpportunity
case StatusValidating, StatusPartnerPending, StatusApplying:
return StageInProgress
case StatusSentWaiting, StatusRejected, StatusContract:
return StageSubmitted
case StatusNotApplied:
return StageNotApplied
default:
return StageOpportunity
}
}
func IsTerminalStatus(status SubmissionStatus) bool {
switch status {
case StatusRejected, StatusContract, StatusNotApplied:
return true
default:
return false
}
}
func AllowedNextStatuses(current SubmissionStatus) []SubmissionStatus {
transitions := map[SubmissionStatus][]SubmissionStatus{
StatusOpportunity: {StatusValidating, StatusNotApplied},
StatusValidating: {StatusPartnerPending, StatusApplying, StatusNotApplied},
StatusPartnerPending: {StatusApplying, StatusNotApplied},
StatusApplying: {StatusSentWaiting, StatusNotApplied},
StatusSentWaiting: {StatusRejected, StatusContract},
StatusRejected: {},
StatusContract: {},
StatusNotApplied: {},
}
return transitions[current]
}
func (ts *TenderSubmission) GetID() string {
return ts.ID.Hex()
}
func (ts *TenderSubmission) SetCreatedAt(timestamp int64) {
ts.CreatedAt = timestamp
}
func (ts *TenderSubmission) SetUpdatedAt(timestamp int64) {
ts.UpdatedAt = timestamp
}
func (ts *TenderSubmission) AddStatusChange(status SubmissionStatus, reason, changedBy, description string) {
ts.History = append(ts.History, StatusHistory{
Status: status,
Reason: reason,
ChangedBy: changedBy,
ChangedAt: time.Now().Unix(),
Description: description,
})
ts.Status = status
ts.Stage = StageForStatus(status)
ts.SetUpdatedAt(time.Now().Unix())
}
// ReopenForApproval resets a terminal submission when the tender is approved again.
// This is only used by the approval sync path, not by user-initiated status updates.
func (ts *TenderSubmission) ReopenForApproval(changedBy, description string) {
ts.AddStatusChange(StatusOpportunity, "approval_resubmitted", changedBy, description)
}
func NewTenderSubmission(tenderID, companyID, customerID string) *TenderSubmission {
now := time.Now().Unix()
ts := &TenderSubmission{
TenderID: tenderID,
CompanyID: companyID,
CustomerID: customerID,
Status: StatusOpportunity,
Stage: StageOpportunity,
}
ts.SetCreatedAt(now)
ts.SetUpdatedAt(now)
ts.AddStatusChange(StatusOpportunity, "", customerID, "Submission workflow started")
return ts
}
func (ts *TenderSubmission) ToResponse() *TenderSubmissionResponse {
history := make([]StatusHistoryResponse, len(ts.History))
for i, h := range ts.History {
history[i] = StatusHistoryResponse{
Status: string(h.Status),
Reason: h.Reason,
ChangedBy: h.ChangedBy,
ChangedAt: h.ChangedAt,
Description: h.Description,
}
}
return &TenderSubmissionResponse{
ID: ts.GetID(),
TenderID: ts.TenderID,
CompanyID: ts.CompanyID,
CustomerID: ts.CustomerID,
Status: string(ts.Status),
Stage: string(ts.Stage),
History: history,
CreatedAt: ts.CreatedAt,
UpdatedAt: ts.UpdatedAt,
}
}
func (ts *TenderSubmission) ToResponseWithTender(tender *TenderDetails) *TenderSubmissionWithTenderResponse {
resp := ts.ToResponse()
return &TenderSubmissionWithTenderResponse{
TenderSubmissionResponse: *resp,
Tender: tender,
}
}
+98
View File
@@ -0,0 +1,98 @@
package tender_submission
import "testing"
func TestStageForStatus(t *testing.T) {
tests := []struct {
status SubmissionStatus
stage SubmissionStage
}{
{StatusOpportunity, StageOpportunity},
{StatusValidating, StageInProgress},
{StatusPartnerPending, StageInProgress},
{StatusApplying, StageInProgress},
{StatusSentWaiting, StageSubmitted},
{StatusRejected, StageSubmitted},
{StatusContract, StageSubmitted},
{StatusNotApplied, StageNotApplied},
}
for _, tt := range tests {
if got := StageForStatus(tt.status); got != tt.stage {
t.Fatalf("StageForStatus(%q) = %q, want %q", tt.status, got, tt.stage)
}
}
}
func TestAllowedNextStatuses(t *testing.T) {
tests := []struct {
current SubmissionStatus
next SubmissionStatus
allowed bool
}{
{StatusOpportunity, StatusValidating, true},
{StatusOpportunity, StatusNotApplied, true},
{StatusOpportunity, StatusApplying, false},
{StatusValidating, StatusPartnerPending, true},
{StatusValidating, StatusApplying, true},
{StatusValidating, StatusNotApplied, true},
{StatusPartnerPending, StatusApplying, true},
{StatusApplying, StatusSentWaiting, true},
{StatusSentWaiting, StatusContract, true},
{StatusSentWaiting, StatusRejected, true},
{StatusContract, StatusRejected, false},
{StatusNotApplied, StatusValidating, false},
}
for _, tt := range tests {
allowed := false
for _, candidate := range AllowedNextStatuses(tt.current) {
if candidate == tt.next {
allowed = true
break
}
}
if allowed != tt.allowed {
t.Fatalf("transition %q -> %q allowed=%v, want %v", tt.current, tt.next, allowed, tt.allowed)
}
}
}
func TestAddStatusChangeUpdatesStage(t *testing.T) {
submission := NewTenderSubmission("tender-1", "company-1", "customer-1")
submission.AddStatusChange(StatusValidating, "", "customer-1", "Accepted tender")
if submission.Status != StatusValidating {
t.Fatalf("status = %q, want %q", submission.Status, StatusValidating)
}
if submission.Stage != StageInProgress {
t.Fatalf("stage = %q, want %q", submission.Stage, StageInProgress)
}
if len(submission.History) != 2 {
t.Fatalf("history length = %d, want 2", len(submission.History))
}
}
func TestValidateStatusTransition(t *testing.T) {
service := &tenderSubmissionService{}
if err := service.validateStatusTransition(StatusOpportunity, StatusValidating); err != nil {
t.Fatalf("expected valid transition, got %v", err)
}
if err := service.validateStatusTransition(StatusContract, StatusRejected); err == nil {
t.Fatal("expected terminal status error")
}
err := service.validateStatusTransition(StatusOpportunity, StatusApplying)
if err == nil {
t.Fatal("expected invalid transition error")
}
transitionErr, ok := err.(*InvalidStatusTransitionError)
if !ok {
t.Fatalf("expected InvalidStatusTransitionError, got %T", err)
}
if transitionErr.CurrentStatus != string(StatusOpportunity) {
t.Fatalf("current status = %q", transitionErr.CurrentStatus)
}
}
+46
View File
@@ -0,0 +1,46 @@
package tender_submission
import (
"errors"
"fmt"
"strings"
)
var (
ErrTenderSubmissionNotFound = errors.New("tender submission not found")
ErrTenderSubmissionExists = errors.New("tender submission already exists for this tender and company")
ErrInvalidStatus = errors.New("invalid tender submission status")
ErrInvalidStage = errors.New("invalid tender submission stage")
ErrTenderNotFound = errors.New("tender not found")
ErrApprovalRequired = errors.New("tender must be approved before starting submission workflow")
ErrInvalidStatusTransition = errors.New("invalid status transition")
ErrTerminalStatus = errors.New("tender submission is in a terminal state and cannot be updated")
ErrConcurrentUpdate = errors.New("tender submission was modified concurrently")
)
// InvalidStatusTransitionError is returned when a status change is not allowed.
type InvalidStatusTransitionError struct {
CurrentStatus string
NewStatus string
Allowed []string
}
func (e *InvalidStatusTransitionError) Error() string {
if len(e.Allowed) == 0 {
return fmt.Sprintf("Tender submission status is already %s and cannot be changed", e.CurrentStatus)
}
return fmt.Sprintf(
"Cannot change tender submission status from %s to %s. Allowed next statuses: %s",
e.CurrentStatus,
e.NewStatus,
strings.Join(e.Allowed, ", "),
)
}
func NewInvalidStatusTransitionError(currentStatus, newStatus string, allowed []string) *InvalidStatusTransitionError {
return &InvalidStatusTransitionError{
CurrentStatus: currentStatus,
NewStatus: newStatus,
Allowed: allowed,
}
}
+100
View File
@@ -0,0 +1,100 @@
package tender_submission
import "tm/pkg/response"
// UpdateSubmissionStatusForm updates the workflow status of a tender submission.
type UpdateSubmissionStatusForm struct {
Status SubmissionStatus `json:"status" valid:"required,in(opportunity|validating|partner_pending|applying|sent_waiting|rejected|contract|not_applied)"`
Reason string `json:"reason,omitempty" valid:"optional,length(0|500)"`
Description string `json:"description,omitempty" valid:"optional,length(0|1000)"`
}
// ListTenderSubmissionsForm filters company tender submissions.
type ListTenderSubmissionsForm struct {
TenderID *string `query:"tender_id" valid:"optional"`
CompanyID *string `query:"company_id" valid:"optional"`
Status []string `query:"status" valid:"optional"`
Stage []string `query:"stage" valid:"optional"`
From *int64 `query:"created_from" valid:"optional"`
To *int64 `query:"created_to" valid:"optional"`
Limit *int `query:"limit" valid:"optional,range(1|100)"`
Offset *int `query:"offset" valid:"optional,range(0|1000000)"`
SortBy *string `query:"sort_by" valid:"optional,in(created_at|updated_at|status|stage)"`
SortOrder *string `query:"sort_order" valid:"optional,in(asc|desc)"`
}
// TenderSubmissionResponse is the API response for a tender submission.
type TenderSubmissionResponse struct {
ID string `json:"id"`
TenderID string `json:"tender_id"`
CompanyID string `json:"company_id"`
CustomerID string `json:"customer_id,omitempty"`
Status string `json:"status"`
Stage string `json:"stage"`
History []StatusHistoryResponse `json:"status_history"`
CreatedAt int64 `json:"created_at"`
UpdatedAt int64 `json:"updated_at"`
}
// StatusHistoryResponse is the API representation of a status change.
type StatusHistoryResponse struct {
Status string `json:"status"`
Reason string `json:"reason,omitempty"`
ChangedBy string `json:"changed_by"`
ChangedAt int64 `json:"changed_at"`
Description string `json:"description,omitempty"`
}
// TenderSubmissionWithTenderResponse includes tender details.
type TenderSubmissionWithTenderResponse struct {
TenderSubmissionResponse
Tender *TenderDetails `json:"tender,omitempty"`
}
// TenderSubmissionListResponse is a paginated list of submissions with tender details.
type TenderSubmissionListResponse struct {
Submissions []*TenderSubmissionWithTenderResponse `json:"submissions"`
Metadata *response.Meta `json:"metadata"`
}
// TenderDetails represents essential tender information for submission responses.
type TenderDetails struct {
ID string `json:"id"`
NoticePublicationID string `json:"notice_publication_id"`
PublicationDate int64 `json:"publication_date"`
Title string `json:"title"`
Description string `json:"description"`
ProcurementTypeCode string `json:"procurement_type_code"`
ProcedureCode string `json:"procedure_code"`
EstimatedValue float64 `json:"estimated_value"`
Currency string `json:"currency"`
Duration string `json:"duration"`
DurationUnit string `json:"duration_unit"`
TenderDeadline int64 `json:"tender_deadline"`
SubmissionDeadline int64 `json:"submission_deadline"`
CountryCode string `json:"country_code"`
BuyerOrganization *OrganizationResponse `json:"buyer_organization"`
Status string `json:"status"`
}
// OrganizationResponse represents buyer organization info.
type OrganizationResponse struct {
Name string `json:"name"`
}
// CompanySubmissionStatsResponse contains submission counts grouped by stage and status.
type CompanySubmissionStatsResponse struct {
CompanyID string `json:"company_id"`
Total int64 `json:"total"`
ByStage map[string]int64 `json:"by_stage"`
ByStatus map[string]int64 `json:"by_status"`
LastUpdated int64 `json:"last_updated"`
}
// AdminSubmissionStatsResponse contains global submission statistics.
type AdminSubmissionStatsResponse struct {
Total int64 `json:"total"`
ByStage map[string]int64 `json:"by_stage"`
ByStatus map[string]int64 `json:"by_status"`
LastUpdated int64 `json:"last_updated"`
}
+334
View File
@@ -0,0 +1,334 @@
package tender_submission
import (
"errors"
"strings"
"tm/internal/customer"
"tm/pkg/logger"
"tm/pkg/response"
"github.com/labstack/echo/v4"
)
// Handler handles HTTP requests for tender submission operations.
type Handler struct {
service Service
logger logger.Logger
}
func NewHandler(service Service, logger logger.Logger) *Handler {
return &Handler{service: service, logger: logger}
}
// ListCompanySubmissions lists tender submissions for the authenticated company.
// @Summary List company tender submissions
// @Description Retrieve paginated tender submission workflows for the authenticated company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param status query []string false "Filter by status"
// @Param stage query []string false "Filter by stage"
// @Param created_from query int false "Filter by created date from (Unix timestamp)"
// @Param created_to query int false "Filter by created date to (Unix timestamp)"
// @Param limit query int false "Page size (default 20, max 100)"
// @Param offset query int false "Pagination offset"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionListResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions [get]
func (h *Handler) ListCompanySubmissions(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
form, err := response.Parse[ListTenderSubmissionsForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
result, err := h.service.ListByCompanyWithTender(c.Request().Context(), companyID, form)
if err != nil {
return response.InternalServerError(c, "Failed to list tender submissions")
}
return response.SuccessWithMeta(c, result.Submissions, result.Metadata, "Tender submissions retrieved successfully")
}
// EnsureSubmission ensures a submission workflow exists for an approved tender.
// @Summary Ensure tender submission workflow
// @Description Create or return the submission workflow for an approved tender
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param tender_id path string true "Tender ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 400 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/tender/{tender_id}/ensure [post]
func (h *Handler) EnsureSubmission(c echo.Context) error {
companyID, err := h.resolveCompanyID(c)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
tenderID := strings.TrimSpace(c.Param("tender_id"))
if tenderID == "" {
return response.BadRequest(c, "Tender ID is required", "")
}
result, err := h.service.EnsureForApprovedTender(c.Request().Context(), tenderID, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) {
return response.NotFound(c, "Tender not found")
}
if errors.Is(err, ErrApprovalRequired) {
return response.BadRequest(c, "Tender must be approved before starting submission workflow", "")
}
return response.InternalServerError(c, "Failed to ensure tender submission")
}
return response.Success(c, result, "Tender submission ensured successfully")
}
// GetSubmissionByID retrieves a tender submission by ID for the authenticated company.
// @Summary Get tender submission by ID
// @Description Retrieve a tender submission workflow with tender details for the authenticated company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/{id} [get]
func (h *Handler) GetSubmissionByID(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
result, err := h.service.GetByIDForCompanyWithTender(c.Request().Context(), id, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// GetSubmissionByTender retrieves a submission by tender and company.
// @Summary Get tender submission by tender
// @Description Retrieve the submission workflow for a specific tender and company
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param tender_id path string true "Tender ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/tender/{tender_id} [get]
func (h *Handler) GetSubmissionByTender(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
tenderID := strings.TrimSpace(c.Param("tender_id"))
if tenderID == "" {
return response.BadRequest(c, "Tender ID is required", "")
}
result, err := h.service.GetByTenderAndCompanyWithTender(c.Request().Context(), tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderNotFound) || errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// UpdateSubmissionStatus updates the workflow status of a submission.
// @Summary Update tender submission status
// @Description Move a tender submission to the next workflow step
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Param body body UpdateSubmissionStatusForm true "Status update"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 400 {object} response.APIResponse
// @Failure 404 {object} response.APIResponse
// @Failure 422 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/{id}/status [patch]
func (h *Handler) UpdateSubmissionStatus(c echo.Context) error {
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
form, err := response.Parse[UpdateSubmissionStatusForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
companyID, err := h.resolveCompanyID(c)
if err != nil {
if errors.Is(err, customer.ErrCompanyNotAssigned) {
return response.Forbidden(c, "Company is not assigned to customer")
}
return response.BadRequest(c, "Company ID is required", "")
}
customerID, _ := customer.GetCustomerIDFromContext(c)
result, err := h.service.UpdateStatus(c.Request().Context(), id, form, companyID, customerID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
var transitionErr *InvalidStatusTransitionError
if errors.As(err, &transitionErr) {
return response.ValidationError(c, "Invalid status transition", err.Error())
}
if errors.Is(err, ErrTerminalStatus) {
return response.ValidationError(c, "Submission is in a terminal state", err.Error())
}
if errors.Is(err, ErrConcurrentUpdate) {
return response.Conflict(c, "Tender submission was modified concurrently")
}
return response.InternalServerError(c, "Failed to update tender submission status")
}
return response.Success(c, result, "Tender submission status updated successfully")
}
// GetCompanySubmissionStats returns submission statistics for the company.
// @Summary Get company tender submission statistics
// @Description Retrieve submission counts grouped by stage and status
// @Tags TenderSubmissions
// @Accept json
// @Produce json
// @Success 200 {object} response.APIResponse{data=CompanySubmissionStatsResponse}
// @Failure 401 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /api/v1/tender-submissions/stats [get]
func (h *Handler) GetCompanySubmissionStats(c echo.Context) error {
companyID, ok := c.Get("company_id").(string)
if !ok || companyID == "" {
return response.Unauthorized(c, "Unauthorized")
}
stats, err := h.service.GetCompanyStats(c.Request().Context(), companyID)
if err != nil {
return response.InternalServerError(c, "Failed to get submission statistics")
}
return response.Success(c, stats, "Submission statistics retrieved successfully")
}
// AdminListSubmissions lists tender submissions for admin users.
// @Summary Admin list tender submissions
// @Description Retrieve paginated tender submissions across companies
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Param company_id query string false "Filter by company ID"
// @Param tender_id query string false "Filter by tender ID"
// @Param status query []string false "Filter by status"
// @Param stage query []string false "Filter by stage"
// @Param limit query int false "Page size"
// @Param offset query int false "Pagination offset"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionListResponse}
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions [get]
func (h *Handler) AdminListSubmissions(c echo.Context) error {
form, err := response.Parse[ListTenderSubmissionsForm](c)
if err != nil {
return response.ValidationError(c, "Invalid request data", err.Error())
}
result, err := h.service.AdminListWithTender(c.Request().Context(), form)
if err != nil {
return response.InternalServerError(c, "Failed to list tender submissions")
}
return response.SuccessWithMeta(c, result.Submissions, result.Metadata, "Tender submissions retrieved successfully")
}
// AdminGetSubmissionByID retrieves a submission by ID for admin users.
// @Summary Admin get tender submission by ID
// @Description Retrieve a tender submission workflow with tender details
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Param id path string true "Submission ID"
// @Success 200 {object} response.APIResponse{data=TenderSubmissionWithTenderResponse}
// @Failure 404 {object} response.APIResponse
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions/{id} [get]
func (h *Handler) AdminGetSubmissionByID(c echo.Context) error {
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return response.BadRequest(c, "Submission ID is required", "")
}
result, err := h.service.GetByIDWithTender(c.Request().Context(), id)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return response.NotFound(c, "Tender submission not found")
}
return response.InternalServerError(c, "Failed to get tender submission")
}
return response.Success(c, result, "Tender submission retrieved successfully")
}
// AdminGetSubmissionStats returns global submission statistics.
// @Summary Admin get tender submission statistics
// @Description Retrieve global submission counts grouped by stage and status
// @Tags Admin-TenderSubmissions
// @Accept json
// @Produce json
// @Success 200 {object} response.APIResponse{data=AdminSubmissionStatsResponse}
// @Failure 500 {object} response.APIResponse
// @Security BearerAuth
// @Router /admin/v1/tender-submissions/stats [get]
func (h *Handler) AdminGetSubmissionStats(c echo.Context) error {
stats, err := h.service.GetGlobalStats(c.Request().Context())
if err != nil {
return response.InternalServerError(c, "Failed to get submission statistics")
}
return response.Success(c, stats, "Submission statistics retrieved successfully")
}
func (h *Handler) resolveCompanyID(c echo.Context) (string, error) {
requestedCompanyID := strings.TrimSpace(c.QueryParam("company_id"))
activeCompanyID, _ := c.Get("company_id").(string)
assignedCompanyIDs, _ := c.Get("company_ids").([]string)
return customer.PickAssignedCompanyID(activeCompanyID, assignedCompanyIDs, requestedCompanyID)
}
+346
View File
@@ -0,0 +1,346 @@
package tender_submission
import (
"context"
"errors"
"time"
"tm/pkg/logger"
"tm/pkg/mongo"
"go.mongodb.org/mongo-driver/v2/bson"
mongodriver "go.mongodb.org/mongo-driver/v2/mongo"
)
const maxListLimit = 100
// Repository defines data access for tender submissions.
type Repository interface {
Create(ctx context.Context, submission *TenderSubmission) error
GetByID(ctx context.Context, id string) (*TenderSubmission, error)
GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*TenderSubmission, error)
Update(ctx context.Context, submission *TenderSubmission) error
UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error
Delete(ctx context.Context, id string) error
ListByCompany(ctx context.Context, companyID string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error)
Search(ctx context.Context, tenderID, companyID *string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error)
GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error)
GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error)
}
type tenderSubmissionRepository struct {
ormRepo mongo.Repository[TenderSubmission]
collection *mongodriver.Collection
logger logger.Logger
}
func NewRepository(mongoManager *mongo.ConnectionManager, logger logger.Logger) Repository {
indexes := []mongo.Index{
*mongo.CreateUniqueIndex("tender_company_idx", bson.D{{Key: "tender_id", Value: 1}, {Key: "company_id", Value: 1}}),
*mongo.NewIndex("tender_id_idx", bson.D{{Key: "tender_id", Value: 1}}),
*mongo.NewIndex("company_id_idx", bson.D{{Key: "company_id", Value: 1}}),
*mongo.NewIndex("status_idx", bson.D{{Key: "status", Value: 1}}),
*mongo.NewIndex("stage_idx", bson.D{{Key: "stage", Value: 1}}),
*mongo.NewIndex("created_at_idx", bson.D{{Key: "created_at", Value: -1}}),
*mongo.NewIndex("updated_at_idx", bson.D{{Key: "updated_at", Value: -1}}),
}
if err := mongoManager.CreateIndexes("tender_submissions", indexes); err != nil {
logger.Warn("Failed to create tender submission indexes", map[string]interface{}{
"error": err.Error(),
})
}
collection := mongoManager.GetCollection("tender_submissions")
ormRepo := mongo.NewRepository[TenderSubmission](collection, logger)
return &tenderSubmissionRepository{ormRepo: ormRepo, collection: collection, logger: logger}
}
func (r *tenderSubmissionRepository) Create(ctx context.Context, submission *TenderSubmission) error {
now := time.Now().Unix()
submission.SetCreatedAt(now)
submission.SetUpdatedAt(now)
if err := r.ormRepo.Create(ctx, submission); err != nil {
if mongo.IsDuplicateKeyError(err) {
return ErrTenderSubmissionExists
}
r.logger.Error("Failed to create tender submission", map[string]interface{}{
"error": err.Error(),
"tender_id": submission.TenderID,
"company_id": submission.CompanyID,
})
return err
}
r.logger.Info("Tender submission created", map[string]interface{}{
"submission_id": submission.GetID(),
"tender_id": submission.TenderID,
"company_id": submission.CompanyID,
"status": submission.Status,
})
return nil
}
func (r *tenderSubmissionRepository) GetByID(ctx context.Context, id string) (*TenderSubmission, error) {
submission, err := r.ormRepo.FindByID(ctx, id)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to get tender submission by ID", map[string]interface{}{
"error": err.Error(),
"submission_id": id,
})
return nil, err
}
return submission, nil
}
func (r *tenderSubmissionRepository) GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*TenderSubmission, error) {
filter := bson.M{"tender_id": tenderID, "company_id": companyID}
submission, err := r.ormRepo.FindOne(ctx, filter)
if err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return nil, ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to get tender submission by tender and company", map[string]interface{}{
"error": err.Error(),
"tender_id": tenderID,
"company_id": companyID,
})
return nil, err
}
return submission, nil
}
func (r *tenderSubmissionRepository) Update(ctx context.Context, submission *TenderSubmission) error {
submission.SetUpdatedAt(time.Now().Unix())
if err := r.ormRepo.Update(ctx, submission); err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to update tender submission", map[string]interface{}{
"error": err.Error(),
"submission_id": submission.GetID(),
})
return err
}
return nil
}
func (r *tenderSubmissionRepository) UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error {
objectID, err := bson.ObjectIDFromHex(submission.GetID())
if err != nil {
return err
}
newUpdatedAt := time.Now().Unix()
submission.SetUpdatedAt(newUpdatedAt)
filter := bson.M{"_id": objectID, "updated_at": expectedUpdatedAt}
result, err := r.collection.UpdateOne(ctx, filter, bson.M{"$set": submission})
if err != nil {
r.logger.Error("Failed to update tender submission with optimistic lock", map[string]interface{}{
"error": err.Error(),
"submission_id": submission.GetID(),
})
return err
}
if result.MatchedCount == 0 {
return ErrConcurrentUpdate
}
return nil
}
func (r *tenderSubmissionRepository) Delete(ctx context.Context, id string) error {
if err := r.ormRepo.Delete(ctx, id); err != nil {
if errors.Is(err, mongo.ErrDocumentNotFound) {
return ErrTenderSubmissionNotFound
}
r.logger.Error("Failed to delete tender submission", map[string]interface{}{
"error": err.Error(),
"submission_id": id,
})
return err
}
return nil
}
func (r *tenderSubmissionRepository) ListByCompany(ctx context.Context, companyID string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error) {
companyIDPtr := companyID
return r.Search(ctx, nil, &companyIDPtr, statuses, stages, from, to, limit, offset, sortBy, sortOrder)
}
func (r *tenderSubmissionRepository) Search(ctx context.Context, tenderID, companyID *string, statuses []SubmissionStatus, stages []SubmissionStage, from, to *int64, limit, offset int, sortBy, sortOrder string) ([]*TenderSubmission, int64, error) {
limit = capListLimit(limit)
if offset < 0 {
offset = 0
}
filter := bson.M{}
if tenderID != nil && *tenderID != "" {
filter["tender_id"] = *tenderID
}
if companyID != nil && *companyID != "" {
filter["company_id"] = *companyID
}
if len(statuses) > 0 {
filter["status"] = bson.M{"$in": statuses}
}
if len(stages) > 0 {
filter["stage"] = bson.M{"$in": stages}
}
if from != nil || to != nil {
dateFilter := bson.M{}
if from != nil {
dateFilter["$gte"] = *from
}
if to != nil {
dateFilter["$lte"] = *to
}
filter["created_at"] = dateFilter
}
if sortBy == "" {
sortBy = "updated_at"
}
if sortOrder == "" {
sortOrder = "desc"
}
pagination := mongo.NewPaginationBuilder().Limit(limit).Skip(offset)
if sortOrder == "asc" {
pagination = pagination.SortAsc(sortBy)
} else {
pagination = pagination.SortDesc(sortBy)
}
result, err := r.ormRepo.FindAll(ctx, filter, pagination.Build())
if err != nil {
r.logger.Error("Failed to search tender submissions", map[string]interface{}{
"error": err.Error(),
})
return nil, 0, err
}
items := make([]*TenderSubmission, len(result.Items))
for i := range result.Items {
items[i] = &result.Items[i]
}
return items, result.TotalCount, nil
}
func (r *tenderSubmissionRepository) GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error) {
pipeline := mongodriver.Pipeline{
{{Key: "$match", Value: bson.M{"company_id": companyID}}},
{{Key: "$facet", Value: bson.M{
"total": bson.A{bson.M{"$count": "count"}},
"by_stage": bson.A{bson.M{"$group": bson.M{"_id": "$stage", "count": bson.M{"$sum": 1}}}},
"by_status": bson.A{bson.M{"$group": bson.M{"_id": "$status", "count": bson.M{"$sum": 1}}}},
}}},
}
results, err := r.ormRepo.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
stats := &CompanySubmissionStatsResponse{
CompanyID: companyID,
ByStage: map[string]int64{},
ByStatus: map[string]int64{},
LastUpdated: time.Now().Unix(),
}
if len(results) == 0 {
return stats, nil
}
facet := results[0]
if totalArr, ok := facet["total"].(bson.A); ok && len(totalArr) > 0 {
if doc, ok := totalArr[0].(bson.M); ok {
stats.Total = int64Value(doc["count"])
}
}
stats.ByStage = countMapFromFacet(facet["by_stage"])
stats.ByStatus = countMapFromFacet(facet["by_status"])
return stats, nil
}
func (r *tenderSubmissionRepository) GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error) {
pipeline := mongodriver.Pipeline{
{{Key: "$facet", Value: bson.M{
"total": bson.A{bson.M{"$count": "count"}},
"by_stage": bson.A{bson.M{"$group": bson.M{"_id": "$stage", "count": bson.M{"$sum": 1}}}},
"by_status": bson.A{bson.M{"$group": bson.M{"_id": "$status", "count": bson.M{"$sum": 1}}}},
}}},
}
results, err := r.ormRepo.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
stats := &AdminSubmissionStatsResponse{
ByStage: map[string]int64{},
ByStatus: map[string]int64{},
LastUpdated: time.Now().Unix(),
}
if len(results) == 0 {
return stats, nil
}
facet := results[0]
if totalArr, ok := facet["total"].(bson.A); ok && len(totalArr) > 0 {
if doc, ok := totalArr[0].(bson.M); ok {
stats.Total = int64Value(doc["count"])
}
}
stats.ByStage = countMapFromFacet(facet["by_stage"])
stats.ByStatus = countMapFromFacet(facet["by_status"])
return stats, nil
}
func countMapFromFacet(raw interface{}) map[string]int64 {
out := map[string]int64{}
arr, ok := raw.(bson.A)
if !ok {
return out
}
for _, item := range arr {
doc, ok := item.(bson.M)
if !ok {
continue
}
key, _ := doc["_id"].(string)
if key == "" {
continue
}
out[key] = int64Value(doc["count"])
}
return out
}
func int64Value(v interface{}) int64 {
switch n := v.(type) {
case int32:
return int64(n)
case int64:
return n
case float64:
return int64(n)
default:
return 0
}
}
func capListLimit(limit int) int {
if limit <= 0 {
return 20
}
if limit > maxListLimit {
return maxListLimit
}
return limit
}
+451
View File
@@ -0,0 +1,451 @@
package tender_submission
import (
"context"
"errors"
"strings"
"tm/internal/tender"
"tm/internal/tender_approval"
"tm/pkg/logger"
"tm/pkg/response"
)
// TenderApprovalReader verifies that a tender is approved before workflow actions.
type TenderApprovalReader interface {
GetByTenderAndCompany(ctx context.Context, tenderID, companyID string) (*tender_approval.TenderApproval, error)
}
// Service defines tender submission business logic.
type Service interface {
EnsureForApprovedTender(ctx context.Context, tenderID, companyID, customerID string) (*TenderSubmissionWithTenderResponse, error)
GetByIDWithTender(ctx context.Context, id string) (*TenderSubmissionWithTenderResponse, error)
GetByIDForCompanyWithTender(ctx context.Context, id, companyID string) (*TenderSubmissionWithTenderResponse, error)
GetByTenderAndCompanyWithTender(ctx context.Context, tenderID, companyID string) (*TenderSubmissionWithTenderResponse, error)
ListByCompanyWithTender(ctx context.Context, companyID string, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error)
UpdateStatus(ctx context.Context, id string, form *UpdateSubmissionStatusForm, companyID, changedBy string) (*TenderSubmissionWithTenderResponse, error)
GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error)
GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error)
AdminListWithTender(ctx context.Context, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error)
OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error
OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error
OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error
}
type tenderSubmissionService struct {
repository Repository
approvalReader TenderApprovalReader
tenderService tender.Service
logger logger.Logger
}
func NewService(repository Repository, approvalReader TenderApprovalReader, tenderService tender.Service, logger logger.Logger) Service {
return &tenderSubmissionService{
repository: repository,
approvalReader: approvalReader,
tenderService: tenderService,
logger: logger,
}
}
func (s *tenderSubmissionService) EnsureForApprovedTender(ctx context.Context, tenderID, companyID, customerID string) (*TenderSubmissionWithTenderResponse, error) {
resolvedTenderID, err := s.tenderService.ResolveMongoID(ctx, tenderID)
if err != nil {
s.logger.Warn("Failed to resolve tender for submission ensure", map[string]interface{}{
"tender_ref": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return nil, ErrTenderNotFound
}
if err := s.requireSubmittedApproval(ctx, resolvedTenderID, companyID); err != nil {
return nil, err
}
submission, err := s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
if !errors.Is(err, ErrTenderSubmissionNotFound) {
return nil, err
}
submission = NewTenderSubmission(resolvedTenderID, companyID, customerID)
if err := s.repository.Create(ctx, submission); err != nil {
if !errors.Is(err, ErrTenderSubmissionExists) {
return nil, err
}
submission, err = s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
return nil, err
}
}
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByIDWithTender(ctx context.Context, id string) (*TenderSubmissionWithTenderResponse, error) {
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByIDForCompanyWithTender(ctx context.Context, id, companyID string) (*TenderSubmissionWithTenderResponse, error) {
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
if submission.CompanyID != companyID {
return nil, ErrTenderSubmissionNotFound
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetByTenderAndCompanyWithTender(ctx context.Context, tenderID, companyID string) (*TenderSubmissionWithTenderResponse, error) {
resolvedTenderID, err := s.tenderService.ResolveMongoID(ctx, tenderID)
if err != nil {
return nil, ErrTenderNotFound
}
submission, err := s.repository.GetByTenderAndCompany(ctx, resolvedTenderID, companyID)
if err != nil {
return nil, err
}
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) ListByCompanyWithTender(ctx context.Context, companyID string, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error) {
limit, offset, sortBy, sortOrder := listParams(form)
statuses := parseStatuses(form.Status)
stages := parseStages(form.Stage)
submissions, total, err := s.repository.ListByCompany(ctx, companyID, statuses, stages, form.From, form.To, limit, offset, sortBy, sortOrder)
if err != nil {
s.logger.Error("Failed to list tender submissions", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
return nil, errors.New("failed to list tender submissions")
}
return &TenderSubmissionListResponse{
Submissions: s.enrichSubmissions(ctx, submissions, false),
Metadata: listMetadata(total, limit, offset),
}, nil
}
func (s *tenderSubmissionService) AdminListWithTender(ctx context.Context, form *ListTenderSubmissionsForm) (*TenderSubmissionListResponse, error) {
limit, offset, sortBy, sortOrder := listParams(form)
statuses := parseStatuses(form.Status)
stages := parseStages(form.Stage)
submissions, total, err := s.repository.Search(ctx, form.TenderID, form.CompanyID, statuses, stages, form.From, form.To, limit, offset, sortBy, sortOrder)
if err != nil {
s.logger.Error("Failed to admin list tender submissions", map[string]interface{}{
"error": err.Error(),
})
return nil, errors.New("failed to list tender submissions")
}
return &TenderSubmissionListResponse{
Submissions: s.enrichSubmissions(ctx, submissions, false),
Metadata: listMetadata(total, limit, offset),
}, nil
}
func (s *tenderSubmissionService) UpdateStatus(ctx context.Context, id string, form *UpdateSubmissionStatusForm, companyID, changedBy string) (*TenderSubmissionWithTenderResponse, error) {
s.logger.Info("Updating tender submission status", map[string]interface{}{
"submission_id": id,
"company_id": companyID,
"new_status": form.Status,
"changed_by": changedBy,
})
submission, err := s.repository.GetByID(ctx, id)
if err != nil {
return nil, err
}
if companyID != "" && submission.CompanyID != companyID {
return nil, ErrTenderSubmissionNotFound
}
expectedUpdatedAt := submission.UpdatedAt
if err := s.validateStatusTransition(submission.Status, form.Status); err != nil {
s.logger.Error("Invalid tender submission status transition", map[string]interface{}{
"submission_id": id,
"current_status": submission.Status,
"new_status": form.Status,
"error": err.Error(),
})
return nil, err
}
submission.AddStatusChange(form.Status, form.Reason, changedBy, form.Description)
if err := s.repository.UpdateIfUpdatedAt(ctx, submission, expectedUpdatedAt); err != nil {
if errors.Is(err, ErrConcurrentUpdate) {
return nil, ErrConcurrentUpdate
}
return nil, errors.New("failed to update tender submission status")
}
s.logger.Info("Tender submission status updated", map[string]interface{}{
"submission_id": id,
"status": submission.Status,
"stage": submission.Stage,
})
return s.responseWithTender(ctx, submission, true)
}
func (s *tenderSubmissionService) GetCompanyStats(ctx context.Context, companyID string) (*CompanySubmissionStatsResponse, error) {
stats, err := s.repository.GetCompanyStats(ctx, companyID)
if err != nil {
s.logger.Error("Failed to get company submission stats", map[string]interface{}{
"company_id": companyID,
"error": err.Error(),
})
return nil, errors.New("failed to get submission statistics")
}
return stats, nil
}
func (s *tenderSubmissionService) GetGlobalStats(ctx context.Context) (*AdminSubmissionStatsResponse, error) {
stats, err := s.repository.GetGlobalStats(ctx)
if err != nil {
s.logger.Error("Failed to get global submission stats", map[string]interface{}{
"error": err.Error(),
})
return nil, errors.New("failed to get submission statistics")
}
return stats, nil
}
func (s *tenderSubmissionService) OnApprovalSubmitted(ctx context.Context, tenderID, companyID, customerID string) error {
existing, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil && !errors.Is(err, ErrTenderSubmissionNotFound) {
return err
}
if existing != nil {
if IsTerminalStatus(existing.Status) {
existing.ReopenForApproval(customerID, "Tender re-approved")
return s.repository.Update(ctx, existing)
}
return nil
}
submission := NewTenderSubmission(tenderID, companyID, customerID)
if err := s.repository.Create(ctx, submission); err != nil {
if errors.Is(err, ErrTenderSubmissionExists) {
existing, getErr := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if getErr != nil {
return getErr
}
if IsTerminalStatus(existing.Status) {
existing.ReopenForApproval(customerID, "Tender re-approved")
return s.repository.Update(ctx, existing)
}
return nil
}
s.logger.Error("Failed to create submission on approval", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"error": err.Error(),
})
return err
}
s.logger.Info("Submission workflow created from approval", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
})
return nil
}
func (s *tenderSubmissionService) OnApprovalRemoved(ctx context.Context, tenderID, companyID string) error {
submission, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return nil
}
return err
}
if submission.Status != StatusOpportunity {
return nil
}
if err := s.repository.Delete(ctx, submission.GetID()); err != nil {
s.logger.Error("Failed to remove opportunity submission after approval toggle", map[string]interface{}{
"submission_id": submission.GetID(),
"error": err.Error(),
})
return err
}
return nil
}
func (s *tenderSubmissionService) OnApprovalRejected(ctx context.Context, tenderID, companyID, changedBy string) error {
submission, err := s.repository.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, ErrTenderSubmissionNotFound) {
return nil
}
return err
}
if IsTerminalStatus(submission.Status) {
return nil
}
if err := s.transitionSubmission(submission, StatusNotApplied, "approval_rejected", changedBy, "Tender approval was rejected"); err != nil {
var transitionErr *InvalidStatusTransitionError
if errors.Is(err, ErrTerminalStatus) || errors.As(err, &transitionErr) {
s.logger.Warn("Skipping approval rejection sync due to invalid transition", map[string]interface{}{
"tender_id": tenderID,
"company_id": companyID,
"current_status": submission.Status,
"target_status": StatusNotApplied,
})
return nil
}
return err
}
return s.repository.Update(ctx, submission)
}
func (s *tenderSubmissionService) transitionSubmission(submission *TenderSubmission, next SubmissionStatus, reason, changedBy, description string) error {
if err := s.validateStatusTransition(submission.Status, next); err != nil {
return err
}
submission.AddStatusChange(next, reason, changedBy, description)
return nil
}
func (s *tenderSubmissionService) requireSubmittedApproval(ctx context.Context, tenderID, companyID string) error {
approval, err := s.approvalReader.GetByTenderAndCompany(ctx, tenderID, companyID)
if err != nil {
if errors.Is(err, tender_approval.ErrTenderApprovalNotFound) {
return ErrApprovalRequired
}
return err
}
if approval.Status != tender_approval.ApprovalStatusSubmitted {
return ErrApprovalRequired
}
return nil
}
func (s *tenderSubmissionService) validateStatusTransition(current, next SubmissionStatus) error {
if IsTerminalStatus(current) {
return ErrTerminalStatus
}
for _, allowed := range AllowedNextStatuses(current) {
if allowed == next {
return nil
}
}
allowed := AllowedNextStatuses(current)
allowedStrings := make([]string, len(allowed))
for i, status := range allowed {
allowedStrings[i] = string(status)
}
return NewInvalidStatusTransitionError(string(current), string(next), allowedStrings)
}
func (s *tenderSubmissionService) responseWithTender(ctx context.Context, submission *TenderSubmission, fullDetails bool) (*TenderSubmissionWithTenderResponse, error) {
tenderResp, err := s.tenderService.GetByID(ctx, submission.TenderID, "")
if err != nil {
s.logger.Warn("Failed to load tender for submission response", map[string]interface{}{
"tender_id": submission.TenderID,
"error": err.Error(),
})
return submission.ToResponseWithTender(nil), nil
}
var details *TenderDetails
if fullDetails {
details = fullTenderDetailsFromResponse(tenderResp)
} else {
details = listTenderDetailsFromResponse(tenderResp)
}
return submission.ToResponseWithTender(details), nil
}
func listParams(form *ListTenderSubmissionsForm) (limit, offset int, sortBy, sortOrder string) {
limit = 20
offset = 0
sortBy = "updated_at"
sortOrder = "desc"
if form == nil {
return limit, offset, sortBy, sortOrder
}
if form.Limit != nil && *form.Limit > 0 {
limit = *form.Limit
}
if limit > maxListLimit {
limit = maxListLimit
}
if form.Offset != nil && *form.Offset >= 0 {
offset = *form.Offset
}
if form.SortBy != nil && strings.TrimSpace(*form.SortBy) != "" {
sortBy = strings.TrimSpace(*form.SortBy)
}
if form.SortOrder != nil && strings.TrimSpace(*form.SortOrder) != "" {
sortOrder = strings.TrimSpace(*form.SortOrder)
}
return limit, offset, sortBy, sortOrder
}
func parseStatuses(values []string) []SubmissionStatus {
if len(values) == 0 {
return nil
}
out := make([]SubmissionStatus, 0, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value == "" {
continue
}
out = append(out, SubmissionStatus(value))
}
return out
}
func parseStages(values []string) []SubmissionStage {
if len(values) == 0 {
return nil
}
out := make([]SubmissionStage, 0, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value == "" {
continue
}
out = append(out, SubmissionStage(value))
}
return out
}
func listMetadata(total int64, limit, offset int) *response.Meta {
pages := int(total) / limit
if limit > 0 && int(total)%limit > 0 {
pages++
}
page := 1
if limit > 0 {
page = (offset / limit) + 1
}
return &response.Meta{
Total: int(total),
Limit: limit,
Offset: offset,
Page: page,
Pages: pages,
}
}
+261
View File
@@ -0,0 +1,261 @@
package tender_submission
import (
"context"
"errors"
"testing"
"tm/pkg/logger"
)
type mockRepository struct {
byTenderCompany map[string]*TenderSubmission
createErr error
updateErr error
deleteErr error
}
func submissionKey(tenderID, companyID string) string {
return tenderID + ":" + companyID
}
func newMockRepository() *mockRepository {
return &mockRepository{byTenderCompany: map[string]*TenderSubmission{}}
}
func (m *mockRepository) Create(_ context.Context, submission *TenderSubmission) error {
if m.createErr != nil {
return m.createErr
}
key := submissionKey(submission.TenderID, submission.CompanyID)
if _, exists := m.byTenderCompany[key]; exists {
return ErrTenderSubmissionExists
}
copy := *submission
m.byTenderCompany[key] = &copy
return nil
}
func (m *mockRepository) GetByID(_ context.Context, id string) (*TenderSubmission, error) {
for _, submission := range m.byTenderCompany {
if submission.GetID() == id {
copy := *submission
return &copy, nil
}
}
return nil, ErrTenderSubmissionNotFound
}
func (m *mockRepository) GetByTenderAndCompany(_ context.Context, tenderID, companyID string) (*TenderSubmission, error) {
submission, ok := m.byTenderCompany[submissionKey(tenderID, companyID)]
if !ok {
return nil, ErrTenderSubmissionNotFound
}
copy := *submission
return &copy, nil
}
func (m *mockRepository) Update(_ context.Context, submission *TenderSubmission) error {
if m.updateErr != nil {
return m.updateErr
}
key := submissionKey(submission.TenderID, submission.CompanyID)
if _, ok := m.byTenderCompany[key]; !ok {
return ErrTenderSubmissionNotFound
}
copy := *submission
m.byTenderCompany[key] = &copy
return nil
}
func (m *mockRepository) UpdateIfUpdatedAt(ctx context.Context, submission *TenderSubmission, expectedUpdatedAt int64) error {
current, err := m.GetByTenderAndCompany(ctx, submission.TenderID, submission.CompanyID)
if err != nil {
return err
}
if current.UpdatedAt != expectedUpdatedAt {
return ErrConcurrentUpdate
}
return m.Update(ctx, submission)
}
func (m *mockRepository) Delete(_ context.Context, id string) error {
if m.deleteErr != nil {
return m.deleteErr
}
for key, submission := range m.byTenderCompany {
if submission.GetID() == id {
delete(m.byTenderCompany, key)
return nil
}
}
return ErrTenderSubmissionNotFound
}
func (m *mockRepository) ListByCompany(context.Context, string, []SubmissionStatus, []SubmissionStage, *int64, *int64, int, int, string, string) ([]*TenderSubmission, int64, error) {
return nil, 0, nil
}
func (m *mockRepository) Search(context.Context, *string, *string, []SubmissionStatus, []SubmissionStage, *int64, *int64, int, int, string, string) ([]*TenderSubmission, int64, error) {
return nil, 0, nil
}
func (m *mockRepository) GetCompanyStats(context.Context, string) (*CompanySubmissionStatsResponse, error) {
return nil, nil
}
func (m *mockRepository) GetGlobalStats(context.Context) (*AdminSubmissionStatsResponse, error) {
return nil, nil
}
func (m *mockRepository) seed(submission *TenderSubmission) {
copy := *submission
m.byTenderCompany[submissionKey(submission.TenderID, submission.CompanyID)] = &copy
}
func newTestService(repo Repository) *tenderSubmissionService {
return &tenderSubmissionService{
repository: repo,
approvalReader: nil,
tenderService: nil,
logger: testLogger{},
}
}
type testLogger struct{}
func (testLogger) Debug(string, map[string]interface{}) {}
func (testLogger) Info(string, map[string]interface{}) {}
func (testLogger) Warn(string, map[string]interface{}) {}
func (testLogger) Error(string, map[string]interface{}) {}
func (testLogger) Fatal(string, map[string]interface{}) {}
func (testLogger) WithFields(map[string]interface{}) logger.Logger {
return testLogger{}
}
func (testLogger) Sync() error { return nil }
func TestOnApprovalSubmittedReopensTerminalSubmission(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusNotApplied, "approval_rejected", "customer-1", "Rejected")
repo.seed(existing)
if err := service.OnApprovalSubmitted(context.Background(), "tender-1", "company-1", "customer-2"); err != nil {
t.Fatalf("OnApprovalSubmitted() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusOpportunity {
t.Fatalf("status = %q, want %q", updated.Status, StatusOpportunity)
}
}
func TestOnApprovalSubmittedCreatesWhenMissing(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
if err := service.OnApprovalSubmitted(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalSubmitted() error = %v", err)
}
created, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if created.Status != StatusOpportunity {
t.Fatalf("status = %q, want %q", created.Status, StatusOpportunity)
}
}
func TestOnApprovalRejectedUsesValidatedTransition(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
repo.seed(existing)
if err := service.OnApprovalRejected(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalRejected() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusNotApplied {
t.Fatalf("status = %q, want %q", updated.Status, StatusNotApplied)
}
}
func TestOnApprovalRejectedSkipsInvalidTransitionFromSentWaiting(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
existing := NewTenderSubmission("tender-1", "company-1", "customer-1")
existing.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
existing.AddStatusChange(StatusApplying, "", "customer-1", "Ready")
existing.AddStatusChange(StatusSentWaiting, "", "customer-1", "Submitted")
repo.seed(existing)
if err := service.OnApprovalRejected(context.Background(), "tender-1", "company-1", "customer-1"); err != nil {
t.Fatalf("OnApprovalRejected() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusSentWaiting {
t.Fatalf("status = %q, want %q", updated.Status, StatusSentWaiting)
}
}
func TestOnApprovalRemovedDeletesOpportunityOnly(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
opportunity := NewTenderSubmission("tender-1", "company-1", "customer-1")
repo.seed(opportunity)
if err := service.OnApprovalRemoved(context.Background(), "tender-1", "company-1"); err != nil {
t.Fatalf("OnApprovalRemoved() error = %v", err)
}
if _, err := repo.GetByTenderAndCompany(context.Background(), "tender-1", "company-1"); !errors.Is(err, ErrTenderSubmissionNotFound) {
t.Fatalf("expected submission to be deleted, got err = %v", err)
}
inProgress := NewTenderSubmission("tender-2", "company-1", "customer-1")
inProgress.AddStatusChange(StatusValidating, "", "customer-1", "Accepted")
repo.seed(inProgress)
if err := service.OnApprovalRemoved(context.Background(), "tender-2", "company-1"); err != nil {
t.Fatalf("OnApprovalRemoved() error = %v", err)
}
updated, err := repo.GetByTenderAndCompany(context.Background(), "tender-2", "company-1")
if err != nil {
t.Fatalf("GetByTenderAndCompany() error = %v", err)
}
if updated.Status != StatusValidating {
t.Fatalf("status = %q, want %q", updated.Status, StatusValidating)
}
}
func TestGetByIDForCompanyWithTenderEnforcesOwnership(t *testing.T) {
repo := newMockRepository()
service := newTestService(repo)
submission := NewTenderSubmission("tender-1", "company-1", "customer-1")
repo.seed(submission)
_, err := service.GetByIDForCompanyWithTender(context.Background(), submission.GetID(), "company-2")
if !errors.Is(err, ErrTenderSubmissionNotFound) {
t.Fatalf("expected not found for other company, got %v", err)
}
}
@@ -0,0 +1,110 @@
package tender_submission
import (
"context"
"tm/internal/tender"
)
func listTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
Title: resp.Title,
Description: resp.Description,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func fullTenderDetailsFromResponse(resp *tender.TenderResponse) *TenderDetails {
if resp == nil {
return nil
}
details := &TenderDetails{
ID: resp.ID,
NoticePublicationID: resp.NoticePublicationID,
PublicationDate: resp.PublicationDate,
Title: resp.Title,
Description: resp.Description,
ProcurementTypeCode: resp.ProcurementTypeCode,
ProcedureCode: resp.ProcedureCode,
EstimatedValue: resp.EstimatedValue,
Currency: resp.Currency,
Duration: resp.Duration,
DurationUnit: resp.DurationUnit,
TenderDeadline: resp.TenderDeadline,
SubmissionDeadline: resp.SubmissionDeadline,
CountryCode: resp.CountryCode,
Status: string(resp.Status),
}
if resp.BuyerOrganization != nil && resp.BuyerOrganization.Name != "" {
details.BuyerOrganization = &OrganizationResponse{Name: resp.BuyerOrganization.Name}
}
return details
}
func uniqueTenderIDs(submissions []*TenderSubmission) []string {
if len(submissions) == 0 {
return nil
}
seen := make(map[string]struct{}, len(submissions))
ids := make([]string, 0, len(submissions))
for _, submission := range submissions {
if submission == nil || submission.TenderID == "" {
continue
}
if _, ok := seen[submission.TenderID]; ok {
continue
}
seen[submission.TenderID] = struct{}{}
ids = append(ids, submission.TenderID)
}
return ids
}
func (s *tenderSubmissionService) loadTendersByIDs(ctx context.Context, tenderIDs []string) map[string]*tender.TenderResponse {
if len(tenderIDs) == 0 {
return map[string]*tender.TenderResponse{}
}
tenders, err := s.tenderService.GetByIDs(ctx, tenderIDs)
if err != nil {
s.logger.Error("Failed to batch load tenders for submissions", map[string]interface{}{
"count": len(tenderIDs),
"error": err.Error(),
})
return map[string]*tender.TenderResponse{}
}
return tenders
}
func (s *tenderSubmissionService) enrichSubmissions(ctx context.Context, submissions []*TenderSubmission, fullDetails bool) []*TenderSubmissionWithTenderResponse {
tenderMap := s.loadTendersByIDs(ctx, uniqueTenderIDs(submissions))
responses := make([]*TenderSubmissionWithTenderResponse, len(submissions))
for i, submission := range submissions {
var details *TenderDetails
if tenderResp, ok := tenderMap[submission.TenderID]; ok {
if fullDetails {
details = fullTenderDetailsFromResponse(tenderResp)
} else {
details = listTenderDetailsFromResponse(tenderResp)
}
}
responses[i] = submission.ToResponseWithTender(details)
}
return responses
}
+23
View File
@@ -252,6 +252,7 @@ type OnboardingRequest struct {
CompanyID string `json:"company_id"`
Documents string `json:"documents"`
WebsiteURL string `json:"website_url"`
Links []string `json:"links,omitempty"`
}
// OnboardingResponse is returned by POST /onboarding.
@@ -388,3 +389,25 @@ func (t *TenderJSON) translationForLanguage(language string) (StoredTranslation,
}
return t.storedTranslationText(language)
}
// hasAnyTranslation reports whether tender.json contains at least one non-empty
// title translation (via translation_status.done or the translations map).
func (t *TenderJSON) hasAnyTranslation() bool {
if t == nil {
return false
}
for _, lang := range t.translationsDone() {
if _, ok := t.storedTranslationText(lang); ok {
return true
}
}
if t.Translations == nil {
return false
}
for _, entry := range t.Translations {
if strings.TrimSpace(entry.Title) != "" {
return true
}
}
return false
}
+83
View File
@@ -654,6 +654,89 @@ func (s *StorageClient) ScanScrapedDocuments(ctx context.Context) ([]ProcedureDo
return results, dailyCounts, nil
}
// ScanTranslatedNotices scans MinIO for tender.json objects with stored translations
// and returns daily notice counts keyed by UTC date (YYYY-MM-DD) plus the total.
func (s *StorageClient) ScanTranslatedNotices(ctx context.Context) (map[string]int64, int64, error) {
s.logger.Debug("Scanning translated notices from MinIO", map[string]interface{}{
"bucket": s.config.MinioBucket,
"prefix": procedurePrefix,
})
dailyCounts := make(map[string]int64)
var total int64
seen := make(map[string]struct{})
var fatalErr error
appendFromPrefix := func(prefix string) bool {
objectCh := s.client.ListObjects(ctx, s.config.MinioBucket, minio.ListObjectsOptions{
Prefix: prefix,
Recursive: true,
})
for object := range objectCh {
if object.Err != nil {
fatalErr = s.logMinIOFailure("list_translated_notices", object.Err, map[string]interface{}{
"prefix": prefix,
})
return false
}
key := object.Key
if !strings.HasSuffix(key, "/tender.json") {
continue
}
if _, dup := seen[key]; dup {
continue
}
seen[key] = struct{}{}
tenderJSON, err := s.readTenderJSONAtKey(ctx, key)
if err != nil {
if errors.Is(err, ErrObjectNotFound) {
continue
}
if errors.Is(err, ErrMinIOUnavailable) {
fatalErr = err
return false
}
s.logger.Debug("Skipping tender.json during translation scan", map[string]interface{}{
"object_key": key,
"error": err.Error(),
})
continue
}
if !tenderJSON.hasAnyTranslation() {
continue
}
total++
modified := normalizeUnixSeconds(object.LastModified.Unix())
if modified > 0 {
date := time.Unix(modified, 0).UTC().Format("2006-01-02")
dailyCounts[date]++
}
}
return true
}
if !appendFromPrefix(procedurePrefix) {
return nil, 0, fatalErr
}
if len(seen) == 0 {
if !appendFromPrefix("") {
return nil, 0, fatalErr
}
}
if fatalErr != nil {
return nil, 0, fatalErr
}
s.logger.Info("Scanned translated notices from storage", map[string]interface{}{
"bucket": s.config.MinioBucket,
"total_count": total,
})
return dailyCounts, total, nil
}
func normalizeUnixSeconds(ts int64) int64 {
if ts > 1_000_000_000_000 {
return ts / 1000
+3
View File
@@ -64,12 +64,15 @@ func NewPagination(c echo.Context) (*Pagination, error) {
sortOrder = "desc"
}
includeTotal := c.QueryParam("include_total") == "true"
return &Pagination{
Limit: limit,
Offset: offset,
Cursor: cursor,
SortBy: sortBy,
SortOrder: sortOrder,
IncludeTotal: includeTotal,
}, nil
}
+16 -1
View File
@@ -18,11 +18,26 @@ func TestNewPaginationDefaults(t *testing.T) {
if err != nil {
t.Fatalf("NewPagination: %v", err)
}
if p.Limit != 10 || p.Offset != 0 || p.Cursor != "" {
if p.Limit != 10 || p.Offset != 0 || p.Cursor != "" || p.IncludeTotal {
t.Fatalf("unexpected pagination: %+v", p)
}
}
func TestNewPaginationIncludeTotal(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/?include_total=true", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
p, err := NewPagination(c)
if err != nil {
t.Fatalf("NewPagination: %v", err)
}
if !p.IncludeTotal {
t.Fatal("expected include_total=true")
}
}
func TestNewPaginationCursorConflict(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/?limit=10&offset=5&cursor=abc", nil)
+1
View File
@@ -43,6 +43,7 @@ type Pagination struct {
Cursor string `query:"cursor" valid:"optional"`
SortBy string `query:"sort_by" valid:"optional"`
SortOrder string `query:"sort_order" valid:"optional,in(asc|desc)"`
IncludeTotal bool `query:"include_total" valid:"optional"`
}
// Success returns a successful response
+1 -1
View File
@@ -19,7 +19,7 @@ const (
DailyJobStatusCompleted = "completed"
TEDScraperJobName = "ted_scraper"
AIPipelineAutoJobName = "ai_pipeline_auto"
AIPipelineDailyJobName = "ai_pipeline_daily"
)
type dailyJobRun struct {
+22
View File
@@ -226,6 +226,7 @@ func (s *TEDScraper) mapPriorInformationNoticeToTender(pin *PriorInformationNoti
t.BuyerOrganization = s.mapOrganization(buyerOrg, "buyer")
}
}
backfillNoticeLocationFromBuyer(t)
reviewID := pin.GetReviewOrganizationID()
if reviewID != "" {
@@ -407,6 +408,7 @@ func (s *TEDScraper) mapToTender(cn *ContractNotice, sourceFileURL, sourceFileNa
t.BuyerOrganization = s.mapOrganization(buyerOrg, "buyer")
}
}
backfillNoticeLocationFromBuyer(t)
// Set review organization
reviewID := cn.GetReviewOrganizationID()
@@ -570,6 +572,7 @@ func (s *TEDScraper) mapContractAwardNoticeToTender(can *ContractAwardNotice, so
t.BuyerOrganization = s.mapOrganization(buyerOrg, "buyer")
}
}
backfillNoticeLocationFromBuyer(t)
// Set all organizations
if orgs := can.GetOrganizations(); orgs != nil {
@@ -667,6 +670,25 @@ func (s *TEDScraper) mapOrganization(tedOrg *Organization, role string) *notice.
return org
}
func backfillNoticeLocationFromBuyer(t *notice.Notice) {
if t == nil || t.BuyerOrganization == nil {
return
}
buyer := t.BuyerOrganization.Address
if strings.TrimSpace(t.CountryCode) == "" && strings.TrimSpace(buyer.CountryCode) != "" {
t.CountryCode = strings.TrimSpace(buyer.CountryCode)
}
if strings.TrimSpace(t.CityName) == "" && strings.TrimSpace(buyer.CityName) != "" {
t.CityName = strings.TrimSpace(buyer.CityName)
}
if strings.TrimSpace(t.PostalCode) == "" && strings.TrimSpace(buyer.PostalZone) != "" {
t.PostalCode = strings.TrimSpace(buyer.PostalZone)
}
if strings.TrimSpace(t.RegionCode) == "" && strings.TrimSpace(buyer.CountrySubentityCode) != "" {
t.RegionCode = strings.TrimSpace(buyer.CountrySubentityCode)
}
}
func backfillNoticeEstimatedValueFromLots(t *notice.Notice) {
if t == nil || t.EstimatedValue > 0 {
return
+64
View File
@@ -0,0 +1,64 @@
package ted
import (
"testing"
"tm/internal/notice"
)
func TestBackfillNoticeLocationFromBuyer(t *testing.T) {
n := &notice.Notice{
BuyerOrganization: &notice.Organization{
Address: notice.Address{
CountryCode: "POL",
CityName: "Warsaw",
PostalZone: "00-001",
CountrySubentityCode: "PL14",
},
},
}
backfillNoticeLocationFromBuyer(n)
if n.CountryCode != "POL" {
t.Fatalf("expected country POL, got %q", n.CountryCode)
}
if n.CityName != "Warsaw" {
t.Fatalf("expected city Warsaw, got %q", n.CityName)
}
if n.PostalCode != "00-001" {
t.Fatalf("expected postal 00-001, got %q", n.PostalCode)
}
if n.RegionCode != "PL14" {
t.Fatalf("expected region PL14, got %q", n.RegionCode)
}
}
func TestBackfillNoticeLocationFromBuyerDoesNotOverwrite(t *testing.T) {
n := &notice.Notice{
CountryCode: "DEU",
CityName: "Berlin",
PostalCode: "10115",
RegionCode: "DE300",
BuyerOrganization: &notice.Organization{
Address: notice.Address{
CountryCode: "POL",
CityName: "Warsaw",
PostalZone: "00-001",
CountrySubentityCode: "PL14",
},
},
}
backfillNoticeLocationFromBuyer(n)
if n.CountryCode != "DEU" {
t.Fatalf("expected country DEU, got %q", n.CountryCode)
}
if n.CityName != "Berlin" {
t.Fatalf("expected city Berlin, got %q", n.CityName)
}
if n.PostalCode != "10115" {
t.Fatalf("expected postal 10115, got %q", n.PostalCode)
}
if n.RegionCode != "DE300" {
t.Fatalf("expected region DE300, got %q", n.RegionCode)
}
}