- Introduced `documents_scraped_filter.go` to handle filtering tenders based on the presence of documents in MinIO, utilizing a caching mechanism to optimize performance.
- Implemented `enrichDocumentsScrapedFilter` and `cachedContractFolderIDsWithDocuments` methods in the `tenderService` for efficient retrieval of contract folder IDs with documents.
- Added unit tests in `documents_scraped_filter_test.go` to validate the new filtering logic and caching behavior, ensuring accurate results and error handling.
- Updated `SearchForm` documentation to clarify the caching aspect of `ContractFolderIDsWithDocuments`.
- Enhanced comments in the handler and repository to reflect the new filtering logic and its implications.
This update improves the tender search functionality by efficiently managing document presence checks, leading to better performance and user experience in the tender management system.
- Introduced a new `widgetCache` structure to manage caching for various dashboard widgets, improving performance and reducing redundant data retrieval.
- Updated the `ClosingSoon` and `Trend` methods in the service to utilize the new caching mechanism, ensuring efficient data access and reducing load on the database.
- Enhanced the `NoticeTypes` and `Countries` methods to implement caching, further optimizing the dashboard's responsiveness.
- Added a new `searchListCache` implementation in the tender service to cache search results, improving the efficiency of tender searches and reducing database load.
- Implemented robust error handling and logging for cache operations, ensuring better visibility into potential issues.
This update significantly enhances the performance and scalability of the dashboard and tender services by integrating effective caching strategies, leading to improved user experience and resource management.
- Updated the tender search handler to include a new `include_total` query parameter, allowing clients to request total count and pagination metadata for search results.
- Refactored the `tenderSearchListProjection` to exclude heavy fields from the list response, optimizing data retrieval.
- Modified the `buildSearchFilter` method to utilize the `processing_metadata.documents_scraped` flag for filtering, improving search accuracy.
- Added unit tests for the new pagination features and search filter logic, ensuring robust validation of the search functionality.
This update significantly enhances the tender search experience by providing more flexible pagination options and improving the efficiency of data handling.
- Introduced `PickAssignedCompanyID` function to determine the appropriate company ID for company-scoped operations, improving company assignment logic.
- Added `ResolveMongoID` method in the tender service to map MongoDB IDs and AI procedure references to canonical tender IDs, enhancing ID resolution.
- Updated `ToggleTenderApproval` to handle company ID validation and improve error handling, ensuring robust processing of tender approvals.
- Enhanced logging for error scenarios in the tender approval service, providing better insights into failures during operations.
- Refactored the `GetByID` and `GetByTenderAndCompany` methods in the tender approval repository to utilize custom error types for improved clarity.
This update significantly improves the handling of company and tender operations, enhancing error management and overall service reliability.
- Removed the `buildRecommendedTenderResponsesParallel` method and replaced it with `buildRecommendedTenderListResponses`, simplifying the response building process.
- Introduced `enrichRecommendedTenderResponsesParallel` to handle enrichment of tender responses in parallel, improving performance.
- Updated the `Recommend` method to utilize the new enrichment logic, ensuring timely and efficient processing of recommendations.
- Cleaned up unused functions and imports in `recommendation_page_cache.go` and `recommendation_response.go`, enhancing code clarity and maintainability.
This update streamlines the recommendation response handling and improves the overall efficiency of the tender recommendation service.
- Removed the `RecommendationPageCacheLanguages` configuration from `AISummarizerConfig` to streamline cache management.
- Updated the `companyService` and `tenderService` to utilize the new `InvalidateRecommendedTendersPageCache` method for cache invalidation, enhancing clarity and efficiency.
- Refactored the `invalidateRecommendedTendersPageCache` method to eliminate unnecessary context parameters, simplifying the function signature.
- Improved the handling of page cache refresh logic by consolidating language handling within the `tenderService`, ensuring consistent behavior across services.
- Cleaned up related tests and removed deprecated functions to maintain code quality and readability.
This update enhances the maintainability of the recommendation caching system by simplifying configuration and improving cache invalidation logic.
- Introduced a new `RecommendedTendersPageCacheRefresher` interface to manage the asynchronous refresh of recommendation pages in Redis, improving cache management.
- Updated the `companyService` to support setting page cache languages and refreshing the recommended tenders page cache based on company IDs.
- Enhanced the `tenderService` to build and invalidate recommended tenders page caches, ensuring timely updates and efficient retrieval of cached recommendations.
- Added configuration options for recommendation page cache languages in the `AISummarizerConfig`, allowing for flexible language support.
- Implemented unit tests for the new caching logic and page refresh functionality, ensuring robust validation of the recommendation caching process.
This update significantly improves the efficiency and responsiveness of the tender recommendation service by integrating enhanced caching mechanisms and page refresh capabilities.
- Introduced a context with a timeout for Redis cache retrieval in the `getCachedAIRecommendations` method, ensuring more robust handling of potential delays.
- Added a new file `recommendation_page.go` to encapsulate the logic for building recommendation pages, improving code organization and readability.
- Implemented a `buildRecommendationPage` method to handle pagination and batch processing of recommendations, enhancing performance and user experience.
- Created a new test file `recommendation_page_test.go` to validate the functionality of the recommendation page building process, ensuring correct pagination and batch resolution.
This update significantly improves the efficiency and maintainability of the tender recommendation service by optimizing caching and implementing structured pagination handling.
- Renamed the test function to `TestBuildRecommendedTenderListResponsesPreservesOrderWithoutStorage` for clarity and updated its logic to reflect changes in the recommendation response building process.
- Removed unnecessary sleep in the test to enhance performance.
- Refactored the `buildRecommendedTenderResponsesParallel` method to `buildRecommendedTenderListResponses`, simplifying the response building without relying on MinIO translation lookups.
- Updated the repository's `mapProcedureReferencesToTenders` function to improve efficiency by grouping candidates by their contract folder IDs, enhancing the mapping process.
This update streamlines the recommendation response handling and enhances the test suite for better maintainability and performance.
- Introduced a caching mechanism for translation responses in the tender recommendation service, improving efficiency by reducing redundant translation requests.
- Refactored the `enrichWithTranslation` method to support an optional translation cache, allowing for faster retrieval of stored translations.
- Implemented parallel processing for building recommended tender responses, utilizing goroutines to enhance performance and responsiveness.
- Added unit tests to validate the new caching behavior and ensure the correct application of translations in various scenarios.
This update significantly improves the performance and responsiveness of the tender recommendation service by integrating caching and parallel processing, enhancing the overall user experience.
- Introduced a new function `dedupeRecommendationResponsesByTenderID` to remove duplicate tender recommendations based on their IDs, ensuring unique entries in the recommendation list.
- Updated the `mergeRecommendedTenders` function to utilize the new deduplication logic, streamlining the merging process of recommendation lists.
- Enhanced the `fetchAIRecommendations` method to apply deduplication on the fetched recommendations, improving data integrity.
- Added unit tests for the new deduplication function to validate its behavior and ensure correct handling of various input scenarios.
This update enhances the recommendation handling by ensuring that duplicate tender entries are effectively managed, improving the overall quality of the recommendations provided to users.
- Introduced a new test file `recommendation_translation_test.go` to validate the behavior of the tender recommendation service regarding translation handling.
- Implemented tests to ensure that the service correctly applies stored translations and retains original text when no translation is available.
- Enhanced the `Recommend` method in the tender service to utilize a structured approach for managing recommended tenders, improving clarity and maintainability.
This update improves the testing coverage for translation handling in tender recommendations, ensuring accurate responses based on available translations and original content.
- Renamed `buildRejectedTenderIDSetForCompanies` to `buildRejectedTenderIDSetsForCompanies` to better reflect its functionality of returning a map of excluded tender IDs by company.
- Updated the implementation to return a nested map structure for better organization of excluded tender IDs per company.
- Added a new test function `TestIsRecommendedTenderExcludedForAllCompanies` to validate the visibility of tenders based on company-specific exclusions.
- Enhanced existing tests to ensure comprehensive coverage of the new logic for handling rejected tenders.
This update improves the clarity and functionality of the tender recommendation process by refining the handling of rejected tenders and expanding test coverage.
- Introduced a new test function `TestMapProcedureReferencesToTenders` to validate the mapping of procedure references to corresponding tenders.
- Enhanced the `GetByProcedureReferences` method in the tender repository to utilize a more efficient filtering mechanism based on unique contract folder IDs.
- Updated pagination logic to dynamically adjust limits based on the number of unique folders and references, ensuring optimal data retrieval.
This update improves the testing coverage for tender mapping functionality and optimizes the repository's data fetching strategy, enhancing overall performance and reliability.
- Added functionality to retrieve merged AI recommendations for multiple companies, improving the relevance of tender suggestions based on company-specific data.
- Introduced normalization functions to clean and deduplicate company IDs, ensuring accurate processing of recommendations.
- Enhanced the company context resolution in customer middleware to support multiple assigned companies, improving the handling of company-specific requests.
- Updated the tender recommendation logic to utilize the new merged recommendations and handle exclusions for rejected tenders accordingly.
- Added unit tests to verify the new recommendation merging logic and company ID normalization, ensuring robust functionality.
This update significantly enhances the tender recommendation process by allowing for more comprehensive and relevant suggestions based on multiple company contexts, improving user experience and satisfaction.
- Updated the tender service to include a new dependency for listing rejected tenders by company, allowing for more refined tender recommendations.
- Introduced a new field in the SearchForm to specify whether to exclude rejected tenders from the recommendation results.
- Enhanced the Recommend method to filter out rejected tenders based on the new exclusion logic, improving the relevance of AI-ranked tender recommendations.
- Added unit tests to verify the exclusion logic for rejected tenders, ensuring robust functionality.
This update improves the tender recommendation process by ensuring that company-rejected tenders are not included in the results, enhancing user experience and satisfaction.
- Implemented the IsRecommendable method in the Tender entity to determine if a tender should be included in AI recommendations based on its status and deadline.
- Added unit tests for the IsRecommendable method to cover various scenarios, ensuring accurate recommendation logic.
- Updated the Recommend method in the tender service to utilize the new IsRecommendable method for improved clarity and functionality.
This update enhances the recommendation logic for tenders, ensuring only appropriate tenders are considered for AI recommendations based on their status and deadlines.
- Introduced `ProcedureReference` struct to encapsulate AI procedure coordinates for better data management.
- Added `GetByProcedureReferences` method in the `TenderRepository` to retrieve multiple tenders based on AI procedure references in a single query.
- Updated the `resolveRecommendedTenders` method in the service layer to utilize the new repository method, improving efficiency in fetching recommended tenders.
- Enhanced error handling and logging for the new repository method to ensure robust operation.
This update improves the handling of AI procedure references, streamlining tender retrieval processes and enhancing overall system performance.
- Introduced `FormatAIProcedureRef` and `ParseAIProcedureRef` functions in the `tender` domain for handling AI service tender references.
- Added unit tests for these functions in `ai_reference_test.go` to ensure correct parsing and formatting behavior.
- Updated the `TenderResponse` struct to include a new `ProcedureRef` field for improved data representation.
- Enhanced the `GetByProcedureReference` method in the repository to retrieve tenders based on the new procedure reference format.
- Modified the `Recommend` method in the service layer to utilize the new procedure reference handling, improving the recommendation process.
This update enhances the handling of AI procedure references, ensuring better data integrity and usability in the tender management system.
- Added new fields `Rank` and `Analysis` to the `TenderResponse` struct for improved data representation.
- Updated the `RecommendTenders` handler to reflect changes in API documentation, including a new summary and description for AI-ranked tender recommendations.
- Improved error handling in the recommendation process, ensuring appropriate responses for missing company IDs and unavailable AI services.
This update enhances the tender response capabilities and improves the clarity of the recommendation API, aligning with the overall goal of providing better insights for users.
- Removed the DocumentSummarizationWorker and its related scheduling logic from the worker bootstrap.
- Updated the AI summarizer client initialization comment for clarity.
- Added a new error type for cases when tender documents have not been scraped yet, enhancing error handling in the tender service.
- Modified API documentation to reflect changes in AI summary retrieval logic, ensuring accurate descriptions of on-demand summarization behavior.
This update streamlines the AI summarization process by eliminating the document summarization worker, improving overall system efficiency and clarity in error handling.
- Updated the AI pipeline service to include a new `ScrapedDocumentMetadataSyncer` interface for persisting scraped document metadata onto tender records.
- Modified the `NewService` function to accept the new metadata syncer dependency.
- Implemented synchronization of scraped document metadata in the `ScrapeDocuments` and `Run` methods.
- Enhanced the tender service to enrich search filters based on scraped documents and added a new method for syncing scraped documents from storage.
- Updated the `SearchForm` to include `ContractFolderIDsWithDocuments` for better handling of scraped documents in queries.
This update improves the integration of scraped document handling within the AI pipeline, enhancing data consistency and operational efficiency in the tender management system.
- Added new routes and handlers for AI pipeline operations, including scraping documents, batch summarization, translation, and syncing with the Opplens AI service.
- Introduced request forms for handling tender references and batch operations.
- Enhanced the AI service with methods for triggering batch operations and managing pipeline runs.
- Updated Swagger documentation to reflect the new AI pipeline endpoints and their functionalities.
This update integrates comprehensive AI pipeline capabilities into the tender management system, improving operational efficiency and user experience.
Paginate un-summarized tenders and mark rows missing notice/folder IDs as handled so the worker cannot spin forever. Return 400/404 for validation and not-found cases on AI summarize/analyze triggers instead of leaking internal errors as 500.
- Added a new endpoint to trigger on-demand agentic analysis for tenders via the AI service.
- Introduced `TriggerAIAnalyze` method in the `TenderHandler` to handle requests and responses for AI analysis.
- Updated the `tenderService` to include `TriggerAIAnalyze` method, which validates input and interacts with the AI summarizer client.
- Enhanced the `AIAnalyzeResponse` and `AIAnalyzeDocument` structures to support the new analysis feature.
- Refactored the `DocumentSummarizationWorker` and `TranslationWorker` to remove deprecated MinIO dependencies, streamlining the AI service interactions.
This update improves the functionality of the tender management system by allowing users to trigger AI analysis on-demand, enhancing the overall user experience and system capabilities.
- Updated `InitAISummarizerClient` to accept `mongoManager` for tracking translation success.
- Introduced new `Statistics` endpoint in the dashboard to fetch scraping and translation statistics.
- Enhanced `TranslationWorker` to utilize the new success counter for tracking successful translations.
- Added necessary data structures and query forms for statistics reporting.
This refactor improves the tracking of AI translation success and provides new insights through the dashboard statistics.
Mazyar