- Changed the default MinIO bucket name from "opplens-documents" to "opplens" across multiple configuration files.
- Introduced caching for dashboard statistics in the service layer to improve performance and reduce redundant data fetching.
- Implemented a mutex for thread-safe access to cached statistics, ensuring data integrity during concurrent requests.
This update streamlines the configuration for the AI summarizer and optimizes the dashboard service, enhancing overall system efficiency.
- Updated the AI pipeline service to include a new `ScrapedDocumentMetadataSyncer` interface for persisting scraped document metadata onto tender records.
- Modified the `NewService` function to accept the new metadata syncer dependency.
- Implemented synchronization of scraped document metadata in the `ScrapeDocuments` and `Run` methods.
- Enhanced the tender service to enrich search filters based on scraped documents and added a new method for syncing scraped documents from storage.
- Updated the `SearchForm` to include `ContractFolderIDsWithDocuments` for better handling of scraped documents in queries.
This update improves the integration of scraped document handling within the AI pipeline, enhancing data consistency and operational efficiency in the tender management system.
- Added new routes and handlers for AI pipeline operations, including scraping documents, batch summarization, translation, and syncing with the Opplens AI service.
- Introduced request forms for handling tender references and batch operations.
- Enhanced the AI service with methods for triggering batch operations and managing pipeline runs.
- Updated Swagger documentation to reflect the new AI pipeline endpoints and their functionalities.
This update integrates comprehensive AI pipeline capabilities into the tender management system, improving operational efficiency and user experience.
- Removed the QueueConfig structure and related queue management files as they are no longer in use.
- Updated the worker initialization to reflect the removal of queue-related configurations.
- Cleaned up the bootstrap process by eliminating deprecated logging related to the queue system.
This update streamlines the worker's configuration and prepares the codebase for future enhancements without the legacy queue management components.
Paginate un-summarized tenders and mark rows missing notice/folder IDs as handled so the worker cannot spin forever. Return 400/404 for validation and not-found cases on AI summarize/analyze triggers instead of leaking internal errors as 500.
- Added a new endpoint to trigger on-demand agentic analysis for tenders via the AI service.
- Introduced `TriggerAIAnalyze` method in the `TenderHandler` to handle requests and responses for AI analysis.
- Updated the `tenderService` to include `TriggerAIAnalyze` method, which validates input and interacts with the AI summarizer client.
- Enhanced the `AIAnalyzeResponse` and `AIAnalyzeDocument` structures to support the new analysis feature.
- Refactored the `DocumentSummarizationWorker` and `TranslationWorker` to remove deprecated MinIO dependencies, streamlining the AI service interactions.
This update improves the functionality of the tender management system by allowing users to trigger AI analysis on-demand, enhancing the overall user experience and system capabilities.
- Updated `InitAISummarizerClient` to accept `mongoManager` for tracking translation success.
- Introduced new `Statistics` endpoint in the dashboard to fetch scraping and translation statistics.
- Enhanced `TranslationWorker` to utilize the new success counter for tracking successful translations.
- Added necessary data structures and query forms for statistics reporting.
This refactor improves the tracking of AI translation success and provides new insights through the dashboard statistics.
- Refactored the NoticeWorker to generate unique tender IDs using the PBL naming convention (SCDYYNNN format) and project names based on client and opportunity details.
- Introduced methods for generating tender IDs and project names, including logic for extracting client and opportunity names, ensuring consistency and clarity in naming.
- Updated the Tender entity to include a new ProjectName field, enhancing the data structure for better project identification.
- Added MongoDB indexing for the new project_name field to optimize query performance.
- Improved error handling and logging during the tender ID generation process, ensuring robustness in the worker's functionality.
- Updated the tender repository to create a unique index on tender_id, enhancing database query performance.
- Introduced new TED mapping functions to convert parsed TED documents into tender entities, improving integration with the tender management system.
- Added error handling for contract notice processing in the TED scraper, ensuring robust logging and error management.
- Removed deprecated eform structures to streamline the codebase and focus on essential components.
Mazyar