- Removed the QueueConfig structure and related queue management files as they are no longer in use.
- Updated the worker initialization to reflect the removal of queue-related configurations.
- Cleaned up the bootstrap process by eliminating deprecated logging related to the queue system.
This update streamlines the worker's configuration and prepares the codebase for future enhancements without the legacy queue management components.
Paginate un-summarized tenders and mark rows missing notice/folder IDs as handled so the worker cannot spin forever. Return 400/404 for validation and not-found cases on AI summarize/analyze triggers instead of leaking internal errors as 500.
- Added a new endpoint to trigger on-demand agentic analysis for tenders via the AI service.
- Introduced `TriggerAIAnalyze` method in the `TenderHandler` to handle requests and responses for AI analysis.
- Updated the `tenderService` to include `TriggerAIAnalyze` method, which validates input and interacts with the AI summarizer client.
- Enhanced the `AIAnalyzeResponse` and `AIAnalyzeDocument` structures to support the new analysis feature.
- Refactored the `DocumentSummarizationWorker` and `TranslationWorker` to remove deprecated MinIO dependencies, streamlining the AI service interactions.
This update improves the functionality of the tender management system by allowing users to trigger AI analysis on-demand, enhancing the overall user experience and system capabilities.
- Updated `InitAISummarizerClient` to accept `mongoManager` for tracking translation success.
- Introduced new `Statistics` endpoint in the dashboard to fetch scraping and translation statistics.
- Enhanced `TranslationWorker` to utilize the new success counter for tracking successful translations.
- Added necessary data structures and query forms for statistics reporting.
This refactor improves the tracking of AI translation success and provides new insights through the dashboard statistics.
- Refactored the NoticeWorker to generate unique tender IDs using the PBL naming convention (SCDYYNNN format) and project names based on client and opportunity details.
- Introduced methods for generating tender IDs and project names, including logic for extracting client and opportunity names, ensuring consistency and clarity in naming.
- Updated the Tender entity to include a new ProjectName field, enhancing the data structure for better project identification.
- Added MongoDB indexing for the new project_name field to optimize query performance.
- Improved error handling and logging during the tender ID generation process, ensuring robustness in the worker's functionality.
- Updated the worker initialization process to include the new GLM SDK, enhancing the worker's capabilities for translation tasks.
- Modified the InitWorker function and NewNoticeWorker constructor to accept the GLM service, ensuring a cohesive integration.
- Implemented the GLM service initialization and logging for successful setup, improving maintainability and usability.
- Updated the NoticeWorker to utilize the GLM SDK for translating notice titles and descriptions, enhancing functionality and user experience.
- Removed the Ollama SDK from the worker initialization process, simplifying the worker's dependencies and enhancing maintainability.
- Updated the InitWorker function and NewNoticeWorker constructor to reflect the removal of the Ollama SDK, ensuring a cleaner and more focused implementation.
- Commented out the Ollama model listing logic for potential future use, maintaining clarity in the main function while reducing unnecessary complexity.
- Introduced a new build step in the Drone CI configuration for the worker service, enabling automated Docker image creation.
- Added a Dockerfile for the worker service, defining the build process and dependencies, ensuring a streamlined deployment for background tasks.
- Enhanced the overall CI/CD pipeline to support the new worker service, improving the project's build and deployment capabilities.
- Updated the ToTender method to accept a tender instance, allowing for direct population of tender fields from the notice entity.
- Simplified the tender creation and update process by checking if the tender ID is zero, streamlining the logic for handling tender entities.
- Removed the commented-out code related to AI translation, improving code clarity and maintainability.
- Enhanced error logging for tender creation and update failures, ensuring better visibility into potential issues during processing.
- Introduced AlertMail configuration in both scraper and worker bootstrap files, allowing for customizable email notifications.
- Updated the TED scraper to utilize the AlertMail configuration for sending completion notifications, improving flexibility in notification management.
- Enhanced error logging in the worker's main function to capture issues when listing Ollama models, ensuring better visibility into potential failures.
- Refactored notification sending logic to check for a valid AlertMail before dispatching emails, ensuring notifications are only sent when configured.
- Improved overall structure and readability of the bootstrap configuration files, aligning with best practices for maintainability.
- Introduced a new worker service with a dedicated main entry point for handling background tasks, including MongoDB connection management and notification service initialization.
- Added a bootstrap package to manage application configuration, logging, and service initialization for the worker.
- Implemented a NoticeWorker to process unprocessed notices and create corresponding tender entities, enhancing the integration of notice management within the tender system.
- Refactored the tender service to remove the Ollama SDK dependency, streamlining the service initialization in the main application.
- Enhanced the notice repository with a method to retrieve unprocessed notices, improving data handling capabilities.
Mazyar