- Introduced the `tender_submission` domain, including entity, repository, service, and handler implementations for managing tender submissions.
- Added new routes for both admin and public access to tender submissions, allowing for listing, ensuring, and retrieving submissions by ID and tender.
- Enhanced the tender approval service to synchronize submission workflows with approval changes, ensuring proper state management.
- Implemented validation and response structures for tender submission operations, improving API consistency and usability.
- Added unit tests for tender submission status transitions and workflow logic, ensuring robust functionality.
This update enhances the tender management system by providing comprehensive support for tender submissions, improving overall workflow and user experience.
- Eliminated the `ContentXML` field from the `Tender` struct in the entity definition, streamlining the data model.
- Removed the `FindTendersWithContentXML` method from the `TenderRepository` interface, as it is no longer necessary.
- Updated the `ToTender` method in the `NoticeWorker` to reflect these changes, ensuring consistency across the codebase.
This update simplifies the tender data structure and repository interface, improving maintainability and clarity in the tender management system.
- Removed the `RecommendationPageCacheLanguages` configuration from `AISummarizerConfig` to streamline cache management.
- Updated the `companyService` and `tenderService` to utilize the new `InvalidateRecommendedTendersPageCache` method for cache invalidation, enhancing clarity and efficiency.
- Refactored the `invalidateRecommendedTendersPageCache` method to eliminate unnecessary context parameters, simplifying the function signature.
- Improved the handling of page cache refresh logic by consolidating language handling within the `tenderService`, ensuring consistent behavior across services.
- Cleaned up related tests and removed deprecated functions to maintain code quality and readability.
This update enhances the maintainability of the recommendation caching system by simplifying configuration and improving cache invalidation logic.
- Introduced a new `RecommendedTendersPageCacheRefresher` interface to manage the asynchronous refresh of recommendation pages in Redis, improving cache management.
- Updated the `companyService` to support setting page cache languages and refreshing the recommended tenders page cache based on company IDs.
- Enhanced the `tenderService` to build and invalidate recommended tenders page caches, ensuring timely updates and efficient retrieval of cached recommendations.
- Added configuration options for recommendation page cache languages in the `AISummarizerConfig`, allowing for flexible language support.
- Implemented unit tests for the new caching logic and page refresh functionality, ensuring robust validation of the recommendation caching process.
This update significantly improves the efficiency and responsiveness of the tender recommendation service by integrating enhanced caching mechanisms and page refresh capabilities.
- Updated the company service to include a new method for scheduling the refresh of cached AI recommendations after the AI pipeline execution.
- Introduced a new interface for managing cached recommendation refreshes, improving the separation of concerns within the service layer.
- Enhanced the worker initialization to include Redis client support, allowing for better management of recommendation caching.
- Added functionality to list company IDs with existing recommendation caches, ensuring efficient updates post-pipeline runs.
- Implemented unit tests to validate the new recommendation refresh logic and ensure proper handling of various scenarios.
This update significantly improves the handling of AI recommendations by integrating caching mechanisms with the AI pipeline, enhancing overall system performance and responsiveness.
- Renamed and refactored the AI pipeline auto run functionality to a daily run, enhancing clarity and purpose.
- Introduced a new `AIPipelineDailyWorker` to manage the daily execution of the AI pipeline, replacing the previous auto run implementation.
- Updated configuration fields and logging messages to reflect the change from auto to daily run, ensuring consistent terminology throughout the codebase.
- Removed the obsolete `ai_pipeline_auto.go` file to streamline the worker structure.
This update improves the maintainability and readability of the AI pipeline management by clearly distinguishing between auto and daily run functionalities.
- Introduced the ability to append external links to company profiles through a new API endpoint.
- Enhanced the `Company` entity to include a `Links` field for storing external resource links.
- Created `AddLinksForm` for validating incoming link data and implemented corresponding logic in the service layer.
- Added error handling for link validation, ensuring only valid URLs are accepted and limiting the number of links to 20.
- Implemented unit tests for link management functions, including sanitization and merging of links.
- Updated relevant API documentation to reflect the new functionality.
This update significantly enhances the company management capabilities by allowing the addition of external links, improving the overall user experience and data richness in company profiles.
- Updated the tender service to include a new dependency for listing rejected tenders by company, allowing for more refined tender recommendations.
- Introduced a new field in the SearchForm to specify whether to exclude rejected tenders from the recommendation results.
- Enhanced the Recommend method to filter out rejected tenders based on the new exclusion logic, improving the relevance of AI-ranked tender recommendations.
- Added unit tests to verify the exclusion logic for rejected tenders, ensuring robust functionality.
This update improves the tender recommendation process by ensuring that company-rejected tenders are not included in the results, enhancing user experience and satisfaction.
- Introduced AIPipelineAutoWorker to manage the execution of the AI pipeline auto run, including startup catch-up and scheduled tasks.
- Enhanced WorkerConfig to include AIPipelineAutoEnabled and AIPipelineAutoInterval settings for better control over AI pipeline execution.
- Added logging for AI pipeline auto run status, including success and error handling, to improve observability.
- Updated daily job tracker to include AIPipelineAutoJobName for tracking AI pipeline job completions.
This update enhances the system's capability to automate AI pipeline executions, improving efficiency and reliability in processing AI tasks.
- Introduced CompanyContextMiddleware to resolve the active company context for customer requests, ensuring that tender recommendations and company-scoped APIs remain in sync with the database.
- Updated public routes to utilize the new CompanyContextMiddleware alongside the existing AuthMiddleware, improving the handling of company-specific requests.
- Added unit tests for the pickActiveCompanyID function to validate the logic for selecting the appropriate company context based on customer assignments and requested company IDs.
This update enhances the accuracy and reliability of company context management in the application, improving user experience and data consistency.
- Updated the dashboard service to integrate Redis caching for improved performance in statistics retrieval.
- Modified the NewService function to accept a Redis client, enabling caching of dashboard statistics.
- Implemented logic to retrieve statistics from Redis, falling back to the database if necessary, and introduced a background process to warm the cache.
- Enhanced error handling and logging for Redis operations to ensure robust statistics management.
- Increased cache duration for scraped documents and adjusted timeout settings for MongoDB queries to optimize performance.
This update significantly improves the responsiveness and efficiency of the dashboard by leveraging Redis for caching statistics.
- Introduced a new `auditlog` package to handle audit logging for user actions, including creation, updates, deletions, and authentication events.
- Enhanced existing services (customer, user) to log relevant actions using the new audit logger, capturing details such as actor ID, action type, target type, and success status.
- Added middleware to enrich request context with metadata for audit logging, ensuring comprehensive tracking of user interactions.
- Integrated Elasticsearch for persistent storage of audit logs, with fallback to file-only logging if Elasticsearch is unavailable.
- Updated API documentation to include new audit log endpoints for administrative access.
This update significantly improves the system's ability to track and audit user actions, enhancing security and accountability within the application.
- Introduced a mutex to ensure only one TED scraper run executes at a time, preventing concurrent executions during startup and scheduled runs.
- Implemented a mechanism to check if today's TED scrape has already been completed during startup, logging appropriate messages for both completed and new runs.
- Added startup catch-up logic for tender translations and unprocessed notices, ensuring that any missed tasks are executed without blocking the application startup.
This update improves the reliability and efficiency of the TED scraper and worker processes, ensuring that all necessary tasks are completed after a server restart.
- Updated `AISummarizerConfig` to allow for a default `RecommendationCacheTTL` of 0, enabling persistent caching until company updates.
- Refactored `StartAIOnboarding` to include cache invalidation and asynchronous recommendation refresh, improving responsiveness during onboarding.
- Introduced `triggerAIOnboardingAsync` method for background processing of AI onboarding and cache refresh, enhancing user experience.
- Improved logging for AI onboarding and recommendation fetching processes, providing better observability and error tracking.
This update optimizes the AI recommendation caching mechanism and onboarding workflow, ensuring a smoother and more efficient experience for users.
- Updated the `companyService` to include Redis caching for AI recommendations, improving performance and reducing redundant AI calls.
- Introduced asynchronous AI onboarding triggered after company profile updates, enhancing user experience by offloading processing.
- Added configuration for recommendation cache TTL in the `AISummarizerConfig`, allowing for flexible cache management.
- Implemented methods for caching, retrieving, and invalidating AI recommendations in the `companyService`, ensuring efficient data handling.
This update enhances the company's AI recommendation capabilities, providing faster responses and a more efficient onboarding process.
- Added a new endpoint in the `tender` handler for retrieving AI-ranked tender recommendations for a company, improving the functionality of the admin panel.
- Updated the `SearchForm` to include a query parameter for `only_active_deadlines`, allowing for more flexible search options.
- Enhanced API documentation for the new endpoint to provide clear usage instructions and expected parameters.
This update improves the tender management system by providing administrators with better tools for accessing relevant tender information.
- Updated the `UploadDocuments` method to sanitize document file IDs before saving, ensuring only valid references are stored.
- Introduced `DetachDocumentFileID` method in the `companyService` to remove file IDs from all companies referencing a deleted file, improving data integrity.
- Enhanced the `companyRepository` with a new method to handle the removal of document file IDs from the database.
- Updated the `filestore` handler to utilize the new detachment functionality when files are deleted, ensuring consistent state across domain entities.
This update improves the management of document file IDs within the company domain, enhancing data integrity and reference handling.
- Introduced a new `NotificationWorker` to promote due scheduled notifications from pending to sent, improving notification management.
- Added `NotificationInterval` configuration to schedule the notification delivery worker, with a default value for flexibility.
- Implemented `MarkDueScheduledAsSent` method in the notification repository to update the status of notifications based on their delivery time.
- Updated the notification service to process due scheduled notifications during relevant operations, ensuring timely delivery.
This update enhances the notification system by automating the delivery of scheduled notifications, improving user engagement and operational efficiency.
- Removed the DocumentSummarizationWorker and its related scheduling logic from the worker bootstrap.
- Updated the AI summarizer client initialization comment for clarity.
- Added a new error type for cases when tender documents have not been scraped yet, enhancing error handling in the tender service.
- Modified API documentation to reflect changes in AI summary retrieval logic, ensuring accurate descriptions of on-demand summarization behavior.
This update streamlines the AI summarization process by eliminating the document summarization worker, improving overall system efficiency and clarity in error handling.
- Updated the document scraper service to include a new ScrapePortalsProvider interface, allowing for dynamic retrieval of supported scraping portals.
- Modified the ListPendingTenders and GetTenderByNoticeID methods to filter tenders based on document URLs that match the configured portals.
- Introduced new error handling for cases when the scrape portals provider is not configured, returning appropriate service unavailable responses.
- Enhanced API documentation to reflect changes in tender retrieval logic and added error response details for unsupported portal scenarios.
This update improves the document scraping functionality by integrating AI portal support, enhancing the overall reliability and flexibility of the tender management system.
- Updated the `ResolvedEstimatedValueAndCurrency` method to aggregate procurement lot values when the tender-level estimated value is not set, improving accuracy in value retrieval.
- Introduced the `AggregateProcurementLotEstimatedValue` function to sum estimated values from procurement lots and return the first found currency.
- Modified the `ToResponseWithLanguage` method to utilize the new estimated value resolution logic.
- Added unit tests for the new functionality, ensuring correct behavior for various scenarios in the `entity_test.go` and `budget_test.go` files.
This update improves the handling of estimated values in tenders, enhancing the overall reliability of the tender management system.
- Introduced the GetScrapePortals method in the AI pipeline handler to list document scraping portals supported by the Opplens AI service.
- Updated the service layer to include GetScrapePortals, which retrieves the portals from the client and handles errors appropriately.
- Enhanced the routes to register the new endpoint for retrieving scrape portals.
- Added a new error type for invalid date ranges in the document scraper, improving validation and error handling.
This update expands the AI pipeline capabilities, allowing for better management of document scraping portals within the tender management system.
- Introduced the ProcedureDocumentsLister interface to list contract folders with scraped documents, enhancing the accuracy of document-scrape statistics.
- Updated the dashboard repository to accept ProcedureDocumentsLister as a dependency, allowing for improved data retrieval.
- Implemented tests for the new functionality, ensuring proper handling of scraped document folder IDs and error propagation.
This update enhances the dashboard's capability to manage and report on scraped documents, improving overall system efficiency and data integrity.
- Changed the default MinIO bucket name from "opplens-documents" to "opplens" across multiple configuration files.
- Introduced caching for dashboard statistics in the service layer to improve performance and reduce redundant data fetching.
- Implemented a mutex for thread-safe access to cached statistics, ensuring data integrity during concurrent requests.
This update streamlines the configuration for the AI summarizer and optimizes the dashboard service, enhancing overall system efficiency.
- Updated the AI pipeline service to include a new `ScrapedDocumentMetadataSyncer` interface for persisting scraped document metadata onto tender records.
- Modified the `NewService` function to accept the new metadata syncer dependency.
- Implemented synchronization of scraped document metadata in the `ScrapeDocuments` and `Run` methods.
- Enhanced the tender service to enrich search filters based on scraped documents and added a new method for syncing scraped documents from storage.
- Updated the `SearchForm` to include `ContractFolderIDsWithDocuments` for better handling of scraped documents in queries.
This update improves the integration of scraped document handling within the AI pipeline, enhancing data consistency and operational efficiency in the tender management system.
- Added `TranslationEnabled` and `TranslationInterval` fields to the worker configuration to manage automatic translation scheduling.
- Updated the worker initialization to log when the translation worker is disabled.
- Improved error handling in the AI summarizer client by introducing `APIStatusError` for better context on API failures, replacing direct error messages with structured error responses.
This update enhances the configurability of the worker and improves error reporting for AI service interactions, contributing to better maintainability and user experience.
- Added new routes and handlers for AI pipeline operations, including scraping documents, batch summarization, translation, and syncing with the Opplens AI service.
- Introduced request forms for handling tender references and batch operations.
- Enhanced the AI service with methods for triggering batch operations and managing pipeline runs.
- Updated Swagger documentation to reflect the new AI pipeline endpoints and their functionalities.
This update integrates comprehensive AI pipeline capabilities into the tender management system, improving operational efficiency and user experience.
- Added new AI onboarding and recommendation endpoints in the company handler for starting onboarding and retrieving ranked tender recommendations.
- Introduced `StartAIOnboarding` and `GetAIRecommendations` methods in the company service to handle AI interactions.
- Updated the company service constructor to include the AI recommendation client.
- Enhanced the AI summarizer client with methods for onboarding and fetching recommendations.
- Added response structures for onboarding and recommended tenders in the company form.
This update enhances the tender management system by integrating AI capabilities for onboarding and tender recommendations, improving user experience and operational efficiency.
- Removed the QueueConfig structure and related queue management files as they are no longer in use.
- Updated the worker initialization to reflect the removal of queue-related configurations.
- Cleaned up the bootstrap process by eliminating deprecated logging related to the queue system.
This update streamlines the worker's configuration and prepares the codebase for future enhancements without the legacy queue management components.
Paginate un-summarized tenders and mark rows missing notice/folder IDs as handled so the worker cannot spin forever. Return 400/404 for validation and not-found cases on AI summarize/analyze triggers instead of leaking internal errors as 500.
- Added a new endpoint to trigger on-demand agentic analysis for tenders via the AI service.
- Introduced `TriggerAIAnalyze` method in the `TenderHandler` to handle requests and responses for AI analysis.
- Updated the `tenderService` to include `TriggerAIAnalyze` method, which validates input and interacts with the AI summarizer client.
- Enhanced the `AIAnalyzeResponse` and `AIAnalyzeDocument` structures to support the new analysis feature.
- Refactored the `DocumentSummarizationWorker` and `TranslationWorker` to remove deprecated MinIO dependencies, streamlining the AI service interactions.
This update improves the functionality of the tender management system by allowing users to trigger AI analysis on-demand, enhancing the overall user experience and system capabilities.
- Updated `InitAISummarizerClient` to accept `mongoManager` for tracking translation success.
- Introduced new `Statistics` endpoint in the dashboard to fetch scraping and translation statistics.
- Enhanced `TranslationWorker` to utilize the new success counter for tracking successful translations.
- Added necessary data structures and query forms for statistics reporting.
This refactor improves the tracking of AI translation success and provides new insights through the dashboard statistics.
Mazyar