- Removed the entrypoint.sh script from the scraper Dockerfile to streamline the build process.
- Updated the Dockerfile to directly set the ENTRYPOINT to the scraper build executable, simplifying the container startup.
- This change enhances maintainability by reducing the number of files and improving clarity in the Docker configuration.
- Removed command line flag parsing for date range scraping in the scraper's main function.
- Introduced configuration fields for one-time scraping and date range (FromDate, ToDate) in the ScraperConfig struct.
- Updated the main function to utilize the new configuration fields, improving the clarity and maintainability of the scraping logic.
- This change enhances the scraper's functionality by centralizing configuration management and reducing command line complexity.
- Added JSON tags to all fields in the CMS entity and its related sections (heroSection, chartSection, cardSection, contactSection, footerSection) to ensure proper serialization and deserialization in API responses.
- This change improves the API's usability by providing clear JSON representations of the CMS data structure.
- Replaced inline entrypoint script creation in the Dockerfile with a separate entrypoint.sh file for better maintainability and clarity.
- The new entrypoint script handles one-time scraping with date range support, improving the scraper's functionality and usability.
- Added an entrypoint script to the scraper Dockerfile to handle one-time scraping with date range support.
- Introduced a new ContentXML field in the Notice entity to store XML content, improving data structure for notices.
- Updated the scraper logic to map XML content to the tender entity, enhancing data processing capabilities.
- Corrected the middleware usage in the CMS routes by ensuring the AuthMiddleware is applied correctly.
- This change enhances the security of the CMS endpoints by enforcing authentication for all CMS-related operations.
- Updated Swagger documentation to include new fields: CompanyName, Country, and Language, improving clarity and usability for API consumers.
- Added a new Type field with enumerations for Company and Organization, enhancing data categorization in the CMS API.
- Reflected changes in the definitions for CMS entities, ensuring accurate representation of the API structure and expected inputs/outputs.
- Improved examples in the documentation to provide better guidance for developers integrating with the CMS API.
- Added a new Type field to the CMS entity to differentiate between Company and Organization types, improving data categorization.
- Updated the CMSForm to include validation for the Type field, ensuring proper input handling.
- Modified the ToResponse method to include the Type field in the CMSResponse, enhancing the API response structure.
- Adjusted the copyFormToEntity method in the CMS service to handle the new Type field, maintaining data integrity based on the selected type.
- Enhanced Swagger documentation to reflect the changes in the CMS API, providing clear examples for the new Type field.
- Introduced a comprehensive MinIO package for managing object storage, including support for hierarchical document storage and general file management.
- Implemented core components such as Config, ConnectionManager, and Service, following Clean Architecture principles.
- Added functionality for file uploads, downloads, bucket management, and structured logging, enhancing usability and maintainability.
- Included detailed error handling and validation for configuration and operations, ensuring robustness.
- Updated documentation with usage examples and configuration guidelines to facilitate integration and understanding of the package's functionality.
- Added new fields to the CMS entity: CompanyName, Country, and Language, enhancing the data structure for better content management.
- Updated the CMSForm to include validation and examples for the new fields, ensuring proper input handling.
- Modified the copyFormToEntity method in the CMS service to map the new fields from the form to the entity, maintaining data integrity.
- Enhanced Swagger documentation to reflect the changes in the CMS API, providing clear examples for the new fields.
- Refactored the NoticeWorker to generate unique tender IDs using the PBL naming convention (SCDYYNNN format) and project names based on client and opportunity details.
- Introduced methods for generating tender IDs and project names, including logic for extracting client and opportunity names, ensuring consistency and clarity in naming.
- Updated the Tender entity to include a new ProjectName field, enhancing the data structure for better project identification.
- Added MongoDB indexing for the new project_name field to optimize query performance.
- Improved error handling and logging during the tender ID generation process, ensuring robustness in the worker's functionality.
- Added command line flags for specifying a date range and a one-time scraping mode in the scraper's main function.
- Implemented validation for the date inputs to ensure both dates are provided and correctly formatted.
- Introduced a new RunOneTimeScraping function to handle the one-time scraping logic, including context management and error handling.
- Updated the TED scraper initialization to support both scheduled and one-time scraping modes, improving flexibility in data retrieval.
- Enhanced logging to provide clear feedback on the scraping process and any errors encountered.
- Updated the worker initialization process to include the new GLM SDK, enhancing the worker's capabilities for translation tasks.
- Modified the InitWorker function and NewNoticeWorker constructor to accept the GLM service, ensuring a cohesive integration.
- Implemented the GLM service initialization and logging for successful setup, improving maintainability and usability.
- Updated the NoticeWorker to utilize the GLM SDK for translating notice titles and descriptions, enhancing functionality and user experience.
- Introduced a new GLM SDK for interacting with the GLM (Zhipu AI) API, designed for chat completions and text generation tasks.
- Implemented core components including Client, Service, and SDK layers following Clean Architecture principles.
- Added configuration management, error handling, and structured logging for improved usability and maintainability.
- Included comprehensive documentation and usage examples to facilitate integration and understanding of the SDK's functionality.
- Enhanced the API with features such as chat completion, streaming responses, and model management, ensuring robust interaction with the GLM service.
- Added a step in the Dockerfile to copy the documentation files from the build stage to the final image.
- This change ensures that the documentation is included in the Docker image, improving accessibility for users and developers.
- Refactored API paths to replace "companies" with "cms" for better alignment with CMS management functionality.
- Updated descriptions, parameters, and responses for CMS-related endpoints to reflect new functionality, including search, create, retrieve, update, and delete operations.
- Enhanced the Swagger documentation to include detailed definitions for CMS entities, forms, and responses, ensuring comprehensive API specifications.
- Improved consistency in naming conventions and descriptions across the CMS API endpoints.
- Introduced a new CMS management feature, including the ability to create, retrieve, update, search, and delete CMS entries.
- Implemented the CMS entity, repository, service, and handler layers following Clean Architecture principles.
- Added API endpoints for CMS operations in Swagger documentation, ensuring comprehensive API specifications.
- Enhanced error handling and validation for CMS data, improving robustness and user experience.
- Updated the main application to initialize the CMS repository and service, integrating them into the existing system.
- Introduced a new contact management feature, including the ability to create, retrieve, search, update, and delete contact messages.
- Implemented the Contact entity, repository, service, and handler layers following Clean Architecture principles.
- Added API endpoints for contact operations in Swagger documentation, ensuring comprehensive API specifications.
- Enhanced error handling and validation for contact data, improving robustness and user experience.
- Updated the main application to initialize the contact repository and service, integrating them into the existing system.
- Modified the Send method in the NotificationHandler to dispatch notifications asynchronously by using a goroutine, improving responsiveness and allowing the handler to return immediately after initiating the send process.
- This change enhances the user experience by not blocking the request while the notification is being sent, ensuring smoother operation of the API.
- Enhanced the GetOJS function to include error handling for reading the response body and parsing CSV records, ensuring that errors are properly returned instead of being ignored.
- Added a check to ensure that the CSV records contain data, returning a meaningful error if the calendar is empty, which improves robustness and clarity in error reporting.
- Removed the Ollama SDK from the worker initialization process, simplifying the worker's dependencies and enhancing maintainability.
- Updated the InitWorker function and NewNoticeWorker constructor to reflect the removal of the Ollama SDK, ensuring a cleaner and more focused implementation.
- Commented out the Ollama model listing logic for potential future use, maintaining clarity in the main function while reducing unnecessary complexity.
- Updated the Send method in the notification service to handle optional image and link fields more gracefully by using local variables.
- Improved logging to ensure that the correct values are sent in the notification context, enhancing clarity and maintainability of the notification process.
- This change ensures that nil values for image and link do not cause dereferencing issues, improving overall robustness of the notification sending logic.
- Introduced a new build step in the Drone CI configuration for the worker service, enabling automated Docker image creation.
- Added a Dockerfile for the worker service, defining the build process and dependencies, ensuring a streamlined deployment for background tasks.
- Enhanced the overall CI/CD pipeline to support the new worker service, improving the project's build and deployment capabilities.
- Updated the ToTender method to accept a tender instance, allowing for direct population of tender fields from the notice entity.
- Simplified the tender creation and update process by checking if the tender ID is zero, streamlining the logic for handling tender entities.
- Removed the commented-out code related to AI translation, improving code clarity and maintainability.
- Enhanced error logging for tender creation and update failures, ensuring better visibility into potential issues during processing.
- Added 'image' and 'link' fields to the NotificationRequest and NotificationResponse structures, improving the flexibility of notification content.
- Updated the 'created_at', 'updated_at', and 'schedule_at' fields to use integer types for Unix timestamps, ensuring consistency in time handling.
- Expanded the tender status enumeration to include 'closed', 'modified', 'suspended', and 'published', enhancing the representation of tender states.
- Reflected these changes in the Swagger documentation, ensuring accurate API specifications for clients.
- Modified the SearchForm struct to utilize query parameters for feedback search criteria, enhancing API consistency and usability.
- Updated validation tags accordingly to reflect the new query parameter structure, ensuring proper validation during request handling.
- Introduced AlertMail configuration in both scraper and worker bootstrap files, allowing for customizable email notifications.
- Updated the TED scraper to utilize the AlertMail configuration for sending completion notifications, improving flexibility in notification management.
- Enhanced error logging in the worker's main function to capture issues when listing Ollama models, ensuring better visibility into potential failures.
- Refactored notification sending logic to check for a valid AlertMail before dispatching emails, ensuring notifications are only sent when configured.
- Improved overall structure and readability of the bootstrap configuration files, aligning with best practices for maintainability.
- Introduced a new worker service with a dedicated main entry point for handling background tasks, including MongoDB connection management and notification service initialization.
- Added a bootstrap package to manage application configuration, logging, and service initialization for the worker.
- Implemented a NoticeWorker to process unprocessed notices and create corresponding tender entities, enhancing the integration of notice management within the tender system.
- Refactored the tender service to remove the Ollama SDK dependency, streamlining the service initialization in the main application.
- Enhanced the notice repository with a method to retrieve unprocessed notices, improving data handling capabilities.
- Moved the initialization of the notification service to follow the MongoDB connection setup for better organization.
- Added comments to clarify the initialization sequence of services, including the TED scraper.
- Removed commented-out code related to stopping the scheduler to clean up the main function.
- Enhanced the overall readability and structure of the main application entry point.
- Replaced the tender repository with a new notice repository, encapsulating notice-related data access methods.
- Introduced the Notice entity to represent tender/contract notices, including relevant fields and methods for managing notice data.
- Updated the TED scraper to utilize the new notice repository for creating and managing notices, enhancing the integration with the tender management system.
- Implemented Ollama SDK initialization in the web bootstrap process, allowing for improved AI interactions.
- Enhanced the tender service to include Ollama SDK for additional functionality, ensuring a more robust service layer.
- Introduced the Ollama SDK, providing a comprehensive interface for interacting with Ollama AI models, including text generation, chat conversations, embeddings, and model management.
- Implemented core components such as Client, Config, and various request/response entities to facilitate API interactions.
- Added structured error handling with specific error types for improved clarity and debugging.
- Included a README and usage guide to assist developers in integrating the SDK into their applications, ensuring a smooth onboarding experience.
- Established a fluent API for chat interactions and streaming responses, enhancing usability and flexibility for developers.
- Updated the tender repository to create a unique index on tender_id, enhancing database query performance.
- Introduced new TED mapping functions to convert parsed TED documents into tender entities, improving integration with the tender management system.
- Added error handling for contract notice processing in the TED scraper, ensuring robust logging and error management.
- Removed deprecated eform structures to streamline the codebase and focus on essential components.
- Added logic to send an asynchronous email notification when a customer's username is updated, improving user communication regarding account changes.
- Introduced a flag to track if the username has changed, ensuring notifications are only sent when necessary.
- Enhanced error handling and logging for successful customer updates, maintaining clarity in operational logs.
- Updated GetByIDs methods in customer and user repositories to convert string IDs to bson.ObjectID, ensuring proper handling of MongoDB object IDs.
- Enhanced error handling for ID conversion to provide clearer feedback when invalid IDs are provided.
- Modified notification service calls in customer and user services to use context.Background() for asynchronous email sending, improving concurrency management.
- Ensured consistent handling of company IDs in the customer repository for retrieving customers by companies.
- Updated the Logout method in customer service to retrieve the customer by ID before invalidating the access token, enhancing error logging for failed retrievals.
- Added error handling for the access token invalidation process, ensuring a clear error message is returned if it fails.
- Implemented logic to clear the device token upon logout, updating the customer record accordingly and logging any errors during the update process.
- Improved logging for successful and failed logout operations to enhance traceability.
- Added 'feature/ted' branch to the Drone CI configuration, allowing for automated builds and tests on this new feature branch.
- This change enhances the CI/CD pipeline by ensuring that developments on the feature branch are properly integrated and tested.
- Updated the GetOJS function to accept a base URL parameter, improving flexibility in constructing the calendar URL.
- Removed the deprecated eform.go file, streamlining the codebase by eliminating unused components.
- Introduced new eform structures for BusinessRegistrationInformationNotice, ContractNotice, ContractAwardNotice, and PriorInformationNotice, enhancing the handling of TED XML data.
- Implemented mapping functions to convert eform notices to the Tender entity, improving integration with the tender management system.
- Added a service layer to encapsulate business logic related to eform processing, adhering to Clean Architecture principles.
- Introduced a new configuration file for the TED scraper, defining database connection settings, logging preferences, and scraping parameters.
- Implemented a CronScheduler to manage scheduled tasks for TED operations, utilizing the robfig/cron library for scheduling.
- Added new entities and structures for handling TED XML data, including eForms and tender-related information, enhancing the scraper's functionality.
- Updated the main application to initialize the TED scraper with the new configuration and set up graceful shutdown handling.
- Removed the deprecated scraping implementation to streamline the codebase and focus on the new architecture.
- Deleted the TED XML parser and scraper implementation files, including parser.go, scraper.go, and ted.go, to streamline the codebase and eliminate unused components.
- Commented out the initialization code in main.go to prevent execution while maintaining the structure for future reference.
- Removed the README.md for the TED package, which contained documentation for the now-deleted parser and scraper functionalities.
- Updated the Tender entity to remove redundant comments on Unix timestamp fields, enhancing code readability.
- Renamed the AwardedEntity type to Awarded for improved clarity and consistency in naming conventions.
- Removed the transformPriorInformationNoticeToContractNotice method from TEDParser, streamlining the parser's functionality and reducing legacy code.
- Added new fields to the Tender entity for handling cancellation, award, and suspension details, improving the entity's capability to manage tender statuses.
- Implemented a DetectNoticeStatus method in the TEDParser to identify the status of TED notices from XML data, enhancing the parser's functionality.
- Updated the ParseXML method to include notice status detection, ensuring accurate status representation in parsed documents.
- Introduced upsert logic in the TED scraper to handle tender records more effectively, allowing for updates or creation based on existing ContractIDs.
- Enhanced logging and error handling throughout the scraping process to improve traceability and robustness.
- Added a comprehensive usage example document to illustrate the new features and usage patterns for the TED XML parser and scraper.
- Modified the ToResponse method in the Tender entity to create an OrganizationResponse instance for BuyerOrganization, ensuring safe access to its Name field.
- This change improves the robustness of the response structure by preventing potential nil dereference errors when BuyerOrganization is not set.
- Updated GetByID methods in user and customer services to return specific "not found" errors for better clarity in error responses.
- Modified NotificationResponse structure to include a new Recipient field, allowing for more flexible recipient handling.
- Changed CreatedAt and UpdatedAt fields in NotificationResponse to use Unix timestamps (int64) for consistency across the application.
- Improved notification retrieval logic to handle potential nil values for user and customer lookups, ensuring robust data handling.
Mazyar