- Added new indexes for the notices collection in the notice repository to improve query performance.
- Introduced indexes for `processing_metadata.processed` and `main_classification`, optimizing data retrieval and management.
- Modified the validation rule for the Country field in the CMSForm struct to ensure it requires exactly 3 characters, enhancing input validation consistency.
- This change improves the robustness of the form handling by enforcing stricter validation criteria.
- Modified the GetUnProcessedNotices method in the notice repository to include a filter for main_classification set to "72000000".
- Changed the sort order from ascending to descending for the created_at field, improving the retrieval of unprocessed notices.
- Added JSON tags to all fields in the CMS entity and its related sections (heroSection, chartSection, cardSection, contactSection, footerSection) to ensure proper serialization and deserialization in API responses.
- This change improves the API's usability by providing clear JSON representations of the CMS data structure.
- Added an entrypoint script to the scraper Dockerfile to handle one-time scraping with date range support.
- Introduced a new ContentXML field in the Notice entity to store XML content, improving data structure for notices.
- Updated the scraper logic to map XML content to the tender entity, enhancing data processing capabilities.
- Added a new Type field to the CMS entity to differentiate between Company and Organization types, improving data categorization.
- Updated the CMSForm to include validation for the Type field, ensuring proper input handling.
- Modified the ToResponse method to include the Type field in the CMSResponse, enhancing the API response structure.
- Adjusted the copyFormToEntity method in the CMS service to handle the new Type field, maintaining data integrity based on the selected type.
- Enhanced Swagger documentation to reflect the changes in the CMS API, providing clear examples for the new Type field.
- Added new fields to the CMS entity: CompanyName, Country, and Language, enhancing the data structure for better content management.
- Updated the CMSForm to include validation and examples for the new fields, ensuring proper input handling.
- Modified the copyFormToEntity method in the CMS service to map the new fields from the form to the entity, maintaining data integrity.
- Enhanced Swagger documentation to reflect the changes in the CMS API, providing clear examples for the new fields.
- Refactored the NoticeWorker to generate unique tender IDs using the PBL naming convention (SCDYYNNN format) and project names based on client and opportunity details.
- Introduced methods for generating tender IDs and project names, including logic for extracting client and opportunity names, ensuring consistency and clarity in naming.
- Updated the Tender entity to include a new ProjectName field, enhancing the data structure for better project identification.
- Added MongoDB indexing for the new project_name field to optimize query performance.
- Improved error handling and logging during the tender ID generation process, ensuring robustness in the worker's functionality.
Mazyar