- Introduced a new configuration file for the TED scraper, defining database connection settings, logging preferences, and scraping parameters.
- Implemented a CronScheduler to manage scheduled tasks for TED operations, utilizing the robfig/cron library for scheduling.
- Added new entities and structures for handling TED XML data, including eForms and tender-related information, enhancing the scraper's functionality.
- Updated the main application to initialize the TED scraper with the new configuration and set up graceful shutdown handling.
- Removed the deprecated scraping implementation to streamline the codebase and focus on the new architecture.
- Deleted the TED XML parser and scraper implementation files, including parser.go, scraper.go, and ted.go, to streamline the codebase and eliminate unused components.
- Commented out the initialization code in main.go to prevent execution while maintaining the structure for future reference.
- Removed the README.md for the TED package, which contained documentation for the now-deleted parser and scraper functionalities.
- Updated the Tender entity to remove redundant comments on Unix timestamp fields, enhancing code readability.
- Renamed the AwardedEntity type to Awarded for improved clarity and consistency in naming conventions.
- Removed the transformPriorInformationNoticeToContractNotice method from TEDParser, streamlining the parser's functionality and reducing legacy code.
- Added new fields to the Tender entity for handling cancellation, award, and suspension details, improving the entity's capability to manage tender statuses.
- Implemented a DetectNoticeStatus method in the TEDParser to identify the status of TED notices from XML data, enhancing the parser's functionality.
- Updated the ParseXML method to include notice status detection, ensuring accurate status representation in parsed documents.
- Introduced upsert logic in the TED scraper to handle tender records more effectively, allowing for updates or creation based on existing ContractIDs.
- Enhanced logging and error handling throughout the scraping process to improve traceability and robustness.
- Added a comprehensive usage example document to illustrate the new features and usage patterns for the TED XML parser and scraper.
- Introduced a new TED scraper in `cmd/scraper` to handle downloading and parsing TED XML files.
- Added configuration management for the scraper, including a new `Config` struct and YAML configuration file.
- Created necessary handler, service, and repository layers for tender management, adhering to Clean Architecture principles.
- Implemented comprehensive logging and error handling throughout the scraper functionality.
- Updated API routes to include tender management operations, enhancing the overall system capabilities.
n.nakhostin