- Introduced AIPipelineAutoWorker to manage the execution of the AI pipeline auto run, including startup catch-up and scheduled tasks.
- Enhanced WorkerConfig to include AIPipelineAutoEnabled and AIPipelineAutoInterval settings for better control over AI pipeline execution.
- Added logging for AI pipeline auto run status, including success and error handling, to improve observability.
- Updated daily job tracker to include AIPipelineAutoJobName for tracking AI pipeline job completions.
This update enhances the system's capability to automate AI pipeline executions, improving efficiency and reliability in processing AI tasks.
- Introduced a mutex to ensure only one TED scraper run executes at a time, preventing concurrent executions during startup and scheduled runs.
- Implemented a mechanism to check if today's TED scrape has already been completed during startup, logging appropriate messages for both completed and new runs.
- Added startup catch-up logic for tender translations and unprocessed notices, ensuring that any missed tasks are executed without blocking the application startup.
This update improves the reliability and efficiency of the TED scraper and worker processes, ensuring that all necessary tasks are completed after a server restart.
Mazyar