Add TED Scraper Configuration and Cron Scheduler Implementation

- Introduced a new configuration file for the TED scraper, defining database connection settings, logging preferences, and scraping parameters.
- Implemented a CronScheduler to manage scheduled tasks for TED operations, utilizing the robfig/cron library for scheduling.
- Added new entities and structures for handling TED XML data, including eForms and tender-related information, enhancing the scraper's functionality.
- Updated the main application to initialize the TED scraper with the new configuration and set up graceful shutdown handling.
- Removed the deprecated scraping implementation to streamline the codebase and focus on the new architecture.
This commit is contained in:
n.nakhostin
2025-09-30 16:03:53 +03:30
parent 0c50935c42
commit 05c7eae8a2
36 changed files with 3427 additions and 311 deletions
+31
View File
@@ -0,0 +1,31 @@
package eform
// =====================================================
// DOCUMENT STRUCTURE
// =====================================================
// Document represents attached documentation
type Document struct {
ID string `json:"id" xml:"id" validate:"required"`
Type DocumentType `json:"type" xml:"type" validate:"required"`
Title string `json:"title" xml:"title" validate:"required,max=500"`
Description string `json:"description,omitempty" xml:"description,omitempty" validate:"max=2000"`
URL string `json:"url,omitempty" xml:"url,omitempty" validate:"omitempty,url"`
RestrictedAccess bool `json:"restricted_access" xml:"restricted_access"`
Language LanguageCode `json:"language" xml:"language" validate:"required,iso639_1"`
}
// DocumentType represents the type of document
type DocumentType string
const (
DocTypeProcurementDocs DocumentType = "procurement-docs"
DocTypeAdditionalInfo DocumentType = "additional-info"
DocTypeTenderSpecs DocumentType = "tender-specs"
DocTypeTenderForm DocumentType = "tender-form"
DocTypeContractDraft DocumentType = "contract-draft"
DocTypeEvaluationCriteria DocumentType = "evaluation-criteria"
DocTypeBillsQuantities DocumentType = "bills-quantities"
DocTypeIllustrations DocumentType = "illustrations"
DocTypeOther DocumentType = "other"
)