Enhance Scraper Dockerfile and Notice Entity
- Added an entrypoint script to the scraper Dockerfile to handle one-time scraping with date range support. - Introduced a new ContentXML field in the Notice entity to store XML content, improving data structure for notices. - Updated the scraper logic to map XML content to the tender entity, enhancing data processing capabilities.
This commit is contained in:
@@ -52,6 +52,7 @@ type Notice struct {
|
||||
SourceFileURL string `bson:"source_file_url" json:"source_file_url"`
|
||||
SourceFileName string `bson:"source_file_name" json:"source_file_name"`
|
||||
ProcessingMetadata ProcessingMetadata `bson:"processing_metadata" json:"processing_metadata"`
|
||||
ContentXML string `bson:"content_xml" json:"content_xml"`
|
||||
|
||||
// Status-specific fields
|
||||
CancellationReason string `bson:"cancellation_reason,omitempty" json:"cancellation_reason,omitempty"`
|
||||
|
||||
Reference in New Issue
Block a user