Enhance Scraper with One-Time Date Range Functionality

- Added command line flags for specifying a date range and a one-time scraping mode in the scraper's main function.
- Implemented validation for the date inputs to ensure both dates are provided and correctly formatted.
- Introduced a new RunOneTimeScraping function to handle the one-time scraping logic, including context management and error handling.
- Updated the TED scraper initialization to support both scheduled and one-time scraping modes, improving flexibility in data retrieval.
- Enhanced logging to provide clear feedback on the scraping process and any errors encountered.
This commit is contained in:
n.nakhostin
2025-11-04 16:53:31 +03:30
parent fafccd0d74
commit 08116981f4
3 changed files with 110 additions and 4 deletions
+3 -1
View File
@@ -45,7 +45,9 @@ func GetOJS(baseURL string, year int, date string) (string, error) {
for _, row := range records[1:] {
rowDate := strings.TrimSpace(row[1])
if rowDate == strings.TrimSpace(date) {
ojs = fmt.Sprintf("%v00%v", year, strings.TrimSpace(fmt.Sprint(row[0])))
// Format: YEAR + zero-padded OJS number to make total length 9 digits
// Examples: 202500004, 202500014, 202500196
ojs = fmt.Sprintf("%d%05s", year, strings.TrimSpace(fmt.Sprint(row[0])))
break
}
}