Refactor document scraper to utilize active publication windows for tender retrieval
continuous-integration/drone/push Build is passing
continuous-integration/drone/push Build is passing
- Updated the ListPendingTenders method to filter tenders based on their active publication submission window instead of just deadlines, enhancing the accuracy of tender listings. - Introduced a new HasActivePublicationWindow method in the Tender entity to determine if the publication-based submission window is still open. - Modified the GetTenderByNoticeID method to reflect the new logic for filtering tenders based on their publication status. - Enhanced the documentation for relevant API endpoints to clarify the changes in tender retrieval criteria. This update improves the precision of tender scraping by ensuring only relevant tenders with active submission windows are processed, contributing to better data management and scraping efficiency.
This commit is contained in:
@@ -24,7 +24,7 @@ func NewHandler(service Service, logger logger.Logger) *Handler {
|
||||
|
||||
// ListPendingTenders retrieves tenders that need document scraping
|
||||
// @Summary List pending tenders for document scraping
|
||||
// @Description Retrieve tenders that have not yet been scraped for documents and whose document URL matches a portal supported by the AI scraping service. Each item includes contract_folder_id, notice_publication_id, document_url, title, and description. Optionally filter by created_at range (Unix timestamps in seconds). By default only tenders with active deadlines are returned; set include_expired=true to include expired tenders.
|
||||
// @Description Retrieve tenders that have not yet been scraped for documents and whose document URL matches a portal supported by the AI scraping service. Each item includes contract_folder_id, notice_publication_id, document_url, title, and description. Optionally filter by created_at range (Unix timestamps in seconds). By default only tenders with an active publication submission window are returned (submission_deadline, or 48 working hours after publication_date); set include_expired=true to include expired tenders.
|
||||
// @Tags Document-Scraper
|
||||
// @Produce json
|
||||
// @Param limit query int false "Number of items per page (default: 20, max: 100)"
|
||||
@@ -61,7 +61,7 @@ func (h *Handler) ListPendingTenders(c echo.Context) error {
|
||||
|
||||
// GetTenderByNoticeID retrieves a specific tender by its notice publication ID
|
||||
// @Summary Get tender by notice ID for document scraping
|
||||
// @Description Retrieve a specific unexpired tender whose document URL matches a supported scrape portal.
|
||||
// @Description Retrieve a specific tender whose publication submission window is still open and whose document URL matches a supported scrape portal.
|
||||
// @Tags Document-Scraper
|
||||
// @Produce json
|
||||
// @Param notice_id path string true "Notice Publication ID"
|
||||
|
||||
Reference in New Issue
Block a user