tm_back/pkg/xmlparser/README.md

Generic XML Parser Package

This package is a generic XML parser implemented using Go Generics that can parse any XML structure into Go structs.

Features

• Generic: Support for any struct type that implements the Parseable interface
• Flexible: Configurable parsing options for different use cases
• Validation: Automatic validation of XML and parsed structures
• Metadata Extraction: Extract XML metadata and parsing information
• Advanced Error Handling: Detailed error and warning reporting
• TED XML Support: Pre-built models for TED XML formats

Installation

go mod tidy

Quick Start

Parse XML from URL

package main

import (
    "fmt"
    "net/http"
    "tm/pkg/xmlparser"
    "tm/pkg/xmlparser/ted"
)

func main() {
    // Create parser for Contract Notice
    parser := xmlparser.NewParser[ted.ContractNotice](nil)
    
    // Fetch XML from TED website
    resp, err := http.Get("https://ted.europa.eu/en/notice/447983-2025/xml")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    
    // Parse XML
    notice, err := parser.Parse(resp.Body)
    if err != nil {
        panic(err)
    }
    
    fmt.Printf("Notice ID: %s\n", notice.GetID())
    fmt.Printf("Notice Type: %s\n", notice.GetNoticeType())
}

Parse XML from String

xmlData := `<?xml version="1.0" encoding="UTF-8"?>
<ContractNotice xmlns="urn:oasis:names:specification:ubl:schema:xsd:ContractNotice-2">
    <ID>sample-notice-123</ID>
    <ContractFolderID>sample-folder-456</ContractFolderID>
    <IssueDate>2025-01-09+02:00</IssueDate>
    <NoticeTypeCode>cn-standard</NoticeTypeCode>
    <NoticeLanguageCode>ENG</NoticeLanguageCode>
</ContractNotice>`

parser := xmlparser.NewParser[ted.ContractNotice](nil)
notice, err := parser.ParseString(xmlData)
if err != nil {
    panic(err)
}

fmt.Printf("Parsed Notice ID: %s\n", notice.GetID())

Key Interfaces

Parseable Interface

Every struct to be parsed must implement this interface:

type Parseable interface {
    Validate() error  // Validate the parsed structure
    GetID() string    // Return unique identifier
}

Parser Interface

type Parser[T Parseable] interface {
    Parse(reader io.Reader) (*T, error)
    ParseString(xmlData string) (*T, error)
    ParseBytes(xmlData []byte) (*T, error)
    Validate(reader io.Reader) error
}

Parser Configuration

options := &xmlparser.ParserOptions{
    StrictValidation:      true,        // Strict XML validation
    AllowEmptyElements:    true,        // Allow empty XML elements
    PreserveWhitespace:    false,       // Preserve whitespace
    IgnoreUnknownElements: false,       // Ignore unknown XML elements
    MaxDepth:              100,         // Maximum nesting depth
    MaxSize:               10 * 1024 * 1024, // Maximum XML size (10MB)
}

parser := xmlparser.NewParser[ted.ContractNotice](options)

Parse with Detailed Results

parser := xmlparser.NewParser[ted.ContractNotice](nil).(*xmlparser.GenericParser[ted.ContractNotice])

result, err := parser.ParseWithResult(strings.NewReader(xmlData))
if err != nil {
    fmt.Printf("Parse error: %v\n", err)
    return
}

// Display parsed information
if result.Data != nil {
    fmt.Printf("Notice ID: %s\n", result.Data.GetID())
}

// Display metadata
if result.Metadata != nil {
    fmt.Printf("XML Version: %s\n", result.Metadata.Version)
    fmt.Printf("Encoding: %s\n", result.Metadata.Encoding)
    fmt.Printf("Namespace: %s\n", result.Metadata.Namespace)
}

// Display errors and warnings
for _, err := range result.Errors {
    fmt.Printf("Error: %s\n", err)
}

for _, warning := range result.Warnings {
    fmt.Printf("Warning: %s\n", warning)
}

TED XML Models

The package includes ready-to-use models for TED XML formats:

Contract Notice

type ContractNotice struct {
    ID                 string
    ContractFolderID   string
    IssueDate          string
    NoticeTypeCode     string
    NoticeLanguageCode string
    ProcurementProject *ProcurementProject
    // ... other fields
}

Creating Custom Models

To create custom models, implement the Parseable interface:

type CustomModel struct {
    XMLName xml.Name `xml:"CustomElement"`
    ID      string   `xml:"id,attr"`
    Name    string   `xml:"name"`
    Value   string   `xml:"value"`
}

func (c CustomModel) Validate() error {
    if c.ID == "" {
        return fmt.Errorf("ID is required")
    }
    if c.Name == "" {
        return fmt.Errorf("Name is required")
    }
    return nil
}

func (c CustomModel) GetID() string {
    return c.ID
}

// Usage
parser := xmlparser.NewParser[CustomModel](nil)
result, err := parser.ParseString(`<CustomElement id="123"><name>Test</name><value>Sample</value></CustomElement>`)

Factory Function for Creating Parsers

func CreateTEDParser[T xmlparser.Parseable](options *xmlparser.ParserOptions) xmlparser.Parser[T] {
    return xmlparser.NewParser[T](options)
}

// Usage
contractParser := CreateTEDParser[ted.ContractNotice](nil)

Practical Examples

The example/ folder contains complete usage examples:

• Parse from URL
• Parse from String
• Using different parser options
• Parse with detailed results

To run the examples:

cd pkg/xmlparser/example
go run main.go

File Structure

pkg/xmlparser/
├── interfaces.go          # Interface definitions
├── parser.go              # Generic parser implementation
├── utils.go               # Utility functions
├── parser_test.go         # Comprehensive tests
├── ted/
│   └── models.go          # TED XML models
├── example/
│   └── main.go            # Usage examples
└── README.md              # This file

TED XML Support

This package is specifically designed for parsing TED (Tenders Electronic Daily) XML:

• Contract Notice: Tender announcements
• Prior Information Notice: Preliminary notices
• Contract Award Notice: Award announcements

Supported Formats

• eForms XML (new TED version)
• TED Schema XML (legacy version)
• UBL 2.3 with eForms Extensions

Error Handling

The parser supports various types of errors:

result, err := parser.ParseString(invalidXML)
if err != nil {
    // General parsing error
    fmt.Printf("Parse failed: %v\n", err)
}

// Detailed errors in result
for _, parseError := range result.Errors {
    fmt.Printf("Parse error: %s\n", parseError)
}

// Warnings
for _, warning := range result.Warnings {
    fmt.Printf("Warning: %s\n", warning)
}

Performance Optimization

• XML size limits (default: 10MB)
• Nesting depth limits (default: 100 levels)
• Stream parsing for large files
• Memory-efficient parsing

Testing

Run comprehensive tests:

# Run all tests
go test ./...

# Run with verbose output
go test -v ./...

# Run benchmarks
go test -bench=. ./...

Utility Functions

The package includes helpful utility functions:

• SanitizeXMLString() - Clean invalid XML characters
• ValidateXMLName() - Validate XML element names
• IsValidNamespace() - Validate namespace URIs
• NormalizeWhitespace() - Normalize text content
• TruncateString() - Truncate strings for display