U TDi%@sdZddlZddlmZmZmZmZddlmZddlm Z m Z m Z m Z Gdddej ZGdd d e ZGd d d ej ZGd d d e ZdS)zScrapy items for product data.N)OptionalListDictAny)datetime) BaseModelHttpUrlFieldfield_validatorc@seZdZdZeZeZeZeZ eZ eZ eZ eZ eZeZeZeZeZeZeZeZeZdS) ProductItemz/Scrapy item for product data (raw dict format).N)__name__ __module__ __qualname____doc__scrapyr namepriceoriginal_pricecurrency descriptionimages product_urlskucategory attributes availability stock_statusrating review_countreviews source_url crawled_atr"r"scrapy_project/items.pyr s$r c@s6eZdZUdZeddddZeed<eddd d Ze e ed <eddd d Z e e ed <edddZ eed<edddZ e eed<eeddZeeed<edddZe eed<edddZe eed<edddZe eed<edddZe eeefed<edd dZe eed!<edd"dZe eed#<eddd$d%d&Ze e ed'<eddd(d Ze eed)<eed*dZeeed+<edd,dZe eed-<eejd.dZeed/<e de!eeeed0d1d2Z"e de!eed0d3d4Z#e!e$dd5d6d7Z%e&d8d9d:Z'e&d8d;d<Z(ee&d8d=d>Z)dS)?Productz*Pydantic model for validated product data..z Product name)Z min_lengthrrNrz Product price)gerrzOriginal price if on salerZVNDz Currency code)defaultrrzProduct description)rrzList of image URLs)default_factoryrrzProduct page URLrzProduct SKU/IDrzProduct categoryrz&Product attributes (size, color, etc.)rzAvailability statusrz%Stock status (in_stock, out_of_stock)rzProduct rating (0-5))r&lerrzNumber of reviewsrzxList of product reviews. Each review is a dict with: author, rating, date, content, images (optional list of image URLs)rzSource website URLr zCrawl timestampr!)vreturncCsdd|DS)zEnsure image URLs are valid.cSs g|]}|r|r|qSr"strip).0Zimgr"r"r# @sz+Product.validate_images..r"clsr+r"r"r#validate_images<szProduct.validate_imagescCs|r |SdS)zClean product name.r-r1r"r"r# validate_nameBszProduct.validate_name)itemr,cCsVt|}t|dtr4|dr*|dgng|d<nt|dtsLg|d<|f|S)z Create Product from Scrapy item.r)dict isinstancegetstrlist)r2r6datar"r"r#from_scrapy_itemHs zProduct.from_scrapy_item)r,c'Csd|jp d|jddd|jpddd|jp,ddddd|jd krBdnddddddddddd|jrj|jrj|jnd|jptd|jp|ddd|jrd |jnddddddddddd &S) z"Convert to WooCommerce CSV format.Zsimpler4r%rZvisibleNZtaxableZin_stock|)&TypeZSKUName Publishedz Is featured?zVisibility in catalogzShort descriptionZ DescriptionzDate sale price startszDate sale price endsz Tax statusz Tax classz In stock?ZStockzLow stock amountzBackorders allowed?zSold individually?z Weight (kg)z Length (cm)z Width (cm)z Height (cm)zAllow customer reviews?z Purchase notez Sale pricez Regular priceZ CategoriesTagszShipping classZImageszDownload limitzDownload expiry daysZParentzGrouped productsZUpsellsz Cross-sellsz External URLz Button textZPosition) rrrrrrrrjoinselfr"r"r#to_woocommerce_csv_rowTsNzProduct.to_woocommerce_csv_rowcCs|r|dSiS)z@Convert to Shopify CSV format (single row, backward compatible).r)to_shopify_csv_rowsrEr"r"r#to_shopify_csv_rowszProduct.to_shopify_csv_rowc0Cs2|jp|jdddd}||jdd|jp6dd|jp@ddddddddd|jpXddd dd d |jpjd|jprdddd|jd |j|jr|jdd nddddddddd dddddddddddd/}g}|jr t |jddD]*\}}| }||d<||d<| |qn"| }d|d<d|d<| ||S)a Convert to Shopify CSV format with multiple rows (one per image). Returns: List of dicts, each representing a row in Shopify CSV format. Each row has the same product data, but different Image Src and Image Position.  -Nr4ZTRUETitlez Default TitleZshopifyZdenyZmanualZFALSEnewZkgZactive)/ZHandlerMz Body (HTML)ZVendorr@rCrBz Option1 Namez Option1 Valuez Option2 Namez Option2 Valuez Option3 Namez Option3 Valuez Variant SKUz Variant GramszVariant Inventory TrackerzVariant Inventory QtyzVariant Inventory PolicyzVariant Fulfillment Servicez Variant PricezVariant Compare At PricezVariant Requires ShippingzVariant TaxablezVariant BarcodezImage Alt Textz Gift Cardz SEO TitlezSEO Descriptionz)Google Shopping / Google Product CategoryzGoogle Shopping / GenderzGoogle Shopping / Age GroupzGoogle Shopping / MPNz"Google Shopping / AdWords Groupingz Google Shopping / AdWords LabelszGoogle Shopping / Conditionz Google Shopping / Custom Productz Google Shopping / Custom Label 0z Google Shopping / Custom Label 1z Google Shopping / Custom Label 2z Google Shopping / Custom Label 3z Google Shopping / Custom Label 4z Variant ImagezVariant Weight UnitzVariant Tax Codez Cost per itemzIncluded / VietnamZStatusr%)startz Image SrczImage Position) rrlowerreplacerrrrr enumeratecopyappend)rFZhandleZbase_rowZrowsZpositionZ image_urlrowr"r"r#rHsz  3 zProduct.to_shopify_csv_rows)*r r rrr rr:__annotations__rrfloatrrrr;rrrrrrrrrrrintrr rnowr!r classmethodr3r5r r=r7rGrIrHr"r"r"r#r$%s:   +r$c@sHeZdZdZeZeZeZeZ eZ eZ eZ dS)ShopItemzIScrapy item for shop/site information (aggregated across multiple pages).N) r r rrrr Z item_typebase_urldomainZpage_urlZ page_typeZ extractedr!r"r"r"r#r\sr\c@seZdZUdZdZeeed<dZeeed<dZ eeed<dZ eeed<dZ eeed<e e dZeeed <e e dZeeed <e e dZeeed <e e dZeeed <e e dZeeed <e edZeeefed<e ejdZeed<dS)ShopzKPydantic model for aggregated shop/site info (best-effort, may be partial).Nr]r^rrlogo)r(emailsphones social_links addressespagesrawr!)r r rrr]rr:rWr^rrr`r r;rarrbrcrdrerr7rfrrrZr!r"r"r"r#r_s r_)rrtypingrrrrrZpydanticrrr r ZItemr r$r\r_r"r"r"r#s .