scrapy 在不同的抓取级别的Request之间传递参数的办法,scrapyrequest,class MySpid


class MySpider(BaseSpider):    name = 'myspider'    start_urls = (        'http://example.com/page1',        'http://example.com/page2',        )    def parse(self, response):        # collect `item_urls`        for item_url in item_urls:            yield Request(url=item_url, callback=self.parse_item)    def parse_item(self, response):        item = MyItem()        # populate `item` fields        yield Request(url=item_details_url, meta={'item': item},            callback=self.parse_details)    def parse_details(self, response):        item = response.meta['item']        # populate more `item` fields        return item

评论关闭