python抓取网页及网页上所有连接的演示代码,python抓取,import urlli
文章由Byrx.net分享于2019-03-23 08:03:19
python抓取网页及网页上所有连接的演示代码,python抓取,import urlli
import urllib, htmllib, formatter, re, sysurl = sys.argv[1]website = urllib.urlopen("http://"+url)data = website.read()website.close()format = formatter.AbstractFormatter(formatter.NullWriter())ptext = htmllib.HTMLParser(format)ptext.feed(data)links = []links = ptext.anchorlistfor link in links: if re.search('http', link) != None: print(link) website = urllib.urlopen(link) data = website.read() website.close() ptext = htmllib.HTMLParser(format) ptext.feed(data) morelinks = ptext.anchorlist for alink in morelinks: if re.search('http', alink) != None: links.append(alink)
评论关闭