文章目录
载入图片
载入各种包
from PIL import Image
from PIL import ImageFilter
import numpy as np
import numpy as np
import pandas as pd
import re
import matplotlib.pyplot as mpl
from urllib.parse import urlparse
import urllib.robotparser
def canFetch(url):parsed_uri = urlparse(url)domain = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)rp = urllib.robotparser.RobotFileParser()rp.set_url(domain + "/robots.txt")try:rp.read()canFetchBool = rp.can_fetch("*", url)
本文来自互联网用户投稿,文章观点仅代表作者本人,不代表本站立场,不承担相关法律责任。如若转载,请注明出处。 如若内容造成侵权/违法违规/事实不符,请点击【内容举报】进行投诉反馈!