使用美汤进行网页抓取（体育数据）

Question

当我尝试加载此代码时，出现两个错误。1：第一个是我无法正确抓取name_text的数据。2：我收到 team = name_text.div.text 的缩进错误。我知道这可能很容易解决，但我尝试了不同的缩进，但似乎没有任何效果。在网站上，我想抓取球队名称和赔率。<div class="size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r" data-automation-id="participant-one">Orlando Magic</div><div class="priceText_f71sibe"><span class="size14_f7opyze medium_f1wf24vo priceTextSize_frw9zm9" data-automation-id="price-text">5.85</span></div>上面的html是从网站上复制的。from bs4 import BeautifulSoupfrom urllib.request import urlopen as uReqmy_url = 'https://www.sportsbet.com.au/betting/basketball-us'uClient = uReq(my_url)page_html = uClient.read()uClient.close()soup = BeautifulSoup(page_html, "html.parser")price_text = soup.findAll("div",{"class":"priceText_f71sibe"})name_text = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r"})filename = "odds.csv"f = open(filename,"w")headers = "Team, odds_team\n"print(name_text)f.write(headers)for price_text in price_texts:team = name_text.div.textodds = price_text.span.textprint(odds)print(team + odds)f.write(team + "," + odds + "\n")f.close()任何帮助都会很棒。干杯。

Answer 1

您的for loop缩进不正确。正确的缩进是：

for price_text in price_texts:

    team = name_text.div.text

    odds = price_text.span.text

    team = name_text.div.text

    odds = price_text.span.text

    print(odds)

    print(team + odds)

    f.write(team + "," + odds + "\n")

f.close()

队前有 4 个空格和赔率。请阅读Python ForLoop 文档。

此外，没有price_texts变量。当你做findAll时你需要分配它，你忘记了一个'S'：

price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})

最后一件事，考虑使用with而不是open()和.close()写入您的文件。

Answer 2

你能试试这个吗？

from bs4 import BeautifulSoup

from urllib.request import urlopen as uReq

my_url = 'https://www.sportsbet.com.au/betting/basketball-us'

uClient = uReq(my_url)

page_html = uClient.read()

uClient.close()

soup = BeautifulSoup(page_html, "html.parser")

price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})

name_texts = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24voparticipantText_fivg86r"})

filename = "odds.csv"

f = open(filename,"w")

headers = "Team, odds_team\n"

print(name_text)

f.write(headers)

odds =''

team=''

for price_text in price_texts:

    odds = price_text.text

for name_text in name_texts:

    team = name_text.text

print(odds)

print(team + odds)

f.write(team + "," + odds + "\n")

f.close()

Answer 3

我在想你可以做的只是迭代并将它们存储到列表中，然后写入文件。不幸的是，我无法在工作中访问该站点，因此无法测试代码，但我相信这应该会提供您正在寻找的输出：

from bs4 import BeautifulSoup

from urllib.request import urlopen as uReq

import csv

from itertools import zip_longest

my_url = 'https://www.sportsbet.com.au/betting/basketball-us'

uClient = uReq(my_url)

page_html = uClient.read()

uClient.close()

soup = BeautifulSoup(page_html, "html.parser")

price_text = soup.findAll("span",{"data-automation-id":"price-text"})

name_text = soup.findAll("div",{"data-automation-id":"participant-one"})

team_list = [ name.text.strip() for name in name_text ]

odds_list = [ price.text.strip() for price in price_text ]

d = [team_list, odds_list]

export_data = zip_longest(*d, fillvalue = '')

with open('odds.csv', 'w', encoding="ISO-8859-1", newline='') as myfile:

      wr = csv.writer(myfile)

      wr.writerow(("Team", "odds_team"))

      wr.writerows(export_data)

myfile.close()