Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
544 views
in Technique[技术] by (71.8m points)

r - Rvest not recognizing css selector

I'm trying to scrape this website:

http://www.racingpost.com/greyhounds/result_home.sd#resultDay=2015-12-26&meetingId=18&isFullMeeting=true

through the rvest package in R.

Unfortunately it seems that rvest doesn't recognize the nodes through the CSS selector.

For example if I try to extract the information in the header of every table (Grade, Prize, Distance), whose CSS selector is ".black" and I run this code:

URL <- read_html("http://www.racingpost.com/greyhounds/result_home.sd#resultDay=2015-12-26&meetingId=18&isFullMeeting=true")
nodes<-html_nodes(URL, ".black") 

nodes comes out to be a null list, so it's not scraping anything.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

It's making an XHR request to generate the HTML. Try this (which should also make it easier to automate the data capture):

library(httr)
library(xml2)
library(rvest)

res <- GET("http://www.racingpost.com/greyhounds/result_by_meeting_full.sd",
           query=list(r_date="2015-12-26",
                      meeting_id=18))

doc <- read_html(content(res, as="text"))

html_nodes(doc, ".black")
## {xml_nodeset (56)}
##  [1] <span class="black">A9</span>
##  [2] <span class="black">£61</span>
##  [3] <span class="black">470m</span>
##  [4] <span class="black">-30</span>
##  [5] <span class="black">H2</span>
##  [6] <span class="black">£105</span>
##  [7] <span class="black">470m</span>
##  [8] <span class="black">-30</span>
##  [9] <span class="black">A7</span>
## [10] <span class="black">£61</span>
## [11] <span class="black">470m</span>
## [12] <span class="black">-30</span>
## [13] <span class="black">A5</span>
## [14] <span class="black">£66</span>
## [15] <span class="black">470m</span>
## [16] <span class="black">-30</span>
## [17] <span class="black">A8</span>
## [18] <span class="black">£61</span>
## [19] <span class="black">470m</span>
## [20] <span class="black">-20</span>
## ...

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...