Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
277 views
in Technique[技术] by (71.8m points)

python - How to remove when scraping a page?

I have made a function by scraping a page and I run but the output gives . I used strip function to remove but its not working. Why and how to remove ? Here is the link: https://ibb.co/VtVV2fb

import scrapy
from .. items import FetchingItem

class SiteFetching(scrapy.Spider):
    name = 'Site'
    start_urls = ['https://www.rev.com/freelancers']
    transcription_page = 'https://www.rev.com/freelancers/transcription'

    def parse(self, response):
        items = {
            'Heading': response.css('#sign-up::text').extract(),
            'Earn_steps': response.css('.pb2 .lh-copy::text , .mb1::text , .mb3 .lh-copy::text').extract(), 
    }

        yield response.follow(self.transcription_page, self.trans_faqs, meta={'items':items})

    def trans_faqs(self, response):
        items = response.meta['items']
        names = {
            'name1': 'FAQ1',
            'name2': 'FAQ2', 
        }
        finder = {
            'find1': '#whatentailed p::text , #whatentailed .mr3::text',
            'find2': '#requirements p::text , #requirements .mr3::text'
        }
        for name, find in zip(names.values(), finder.values()):
            items[name] = list(map(str.strip,response.css(find).extract()))
        yield items
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Whenever I have a string with tabs or newlines, I found that replacing them with '' works for me.

For example, if you have both and in your string variable, you could do this:

string_variable.replace('
','').replace('	','')

Works well so far.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...