Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Ask a Question
Welcome To Ask or Share your Answers For Others

Categories

Recent questions tagged robots.txt

0 votes
947 views
1 answer
robots.txt on domain instead of subdomain?
    I am researching this. I need suggestions. I have a domain that I own. It is pointed to a ... stackoverflow.com/questions/66067899/robots-txt-on-domain-instead-of-subdomain...
asked Oct 6, 2021 in Technique[技术] by 深蓝 (71.8m points)
0 votes
4.4k views
1 answer
robots.txt ignore only slug only
    I want to achieve that behavior: Allow: /plans and Disallow: /plans/* crawl: www.example.com/plans Do not crawl: ... /abcd-edfg-3fdv-3455 www.example.com/plans/3fdv-3455-edfg-avhn...
asked Jan 24, 2021 in Technique[技术] by 深蓝 (71.8m points)
To see more, click for the full list of questions or popular tags.
Ask a question:
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

Just Browsing Browsing

2.1m questions

2.1m answers

60 comments

57.0k users

Most popular tags

Powered by Question2Answer
Donate
Theme by Q2A Market&&OStack.cn
...