Skip to content

Issues: rmax/scrapy-redis

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
29 Open 164 Closed
29 Open 164 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Should we auto reconnect to redis? feature help wanted
#167 opened May 1, 2020 by botzill
2
Tried to stop a LoopingCall that was not running. bug
#60 opened Jul 16, 2016 by rmax
@rmax
Warn if dupefilter is not scrapy-redis compatible feature
#61 opened Jul 16, 2016 by rmax
@rmax
12x times faster next_request()/spider_idle() logic feature
#74 opened Sep 19, 2016 by 77cc33
7
Redis Sitemap Spider improvement
#102 opened Jun 8, 2017 by dariuschira
@rmax
Config to set the spider do not crash when the redis is unusble improvement question
#108 opened Aug 21, 2017 by kingname
1
Suggesting new way to schedule requests feature
#119 opened Apr 1, 2018 by botzill
1
4
Read one entry from redis queue feature
#136 opened Dec 4, 2018 by t75bernd
1
2019-04-03 12:25:25 [scrapy.extensions.logstats] INFO: Crawled 18852 pages (at 0 pages/min), scraped 1762 items (at 0 items/min) bug improvement question
#145 opened Apr 3, 2019 by python-D
2
How do I receive url from redis more quickly ? improvement question
#147 opened Apr 26, 2019 by CoderZHV
1
My scrapy program cannot keep pulling data from redis queue bug question
#148 opened Jun 25, 2019 by Cehae
1
scrapy wouldn't create new request after i pause crawl and resume crawl, and redis wouldn't create spidername:requests set again, even i set SCHEDULER_PERSIST = True at settings.py. bug question
#149 opened Jul 24, 2019 by Vickey-Wu
Ensure serialized requests can be deserialized. feature
#54 opened Jun 23, 2016 by rmax v0.7
@rmax @LuckyPigeon
2
define redis record as url +parameters and access parameters in parse method feature
#156 opened Nov 4, 2019 by ni-pixel
Looking for co-maintainers! help wanted
#171 opened Jun 5, 2020 by rmax
9
Logs report a warning: Spider.make_requests_from_url method is deprecated... help wanted improvement
#181 opened Dec 11, 2020 by LittleHello
2
When does the crawler stop improvement question
#182 opened Dec 25, 2020 by forgeries
4
[RFC]A new journey feature
#203 opened Oct 21, 2021 by whg517
4 of 6 tasks
@LuckyPigeon
14
[FEATURE] Finish TODO tasks feature good first issue improvement
#219 opened Jan 14, 2022 by LuckyPigeon
Is there a way to stop spider check duplicate with redis ? feature improvement question
#242 opened Apr 2, 2022 by milkeasd
7
[docs] Chinese documentation documentation good first issue help wanted
#243 opened Apr 5, 2022 by LuckyPigeon
1 task
Redis data persistence documentation
#248 opened Apr 11, 2022 by MRPTY1
4
make_request_from_data implementation in RedisMixin question
#267 opened Jan 24, 2023 by rjbks
2
Playwright? question
#271 opened Apr 13, 2023 by martmalo
2
AttributeError: 'BossSpider' object has no attribute 'settings'
#301 opened Sep 29, 2024 by gaye746560359
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.